Commit Graph

734 Commits

Author SHA1 Message Date
Sam e4208113a8 improve report and add regular logging 2018-07-27 16:22:14 +10:00
Sam 5e262265a2 update script to provide more mem stats 2018-07-27 12:51:23 +10:00
Godfrey Chan 5affdcbd59 Bump Ruby version in some docs 2018-07-25 14:38:10 -07:00
Vinoth Kannan 1390eb2957 Disable bootstrap mode before start importing 2018-07-25 12:12:26 +05:30
Sam f0a23d50b4 DEV: add script for testing memory usage in sidekiq 2018-07-24 17:57:02 +10:00
Neil Lalonde bf7ebecb76 FIX: discourse_merger: many foreign keys were not being updated 2018-07-22 22:05:07 -04:00
Neil Lalonde 4e09206061 FIX: set uploads sequence after copying uplaods in discourse_merger 2018-07-19 11:07:15 -04:00
Régis Hanol e8e9b5cea4 FIX: clean URLs in SMF1 importer 2018-07-19 13:17:43 +02:00
Régis Hanol 63e5349209 FIX: [img] BBCode tags might have parameters 2018-07-19 13:11:01 +02:00
Régis Hanol 5434cf02a3 FIX: smf1 importer was swallowing some data 2018-07-19 10:29:54 +02:00
Neil Lalonde def2653fc8 FIX: discourse_merger: copied topic_link records had wrong url, and update all internal links to use new topic URLs in copied posts 2018-07-18 16:45:48 -04:00
Neil Lalonde 24da2940a7 FIX: copy uploads quickly in discourse_merger.rb, and fix user avatar upload id for copied users 2018-07-18 16:42:59 -04:00
Neil Lalonde dbfa491ee2 FIX: avatars in discourse_merger.rb 2018-07-17 21:40:24 -04:00
Neil Lalonde f146f94ef6 FIX: errors when copying post_uploads in discourse_merger.rb 2018-07-17 16:47:23 -04:00
Neil Lalonde 04077a7df6 WIP: a fast method of copying uploads in discourse_merger.rb. not working yet. 2018-07-17 16:46:32 -04:00
Neil Lalonde 2786c79354 another check to avoid unique index error in discourse_merger.rb 2018-07-16 13:34:41 -04:00
Neil Lalonde 8d11df6971 FIX: support amazon S3 upload urls in discourse_merger.rb 2018-07-13 16:10:31 -04:00
Neil Lalonde 71814009bd FIX: badges for merged users don't get merged by discourse_merger.rb 2018-07-12 17:43:21 -04:00
Neil Lalonde cba292cb56 FIX: personal messages not being copied by discourse_merger.rb 2018-07-12 17:41:16 -04:00
Régis Hanol c818550172 Support custom avatar in SMF1 importer 2018-07-12 17:38:07 +02:00
Régis Hanol 5c4534d895 Update SMF1 import
- Properly import avatar when they use an external image
- Don't import the same attachment twice
2018-07-12 16:55:30 +02:00
Neil Lalonde c33ee13c4c FIX: discourse_merger halts when topic has nil category 2018-06-29 12:21:25 -04:00
Sam f4f95ce956 correct linting 2018-06-29 16:04:38 +10:00
David Lee 8f43872bff Add Question2Answer import script 2018-06-29 15:48:01 +10:00
Arpit Jalan c73f98c289 FIX: invert from and to user id in smf1 import script 2018-06-28 12:30:28 +05:30
Gerhard Schlager fb022098f6 Base importer: Calculate category colors depending on parent category 2018-06-27 20:27:11 +02:00
Vinoth Kannan 652b32484f Assign default value for message template matches 2018-06-26 05:16:03 +05:30
Vinoth Kannan f3011c709b Extract html content from lithium message template 2018-06-26 05:07:32 +05:30
Neil Lalonde a1c0d0e6e5 fixes to discourse_merger: failures for Uploads, UserBadges, PostUploads hack 2018-06-21 12:16:05 -04:00
Neil Lalonde b9cb97df7f add support for badges in discourse_merger 2018-06-19 15:11:48 -04:00
Neil Lalonde dbcbd8d939 close connections in discourse_merger 2018-06-19 10:34:05 -04:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
OsamaSayegh 91b73e0c2d FIX: remap shouldn't fail silently when an error occurs 2018-06-19 14:49:43 +10:00
Vinoth Kannan 4ffa4a28b0 FIX: duplicate_emails get overridden in new batch of import 2018-06-19 00:21:48 +05:30
Vinoth Kannan 750367007c REFACTOR: Import user visits from lithium database 2018-06-18 20:38:57 +05:30
Vinoth Kannan ef4a86456b Add attachment folder name in prefix for lithium import 2018-06-18 18:29:14 +05:30
Gerhard Schlager 3f167ae5ce Use short upload URL in import scripts 2018-06-17 22:57:32 +02:00
Gerhard Schlager 88ca838e02 Create avatar from file in base importer 2018-06-17 22:57:31 +02:00
Gerhard Schlager 84d9b2e473 Use correct post id in zendesk importer 2018-06-17 22:57:31 +02:00
Vinoth Kannan 2a0f409b9d Use lowercased email addresses to check duplicates 2018-06-16 20:34:37 +05:30
Vinoth Kannan ac44374a59 Import user visits from user_log table 2018-06-16 19:10:55 +05:30
Neil Lalonde 20ceadffaf FEATURE: script to merge two discourse sites 2018-06-15 17:13:36 -04:00
Sam c56bd2ac16 add memory analysis script 2018-06-14 12:18:36 +10:00
discoursehosting fc973f9363 Improve the VBulletin importer (#5922) 2018-06-12 20:41:21 +02:00
Neil Lalonde 1ba8e8948d FIX: add support for string avatar_type values in PHPBB3 importer 2018-06-07 18:14:11 -04:00
Arpit Jalan b4e0cddcc9 disable all outgoing emails in base importer 2018-06-07 22:49:38 +05:30
Vinoth Kannan 620a1524cb Use plus addressing email address for duplicates 2018-06-07 19:11:55 +05:30
Arpit Jalan f9ab3848ed FEATURE: support disabling emails for non-staff users 2018-06-07 18:31:08 +05:30
Guo Xiang Tan ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Régis Hanol 127398c68e FIX: import comments of 1st post in SE importer 2018-06-05 18:22:42 +02:00