788719d271
DEV: speed up posts base imports
2019-01-04 15:30:17 +01:00
71a5369fef
FIX: do not convert quote tags to markdown
2018-12-11 20:09:46 +05:30
c39a1022cc
PERF: user imports would slow down the more users were imported
2018-10-22 11:14:13 +02:00
afa22a0c6f
REFACTOR: more 'fake_email' to base importer
2018-10-22 11:12:40 +02:00
5f2fb0fe33
Show original options when an error happens while importing an user
2018-10-18 10:21:12 +02:00
15f657309a
FEATURE: Zendesk importer that uses its API to get data
2018-08-28 10:21:39 -04:00
1390eb2957
Disable bootstrap mode before start importing
2018-07-25 12:12:26 +05:30
fb022098f6
Base importer: Calculate category colors depending on parent category
2018-06-27 20:27:11 +02:00
5f64fd0a21
DEV: remove exec_sql and replace with mini_sql
...
Introduce new patterns for direct sql that are safe and fast.
MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API
- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder
See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
b4e0cddcc9
disable all outgoing emails in base importer
2018-06-07 22:49:38 +05:30
f9ab3848ed
FEATURE: support disabling emails for non-staff users
2018-06-07 18:31:08 +05:30
aeb511e8ff
FEATURE: SMF1 importer
2018-05-28 11:02:19 +02:00
eceeef8413
Imported categories use colors from settings instead of brown
2018-05-22 21:55:54 +02:00
6e8bf1271f
increase purge site settings for imports in case jobs like clean_up_uploads cause a mess
2018-04-09 12:44:59 -04:00
2834705bd1
Improve base import script
...
* Make sure the category description is imported correctly
(the about topic usually had the wrong excerpt).
* Allow import scripts to mark topics as closed or archived.
* Allow import scripts to store the topic's original id.
It will be stored in topic_custom_fields as import_topic_id.
2018-03-16 22:39:06 +01:00
25ec077eca
rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length'
2018-02-01 13:25:29 +05:30
bb54eb1192
Improvements to mbox importer
...
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
2018-01-17 12:04:57 +01:00
a93bf186c5
FIX: validation of website failed in base importer
2017-12-28 14:51:43 +01:00
94ceb06b02
fix user lookup by email in disqus import script
2017-11-16 16:26:18 +01:00
aec5de451b
FIX: compare imported ids as string instead of converting to integer, same as LookupContainer
2017-10-23 14:01:57 -04:00
0d257058c9
make rubocop 🤖 happy
2017-10-23 10:21:50 +02:00
20e7e285d6
add users with invalid email addresses ( #5224 )
...
* add users with invalid email addresses
* start to add suspend user stff
* don't create a suspend_user function
* include reason for suspension
2017-10-23 10:18:44 +02:00
5012d46cbd
Add rubocop to our build. ( #5004 )
2017-07-28 10:20:09 +09:00
ed5d322196
FIX: handle existing users in imports
2017-07-24 16:22:56 +05:30
d38b55e0c8
Merge pull request #4959 from farmdawgnation/minor-mbox-import-fix
...
Add a nil check on the connection before attempting to exec it
2017-07-21 09:45:15 +09:00
d0b027d88d
FEATURE: phase 1 of supporting multiple email addresses
2017-07-20 11:22:27 +09:00
5b11391588
Add a nil check on the connection before attempting to exec it
2017-07-07 11:11:43 -04:00
e155cb6db1
FIX: JIVE API Importer
...
- Added the 'hierarchical=false' query parameter to retrieve comments (replies) chronologically
- Calling 'create_post' (singular) wasn't updating the LookupContainer and thus we couldn't populate the 'reply_to_post_number' column
2017-04-26 15:36:35 +02:00
0d250c3935
50% faster vBulletin 4 importer
2017-02-01 14:33:09 +01:00
f687bfb7e2
Allow to process imported content. (add additional callbacks)
2016-10-26 16:51:34 +01:00
30f2081c1b
importer/base.rb: improve progress reporting
2016-09-20 11:27:45 -07:00
72d7c055f4
Lots of ugprades to the Disqus importer script
2016-08-17 15:10:46 -04:00
fccc0c93c5
Store the original username during import ( #4349 )
2016-07-27 22:30:15 +02:00
56d2626187
support 'read_restricted' categories in importers
2016-07-27 18:38:23 +02:00
3b119dac6d
FIX: importer performance of update_tl0, and give feedback during vanilla_mysql's create_permalinks
2016-07-06 10:58:43 -04:00
b42f28d4c3
Improved mailing list import.
...
Now uses a SQLite database to store messages rather than JSON files
for performance and memory considerations.
2016-06-14 11:45:17 -04:00
f07b3e7df9
PERF: base importer updates user_stats with sql (mostly)
2016-05-17 18:39:04 -04:00
b716886240
update bbPress importer
2016-03-07 18:22:15 +01:00
8a486d8cea
Allow importers to set empty names
2016-02-21 23:35:47 +01:00
eb166e78b6
Don't try to import invalid websites
2016-02-21 17:38:04 +01:00
08e10c2a9d
FEATURE: Zoho importer
2015-12-04 15:06:46 -05:00
26006f12a2
Increased 'error resistance' in base importer class
2015-12-01 15:38:21 +01:00
6c6d406e63
Small fixes to import base.
2015-10-29 16:47:27 -04:00
6238a43f93
Fix base and vBulletin importer
2015-10-21 19:07:31 +02:00
606aeb9d55
improvements to importer
...
- improve perf of test for existing posts
- always use a system guardian when importing posts
- for lithuim importer requery raw (transform is not repeatable)
2015-10-16 10:41:21 +11:00
29d27ec2ef
FIX: Prevent cast error when import_id contains strings and numbers
...
Some importers insert numbers and strings as import_id into the same
custom_fields table. This change prevents cast errors during imports.
2015-09-30 16:38:06 +02:00
821124f3fd
FEATURE: Skip batches if all records exist
...
Update all import scripts to take advantage of all_records_exist?
2015-09-21 17:13:02 -07:00
200ee15991
FIX: Importer not handling usernames correctly
2015-09-21 16:27:47 -07:00
8c03dd16af
Adds support for importing password hashes used by "migratepassword" plugin
...
Adds setting to phpBB3 importer for importing passwords (default: off)
Plugin: https://github.com/discoursehosting/discourse-migratepassword
2015-08-20 22:22:47 +02:00
bb93a345eb
UX: Use smaller messages for moderator actions.
2015-07-27 10:09:21 -04:00
Régis Hanol
Arpit Jalan
Neil Lalonde
Vinoth Kannan
Gerhard Schlager
Sam
Jay Pfaffman
Guo Xiang Tan
Leo McArdle
Matt Farmer
Dmitry Demenchuk
Jay Pfaffman
Robin Ward
Kane York