9bff0882c3
FEATURE: Nokogumbo ( #9577 )
...
* FEATURE: Nokogumbo
Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
067696df8f
DEV: Apply Rubocop redundant return style
2019-11-14 15:10:51 -05:00
0f3c3bc309
Make import scripts work with frozen strings
2019-05-30 22:22:24 +02:00
30990006a9
DEV: enable frozen string literal on all files
...
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.
Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
24347ace10
FIX: Properly associate user_profiles background urls via upload id.
...
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
2019-05-02 14:58:24 +08:00
3c9c95ac83
Update Rubocop to 0.60
2018-12-04 10:48:16 +01:00
9248ad1905
DEV: Enable `Style/SingleLineMethods` and `Style/Semicolon` in Rubocop ( #6717 )
2018-12-04 11:48:13 +08:00
afa22a0c6f
REFACTOR: more 'fake_email' to base importer
2018-10-22 11:12:40 +02:00
434035f167
FIX: Link post to uploads in `PostCreator`.
...
* This ensures that uploads are linked to their post on creation
instead of a background job which may be delayed if Sidekiq
is facing difficulties.
2018-09-06 11:18:11 +08:00
652b32484f
Assign default value for message template matches
2018-06-26 05:16:03 +05:30
f3011c709b
Extract html content from lithium message template
2018-06-26 05:07:32 +05:30
5f64fd0a21
DEV: remove exec_sql and replace with mini_sql
...
Introduce new patterns for direct sql that are safe and fast.
MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API
- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder
See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
4ffa4a28b0
FIX: duplicate_emails get overridden in new batch of import
2018-06-19 00:21:48 +05:30
750367007c
REFACTOR: Import user visits from lithium database
2018-06-18 20:38:57 +05:30
ef4a86456b
Add attachment folder name in prefix for lithium import
2018-06-18 18:29:14 +05:30
2a0f409b9d
Use lowercased email addresses to check duplicates
2018-06-16 20:34:37 +05:30
ac44374a59
Import user visits from user_log table
2018-06-16 19:10:55 +05:30
620a1524cb
Use plus addressing email address for duplicates
2018-06-07 19:11:55 +05:30
bb12fa3fdc
Migrate user mentions in lithium import
2018-05-21 18:19:22 +05:30
b229c112f6
FIX: variable name typo
2018-05-21 13:47:30 +05:30
09151190f9
FIX: Use avatar_dir to import user avatars
2018-05-21 13:43:23 +05:30
c9c3a83261
Importing lithium post images and attachments
2018-05-21 13:34:52 +05:30
f3385a74cb
Importing lithium topic tags
2018-05-19 11:24:48 +05:30
ba0dd5889d
Improvements in importing the lithium pms
2018-05-18 22:57:15 +05:30
9f92fdded0
Improvements in lithium topic and post import
2018-05-18 18:53:18 +05:30
9d4d6276b7
Import user profile fields and avatars
2018-05-18 17:11:20 +05:30
0639b902dc
Import lithium nodes into categories
2018-05-18 01:34:31 +05:30
ade7c94b43
Import lithium roles into groups
2018-05-16 17:26:15 +05:30
5012d46cbd
Add rubocop to our build. ( #5004 )
2017-07-28 10:20:09 +09:00
d0c5205a52
Feature: Change markdown engine to markdown it
...
This commit removes the old evilstreak markdownjs engine.
- Adds specs to WhiteLister and changes it to stop using globals
(Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
79a084dd58
Revert "remove old markdown engine work-in-progress"
...
This reverts commit ee470b5317
2017-07-12 18:10:51 -04:00
ee470b5317
remove old markdown engine work-in-progress
2017-07-12 17:44:40 -04:00
9641d2413d
REFACTOR: upload workflow creation into UploadCreator
...
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
a6be3d7cbe
skip post processing stuff we did not import
2015-10-16 10:41:21 +11:00
606aeb9d55
improvements to importer
...
- improve perf of test for existing posts
- always use a system guardian when importing posts
- for lithuim importer requery raw (transform is not repeatable)
2015-10-16 10:41:21 +11:00
821124f3fd
FEATURE: Skip batches if all records exist
...
Update all import scripts to take advantage of all_records_exist?
2015-09-21 17:13:02 -07:00
02a38eebbb
correct logic in importer
2015-08-03 18:30:26 +10:00
a1f02d4baa
correct logic, add missing permalink creator
2015-08-03 17:35:35 +10:00
fd82107df8
correct bugs in lithium importer
2015-08-03 17:16:19 +10:00
64bbf2c1c4
correct closing logic for wd importer
2015-08-03 16:18:28 +10:00
f7ecfb3a83
handle image uploads
2015-07-29 18:34:10 +10:00
c699a6fca9
PMs now import for my current data dump
2015-07-28 18:16:29 +10:00
7891ed8145
Work in progress lithium importer
...
* PM import partially implemented
* Proper category names
* Smiley mappings
2015-07-28 17:46:17 +10:00
3a54923116
FIX: permalink normalization not applied at constraint
...
implement permalink import for lithium
2015-07-22 13:40:45 +10:00
d873e68059
bypass table when normalizing.
2015-07-21 10:45:18 +10:00
c552f846ae
add index
2015-07-20 16:56:46 +10:00
24eb12778f
work in progress lithium importer
2015-07-17 17:16:47 +10:00
bbfd18aa59
wip lithuim importer
2015-07-17 12:43:45 +10:00
35e1b52d54
remove 0 view filter
2015-07-16 16:57:55 +10:00
88578f74d0
another subst
2015-07-16 16:24:46 +10:00
Krzysztof Kotlarek
Penar Musaraj
Gerhard Schlager
Sam Saffron
Guo Xiang Tan
Régis Hanol
David Taylor
Vinoth Kannan
Guo Xiang Tan
Kane York