Commit Graph

71 Commits

Author SHA1 Message Date
Guo Xiang Tan 918633aa12 FIX: `upload:migrate_to_s3` rake task not remapping properly. 2018-10-10 15:09:21 +08:00
Guo Xiang Tan 24c55bd613 Add dry run option to `UploadRecovery`. 2018-09-12 21:53:01 +08:00
Guo Xiang Tan c053f8ccf6 New rake task `uploads:recover`. 2018-09-12 01:52:30 -07:00
Guo Xiang Tan 6d01e0aa04 DEV: Print the error class in `uploads:list_posts_with_broken_images`. 2018-09-12 01:06:51 -07:00
Guo Xiang Tan 94ff428571 Pass the right value to rake task. 2018-09-10 20:07:28 +08:00
Guo Xiang Tan 4a966c639d DEV: Update `uploads:list_posts_with_broken_images` to recover from tombstone. 2018-09-10 17:01:41 +08:00
Guo Xiang Tan 68572b8afc Print error messages on why upload fails to save. 2018-09-10 16:02:13 +08:00
Guo Xiang Tan 0aca80e92a Fixes to `uploads:list_posts_with_broken_images`. 2018-09-10 15:16:29 +08:00
Guo Xiang Tan 8496537590 Add `RECOVER_FROM_S3` to `uploads:list_posts_with_broken_images` rake task. 2018-09-10 15:14:30 +08:00
Guo Xiang Tan 72834f19ff DEV: Add rake tasks to list posts with broken images. 2018-09-05 16:54:15 +08:00
Guo Xiang Tan 33f4aa2835 Fix `migrate_to_s3` rake task, `File#close!` is not a valid method. 2018-08-16 13:26:58 +08:00
Sam 1fc2597626 better error handling for upload extension fixer 2018-08-10 11:28:22 +10:00
Guo Xiang Tan 0879610ffd Add missing require in `uploads:fix_incorrect_extensions`. 2018-08-08 15:39:37 +08:00
Guo Xiang Tan 17047806b9 Add a rake task to fix uploads with wrong extension. 2018-08-08 13:15:17 +08:00
Régis Hanol 14bbd5d167 FIX: use the right URL when downloading the file from S3 2018-08-02 10:04:41 +02:00
Régis Hanol 0f1137dafa FIX: 'migrate_from_s3' rake task wasn't handling short urls 2018-08-01 22:58:46 +02:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Régis Hanol 9e7d5a3cdf FIX: 'uploads:recover_from_tombstone' rake task wasn't restoring attachments 2018-03-28 15:51:47 +02:00
Guo Xiang Tan 5782a23a69 Allow `uploads:missing` to skip optimized image check. 2018-03-12 20:08:53 +08:00
Régis Hanol cbf4204ecc FIX: uploads:migrate_from_s3 wasn't destroying records before creating new ones 2018-01-24 13:22:07 +01:00
Régis Hanol 28d432263e rework the uploads:migrate_from_s3 rake task 2018-01-23 18:48:06 +01:00
Sam 83a346cd68 FIX: migrate_from_s3 rake task not working 2018-01-22 11:42:09 +11:00
Régis Hanol b6c9d7a9d2 move requires to top of file 2018-01-19 17:51:42 +01:00
Régis Hanol 2d49250107 FIX: support recovering images using 'upload://' scheme 2018-01-19 15:40:17 +01:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Régis Hanol b7e6620109 FIX: 'uploads:migrate_from_s3' rake task 2017-06-23 10:15:11 +02:00
Régis Hanol 94cee37edd FIX: uploads:recover_from_tombstone rake task 2017-06-22 17:00:05 +02:00
Robin Ward cdbe027c1c Refactor `FileHelper` to use keyword arguments. 2017-05-24 13:54:26 -04:00
Régis Hanol 9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
Guo Xiang Tan b5d6d3c326 FIX: Make recover from tombstone script work for multisite. 2016-09-22 12:28:36 +08:00
Guo Xiang Tan 333d6f9f10 More improvements to restore from tombstone script. 2016-09-22 10:39:39 +08:00
Guo Xiang Tan 1d4451db5d Improve restore from tombstone script by not having to loop through all posts. 2016-09-21 18:01:59 +08:00
Guo Xiang Tan b62b6ba12c Improve restore from tombstone rake task. 2016-09-21 17:47:27 +08:00
Guo Xiang Tan 3af5b19002 Add rake task to restore missing uploads from tombstone. 2016-09-21 16:56:32 +08:00
Guo Xiang Tan e4b75f604c FIX: Make clean up upload script a safer task to run. 2016-09-05 10:06:02 +08:00
Guo Xiang Tan df8d24734a Regenerate missing optimized images for all dbs by default. 2016-09-02 13:06:31 +08:00
Guo Xiang Tan e4883725e6 Script to analyze uploads on a site. 2016-09-02 12:07:06 +08:00
Guo Xiang Tan 82324db71a Allow `uploads:missing` Rake task to be run for a single site. 2016-09-02 10:22:39 +08:00
Guo Xiang Tan 2349ab5a51 `puts` not `putc`. 2016-08-29 10:30:10 +08:00
Guo Xiang Tan e57ed5223a Fix rake task. 2016-08-26 15:46:42 +08:00
Guo Xiang Tan 2448db4716 Prioritize user avatars when regenerating missing optimized images. 2016-08-25 18:29:52 +08:00
Guo Xiang Tan f19232646f Rescue all errors when backfilling shas for uploads. 2016-08-23 15:05:37 +08:00
Guo Xiang Tan 574b8fdf41 Allow backfilling of shas to complete. 2016-08-23 10:34:32 +08:00
Guo Xiang Tan 5222022842 Use another rake task to regenerate optimized images for multisite. 2016-08-19 15:52:09 +08:00
Régis Hanol 9971e62254 FIX: migrate_from_s3 rake task 2016-07-25 12:12:10 +02:00
Régis Hanol 557e59c28e check the uploads doesn't already exists before copying it 2016-04-13 16:33:00 +02:00
Régis Hanol 8e4113a468 add multisite support for the 'uploads:gather' rake task 2016-04-12 16:00:25 +02:00
Régis Hanol 71fe66c8c2 'uploads:gather' should only work on local uploads 2016-04-11 21:17:33 +02:00
Régis Hanol 90dfc96206 new 'uploads:gather' rake task 2016-04-11 20:42:40 +02:00