Commit Graph

102 Commits

Author SHA1 Message Date
Régis Hanol 4e78abb537 let's try 3 times to download images locally 2017-10-11 23:11:44 +02:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Sam 8ecf313a81 FIX: correctly raise errors when downloads fail
This corrects an issue where we are hitting Gravatar for 404 over and over

Also ensures file download properly reports errors
2017-09-28 16:35:43 +10:00
Régis Hanol 3a75242c38 add more logs to pull hotlinked images 2017-09-28 01:00:13 +02:00
Régis Hanol dd07094bd7 bump log level to debug download issues 2017-09-27 23:26:07 +02:00
Vinoth Kannan 6e9671c2c3 UX: Placeholder images color changed & tootip added 2017-09-13 15:16:38 +05:30
Vinoth Kannan 0fb7831749 FEATURE: Add placeholders to broken and large image files (#5113) 2017-09-01 10:26:13 -04:00
Régis Hanol a6dff79c2c change log level to info when failing to download a hotlinked image 2017-07-12 11:06:28 +02:00
Guo Xiang Tan e92acb4c40 FIX: `Jobs::PullHotlinkedImages#is_valid_image_src` returns true for a generic string. 2017-07-06 18:31:15 +09:00
Guo Xiang Tan 061aa261da Log site name when logging to Logster in `Jobs::PullHotlinkedImages`. 2017-07-05 10:34:24 +09:00
Guo Xiang Tan 4e8b80c157 Remove unnecessary `ensure` block. 2017-07-05 10:21:42 +09:00
Régis Hanol 5d63a7f4a6 FIX: pull hotlinked images even when they have no extension 2017-06-13 13:27:05 +02:00
Vinoth Kannan 209383faab FEATURE: pull onebox images 2017-06-06 18:51:58 +05:30
Robin Ward cdbe027c1c Refactor `FileHelper` to use keyword arguments. 2017-05-24 13:54:26 -04:00
Régis Hanol 9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
Leo McArdle b60e9b7330 FIX: Handle img src starting with "//" in pull_hotlinked_images job 2017-01-16 10:50:07 +00:00
Arpit Jalan 929730c008 FIX: properly insert images in markdown inline format (take 2) 2016-09-01 17:56:39 +05:30
Arpit Jalan 54a1180140 FIX: properly insert images in markdown inline format 2016-09-01 11:55:40 +05:30
Sam 8ec7fd84fd FEATURE: prioritize sidekiq jobs
This commit introduces 3 queues for sidekiq

"critical" for urgent jobs (weighted at 4x weight)
"default" for standard jobs(weighted at 2x weight)
"low" for less important jobs


"critical jobs"

Reset Password emails has been seperated to its own job
Heartbeat which is required to keep sidekiq running
Test email which needs to return real quick


"low priority jobs"

Notify mailing list
Pull hotlinked images
Update gravatar

"default"

All the rest

Note: for people running sidekiq from command line use

bin/sidekiq -q critical,4 -q default,2 -q low
2016-04-07 12:56:43 +10:00
Régis Hanol 40b099f1a6 FIX: keep whitespaces when replacing direct link to external images with local images 2016-02-15 12:34:45 +01:00
Régis Hanol 78de89f7a1 FIX: PullHotlinkedImages was messing with URL when using Markdown references 2015-11-09 16:37:51 +01:00
Régis Hanol 5e3da94c46 FIX: prevent infinite loop in PullHotlinkedImages job 2015-10-30 22:46:46 +01:00
Sam 82a6176b08 lower the volume on failed to pull hotlinked image
add more diagnostics
2015-08-19 12:32:45 +10:00
Régis Hanol 827ea641b0 FIX: Use File.size instead of IO.size 2015-08-17 18:57:28 +02:00
Régis Hanol b6cd4af2ba FIX: follow redirects when pulling hotlinked images 2015-08-14 12:46:52 +02:00
Régis Hanol 189cb3ff12 FEATURE: move migrate_to_new_scheme into a background job
- new hidden site setting 'migrate_to_new_scheme' (defaults to false)
- new rake tasks to toggle migration to new scheme
- FIX: migrate_to_new_scheme also works with CDN
- PERF: improve perf of the DbHelper.remap method
- REFACTOR: UrlHelper is now a class
2015-06-12 12:07:57 +02:00
Sam 0a2a248acc should be using site setting not global 2015-05-27 11:17:46 +10:00
Régis Hanol a797f7c664 FIX: properly handle images when using 's3_cdn_url' 2015-05-26 11:47:33 +02:00
Régis Hanol e7f251c105 LOTS of changes to properly handle post/topic revisions
FIX: history revision can now properly be hidden
FIX: PostRevision serializer is now entirely dynamic to properly handle
hidden revisions
FIX: default history modal to "side by side" view on mobile
FIX: properly hiden which revision has been hidden
UX: inline category/user/wiki/post_type changes with the revision
details
FEATURE: new '/posts/:post_id/revisions/latest' endpoint to retrieve
latest revision
UX: do not show the hide/show revision button on mobile (no room for
them)
UX: remove CSS transitions on the buttons in the history modal
FIX: PostRevisor now handles all the changes that might create new
revisions
FIX: PostRevision.ensure_consistency! was wrong due to off by 1
mistake...
refactored topic's callbacks for better readability
extracted 'PostRevisionGuardian'
2014-10-27 22:06:43 +01:00
Régis Hanol f283e2a996 FIX: there's no need to try to download relative images 2014-09-26 18:27:10 +02:00
Régis Hanol c7330ed73f BUGFIX: errors when post-processing 'data images' 2014-07-18 17:54:18 +02:00
Régis Hanol 18489ba83a BUGFIX: proper regex escaping 2014-06-11 16:37:26 +02:00
Sam a2e2d0e886 Merge pull request #2316 from mutiny/refactor-where-first
Refactor `where(...).first` to `find_by(...)`
2014-05-08 09:10:45 +10:00
Régis Hanol 4b9acd6806 BUGFIX: make sure we do not try to pull images from the CDN 2014-05-07 19:49:16 +02:00
Louis Rose 1574485443 Perform the where(...).first to find_by(...) refactoring.
This refactoring was automated using the command: bundle exec "ruby refactorings/where_dot_first_to_find_by/app.rb"
2014-05-06 14:41:59 +01:00
Régis Hanol c52ee665b4 FEATURE: do not bump topic when pulling hotlinked images 2014-04-24 16:07:47 +02:00
riking 7da5d3fe78 Backoff-retry for hotlinked image pull + some style fixes 2014-04-23 22:25:43 -07:00
riking b309deef11 Add site setting for domains to never download images from 2014-04-23 22:24:35 -07:00
Régis Hanol 5fd4d88491 BUGFIX: pull hotlinked images job wasn't properly handling the InvalidParameters exception 2014-04-22 15:32:48 +02:00
Régis Hanol 9cd8476453 REFACTOR: use an options hash instead of multiple nil-able parameters 2014-04-15 17:17:10 +02:00
Régis Hanol 542d54e6bf BUGFIX: uploads to S3 2014-04-15 13:04:14 +02:00
Régis Hanol 2505d18aa9 FEATURE: support email attachments 2014-04-14 22:55:57 +02:00
Régis Hanol 9da773af7a BUGFIX: do not pull images we already host locally 2014-01-15 01:14:41 +01:00
DV Suresh 24886c300b Correct few spelling in the comments 2013-12-21 01:19:22 -06:00
Régis Hanol 418856a789 make sure assets always use schemaless absolute uris 2013-12-06 11:16:13 +01:00
Régis Hanol b0d132ee2e make sure image urls have a scheme before pulling them in 2013-11-25 19:47:53 +01:00
Régis Hanol f65cde3cda do not bump posts when rebaking 2013-11-22 01:52:26 +01:00
Régis Hanol 31fd5b85bc FIX markdown hotlinked images were not properly pulled 2013-11-20 13:10:08 +01:00
Régis Hanol 482b752046 add edit reason when editing a post 2013-11-15 23:28:49 +01:00
Régis Hanol 16267e4887 add download_remote_images_to_local site setting 2013-11-15 15:22:18 +01:00
Régis Hanol 7e7d951152 do not pull hotlinked images when max_image_size_kb == 0 2013-11-13 17:30:48 +01:00
Régis Hanol 37fd7ab574 pull hotlinked images 2013-11-05 19:07:29 +01:00