Commit Graph

46 Commits

Author SHA1 Message Date
Sam Saffron 7429700389 FIX: ensure we can download maxmind without redis or db config
This also corrects FileHelper.download so it supports "follow_redirect"
correctly (it used to always follow 1 redirect) and adds a `validate_url`
param that will bypass all uri validation if set to false (default is true)
2019-05-28 10:28:57 +10:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam 4232d32699 PERF: reduce workload when optimizing images
Previously, we would initialize an ImageOptim object each time we resize.

This object init is mega expensive (170ms on a VERY fast machine):

```
[1] pry(main)> Benchmark.measure { FileHelper.image_optim   }
=> #<Benchmark::Tms:0x00007f55440c1de0
 @cstime=0.055742,
 @cutime=0.141031,
 @label="",
 @real=0.17165619300794788,
 @stime=0.0002750000000000252,
 @total=0.19890400000000008,
 @utime=0.0018560000000000798>

```

This happens cause during init it hunts for all the right binaries and sets
up internals.

We now memoize this object to avoid a huge amount of pointless work.
2019-01-09 12:28:18 +11:00
Sam 766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Guo Xiang Tan 6c65718301 Include response body when raising an error in `FileHelper#download`. 2018-09-13 15:43:58 +08:00
Guo Xiang Tan 71caf7521d Drop `tif`, `tiff`, `webp` and `bmp` from supported images.
https://meta.discourse.org/t/cr2-raw-files-are-being-treated-as-tiff-files/96775/3?u=tgxworld
2018-09-12 09:29:54 +08:00
Guo Xiang Tan e1b16e445e Rename `FileHelper.is_image?` -> `FileHelper.is_supported_image?`. 2018-09-12 09:22:28 +08:00
Guo Xiang Tan d59e635a77 Rename `FileHelper.images` to `FileHelper.supported_images`. 2018-09-12 09:22:28 +08:00
Guo Xiang Tan 16c0ebe8a8 Fix the build. 2018-08-17 16:53:07 +08:00
Guo Xiang Tan a26ef7738f FIX: `FileHelper#download` should return nil if max size is exceeded. 2018-08-17 16:19:59 +08:00
Guo Xiang Tan b94633e844 FIX: `FileHelper` should prioritize response content-type.
Request to a URL with `.png` extension may return a jpg
instead causing us to attach the wrong extension to an
upload.
2018-07-30 10:54:36 +08:00
Guo Xiang Tan ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Régis Hanol 0559a4736a FIX: don't double request when downloading a file 2018-02-24 12:35:57 +01:00
Régis Hanol ca1fd774a1 Revert "WIP"
This reverts commit 2cf5479678.
2018-02-22 18:15:42 +01:00
Régis Hanol 2cf5479678 WIP 2018-02-22 17:56:56 +01:00
Gerhard Schlager d1f257d275 FinalDestination should only log when verbose is enabled 2017-10-31 17:16:59 +01:00
Gerhard Schlager 8c27f28dcb add more logging to FinalDestination 2017-10-31 12:26:35 +01:00
Sam 8ecf313a81 FIX: correctly raise errors when downloads fail
This corrects an issue where we are hitting Gravatar for 404 over and over

Also ensures file download properly reports errors
2017-09-28 16:35:43 +10:00
Sam b80320da5e no verbose logging for failed downloads of gravatars 2017-09-28 11:32:26 +10:00
Régis Hanol 165b388158 fix indentation 2017-09-28 01:07:43 +02:00
Régis Hanol 3a75242c38 add more logs to pull hotlinked images 2017-09-28 01:00:13 +02:00
Régis Hanol c7c93e7159 FEATURE: new 'strip image metadata' site setting 2017-07-25 11:48:39 +02:00
Guo Xiang Tan 02094517e9 Stop rescuing `nil` which is a really bad thing to do. 2017-07-05 10:22:31 +09:00
Régis Hanol ca539854c8 FIX: use 'jpg' instead of 'jpe' extension for 'image/jpeg' content type 2017-06-22 12:53:56 +02:00
Régis Hanol 5d63a7f4a6 FIX: pull hotlinked images even when they have no extension 2017-06-13 13:27:05 +02:00
Robin Ward 0a08c18a14 FIX: Don't rate limit gravatar downloads 2017-05-24 13:54:26 -04:00
Robin Ward cdbe027c1c Refactor `FileHelper` to use keyword arguments. 2017-05-24 13:54:26 -04:00
Robin Ward 3b0cbf7013 FIX: Always allow downloads from CDN 2017-05-23 16:32:54 -04:00
Robin Ward f101e052e6 FIX: Roll back file_helper changes, relative URLs were broken 2017-05-23 15:50:48 -04:00
Robin Ward 36e477750c FIX: Use same code path for downloading images 2017-05-23 14:51:30 -04:00
Robin Ward 31f27006e1 FIX: We can download protocol relative urls 2017-05-15 15:32:55 -04:00
Régis Hanol 3ce3abef8f FIX: add Content-Disposition and Content-Type headers when downloading attachments 2017-02-20 15:59:01 +01:00
Arpit Jalan da2f1fda15 FIX: increase read_timeout when downloading avatar 2016-03-24 17:40:03 +05:30
Régis Hanol bebeeffc30 FIX: ico are images too 2015-12-31 11:40:23 +01:00
Régis Hanol a3e76dc193 FIX: allow HTTP <-> HTTPS redirections when downloading images 2015-08-17 19:21:30 +02:00
Régis Hanol f0b35b2fe2 FIX: file helper to download file using chunks of 512KB 2015-05-19 12:39:46 +02:00
Régis Hanol 961f676b91 FIX: proper hack to support underscores in URLs 2014-11-12 18:15:50 +01:00
Régis Hanol 7244266174 FIX: support for underscores in URLs (when using S3 storage) 2014-11-12 17:08:11 +01:00
Jeff Atwood d926cc13ac FEATURE: add webp and svg to known image formats 2014-10-13 12:43:39 -07:00
Régis Hanol 488d4eebbe SECURITY: do not follow redirect by default when downloading hotlinked images 2014-07-27 17:21:47 +02:00
Sam a864f8aefd BUGFIX: sometimes we have stringio, it has no close 2014-05-27 10:08:03 +10:00
Régis Hanol 51750f7d0e TRIVIAL: slightly better information when logging download error 2014-05-12 16:57:52 +02:00
Régis Hanol 4371374ba6 FEATURE: support for enabling all upload file types
BUGFIX: authorized extensions is now case insensitive
2014-04-29 19:12:35 +02:00
Régis Hanol 76d373161d BUGIFX: generate avatars when using S3 store 2014-04-22 17:11:06 +02:00
Régis Hanol 542d54e6bf BUGFIX: uploads to S3 2014-04-15 13:04:14 +02:00
Régis Hanol 2505d18aa9 FEATURE: support email attachments 2014-04-14 22:55:57 +02:00