Commit Graph

103 Commits

Author SHA1 Message Date
Robin Ward 95f263995d FIX: Previous annotations were broken 2019-01-11 14:30:19 -05:00
Robin Ward a3839495e0 Update annotations 2019-01-11 12:19:43 -05:00
Sam 7e4a43afc6 PERF: run ImageMagick conversions with nice 10
This lowers the priority of all image resizes so they do not clog CPU on
machines running low on resources
2019-01-09 09:21:58 +11:00
Sam 9c91e68351 PERF: remove image optimization throttling from Sidekiq
Previously we only allowed one image optimization per machine, this meant there
was cross talk between avatar resizing and Sidekiq. This could lead to large
amounts of starvation when optimized image version changed which in turn could
block the Sidekiq queue.

This increases amount of allowed load on machines but this is preferable to
having crosstalk between avatar resizing and Sidekiq.
2019-01-04 18:44:44 +11:00
Sam 570877da3c FEATURE: store thumbnail algorithm version in optimized image table
Previously we had no idea what algorithm generated thumbnails, this starts tracking the version.

We also bumped up the version to force all optimized images to be generated. This is important cause we recently introduced pngquant which results in much smaller images.
2019-01-03 17:07:30 +11:00
Sam 766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Robin Ward 6080e3a2c0 FIX: Broken spec 2018-12-18 14:55:09 -05:00
Robin Ward 662cfc416b FEATURE: Show a blurry preview when lazy loading images
This generates a 10x10 PNG thumbnail for each lightboxed image.
If Image Lazy Loading is enabled (IntersectionObserver API) then
we'll load the low res version when offscreen. As the image scrolls
in we'll swap it for the high res version.

We use a WeakMap to track the old image attributes. It's much less
memory than storing them as `data-*` attributes and swapping them
back and forth all the time.
2018-12-19 01:57:30 +08:00
Sam 0a442e319c FIX: correct svg handling for images
We regressed and optimized images no longer worked with svg

The following adds the correct logic to simply copy file for svgs
and bypasses resizing for svg avatars
2018-11-07 15:29:26 +11:00
Arpit Jalan 72be638728 FEATURE: add external details to user fields 2018-09-20 08:10:51 +05:30
Sam e0e6dae6a7 minor cleanup to previous commit from code review 2018-09-19 16:07:29 +10:00
Sam 5302709343 FIX: in redis readonly raise an exception from DistributedMutex
If we detect redis is in readonly we can not correctly get a mutex
raise an exception to notify caller

When getting optimized images avoid the distributed mutex unless
for some reason it is the first call and we need to generate a thumb

In redis readonly no thumbnails will be generated
2018-09-19 15:50:58 +10:00
Sam 9ab1fb7dfc FEATURE: correctly store width and height on uploads
Previously we used width and height for thumbnails, new code ensures

1. We auto correct width and height
2. We added extra columns for thumbnail_width and height, this is determined
 by actual upload and no longer passed in as a side effect
3. Optimized Image now stores filesize which can be used for analysis, decisions

Also

- fixes Android image manifest as a side effect
- fixes issue where a thumbnail generated that is smaller than the upload is no longer used
2018-08-28 12:59:22 +10:00
Sam 8b5e42ea16 FIX: always test and coerce to image on upload
In the past the filename of the origin was used as the source
for the extension of the file when optimizing on upload.

We now use the actual calculated extension based on upload data.
2018-08-20 12:18:59 +10:00
Sam 9628c3cf97 FEATURE: automatically correct extension for bad uploads
This fixes with post thumbnails on the fly
2018-08-17 14:00:27 +10:00
Sam f62073a22a correct regression uploading images 2018-08-16 18:49:08 +10:00
Sam 796164b58c FIX: automatically correct bad avatars on access
Also start relying on upload extension for optimized images
2018-08-16 16:32:56 +10:00
Sam 33b5cc6bbd FIX: Add gif to list of allowed decoders 2018-08-15 13:56:57 +10:00
Sam cf788582cc DEV: split optimize image failure warnings in logs 2018-08-15 13:27:24 +10:00
Sam 83fd308963 FEATURE: group error message regarding image optimization failures 2018-08-13 14:48:25 +10:00
Gerhard Schlager 6d813c2b52 FIX: Importers failed to import avatars 2018-08-12 22:02:17 +02:00
Guo Xiang Tan f8b367cc9c Fix the build. 2018-07-26 10:17:51 +08:00
Guo Xiang Tan 30242972d3 Add spec for 4bf3bf6786. 2018-07-26 09:16:14 +08:00
Régis Hanol f94aeaf6cf SECURITY: force IM decoder based on file extension - part 3 2018-07-25 23:55:06 +02:00
Régis Hanol 800c57c6ab SECURITY: force IM decoder based on file extension - part 2 2018-07-25 23:08:02 +02:00
Régis Hanol 4bf3bf6786 SECURITY: force IM decoder based on file extension 2018-07-25 22:00:04 +02:00
Guo Xiang Tan 487ef9773f Better logging when optimizing image fails. 2018-07-18 14:11:23 +08:00
Guo Xiang Tan 1d74ccaaf8 Add compatibility for ImageMagick7. 2018-07-17 15:50:58 +08:00
Guo Xiang Tan 142571bba0 Remove use of `rescue nil`.
* `rescue nil` is a really bad pattern to use in our code base.
  We should rescue errors that we expect the code to throw and
  not rescue everything because we're unsure of what errors the
  code would throw. This would reduce the amount of pain we face
  when debugging why something isn't working as expexted. I've
  been bitten countless of times by errors being swallowed as a
  result during debugging sessions.
2018-04-02 13:52:51 +08:00
Guo Xiang Tan 226ace1643 Update annotations. 2018-02-20 14:28:58 +08:00
Sam c42bb94362 PERF: slow down optimizing images per host 2018-01-24 08:53:17 +11:00
Sam bfd052a317 PERF: guard so we only create one optimized image per process
This protectd sidekiq and other cases where optimized images are created on
demand so they do not dominate the machine.
2018-01-24 08:22:28 +11:00
Arpit Jalan daeb7694bc update annotations 2017-12-05 21:03:20 +05:30
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Régis Hanol c7c93e7159 FEATURE: new 'strip image metadata' site setting 2017-07-25 11:48:39 +02:00
Régis Hanol aa69f78fed let's not use progressing rendering on images 2017-06-26 23:19:48 +02:00
Régis Hanol 6e5c59e5ec FIX: image orientation wasn't properly working 2017-06-22 16:53:49 +02:00
Robin Ward cdbe027c1c Refactor `FileHelper` to use keyword arguments. 2017-05-24 13:54:26 -04:00
Régis Hanol 9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
Robin Ward 24e1892d28 Silence logs for OptimizedImage failures 2017-05-09 14:59:46 -04:00
Guo Xiang Tan ddc8abf9d8 Fix broken optimize image command. 2017-03-20 15:07:38 +08:00
Guo Xiang Tan e7c972ac89 FIX: Don't use backticks that take in inputs. 2017-03-17 15:33:51 +08:00
Régis Hanol c4552ef8c1 slightly shorter regexp for OptimizedImage.safe_path? 2017-01-02 16:28:14 +01:00
Sam 15b5fddd49 SECURITY: protect upload params, only allow very strict filenames 2016-12-19 10:16:18 +11:00
Guo Xiang Tan e4b75f604c FIX: Make clean up upload script a safer task to run. 2016-09-05 10:06:02 +08:00
Guo Xiang Tan efd7cbd887 Remove limit default.
Having the limit makes it harder to migrate all `Upload`/`OptimizedImage`
since the count has to be figured out and passed to the method.
2016-09-02 10:55:11 +08:00
Régis Hanol 11172b7c2d FIX: cropping GIF wasn't working 2016-07-27 18:48:02 +02:00
Régis Hanol 667dd54a23 FEATURE: new 'crop_tall_images' site setting 2016-05-23 16:18:30 +02:00
Marcus Rückert 320d5d64e8 Use sRGB Color Profile when converting images
With the conversion done when uploading or fetching an image we also
lose any embedded color profiles. [This leads to images possibly not rendered
properly in the browsers.][1] To fix the issue we tell imagemagick to render
the image from the embedded color profile to sRGB, which is the color
space used by most browsers.

RT_sRGB.icm is taken from the [RawTherapee repository][2] and is licensed as Public Domain.

[1]: https://meta.discourse.org/t/image-embedded-color-profile/40519
[2]: https://github.com/Beep6581/RawTherapee/blob/master/rtdata/iccprofiles/output/RT_sRGB.icm
2016-03-07 23:42:33 +01:00
Sam 32c681c96b annotate models 2016-02-23 10:33:53 +11:00