Commit Graph

48 Commits

Author SHA1 Message Date
Jarek Radosz e00abbe1b7 DEV: Clean up S3 specs, stubs, and helpers
Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-28 12:02:25 +01:00
Martin Brennan 1f7b44de67
DEV: Remove flaky secure optimized image spec 2020-08-04 16:29:41 +10:00
Sam Saffron 3f054d3350
DEV: merge 2 specs
No need to run 2 specs here when 1 will do
2020-08-03 16:53:37 +10:00
Jarek Radosz 64ce12a758
FIX: `OptimizedImage#filesize` (#10095)
`OptimizedImage#filesize` calls `Discourse.store.download` with an OptimizedImage as an argument. It would in turn attempt to call `#original_filename` and `#secure?` on that object. Both would fail as these methods do not exist on OptimizedImage, only on Upload. We didn't know about these issues because:
1. `#calculate_filesize` is not called often, because the filesize is saved on OptimizedImage creation, so it's used mostly for manual filesize recalculation
2. we were using `rescue nil` which swallows all errors
2020-07-06 17:01:29 +02:00
Jarek Radosz 669c940ec3 Revert "DEV: Remove the remaining ENV["TRAVIS"] usage (#10041)"
This reverts commit 78aff841e3.

See https://review.discourse.org/t/dev-remove-the-remaining-env-travis-usage-10041/12737/4?u=cvx
2020-06-16 19:42:00 +02:00
Jarek Radosz 78aff841e3
DEV: Remove the remaining ENV["TRAVIS"] usage (#10041) 2020-06-16 17:41:15 +02:00
Sam Saffron 494379201d
DEV: attempt to stabilize flaky spec
Spec was checking implementation when it did not need to, temp file was blank
so optimized image should be blank
2020-03-06 12:57:12 +11:00
Martin Brennan 7c32411881
FEATURE: Secure media allowing duplicated uploads with category-level privacy and post-based access rules (#8664)
### General Changes and Duplication

* We now consider a post `with_secure_media?` if it is in a read-restricted category.
* When uploading we now set an upload's secure status straight away.
* When uploading if `SiteSetting.secure_media` is enabled, we do not check to see if the upload already exists using the `sha1` digest of the upload. The `sha1` column of the upload is filled with a `SecureRandom.hex(20)` value which is the same length as `Upload::SHA1_LENGTH`. The `original_sha1` column is filled with the _real_ sha1 digest of the file. 
* Whether an upload `should_be_secure?` is now determined by whether the `access_control_post` is `with_secure_media?` (if there is no access control post then we leave the secure status as is).
* When serializing the upload, we now cook the URL if the upload is secure. This is so it shows up correctly in the composer preview, because we set secure status on upload.

### Viewing Secure Media

* The secure-media-upload URL will take the post that the upload is attached to into account via `Guardian.can_see?` for access permissions
* If there is no `access_control_post` then we just deliver the media. This should be a rare occurrance and shouldn't cause issues as the `access_control_post` is set when `link_post_uploads` is called via `CookedPostProcessor`

### Removed

We no longer do any of these because we do not reuse uploads by sha1 if secure media is enabled.

* We no longer have a way to prevent cross-posting of a secure upload from a private context to a public context.
* We no longer have to set `secure: false` for uploads when uploading for a theme component.
2020-01-16 13:50:27 +10:00
Mark VanLandingham 09d9baa6d7
FIX: Update S3 stubs for more aws-sdk API changes (#8534) 2019-12-11 11:26:52 -08:00
dependabot-preview[bot] b90a592146 DEV: Bump aws-sdk-sns from 1.13.0 to 1.21.0 (#8490)
Bumps [aws-sdk-sns](https://github.com/aws/aws-sdk-ruby) from 1.13.0 to 1.21.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-sns/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/1.13.0...1.21.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 06:13:17 -08:00
Guo Xiang Tan 20e783337d DEV: Remove use of fake external store in optimized images spec.
Testing against fake stuff is bad because we're not testing against reality when
reality changes.
2019-05-29 20:13:23 +08:00
Sam Saffron 4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Guo Xiang Tan 839a54b97b FIX: Destroy `OptimizedImage` record even if `Upload` record is invalid. 2019-03-22 16:47:06 +08:00
Penar Musaraj c50db76f5d FIX: do not treat TIFF, BMP, WEBP as images
Treating TIFF and BMP as images cause us to add them to IMG tags, this is very inconsistent across browsers.

You can still upload these files they will simply not be displayed in IMG tags.
2019-02-11 16:28:43 +11:00
Sam 570877da3c FEATURE: store thumbnail algorithm version in optimized image table
Previously we had no idea what algorithm generated thumbnails, this starts tracking the version.

We also bumped up the version to force all optimized images to be generated. This is important cause we recently introduced pngquant which results in much smaller images.
2019-01-03 17:07:30 +11:00
Neil Lalonde 3236f2668f add note to specs about ImageMagick 7 requirement 2019-01-02 12:03:57 -05:00
Sam 766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Jeff Atwood 96aca20fd7 add image optim version note to relevant tests 2019-01-01 01:25:00 -08:00
Robin Ward 662cfc416b FEATURE: Show a blurry preview when lazy loading images
This generates a 10x10 PNG thumbnail for each lightboxed image.
If Image Lazy Loading is enabled (IntersectionObserver API) then
we'll load the low res version when offscreen. As the image scrolls
in we'll swap it for the high res version.

We use a WeakMap to track the old image attributes. It's much less
memory than storing them as `data-*` attributes and swapping them
back and forth all the time.
2018-12-19 01:57:30 +08:00
Sam 0a442e319c FIX: correct svg handling for images
We regressed and optimized images no longer worked with svg

The following adds the correct logic to simply copy file for svgs
and bypasses resizing for svg avatars
2018-11-07 15:29:26 +11:00
Joffrey JAFFEUX 78954672f9 FIX: uses hex to compare images
It prevents some terminals from crashing in case of errors and dumping the whole file content into the terminal.
2018-11-05 09:47:15 -05:00
Sam adab7a3a48 improve test, also ensure no zero size is generated 2018-10-23 08:50:07 +11:00
Sam bea8d337b2 DEV: ensure resizing test does not raise bad error
Current resizing test was showing binary diff in terminal and failing
in latest image magick 7, this fixes both issues
2018-10-23 08:45:06 +11:00
Sam 9ab1fb7dfc FEATURE: correctly store width and height on uploads
Previously we used width and height for thumbnails, new code ensures

1. We auto correct width and height
2. We added extra columns for thumbnail_width and height, this is determined
 by actual upload and no longer passed in as a side effect
3. Optimized Image now stores filesize which can be used for analysis, decisions

Also

- fixes Android image manifest as a side effect
- fixes issue where a thumbnail generated that is smaller than the upload is no longer used
2018-08-28 12:59:22 +10:00
Sam 8b5e42ea16 FIX: always test and coerce to image on upload
In the past the filename of the origin was used as the source
for the extension of the file when optimizing on upload.

We now use the actual calculated extension based on upload data.
2018-08-20 12:18:59 +10:00
Guo Xiang Tan f8b367cc9c Fix the build. 2018-07-26 10:17:51 +08:00
Guo Xiang Tan 30242972d3 Add spec for 4bf3bf6786. 2018-07-26 09:16:14 +08:00
Régis Hanol 4bf3bf6786 SECURITY: force IM decoder based on file extension 2018-07-25 22:00:04 +02:00
Guo Xiang Tan d839a42bf9 Skip imagemagick tests on Travis. 2018-07-17 17:11:05 +08:00
Guo Xiang Tan 1d74ccaaf8 Add compatibility for ImageMagick7. 2018-07-17 15:50:58 +08:00
Sam 15b5fddd49 SECURITY: protect upload params, only allow very strict filenames 2016-12-19 10:16:18 +11:00
Andy Waite 3e50313fdc Prepare for separation of RSpec helper files
Since rspec-rails 3, the default installation creates two helper files:
* `spec_helper.rb`
* `rails_helper.rb`

`spec_helper.rb` is intended as a way of running specs that do not
require Rails, whereas `rails_helper.rb` loads Rails (as Discourse's
current `spec_helper.rb` does).

For more information:

https://www.relishapp.com/rspec/rspec-rails/docs/upgrade#default-helper-files

In this commit, I've simply replaced all instances of `spec_helper` with
`rails_helper`, and renamed the original `spec_helper.rb`.

This brings the Discourse project closer to the standard usage of RSpec
in a Rails app.

At present, every spec relies on loading Rails, but there are likely
many that don't need to. In a future pull request, I hope to introduce a
separate, minimal `spec_helper.rb` which can be used in tests which
don't rely on Rails.
2015-12-01 20:39:42 +00:00
Sam Saffron 97f03e699d FIX: mixed local s3 store missing avatars 2015-06-01 11:17:42 +10:00
Régis Hanol 5a143c0c6e storage engines refactor 2015-05-29 18:39:47 +02:00
Sam eeda367e70 FIX: should be able to serve optimized image from local if its ... local... 2015-05-26 12:32:52 +10:00
Régis Hanol e36272cb68 fix the build 2015-02-09 17:13:22 +01:00
Luciano Sousa b52f12948a models with rspec3 syntax 2015-01-05 13:04:23 -03:00
Régis Hanol de76b512c1 fix most deprecations in the specs (still some left) 2014-09-25 17:44:48 +02:00
Sam ffa11bd1d6 remove cruft 2014-09-25 09:50:43 +10:00
Sam 504cfcff96 Fix specs for avatars
Implement avatar picker
Correct avatar related jobs
2014-05-27 10:08:03 +10:00
Régis Hanol 2505d18aa9 FEATURE: support email attachments 2014-04-14 22:55:57 +02:00
Régis Hanol 37fd7ab574 pull hotlinked images 2013-11-05 19:07:29 +01:00
Régis Hanol c867b67a0b custom avatar support 2013-08-13 22:08:29 +02:00
Régis Hanol ed9417fa3b enable thumbnailing on S3
- added url to optimized image model
- refactored s3_store & local_store
2013-07-31 23:26:34 +02:00
Régis Hanol ac7253a938 refactor CookedPostProcessor & specs 2013-07-08 01:39:08 +02:00
Régis Hanol af45b5a11e proper column naming
silly schemaless database habits are hard to kill
2013-06-17 04:02:17 +02:00
Régis Hanol 510bac4b27 refactored a bit & tested thumbnails creation 2013-06-17 02:49:34 +02:00
Régis Hanol 5de03814fb created `optimized_image` model 2013-06-17 02:49:34 +02:00