Commit Graph

40 Commits

Author SHA1 Message Date
Rishabh ad6ad3f679 DEV: Remove SiteSetting.s3_force_path_style (#7210)
- s3_force_path_style was added as a Minio specific url scheme but it has never been well supported in our code base.
- Our new migrate_to_s3 rake task does not work reliably with path style urls too
- Minio has also added support for virtual style requests i.e the same scheme as AWS S3/DO Spaces so we can rely on that instead of using path style requests.
- Add migration to drop s3_force_path_style from the site_settings table
2019-03-20 14:58:20 +01:00
Vinoth Kannan cc496de10e FIX: Remove double quotes from etag value in API response
https://github.com/aws/aws-sdk-ruby/issues/1134
2019-02-08 14:31:19 +05:30
Gerhard Schlager ba724d7f25 FIX: S3 endpoint broke bucket creation in non-default region 2019-02-05 18:17:02 +01:00
Vinoth Kannan b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30
Rishabh f181e9cc08
FIX: Add compatibility for bucket folder paths in migrate_to_s3 task (#6855)
* FIX: Add compatibility for bucket folder paths in migrate_to_s3 task
* Refactor bucket_name split logic into S3Helper
2019-01-08 20:04:48 +05:30
Vinoth Kannan 82d7f9ce5e fix the build
Checking size for a file object directly will cause issue if it is a closed stream
2019-01-04 13:25:11 +05:30
Vinoth Kannan 940a61037c DEV: Add option to pass s3 client in param 2019-01-04 12:16:09 +05:30
Vinoth Kannan 75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
Régis Hanol 5381096bfd PERF: new 'migrate_to_s3' rake task 2018-12-26 17:34:49 +01:00
Rishabh cae5ba7356 FIX: Ensure that multisite s3 uploads are tombstoned correctly (#6769)
* FIX: Ensure that multisite uploads are tombstoned into the correct paths

* Move multisite specs to spec/multisite/s3_store_spec.rb
2018-12-19 13:32:32 +08:00
Vinoth Kannan fd272eee44 FEATURE: Make uploads:missing task compatible with s3 uploads 2018-11-27 00:54:51 +05:30
Guo Xiang Tan 84d4c81a26 FEATURE: Support backup uploads/downloads directly to/from S3.
This reverts commit 3c59106bac.
2018-10-15 09:43:31 +08:00
Guo Xiang Tan 3c59106bac Revert "FEATURE: Support backup uploads/downloads directly to/from S3."
This reverts commit c29a4dddc1.

We're doing a beta bump soon so un-revert this after that is done.
2018-10-11 11:08:23 +08:00
Gerhard Schlager c29a4dddc1 FEATURE: Support backup uploads/downloads directly to/from S3. 2018-10-11 10:38:43 +08:00
Rishabh 4f46aa1ba3 FEATURE: Add SiteSetting for s3_configure_tombstone_policy
Add SiteSetting for s3_configure_tombstone_policy, skip policy generation if turned off (default on)
2018-09-17 10:57:50 +10:00
Guo Xiang Tan 4a966c639d DEV: Update `uploads:list_posts_with_broken_images` to recover from tombstone. 2018-09-10 17:01:41 +08:00
Guo Xiang Tan df04e69cde FIX: `S3Helper#list` creates incorrect prefix. 2018-09-10 16:34:40 +08:00
Neil Lalonde ac40fede66 Revert "FIX: subfolder support for S3 CDN" 2018-08-20 10:30:44 -04:00
Neil Lalonde 0b8f6d3637 FIX: subfolder support for S3 CDN 2018-08-20 10:22:03 -04:00
Guo Xiang Tan 1ea23b1eae FIX: Wrong order for `S3Helper#copy_file`. 2018-08-08 15:58:54 +08:00
Guo Xiang Tan aafff740d2 Add `FileStore::S3Store#copy_file`. 2018-08-08 11:30:34 +08:00
Rishabh a6c589d882 FEATURE: Add custom S3 Endpoint and DigitalOcean Spaces/Minio support for Backups (#6045)
- Add custom S3 Endpoints and DigitalOcean Spaces support
- Add Minio support using 'force_path_style' option and fix uploads to custom endpoint
2018-07-16 14:44:55 +10:00
Andrew Schleifer 4be0e31459 fix s3_cdn_url when the s3 bucket contains a folder 2018-05-23 15:51:02 -05:00
Andrew Schleifer ff15d95983 FIX s3_helper.list for buckets with folders
s3_bucket_folder_path does not contain a trailing slash so it was
smashingstufftogether
2018-05-22 20:09:08 -05:00
Sam 4f28c71b50 FIX: error setting tombstone bucket when set to old version 2017-11-13 15:36:45 +11:00
Sam a8b4255bae Correct rule installation in AWS 2017-10-09 10:26:58 +11:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Sam ac01885b60 FEATURE: rake tasks for uploading assets to S3
This opens the door to serving application.js and so on from s3.

Also updates s3 gem for some tagging support
2017-10-03 18:00:53 +11:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan 3141c179f7 REFACTOR: Get bucket name from S3Helper. 2016-08-19 14:08:37 +08:00
Guo Xiang Tan 78bec20424 FIX: Check options and not just site settings. 2016-08-17 16:16:00 +08:00
Guo Xiang Tan 11356fb9f4 Allow custom s3 options for `S3Helper`. 2016-08-16 11:13:59 +08:00
Guo Xiang Tan 1916924304 Split options into another method so that plugins can use it. 2016-08-15 22:04:24 +08:00
Guo Xiang Tan 0433163866 FEATURE: Support subfolders in `SiteSetting.s3_backup_bucket`. 2016-08-15 16:14:51 +08:00
Guo Xiang Tan aa5de3c40a FEATURE: Support subfolders in S3 bucket name.
This commit also fixes a bug where s3 uploads are not
moved to a tombstone folder when removed.
2016-08-15 13:07:41 +08:00
Guo Xiang Tan 3378ee223f FIX: Incorrect path being passed to `S3Store#remove_file`. 2016-08-15 11:35:30 +08:00
Régis Hanol 85d4d3223c FIX: crop avatars on the server instead of the client
FIX: support for dots in S3 bucket names
2015-05-26 15:54:25 +02:00
Régis Hanol bb0c2813ac FEATURE: generate (avatar) thumbnails in a background task
FIX: keep the "uploading..." indicator until the server replies via the MessageBus
FIX: text was disapearing when uploading an avatar

PERF: always use a region for S3 (defaults to 'us-east-1')
FEATURE: ApplyCDN middleware when using S3
FIX: use the same pattern to store files on S3 and locally
PERF: keep a local cache of uploads when generating thumbnails
FEATURE: migrate_to_s3 rake task
2015-05-25 17:59:00 +02:00
Sam f079dd63ed PERF: remove "fog" dependency 2015-05-25 17:57:06 +10:00
Régis Hanol bfdbb70b3b FIX: automatic backup uploads to S3 when using a region 2014-09-24 22:52:09 +02:00