David Taylor b0416cb1c1 FEATURE: Upload to s3 in parallel to speed up backup restores (#13391) ... Uploading lots of small files can be made significantly faster by parallelizing the `s3.put_object` calls. In testing, an UPLOAD_CONCURRENCY of 10 made a large restore 10x faster. An UPLOAD_CONCURRENCY of 20 made the same restore 18x faster. This commit is careful to parallelize as little as possible, to reduce the chance of concurrency issues. In the worker threads, no database transactions are performed. All modification of shared objects is controlled with a mutex. Unfortunately we do not have any existing tests for the `ToS3Migration` class. This change has been tested with a large site backup (120k uploads totalling 45GB)		2021-06-16 10:34:39 +01:00
..
base_store.rb	FEATURE: Use path from existing URL of uploads and optimized images (#13177)	2021-05-27 17:42:25 +02:00
local_store.rb	FEATURE: Use path from existing URL of uploads and optimized images (#13177)	2021-05-27 17:42:25 +02:00
s3_store.rb	FEATURE: Use path from existing URL of uploads and optimized images (#13177)	2021-05-27 17:42:25 +02:00
to_s3_migration.rb	FEATURE: Upload to s3 in parallel to speed up backup restores (#13391)	2021-06-16 10:34:39 +01:00