Commit Graph

5726 Commits

Author SHA1 Message Date
Guo Xiang Tan 51b19e945c DEV: Stablize the multisite tests.
SiteSettingExtension triggers message bus which re-establishes a
DB connection in `SiteSettingExtension#process_message`. That happens
concurrently and a test that requires a connection to the db will
fail when the reconnection is happening.
2019-01-16 10:29:36 +08:00
Gerhard Schlager e180e14a55 DEV: Reload current client locale file in development mode 2019-01-15 22:55:51 +01:00
Rishabh ff8f9dc1c9 FIX: prefix should precede folder path (follow-up on 10fbb07e) 2019-01-15 15:58:19 +05:30
Rishabh 10fbb07e1a FIX: include folder name in prefix for listing files on S3 (follow-up on 3ec38f5a)
Fix the destination url in remap since it's already a part of s3_base_url
2019-01-15 15:23:55 +05:30
Guo Xiang Tan ec58c33e9e DEV: Improve postgresql fallover and multisite tests. 2019-01-15 12:52:45 +08:00
Neil Lalonde 6c2333a780 Version bump to v2.2.0.beta8 2019-01-14 17:03:18 -05:00
Penar Musaraj b28d68efbf DEV: Use yarn to manage Highlight.js dependency
Moves Highlight.js files to vendor/assets/javascripts

Adds Highlight.js in yarn package management

Removes old rake task and reliance on NPM to build Highlight.js

Highlight.js is now integrated in the "javascript:update" rake task
2019-01-14 12:46:35 -05:00
Florian Heidenreich 39983f9d95 FIX: list staff users within the last month 2019-01-14 10:29:00 -05:00
Arpit Jalan a121d40771
FIX: do not show PM topics when moving posts to an existing public topic (#6876) 2019-01-14 15:00:45 +05:30
Penar Musaraj 47cbfb1498 FEATURE: plugin support for custom icons
Plugins can now add their own SVG sprites in "plugin-name/svg-icons/".

Example: save the following as "plugin-name/svg-icons/plugin-icons.svg"

```
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
  <symbol id="testplugin-compress" viewBox="0 0 1792 1792">
    <path d="M896 960v448q0 26-19 45t-45 19-45-19l-144-144-332 332q-10 10-23 10t-23-10l-114-114q-10-10-10-23t10-23l332-332-144-144q-19-19-19-45t19-45 45-19h448q26 0 45 19t19 45zm755-672q0 13-10 23l-332 332 144 144q19 19 19 45t-19 45-45 19h-448q-26 0-45-19t-19-45v-448q0-26 19-45t45-19 45 19l144 144 332-332q10-10 23-10t23 10l114 114q10 10 10 23z"></path>
  </symbol>
</svg>
```
and then register the icon by adding this in plugin.rb
```
register_svg_icon "testplugin-compress"
```
2019-01-12 09:59:12 -05:00
Robin Ward dbe42068a2 REFACTOR: Move option to return emails into the serializer
This makes more sense than having the guardian take an accessor.
The logic belongs in the Serializer, where the JSON is calculated.

Also removed some of the DRYness in the spec. It's fewer lines
and made it easier to test the option on the serializer.
2019-01-11 11:17:23 -05:00
Vinoth Kannan f94c0283b2
FIX: Use correct version when generating file path for optimized image (#6871) 2019-01-11 18:35:38 +05:30
Penar Musaraj 4852cee946 FIX: add missing “merge selected posts” icon 2019-01-10 17:19:24 -05:00
Penar Musaraj 3e5e6c82be FIX: add missing copyright icon
Icon is used in the admin UI for themes/theme components
2019-01-10 14:26:53 -05:00
Joffrey JAFFEUX f9648de897
DEV: upgrades from Ember 2.13 to Ember 3.5.1 (#6808)
Co-Authored-By: Bianca Nenciu <nbianca@users.noreply.github.com>
Co-Authored-By: David Taylor <david@taylorhq.com>
2019-01-10 11:06:01 +01:00
Sam 35b59cfa78 SECURITY: escape title HTML for inline onebox 2019-01-10 12:02:05 +11:00
Penar Musaraj e11c6ffa89 FEATURE: allow extending CSP base-uri and object-src
Plus, ensure :none is stripped, it cannot be combined with other sources
2019-01-09 15:34:14 -05:00
Kyle Zhao dec8e5879a FEATURE: set CSP base-uri and object-src to none (#6863) 2019-01-09 15:04:50 -05:00
Sam 4232d32699 PERF: reduce workload when optimizing images
Previously, we would initialize an ImageOptim object each time we resize.

This object init is mega expensive (170ms on a VERY fast machine):

```
[1] pry(main)> Benchmark.measure { FileHelper.image_optim   }
=> #<Benchmark::Tms:0x00007f55440c1de0
 @cstime=0.055742,
 @cutime=0.141031,
 @label="",
 @real=0.17165619300794788,
 @stime=0.0002750000000000252,
 @total=0.19890400000000008,
 @utime=0.0018560000000000798>

```

This happens cause during init it hunts for all the right binaries and sets
up internals.

We now memoize this object to avoid a huge amount of pointless work.
2019-01-09 12:28:18 +11:00
Sam df460b4abd PERF: run sidekiq with nice 5
This ensures that unicorn master forks of sidekiq run with a lower priority
than the webs. It means that a busy sidekiq is less likely to impact web
performance
2019-01-09 09:29:14 +11:00
Régis Hanol 3ec38f5a3b Revert "FIX: migrate_to_s3 rake task with folder path"
This reverts commit 97fd12e8af.
2019-01-08 19:44:31 +01:00
Régis Hanol 97fd12e8af
FIX: migrate_to_s3 rake task with folder path 2019-01-08 18:56:18 +01:00
Rishabh f181e9cc08
FIX: Add compatibility for bucket folder paths in migrate_to_s3 task (#6855)
* FIX: Add compatibility for bucket folder paths in migrate_to_s3 task
* Refactor bucket_name split logic into S3Helper
2019-01-08 20:04:48 +05:30
Guo Xiang Tan ec27db78be FIX: Set unique post key for a user outside of transaction.
Previously, the Redis key was set within the transaction and the key
isn't deleted if the transaction is not successful.

Note that this isn't tested because we don't have a repro of what can
raise an error within the transaction.

https://meta.discourse.org/t/body-is-too-similar-to-what-you-previously-posted-even-when-previous-post-didnt-go-through/105436
2019-01-08 15:22:22 +08:00
Rishabh efc481d9c0 DEV: Use puts instead of printing newline (follow up on c5b7bda1) 2019-01-05 01:20:00 +05:30
Robin Ward 5eaf3cb104 Adjusts the `minimum_flag_threshold` for TL3/TL4 actions
Before this patch, a high trust level user could flag something
and have an action be taken, as well as skipping the flag queue.

Now, if a TL3/TL4 cause an action, the flag will skip the minimum
visibility check and allow staff to review it.
2019-01-04 13:16:44 -05:00
Gerhard Schlager c0a8bb9a91 FEATURE: Include "via <site_name>" in email From header 2019-01-04 17:06:19 +01:00
Régis Hanol 788719d271 DEV: speed up posts base imports 2019-01-04 15:30:17 +01:00
Régis Hanol 95e5f8380d FEATURE: Allow plugins to add custom emoji translations
FIX: buildTranslationTree was erroring when translations overlapped (ie. ":-)" and ":-))")
FIX: emoji translations wasn't working properly when translations overlapped
2019-01-04 15:27:46 +01:00
Rishabh c5b7bda198 DEV: Show migrate_to_s3 output on a new line 2019-01-04 18:09:54 +05:30
Vinoth Kannan 902f535111 FIX: upload method in S3Helper will expect a file object param 2019-01-04 15:30:45 +05:30
Vinoth Kannan 82d7f9ce5e fix the build
Checking size for a file object directly will cause issue if it is a closed stream
2019-01-04 13:25:11 +05:30
Vinoth Kannan 940a61037c DEV: Add option to pass s3 client in param 2019-01-04 12:16:09 +05:30
Vinoth Kannan 75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
Guo Xiang Tan 5f0f7f909d FIX: Incorrect CDN URL for site setting uploads when s3 is enabled. 2019-01-04 07:52:13 +08:00
Sam e2dca641c6 handle exceptions in s3:correct_acl task
We need to handle arbitrary exceptions in this task, especially since the
task is not easily resumable.

Simply output problem uploads as you hit them for now.
2019-01-04 08:32:09 +11:00
Sam 05a3e3670f FEATURE: add rake task that resets ACL on every object in S3
Some previous migrations to S3 may have bad ACLs set on objects. This
introduces a new rake task (`rake s3:correct_acl`) that will reset ACL on
every S3 object.

Vast majority of users will never have to run it, but if you have ACL issues
this is the atomic solution.
2019-01-04 08:13:43 +11:00
Gerhard Schlager b089ac1537 FIX: Posting without bump raised an error for TL4 2019-01-03 14:14:02 +01:00
Neil Lalonde 9ee08908ad Version bump to v2.2.0.beta7 2019-01-02 15:32:45 -05:00
Gerhard Schlager c30996129f FEATURE: Allow TL4 users to reset bump date 2019-01-02 16:57:05 +01:00
Gerhard Schlager e8053d6e7d FIX: Polls didn't work in imported posts
Imports skip validation of posts, but polls are only created during the validation phase.
2019-01-02 15:26:57 +01:00
Gerhard Schlager a474bf966c DEV: Skip prettier in single plugin test if patterns aren't found
This reverts d06ca90c94
It didn't work because yarn doesn't return the original exit code of the failed command.
2019-01-02 14:36:09 +01:00
Gerhard Schlager d06ca90c94 DEV: Ignore prettier error when there are no ES6 files 2019-01-02 11:30:09 +01:00
Sam 766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Guo Xiang Tan 9e50813252 FIX: Pass all necessary options in `migrate_to_s3` rake task. 2019-01-02 09:11:23 +08:00
Guo Xiang Tan 0daaae1cf3 DEV: Fix rubocop after update. 2019-01-02 05:47:29 +08:00
David Taylor 23c65feb6c DEV: Prevent deprecation warnings being incorrectly logged 2018-12-31 14:41:30 +00:00
Sam a19170a4c2 DEV: avoid require_dependency for some libs
This avoids require dependency on method_profiler and anon cache.

It means that if there is any change to these files the reloader will not pick it up.

Previously the reloader was picking up the anon cache twice causing it to double load on boot.

This caused warnings.

Long term my plan is to give up on require dependency and instead use:

https://github.com/Shopify/autoload_reloader
2018-12-31 10:53:30 +11:00
Régis Hanol 5381096bfd PERF: new 'migrate_to_s3' rake task 2018-12-26 17:34:49 +01:00
Guo Xiang Tan 2cbb513c98 FIX: Don't use `Redis#keys` in production.
As per the documentation for KEYS

```
Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout.
```

Instead SCAN

```
Since these commands allow for incremental iteration, returning only a small number of elements per call, they can be used in production without the downside of commands like KEYS or SMEMBERS that may block the server for a long time (even several seconds) when called against big collections of keys or elements.
```
2018-12-25 15:12:59 +02:00