Commit Graph

594 Commits

Author SHA1 Message Date
Sam Saffron eb105ba79d DEV: revert upgrade of rack to version 2.0.8
We can not upgrade rack cause it breaks Sidekiq web.

I can not find a trivial fix short of disabling sessions in Sidekiq which
is a security concern.

We need to figure out how to reuse sessions with our Rails application in
Sidekiq.

This gets extra complex cause we use a special cookie store for sessions.

9e399b42b9/lib/discourse_cookie_store.rb (L3-L21)
2020-01-13 18:07:16 +11:00
Martin Brennan beb91e7eff
FIX: require: false for rotp gem (#8540)
The ROTP gem is only used in a very small amount of places in the app, we don't need to globally require it.

Also set the Addressable gem to not have a specific version range, as it has not been a problem yet.

Some slight refactoring of UserSecondFactor here too to use SecondFactorManager to avoid code repetition
2019-12-17 10:33:51 +10:00
Martin Brennan edbc356593
FIX: Replace deprecated URI.encode, URI.escape, URI.unescape and URI.unencode (#8528)
The following methods have long been deprecated in ruby due to flaws in their implementation per http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/29293?29179-31097:

URI.escape
URI.unescape
URI.encode
URI.unencode
escape/encode are just aliases for one another. This PR uses the Addressable gem to replace these methods with its own encode, unencode, and encode_component methods where appropriate.

I have put all references to Addressable::URI here into the UrlHelper to keep them corralled in one place to make changes to this implementation easier.

Addressable is now also an explicit gem dependency.
2019-12-12 12:49:21 +10:00
Sam Saffron b6acfb7847 DEV: upgrade redis-namespace gem
New release has a few extra commands namespaced, nothing we use.

Also added a comment about why this is explicitly required.
2019-12-12 13:36:08 +11:00
Mark VanLandingham 06c6062ed2
DEV: Lock sassc gem at version 2.0.1 with note (#8523) 2019-12-11 06:22:39 -08:00
Sam Saffron 4de39f6596 DEV: hold back mocha upgrade
This breaks our test suite and we want to properly document this.
2019-12-11 12:43:07 +11:00
Sam Saffron 3e0454c97b DEV: add a note about sprockets being held back
We want to upgrade to version 4, but it does not work atm.
2019-12-10 12:31:16 +11:00
Sam Saffron a06fccae1b DEV: update dependencies and add notes about exceptions
Previously it was unclear why certain gems are being held back cause Gemfile
had no comment explaining it.

I tried to add some explanation from memory and remove some exceptions that
seemed to be superfluous.

This upgrades shoulda to latest, it appears to work once a couple of assertions
are removed

Also update http accept language used to auto detect language from http header
this is tested

Zeitwerk small update seems fine
2019-12-06 13:00:28 +11:00
Arpit Jalan cab9c7c77e Bump onebox version.
- FIX: use dedicated Vimeo onebox for all video types
2019-11-27 16:22:25 +05:30
Arpit Jalan 7543db086a Bump onebox version.
- FIX: Amazon video oneboxes were not working.
2019-11-20 14:47:59 +05:30
David Taylor eaf6096890 DEV: Use rubocop-discourse gem to add custom chdir cop
Followup to b27e009655
2019-11-18 15:39:41 +00:00
Sam Saffron 26c0199c01 DEV: update Rails to version 6.0.1
This version of Rails eliminates a monkey patch that is no longer needed!

Additionally it preps us for Ruby 2.7 support.
2019-11-08 16:56:30 +11:00
Arpit Jalan c5df853dea Bump onebox version.
- fix for gfycat onebox in email
2019-11-07 10:03:12 +05:30
Arpit Jalan cb9702bf7a Bump onebox version.
- Remove native caching
- FIX: dropbox videos were not loading
2019-11-04 10:46:20 +05:30
Arpit Jalan 12409f63a0 Bump onebox version.
- FIX: Follow redirect returns url if response code is 200
- FIX: do not resize xkcd image
2019-10-22 12:26:01 +05:30
Krzysztof Kotlarek 858cf5836c
FIX: update Redis gem to version 4.1.3
I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

Redis gem is manipulating Redis config https://github.com/redis/redis-rb/blob/master/lib/redis/client.rb#L95
therefore we cannot pass the frozen config object.

Pass of the copy of the object is protecting original config
2019-10-21 09:59:24 +11:00
Sam Saffron ae2a56999e Revert "FIX: update Redis gem to version 4.1.3 (#8197)"
This reverts commit ab74a50d85.

We really want to upgrade redis, but discovered some edge cases
around failover we need to test.

Holding off on the upgrade till a bit more testing happens
2019-10-17 11:41:46 +11:00
Krzysztof Kotlarek ab74a50d85 FIX: update Redis gem to version 4.1.3 (#8197)
* FIX: update Redis gem to version 4.1.3

I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

* FIX: get rid of redis freedom patch
2019-10-17 08:49:23 +11:00
David Taylor 061c8874f5 FIX: Correct line count link in GitHub commit onebox
Bump onebox version
2019-10-15 23:52:59 +01:00
Sam Saffron c3cc96084c FIX: remove hiredis gem which is no longer needed
Previously some local micro-benchmarks revealed it was not giving any perf
benefits.

Now that we upgraded to 2.6.5 we are seeing some segfaults.

No need to carry this dependency around anymore.

We can re-evaluate in future if it improves perf and fix the segfaults.
2019-10-15 18:17:14 +11:00
romanrizzi 9845963105 FEATURE: Use the 'ugc' rel attribute alongside 'nofollow' 2019-10-14 15:21:48 -03:00
David Taylor 939a746dcd UX: Use theme colors for GitHub issue labels
Bump onebox version to pull tag rendering bug fix
2019-10-09 12:28:48 +01:00
David Taylor 3edd514c72 FEATURE: Redesigned GitHub oneboxes
Bump onebox version, and add new styling

Commit, PR and Issue oneboxes are updated with a new design. Timestamps are now localized using local-dates (if installed).
2019-10-09 11:47:58 +01:00
David Taylor e7cc7def8b UX: Stop using fixed-width font to render github issue description
Bump onebox version
2019-10-08 11:48:05 +01:00
David Taylor 615039f228 FEATURE: Improve GitHub commit, PR and issue onebox rendering
Bump onebox version to include new github rendering, and add relevant CSS

Avatars are reduced in size significantly, and icons are added to easily differentiate PRs and commits. The 'Issue:' prefix is removed from issue oneboxes, to make them consistent with commits and PRs.
2019-10-07 19:26:10 +01:00
Sam Saffron 8d5f47dded PREF: optimise preloading application
We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)

This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes

1. We attempt to localize a string on each site, ensuring we warmup
the i18n

2. We preload all our templates (compiling .erb to class)

3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
2019-10-07 00:33:37 -04:00
Martin Brennan 68d35b14f4 FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099)
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).

Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
2019-10-01 19:08:41 -07:00
Krzysztof Kotlarek 32b8a2ccff DEV: Upgrade Discourse to Rails 6 (#8083)
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
2019-09-12 10:41:50 +10:00
Arpit Jalan 4195548a17 Bump onebox version.
- indicate and link to Flickr Album
2019-09-11 23:23:11 +05:30
Sam Saffron ed00f35306 FEATURE: improve performance of anonymous cache
This commit introduces 2 features:

1. DISCOURSE_COMPRESS_ANON_CACHE (true|false, default false): this allows
you to optionally compress the anon cache body entries in Redis, can be
useful for high load sites with Redis that lives on a separate server to
to webs

2. DISCOURSE_ANON_CACHE_STORE_THRESHOLD (default 2), only pop entries into
redis if we observe them more than N times. This avoids situations where
a crawler can walk a big pile of topics and store them all in Redis never
to be used. Our default anon cache time for topics is only 60 seconds. Anon
cache is in place to avoid the "slashdot" effect where a single topic is
hit by 100s of people in one minute.
2019-09-04 17:18:32 +10:00
Arpit Jalan e9c971ba77 Bump onebox version.
- allow oneboxing for `www.amazon.com.mx`
2019-08-26 16:44:10 +05:30
Arpit Jalan 038bf02e33 Bump onebox version.
- strip whitespace from Twitter onebox
2019-08-21 10:19:54 +05:30
Arpit Jalan f27564a0a0 Bump onebox version.
- normalize and decode html entities for image URL
2019-08-19 19:01:23 +05:30
Arpit Jalan 44f4801087 Bump onebox version.
- do not double encode percentage in url
- support hashbang in url
2019-08-12 08:42:50 +05:30
Guo Xiang Tan 740e2a6025 DEV: Add comment about fork of `image_optim`. 2019-08-08 13:03:35 +08:00
Arpit Jalan bc5daa1466 Bump onebox version.
- prioritize `card_html` over `article_html`
2019-08-05 11:04:58 +05:30
Neil Lalonde 9656a21fdb
FEATURE: customization of html emails (#7934)
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.

Customizations can be added and edited in Admin > Customize > Email Style.

Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.

As part of this work, RTL locales are now rendered correctly for all emails.
2019-07-30 15:05:08 -04:00
Arpit Jalan 6a0787445c Bump onebox version.
- Deprioritize Twitter card in generic onebox
2019-07-25 17:13:23 +05:30
Roman Rizzi f5c707c97a
FEATURE: Gz to zip for exports (#7889)
* Revert "Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)""

This reverts commit f89bd55576.

* Replace .tar.zip with .zip
2019-07-18 09:34:48 -03:00
Joffrey JAFFEUX b3eb67976d
DEV: Upgrades to Ember 3.10 (#7871)
Co-Authored-By: majakomel <maja.komel@gmail.com>
2019-07-16 12:45:15 +02:00
Arpit Jalan 25830c73be Bump onebox version.
- use custom placeholder HTML for generic whitelisted oneboxes
- optimize usage of custom placeholder HTML
2019-07-11 18:31:51 +05:30
romanrizzi f89bd55576 Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)"
This reverts commit 8b2580e20f.
2019-07-10 11:38:51 -03:00
Roman Rizzi 8b2580e20f
FEATURE: admin/user exports are compressed using the zip format (#7784)
* FEATURE: admin/user exports are compressed using the zip format

* Update translations. Theme exporter now exports .zip file. Theme importer supports .zip and .gz files

* Fix controller test, updated locale and skip saving the csv export to disk
2019-07-10 11:13:03 -03:00
Arpit Jalan f0f271cd5f Bump onebox version.
- remove additional whitespace from Twitter onebox
2019-07-09 13:12:03 +05:30
Arpit Jalan bb8cf81089 Bump onebox version.
- better placeholders for audio/video/trello/typeform oneboxes
- added CSS for audio/video/trello/typeform onebox placeholders
2019-07-08 21:40:33 +05:30
Arpit Jalan feb828172b Bump onebox version.
- improved spacing for quoted twitter onebox
2019-07-06 09:41:01 +05:30
Arpit Jalan 5bc1fd23b0 Bump onebox version.
- update HTML for twitter quoted onebox
- updated CSS for twitter quoted onebox
2019-07-05 19:35:36 +05:30
Arpit Jalan 1ebc3cce4a Bump onebox version.
- twitter oneboxing using API was broken
2019-07-04 11:41:07 +05:30
Arpit Jalan 32edaa3843 Bump onebox version
- support for Twitter quoted tweets
2019-07-04 11:01:30 +05:30
Abroskin Alexander 0872a1182d DEV: Replace Overcommit with Lefthook (#7826)
Overcommit uses prebuilt hooks and require global installation.
To avoid this issues replace it with Lefthook.
Lefthook will be installed with npm packages. New contributors
will have fully consistent git hooks.
2019-07-02 11:29:52 +02:00