Commit Graph

634 Commits

Author SHA1 Message Date
David Taylor eaf6096890 DEV: Use rubocop-discourse gem to add custom chdir cop
Followup to b27e009655
2019-11-18 15:39:41 +00:00
Sam Saffron 26c0199c01 DEV: update Rails to version 6.0.1
This version of Rails eliminates a monkey patch that is no longer needed!

Additionally it preps us for Ruby 2.7 support.
2019-11-08 16:56:30 +11:00
Arpit Jalan c5df853dea Bump onebox version.
- fix for gfycat onebox in email
2019-11-07 10:03:12 +05:30
Arpit Jalan cb9702bf7a Bump onebox version.
- Remove native caching
- FIX: dropbox videos were not loading
2019-11-04 10:46:20 +05:30
Arpit Jalan 12409f63a0 Bump onebox version.
- FIX: Follow redirect returns url if response code is 200
- FIX: do not resize xkcd image
2019-10-22 12:26:01 +05:30
Krzysztof Kotlarek 858cf5836c
FIX: update Redis gem to version 4.1.3
I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

Redis gem is manipulating Redis config https://github.com/redis/redis-rb/blob/master/lib/redis/client.rb#L95
therefore we cannot pass the frozen config object.

Pass of the copy of the object is protecting original config
2019-10-21 09:59:24 +11:00
Sam Saffron ae2a56999e Revert "FIX: update Redis gem to version 4.1.3 (#8197)"
This reverts commit ab74a50d85.

We really want to upgrade redis, but discovered some edge cases
around failover we need to test.

Holding off on the upgrade till a bit more testing happens
2019-10-17 11:41:46 +11:00
Krzysztof Kotlarek ab74a50d85 FIX: update Redis gem to version 4.1.3 (#8197)
* FIX: update Redis gem to version 4.1.3

I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

* FIX: get rid of redis freedom patch
2019-10-17 08:49:23 +11:00
David Taylor 061c8874f5 FIX: Correct line count link in GitHub commit onebox
Bump onebox version
2019-10-15 23:52:59 +01:00
Sam Saffron c3cc96084c FIX: remove hiredis gem which is no longer needed
Previously some local micro-benchmarks revealed it was not giving any perf
benefits.

Now that we upgraded to 2.6.5 we are seeing some segfaults.

No need to carry this dependency around anymore.

We can re-evaluate in future if it improves perf and fix the segfaults.
2019-10-15 18:17:14 +11:00
romanrizzi 9845963105 FEATURE: Use the 'ugc' rel attribute alongside 'nofollow' 2019-10-14 15:21:48 -03:00
David Taylor 939a746dcd UX: Use theme colors for GitHub issue labels
Bump onebox version to pull tag rendering bug fix
2019-10-09 12:28:48 +01:00
David Taylor 3edd514c72 FEATURE: Redesigned GitHub oneboxes
Bump onebox version, and add new styling

Commit, PR and Issue oneboxes are updated with a new design. Timestamps are now localized using local-dates (if installed).
2019-10-09 11:47:58 +01:00
David Taylor e7cc7def8b UX: Stop using fixed-width font to render github issue description
Bump onebox version
2019-10-08 11:48:05 +01:00
David Taylor 615039f228 FEATURE: Improve GitHub commit, PR and issue onebox rendering
Bump onebox version to include new github rendering, and add relevant CSS

Avatars are reduced in size significantly, and icons are added to easily differentiate PRs and commits. The 'Issue:' prefix is removed from issue oneboxes, to make them consistent with commits and PRs.
2019-10-07 19:26:10 +01:00
Sam Saffron 8d5f47dded PREF: optimise preloading application
We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)

This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes

1. We attempt to localize a string on each site, ensuring we warmup
the i18n

2. We preload all our templates (compiling .erb to class)

3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
2019-10-07 00:33:37 -04:00
Martin Brennan 68d35b14f4 FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099)
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).

Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
2019-10-01 19:08:41 -07:00
Krzysztof Kotlarek 32b8a2ccff DEV: Upgrade Discourse to Rails 6 (#8083)
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
2019-09-12 10:41:50 +10:00
Arpit Jalan 4195548a17 Bump onebox version.
- indicate and link to Flickr Album
2019-09-11 23:23:11 +05:30
Sam Saffron ed00f35306 FEATURE: improve performance of anonymous cache
This commit introduces 2 features:

1. DISCOURSE_COMPRESS_ANON_CACHE (true|false, default false): this allows
you to optionally compress the anon cache body entries in Redis, can be
useful for high load sites with Redis that lives on a separate server to
to webs

2. DISCOURSE_ANON_CACHE_STORE_THRESHOLD (default 2), only pop entries into
redis if we observe them more than N times. This avoids situations where
a crawler can walk a big pile of topics and store them all in Redis never
to be used. Our default anon cache time for topics is only 60 seconds. Anon
cache is in place to avoid the "slashdot" effect where a single topic is
hit by 100s of people in one minute.
2019-09-04 17:18:32 +10:00
Arpit Jalan e9c971ba77 Bump onebox version.
- allow oneboxing for `www.amazon.com.mx`
2019-08-26 16:44:10 +05:30
Arpit Jalan 038bf02e33 Bump onebox version.
- strip whitespace from Twitter onebox
2019-08-21 10:19:54 +05:30
Arpit Jalan f27564a0a0 Bump onebox version.
- normalize and decode html entities for image URL
2019-08-19 19:01:23 +05:30
Arpit Jalan 44f4801087 Bump onebox version.
- do not double encode percentage in url
- support hashbang in url
2019-08-12 08:42:50 +05:30
Guo Xiang Tan 740e2a6025 DEV: Add comment about fork of `image_optim`. 2019-08-08 13:03:35 +08:00
Arpit Jalan bc5daa1466 Bump onebox version.
- prioritize `card_html` over `article_html`
2019-08-05 11:04:58 +05:30
Neil Lalonde 9656a21fdb
FEATURE: customization of html emails (#7934)
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.

Customizations can be added and edited in Admin > Customize > Email Style.

Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.

As part of this work, RTL locales are now rendered correctly for all emails.
2019-07-30 15:05:08 -04:00
Arpit Jalan 6a0787445c Bump onebox version.
- Deprioritize Twitter card in generic onebox
2019-07-25 17:13:23 +05:30
Roman Rizzi f5c707c97a
FEATURE: Gz to zip for exports (#7889)
* Revert "Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)""

This reverts commit f89bd55576.

* Replace .tar.zip with .zip
2019-07-18 09:34:48 -03:00
Joffrey JAFFEUX b3eb67976d
DEV: Upgrades to Ember 3.10 (#7871)
Co-Authored-By: majakomel <maja.komel@gmail.com>
2019-07-16 12:45:15 +02:00
Arpit Jalan 25830c73be Bump onebox version.
- use custom placeholder HTML for generic whitelisted oneboxes
- optimize usage of custom placeholder HTML
2019-07-11 18:31:51 +05:30
romanrizzi f89bd55576 Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)"
This reverts commit 8b2580e20f.
2019-07-10 11:38:51 -03:00
Roman Rizzi 8b2580e20f
FEATURE: admin/user exports are compressed using the zip format (#7784)
* FEATURE: admin/user exports are compressed using the zip format

* Update translations. Theme exporter now exports .zip file. Theme importer supports .zip and .gz files

* Fix controller test, updated locale and skip saving the csv export to disk
2019-07-10 11:13:03 -03:00
Arpit Jalan f0f271cd5f Bump onebox version.
- remove additional whitespace from Twitter onebox
2019-07-09 13:12:03 +05:30
Arpit Jalan bb8cf81089 Bump onebox version.
- better placeholders for audio/video/trello/typeform oneboxes
- added CSS for audio/video/trello/typeform onebox placeholders
2019-07-08 21:40:33 +05:30
Arpit Jalan feb828172b Bump onebox version.
- improved spacing for quoted twitter onebox
2019-07-06 09:41:01 +05:30
Arpit Jalan 5bc1fd23b0 Bump onebox version.
- update HTML for twitter quoted onebox
- updated CSS for twitter quoted onebox
2019-07-05 19:35:36 +05:30
Arpit Jalan 1ebc3cce4a Bump onebox version.
- twitter oneboxing using API was broken
2019-07-04 11:41:07 +05:30
Arpit Jalan 32edaa3843 Bump onebox version
- support for Twitter quoted tweets
2019-07-04 11:01:30 +05:30
Abroskin Alexander 0872a1182d DEV: Replace Overcommit with Lefthook (#7826)
Overcommit uses prebuilt hooks and require global installation.
To avoid this issues replace it with Lefthook.
Lefthook will be installed with npm packages. New contributors
will have fully consistent git hooks.
2019-07-02 11:29:52 +02:00
Arpit Jalan 7f14e185cc Bump onebox version.
- prevent double escaping of URL
2019-06-27 23:35:35 +05:30
Arpit Jalan 168a38dc29 Bump onebox version.
- better Twitch placeholder
- CSS for said placeholder
2019-06-26 23:22:29 +05:30
Arpit Jalan 2af4002817 Bump onebox version.
- Add a placeholder icon for Twitch onebox preview
- Add CSS for showing fontawesome play icon for placeholder class
2019-06-26 13:40:14 +05:30
Guo Xiang Tan c3381b845b DEV: Install diffy in production for `rake posts:inline_uploads` task. 2019-06-14 14:42:53 +08:00
Guo Xiang Tan 29259b46ae DEV: Verbose mode for `posts:inline_uploads` rake task. 2019-06-10 08:59:11 +08:00
Guo Xiang Tan 1991af2abb DEV: Switch `InlineUploads` to a regexp based implementation. 2019-06-04 15:54:25 +08:00
Guo Xiang Tan d93e5fb00d DEV: Class that converts MD with old attachment links to new MD. 2019-06-04 15:54:25 +08:00
David Taylor 8511bfe583 Bump omniauth-google-oauth2 gem version
Pinning to an old version is no longer required following 8b4d6dafea
2019-06-03 19:17:00 +01:00
Joffrey JAFFEUX ee43b36b64
Bump onebox version. (#7666)
Fixes multiple possible sources of exceptions due to frozen strings. Wikipedia onebox was definitely failing before this patch.
2019-05-31 17:04:34 +02:00
Joffrey JAFFEUX 75d413ad11
Bump onebox version. (#7665)
Fixes a regression with soundclound onebox due to frozen string literal.
2019-05-31 16:41:33 +02:00
Arpit Jalan bf3c781f26 Bump onebox version.
- add frozen string literal to all the files
2019-05-28 17:39:42 +05:30
Arpit Jalan ce89f19250 Bump onebox version.
- use Vimeo engine for private links only
- if og:video_url is missing, make one using Vimeo ID
2019-05-20 12:24:43 +05:30
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Arpit Jalan d679c4e0eb Bump onebox version.
- FIX: encode the URL per RFC 3986 spec
2019-05-06 18:17:42 +05:30
romanrizzi 2ebe9e3a8b Bump onebox version 2019-04-30 10:07:48 -03:00
Guo Xiang Tan 09b3d0c2a0
DEV: Only install danger on Travis. (#7452) 2019-04-29 14:45:24 +08:00
Sam Saffron 09638fcd27 DEV: add note explaining why omniauth-google-oauth2 is pinned 2019-04-29 15:47:05 +10:00
Maja Komel 4b455e741e DEV: Ember 3.8.0
Co-Authored-By: majakomel <maja.komel@gmail.com>
2019-04-26 12:16:21 +02:00
Arpit Jalan e8f51815e5 Bump onebox version.
- Update github_blob engine to support displaying stl files
- FEATURE: add `data-original-href` attribute to Vimeo iframes
- Add poster image for video oneboxes
2019-04-24 13:59:14 +05:30
Guo Xiang Tan b3dcaacdf4 Update Rails to 5.2.3. 2019-04-20 10:49:54 +09:00
Daniel Waterworth 7e3628d11f Added test-prof as a dependency (#7395)
test-prof is a collection of tools for analyzing test-suite performance.
2019-04-19 10:52:31 +02:00
Nicolas Sebastian Vidal 2b8487b0ea Removed "shoulda" gem in favor of "shoulda-matchers" and update (#7387)
* Update shoulda gem

* Remove shoulda gem in favor of shoulda-matchers only
2019-04-18 07:41:37 +10:00
Roman Rizzi 76e76140e1 Bump onebox version 2019-04-12 10:28:36 -03:00
Arpit Jalan 9c1d1777db Bump onebox version
- adds support for oneboxing google drive files
- add styling for google drive onebox favicon
2019-04-10 13:37:24 +05:30
Maja Komel b0053f3a1c FEATURE: bump onebox version, add styling for new reddit image onebox 2019-04-04 11:24:30 +02:00
David Taylor b375dcb14a DEV: Introduce parallel rspec testing
Adds the parallel_tests gem, and redis/postgres configuration for running rspec tests in parallel. To use:

```
rake parallel:rake[db:create]
rake parallel:rake[db:migrate]
rake parallel:spec
```

This brings the test suite from 12m20s to 3m11s on my macOS machine
2019-04-01 11:06:47 -04:00
Gerhard Schlager cadd1d670f DEV: Add simplecov as test dependency (#7271) 2019-04-01 16:00:11 +11:00
David Taylor 59491f3047 FIX: Add `sassc-rails` for plugins using sprockets to compile scss
This did not affect core because we have a custom stylesheet pipeline
2019-03-21 23:23:29 +00:00
Maja Komel f3d0d8fe7d Upgrade to Ember 3.7.0 2019-03-20 14:43:25 +01:00
Roman Rizzi bd8e46a9c1 SECURITY: Upgrading Rails version to 5.2.2.1 2019-03-13 16:24:54 -03:00
Roman Rizzi 77931b70c3
Revert "DEV: Upgrade to Ember 3.7.0 (#6977)" (#7165)
This reverts commit 3eebf8be73.
2019-03-13 15:49:47 -03:00
Maja Komel 3eebf8be73
DEV: Upgrade to Ember 3.7.0 (#6977)
* Upgrade to Ember 3.7.0

* use ember source 3.7.0.2

* fix mobile header

* fix navigation
2019-03-13 12:16:06 +01:00
Arpit Jalan a9648e8fd1 onebox version bump
- FIX: respect code indentation
2019-03-07 17:55:47 +05:30
Vinoth Kannan 1b1f9831b0 Bump onebox version
discourse/onebox@4dd5a62
2019-03-06 11:58:41 +05:30
Gerhard Schlager 33129efdb5 Revert "Bump onebox version"
This reverts commit 345f6237cb.
2019-03-05 22:51:02 +01:00
Gerhard Schlager 345f6237cb Bump onebox version
f2b361fc28
2019-03-05 22:18:49 +01:00
Gerhard Schlager c36c9c2ee5 FEATURE: Import script for AnswerBase
Improves the generic database used by some import scripts:
* Adds additional columns for users
* Adds support for attachments
* Allows setting the data type for keys (numeric or string) to ensure correct sorting
2019-02-28 22:08:12 +01:00
Arpit Jalan 2d247cc4e9 Bump onebox version
- deafult to dedicated vimeo and gfycat engine
2019-02-26 10:50:27 +05:30
Sam Saffron 83f13ecf82 FEATURE: bump onebox dependency
- Adds support of kaltura oneboxes
- Adds support for typeform oneboxes
2019-02-19 15:22:43 +11:00
Régis Hanol 4d674acc25 FEATURE: AWS SNS bounce notifications webhooks 2019-02-13 21:26:40 +01:00
Vinoth Kannan 36ff971c9c FIX: Bump onebox version to include imgur security fix 2019-02-13 01:11:24 +05:30
Sam fe7c10b409 DEV: fix seed-fu require for rails 6 2019-02-06 17:33:36 +11:00
Sam 448ea663c3 DEV: remove seed-fu pinning from rails master
This pinning should no longer be needed
2019-02-06 16:54:06 +11:00
Sam 635bc72ec5 DEV: pin redis to version 4.0.1
Version 4.1.0 returns frozen hashes which conflict with our monkey patch

We will follow up unpinning this later
2019-02-05 09:08:44 +11:00
Sam 9f5bbd663d DEV: update mini_racer and message_bus
Two very low risk updates, message_bus has been released no need to depend
on pre-release.

mini_racer update is for a very minor change (shared isolates are not used
in discourse so it is not a fix we technically need)
2019-02-04 17:46:38 +11:00
Sam 1d2c4b0eee DEV: we are waiting on an annotate gem release
Once version 2.7.5 is released per: https://github.com/ctran/annotate_models/pull/595
we can drop this conditional.
2019-02-04 16:11:19 +11:00
Sam 1816bdf46e DEV: upgrade mail gem from pre-release
2.7.1 version of the mail gem was released! We no longer need to depend on
the pre-release.
2019-02-04 16:10:13 +11:00
Sam 377f3efb60 DEV: remove foreman gem and unsupported Procfile
Launching Discourse no longer should require foreman in dev. We can simply
use `bin/unicorn` which automatically launches a sidekiq worker.

The foreman gem depends on thor ~> 0.19.1 which is no longer supported in
rails 6. So this pre-emptively prepares us for it.
2019-02-04 15:05:54 +11:00
Guo Xiang Tan 53c0ad9388 Revert "DEV: Only install danger on Travis."
This reverts commit 792dd033e6.
2019-01-21 11:31:16 +08:00
Guo Xiang Tan 792dd033e6 DEV: Only install danger on Travis. 2019-01-21 09:46:32 +08:00
Joffrey JAFFEUX f9648de897
DEV: upgrades from Ember 2.13 to Ember 3.5.1 (#6808)
Co-Authored-By: Bianca Nenciu <nbianca@users.noreply.github.com>
Co-Authored-By: David Taylor <david@taylorhq.com>
2019-01-10 11:06:01 +01:00
David Taylor 1f0708981f FIX: Bump onebox version for gfycat aspect ratio fix 2019-01-09 18:00:28 +00:00
David Taylor 286cc72c8b FIX: Gyfcat onebox should have fixed aspect ratio videos
(Fixed upstream in the onebox gem)
2019-01-09 17:15:15 +00:00
David Taylor fe20cb4b56 FIX: Enforce a fixed height on generic oneboxed videos
This prevents 'jumping' as the video loads. This change will require posts to be rebaked before it takes effect.
2019-01-08 16:22:03 +00:00
Arpit Jalan c76c44bc66 bump onebox version
- FEATURE: Add support for Twitter cards.
- FIX: add more https hosts
2019-01-08 09:20:08 +05:30
Arpit Jalan a8a0f97157 bump onebox version
- FEATURE: support YouTube `rel` parameter
2018-12-25 06:36:51 +05:30
Joffrey JAFFEUX 7b253dbe4a
FIX: gfycat oneboxing 2018-12-22 11:16:18 +01:00
Arpit Jalan 59cb907f25 Bump onebox version
- update Twitter status icons
2018-12-13 06:25:50 +05:30
Sam fcb3f1e219 DEV: upgrade Rails to version 5.2.2 2018-12-10 11:29:28 +11:00
Sam 8868cfd2e4 FIX: redis leak when visiting large amounts of topics
Message bus uses a key to keep track of the last id for each channel
this key was never expired even if channel data expired

This change ensures we also expire the tracking key, it means a lot to us
cause each topic has a channel, so if you have 1 million topics that is
1 million keys that may persist forever
2018-11-30 14:41:15 +11:00