Commit Graph

470 Commits

Author SHA1 Message Date
Guo Xiang Tan 878f06f1fe DEV: Remove custom connection reaper.
Rails 6 fixed the reaper to use one thread to reap all the connection pools.
2020-05-26 09:09:46 +08:00
Michael Brown d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood 20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Guo Xiang Tan 2f03a879f9
DEV: Require `rails_failover` before global settings. 2020-05-20 16:06:40 +08:00
Guo Xiang Tan f7f436e536
DEV: Install `rails_failover` gem to test our Redis changes. 2020-05-20 15:40:27 +08:00
Guo Xiang Tan 96c02caba7
DEV: Change use of Redis `flushall` to `flushdb`.
FLUSHALL removes all keys from all databases. Instead we only want to
remove keys from the current Redis database.
2020-05-19 10:20:00 +08:00
Sam Saffron 609e929186
Revert "Revert "DEV: upgrade to Rails 6.0.3""
This reverts commit 2ff8b4f5d9.

Attempt #2 at a Rails update this time we also update the
rails_multisite gem to allow for cleaner reordering
2020-05-08 11:49:22 +10:00
David Taylor 2ff8b4f5d9
Revert "DEV: upgrade to Rails 6.0.3"
This was causing issues during multisite:migrate

https://meta.discourse.org/t/multisite-migrate-broken-since-rails-6-0-3-update/150691

This reverts commit 136a545653.
2020-05-07 11:44:39 +01:00
Sam Saffron 136a545653
DEV: upgrade to Rails 6.0.3
Upgrades Rails to latest, this version has better compatibility
with Ruby 2.7

During the upgrade we needed a new cleaner mechanism for configuring
message bus.

All tests are green.

If anything weird pops up please revert.
2020-05-07 15:53:40 +10:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
Jarek Radosz 17cf300b71
DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
Dan Ungureanu 3d9c320aab
PERF: Cache Category.subcategory_ids (#9350)
Also reset category cache after backup restore.
2020-04-09 15:42:24 +03:00
Kane York 138d4aebde FEATURE: Webhooks and Event for user being granted a badge
Adding a webhook for badge revocation is left for future work as it's relatively rare.
2020-04-08 14:38:30 -07:00
Daniel Waterworth bca126f3f5 REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.

Follow-up-to: ca1208a636
2020-04-02 16:44:44 +01:00
Daniel Waterworth ca1208a636 Revert "REFACTOR: Move the multisite middleware to the front"
Looks like this is causing problems.

Follow-up-to: a91843f0dc
2020-04-02 15:20:28 +01:00
Daniel Waterworth a91843f0dc REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.
2020-04-02 10:15:38 +01:00
Sam Saffron 25f1f23288
FEATURE: Stricter rules for user presence
Previously we would consider a user "present" and "last seen" if the
browser window was visible.

This has many edge cases, you could be considered present and around for
days just by having a window open and no screensaver on.

Instead we now also check that you either clicked, transitioned around app
or scrolled the page in the last minute in combination with window
visibility

This will lead to more reliable notifications via email and reduce load of
message bus for cases where a user walks away from the terminal
2020-03-26 17:36:52 +11:00
Martin Brennan 097851c135
FIX: Change secure media to encompass attachments as well (#9271)
If the “secure media” site setting is enabled then ALL files uploaded to Discourse (images, video, audio, pdf, txt, zip etc. etc.) will follow the secure media rules. The “prevent anons from downloading files” setting will no longer have any bearing on upload security. Basically, the feature will more appropriately be called “secure uploads” instead of “secure media”.

This is being done because there are communities out there that would like all attachments and media to be secure based on category rules but still allow anonymous users to download attachments in public places, which is not possible in the current arrangement.
2020-03-26 07:16:02 +10:00
David Taylor fb8b0ca197
DEV: Remove unused middleware (#9203)
This has not been used since January 2014, per 166a8d29
2020-03-16 12:37:43 +00:00
Gerhard Schlager 8022e51179 FIX: Failed to restore backups from versions without translation overrides
Rails calls I18n.translate during initialization and by default translation overrides are used. Database migrations would fail if the system tried to migrate from an old version that didn't have the `translation_overrides` table with all its columns yet.

This makes restoring really old backups work again. Running `DISABLE_TRANSLATION_OVERRIDES=1 rake db:migrate` will allow you to upgrade such an old database as well.
2020-03-14 00:00:22 +01:00
Robin Ward a3f0543f99
Support for transpiling `.js` files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
Roman Rizzi 537f87562e
FIX: We need to skip users with associated reviewables when auto-approving (#9080)
* FIX: We need to skip users with associated reviewables when auto-approving them

* Update spec/initializers/track_setting_changes_spec.rb

* Update spec/initializers/track_setting_changes_spec.rb

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-03-02 14:33:52 -05:00
Martin Brennan 14adddd18d
FIX: Ignore secure-media-uploads for miniprofiler (#9070) 2020-02-28 12:11:30 +10:00
Matt Palmer 377d2d3fad DEV: Silence spurious rubocop lint warning 2020-02-19 13:10:30 +11:00
Matt Palmer a14a7f1cb8 DEV: Add optional support for running byebug when a PG Clash happens
Tracking down concurrency issues from backtraces and manual repros is a fraught process.
Sometimes you've just got to get your hands dirty and do a live debug.
2020-02-19 12:50:37 +11:00
David Taylor 0b09f5299d
DEV: Improve pg connection access logging
`ensure` that the accessing thread is set to nil after an action
2020-02-18 16:58:47 +00:00
David Taylor ea49ca7ef5
DEV: Handle nil backtraces in pg access logs 2020-02-18 15:45:44 +00:00
David Taylor 2bdd1275ce
DEV: Initialize pg access log mutex in non-sidekiq processes
Followup to be3e4ab3f5
2020-02-18 14:20:28 +00:00
David Taylor be3e4ab3f5
DEV: Report simultaneous use of PG::Connection objects 2020-02-18 13:50:15 +00:00
Sam Saffron 28292d2759
PERF: avoid shelling to get hostname aggressively
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways

1. We cache the result in `Discourse.os_hostname` so it is only ever called once

2. We prefer to use Socket.gethostname which avoids making a shell command

This improves performance as we are not spawning hostname processes throughout
the app lifetime
2020-02-18 15:13:19 +11:00
David Taylor cd3fab9ccc
DEV: Allow raw PG tracing to be enabled only for sidekiq processes 2020-02-17 18:14:14 +00:00
David Taylor 0c6f2892c6
DEV: Add raw PG connection tracing behind an environment variable
This should be useful for debugging connection problems. Warning: this will generate some large files, and will likely impact performance
2020-02-17 16:21:26 +00:00
David Taylor 5919618a87
DEV: Drop legacy OpenID 2.0 support (#8894)
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.

Context at https://meta.discourse.org/t/-/113249
2020-02-07 17:32:35 +00:00
OsamaSayegh a516c5df82 DEV: Bump logster version to 2.6.1 and enable new logster feature
Logster 2.6.1 includes a few new features and fixes. More details here: 58bb5c5368/CHANGELOG.md
2020-02-07 13:35:26 +00:00
Sam Saffron f8e92298f2 DEV: default Oj to compat mode
Out-of-the-box Oj uses :object mode, this shifts us to use :compat mode
by default which is safer.
It means any de-serialization going forward will default to this mode.

If we wish to serialize or deserialize arbitrary objects going forward with
no json interfaces we will have to opt in.
2020-01-16 07:52:28 +11:00
David Taylor bc4c40abd4
DEV: Remove unsafe-eval from development CSP (#8569)
- Refactor source_url to avoid using eval in development
- Precompile handlebars in development
- Include template compilers when running qunit
- Remove unsafe-eval in development CSP
- Include unsafe-eval only for qunit routes in development
2019-12-30 12:17:12 +00:00
Gerhard Schlager 7aea7f2cae FIX: Track correct site setting 2019-12-24 14:11:37 +01:00
Robin Ward ce78eff888 FIX: Migration paths were being forgotten
According to the [Rails
Source](https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L20)
the `ActiveRecord::Migrator.migrations_paths` are overwritten with the
value of `ActiveRecord::Tasks::DatabaseTasks.migrations_paths` every
time the config is loaded.

This caused a bug for Discourse development where if you ran:

`rake db:drop db:create db:migrate` in one line, you would not get our
post migrations, as those had a custom value for `migrations_paths`.

The fix is to use `ActiveRecord::Tasks::DatabaseTasks.migrations_paths`
to set up all our custom paths. Everything seems to work as expected.
2019-12-16 14:13:47 -05:00
Vinoth Kannan e51091f199 REFACTOR: do `X-Frame-Options` header removal in application controller.
Co-authored-by: Sam <sam.saffron@gmail.com>
Previous commit: f7084a4339
2019-12-06 18:25:32 +05:30
Vinoth Kannan f7084a4339 FEATURE: add site setting to remove `X-Frame-Options` header. 2019-12-06 03:15:09 +05:30
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Arpit Jalan 6a417c308f FIX: include onebox default options in development environment 2019-11-07 15:42:53 +05:30
Gerhard Schlager 61b1f9c36b FEATURE: Load translation overrides without JS `eval` 2019-11-05 19:16:38 +01:00
Dan Ungureanu 5b84307774
FIX: Ensure that scheduled jobs are loaded. (#8183)
In development, the scheduled jobs are loaded lazily and MiniScheduler
cannot discover them (/sidekiq/scheduler does not show any jobs).
2019-10-14 12:14:16 +03:00
Daniel Waterworth 4c9ed7bd85 FIX: Fix rake db:create after zeitwerk changes
Post-zeitwerk, rails has deprecated autoloading modules during
initialization and forces all autoloaded modules to be reloaded after
initialization.

Requiring the file explicitly prevents autoloading and therefore
prevents the state on SiteSetting being trashed which was causing the
problem here.
2019-10-08 12:22:34 +01:00
Krzysztof Kotlarek 302e8f4393 FIX: Use migrations path for post_migrate (#8133)
That is a problem after upgrade to Rails 6. It was partially fixed here: 025d4ee91f
2019-10-02 15:28:38 +10:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Krzysztof Kotlarek f64c9f37fa FIX: Remove versions from Active Record warm up (#8105) 2019-09-18 17:59:51 -04:00
Robin Ward 30bba6252d Allow CORS combined with HTTP Auth 2019-09-09 15:02:20 -04:00
Sam Saffron 098f9e8b5b PERF: Run multiple threads for regular job schedules
Under extreme load on large databases certain regular jobs can take quite
a while to run. We need to ensure we never starve a sidekiq from running
mini scheduler, cause without it we are unable to queue stuff such as
heartbeat jobs.
2019-08-29 15:34:36 +10:00