Commit Graph

717 Commits

Author SHA1 Message Date
Alan Guo Xiang Tan 384068f4c7
DEV: Remove webrick dependency which we no longer need (#22738)
The `discourse-prometheus` plugin has since specificed the depedency on
webrick in the plugin so we no longer need to carry this in core.

See c4b675f0fe
2023-07-21 13:12:32 +08:00
Alan Guo Xiang Tan 351005ef1b
DEV: Upgrade MessageBus to latest version. (#22355)
Previously it was pinned to 4.3.2 because 4.3.3 was broken. This has now
been resolved so we no longer need to pin MessageBus.
2023-06-30 08:14:16 +08:00
Gerhard Schlager 50f710bcae
DEV: Bump oj from 3.13.14 to 3.15.0 (#22352)
https://github.com/ohler55/oj/issues/789 has been fixed and SSE4.2 is disabled by default.
2023-06-30 07:39:55 +08:00
Greg Molnar 4d3999de10
DEV: Upgrade Rails to 7.0.5.1 (#22305)
See discuss.rubyonrails.org/t/cve-2023-28362-possible-xss-via-user-supplied-values-to/83132

Impact of this vulnerability has been assess to be very low for Discourse since XSS attacks are mitigated by Discourse's default CSP.
2023-06-28 07:18:49 +08:00
Sam 9e241e82e9
DEV: use HTML5 version of loofah (#21522)
https://meta.discourse.org/t/markdown-preview-and-result-differ/263878

The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-20 09:49:22 +08:00
Martin Brennan a510de1f6e
DEV: Roll back MessageBus version (#22199)
We are having issues with a lot of MessageBus updates not coming
through, it seems like the poll is not reconnecting after hanging
up. Pinning to the version before this commit to check:

a2a46fde87
2023-06-20 10:24:47 +10:00
David Taylor d3e5251704
PERF: Use `OpenSSL::KDF` for Pbkdf2 implementation (#20982)
This was introduced to the standard library in Ruby 2.4. In my testing, it produces the same result, and is around 8x faster than our pure-ruby implementation
2023-04-05 17:00:05 +01:00
Jarek Radosz ef0016cdba
DEV: Drop our mail gem fork (#20805)
Didn't happen in https://github.com/discourse/discourse/pull/16622 but now that the official release is fixed - let's do this :P
2023-03-24 20:10:28 +01:00
Blake Erickson d0c6b33cc2
SECURITY: Bump Rails to v7.0.4.3 (#20675) 2023-03-14 10:19:31 -06:00
Alan Guo Xiang Tan d5ebcc3309
DEV: Remove ruby-lsp from Gemfile (#20595) 2023-03-09 05:34:54 +08:00
Alan Guo Xiang Tan 990b710ade
DEV: Add ruby_lsp gem to development (#20517)
In order to use the ruby-lsp vscode extension, the ruby_lsp gem needs to
be added to the project's Gemfile. That may soon change with
https://github.com/Shopify/vscode-ruby-lsp/pull/419 but this will do for
now.
2023-03-03 06:45:52 +08:00
Penar Musaraj 3cadeaf90f
FIX: Add support for Europe/Kyiv timezone (#20394) 2023-02-21 11:52:04 -05:00
Osama Sayegh 75ae70c27c
DEV: rtlcss_wrapper renamed to rtlcss (#20331)
The `rtlcss_wrapper` gem has been renamed to `rtlcss` per bd89847a39.
2023-02-16 18:15:56 +03:00
Martin Brennan 60ad836313
DEV: Chat service object initial implementation (#19814)
This is a combined work of Martin Brennan, Loïc Guitaut, and Joffrey Jaffeux.

---

This commit implements a base service object when working in chat. The documentation is available at https://discourse.github.io/discourse/chat/backend/Chat/Service.html

Generating documentation has been made as part of this commit with a bigger goal in mind of generally making it easier to dive into the chat project.

Working with services generally involves 3 parts:

- The service object itself, which is a series of steps where few of them are specialized (model, transaction, policy)

```ruby
class UpdateAge
  include Chat::Service::Base

  model :user, :fetch_user
  policy :can_see_user
  contract
  step :update_age

  class Contract
    attribute :age, :integer
  end

  def fetch_user(user_id:, **)
    User.find_by(id: user_id)
  end

  def can_see_user(guardian:, **)
    guardian.can_see_user(user)
  end

  def update_age(age:, **)
    user.update!(age: age)
  end
end
```

- The `with_service` controller helper, handling success and failure of the service within a service and making easy to return proper response to it from the controller

```ruby
def update
  with_service(UpdateAge) do
    on_success { render_serialized(result.user, BasicUserSerializer, root: "user") }
  end
end
```

- Rspec matchers and steps inspector, improving the dev experience while creating specs for a service

```ruby
RSpec.describe(UpdateAge) do
  subject(:result) do
    described_class.call(guardian: guardian, user_id: user.id, age: age)
  end

  fab!(:user) { Fabricate(:user) }
  fab!(:current_user) { Fabricate(:admin) }

  let(:guardian) { Guardian.new(current_user) }
  let(:age) { 1 }

   it { expect(user.reload.age).to eq(age) }
end
```

Note in case of unexpected failure in your spec, the output will give all the relevant information:

```
  1) UpdateAge when no channel_id is given is expected to fail to find a model named 'user'
     Failure/Error: it { is_expected.to fail_to_find_a_model(:user) }

       Expected model 'foo' (key: 'result.model.user') was not found in the result object.

       [1/4] [model] 'user' 
       [2/4] [policy] 'can_see_user'
       [3/4] [contract] 'default'
       [4/4] [step] 'update_age'

       /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/update_age.rb:32:in `fetch_user': missing keyword: :user_id (ArgumentError)
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `instance_exec'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:219:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `block in run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `each'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:411:in `run'
       	from <internal:kernel>:90:in `tap'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:302:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/spec/services/update_age_spec.rb:15:in `block (3 levels) in <main>'
```
2023-02-13 13:09:57 +01:00
Rafael dos Santos Silva 6e522e4aad
DEV: Move to Sass compilation to dart-sass (#19910)
This PR is a major change to Sass compilation in Discourse.

The new version of sass-ruby moves to dart-sass putting we back on the supported version of Sass. It does so while keeping compatibility with the existing method signatures, so minimal change is needed in Discourse for this change.

This moves us

From:
  - sassc 2.0.1 (Feb 2019)
  - libsass 3.5.2 (May 2018)

To:
  - dart-sass 1.58

This update applies the following breaking changes:

> 
> These breaking changes are coming soon or have recently been released:
> 
>  [Functions are stricter about which units they allow](https://sass-lang.com/documentation/breaking-changes/function-units) beginning in Dart Sass 1.32.0.
> 
>  [Selectors with invalid combinators are invalid](https://sass-lang.com/documentation/breaking-changes/bogus-combinators) beginning in Dart Sass 1.54.0.
> 
>  [/ is changing from a division operation to a list separator](https://sass-lang.com/documentation/breaking-changes/slash-div) beginning in Dart Sass 1.33.0.
> 
>  [Parsing the special syntax of @-moz-document will be invalid](https://sass-lang.com/documentation/breaking-changes/moz-document) beginning in Dart Sass 1.7.2.
> 
>  [Compound selectors could not be extended](https://sass-lang.com/documentation/breaking-changes/extend-compound) in Dart Sass 1.0.0 and Ruby Sass 4.0.0.


SCSS files have been migrated automatically using `sass-migrator division app/assets/stylesheets/**/*.scss`
2023-02-07 12:24:57 -03:00
Osama Sayegh f94951147e
FIX: Replace R2 gem with rtlcss for generating RTL CSS (#19636)
We've had a couple of problems with the R2 gem where it generated a broken RTL CSS bundle that caused a badly broken layout when Discourse is used in an RTL language, see a3ce93b and 5926386. For this reason, we're replacing R2 with `rtlcss` that can handle modern CSS features better than R2 does.

`rltcss` is written in JS and available as an npm package. Calling the `rltcss` from rubyland is done via the `rtlcss_wrapper` gem which contains a distributable copy of the `rtlcss` package and loads/calls it with Mini Racer. See https://github.com/discourse/rtlcss_wrapper for more details.

Internal topic: t/76263.
2023-02-01 14:21:15 +03:00
David Taylor 87316d7a10
SECURITY: Bump Rails to v7.0.4.1 (#19956) 2023-01-23 15:38:49 -05:00
David Taylor ef437a1e41
DEV: Bump Rails to 7.0.4 (#19881) 2023-01-17 09:54:50 +00:00
David Taylor ce6335693a
DEV: Bump sprockets to include ERB kwargs fix (#19850)
This should resolve these warnings under Ruby 3.1

```
warning: Passing safe_level with the 2nd argument of ERB.new is deprecated
```

Unfortunately Sprockets 3.x has not seen a rubygems release since 2018, so we need to fetch these improvements via git.
2023-01-13 10:52:05 +00:00
Rafael dos Santos Silva 192d8c25e6
DEV: Move back to web-push gem (#19849)
Our fork was needed for OpenSSL 3 and Ruby 2.X compatibility.

The OpenSSL 3 part was merged into the gem for version 3.

Discourse dropped support for Ruby 2.X.

That means we don't need our fork anymore.
2023-01-12 12:12:19 -03:00
David Taylor 0cf6421716
DEV: Apply syntax_tree formatting to `Gemfile` 2023-01-09 11:13:33 +00:00
David Taylor 93e2dad656
DEV: Introduce syntax_tree code formatter (#19775)
This commit introduces the necessary gems and config, but adds all our ruby code directories to the `--ignore-files` list.

Future commits will apply syntax_tree to parts of the codebase, removing the ignore patterns as we go
2023-01-07 11:11:08 +00:00
Rafael dos Santos Silva 7b53973bd8
DEV: Use WebPush fork for OpenSSL 3 compat (#19627)
* DEV: Use WebPush fork for OpenSSL 3 compat

* add some context on gemfile changes
2022-12-27 15:28:13 -03:00
Rafael dos Santos Silva e430f22936
FIX: Ensure we have a patched version of CGI gem (#19588)
* FIX: Ensure we have a patched version of CGI gem

Per https://github.com/ruby/cgi/pull/29 the current shipped version of
the CGI gem doesn't allow for leading dots in domain names, which breaks
setting cookies like `.example.com`.

* Update Gemfile

Co-authored-by: Jarek Radosz <jradosz@gmail.com>

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2022-12-22 19:52:20 -03:00
Rafael dos Santos Silva aa1e8790e5
FIX: Prometheus plugin expects webrick in production (#19562)
* FIX: Prometheus plugin expects webrick in production

* newline
2022-12-21 18:20:07 -03:00
Rafael dos Santos Silva 7c5744a4cb
DEV: Ruby 3.1 compat (#19543)
* DEV: Ruby 3.1 compat

* DEV: Add webrick to test/dev

* DEV: Review ruby2_keywords license

* move ruby2_keywords to ignored

* move openssl to ignored
2022-12-21 01:20:43 -03:00
Leonardo Mosquera bfecbde837
Fixes for vBulletin bulk importer (#17618)
* Allow taking table prefix from env var

* FIX: remove unused column references

The columns `filedata` and `extension` are not present in a v4.2.4
database, and they aren't used in the method anyways.

* FIX: report progress for tables without imported_id

* FIX: effectively check for AR validation errors

NOTE: other migration scripts also have this problem; see /t/58202

* FIX: properly count Posts when importing attachments

* FIX: improve logging

* Remove leftover comment

* FIX: show progress when exporting Permalink file

* PERF: stream Permalink file

The current way results in tons of memory usage; write once per line instead

* Document fixes needed

* WIP - deduplicate category names

* Ignore non alphanumeric chars for grouping

* FIX: properly deduplicate user emails by merging accounts

* FIX: don't merge empty UserEmails

* Improve logging

* Merge users AFTER fixing primary key sequences

* Parallelize user merging

* Save duplicated users structure for debugging purposes

* Add progress logging for the (multiple hour) user merging step
2022-11-28 16:30:19 -03:00
David Taylor 84bec1cbae
DEV: Cleanup legacy asset compilation gems and code (#19177)
We now use Ember CLI (core/plugins) and DiscourseJSProcessor (themes) for all Ember and template compilation. This commit removes the remnants of the legacy Sprockets-based Ember compilation system.

Sprockets, and its DiscourseJSProcess-based Babel transformations, is still in use for a few assets. Ideally that will be removed/replaced in the near future.
2022-11-24 12:13:59 +00:00
Jarek Radosz b5005e3574
DEV: Always use non-builtin net-* and digest gems (#18560)
This will possibly fix the issue we're having with Dependabot. It seems it now uses a different ruby version (i.e. 3.1+)
2022-10-12 15:22:22 +02:00
Krzysztof Kotlarek e6b5b6eae3
DEV: update to patched discourse-seed-fu (#18493)
Original seed-fu was cloned and patched with David's fix 576b69a368
2022-10-07 09:16:04 +11:00
David Taylor e8b1021cb6
DEV: Add faraday and faraday-retry as explicit dependencies (#18473)
`Faraday` is very commonly used in official and third-party plugins, and we will likely be increasing our use of it in core. This commit adds it as a direct dependency and adds the official faraday-retry gem which is very commonly used (e.g. by Octokit).
2022-10-05 13:19:04 +01:00
Martin Brennan 57caf08e13
DEV: Minimal first pass of rails system test setup (#16311)
This commit introduces rails system tests run with chromedriver, selenium,
and headless chrome to our testing toolbox.

We use the `webdrivers` gem and `selenium-webdriver` which is what
the latest Rails uses so the tests run locally and in CI out of the box.

You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra
verbose logs of what selenium is doing to communicate with the system
tests.

By default JS logs are verbose so errors from JS are shown when
running system tests, you can disable this with
`SELENIUM_DISABLE_VERBOSE_JS_LOGS=1`

You can use `SELENIUM_HEADLESS=0` to run the system
tests inside a chrome browser instead of headless, which can be useful to debug things
and see what the spec sees. See note above about `bin/ember-cli` to avoid
surprises.

I have modified `bin/turbo_rspec` to exclude `spec/system` by default,
support for parallel system specs is a little shaky right now and we don't
want them slowing down the turbo by default either.

### PageObjects and System Tests

To make querying and inspecting parts of the page easier
and more reusable inbetween system tests, we are using the
concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in
our system tests. A "Page" here is generally corresponds to
an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`,
and this contains logic for querying components within the topic
such as "Posts".

I have also split "Modals" into their own entity. Further down the
line we may want to explore creating independent "Component"
contexts.

Capybara DSL should be included in each PageObject class,
reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl

For system tests, since they are so slow, we want to focus on
the "happy path" and not do every different possible context
and branch check using them. They are meant to be overarching
tests that check a number of things are correct using the full stack
from JS and ember to rails to ruby and then the database.

### CI Setup

Whenever a system spec fails, a screenshot
is taken and a build artifact is produced _after the entire CI run is complete_,
which can be downloaded from the Actions UI in the repo.

Most importantly, a step to build the Ember app using Ember CLI
is needed, otherwise the JS assets cannot be found by capybara:

```
- name: Build Ember CLI
  run: bin/ember-cli --build
```

A new `--build` argument has been added to `bin/ember-cli` for this
case, which is not needed locally if you already have the discourse
rails server running via `bin/ember-cli -u` since the whole server is built and
set up by default.

Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 11:48:16 +10:00
David Taylor 53ee8746f6
DEV: Bump rubocop config and switch back to rubygems (#18379) 2022-09-27 12:07:09 +01:00
Loïc Guitaut cc515e2c1c DEV: Remove `before(:all)` from RSpec suite 2022-07-18 14:21:56 +02:00
Sam 8dad778fcc
FIX: support systems without SSE 4.2 (#17505)
Older CPUs do not support the instruction set used internally by the oj gem
starting with version 3.13.15.

This pins the version to a working one while oj patch the project to support
CPUs without the instructions (eg: AMD Opteron 4180).

see:

https://meta.discourse.org/t/discourse-update-keeps-failing/231862
https://github.com/ohler55/oj/issues/789
a455c389c0
2022-07-15 10:06:04 +10:00
David Taylor ee07f6da7d
SECURITY: Bump Rails to 7.0.3.1 (#17469)
https://discuss.rubyonrails.org/t/81017
2022-07-13 11:17:46 +01:00
Loïc Guitaut fff3ff11c0 FIX: Make disabling TLS in mail possible again
Following the Rails 7 upgrade, the `DISCOURSE_SMTP_ENABLE_START_TLS`
setting doesn’t work anymore. This is because Rails upgraded the
`net-smtp` gem to the 0.3.1 version which enables `starttls` by default.
The `mail` gem doesn’t support this new behavior yet and doesn’t know
how to disable TLS. This should be fixed in an upcoming release.

Meanwhile applying this patch allows us to get back the previous
behavior which is expected by many.
2022-06-08 14:04:05 +02:00
David Taylor 8bc8dbc134
Revert "DEV: Drop our `mail` gem fork (#16622)" (#16853)
This reverts commit 0d30c19b7e.

Pending resolution of https://github.com/mikel/mail/issues/1489
2022-05-17 16:11:39 +01:00
Loïc Guitaut 5fb6dd5664 DEV: Upgrade to Rails 7.0.3 2022-05-10 12:01:00 +02:00
Sam 2df3c65ba9
FIX: add support for pipelined and multi redis commands (#16682)
Latest redis interoduces a block form of multi / pipelined, this was incorrectly
passed through and not namespaced.

Fix also updates logster, we held off on upgrading it due to missing functions
2022-05-10 08:19:02 +10:00
Jarek Radosz 0d30c19b7e
DEV: Drop our `mail` gem fork (#16622)
Bianca's patch was finally released (in rc release for now, but still!)
2022-05-04 16:53:53 +02:00
Jarek Radosz 79c854d9be
DEV: Remove RTLit gem (#16620)
Its only use was removed 7 years ago in #3377.
2022-05-04 14:11:12 +08:00
Jarek Radosz d8ce4228da
DEV: Lock logster to 2.11.0 (#16605)
2.11.1+ is not compatible with DiscourseRedis implementation. This fixes error reporting.
2022-05-03 13:41:09 +02:00
Loïc Guitaut 008b700a3f DEV: Upgrade to Rails 7
This patch upgrades Rails to version 7.0.2.4.
2022-04-28 11:51:03 +02:00
David Taylor c88ca23e8f
Revert "DEV: Update to Sprockets 4.0 (#16467)" (#16524)
This reverts commit 01107e418e.

We have seen some random occurrences of corrupted assets, and think it may be related to the sprockets 4 update. Reverting for investigation
2022-04-20 22:17:29 +01:00
David Taylor 01107e418e
DEV: Update to Sprockets 4.0 (#16467)
The main difference is that Sprockets 4.0 no longer tries to compile everything by default. This is good for us, because we can remove all our custom 'exclusion' logic which was working around the old sprockets 3.0 behavior.

The other big change is that lambdas can no longer be added to the `config.assets.precompile` array. Instead, we can do the necessary globs ourselves, and add the desired files manually.

A small patch is required to make ember-rails compatible. Since we plan to remove this dependency in the near future, I do not intend to upstream this change.

I have compared the `bin/rake assets:precompile` output before and after this change, and verified that all files are present.
2022-04-13 15:03:50 +01:00
David Taylor 78f7e8fe2f
Revert "DEV: Update to Sprockets 4.0 (#16459)" (#16466)
This reverts commit ec7efbde1a.

This is causing problems in non-ember-cli environments. Reverting for now.
2022-04-13 11:04:13 +01:00
David Taylor ec7efbde1a
DEV: Update to Sprockets 4.0 (#16459)
The main difference is that Sprockets 4.0 no longer tries to compile everything by default. This is good for us, because we can remove all our custom 'exclusion' logic which was working around the old sprockets 3.0 behavior.

The other big change is that lambdas can no longer be added to the `config.assets.precompile` array. Instead, we can do the necessary globs ourselves, and add the desired files manually.

A small patch is required to make ember-rails compatible. Since we plan to remove this dependency in the near future, I do not intend to upstream this change.

I have compared the `bin/rake assets:precompile` output before and after this change, and verified that all files are present.
2022-04-13 10:21:59 +01:00
Alan Guo Xiang Tan 01c5f9337a
DEV: Upgrade Rails to 6.1.4.7. (#16143)
There was a security release in 6.1.4.6 to fix: https://github.com/rails/rails/security/advisories/GHSA-wh98-p28r-vrc9

It doesn't affect us as Discourse doesn't use thread local variables
but we should still upgrade as a matter of caution.
2022-03-09 10:10:51 +08:00
Jarek Radosz aaf432df86
DEV: Remove mock_redis (#15985)
Was used just in one spec file. And we prefer to run specs against a real redis server.
2022-02-18 01:14:38 +01:00