Commit Graph

134 Commits

Author SHA1 Message Date
Joffrey JAFFEUX e495a2fc3f
DEV: Enable parallel system specs in GitHub actions CI (#21251)
Also skips/improves few flakey specs
2023-04-26 13:02:19 +02:00
Jarek Radosz 00630e4c74
DEV: Remove `RUBY_GLOBAL_METHOD_CACHE_SIZE` (#21249)
It doesn't do anything since ruby 3.0.0.preview1. It was removed in https://github.com/ruby/ruby/pull/2888
2023-04-26 10:39:39 +02:00
Joffrey JAFFEUX d4c6457065
DEV: increase plugin system tests timeout (#21247) 2023-04-26 10:08:10 +02:00
Jarek Radosz 43e0025141
Revert "DEV: Merge package.json files (#21172)" (#21182)
This reverts commit 49a1e1cd0e.

Is causing issues in prod-adjacent environments (Jenkins)
2023-04-20 14:57:40 +02:00
Jarek Radosz 49a1e1cd0e
DEV: Merge package.json files (#21172)
This means: a single yarn.lock and removing one of the package.json files
2023-04-20 12:46:12 +02:00
Alan Guo Xiang Tan 3cb9fd739a
DEV: Run system tests with documentation format on github actions (#21069)
Allows us to see the tests which have timed out
2023-04-12 14:47:05 +08:00
NullVoxPopuli bdaaac9c9f
DEV: Setup lint to the future (#20990)
## How does this work?

Any time a lint rule is added or changed, you can run `yarn lint:fix` to handle all the auto-fixable situations.
But not all lints are auto-fixable -- for those, lint-to-the-future has tooling to automatically ignore present violations.
An alias has been added for lint-to-the-future to ignore new violations, `yarn lttf:ignore`.
The command will add lint-ignore declarations throughout all the files with present violations, which should then be committed.

An excerpt from lint-to-the-future's [README](https://github.com/mansona/lint-to-the-future#lint-to-the-future-dashboard):

> The point of Lint to the Future is to allow you to progressively update your codebase using new lint rules without overwhelming you with the task. You can easily ignore lint rules using project-based ignores in your config files but that doesn't prevent you from making the same errors in new files.

> We chose to do the ignores on a file basis as it is a perfect balance and it means that the tracking/graphing aspects of Lint to the Future provide you with achievable goals, especially in large codebases.

## How do I view progress?

lint-to-the-future provides graphs of violations-over-time per lint rule in a dashboard format, so we can track how well we're doing at cleaning up the violations.

To view the dashboard locally, run `yarn lint-progress` and visit `http://localhost:8084` (or whatever the port it chose, as it will choose a new port if 8084 is preoccupied)

Also there is a `list` command which shows a JSON object of:
```ts
{
  [date: string]: { // yyyy-mm-dd
    [pluginName: string]: {
      [fileName: string]: string[]; // list of files with violations
    }
  }
}
```


```bash
yarn lint-to-the-future list --stdout
```

## What about lint-todo?

Lint todo is another system available for both eslint and ember-template-lint that _forces_ folks to "leave things better than they found them" by being transparent / line-specific ignoring of violations. 
It was decided that for _this_ project, it made more sense, and would be less disruptive to new contributors to have the ignore declarations explicitly defined in each file (whereas in lint-todo, they are hidden).
To effectively use lint-todo, a whole team needs to agree to the workflow, and in open source, we want "just anyway" to be able to contribute, and throwing surprises at them can deter contributions.
2023-04-06 17:25:01 +01:00
Daniel Waterworth 52999d1422
DEV: Show which system tests are being executed in CI (#20795)
... to see if it's always the same test causing timeouts
2023-03-23 15:12:14 -05:00
David Taylor 150a6601c0
DEV: Check Zeitwerk eager loading in GitHub CI (#20699)
In production, `eager_load=true`. This sometimes leads to boot errors which are not present in dev/test environments. Running `zeitwerk:check` in CI will help us to pick up on any errors early.

This commit also introduces a `DISCOURSE_ZEITWERK_EAGER_LOAD` environment variable to make it easier to toggle the behaviour when developing locally.
2023-03-16 14:22:16 +00:00
Daniel Waterworth 5324216740 DEV: Use rspec instead of turbo_rspec with one core 2023-03-15 16:46:48 -05:00
Rafael dos Santos Silva c3ae555ac2
DEV: Show correct version of Ruby we are using on CI (#20415) 2023-02-22 15:39:49 -03:00
Joffrey JAFFEUX a5e45d90fe
DEV: removes github page documentation page generation (#20268)
It was causing errors on C, will need to investigate a better solution in the future.
2023-02-13 19:39:05 +01:00
Martin Brennan 60ad836313
DEV: Chat service object initial implementation (#19814)
This is a combined work of Martin Brennan, Loïc Guitaut, and Joffrey Jaffeux.

---

This commit implements a base service object when working in chat. The documentation is available at https://discourse.github.io/discourse/chat/backend/Chat/Service.html

Generating documentation has been made as part of this commit with a bigger goal in mind of generally making it easier to dive into the chat project.

Working with services generally involves 3 parts:

- The service object itself, which is a series of steps where few of them are specialized (model, transaction, policy)

```ruby
class UpdateAge
  include Chat::Service::Base

  model :user, :fetch_user
  policy :can_see_user
  contract
  step :update_age

  class Contract
    attribute :age, :integer
  end

  def fetch_user(user_id:, **)
    User.find_by(id: user_id)
  end

  def can_see_user(guardian:, **)
    guardian.can_see_user(user)
  end

  def update_age(age:, **)
    user.update!(age: age)
  end
end
```

- The `with_service` controller helper, handling success and failure of the service within a service and making easy to return proper response to it from the controller

```ruby
def update
  with_service(UpdateAge) do
    on_success { render_serialized(result.user, BasicUserSerializer, root: "user") }
  end
end
```

- Rspec matchers and steps inspector, improving the dev experience while creating specs for a service

```ruby
RSpec.describe(UpdateAge) do
  subject(:result) do
    described_class.call(guardian: guardian, user_id: user.id, age: age)
  end

  fab!(:user) { Fabricate(:user) }
  fab!(:current_user) { Fabricate(:admin) }

  let(:guardian) { Guardian.new(current_user) }
  let(:age) { 1 }

   it { expect(user.reload.age).to eq(age) }
end
```

Note in case of unexpected failure in your spec, the output will give all the relevant information:

```
  1) UpdateAge when no channel_id is given is expected to fail to find a model named 'user'
     Failure/Error: it { is_expected.to fail_to_find_a_model(:user) }

       Expected model 'foo' (key: 'result.model.user') was not found in the result object.

       [1/4] [model] 'user' 
       [2/4] [policy] 'can_see_user'
       [3/4] [contract] 'default'
       [4/4] [step] 'update_age'

       /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/update_age.rb:32:in `fetch_user': missing keyword: :user_id (ArgumentError)
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `instance_exec'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:219:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `block in run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `each'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:411:in `run'
       	from <internal:kernel>:90:in `tap'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:302:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/spec/services/update_age_spec.rb:15:in `block (3 levels) in <main>'
```
2023-02-13 13:09:57 +01:00
David Taylor 488b8b369a DEV: Fix syntax_tree in GitHub CI
This broke because of directory ownership errors during `git ls-files`. This commit fixes the permissions and adds bash flags so that those kind of errors will blow up the step in future.
2023-02-02 13:03:11 +00:00
David Taylor 6b01105cb9 DEV: Add hbs prettier checks to GitHub CI
We have already formatted all hbs files, but we didn't update the linting CI check to include them.
2023-02-02 13:03:11 +00:00
David Taylor a6b680f4fe
DEV: Fix GitHub CI permissions issues (#20069)
The `git` version in our discourse_test docker image was recently updated to include a permissions check before running any git commands. For this to pass, the owner of the discourse directory needs to match the user running any git commands.

Under GitHub actions, by default the working directory is created with uid=1000 as the owner. We run all our tests as `root`, so this mismatch causes git to raise the permissions error. We can't switch to run the entire workflow as the `discourse (uid=1000)` user because our discourse_test image is not configured to allow `discourse` access to postgres/redis directories. For now, this commit updates the working directory's owner to match the user running the workflow.
2023-01-30 15:39:43 +00:00
David Taylor b96869d5fb
DEV: Disable parallel system specs in GitHub actions (#20023)
We have some flakiness which needs to be resolved. Followup to e717529d80
2023-01-26 14:37:47 +00:00
David Taylor e717529d80
DEV: Enable parallel system specs in GitHub actions CI (#19584) 2023-01-26 13:26:02 +00:00
Rafael dos Santos Silva 60ebbfd7e7
DEV: Stop testing with Ruby 3.2 for now (#19909) 2023-01-18 12:04:49 -03:00
Joffrey JAFFEUX f29b956339
DEV: introduces documentation for chat (#19772)
Note this commit also slightly changes internal API: channel instead of getChannel and updateCurrentUserChannelNotificationsSettings instead of updateCurrentUserChatChannelNotificationsSettings.

Also destroyChannel takes a second param which is the name confirmation instead of an optional object containing this confirmation. This is to enforce the fact that it's required.

In the future a top level jsdoc config file could be used instead of the hack tempfile, but while it's only an experiment for chat, it's probably good enough.
2023-01-18 12:36:16 +01:00
Rafael dos Santos Silva 076b3a6514
DEV: Key bundler CI cache on Ruby version (#19868) 2023-01-13 11:39:49 -03:00
Rafael dos Santos Silva 8e7e6e14c7
DEV: Add Ruby 3.2 to test matrix (#19862)
* DEV: Add Ruby 3.2 to test matrix

* DEV: Update test name
2023-01-13 09:22:33 -03:00
David Taylor 93e2dad656
DEV: Introduce syntax_tree code formatter (#19775)
This commit introduces the necessary gems and config, but adds all our ruby code directories to the `--ignore-files` list.

Future commits will apply syntax_tree to parts of the codebase, removing the ignore patterns as we go
2023-01-07 11:11:08 +00:00
Alan Guo Xiang Tan e58277adf3
DEV: Increase Capybara.default_max_wait_time on github actions (#19750)
Our working theory is that system tests on Github run on much less
powerful hardware as compared to running the tests on our work machines.
Hopefully, increasing the wait time now will help reduce some flakes
that we're seeing on Github.
2023-01-05 08:50:35 +08:00
David Taylor 335893ae91
DEV: Correct private-fork -> private-mirror (#19560)
Followup to 7eb9482ba9
2022-12-21 16:52:35 +00:00
David Taylor 7eb9482ba9
DEV: Skip 'push' workflow events for discourse-private-mirror (#19556)
We don't want 'push' workflows to run on this private fork (which is used for developing security-fixes before public disclosure)
2022-12-21 16:44:38 +00:00
Martin Brennan 8b3c6cd396
DEV: Fix github workflow system spec screenshot location (#19435)
These screenshots are located at paths like:

/__w/discourse/discourse/tmp/capybara/failures_r_spec_example_groups_quoting_chat_message_transcripts_copying_quote_transcripts_with_the_clipboard_quotes_multiple_chat_messages_into_a_topic_134.png

not /tmp/screenshots. This should fix the issue. Also makes plugin system specs
use documentation format and profile.
2022-12-13 15:36:30 +10:00
Alan Guo Xiang Tan 5103268312
DEV: Run system tests with documentation and profiling on actions (#19271)
When a test takes too long, we want to know which test and at what step
2022-12-01 05:54:17 +08:00
Jarek Radosz cccb6e9b0b
DEV: Bump Dependabot's bundler PR limit (#19081) 2022-11-17 21:48:13 +01:00
David Taylor f8939bd294
DEV: Bump `@actions/checkout` to v3 in frontend tests (#18989)
v2 uses Node 12, which is deprecated
2022-11-11 13:31:28 +00:00
David Taylor 70a990da03
DEV: Update GitHub actions set-output uses (#18988)
https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
2022-11-11 13:12:08 +00:00
Joffrey JAFFEUX 11f3618b80
DEV: initial system tests for chat and plugins (#18881)
This is a very basic to ensure it's working and open future possible work
2022-11-04 15:06:24 +01:00
David Taylor 6c25b28312
DEV: Fix `labeler.yml` glob configuration (#18846)
Followup to 449f7d5ed5
2022-11-02 15:59:59 +00:00
David Taylor 449f7d5ed5
DEV: Automatically label chat PRs (#18843) 2022-11-02 15:43:59 +00:00
Jarek Radosz 43c2841a69
DEV: Switch back to mainline `licensed` gem (#18641)
The bundler issue has been fixed
2022-10-18 14:34:44 +02:00
Martin Brennan 57caf08e13
DEV: Minimal first pass of rails system test setup (#16311)
This commit introduces rails system tests run with chromedriver, selenium,
and headless chrome to our testing toolbox.

We use the `webdrivers` gem and `selenium-webdriver` which is what
the latest Rails uses so the tests run locally and in CI out of the box.

You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra
verbose logs of what selenium is doing to communicate with the system
tests.

By default JS logs are verbose so errors from JS are shown when
running system tests, you can disable this with
`SELENIUM_DISABLE_VERBOSE_JS_LOGS=1`

You can use `SELENIUM_HEADLESS=0` to run the system
tests inside a chrome browser instead of headless, which can be useful to debug things
and see what the spec sees. See note above about `bin/ember-cli` to avoid
surprises.

I have modified `bin/turbo_rspec` to exclude `spec/system` by default,
support for parallel system specs is a little shaky right now and we don't
want them slowing down the turbo by default either.

### PageObjects and System Tests

To make querying and inspecting parts of the page easier
and more reusable inbetween system tests, we are using the
concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in
our system tests. A "Page" here is generally corresponds to
an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`,
and this contains logic for querying components within the topic
such as "Posts".

I have also split "Modals" into their own entity. Further down the
line we may want to explore creating independent "Component"
contexts.

Capybara DSL should be included in each PageObject class,
reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl

For system tests, since they are so slow, we want to focus on
the "happy path" and not do every different possible context
and branch check using them. They are meant to be overarching
tests that check a number of things are correct using the full stack
from JS and ember to rails to ruby and then the database.

### CI Setup

Whenever a system spec fails, a screenshot
is taken and a build artifact is produced _after the entire CI run is complete_,
which can be downloaded from the Actions UI in the repo.

Most importantly, a step to build the Ember app using Ember CLI
is needed, otherwise the JS assets cannot be found by capybara:

```
- name: Build Ember CLI
  run: bin/ember-cli --build
```

A new `--build` argument has been added to `bin/ember-cli` for this
case, which is not needed locally if you already have the discourse
rails server running via `bin/ember-cli -u` since the whole server is built and
set up by default.

Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 11:48:16 +10:00
David Taylor 885e133cac
DEV: Cache `turbo_rspec_runtime.log` in CI (#18318) 2022-09-21 22:13:25 +01:00
David Taylor a11aea4fe2
DEV: Update github workflow for 8-core workers (#18271) 2022-09-21 18:13:13 +01:00
David Taylor b0a9d8b761
DEV: Improve labelling of Firefox Evergreen/ESR CI runs (#18313)
Both versions are used with `--headless`, so labelling one "Firefox" and the other "Firefox Headless" doesn't really make sense. Evergreen / ESR are better descriptions.
2022-09-21 15:34:26 +01:00
David Taylor 42d226f727
DEV: Ensure GitHub workflows cancel cleanly (#18314)
We added `always()` on some steps so that they run even if previous steps fail. That helps give us a picture of all failures in one run, rather than having to re-run the workflow after fixing the first failure.

However, when we explicitly cancel a job, we should skip running these steps. `!cancelled()` is a better substitute for `always()` in this case.
2022-09-21 14:32:21 +01:00
David Taylor e06b9d4a52
DEV: Remove support for legacy plugin JS compilation pipeline (#18293)
This became the default in b1755137
2022-09-21 12:38:02 +01:00
David Taylor ef39193a06
DEV: Add `rake plugins:turbo_spec` task (#18289)
This leans on our existing `turbo_rspec` implementation to run plugin specs in parallel on all available cores
2022-09-20 15:42:54 +01:00
Jarek Radosz 494a6317d7
DEV: Update dependabot config (#18218) 2022-09-12 09:27:10 +02:00
Jarek Radosz 07aa324f61
DEV: Try to kickstart npm updates (#18152)
Dependabot hasn't picked up the previous config change. Let's see if it picks up this one.
2022-09-01 11:46:02 +02:00
Jarek Radosz 2595e368b1
DEV: Enable dependabot for frontend deps (#18138) 2022-08-31 01:08:23 +02:00
Jarek Radosz 0d4b1f25f0
DEV: Reformat dependabot config whitespace (#18126)
In preparation for npm addition
2022-08-30 10:02:33 +08:00
David Taylor 33a2624f09
DEV: Introduce flag for compiling Plugin JS with Ember CLI (#17965)
When `EMBER_CLI_PLUGIN_ASSETS=1`, plugin application JS will be compiled via Ember CLI. In this mode, the existing `register_asset` API will cause any registered JS files to be made available in `/plugins/{plugin-name}_extra.js`. These 'extra' files will be loaded immediately after the plugin app JS file, so this should not affect functionality.

Plugin compilation in Ember CLI is implemented as an addon, similar to the existing 'admin' addon. We bypass the normal Ember CLI compilation process (which would add the JS to the main app bundle), and reroute the addon Broccoli tree into a separate JS file per-plugin. Previously, Sprockets would add compiled templates directly to `Ember.TEMPLATES`. Under Ember CLI, they are compiled into es6 modules. Some new logic in `discourse-boot.js` takes care of remapping the new module names into the old-style `Ember.TEMPLATES`.

This change has been designed to be a like-for-like replacement of the old plugin compilation system, so we do not expect any breakage. Even so, the environment variable flag will allow us to test this in a range of environments before enabling it by default.

A manual silence implementation is added for the build-time `ember-glimmer.link-to.positional-arguments` deprecation while we work on a better story for plugins.
2022-08-22 09:56:39 +01:00
Jarek Radosz 5d1cf006ab
DEV: Tweak core_frontend_tests timeouts (#17902)
Each test chunk takes about 10 minutes, so those timeouts can be decreased from 20 to 15.

And there are three of those chunks so total can be a bit over 30 minutes, hence the bump to 35.
2022-08-14 17:30:15 +02:00
Jarek Radosz 89d43235c4
DEV: Set permissions for GitHub actions (#17733)
Included permissions for the action.

https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions
https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

Co-authored-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-07-30 17:22:03 +02:00
Gerhard Schlager 010bb20f53
DEV: Workaround for licensed gem incompatibility with latest Bundler (#17704) 2022-07-28 12:26:24 +02:00