Commit Graph

160 Commits

Author SHA1 Message Date
Jarek Radosz 7b78895930
DEV: Group babel updates together (#23322)
Grouped Dependabot updates are now available (https://github.blog/changelog/2023-08-24-grouped-version-updates-for-dependabot-are-generally-available/) so we can finally make it create a single PR for both `@babel/core` and `@babel/standalone` 😌
2023-08-30 10:38:26 +02:00
Martin Brennan cf42466dea
DEV: Add S3 upload system specs using minio (#22975)
This commit adds some system specs to test uploads with
direct to S3 single and multipart uploads via uppy. This
is done with minio as a local S3 replacement. We are doing
this to catch regressions when uppy dependencies need to
be upgraded or we change uppy upload code, since before
this there was no way to know outside manual testing whether
these changes would cause regressions.

Minio's server lifecycle and the installed binaries are managed
by the https://github.com/discourse/minio_runner gem, though the
binaries are already installed on the discourse_test image we run
GitHub CI from.

These tests will only run in CI unless you specifically use the
CI=1 or RUN_S3_SYSTEM_SPECS=1 env vars.

For a history of experimentation here see https://github.com/discourse/discourse/pull/22381

Related PRs:

* https://github.com/discourse/minio_runner/pull/1
* https://github.com/discourse/minio_runner/pull/2
* https://github.com/discourse/minio_runner/pull/3
2023-08-23 11:18:33 +10:00
Mark VanLandingham fbf7b106cc
DEV: Bump selenium-webdriver version to fix system spec running (#23117)
We can no long user Webdriver - SeleniumHQ/selenium#11066. Bumping selenium-webdriver did the trick, as well as manually setting the user_agent for mobile system specs. Unsure what changed to make this necessary, but it is necessary to get the app to boot in mobile view.
2023-08-16 15:07:03 -05:00
Jarek Radosz 8bebd8fd99
DEV: Format .thor files (#23059) 2023-08-10 13:59:13 +02:00
Krzysztof Kotlarek 77b4e42f61
FIX: specify chrome version (#22681) 2023-07-19 15:06:56 +10:00
Alan Guo Xiang Tan 433cb7092d
DEV: Use documentation format for core specs on CI (#22531)
The documentation format makes it easier to link a failing test to the
process it was launched in.
2023-07-11 10:12:51 +08:00
Alan Guo Xiang Tan 7b40ec859f
DEV: Reduce parallel test processors for system tests to 4 (#22456)
Why this change?

This is abit of a trial and error but we're starting to see selenium
session not created errors on CI. One of the reason for this is that the
system has run out of resources to create a new tab.

This commit reduces the number of parallel test processors in an attempt
to increase the amount of resources available to each test process and
hopefully lead to more stable CI system tests.
2023-07-06 10:34:44 +08:00
Alan Guo Xiang Tan b764c53cc0
Revert "DEV: Reduce number of parallel test processors to 3 for system tests (#22423)" (#22453)
This reverts commit 865f7a9852.

The flakiness that we have been seeing and fixing on CI were not related
to system resource problems. Therefore, we can bump this up back to 5.
2023-07-06 08:47:43 +08:00
Alan Guo Xiang Tan 858cc6aff2
DEV: Log server errors encountered when running system tests on CI (#22427)
Why is this change required?

We've been seeing flaky tests due to server errors on CI but are unable
to debug it because we do not log any of the errors. This change gives
us a fighting chance the next time we encounter a server error during
system test runs.

See
https://github.com/discourse/discourse/actions/runs/5459248864/jobs/9935049920?pr=22424
for an example of server errors encountered during system tests.
2023-07-05 09:46:48 +08:00
Alan Guo Xiang Tan 865f7a9852
DEV: Reduce number of parallel test processors to 3 for system tests (#22423)
This is an experiment to see if not saturating resources on a runner
with 8 vCPU leads to more stability on our CI.
2023-07-05 08:08:46 +08:00
Jarek Radosz 876ff17cc2
DEV: Update eslint/prettier (#22226) 2023-06-21 20:59:03 +02:00
David Taylor 251d6f0627
DEV: Introduce Zeitwerk reloading check in CI (#22151)
This should help us to catch the most catastrophic cases where plugins fail to auto-reload in development mode.
2023-06-16 14:33:14 +01:00
Alan Guo Xiang Tan 5897709a90
DEV: Use runtime info to split test files for parallel testing (#22060)
Using the runtime information, we will be able to more efficiently group
the test files across the test processes hence leading to better
utilization of resources.
2023-06-12 09:07:17 +08:00
Daniel Waterworth 67afd85aae
Revert "DEV: Use runtime info to split test files for parallel testing (#21896)" (#22016)
This reverts commit 14ed971db6.

This prevented the core backend tests from running in GitHub CI
2023-06-08 15:13:26 -05:00
Alan Guo Xiang Tan 14ed971db6
DEV: Use runtime info to split test files for parallel testing (#21896)
Using the runtime information, we will be able to more efficiently group
the test files across the test processes hence leading to better
utilization of resources.
2023-06-05 08:01:41 +08:00
Alan Guo Xiang Tan ba251dec6b
DEV: Increase number of parallel process for system test to 5 on CI (#21878)
4dd053a69c addressed most of the
instability we were seeing with system tests on CI and locally. Let's
try pushing the number of parallel processes up to squeeze as much time
savings as possible from the runner.
2023-06-01 10:41:11 +08:00
Alan Guo Xiang Tan 2c1c7d1e4f DEV: Use `--profile` when running system tests in CI 2023-05-30 13:46:14 +09:00
Alan Guo Xiang Tan 123a77a2bc
DEV: Correct Capybara default max wait time setting in CI (#21801)
We were not setting capybara's default max wait time correctly in CI due
to a spelling error.

This regressd in fc17045876
2023-05-29 10:41:24 +08:00
Alan Guo Xiang Tan 038318bffc
DEV: Bump CAPYBARA_DEFAULT_MAX_WAIT_TIME to 10 seconds on CI (#21711)
We're running on pretty crappy hardware on Github's CI and this has an
impact on the stability of our system tests on CI. Therefore, we are
bumping `CABPYARA_DEFAULT_MAX_WAIT_TIME` to 10 seconds to account for
the less than ideal hardware we're running the system tests on.

This change trades off speed for stability but speed is already bad on
CI so stability is more important for our case.
2023-05-25 09:25:08 +08:00
Jarek Radosz fc17045876
DEV: Clean up workflow files (#21526) 2023-05-12 14:00:04 +02:00
Jarek Radosz 19ac90536f
DEV: Restore the documentation format in system tests (#21471) 2023-05-12 11:13:52 +02:00
David Taylor d6f94e0916
DEV: Revert buildjet (#21475)
This reverts commits 17603794b5 and 41bdf8cfcd.
2023-05-11 22:25:30 +02:00
Natalie Tay 17603794b5
DEV: Reduce plugin system test parallel processors (#21466) 2023-05-10 15:43:41 +08:00
Natalie Tay 41bdf8cfcd
DEV: Use BuildJet for some test jobs (#21404) 2023-05-10 10:39:27 +08:00
Alan Guo Xiang Tan 1f6d57ab93
DEV: Run turbo rspecs with verbose output on CI (#21379)
In CI, we the output to be as verbose as possible so that when it fails we have the necessary information to debug the failures.
2023-05-04 10:34:02 +08:00
David Taylor c6f5b4297d
DEV: Use `--frozen-lockfile` in GitHub CI (#21338)
This will help us to catch missing lockfile changes before they are merged into `main`
2023-05-02 16:25:22 +01:00
Joffrey JAFFEUX e495a2fc3f
DEV: Enable parallel system specs in GitHub actions CI (#21251)
Also skips/improves few flakey specs
2023-04-26 13:02:19 +02:00
Jarek Radosz 00630e4c74
DEV: Remove `RUBY_GLOBAL_METHOD_CACHE_SIZE` (#21249)
It doesn't do anything since ruby 3.0.0.preview1. It was removed in https://github.com/ruby/ruby/pull/2888
2023-04-26 10:39:39 +02:00
Joffrey JAFFEUX d4c6457065
DEV: increase plugin system tests timeout (#21247) 2023-04-26 10:08:10 +02:00
Jarek Radosz 43e0025141
Revert "DEV: Merge package.json files (#21172)" (#21182)
This reverts commit 49a1e1cd0e.

Is causing issues in prod-adjacent environments (Jenkins)
2023-04-20 14:57:40 +02:00
Jarek Radosz 49a1e1cd0e
DEV: Merge package.json files (#21172)
This means: a single yarn.lock and removing one of the package.json files
2023-04-20 12:46:12 +02:00
Alan Guo Xiang Tan 3cb9fd739a
DEV: Run system tests with documentation format on github actions (#21069)
Allows us to see the tests which have timed out
2023-04-12 14:47:05 +08:00
NullVoxPopuli bdaaac9c9f
DEV: Setup lint to the future (#20990)
## How does this work?

Any time a lint rule is added or changed, you can run `yarn lint:fix` to handle all the auto-fixable situations.
But not all lints are auto-fixable -- for those, lint-to-the-future has tooling to automatically ignore present violations.
An alias has been added for lint-to-the-future to ignore new violations, `yarn lttf:ignore`.
The command will add lint-ignore declarations throughout all the files with present violations, which should then be committed.

An excerpt from lint-to-the-future's [README](https://github.com/mansona/lint-to-the-future#lint-to-the-future-dashboard):

> The point of Lint to the Future is to allow you to progressively update your codebase using new lint rules without overwhelming you with the task. You can easily ignore lint rules using project-based ignores in your config files but that doesn't prevent you from making the same errors in new files.

> We chose to do the ignores on a file basis as it is a perfect balance and it means that the tracking/graphing aspects of Lint to the Future provide you with achievable goals, especially in large codebases.

## How do I view progress?

lint-to-the-future provides graphs of violations-over-time per lint rule in a dashboard format, so we can track how well we're doing at cleaning up the violations.

To view the dashboard locally, run `yarn lint-progress` and visit `http://localhost:8084` (or whatever the port it chose, as it will choose a new port if 8084 is preoccupied)

Also there is a `list` command which shows a JSON object of:
```ts
{
  [date: string]: { // yyyy-mm-dd
    [pluginName: string]: {
      [fileName: string]: string[]; // list of files with violations
    }
  }
}
```


```bash
yarn lint-to-the-future list --stdout
```

## What about lint-todo?

Lint todo is another system available for both eslint and ember-template-lint that _forces_ folks to "leave things better than they found them" by being transparent / line-specific ignoring of violations. 
It was decided that for _this_ project, it made more sense, and would be less disruptive to new contributors to have the ignore declarations explicitly defined in each file (whereas in lint-todo, they are hidden).
To effectively use lint-todo, a whole team needs to agree to the workflow, and in open source, we want "just anyway" to be able to contribute, and throwing surprises at them can deter contributions.
2023-04-06 17:25:01 +01:00
Daniel Waterworth 52999d1422
DEV: Show which system tests are being executed in CI (#20795)
... to see if it's always the same test causing timeouts
2023-03-23 15:12:14 -05:00
David Taylor 150a6601c0
DEV: Check Zeitwerk eager loading in GitHub CI (#20699)
In production, `eager_load=true`. This sometimes leads to boot errors which are not present in dev/test environments. Running `zeitwerk:check` in CI will help us to pick up on any errors early.

This commit also introduces a `DISCOURSE_ZEITWERK_EAGER_LOAD` environment variable to make it easier to toggle the behaviour when developing locally.
2023-03-16 14:22:16 +00:00
Daniel Waterworth 5324216740 DEV: Use rspec instead of turbo_rspec with one core 2023-03-15 16:46:48 -05:00
Rafael dos Santos Silva c3ae555ac2
DEV: Show correct version of Ruby we are using on CI (#20415) 2023-02-22 15:39:49 -03:00
Joffrey JAFFEUX a5e45d90fe
DEV: removes github page documentation page generation (#20268)
It was causing errors on C, will need to investigate a better solution in the future.
2023-02-13 19:39:05 +01:00
Martin Brennan 60ad836313
DEV: Chat service object initial implementation (#19814)
This is a combined work of Martin Brennan, Loïc Guitaut, and Joffrey Jaffeux.

---

This commit implements a base service object when working in chat. The documentation is available at https://discourse.github.io/discourse/chat/backend/Chat/Service.html

Generating documentation has been made as part of this commit with a bigger goal in mind of generally making it easier to dive into the chat project.

Working with services generally involves 3 parts:

- The service object itself, which is a series of steps where few of them are specialized (model, transaction, policy)

```ruby
class UpdateAge
  include Chat::Service::Base

  model :user, :fetch_user
  policy :can_see_user
  contract
  step :update_age

  class Contract
    attribute :age, :integer
  end

  def fetch_user(user_id:, **)
    User.find_by(id: user_id)
  end

  def can_see_user(guardian:, **)
    guardian.can_see_user(user)
  end

  def update_age(age:, **)
    user.update!(age: age)
  end
end
```

- The `with_service` controller helper, handling success and failure of the service within a service and making easy to return proper response to it from the controller

```ruby
def update
  with_service(UpdateAge) do
    on_success { render_serialized(result.user, BasicUserSerializer, root: "user") }
  end
end
```

- Rspec matchers and steps inspector, improving the dev experience while creating specs for a service

```ruby
RSpec.describe(UpdateAge) do
  subject(:result) do
    described_class.call(guardian: guardian, user_id: user.id, age: age)
  end

  fab!(:user) { Fabricate(:user) }
  fab!(:current_user) { Fabricate(:admin) }

  let(:guardian) { Guardian.new(current_user) }
  let(:age) { 1 }

   it { expect(user.reload.age).to eq(age) }
end
```

Note in case of unexpected failure in your spec, the output will give all the relevant information:

```
  1) UpdateAge when no channel_id is given is expected to fail to find a model named 'user'
     Failure/Error: it { is_expected.to fail_to_find_a_model(:user) }

       Expected model 'foo' (key: 'result.model.user') was not found in the result object.

       [1/4] [model] 'user' 
       [2/4] [policy] 'can_see_user'
       [3/4] [contract] 'default'
       [4/4] [step] 'update_age'

       /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/update_age.rb:32:in `fetch_user': missing keyword: :user_id (ArgumentError)
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `instance_exec'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:219:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `block in run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `each'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:411:in `run'
       	from <internal:kernel>:90:in `tap'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:302:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/spec/services/update_age_spec.rb:15:in `block (3 levels) in <main>'
```
2023-02-13 13:09:57 +01:00
David Taylor 488b8b369a DEV: Fix syntax_tree in GitHub CI
This broke because of directory ownership errors during `git ls-files`. This commit fixes the permissions and adds bash flags so that those kind of errors will blow up the step in future.
2023-02-02 13:03:11 +00:00
David Taylor 6b01105cb9 DEV: Add hbs prettier checks to GitHub CI
We have already formatted all hbs files, but we didn't update the linting CI check to include them.
2023-02-02 13:03:11 +00:00
David Taylor a6b680f4fe
DEV: Fix GitHub CI permissions issues (#20069)
The `git` version in our discourse_test docker image was recently updated to include a permissions check before running any git commands. For this to pass, the owner of the discourse directory needs to match the user running any git commands.

Under GitHub actions, by default the working directory is created with uid=1000 as the owner. We run all our tests as `root`, so this mismatch causes git to raise the permissions error. We can't switch to run the entire workflow as the `discourse (uid=1000)` user because our discourse_test image is not configured to allow `discourse` access to postgres/redis directories. For now, this commit updates the working directory's owner to match the user running the workflow.
2023-01-30 15:39:43 +00:00
David Taylor b96869d5fb
DEV: Disable parallel system specs in GitHub actions (#20023)
We have some flakiness which needs to be resolved. Followup to e717529d80
2023-01-26 14:37:47 +00:00
David Taylor e717529d80
DEV: Enable parallel system specs in GitHub actions CI (#19584) 2023-01-26 13:26:02 +00:00
Rafael dos Santos Silva 60ebbfd7e7
DEV: Stop testing with Ruby 3.2 for now (#19909) 2023-01-18 12:04:49 -03:00
Joffrey JAFFEUX f29b956339
DEV: introduces documentation for chat (#19772)
Note this commit also slightly changes internal API: channel instead of getChannel and updateCurrentUserChannelNotificationsSettings instead of updateCurrentUserChatChannelNotificationsSettings.

Also destroyChannel takes a second param which is the name confirmation instead of an optional object containing this confirmation. This is to enforce the fact that it's required.

In the future a top level jsdoc config file could be used instead of the hack tempfile, but while it's only an experiment for chat, it's probably good enough.
2023-01-18 12:36:16 +01:00
Rafael dos Santos Silva 076b3a6514
DEV: Key bundler CI cache on Ruby version (#19868) 2023-01-13 11:39:49 -03:00
Rafael dos Santos Silva 8e7e6e14c7
DEV: Add Ruby 3.2 to test matrix (#19862)
* DEV: Add Ruby 3.2 to test matrix

* DEV: Update test name
2023-01-13 09:22:33 -03:00
David Taylor 93e2dad656
DEV: Introduce syntax_tree code formatter (#19775)
This commit introduces the necessary gems and config, but adds all our ruby code directories to the `--ignore-files` list.

Future commits will apply syntax_tree to parts of the codebase, removing the ignore patterns as we go
2023-01-07 11:11:08 +00:00
Alan Guo Xiang Tan e58277adf3
DEV: Increase Capybara.default_max_wait_time on github actions (#19750)
Our working theory is that system tests on Github run on much less
powerful hardware as compared to running the tests on our work machines.
Hopefully, increasing the wait time now will help reduce some flakes
that we're seeing on Github.
2023-01-05 08:50:35 +08:00