Commit Graph

9557 Commits

Author SHA1 Message Date
David Taylor 055d29d898
DEV: Correct sourceMappingURL regex (#23504)
This comment isn't necessarily on a line by itself, so we need to remove the `^` from the regex. This will fix `EMBER_ENV=development bin/rake assets:precompile`
2023-09-11 11:39:55 +01:00
David Taylor b07445ced8
DEV: Disable Webpack parallelization for low-memory environments (#23487)
This reduces memory usage for Embroider-based builds on low-memory servers (e.g. entry-level Digital Ocean droplets)
2023-09-11 09:32:37 +01:00
David Taylor 9667485951
DEV: Stop building test assets in production under Embroider (#23388)
Until now, we have allowed testing themes in production environments via `/theme-qunit`. This was made possible by hacking the ember-cli build so that it would create the `tests.js` bundle in production. However, this is fundamentally problematic because a number of test-specific things are still optimized out of the Ember build in production mode. It also makes asset compilation significantly slower, and makes it more difficult for us to update our build pipeline (e.g. to introduce Embroider).

This commit removes the ability to run qunit tests in production builds of the JS app when the Embdroider flag is enabled. If a production instance of Discourse exists exclusively for the development of themes (e.g. discourse.theme-creator.io) then they can add `EMBER_ENV: development` to their `app.yml` file. This will build the entire app in development mode, and has a significant performance impact. This must not be used for real production sites.

This commit also refactors many of the request specs into system specs. This means that the tests are guaranteed to have Ember assets built, and is also a better end-to-end test than simply checking for the presence of certain `<script>` tags in the HTML.
2023-09-11 09:12:37 +01:00
David Taylor 231ea8faa2
DEV: Correctly identify Embroider chunks (#23488)
This method is used by assets:precompile to decide whether to apply `terser` to a file. Embroider chunks do not necessarily start with `chunk.`, and so they were incorrectly being re-terser'd by our assets:precompile task. This is inefficient, and also led to broken sourcemaps on some assets.
2023-09-10 10:18:43 +01:00
Bianca Nenciu 4db5310135
DEV: Remove unused topic_create_allowed_category_ids (#23463) 2023-09-08 12:03:22 +03:00
Alan Guo Xiang Tan dc76d82f24
DEV: Fix broken conditional in `docker:test` Rake task (#23477)
Broke in ef73d20832
2023-09-08 12:16:02 +08:00
Alan Guo Xiang Tan ef73d20832
DEV: Fix `docker:test:setup` Rake task that was missing the DB (#23474)
This is a follow up to 9caba30d5c

In that commit, we were migrating the database but we didn't actually
ensure that the database was created and that plugins were updated
before the databases were migrated.
2023-09-08 10:16:23 +08:00
David Taylor 75ce01a69b
DEV: Ensure Embroider sourcemaps are collected by Sprockets (#23468)
Names of sourcemaps are not necessarily equal to the js file names. Instead, we can check the `sourceMappingURL` comment to find the map's filename.
2023-09-07 22:20:52 +01:00
Alan Guo Xiang Tan 1f0a78fb82
DEV: Remove accidentally hardcoded Redis port in `docker.rake` (#23455)
Follow up to 9caba30d5c
2023-09-07 14:22:14 +08:00
Alan Guo Xiang Tan 9caba30d5c
DEV: Add `docker:test:setup` Rake task (#23430)
## What is the context here?

The `docker.rake` Rakefile contains Rake tasks that are meant to be run
in the `discourse/discourse_test:release` Docker image. For example, we
have the `docker:test` Rake task that makes it easier to run the test
suite for a particular Discourse commit.

Why are we introducing a `docker:test:setup` Rake task?

While we have the `docker:test` Rake task, it is very limited in the
test commands that can be executed. It is very useful for automated
testing but not very useful for running tests in the development
environment. Therefore, we are introducing a `docker:test:setup` rake
task that can be used to set up the test environment for running tests.

The envisioned example usage is something like this:

```
docker run -d --name=discourse_test --entrypoint=/sbin/boot discourse/discourse_test:release
docker exec -u discourse:discourse discourse_test ruby script/docker_test.rb --no-tests
docker exec -u discourse:discourse discourse_test bundle exec rake docker:test:setup
docker exec -u discourse:discourse discourse_test bundle exec rspec <path to file>
```
2023-09-07 13:46:23 +08:00
Ted Johansson ad58b6d604
DEV: Validate before and bumped_before options in TopicQuery (#23451) 2023-09-07 14:38:03 +10:00
Ted Johansson e74560f062
FIX: Don't mix up action labels between different reviewables (#23365)
Currently, if the review queue has both a flagged post and a flagged chat message, one of the two will have some of the labels of their actions replaced by those of the other. In other words, the labels are getting mixed up. For example, a flagged chat message might show up with an action labelled "Delete post".

This is happening because when using bundles, we are sending along the actions in a separate part of the response, so they can be shared by many reviewables. The bundles then index into this bag of actions by their ID, which is something generic describing the server action, e.g. "agree_and_delete".

The problem here is the same action can have different labels depending on the type of reviewable. Now that the bag of actions contains multiple actions with the same ID, which one is chosen is arbitrary. I.e. it doesn't distinguish based on the type of the reviewable.

This change adds an additional field to the actions, server_action, which now contains what used to be the ID. Meanwhile, the ID has been turned into a concatenation of the reviewable type and the server action, e.g. post-agree_and_delete.

This still provides the upside of denormalizing the actions while allowing for different reviewable types to have different labels and descriptions.

At first I thought I would prepend the reviewable type to the ID, but this doesn't work well because the ID is used on the server-side to determine which actions are possible, and these need to be shared between different reviewables. Hence the introduction of server_action, which now serves that purpose.

I also thought about changing the way that the bundle indexes into the bag of actions, but this is happening through some EmberJS mechanism, so we don't own that code.
2023-09-06 10:57:30 +08:00
Alan Guo Xiang Tan 7cc5501cfd
DEV: Remove outdated ENV in docker.rake that is no longer being used (#23428)
We now have `USE_TURBO` which relis on `turbo_rspec`.
2023-09-06 09:07:57 +08:00
Jarek Radosz ee3ac739f3
DEV: Don't run dev-only code in specs (#23427)
This was causing the following notice to be printed out when running system specs:

```
I did no detect a custom `config/dev.yml` file, creating one for you where you can amend defaults.
```

(since 61571bee43)
2023-09-06 02:14:57 +02:00
Martin Brennan c532f6eb3d
FEATURE: Secure uploads in PMs only (#23398)
This adds a new secure_uploads_pm_only site setting. When secure_uploads
is true with this setting, only uploads created in PMs will be marked
secure; no uploads in secure categories will be marked as secure, and
the login_required site setting has no bearing on upload security
either.

This is meant to be a stopgap solution to prevent secure uploads
in a single place (private messages) for sensitive admin data exports.
Ideally we would want a more comprehensive way of saying that certain
upload types get secured which is a hybrid/mixed mode secure uploads,
but for now this will do the trick.
2023-09-06 09:39:09 +10:00
Martin Brennan de9b567c19
FIX: Admins not able to convert topics if they aren't in personal_message_enabled_groups (#23399)
Admins are always able to send PMs, so it doesn't make
sense that they shouldn't be able to convert topics just
because they aren't in personal_message_enabled_groups.
2023-09-06 09:17:40 +10:00
David Taylor 534f62cf5b
FIX: Respect default category sort when `filter=default` (#23411)
Previously we would respect it if the filter was `nil`, but if `default` was explicitly passed then it would ignore the category order settings. This explicit passing of `filter=default` happens for some types of navigations in the JS app.

This extends the fix from 92bc61b4be
2023-09-05 19:05:30 +01:00
Alan Guo Xiang Tan 738343d4d2
DEV: Remove unused line of comment (#23401)
Wizard tests have been merged into core so the env isn't being used
anymore.
2023-09-05 14:28:23 +08:00
David Taylor 082ccdbd66
DEV: Reduce theme-qunit smoke test timeout (#23394)
The theme tests we use for the smoke-test typically take 3-4 seconds to complete. This commit reduces the timeout from 10 minutes to 20 seconds, so that failures are detected more quickl
2023-09-04 23:10:40 +01:00
David Taylor b59f1ad4ee
DEV: Improve strategy for identifying ember-cli JS chunks (#23382)
Our Ember build compiles assets into multiple chunks. In the past, we used the output from ember-auto-import-chunks-json-generator to give Rails a map of those chunks. However, that addon is specific to ember-auto-import, and is not compatible with Embroider.

Instead, we can switch to parsing the html files which are output by ember-cli. These are guaranteed to have the correct JS files in the correct place. A <discourse-chunked-script> will allow us to easily identify which chunks belong to which entrypoint.

In future, as we update more entrypoints to be compiled by Embroider/Webpack, we can easily introduce new wrappers.

Previously applied in 2c58d45 and reverted in 24d46fd. This version has been updated for subfolder support.
2023-09-04 13:56:34 +01:00
Chris Rendle-Short 7ea3079e3e
FIX: Update DIRECT_UPLOAD CORS ruleset to include new Amazon signing headers (#23379)
New headers were added to upload PUT requests as part of a MinIO update (cf42466). This change updates the asset bucket CORS ruleset to allow the new headers in the preflight request.

See https://dev.discourse.org/t/111136



Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2023-09-04 16:09:52 +10:00
Joffrey JAFFEUX f1d8cd529e
Revert "Revert "PERF: Cache each theme field value once (#23192)" (#23354)" (#23356)
This reverts commit 9821ca9413.
2023-08-31 14:12:03 -05:00
Joffrey JAFFEUX 24d46fd981
Revert "DEV: Improve strategy for identifying ember-cli JS chunks (#23336)" (#23355)
This reverts commit 2c58d456dd.
2023-08-31 19:28:48 +02:00
Joffrey JAFFEUX 9821ca9413
Revert "PERF: Cache each theme field value once (#23192)" (#23354)
This reverts commit 82a56334a3.
2023-08-31 19:04:43 +02:00
Daniel Waterworth 82a56334a3
PERF: Cache each theme field value once (#23192)
Previously, theme fields from components would be cached for each of
their parent themes.
2023-08-31 11:24:02 -05:00
Penar Musaraj 006a5166e5
DEV: Refactor rp_id and rp_name (#23339)
They're both constant per-instance values, there is no need to store them
in the session. This also makes the code a bit more readable by moving
the `session_challenge_key` method up to the `DiscourseWebauthn` module.
2023-08-31 09:11:23 -04:00
David Taylor 2c58d456dd
DEV: Improve strategy for identifying ember-cli JS chunks (#23336)
Our Ember build compiles assets into multiple chunks. In the past, we used the output from `ember-auto-import-chunks-json-generator` to give Rails a map of those chunks. However, that addon is specific to ember-auto-import, and is not compatible with Embroider.

Instead, we can switch to parsing the html files which are output by ember-cli. These are guaranteed to have the correct JS files in the correct place. A `<discourse-chunked-script>` will allow us to easily identify which chunks belong to which entrypoint.

In future, as we update more entrypoints to be compiled by Embroider/Webpack, we can easily introduce new wrappers.
2023-08-30 18:47:06 +01:00
Martin Brennan 2965519c76
DEV: Add auto _map extensions for list site settings with no type (#23331)
Followup to eea74e0e32. Site settings
which are a list without a list_type should also have the _map
extension added which returns an array based on split("|").

For example:

```
SiteSetting.post_menu_map
=> ["read", "like"]
```
2023-08-30 16:14:06 +10:00
Renato Atilio 58b49bce41
FEATURE: support to initial values for form templates through /new-topic (#23313)
* FEATURE: adds support for initial values through /new-topic to form templates
2023-08-29 18:41:33 -03:00
Blake Erickson 5d438f805c
DEV: Add rake command to help detect dead settings (#23300)
* DEV: Add rake command to help detect dead settings

Some Site Settings may still exist but are no longer being used in the
core discourse code or in related plugins. This rake task will help
identify any unused (aka: dead) settings by using the `rg` command to
search for them.

You can execute the rake task by using this command:

`LOAD_PLUGINS=1 bin/rails "site_settings:find_dead"`

* Add env variable, apply feedback
2023-08-29 09:42:52 -06:00
Jarek Radosz 09446baf10
DEV: Fix `redis.sadd` warnings (#23244)
```
Redis#sadd will always return an Integer in Redis 5.0.0. Use Redis#sadd? instead
```
2023-08-28 12:58:47 +08:00
David Taylor 92bc61b4be
FIX: Respect default category sort order when navigating within app (#23270)
When navigating around, we make ajax requests with a parameter like `?filter=latest`. This results in the TopicQuery being set up with `filter: "latest"` as a string. The logic introduced in fd9a5bc0 checks for equality with `:latest` and `:unseen` symbols, which didn't work correctly in this situation

This commit makes the logic detect both strings and symbols, and adds a spec for the behaviour.
2023-08-25 16:49:49 +01:00
Jarek Radosz 3ce3c2ff81
DEV: Add a spec for `assets:precompile:js_processor` (#23220) 2023-08-25 11:44:30 +02:00
Kelv 93c2de3ba5
DEV: add geoblocking and tag-by-group plugins (#23255) 2023-08-25 16:22:54 +08:00
Jarek Radosz e60d227c8f
DEV: Avoid constant redefinition warnings in specs (#23241)
Specs sometimes do `Discourse::Application.load_tasks` which re-loads rake task files, causing constant redefinition.
2023-08-24 23:16:32 +02:00
Rafael dos Santos Silva d10e9a6c1d
FEATURE: Onebox and Download for WEBP and AVIF (#23235)
This adds support for oneboxing WEBP and AVIF images in posts and fixing
oneboxing fixes download remote images for those formats too.

Reported in https://meta.discourse.org/t/-/276433?u=falco
2023-08-24 16:44:06 -03:00
Jarek Radosz 1e59e18ad2
FIX: Compile js-processor before db:migrate (#23229)
In production env it's possible to have migrations run before js-processor is available.
2023-08-24 19:24:43 +02:00
Jarek Radosz 70f1cc5552
DEV: Use esbuild to make DiscourseJsProcessor (#23223)
Reverts e2705df and re-lands #23187 and #23219.

The issue was incorrect order of execution of Rails' `assets:precompile` task in our own precompilation stack.

Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 16:36:22 +02:00
David Taylor e2705df0f4
Revert "DEV: Use esbuild to make DiscourseJsProcessor (#23187)" (#23221)
This reverts commit 4dfe25d062 and 4fdeb6281e. We are investigating an issue related to asset compilation and S3 assets
2023-08-24 13:25:44 +01:00
Jarek Radosz 4fdeb6281e
FIX: `assets:precompile:js_processor` task issue (#23219)
* Fix the reference
* Use mutex in non-prod only (…and don't try to build the processor in runtime in prod)
2023-08-24 13:19:57 +02:00
Jarek Radosz 4dfe25d062
DEV: Use esbuild to make DiscourseJsProcessor (#23187)
Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 12:43:59 +02:00
Martin Brennan cf42466dea
DEV: Add S3 upload system specs using minio (#22975)
This commit adds some system specs to test uploads with
direct to S3 single and multipart uploads via uppy. This
is done with minio as a local S3 replacement. We are doing
this to catch regressions when uppy dependencies need to
be upgraded or we change uppy upload code, since before
this there was no way to know outside manual testing whether
these changes would cause regressions.

Minio's server lifecycle and the installed binaries are managed
by the https://github.com/discourse/minio_runner gem, though the
binaries are already installed on the discourse_test image we run
GitHub CI from.

These tests will only run in CI unless you specifically use the
CI=1 or RUN_S3_SYSTEM_SPECS=1 env vars.

For a history of experimentation here see https://github.com/discourse/discourse/pull/22381

Related PRs:

* https://github.com/discourse/minio_runner/pull/1
* https://github.com/discourse/minio_runner/pull/2
* https://github.com/discourse/minio_runner/pull/3
2023-08-23 11:18:33 +10:00
Jarek Radosz 9b63ac473b
FIX: `next` vs `return` in maxmind task (#23196)
(and added the dependence on `environment` for trying the task independently of `assets:precompile`)
2023-08-22 23:00:02 +02:00
Jarek Radosz c9de84c63d
DEV: Extract maxmind refresh logic to its own task (#23195) 2023-08-22 22:27:16 +02:00
Penar Musaraj 4c3ee1b211
DEV: Remove unused code (#23171)
This method slipped in by accident in 10c6b2a and is not used.
2023-08-21 15:01:05 -04:00
Mark VanLandingham 730f652255
DEV: Add plugin modifier locations for user search locations (#23169) 2023-08-21 12:23:42 -05:00
David Taylor 82b16f4f47
DEV: Do not manipulate theme module paths at build-time (#23148)
Manipulating theme module paths means that the paths you author are not the ones used at runtime. This can lead to some very unexpected behavior and potential module name clashes. It also meant that the refactor in 16c6ab8661 was unable to correctly match up theme connector js/templates.

While this could technically be a breaking change, I think it is reasonably safe because:

1. Themes are already forced to use relative paths when referencing their own modules (since they're namespaced based on the site-specific id). The only time this might be problematic is when theme tests reference modules in the theme's main `javascripts` directory

2. For things like components/services/controllers/etc. our custom Ember resolver works backwards from the end of the path, so adding `discourse/` in the middle will not affect resolution.
2023-08-18 18:15:23 +01:00
Selase Krakani 87ebbec9b2
FIX: Pending post deletion by creator (#23130)
`ReviewableQueuedPost` got refactored a while back to use the more
appropriate `target_created_by` for the user of the post being queued
instead of `created_by`. The change was not extended to the `DELETE
/review/:id` endpoint leading to error responses for a user attempting
to deleting their own queued post.

This fix extends the `Reviewable` lookup implementation in
`ReviewablesController#destroy` and Guardian implementation to account
for this change.
2023-08-18 15:30:59 +00:00
Penar Musaraj 10c6b2a0c2
WIP: Rename Webauthn to DiscourseWebauthn (#23077) 2023-08-18 08:39:10 -04:00
Osama Sayegh 09d3709ec9
FEATURE: New topics vs replies toggle for the new new view (#22920)
This PR adds a new toggle to switch the (new) /new list between showing topics with new replies (a.k.a unread topics), new topics, or everything mixed together.
2023-08-18 12:44:04 +08:00