Commit Graph

9572 Commits

Author SHA1 Message Date
David Taylor e0daacf3ef
DEV: Drop `/theme-qunit` from smoke test (#23562)
We will soon be dropping support for `/theme-qunit` in production, so this will start failing if we don't remove it. Plus, we now have system specs which verify the end-to-end functionality of the Theme QUnit system.

This was the last thing which was using the legacy `run-qunit` script, so that can also be dropped.
2023-09-13 16:14:27 +01:00
David Battersby 6e2b484f12
FIX: prevent lightbox images from double escaping titles (#23458)
This change fixes an issue where lightbox images are showing escaped text in the link title and lightbox image description area.
2023-09-13 14:33:08 +08:00
Sam 267e8ebaa6
FIX: min_personal_message_post_length not applying to first post (#23531)
* FIX: min_personal_message_post_length not applying to first post

Due to the way PostCreator is wired, we were not applying min_personal_message_post_length
to the first post.

This meant that admins could not configure it so PMs have different
limits.

The code was already pretending that this works, but had no reliable way
of figuring out if we were dealing with a private message
2023-09-13 15:43:54 +10:00
Penar Musaraj f6326d03f0
DEV: Bump max theme sprite size to 1MB (#23556)
There is one repo with very large sprites, and it's causing missing
icons on some instances.
2023-09-13 15:00:26 +10:00
Roman Rizzi 75e8a6bf90
Bump version to v3.2.0.beta2-dev 2023-09-12 15:41:11 -03:00
Roman Rizzi a9cc379121
Bump version to v3.2.0.beta1 2023-09-12 15:41:11 -03:00
Bianca Nenciu 6f782d8e45
SECURITY: Add limits for themes and theme assets
This commit adds limits to themes and theme components on the:

- file size of about.json and .discourse-compatibility
- file size of theme assets
- number of files in a theme
2023-09-12 15:31:31 -03:00
Daniel Waterworth 290306a932
SECURITY: Reduce maximum size of SVG sprite cache to prevent DoS
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-09-12 15:31:28 -03:00
OsamaSayegh c1b5faa5fd
SECURITY: Limit name field length of TOTP authenticators and security keys 2023-09-12 15:31:17 -03:00
David Taylor 8b51a89919
DEV: Do not squash commits in `version_bump:stage_security_fixes` (#23547)
Sometimes fixes will deliberately keep commits separate, and we don't want to undo that
2023-09-12 18:00:42 +01:00
Loïc Guitaut b7d7099d08 DEV: Add link to PR when generating release notes 2023-09-12 09:26:46 +02:00
Sam f25849501d
FEATURE: allow consumers to parse a search string (#23528)
This extends search so it can have consumers that:

1. Can split off "term" from various advanced filters and orders
2. Can build a relation of either order or filter

It also moves a lot of stuff around in the search class for clarity.

Two new APIs are exposed:

`.apply_filter` to apply all the special filters to a posts/topics relation
`.apply_order` to force a particular order (eg: order:latest)

This can then be used by semantic search in Discourse AI
2023-09-12 16:21:01 +10:00
Alan Guo Xiang Tan 07c29f3066
Revert "DEV: Run core system tests by default in docker test image (#23517)" (#23525)
This reverts commit 40acb9a111.

Reverting because test runs are breaking due to this change
2023-09-12 11:45:49 +10:00
Alan Guo Xiang Tan d2e4b32c87
DEV: Add support for uploading a theme from a directory in system tests (#23402)
Why this change?

Currently, we do not have an easy way to test themes and theme components
using Rails system tests. While we support QUnit acceptance tests for
themes and theme components, QUnit acceptance tests stubs out the server
and setting up the fixtures for server responses is difficult and can lead to a
frustrating experience. System tests on the other hand allow authors to
set up the test fixtures using our fabricator system which is much
easier to use.

What does this change do?

In order for us to allow authors to run system tests with their themes
installed, we are adding a `upload_theme` helper that is made available
when writing system tests. The `upload_theme` helper requires a single
`directory` parameter where `directory` is the directory of the theme
locally and returns a `Theme` record.
2023-09-12 07:38:47 +08:00
Daniel Waterworth 40acb9a111
DEV: Run core system tests by default in docker test image (#23517) 2023-09-11 16:04:33 -05:00
David Taylor 055d29d898
DEV: Correct sourceMappingURL regex (#23504)
This comment isn't necessarily on a line by itself, so we need to remove the `^` from the regex. This will fix `EMBER_ENV=development bin/rake assets:precompile`
2023-09-11 11:39:55 +01:00
David Taylor b07445ced8
DEV: Disable Webpack parallelization for low-memory environments (#23487)
This reduces memory usage for Embroider-based builds on low-memory servers (e.g. entry-level Digital Ocean droplets)
2023-09-11 09:32:37 +01:00
David Taylor 9667485951
DEV: Stop building test assets in production under Embroider (#23388)
Until now, we have allowed testing themes in production environments via `/theme-qunit`. This was made possible by hacking the ember-cli build so that it would create the `tests.js` bundle in production. However, this is fundamentally problematic because a number of test-specific things are still optimized out of the Ember build in production mode. It also makes asset compilation significantly slower, and makes it more difficult for us to update our build pipeline (e.g. to introduce Embroider).

This commit removes the ability to run qunit tests in production builds of the JS app when the Embdroider flag is enabled. If a production instance of Discourse exists exclusively for the development of themes (e.g. discourse.theme-creator.io) then they can add `EMBER_ENV: development` to their `app.yml` file. This will build the entire app in development mode, and has a significant performance impact. This must not be used for real production sites.

This commit also refactors many of the request specs into system specs. This means that the tests are guaranteed to have Ember assets built, and is also a better end-to-end test than simply checking for the presence of certain `<script>` tags in the HTML.
2023-09-11 09:12:37 +01:00
David Taylor 231ea8faa2
DEV: Correctly identify Embroider chunks (#23488)
This method is used by assets:precompile to decide whether to apply `terser` to a file. Embroider chunks do not necessarily start with `chunk.`, and so they were incorrectly being re-terser'd by our assets:precompile task. This is inefficient, and also led to broken sourcemaps on some assets.
2023-09-10 10:18:43 +01:00
Bianca Nenciu 4db5310135
DEV: Remove unused topic_create_allowed_category_ids (#23463) 2023-09-08 12:03:22 +03:00
Alan Guo Xiang Tan dc76d82f24
DEV: Fix broken conditional in `docker:test` Rake task (#23477)
Broke in ef73d20832
2023-09-08 12:16:02 +08:00
Alan Guo Xiang Tan ef73d20832
DEV: Fix `docker:test:setup` Rake task that was missing the DB (#23474)
This is a follow up to 9caba30d5c

In that commit, we were migrating the database but we didn't actually
ensure that the database was created and that plugins were updated
before the databases were migrated.
2023-09-08 10:16:23 +08:00
David Taylor 75ce01a69b
DEV: Ensure Embroider sourcemaps are collected by Sprockets (#23468)
Names of sourcemaps are not necessarily equal to the js file names. Instead, we can check the `sourceMappingURL` comment to find the map's filename.
2023-09-07 22:20:52 +01:00
Alan Guo Xiang Tan 1f0a78fb82
DEV: Remove accidentally hardcoded Redis port in `docker.rake` (#23455)
Follow up to 9caba30d5c
2023-09-07 14:22:14 +08:00
Alan Guo Xiang Tan 9caba30d5c
DEV: Add `docker:test:setup` Rake task (#23430)
## What is the context here?

The `docker.rake` Rakefile contains Rake tasks that are meant to be run
in the `discourse/discourse_test:release` Docker image. For example, we
have the `docker:test` Rake task that makes it easier to run the test
suite for a particular Discourse commit.

Why are we introducing a `docker:test:setup` Rake task?

While we have the `docker:test` Rake task, it is very limited in the
test commands that can be executed. It is very useful for automated
testing but not very useful for running tests in the development
environment. Therefore, we are introducing a `docker:test:setup` rake
task that can be used to set up the test environment for running tests.

The envisioned example usage is something like this:

```
docker run -d --name=discourse_test --entrypoint=/sbin/boot discourse/discourse_test:release
docker exec -u discourse:discourse discourse_test ruby script/docker_test.rb --no-tests
docker exec -u discourse:discourse discourse_test bundle exec rake docker:test:setup
docker exec -u discourse:discourse discourse_test bundle exec rspec <path to file>
```
2023-09-07 13:46:23 +08:00
Ted Johansson ad58b6d604
DEV: Validate before and bumped_before options in TopicQuery (#23451) 2023-09-07 14:38:03 +10:00
Ted Johansson e74560f062
FIX: Don't mix up action labels between different reviewables (#23365)
Currently, if the review queue has both a flagged post and a flagged chat message, one of the two will have some of the labels of their actions replaced by those of the other. In other words, the labels are getting mixed up. For example, a flagged chat message might show up with an action labelled "Delete post".

This is happening because when using bundles, we are sending along the actions in a separate part of the response, so they can be shared by many reviewables. The bundles then index into this bag of actions by their ID, which is something generic describing the server action, e.g. "agree_and_delete".

The problem here is the same action can have different labels depending on the type of reviewable. Now that the bag of actions contains multiple actions with the same ID, which one is chosen is arbitrary. I.e. it doesn't distinguish based on the type of the reviewable.

This change adds an additional field to the actions, server_action, which now contains what used to be the ID. Meanwhile, the ID has been turned into a concatenation of the reviewable type and the server action, e.g. post-agree_and_delete.

This still provides the upside of denormalizing the actions while allowing for different reviewable types to have different labels and descriptions.

At first I thought I would prepend the reviewable type to the ID, but this doesn't work well because the ID is used on the server-side to determine which actions are possible, and these need to be shared between different reviewables. Hence the introduction of server_action, which now serves that purpose.

I also thought about changing the way that the bundle indexes into the bag of actions, but this is happening through some EmberJS mechanism, so we don't own that code.
2023-09-06 10:57:30 +08:00
Alan Guo Xiang Tan 7cc5501cfd
DEV: Remove outdated ENV in docker.rake that is no longer being used (#23428)
We now have `USE_TURBO` which relis on `turbo_rspec`.
2023-09-06 09:07:57 +08:00
Jarek Radosz ee3ac739f3
DEV: Don't run dev-only code in specs (#23427)
This was causing the following notice to be printed out when running system specs:

```
I did no detect a custom `config/dev.yml` file, creating one for you where you can amend defaults.
```

(since 61571bee43)
2023-09-06 02:14:57 +02:00
Martin Brennan c532f6eb3d
FEATURE: Secure uploads in PMs only (#23398)
This adds a new secure_uploads_pm_only site setting. When secure_uploads
is true with this setting, only uploads created in PMs will be marked
secure; no uploads in secure categories will be marked as secure, and
the login_required site setting has no bearing on upload security
either.

This is meant to be a stopgap solution to prevent secure uploads
in a single place (private messages) for sensitive admin data exports.
Ideally we would want a more comprehensive way of saying that certain
upload types get secured which is a hybrid/mixed mode secure uploads,
but for now this will do the trick.
2023-09-06 09:39:09 +10:00
Martin Brennan de9b567c19
FIX: Admins not able to convert topics if they aren't in personal_message_enabled_groups (#23399)
Admins are always able to send PMs, so it doesn't make
sense that they shouldn't be able to convert topics just
because they aren't in personal_message_enabled_groups.
2023-09-06 09:17:40 +10:00
David Taylor 534f62cf5b
FIX: Respect default category sort when `filter=default` (#23411)
Previously we would respect it if the filter was `nil`, but if `default` was explicitly passed then it would ignore the category order settings. This explicit passing of `filter=default` happens for some types of navigations in the JS app.

This extends the fix from 92bc61b4be
2023-09-05 19:05:30 +01:00
Alan Guo Xiang Tan 738343d4d2
DEV: Remove unused line of comment (#23401)
Wizard tests have been merged into core so the env isn't being used
anymore.
2023-09-05 14:28:23 +08:00
David Taylor 082ccdbd66
DEV: Reduce theme-qunit smoke test timeout (#23394)
The theme tests we use for the smoke-test typically take 3-4 seconds to complete. This commit reduces the timeout from 10 minutes to 20 seconds, so that failures are detected more quickl
2023-09-04 23:10:40 +01:00
David Taylor b59f1ad4ee
DEV: Improve strategy for identifying ember-cli JS chunks (#23382)
Our Ember build compiles assets into multiple chunks. In the past, we used the output from ember-auto-import-chunks-json-generator to give Rails a map of those chunks. However, that addon is specific to ember-auto-import, and is not compatible with Embroider.

Instead, we can switch to parsing the html files which are output by ember-cli. These are guaranteed to have the correct JS files in the correct place. A <discourse-chunked-script> will allow us to easily identify which chunks belong to which entrypoint.

In future, as we update more entrypoints to be compiled by Embroider/Webpack, we can easily introduce new wrappers.

Previously applied in 2c58d45 and reverted in 24d46fd. This version has been updated for subfolder support.
2023-09-04 13:56:34 +01:00
Chris Rendle-Short 7ea3079e3e
FIX: Update DIRECT_UPLOAD CORS ruleset to include new Amazon signing headers (#23379)
New headers were added to upload PUT requests as part of a MinIO update (cf42466). This change updates the asset bucket CORS ruleset to allow the new headers in the preflight request.

See https://dev.discourse.org/t/111136



Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2023-09-04 16:09:52 +10:00
Joffrey JAFFEUX f1d8cd529e
Revert "Revert "PERF: Cache each theme field value once (#23192)" (#23354)" (#23356)
This reverts commit 9821ca9413.
2023-08-31 14:12:03 -05:00
Joffrey JAFFEUX 24d46fd981
Revert "DEV: Improve strategy for identifying ember-cli JS chunks (#23336)" (#23355)
This reverts commit 2c58d456dd.
2023-08-31 19:28:48 +02:00
Joffrey JAFFEUX 9821ca9413
Revert "PERF: Cache each theme field value once (#23192)" (#23354)
This reverts commit 82a56334a3.
2023-08-31 19:04:43 +02:00
Daniel Waterworth 82a56334a3
PERF: Cache each theme field value once (#23192)
Previously, theme fields from components would be cached for each of
their parent themes.
2023-08-31 11:24:02 -05:00
Penar Musaraj 006a5166e5
DEV: Refactor rp_id and rp_name (#23339)
They're both constant per-instance values, there is no need to store them
in the session. This also makes the code a bit more readable by moving
the `session_challenge_key` method up to the `DiscourseWebauthn` module.
2023-08-31 09:11:23 -04:00
David Taylor 2c58d456dd
DEV: Improve strategy for identifying ember-cli JS chunks (#23336)
Our Ember build compiles assets into multiple chunks. In the past, we used the output from `ember-auto-import-chunks-json-generator` to give Rails a map of those chunks. However, that addon is specific to ember-auto-import, and is not compatible with Embroider.

Instead, we can switch to parsing the html files which are output by ember-cli. These are guaranteed to have the correct JS files in the correct place. A `<discourse-chunked-script>` will allow us to easily identify which chunks belong to which entrypoint.

In future, as we update more entrypoints to be compiled by Embroider/Webpack, we can easily introduce new wrappers.
2023-08-30 18:47:06 +01:00
Martin Brennan 2965519c76
DEV: Add auto _map extensions for list site settings with no type (#23331)
Followup to eea74e0e32. Site settings
which are a list without a list_type should also have the _map
extension added which returns an array based on split("|").

For example:

```
SiteSetting.post_menu_map
=> ["read", "like"]
```
2023-08-30 16:14:06 +10:00
Renato Atilio 58b49bce41
FEATURE: support to initial values for form templates through /new-topic (#23313)
* FEATURE: adds support for initial values through /new-topic to form templates
2023-08-29 18:41:33 -03:00
Blake Erickson 5d438f805c
DEV: Add rake command to help detect dead settings (#23300)
* DEV: Add rake command to help detect dead settings

Some Site Settings may still exist but are no longer being used in the
core discourse code or in related plugins. This rake task will help
identify any unused (aka: dead) settings by using the `rg` command to
search for them.

You can execute the rake task by using this command:

`LOAD_PLUGINS=1 bin/rails "site_settings:find_dead"`

* Add env variable, apply feedback
2023-08-29 09:42:52 -06:00
Jarek Radosz 09446baf10
DEV: Fix `redis.sadd` warnings (#23244)
```
Redis#sadd will always return an Integer in Redis 5.0.0. Use Redis#sadd? instead
```
2023-08-28 12:58:47 +08:00
David Taylor 92bc61b4be
FIX: Respect default category sort order when navigating within app (#23270)
When navigating around, we make ajax requests with a parameter like `?filter=latest`. This results in the TopicQuery being set up with `filter: "latest"` as a string. The logic introduced in fd9a5bc0 checks for equality with `:latest` and `:unseen` symbols, which didn't work correctly in this situation

This commit makes the logic detect both strings and symbols, and adds a spec for the behaviour.
2023-08-25 16:49:49 +01:00
Jarek Radosz 3ce3c2ff81
DEV: Add a spec for `assets:precompile:js_processor` (#23220) 2023-08-25 11:44:30 +02:00
Kelv 93c2de3ba5
DEV: add geoblocking and tag-by-group plugins (#23255) 2023-08-25 16:22:54 +08:00
Jarek Radosz e60d227c8f
DEV: Avoid constant redefinition warnings in specs (#23241)
Specs sometimes do `Discourse::Application.load_tasks` which re-loads rake task files, causing constant redefinition.
2023-08-24 23:16:32 +02:00