Commit Graph

1268 Commits

Author SHA1 Message Date
Jarek Radosz 1e59e18ad2
FIX: Compile js-processor before db:migrate (#23229)
In production env it's possible to have migrations run before js-processor is available.
2023-08-24 19:24:43 +02:00
Jarek Radosz 70f1cc5552
DEV: Use esbuild to make DiscourseJsProcessor (#23223)
Reverts e2705df and re-lands #23187 and #23219.

The issue was incorrect order of execution of Rails' `assets:precompile` task in our own precompilation stack.

Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 16:36:22 +02:00
David Taylor e2705df0f4
Revert "DEV: Use esbuild to make DiscourseJsProcessor (#23187)" (#23221)
This reverts commit 4dfe25d062 and 4fdeb6281e. We are investigating an issue related to asset compilation and S3 assets
2023-08-24 13:25:44 +01:00
Jarek Radosz 4fdeb6281e
FIX: `assets:precompile:js_processor` task issue (#23219)
* Fix the reference
* Use mutex in non-prod only (…and don't try to build the processor in runtime in prod)
2023-08-24 13:19:57 +02:00
Jarek Radosz 4dfe25d062
DEV: Use esbuild to make DiscourseJsProcessor (#23187)
Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 12:43:59 +02:00
Jarek Radosz 9b63ac473b
FIX: `next` vs `return` in maxmind task (#23196)
(and added the dependence on `environment` for trying the task independently of `assets:precompile`)
2023-08-22 23:00:02 +02:00
Jarek Radosz c9de84c63d
DEV: Extract maxmind refresh logic to its own task (#23195) 2023-08-22 22:27:16 +02:00
Sam e654edf844
FEATURE: do not bump topics when retroactively closing (#23115)
The category feature that automatically closes topics does it silently

This amends it so `rake topics:apply_autoclose` which does retroactive
closing will also do so silently.
2023-08-16 11:20:47 +10:00
Jarek Radosz 8bebd8fd99
DEV: Format .thor files (#23059) 2023-08-10 13:59:13 +02:00
Jarek Radosz 94649565ce
DEV: Correct `Style/RedundantReturn` rubocop issues (#23052) 2023-08-10 02:03:38 +02:00
Gerhard Schlager eabea3e8fd DEV: Create missing user profiles in "import:ensure_consistency" rake task 2023-08-09 20:56:14 +02:00
Gerhard Schlager b2fee68b3f DEV: Add rake task for generating avatars from SSO 2023-08-09 20:56:14 +02:00
David Taylor 13b13a758c
Remove discourse-canned-replies from official list (#22932)
This plugin is no longer supported, and so we no longer need to run its tests in CI

(removing the comment and the 'Canned Replies' value from the array caused syntax_tree to change to the `%w` syntax)
2023-08-02 12:48:20 +01:00
David Taylor 8a0d00b866
DEV: Introduce version_bump rake tasks (#22817)
This commit introduces five rake tasks to help us with version bump procedures:

- `version_bump:beta` and `version_bump:minor_stable` are for our minor releases
- `version_bump:major_stable_prepare` and `version_bump:major_stable_merge` are for our major release process
- `version_bump:stage_security_fixes` is to collate multiple security fixes from private branches into a single branch for release

The scripts will stage the necessary commits in a branch and prompt you to create a PR for review. No changes to release branches or tags will be made without the PR being approved, and explicit confirmation of prompts in the scripts.

To avoid polluting the operator's primary working tree, the scripts create a temporary git worktree in a temporary directory and perform all checkouts/commits there.
2023-07-31 16:05:13 +01:00
Jarek Radosz 3a11c82547
DEV: Update minitest to 5.19.0 (#22821) 2023-07-27 12:18:40 +02:00
Isaac Janzen 37942cb8bb
DEV: Convert `admin-incoming-email` modal to component-based API (#22701)
- Convert `admin-incoming-email` modal to component-based API
- Testing that the modal was working in local development was extremely challenging due to the need for `rejected` and `bounced` emails. Something that is not easy to stub in a local dev environment. To make this process more smooth for future developers I have added a new rake task:

```
desc "Creates sample email logs"
task "email_logs:populate" => ["db:load_config"] do |_, args|
  DiscourseDev::EmailLog.populate!
end
```

That will generate fully functional email logs in development to be toyed with.

<img width="787" alt="Screenshot 2023-07-20 at 3 27 04 PM" src="https://github.com/discourse/discourse/assets/50783505/47b3fe34-cd7e-49a5-8fe6-768c0fbd1aa2">
2023-07-20 16:31:20 -05:00
David Taylor 9c915345ea
DEV: Simplify qunit target selection (#22591)
Previously we had three query parameters to control which tests would be run. The default was to run all core/plugin tests together, which would almost always lead to errors and does not match the way we run tests in CI.

This commit removes the three old parameters (skip_core, skip_plugins and single_plugin), and introduces a new 'target' parameter. This can have a value of 'core', 'plugins', 'all', or a specific plugin name. The default is 'core'. Attempting to use the old parameters will raise an error.
2023-07-13 14:20:00 +01:00
Matt Palmer bd9c919e06
FIX: don't use etags for post-upload verification (#21923)
They don't work for server-side encryption with customer keys, and so instead we just use Content-MD5 to ensure there was no corruption in transit, which is the best we can do.

See also: https://meta.discourse.org/t/s3-uploads-incompatible-with-server-side-encryption/266853
2023-07-07 09:53:49 +02:00
Osama Sayegh d2d6d727de
FEATURE: Add default site settings to control the defaults of navigation menu preferences (#22485)
Follow-up to b27e12445d

This commit adds 2 new site settings `default_sidebar_link_to_filtered_list` and `default_sidebar_show_count_of_new_items` to control the default values for the navigation menu preferences that were added in the linked commit (`sidebar_link_to_filtered_list` and `sidebar_show_count_of_new_items` respectively).
2023-07-07 04:52:10 +03:00
Jarek Radosz 6b4762e450
DEV: Delete `uglify` asset codepath (#22346)
We no longer process any thrid-party assets with uglify
2023-06-29 12:23:46 +02:00
Neil Lalonde d6a82f1034
DEV: Don't print plugin changes in details blocks (#22328) 2023-06-28 11:28:08 -04:00
Sam 0de3b279ce
FEATURE: add db:resize:notification_id task for growing table (#20505)
Under exceptional cases people may need to resize the notification table.
This only happens on forums with a total of more than 2.5 billion notifications.

This rake task can be used to convert all the notification columns to
bigint to make more room.
2023-06-21 09:57:16 -04:00
Emmanuel Ferdman 722180edba
DEV: Typo in an output message of uploads.rake (#22209)
Signed-off-by: emmanuel-ferdman <emmanuelferdman@gmail.com>
2023-06-21 12:00:26 +08:00
David Taylor 9c926ce645
PERF: Improve workbox loading strategy (#22019)
Previously workbox JS was vendored into our git repository, and would be loaded from the `public/javascripts` directory with a 1 day cache lifetime. The main aim of this commit is to add 'cachebuster' to the workbox URL so that the cache lifetime can be increased.

- Remove vendored copies of workbox.
- Use ember-cli/broccoli to collect workbox files from node_modules into assets/workbox-{digest}
- Add assets to sprockets manifest so that they're collected from the ember-cli output directory (and uploaded to s3 when configured)

Some of the sprockets-related changes in this commit are not ideal, but we hope to remove sprockets in the not-too-distant future.
2023-06-09 11:14:11 +01:00
Matt Palmer a98d2a8086
FEATURE: allow S3 ACLs to be disabled (#21769)
AWS recommends running buckets without ACLs, and to use resource policies to manage access control instead.
This is not a bad idea, because S3 ACLs are whack, and while resource policies are also whack, they're a more constrained form of whack.
Further, some compliance regimes get antsy if you don't go with the vendor's recommended settings, and arguing that you need to enable ACLs on a bucket just to store images in there is more hassle than it's worth.
The new site setting (s3_use_acls) cannot be disabled when secure
uploads is enabled -- the latter relies on private ACLs for security
at this point in time. We may want to reexamine this in future.
2023-06-06 15:47:40 +10:00
David Taylor 0330f51d75
DEV: Ensure `maxminddb:fetch` works as standalone task (#21893)
`DiscourseIpInfo` expects zeitwerk auto-loading to be available, so we need to ensure the rake task loads the full rails environment. Normally we run this task as part of assets:precompile, so the app is already initialized. This commit only affects the case where the maxmind task is run directly.
2023-06-01 21:46:49 +01:00
Bianca Nenciu 61a0ae3755
FEATURE: Create legal topics for set company name (#21620)
Legal topics, such as the Terms of Service and Privacy Policy topics
do not make sense if the entity creating the community is not a company.
These topics will be created and updated only when the company name is
present and deleted when it is not.
2023-05-24 23:05:36 +03:00
Jarek Radosz 7f85624a01
DEV: Fix plugin:spec task return code (#21661)
Regressed in eec10efc3d. It means that backend plugin spec failures in CI were not failing the spec suite.

Fixes recent regressions and skips two of them - to be handled next week.

---------

Co-authored-by: Andrei Prigorshnev <a.prigorshnev@gmail.com>
2023-05-19 16:17:02 +02:00
Régis Hanol db9d998de3
FIX: improve mailman email parsing (#21627)
https://meta.discourse.org/t/improving-mailman-email-parsing/253041

When mirroring a public mailling list which uses mailman, there were some cases where the incoming email was not associated to the proper user.

As it happens, for various (undertermined) reasons, the email from the sender is often not in the `From` header but can be in any of the following headers: `Reply-To`, `CC`, `X-Original-From`, `X-MailFrom`.

It might be in other headers as well, but those were the ones we found the most reliable.
2023-05-19 10:33:48 +02:00
Bianca Nenciu f0ec1fad8c
FIX: Update category tag stats with new or deleted (#21531)
The old method updated only existing records, without considering that
new tags might have been created or some tags might not exist anymore.
This was usually not a problem because the stats were also updated by
other code paths.

However, the ensure consistency job should be more solid and help when
other code paths fail or after importing data.

Also, update category tag stats too should happen when updating other
category stats as well.
2023-05-18 12:46:44 +03:00
Joffrey JAFFEUX 0b4e14aa74
DEV: prevents `rake emoji:update` to run on prod (#21594)
This task is only designed to help generate a pull request to update official emoji images. It should never be run directly on production.
2023-05-16 20:46:57 +02:00
Jarek Radosz eec10efc3d
DEV: Enable color CI output and tweak formatting (#21527)
* Color for turbo_rspec in CI (`progress` and `documentation` formats)
* Show "DONE" only when `documentation` formatter is used
* Fix formatting
* Collapse RSpec commands
* Add line wrapping to the `progress` formatter (to mitigate GH Actions issue)
2023-05-12 18:22:15 +02:00
Jarek Radosz 19ac90536f
DEV: Restore the documentation format in system tests (#21471) 2023-05-12 11:13:52 +02:00
Ted Johansson b837459e1d
DEV: Add both safe and unsafe Discourse.store.download methods (#21498)
* DEV: Add both safe and unsafe Discourse.store.download methods

* DEV: Update call sites that can use the safe store download method
2023-05-11 17:27:27 +08:00
Daniel Waterworth bc236119d3
DEV: Increase docker.rake system test timeout time to match CI (#21339) 2023-05-02 12:12:03 -05:00
Daniel Waterworth a4dd758f5c
DEV: Add 20m timeout to system tests (#21323) 2023-05-01 13:23:48 -05:00
Mark VanLandingham 7f0682f4f2
DEV: Wrap remote theme update in transaction (#21302) 2023-04-28 12:11:53 -05:00
Jan Cernik afe3e36363
DEV: Remove lazy-yt and replace with lazy-videos (#20722)
- Refactors the old plugin to remove jquery usage
- Adds support for Vimeo videos (default on) and Tiktok (experimental and default off)
2023-03-29 11:54:25 -04:00
Krzysztof Kotlarek 4047073292
FIX: display validation under custom sidebar fields (#20772)
Before, incorrectly filled fields were marked with red border. Now, additional information under the field is displayed to notify the user what is incorrect.

/t/93696
2023-03-27 13:03:16 +11:00
Jarek Radosz 627f69738f
DEV: Capture output in hashtags spec (#20773) 2023-03-23 11:47:14 +10:00
Sam 0364ef5efe
FIX: MaxMindDB download failures caused a crash on rebuild (#20737)
We did not properly define the "name" variable leading to the error handling
to crash.

This quick fix ensures it can gracefully fail.
2023-03-20 18:23:54 +11:00
Daniel Waterworth 293cb7bde2
FIX: An ember build is required to run the system tests (#20725) 2023-03-17 13:20:49 -05:00
Daniel Waterworth 164b60cd07 DEV: Optionally, run system tests in docker:test 2023-03-15 16:46:48 -05:00
Daniel Waterworth 84f590ab83
DEV: Store theme sprites in the DB (#20501)
Let's avoid fetching sprites from the CDN during page rendering.
2023-03-14 13:11:45 -05:00
Jay Pfaffman 4350a903ec
clear security keys in disable_2fa rake task (#20586) 2023-03-09 15:38:59 -05:00
Constanza c051992098
FIX: mark posted as true for post authors in the TopicUser table during ensure_consistency task (#20473)
The ensure_consistency rake task was not marking posted as true for post authors in the TopicUser table, post migration. Create another step to set posted='t'.
2023-02-28 20:45:32 -04:00
Penar Musaraj c052df412a
DEV: Add support for aliases in HighlightJS languages (#20380)
We were only supporting the main name of each HighlightJS language. So, by default, you could not use `js` or `jsx` to highlight Javascript, given they are aliases for `javascript`.

This PR adds a list of aliases as a constant to core (built via a rake task), and then checks against the `highlighted_languages` site settings plus the list of aliases when processing a code block.
2023-02-23 15:06:06 -05:00
Martin Brennan 4ab1f76499
DEV: Fix bin/turbo_rspec runtime recording (#20407)
This commit 57caf08e13 broke
`bin/turbo_rspec` timing recording via `TurboTests::Runner`,
because we changed to using all `spec/*` folders except
`spec/system` as default for the runner, rather than
the old `['spec']` array, which is what `TurboTests::Runner`
was relying on to determine whether to record test run
time with `ParallelTests::RSpec::RuntimeLogger`.

Instead, we can just pass a new `use_runtime_info` boolean to the
runner class and use it when running against the default set of
spec files using `bin/turbo_rspec` and the turbo rspec rake task.
2023-02-23 07:47:11 +10:00
Loïc Guitaut f7c57fbc19 DEV: Enable `unless` cops
We discussed the use of `unless` internally and decided to enforce
available rules from rubocop to restrict its most problematic uses.
2023-02-21 10:30:48 +01:00
Osama Sayegh f91631b625
DEV: Update javascript:update_constants rake task following template colocation (#20365)
As of ba3f62f576, handlebars templates are colocated with js files so the path to hbs templates referenced by this rake task is no longer valid. This commit fixes the path to hbs templates and updates a couple of files that are generated by the rake task.
2023-02-20 06:20:47 +03:00