Commit Graph

1211 Commits

Author SHA1 Message Date
David Taylor eebe2f256c
DEV: Skip s3 asset deletion when in readonly mode (#19611)
In some situations (e.g. disaster recovery), it may make sense to spin up a temporary readonly version of a cluster. In that situation, the s3 `expire_missing_assets` job would delete assets which are still in use by the canonical read-write version of the cluster.

To avoid that, this commit will skip deletion if the site is currently in readonly mode.
2022-12-30 11:35:40 +00:00
Alan Guo Xiang Tan 0da79561c3
DEV: Improve/Fix script/bench.rb (#19646)
1. Fix bug where we were not waiting for all unicorn workers to start up
before running benchmarks.

2. Fix a bug where headers were not used when benchmarking. Admin
benchmarks were basically running as anon user.

3. Disable rate limits when in profile env. We're pretty much going to
hit the rate limit every time as a normal user.

4. Benchmark against topic with a fixed posts count of 100. Previously profiling script was just randomly creating posts
and we would benchmark against a topic with a fixed posts count of 30.
Sometimes, the script fails because no topics with a posts count of 30
exists.

5. Benchmarks are not run against a normal user on top of anon and
admin.

6. Add script option to select tests that should be run.
2022-12-30 07:25:11 +08:00
David Taylor 104a16610a DEV: Configure prettier for hbs templates 2022-12-28 13:11:12 +00:00
David Taylor a6af981e1b
DEV: Add retry logic to `plugin:install_all_official` (#19551)
This task sometimes fails in CI due to temporary network issues. Retrying twice should help resolve those situations without needing to manually restart the job.
2022-12-21 15:31:03 +00:00
David Taylor ca72d8d030
PERF: Adjust node memory threshold for assets:precompile (#19040)
Previously we were forcing node's max-old-space-size to be 2GB. This override was added in a01b1dd6 to avoid issues caused by a lower default node heap_size_limit on machines with less memory.

This commit makes that `max-old-space-size` override more specific so that it only applies to machines with less memory. Other machines will go use Node's defaults.

The override is also lowered to 1GB. This is still high enough for the build to complete, while reducing memory usage.

https://meta.discourse.org/t/245547
2022-11-15 22:02:13 +00:00
Jarek Radosz e54a3d5ea9
DEV: Add `START_ID` to `uploads:downsize` task (#18992) 2022-11-11 22:51:48 +01:00
Jarek Radosz dc8a7e74f4
FIX: Allow attr updates of over-size-limit uploads (#18986) 2022-11-11 17:56:11 +01:00
Jarek Radosz bc22fe4fdf
DEV: Convert the downsizing script to a rake task (#18976)
…to make it testable!
2022-11-11 13:00:44 +01:00
David Taylor 8700c5ee6b
PERF: Make stylesheet hashes consistent between deploys (#18909)
Previously the stylesheet cachebusting hash was based on the maximum mtime of files. This works well in development and during in-container updates (e.g. via docker_manager). However, when a fresh docker image is created for each deploy, the file mtimes will change even if the contents has not.

This commit changes the production logic to calculate the cachebuster from the filenames and contents of the relevant assets. This should be consistent across deploys, thereby improving cache hits and improving page load times.
2022-11-07 16:13:35 +00:00
David Taylor f30f9ec5d9
PERF: Update `s3:expire_missing_assets` to delete in batches (#18908)
Some sites may have thousands of stale assets - deleting them one-by-one is very slow.

Followup to e8570b5cc9
2022-11-07 12:53:14 +00:00
David Taylor e8570b5cc9
Fix and improve `s3:expire_missing_assets` task (#18863)
- Ensure it works with prefixed S3 buckets
- Perform a sanity check that all current assets are present on S3 before starting deletion
- Remove the lifecycle rule configuration and delete expired assets immediately. This task should be run post-deploy anyway, so adding a 10-day window is not required
2022-11-07 10:44:45 +00:00
Vinoth Kannan dea44ec923
FEATURE: new site setting to hide user profiles by default. (#18864)
Previously, we didn't have a site-wide setting to set the default behavior for user profile visibility and user presence features. But we already have a user preference for that.
2022-11-06 16:44:17 +05:30
Daniel Waterworth 6f48502dd2
DEV: Add rake task to install a theme from an archive (#18890) 2022-11-04 16:42:07 -05:00
David Taylor b18ebc9648
PERF: Correct should_skip? logic in `s3:upload` (#18862)
This task is supposed to skip uploading if the asset is already present in S3. However, when a bucket 'folder path' was configured, this logic was broken and so the assets would be re-uploaded every time.

This commit fixes that logic to include the bucket 'folder path' in the check
2022-11-04 17:50:46 +00:00
Joffrey JAFFEUX 11f3618b80
DEV: initial system tests for chat and plugins (#18881)
This is a very basic to ensure it's working and open future possible work
2022-11-04 15:06:24 +01:00
Daniel Waterworth 167181f4b7
DEV: Quote values when constructing SQL (#18827)
All of these cases should already be safe, but still good to quote for
"defense in depth".
2022-11-01 14:05:13 -05:00
David Taylor 1da5aa9027
DEV: Parallelize core JS tests in docker.rake (#18756) 2022-10-26 12:00:35 +01:00
Jan Cernik 08476f17ff
FEATURE: Add dark mode option for category logos (#18460)
Adds a new upload field for a second dark mode category logo. 
This alternative will be used when the browser is in dark mode (similar to the global site setting for a dark logo).
2022-10-07 11:00:44 -04:00
Penar Musaraj 4d8011032e
DEV: Add a rake task to export/import translation overrides (#18487)
Use `bin/rake export:translation_overrides` to export to a file. Then,
copy that file to a new site and run `bin/rake import:file["filename"].
2022-10-05 15:22:16 -04:00
Dan Gebhardt 03b7b7d1bc
DEV: Remove usage of {{action}} modifiers - Take 2 (#18476)
This PR enables the [`no-action-modifiers`](https://github.com/ember-template-lint/ember-template-lint/blob/master/docs/rule/no-action-modifiers.md) template lint rule and removes all usages of the `{{action}}` modifier in core.

In general, instances of `{{action "x"}}` have been replaced with `{{on "click" (action "x")}}`. 

In many cases, such as for `a` elements, we also need to prevent default event handling to avoid unwanted side effects. While the `{{action}}` modifier internally calls `event.preventDefault()`, we need to handle these cases more explicitly. For this purpose, this PR also adds the [ember-event-helpers](https://github.com/buschtoens/ember-event-helpers) dependency so we can use the `prevent-default` handler. For instance:

```
<a href {{on "click" (prevent-default (action "x"))}}>Do X</a>
```

Note that `action` has not in general been refactored away as a helper yet. In general, all event handlers should be methods on the corresponding component and referenced directly (e.g. `{{on "click" this.doSomething}}`). However, the `action` helper is used extensively throughout the codebase and often references methods in the `actions` hash on controllers or routes. Thus this refactor will also be extensive and probably deserves a separate PR.

Note: This work was done to complement #17767 by minimizing the potential impact of the `action` modifier override, which uses private API and arguably should be replaced with an AST transform.

This is a followup to #18333, which had to be reverted because it did not account for the default treatment of modifier keys by the {{action}} modifier.

Commits:
* Enable `no-action-modifiers` template lint rule
* Replace {{action "x"}} with {{on "click" (action "x")}}
* Remove unnecessary action helper usage
* Remove ctl+click tests for user-menu
   These tests now break in Chrome when used with addEventListener. As per the comment, they can probably be safely removed.
* Prevent default event handlers to avoid unwanted side effects
   Uses `event.preventDefault()` in event handlers to prevent default event handling. This had been done automatically by the `action` modifier, but is not always desirable or necessary.
* Restore UserCardContents#showUser action to avoid regression
   By keeping the `showUser` action, we can avoid a breaking change for plugins that rely upon it, while not interfering with the `showUser` argument that's been passed.
* Revert EditCategoryTab#selectTab -> EditCategoryTab#select
   Avoid potential breaking change in themes / plugins
* Restore GroupCardContents#showGroup action to avoid regression
   By keeping the `showGroup` action, we can avoid a breaking change for plugins that rely upon it, while not interfering with the `showGroup` argument that's been passed.
* Restore SecondFactorAddTotp#showSecondFactorKey action to avoid regression
   By keeping the `showSecondFactorKey` action, we can avoid a breaking change for plugins that rely upon it, while not interfering with the `showSecondFactorKey` property that's maintained on the controller.
* Refactor away from `actions` hash in ChooseMessage component
* Modernize EmojiPicker#onCategorySelection usage
* Modernize SearchResultEntry#logClick usage
* Modernize Discovery::Categories#showInserted usage
* Modernize Preferences::Account#resendConfirmationEmail usage
* Modernize MultiSelect::SelectedCategory#onSelectedNameClick usage
* Favor fn over action in SelectedChoice component
* Modernize WizardStep event handlers
* Favor fn over action usage in buttons
* Restore Login#forgotPassword action to avoid possible regression
* Introduce modKeysPressed utility
   Returns an array of modifier keys that are pressed during a given `MouseEvent` or `KeyboardEvent`.
* Don't interfere with click events on links with `href` values when modifier keys are pressed
2022-10-05 13:08:54 +01:00
David Taylor 585c584fdb
Revert "DEV: Remove usage of `{{action}}` modifiers (#18333)" (#18469)
This reverts commit ba27ee1637.

We found some issues with handling of cmd/ctrl/shift + click on `<a` elements
2022-10-04 12:27:26 +01:00
Dan Gebhardt ba27ee1637
DEV: Remove usage of `{{action}}` modifiers (#18333)
This PR enables the [`no-action-modifiers`](https://github.com/ember-template-lint/ember-template-lint/blob/master/docs/rule/no-action-modifiers.md) template lint rule and removes all usages of the `{{action}}` modifier in core.

In general, instances of `{{action "x"}}` have been replaced with `{{on "click" (action "x")}}`. 

In many cases, such as for `a` elements, we also need to prevent default event handling to avoid unwanted side effects. While the `{{action}}` modifier internally calls `event.preventDefault()`, we need to handle these cases more explicitly. For this purpose, this PR also adds the [ember-event-helpers](https://github.com/buschtoens/ember-event-helpers) dependency so we can use the `prevent-default` handler. For instance:

```
<a href {{on "click" (prevent-default (action "x"))}}>Do X</a>
```

Note that `action` has not in general been refactored away as a helper yet. In general, all event handlers should be methods on the corresponding component and referenced directly (e.g. `{{on "click" this.doSomething}}`). However, the `action` helper is used extensively throughout the codebase and often references methods in the `actions` hash on controllers or routes. Thus this refactor will also be extensive and probably deserves a separate PR.

Note: This work was done to complement #17767 by minimizing the potential impact of the `action` modifier override, which uses private API and arguably should be replaced with an AST transform.

Commits:
* Enable `no-action-modifiers` template lint rule
* Replace {{action "x"}} with {{on "click" (action "x")}}
* Remove unnecessary action helper usage
* Remove ctl+click tests for user-menu
   These tests now break in Chrome when used with addEventListener. As per the comment, they can probably be safely removed.
* Prevent default event handlers to avoid unwanted side effects
   Uses `event.preventDefault()` in event handlers to prevent default event handling. This had been done automatically by the `action` modifier, but is not always desirable or necessary.
* Restore UserCardContents#showUser action to avoid regression
   By keeping the `showUser` action, we can avoid a breaking change for plugins that rely upon it, while not interfering with the `showUser` argument that's been passed.
* Revert EditCategoryTab#selectTab -> EditCategoryTab#select
   Avoid potential breaking change in themes / plugins
* Restore GroupCardContents#showGroup action to avoid regression
   By keeping the `showGroup` action, we can avoid a breaking change for plugins that rely upon it, while not interfering with the `showGroup` argument that's been passed.
* Restore SecondFactorAddTotp#showSecondFactorKey action to avoid regression
   By keeping the `showSecondFactorKey` action, we can avoid a breaking change for plugins that rely upon it, while not interfering with the `showSecondFactorKey` property that's maintained on the controller.
* Refactor away from `actions` hash in ChooseMessage component
* Modernize EmojiPicker#onCategorySelection usage
* Modernize SearchResultEntry#logClick usage
* Modernize Discovery::Categories#showInserted usage
* Modernize Preferences::Account#resendConfirmationEmail usage
* Modernize MultiSelect::SelectedCategory#onSelectedNameClick usage
* Favor fn over action in SelectedChoice component
* Modernize WizardStep event handlers
* Favor fn over action usage in buttons
* Restore Login#forgotPassword action to avoid possible regression
2022-10-04 10:42:46 +02:00
Martin Brennan 8ebd5edd1e
DEV: Rename secure_media to secure_uploads (#18376)
This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality.

This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site.

Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing.

This also keeps compatibility with the `secure-media-uploads` path, and changes new
secure URLs to be `secure-uploads`.

Deprecated settings:

* secure_media -> secure_uploads
* secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails
* secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 09:24:33 +10:00
Joshua Rosenfeld 9662ca7ee4
DEV: Print plugin changes in details blocks (#18345)
Update formatting of the `release_note` rake task. Use `[details]` blocks for each change category so more plugins fit on one screen.
2022-09-23 10:29:00 -04:00
Bianca Nenciu 7d3e763f01
FIX: Query correct model in posts:inline_uploads (#18330)
Follow up to commit 9db8f00b3d.
2022-09-23 15:00:02 +03:00
Keegan George a23d19fab0
DEV: Emoji picker keyboard accessibility updates (#18331)
This PR makes some updates to the prior keyboard accessibility commit (eb98746):
- Makes `tabindex` attribute only appear on emoji markup in the emoji picker.
- After pressing the Esc key, focus returns to the <textarea/> input (composer editor or chat input)
2022-09-22 15:21:34 -07:00
David Taylor 0c743a591f
DEV: Allow `rake qunit` filter to be used alongside parallel option (#18326)
Followup to 61f5c8716d
2022-09-22 10:28:02 +01:00
Sergei Mellow 61f5c8716d
DEV: Update `rake qunit:test` to support filtering (#18249) 2022-09-21 14:00:50 -04:00
David Taylor e06b9d4a52
DEV: Remove support for legacy plugin JS compilation pipeline (#18293)
This became the default in b1755137
2022-09-21 12:38:02 +01:00
David Taylor ce21205d39
DEV: Add support for running plugin qunit in parallel (#18291)
For example, to run in three concurrent browsers and assemble the results:

```
QUNIT_PARALLEL=3 bin/rake "plugin:qunit[discourse-chat]"
```
2022-09-20 18:11:26 +01:00
David Taylor ef39193a06
DEV: Add `rake plugins:turbo_spec` task (#18289)
This leans on our existing `turbo_rspec` implementation to run plugin specs in parallel on all available cores
2022-09-20 15:42:54 +01:00
David Taylor 1bd1664ae0
DEV: Compile markdown-it-bundle with ember-cli (#18104)
We were already compiling the markdown bundle via ember-cli, but that version was only being used in the test environment. This commit improves the implementation, and updates the filename so it's also used in production.

This commit also
- Removes the vendored copy of `markdown-it.js` and fetches from node_modules instead
- Updates `pretty_text.rb` to remove the custom sprockets-manifest-parsing
- Removes `pretty-text-bundle.js`, which was only being used by `pretty_text.rb`
2022-08-29 19:11:59 +01:00
David Taylor 23ec819e70
DEV: Introduce debugging rake task for PostRevision issues (#17998)
See https://meta.discourse.org/t/232885 for context. This rake task enumerates problematic records and prints out a sample of each failure.
2022-08-19 11:30:46 +01:00
David Taylor 14cc67c9b2
DEV: Run `yarn install` for all `rake docker:test` invocations (#17844)
Previously we were only `yarn install`ing for linting and qunit runs. The Rails app now relies on a number of `node_modules` dependencies (e.g. for pretty_text, and discourse_js_processor), so we need to make sure they're available.
2022-08-09 12:28:52 +01:00
Daniel Waterworth e0ece3a77e
Revert "DEV: Improve multisite db scripts in dev (#17337)" (#17801)
This reverts commit 2e4056d185.

Unfortunately, this broke db:create and db:drop when multisite config
wasn't present.
2022-08-04 16:15:06 -05:00
Sérgio Saquetim c99f658a9e
DEV: Allow to specify seed on rake task plugin:spec (#17770)
Allow users to specify the seed of the tests using the env variable RSPEC_SEED

Example:

bundle exec rake "plugin:spec[plugin-name]" RSPEC_SEED=65536
This is useful while fixing flaky tests.
2022-08-02 23:52:53 -03:00
David Taylor 1121062aa9
DEV: Run prettier and rubocop in parallel for CI (#17357) 2022-07-07 13:29:14 +08:00
tshenry d1a15d4f8d
FIX: Should be UploadReference instead of UploadReferences (#17361)
This fixes a couple of typos that were introduced in 9db8f00
2022-07-06 11:40:54 -07:00
Jarek Radosz 2e4056d185
DEV: Improve multisite db scripts in dev (#17337)
## Without multisite.yml config

No change. `bin/rails db:create` / `db:migrate` / `db:drop` should work the same.

## With multisite.yml config

### db:create

`bin/rails db:create` creates development, test, and all databases from the multisite config

`RAILS_DB=[site] bin/rails db:create` creates the database for the specified site from the multisite config

### db:migrate

`bin/rails db:migrate` migrates the development database and all databases from the multisite config

`RAILS_ENV=test bin/rails db:migrate` migrates the test database and `discourse_test_multisite`

`RAILS_DB=[site] bin/rails db:migrate` migrates the database for the specified site from the multisite config

### db:drop

`bin/rails db:drop` drops development, test, and all databases from the multisite config

`RAILS_DB=[site] bin/rails db:create` drops the database for the specified site from the multisite config
2022-07-06 10:39:03 +02:00
Jarek Radosz c5f0aa2f32
DEV: Remove handlebars from the old package.json (#17319) 2022-07-04 15:05:03 +02:00
Jarek Radosz aa7792cf93
DEV: Use npm bootstrap (#17315) 2022-07-04 11:36:51 +02:00
Jarek Radosz a1170970db
FIX: Add wizard.js to ember-cli assets (#17153) 2022-06-20 17:18:35 +02:00
Jarek Radosz 2c1fc28d00
DEV: Remove ember-cli flags from the backend (#17147)
…and other auxiliary code

* Restore `QUNIT_EMBER_CLI` flag warning
* Add `ALLOW_EMBER_CLI_PROXY_BYPASS`
2022-06-20 16:33:05 +02:00
David Taylor 1b4692039e DEV: Remove legacy JS manifests and vendored scripts
Now that we've switched to Ember CLI, these things are no longer used.

- These sprockets manifests are superceded by the assets generated by ember cli
- These vendored scripts are now fetched by ember-auto-import at compile time
2022-06-20 15:01:06 +01:00
David Taylor 12980418ae
DEV: Disable the use of 'legacy' Ember assets (#17127)
Anyone still using `EMBER_CLI_PROD_ASSETS=0` in development or production will be gracefully switched to Ember CLI. In development, a repeated message will be logged to STDERR.

Similarly, passing `QUNIT_EMBER_CLI=0` to the qunit rake task will now do nothing. A warning will be printed, and ember-cli mode will be used. Note that we've chosen not to fail the task, so that existing plugin/theme CI jobs don't immediately start failing. We may switch to a hard fail in the coming days/weeks.
2022-06-17 16:51:28 +01:00
Jarek Radosz 91bc4442d1
DEV: Remove wizard step from docker tests (#17125)
Wizard tests are now in core.
2022-06-17 15:35:18 +02:00
Jarek Radosz fcb4e5a1a1
DEV: Make wizard an ember addon (#17027)
Co-authored-by: David Taylor <david@taylorhq.com>
2022-06-17 14:50:21 +02:00
Gerhard Schlager 888f50543d
DEV: Fix `javascript:update` rake task (#17098)
* The `javascript:update` rake task failed because recent versions of chart.js use a lowercase filename (`chart.min.js` instead of `Chart.min.js`)

* Changed `loadScript()` to use lowercase keys to lookup scripts

* `svg-arrow.css` seems to have changed slightly (linebreak at the end of file)
2022-06-15 19:49:04 +02:00
Gerhard Schlager 96e87af605
FIX: Rake tasks related to uploads were broken (#17085) 2022-06-14 11:05:03 +02:00
Bianca Nenciu 9db8f00b3d
FEATURE: Create upload_references table (#16146)
This table holds associations between uploads and other models. This can be used to prevent removing uploads that are still in use.

* DEV: Create upload_references
* DEV: Use UploadReference instead of PostUpload
* DEV: Use UploadReference for SiteSetting
* DEV: Use UploadReference for Badge
* DEV: Use UploadReference for Category
* DEV: Use UploadReference for CustomEmoji
* DEV: Use UploadReference for Group
* DEV: Use UploadReference for ThemeField
* DEV: Use UploadReference for ThemeSetting
* DEV: Use UploadReference for User
* DEV: Use UploadReference for UserAvatar
* DEV: Use UploadReference for UserExport
* DEV: Use UploadReference for UserProfile
* DEV: Add method to extract uploads from raw text
* DEV: Use UploadReference for Draft
* DEV: Use UploadReference for ReviewableQueuedPost
* DEV: Use UploadReference for UserProfile's bio_raw
* DEV: Do not copy user uploads to upload references
* DEV: Copy post uploads again after deploy
* DEV: Use created_at and updated_at from uploads table
* FIX: Check if upload site setting is empty
* DEV: Copy user uploads to upload references
* DEV: Make upload extraction less strict
2022-06-09 09:24:30 +10:00