Commit Graph

1149 Commits

Author SHA1 Message Date
David Taylor 127ba698a7
DEV: Allow running theme-qunit tests via testem (#16540)
This allows `QUNIT_EMBER_CLI=1 bin/rake theme:qunit[...]` to test themes using `testem` with Ember-CLI-generated assets
2022-04-22 15:04:01 +01:00
Jarek Radosz 1209d51a7f
DEV: Don't use `Dir.chdir()` (#16535)
`system()` provides `chdir:` option for that.
2022-04-22 11:17:10 +02:00
David Taylor 22a7905f2d
DEV: Allow Ember CLI assets to be used by development Rails app (#16511)
Previously, accessing the Rails app directly in development mode would give you assets from our 'legacy' Ember asset pipeline. The only way to run with Ember CLI assets was to run ember-cli as a proxy. This was quite limiting when working on things which are bypassed when using the ember-cli proxy (e.g. changes to `application.html.erb`). Also, since `ember-auto-import` introduced chunking, visiting `/theme-qunit` under Ember CLI was failing to include all necessary chunks.

This commit teaches Sprockets about our Ember CLI assets so that they can be used in development mode, and are automatically collected up under `/public/assets` during `assets:precompile`. As a bonus, this allows us to remove all the custom manifest modification from `assets:precompile`.

The key changes are:
- Introduce a shared `EmberCli.enabled?` helper
- When ember-cli is enabled, add ember-cli `/dist/assets` as the top-priority Rails asset directory
- Have ember-cli output a `chunks.json` manifest, and teach `preload_script` to read it and append the correct chunks to their associated `afterFile`
- Remove most custom ember-cli logic from the `assets:precompile` step. Instead, rely on Rails to take care of pulling the 'precompiled' assets into the `public/assets` directory. Move the 'renaming' logic to runtime, so it can be used in development mode as well.
- Remove fingerprinting from `ember-cli-build`, and allow Rails to take care of things

Long-term, we may want to replace Sprockets with the lighter-weight Propshaft. The changes made in this commit have been made with that long-term goal in mind.

tldr: when you visit the rails app directly, you'll now be served the current ember-cli assets. To keep these up-to-date make sure either `ember serve`, or `ember build --watch` is running. If you really want to load the old non-ember-cli assets, then you should start the server with `EMBER_CLI_PROD_ASSETS=0`. (the legacy asset pipeline will be removed very soon)
2022-04-21 16:26:34 +01:00
Jarek Radosz 69dbc207ab
DEV: Improve `plugin:versions` task (#16391)
1. Sort plugins by name
2. Include plugins that are a symbolic link to a submodule repo (in those cases `.git` isn't a directory but a file that looks like e.g. `gitdir: ../../.git/modules/plugins/name-here`)
2022-04-06 18:29:39 +02:00
Jarek Radosz 9060a045c2
DEV: Minor linting fixes (#16359)
1. `test/run-qunit.js` wasn't eslinted (I'm not adding it to the CI workflow for now, just fixed the issues)
2. "…" utf character isn't rendered correctly in Jenkins, replaced with three dots
3. Don't try to lint `tmp` when doing `eslint .` in the root dir
2022-04-02 21:15:38 +02:00
David Taylor 9df28fe4b3
DEV: Only raise `rake themes:update` errors when flag provided (#16254)
Switching behavior based on multisite/single-site configuration can create some difficult-to-debug situations. The flag is much more obvious.
2022-03-22 17:02:14 +00:00
Jarek Radosz e4a92a8113
DEV: Add `SKIP_DB_CREATE` option to docker:test (#16225)
So that the database creation can be run separately from the specs.
2022-03-21 13:21:06 +01:00
Jarek Radosz bf8dc394bd
DEV: Add chromium to ChromeInstalledChecker (#16224) 2022-03-19 11:00:06 +01:00
David Taylor c58c2f53e8
FIX: Skip ember-cli build memory optimisation for multiple rake tasks (#16198) 2022-03-16 12:02:21 +00:00
Jarek Radosz dec68d780c
DEV: Don't install plugin gems twice (#16192)
Missing plugin gems are installed when the app is being loaded.

That means when you run `bin/rails plugin:install_all_gems` it first installs missing gems and then reinstalls all gems…

Also, the method these rake tasks were using to install gems was very crude, and the regex there was incorrect which resulted in failures in certain cases. Though that didn't matter since those gems were being installed using a correct method just moments before…
2022-03-15 15:08:05 +01:00
David Taylor ea1865cc01
DEV: Ensure yarn dependencies are installed during docker.rake (#16185)
Followup to 349a4acc9f
2022-03-14 17:53:21 +00:00
David Taylor 349a4acc9f
DEV: Use Ember CLI for docker.rake CI testing (#16182) 2022-03-14 17:32:03 +00:00
David Taylor 21cc57674c
FIX: Do not run Ember CLI chunk files through babel twice (#16183) 2022-03-14 16:11:13 +00:00
Jarek Radosz 8d5281115e
DEV: Use the --no-rebase option of `git pull` (#16180)
…to avoid repeatedly printed notes:

```
hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint:
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint:
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
```
2022-03-14 15:13:06 +01:00
Jarek Radosz 3db4b3bc5e
DEV: Drop lodash (#16110) 2022-03-06 18:15:25 +01:00
Jarek Radosz f9e1ba4f8f
DEV: Remove remnants of nginx-perf-report plugin (#16107)
The plugin is no longer official as of Nov 18, 2019 (e2ccb0c608)
2022-03-05 17:22:23 +01:00
David Taylor 9415ec91a1
DEV: Make load order consistent for `rake plugin:spec` (#16076)
`Dir.glob` doesn't guarantee any particular order for results. However, it does appear to be consistent on a given machine. This means that specs can consistently pass on one machine while consistently failing on another. This can lead to some very confusing situations!

This commit sorts the spec files alphabetically so that load order is consistent across environments.

Note that the order in which tests are **run** is not affected by this change. Run order is still randomized by RSpec
2022-03-01 17:46:47 +00:00
Jarek Radosz 2fc70c5572
DEV: Correctly tag heredocs (#16061)
This allows text editors to use correct syntax coloring for the heredoc sections.

Heredoc tag names we use:

languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS
other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 20:50:55 +01:00
Jarek Radosz 29e601c13a
DEV: Fix the javascript:update rake task (#16057)
The source of the moment tz name package has changed in #15761
2022-02-26 02:06:12 +01:00
Dan Ungureanu ae5eab06ad
FEATURE: Add ENV to control themes:update errors (#16051)
This commit introduces a new environment variable `RAISE_THEME_ERRORS`
that can control what happens when `theme:update` Rake task errors. It
can have three possible values: `0` to always print errors, `1` to
always raise on error, or be absent to use the default behavior which
raises errors only for default sites.
2022-02-25 11:54:42 +02:00
Joffrey JAFFEUX 8244b4b163
DEV: prevents warnings with EMBER_CLI being redefined (#16035)
Example error:

```
/__w/discourse/discourse/lib/tasks/assets.rake:3: warning: already initialized constant EMBER_CLI
/__w/discourse/discourse/lib/tasks/assets.rake:3: warning: previous definition of EMBER_CLI was here
```
2022-02-23 10:51:31 +01:00
David Taylor 6ab4d26d84
PERF: Terminate `rake` process during `ember build` (#15920)
The `assets:precompile` rake task loads the full Ruby app, which can consume around 500mb of RAM by itself. Using `exec` to run `ember build` allows us to free up the Ruby memory and make more space for `ember build`
2022-02-14 11:49:46 +00:00
David Taylor a01b1dd648
PERF: Update ember-auto-import and webpack (#15919)
This makes a small improvement to 'cold cache' ember-cli build times, and a large improvement to 'warm cache' build times

The ember-auto-import update means that vendor is now split into multiple files for efficiency. These are named `chunk.*`, and should be included immediately after the `vendor.js` file. This commit also updates the rails app to render script tags for these chunks.

This change was previously merged, and caused memory-related errors on RAM-constrained machines. This was because Webpack 5 switches from multiple worker processes to a single multi-threaded process. This meant that it was hitting node's default heap size limit (~500mb on a 1GB RAM server). Discourse's standard install procedure recommends adding 2GB swap to 1GB-RAM machines, so we can afford to override's Node's default via the `--max-old-space-size` flag.
2022-02-14 11:21:39 +00:00
David Taylor 8025768fd2
DEV: Embed source content in JS sourcemaps (#15914)
This removes the need for a further request for the 'original' source file, and brings our custom terser arguments in line with the ember-cli defaults
2022-02-11 20:38:53 +00:00
David Taylor 51b88c0e2c
DEV: Ensure Ember CLI sourcemaps are uploaded to S3 (#15893)
Ember CLI gives sourcemaps their own digest. Our `s3.rake` logic assumes that the digest portion of sourcemap filenames remains the same.

The Ember CLI sourcemaps are included in the manifest file, so we can ensure they are uploaded by letting them past the MiniMime check.

Followup to abefb1beff
2022-02-10 19:00:47 +00:00
David Taylor fcae4a8faf
DEV: Do not include CDN domain in the sourceMappingURL (#15887)
If no path is supplied, browsers will look for the map on the same path as the JS file itself. This fixes two problems that we see in production:

1. When compiling assets against one CDN, and then re-using them on a site with a different CDN, the sourceMappingUrls would be incorrect and print warnings in the console

2. If both an S3 CDN and an app CDN are configured, we were using the S3 CDN for the JS and the app CDN for the map. This commit will make sure we use the S3 CDN for both.
2022-02-10 15:37:44 +00:00
David Taylor abefb1beff
DEV: Do not run ember-cli output through terser again (#15889)
ember-cli already runs terser on its output. Running it through again provides no benefit, takes longer, and also breaks source mapping of these assets in production.
2022-02-10 15:35:54 +00:00
David Taylor ecc07fd8dc
DEV: Make Ember CLI assets the default in production (#15861)
This was reverted in e92f57255d due to memory usage concerns. This memory issue was resolved by 4cceb55621.
2022-02-08 10:03:53 +00:00
David Taylor e92f57255d
Revert "DEV: Make Ember CLI assets the default in production (#15843)" (#15852)
This reverts 1b622667bc

We have had reports of issues rebuilding under memory-constrained environments. Reverting while we investigate further.
2022-02-07 20:31:10 +00:00
David Taylor 1b622667bc
DEV: Make Ember CLI assets the default in production (#15843)
This can be disabled by setting `EMBER_CLI_PROD_ASSETS=0`, but this option will not be available for long. If your theme/plugin/site has issues under Ember CLI, please open a topic on https://meta.discourse.org
2022-02-07 15:25:57 +00:00
David Taylor c3a54eb30f DEV: Use strings for :user_email job type argument
Job arguments go via JSON, and so symbols will appear as strings in the Job's `#execute` method. The latest version of Sidekiq has started warning about this to reduce developer confusion.
2022-02-04 18:28:18 +00:00
Rafael dos Santos Silva 3f694e4ab5
FEATURE: Use native color-picker (#15748) 2022-02-01 11:18:13 -03:00
David Taylor 6c3df84a93
DEV: In themes:update, only update themes which are out-of-date (#15676)
Running `update_from_remote` and `save!` cause a number of side-effects, including instructing all clients to reload CSS files. If there are no changes, then this is wasteful, and can even cause a 'flicker' effect on clients as they reload CSS.

This commit checks if any updates are available before triggering `update_from_remote` / `save!`. This should be much faster, and stop the 'flickering' UX from happening on every themes:update run.

It also improves the output of the command to include the from/to commit hashes, which may be useful for debugging issues. For example:

```
Checking 'Alien Night | A Dark Discourse Theme' for 'default'... already up to date
Checking 'Star Wars' for 'default'... updating from d8a170dd to 66b9756f
Checking 'Media Overlay' for 'default'... already up to date
```
2022-01-21 18:23:26 +00:00
Dan Ungureanu f0c1a4fab8
FIX: Create email token with correct scope (#15658)
`account_created` email contains a URL to `/u/password-reset/TOKEN`
which means that the correct scope for the email token is
`password_reset`, not `signup`.
2022-01-20 16:38:56 +02:00
Gerhard Schlager 27f1630b01
DEV: Try to download missing uploads from origin URL (#15629) 2022-01-19 11:05:58 +01:00
David Taylor 5c23c6cdab DEV: Update `rake qunit:test` and `rake plugin:qunit` to use `testem`
For now this is still gated behind a `QUNIT_EMBER_CLI=1` environment variable, but will eventually become the default so that we can remove `run-qunit.js`.
2022-01-18 10:16:29 +00:00
David Taylor ed2f700440
DEV: Wait for initdb to complete in docker.rake (#15614)
On slower hardware it can take a while to init the database. If we don't wait, the `rake db:create` step will fail.
2022-01-17 17:45:39 +00:00
Rafael dos Santos Silva 8cfd791e0f
FEATURE: Add missing emojis (#15582)
* FEATURE: Add missing emojis

* More missing emojis
2022-01-14 17:51:13 -03:00
Robin Ward 6272edd121 DEV: Support for running theme test with Ember CLI (third attempt)
The second attempt fixed issues with smoke test.

This one makes sure minification only happens in production mode.
2022-01-13 16:02:07 -05:00
Martin Brennan 107239a442
Revert "DEV: Support for running theme test with Ember CLI (second attempt)" (#15559)
This reverts commit 2c7906999a.

The changes break some things in local development (putting JS files
into minified files, not allowing debugger, and others)
2022-01-13 10:05:35 +10:00
Robin Ward 2c7906999a DEV: Support for running theme test with Ember CLI (second attempt)
This PR includes support for running theme tests in legacy ember
production envrionments.
2022-01-12 15:43:29 -05:00
David Taylor 252bb87ab3
Revert "DEV: Support for running theme test with Ember CLI" (#15547)
This reverts commit ea84a82f77.

This is causing problems with `/theme-qunit` on legacy, non-ember-cli production sites. Reverting while we work on a fix
2022-01-11 23:38:59 +00:00
Robin Ward ea84a82f77 DEV: Support for running theme test with Ember CLI
This is quite complex as it means that in production we have to build
Ember CLI test files and allow them to be used by our Rails application.

There is a fair bit of glue we can remove in the future once we move to
Ember CLI completely.
2022-01-11 15:42:13 -05:00
David Taylor b3e52f99e6
FIX: LOAD_PLUGINS=0 in dev/prod, warn in plugin:pull_compatible_all (#15537)
The `plugin:pull_compatible_all` task is intended to take incompatible plugins and downgrade them to an earlier version. Problem is, when running the rake task in development/production environments, the plugins have already been activated. If an incompatible plugin raises an error in `plugin.rb` then the rake task will be unable to start.

This commit centralises our LOAD_PLUGINS detection, adds support for LOAD_PLUGINS=0 in dev/prod, and adds a warning to `plugin:pull_compatible_all` if it's run with plugins enabled.
2022-01-11 12:30:22 +00:00
Rafael dos Santos Silva 06bba76719
DEV: Update emojis constants (#15506)
Follow up to d1cfabd so the new emojis are usable by the app
2022-01-10 14:53:52 -03:00
Jarek Radosz 5a50f18c0c
DEV: Avoid `$` globals (#15453)
Also:
* Remove an unused method (#fill_email)
* Replace a method that was used just once (#generate_username) with `SecureRandom.alphanumeric`
* Remove an obsolete dev puma `tmp/restart` file logic
2022-01-08 23:39:46 +01:00
Jarek Radosz 793c3ae7d4
FIX: Actually save fields on themes:update (#15497)
`RemoteTheme#update_from_remote` does not save theme fields on its own.

Fixes theme/theme component autoupdates working only partially.
2022-01-08 05:41:20 +01:00
Rafael dos Santos Silva 0a0e06fcf8
FEATURE: Update emojis (#15468)
* FEATURE: Update emojis

* Add extra aliases for backwards compatibility
2022-01-07 15:49:56 -03:00
Peter Zhu c5fd8c42db
DEV: Fix methods removed in Ruby 3.2 (#15459)
* File.exists? is deprecated and removed in Ruby 3.2 in favor of
File.exist?
* Dir.exists? is deprecated and removed in Ruby 3.2 in favor of
Dir.exist?
2022-01-05 18:45:08 +01:00
Martin Brennan 099b679fc5
DEV: Clean up old bookmark code (#15455)
The rake task deleted here was added back in Feb 2020
when bookmarks were first converted from PostAction
records, it is no longer needed. The ignored columns
were removed in ed83d7573e.
2022-01-05 10:02:02 +10:00