Commit Graph

9721 Commits

Author SHA1 Message Date
Krzysztof Kotlarek 5055e431a8
DEV: Custom generator for rename site setting migration (#24841)
Ability to automatically generate migration when site setting name is changed.
Example usage: `rails generate site_setting_rename_migration site_description contact_email`
2023-12-13 09:58:45 +11:00
Rafael dos Santos Silva de936f07e5
PERF: Update node_options during ember build for low end servers (#24850)
See https://meta.discourse.org/t/286643/14

Co-authored-by: David Taylor <david@taylorhq.com>
2023-12-12 17:05:20 +00:00
Angus McLeod 95c61b88dc
Apply embed unlisted setting consistently (#24294)
Applies the embed_unlisted site setting consistently across topic embeds, including those created via the WP Discourse plugin. Relatedly, adds a embed exception to can_create_unlisted_topic? check. Users creating embedded topics are not always staff.
2023-12-12 09:35:26 -05:00
Krzysztof Kotlarek 7dd150bc95
DEV: Convert min_trust_to_edit_wiki_post to groups (#24766)
This change converts the min_trust_to_edit_wiki_post site setting to edit_wiki_post_allowed_groups.

See: https://meta.discourse.org/t/283408

Hides the old setting
Adds the new site setting
Add a deprecation warning
Updates to use the new setting
Adds a migration to fill in the new setting if the old setting was changed
Adds an entry to the site_setting.keywords section
Updates tests to account for the new change
After a couple of months, we will remove the email_in_min_trust setting entirely.

Internal ref: /t/117248
2023-12-12 15:20:37 +11:00
Gerhard Schlager dc8c6b8958 DEV: Lots of improvements to the generic_bulk import script
Notable changes:
* Imports a lot more tables from core and plugins
  * site settings
  * uploads with necessary upload references
  * groups and group members
  * user profiles
  * user options
  * user fields & values
  * muted users
  * user notes (plugin)
  * user followers (plugin)
  * user avatars
  * tag groups and tags
  * tag users (notification settings for tags / user)
  * category permissions
  * polls with options and votes
  * post votes (plugin)
  * solutions (plugin)
  * gamification scores (plugin)
  * events (plugin)
  * badges and badge groupings
  * user badges
  * optimized images
  * topic users (notification settings for topics)
  * post custom fields
  * permalinks and permalink normalizations

* It creates the `migration_mappings` table which is used to store the mapping for a handful of imported tables

* Detects duplicate group names and renames them

* Pre-cooking for attachments, images and mentions

* Outputs instructions when gems are missing

* Supports importing uploads from a DB generated by `uploads_importer.rb`

* Checks that all required plugins exists and enables them if needed

* A couple of optimizations and additions in `import.rake`
2023-12-11 16:23:07 +01:00
Penar Musaraj 28956a5415
DEV: Switch to sassc-embedded (#24639)
Sassc-embedded fixes a performance issue with a leaking DartSass process. And it also fixes an issue with source map file paths (without any extra flags).
2023-12-08 10:34:03 -05:00
David Taylor e4c373194d
DEV: Refactor Wizard components (#24770)
This commit refactors the Wizard component code in preparation for moving it to the 'static' directory for Embroider route-splitting. It also includes a number of general improvements and simplifications.

Extracted from https://github.com/discourse/discourse/pull/23678

Co-authored-by: Godfrey Chan <godfreykfc@gmail.com>
2023-12-07 16:33:38 +00:00
Martin Brennan 7afb5fc481
DEV: Use Discourse::SYSTEM_USER_ID in fixtures/009_users (#24743)
I couldn't find where we created the system user and
this is why -- everywhere else in the app we reference
SYSTEM_USER_ID but here.
2023-12-07 09:04:45 +10:00
Jarek Radosz 694b5f108b
DEV: Fix various rubocop lints (#24749)
These (21 + 3 from previous PRs) are soon to be enabled in rubocop-discourse:

Capybara/VisibilityMatcher
Lint/DeprecatedOpenSSLConstant
Lint/DisjunctiveAssignmentInConstructor
Lint/EmptyConditionalBody
Lint/EmptyEnsure
Lint/LiteralInInterpolation
Lint/NonLocalExitFromIterator
Lint/ParenthesesAsGroupedExpression
Lint/RedundantCopDisableDirective
Lint/RedundantRequireStatement
Lint/RedundantSafeNavigation
Lint/RedundantStringCoercion
Lint/RedundantWithIndex
Lint/RedundantWithObject
Lint/SafeNavigationChain
Lint/SafeNavigationConsistency
Lint/SelfAssignment
Lint/UnreachableCode
Lint/UselessMethodDefinition
Lint/Void

Previous PRs:
Lint/ShadowedArgument
Lint/DuplicateMethods
Lint/BooleanSymbol
RSpec/SpecFilePathSuffix
2023-12-06 23:25:00 +01:00
David Taylor 48ec946702
UX: Restore category badge colours on 404 page (#24754) 2023-12-06 17:49:19 +00:00
Jarek Radosz 6a66dc1cfb
DEV: Fix Lint/BooleanSymbol (#24747) 2023-12-06 13:19:09 +01:00
Jarek Radosz 138bf486d3
DEV: Fix Lint/DuplicateMethods (#24746) 2023-12-06 13:18:34 +01:00
Jarek Radosz 4280c01153
DEV: Fix Lint/ShadowedArgument (#24733) 2023-12-06 13:16:10 +01:00
David Battersby 8b46dc8bb5
FEATURE: Add thumbnails for chat image uploads (#24328)
Introduces the concept of image thumbnails in chat, prior to this we uploaded and used full size chat images within channels and direct messages.

The following changes are covered:
- Post processing of image uploads to create the thumbnail within Chat::MessageProcessor
- Extract responsive image ratios into CookedProcessorMixin (used for creating upload variations)
- Add thumbnail to upload serializer from plugin.rb
- Convert chat upload template to glimmer component using .gjs format
- Use thumbnail image within chat upload component (stores full size img in orig-src data attribute)
- Old uploads which don't have thumbnails will fallback to full size images in channels/DMs
- Update Magnific lightbox to use full size image when clicked
- Update Glimmer lightbox to use full size image (enables zooming for chat images)
2023-12-06 14:59:18 +08:00
Martin Brennan 30d5e752d7
DEV: Revert guardian changes (#24742)
I took the wrong approach here, need to rethink.

* Revert "FIX: Use Guardian.basic_user instead of new (anon) (#24705)"

This reverts commit 9057272ee2.

* Revert "DEV: Remove unnecessary method_missing from GuardianUser (#24735)"

This reverts commit a5d4bf6dd2.

* Revert "DEV: Improve Guardian devex (#24706)"

This reverts commit 77b6a038ba.

* Revert "FIX: Introduce Guardian::BasicUser for oneboxing checks (#24681)"

This reverts commit de983796e1.
2023-12-06 16:37:32 +10:00
Martin Brennan 9057272ee2
FIX: Use Guardian.basic_user instead of new (anon) (#24705)
c.f. de983796e1

There will soon be additional login_required checks
for Guardian, and the intent of many checks by automated
systems is better fulfilled by using BasicUser, which
simulates a logged in TL0 forum user, rather than an
anon user.

In some cases the use of anon still makes sense (e.g.
anonymous_cache), and in that case the more explicit
`Guardian.anon_user` is used
2023-12-06 11:56:21 +10:00
Martin Brennan a5d4bf6dd2
DEV: Remove unnecessary method_missing from GuardianUser (#24735)
Followup to 77b6a038ba, this
was a mistake and should have been removed before merge.
2023-12-06 11:24:19 +10:00
Martin Brennan 77b6a038ba
DEV: Improve Guardian devex (#24706)
It's quite confusing for blank? to be overridden
on AnonymousUser and BasicUser to represent
whether the fake user is authenticated or not;
we can achieve the same thing more clearly with
a wrapper GuardianUser class around these
user classes. Also fixes an issue where
`def user` would be returning nil.
2023-12-06 10:57:04 +10:00
Martin Brennan de983796e1
FIX: Introduce Guardian::BasicUser for oneboxing checks (#24681)
Through internal discussion, it has become clear that
we need a conceptual Guardian user that bridges the
gap between anon users and a logged in forum user with
an absolute baseline level of access to public topics,
which can be used in cases where:

1. Automated systems are running which shouldn't see any
   private data
1. A baseline level of user access is needed

In this case we are fixing the latter; when oneboxing a local
topic, and we are linking to a topic in another category from
the current one, we need to operate off a baseline level of
access, since not all users have access to the same categories,
and we don't want e.g. editing a post with an internal link to
expose sensitive internal information.
2023-12-05 09:25:23 +10:00
David Taylor 3aeff56faf
DEV: Run assets:precompile ember build with `CI=1` (#24696)
This will improve the output to print the current step (rather than the existing behavior which just says 'building...')
2023-12-04 15:49:50 +00:00
Jarek Radosz 7196613e2e
DEV: Fix various spec linting issues (#24672)
Duplicated specs, incorrect descriptions, incorrect assertions, incorrect filenames, old todo
2023-12-04 13:45:19 +01:00
David Taylor c2887d3f8c
DEV: Remove unused Ember::Handlebars freedom patch (#24688)
We no longer compile Ember templates in ruby, so this is unused
2023-12-04 12:40:08 +00:00
Jarek Radosz 74011232e9
FIX: Request html when fetching inline onebox data (#24674)
We do expect to receive html
2023-12-04 11:36:42 +10:00
David Taylor ecf7a4f0c6
FIX: Ensure app-cdn CORS is not overridden by cors_origin setting (#24661)
We add `Access-Control-Allow-Origin: *` to all asset requests which are requested via a configured CDN. This is particularly important now that we're using browser-native `import()` to load the highlightjs bundle. Unfortunately, user-configurable 'cors_origins' site setting was overriding the wldcard value on CDN assets and causing CORS errors.

This commit updates the logic to give the `*` value precedence, and adds a spec for the situation. It also invalidates the cache of hljs assets (because CDNs will have cached the bad Access-Control-Allow-Origin header).

The rack-cors middleware is also slightly tweaked so that it is always inserted. This makes things easier to test and more consistent.
2023-12-01 12:57:11 +00:00
Ted Johansson 54e813e964
FIX: Don't error out when trying to retrieve title and URL won't encode (#24660) 2023-12-01 15:03:06 +08:00
Martin Brennan c58cd697d2
FIX: Further improvements for plugin list (#24622)
Followup e37fb3042d

* Automatically remove the prefix `Discourse ` from all the plugin titles to avoid repetition
* Remove the :discourse_dev: icon from the author. Consider a "By Discourse" with no labels as official
* We add a `label` metadata to plugin.rb
  * Only plugins made by us in `discourse` and `discourse-org` GitHub organizations will show these in the list
* Make the plugin author font size a little smaller
* Make the commit sha look like a link so it's more obvious it goes to the code

Also I added some validation and truncation for plugin metadata
parsing since currently you can put absolutely anything in there
and it will show on the plugin list.
2023-11-30 10:53:17 +10:00
Daniel Waterworth eef93ac926
DEV: Allow setting max_length for field types using the plugin API (#24635) 2023-11-29 14:17:12 -06:00
David Taylor 265a8cd2b2
DEV: Fixup plugin initialization guard (#24628)
In development, I sometimes get `nil` `location.absolute_path` values. It looks like this is sometimes expected (https://bugs.ruby-lang.org/issues/10561) so we should fallback to `.path` and add a nil check.
2023-11-29 11:44:09 +00:00
Blake Erickson 21d614215b
DEV: Use staged user check instead (#24578)
This change refactors the check `user.groups.any?` and instead uses
`user.staged?` to check if the user is staged or not.

Also fixes several tests to ensure the users have their auto trust level
groups created.

Follow up to:

- 8a45f84277
- 447d9b2105
- c89edd9e86
2023-11-28 07:34:02 -07:00
David Taylor 5783f231f8
DEV: Introduce `DISCOURSE_ASSET_URL_SALT` (#24596)
This value is included when generating static asset URLs. Updating the value will allow site operators to invalidate all asset urls to recover from configuration issues which may have been cached by CDNs/browsers.
2023-11-28 11:28:40 +00:00
David Taylor 16b6e86932 DEV: Introduce feature-flag for Ember 5 upgrade
This commit introduces the scaffolding for us to easily switch between Ember 3.28 and Ember 5 on the `main` branch of Discourse. Unfortunately, there is no built-in system to apply this kind of flagging within yarn / ember-cli. There are projects like `ember-try` which are designed for running against multiple version of a dependency, but they do not allow us to 'lock' dependency/sub-dependency versions, and are therefore unsuitable for our use in production.

Instead, we will be maintaining two root `package.json` files, and two `yarn.lock` files. For ember-3, they remain as-is. For ember5, we use a yarn 'resolution' to override the version for ember-source across the entire yarn workspace.

To allow for easy switching with minimal diff against the repository, `package.json` and `yarn.lock` are symlinks which point to `package-ember3.json` and `yarn-ember3.lock` by default. To switch to Ember 5, we can run `script/switch ember version 5` to update the symlinks to point to `package-ember5.json` and `package-ember3.json` respectively. In production, and when using `bin/ember-cli` for development, the ember version can also be upgraded using the `EMBER_VERSION=5` environment variable.

When making changes to dependencies, these should be made against the default `ember3` versions, and then `script/regen_ember_5_lockfile` should be used to regenerate `yarn-ember5.lock` accordingly. A new 'Ember Version Lockfiles' GitHub workflow will automate this process on Dependabot PRs.

When running a local environment against Ember 5, the two symlink changes will show up as git diffs. To avoid us accidentally committing/pushing that change, another GitHub workflow is introduced which checks the default Ember version and raises an error if it is greater than v3.

Supporting two ember versions simultaneously obviously carries significant overhead, so our aim will be to get themes/plugins updated as quickly as possible, and then drop this flag.
2023-11-27 16:40:22 +00:00
Martin Brennan 09c446c1ae
FIX: Handle missing git repo details in plugin list (#24539)
Followup to e37fb3042d,
in some cases we cannot get git information for the
plugin folder (e.g. permission issues), so we need
to only try and get information about it if
commit_hash is present.
2023-11-24 12:08:10 +10:00
Sam c2fd090d7d
DEV: revert missing license for maxmind changes (#24538)
Reverts
 
 - DEV: maxmind license checking failing tests #24534 
 - UX: Show if MaxMind key is missing on IP lookup #18993

These changes are leading to surprising results, our logs are now filling up with warnings on dev environments 

We need the change to be redone
2023-11-24 11:31:11 +11:00
Sam 85d2b5fa48
DEV: maxmind license checking failing tests (#24534)
This improves the implementation of #18993

1. Error message displayed to user is clearer
2. open_db will also be called, even if license key is blank, as it was previously
3. This in turn means no need to keep stubbing 'maxmind_license_key'
2023-11-24 09:38:46 +11:00
MichaIng c58a41cb3e
UX: Show on IP lookup if MaxMind key is missing (#18993)
as discussed in https://meta.discourse.org/t/maxminddb-not-found-error/148512/7.
 
shows a warning to the admin if no license for maxmind is found
2023-11-24 08:02:05 +11:00
Alan Guo Xiang Tan aaadce0652
Revert "FIX: `plugin:install_all_gems` Rake task not installing plugin gem (#24522)" (#24524)
This breaks the `plugin:install_all_gems` Rake task when used before
Redis is running. Need to go back to the drawing board.

This reverts commit 189aa5fa4e.
2023-11-23 13:01:54 +08:00
Blake Erickson c89edd9e86
DEV: Convert email_in_min_trust to groups (#24515)
This change converts the `email_in_min_trust` site setting to
`email_in_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change

After a couple of months we will remove the
`email_in_min_trust` setting entirely.

Internal ref: /t/115696
2023-11-22 18:03:28 -07:00
Alan Guo Xiang Tan 189aa5fa4e
FIX: `plugin:install_all_gems` Rake task not installing plugin gem (#24522)
Why this change?

This regressed in dec68d780c where the
commit assumes that plugin gems are always installed when the
`plugin:install_all_gems` Rake task is ran as it would run the our Rails
initializers which activates plugins and install the gems. However, this
assumption only holds true when the `LOAD_PLUGINS` is present and set to
`1`.

What does this change do?

This commit changes the `plugin:install_all_gems` to load the Rails
environment with `LOAD_PLUGINS` set to `1` such that the plugin gems
will be installed as part of our initialization process for the app.

The commit also removes the `plugin:install_gems` Rake task which is
currently a noop and does not seem to be used anywhere..
2023-11-23 08:29:51 +08:00
Gerhard Schlager 5b97f79569
DEV: Replace `starts_with?` with `start_with?` in `HtmlToMarkdown` (#24521)
This allows us to use that class without loading Rails, e.g. in imports (converters).
2023-11-23 00:57:24 +01:00
Gerhard Schlager 9f3399ca8f
FIX: Remove multiple consecutive dashes from tag names (#24520) 2023-11-23 00:57:12 +01:00
Alan Guo Xiang Tan 1e290eed7b
PERF: Switch `plugins:update_all` to git pull concurrently (#24513)
Why this change?

Similar to d0117ff6e3, `plugins:update_all` spends most of its time waiting
on the network. On my local machine, this takes up to 2 mins when I have
all the official plugins installed. On a 32 cores machine, the total
time is cut down to 4 seconds.

What does this change do?

1. Move the logic in the `plugin:update` Rake task into a method.
2. Updates the `plugin:update` and `plugin:update_all` to rely on the
   new method.
3. Wraps the method call to update a plugin in `plugin:update_all` in a
   `Concurrent::Promise`

This change also adds the `--quiet` option to the `git pull` option
since the `git pull` output is just noise for 99% of the time.
2023-11-23 07:08:32 +08:00
Martin Brennan e395e5e002
FIX: Use plugin category name for plugin list (#24477)
Followup to e37fb3042d

Some plugins like discourse-ai and discourse-saml do not
nicely change from kebab-case to Title Case (e.g. Ai, Saml),
and anyway this method of getting the plugin name is not
translated either.

Better to use the plugin setting category if it exists,
since that is written by a human and is translated.
2023-11-23 08:40:55 +10:00
Blake Erickson 8a45f84277
DEV: Convert approve_new_topics_unless_trust_level to groups (#24504)
* DEV: Convert approve_new_topics_unless_trust_level to groups

This change converts the `approve_new_topics_unless_trust_level` site
setting to `approve_new_topics_unless_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change

After a couple of months we will remove the
`approve_new_topics_unless_trust_level` setting entirely.

Internal ref: /t/115696

* add missing translation

* Add keyword entry

* Add migration
2023-11-22 10:44:59 -07:00
Alan Guo Xiang Tan d0117ff6e3
PERF: Switch `plugin:install_all_official` to clone plugins concurrently (#24511)
Why this change?

`plugin:install_all_official` is quite slow at the moment taking roughly
1 minute and 51 seconds on my machine. Since most of the time is spent
waiting on the network, we can actually speed up the Rake task
significantly by executing the cloning concurrently. With a 8 cores
machine, cloning all plugins will only take 15 seconds.

What does this change do?

This change wraps the `git clone` operation in the
`plugin:install_all_official` Rake task in a `Concurrent::Promise` which
basically runs the `git clone` operation in a Thread. The `--quiet`
option has also been added to `git clone` since running stuff
concurrently messes up the output. That could be fixed but it has been
determined to be not worth it since the output from `git clone` is
meaningless to us.
2023-11-22 21:43:42 +08:00
Alan Guo Xiang Tan 48f3c638cc
DEV: Support customizing `docker:test:setup` with envs (#24508)
Why this change?

There are instances where we would like to customize what the
`docker:test:setup` Rake task does.

What does this change do?

Adds a bunch of env variables that could be set to customize what the
`docker:test:setup` Rake test does.
2023-11-22 15:12:02 +08:00
Martin Brennan 4e7929abb6
FIX: Wrong argument error being thrown in UrlHelper (#24506)
We were throwing ArgumentError in UrlHelper.normalised_encode,
but it was incorrect -- we were passing ArgumentError.new
2 arguments which is not supported. Fix this and have a hint
of which URL is causing the issue for debugging.
2023-11-22 15:19:40 +10:00
Alan Guo Xiang Tan 86da47f58d
FIX: `docker:test` Rake task did not run system tests in parallel (#24507)
Why this change?

We support a `USE_TURBO` environment variable which tells the
`docker:test` rake task to run rspec tests in parallel. However, this
currently does not apply to system tests.

What does this change do?

This commit runs system specs for both core and plugins using
`./bin/turbo_rspec` when the `USE_TURBO` environment is present. Note
that when running system specs, we will only spawn X number of test
processes where X is half the number of available CPU cores. This is
done because we have to leave CPU resources for the chrome processes
that will be created.
2023-11-22 11:51:43 +08:00
Blake Erickson 447d9b2105
DEV: Convert approve_unless_trust_level to groups (#24357)
This change converts the `approve_unless_trust_level` site setting to
`approve_unless_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Adds the new site setting
- Adds a deprecation warning
- Updates core to use the new settings.
- Adds a migration to fill in the new setting of the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates many tests to account for the new change

After a couple of months we will remove the `approve_unless_trust_level`
setting entirely.

Internal ref: /t/115696
2023-11-21 11:31:42 -07:00
Roman Rizzi 75e2c6b506
DEV: Pass the user who requested the summary to the strategy. (#24489)
This change allows the `discourse-ai` plugin to log the user who requested the summary in the `AiApiAuditLog`.
2023-11-21 13:27:27 -03:00
Martin Brennan e37fb3042d
FEATURE: Admin plugin list redesign (#24363)
* Remove checkmark for official plugins
* Add author for plugin, which is By Discourse for all discourse
  and discourse-org github plugins
* Link to meta topic instead of github repo
* Add experimental flag for plugin metadata and show this as a
  badge on the plugin list if present

---------

Co-authored-by: chapoi <101828855+chapoi@users.noreply.github.com>
2023-11-21 09:37:11 +10:00