Commit Graph

10262 Commits

Author SHA1 Message Date
Mark VanLandingham 882c2da29e
DEV: Add modifier to SuggestedTopicsBuilder#add_results (#29164) 2024-10-10 10:03:26 -05:00
Kelv 32e261ef73
DEV: Migrate user passwords data to UserPassword table (#28746)
* Add migrations to ensure password hash is synced across users & user_passwords

* Persist password-related data in user_passwords instead of users

* Merge User#expire_old_email_tokens with User#expire_tokens_if_password_changed

* Add post deploy migration to mark password-related columns from users table as read-only

* Refactored UserPassword#confirm_password? and changes required to accommodate hashing the password after validations
2024-10-10 09:23:06 +08:00
Alan Guo Xiang Tan c1f25cdf5b
FIX: Unicorn master and Sidekiq reopening logs at the same time (#29137)
In our production environment, we have been seeing Sidekiq processes
getting stuck randomly when a USR1 signal is sent to the Unicorn master
process. We have not been able to identify the root cause of why the
Sidekiq process gets stuck. We however noticed that when the Unicorn
master process receives a USR1 signal, it will reopen the logs for the
Unicorn master process first before sending a USR1 signal for the
Unicorn worker processes to reopen the logs. We figured that we should
do the same for the Sidekiq process as well when a USR1 signal.

In this commit, we introduce an arbitrary delay of 1 second before we
the Sidekiq process reopens its log files so as to allow enough time for the Unicorn
master to finish reopening it logs first.

We also do not send reopen logs for the Sidekiq process if the `DISCOURSE_LOG_SIDEKIQ`
env is not present because there is no need to reopen any logs.
2024-10-10 08:01:40 +08:00
Alan Guo Xiang Tan ed6c9d1545
DEV: Call Discourse.redis.flushdb after the end of each test (#29117)
There have been too many flaky tests as a result of leaking state in
Redis so it is easier to resolve them by ensuring we flush Redis'
database.

Locally on my machine, calling `Discourse.redis.flushdb` takes around
0.1ms which means this change will have very little impact on test
runtimes.
2024-10-09 07:19:31 +08:00
Loïc Guitaut 229773e7a8 DEV: Drop `OpenStruct` for the context object in services
While using `OpenStruct` is nice, it’s generally not a very good idea as
it usually leads to performance problems.

The `OpenStruct` source code even says basically to avoid it.

Since the context object is crucial in our services, this patch replaces
`OpenStruct` with a custom implementation instead.
2024-10-08 10:34:55 +02:00
Alan Guo Xiang Tan 4ba8d3b76b
Bump version to v3.4.0.beta3-dev 2024-10-07 12:14:10 +08:00
Alan Guo Xiang Tan 9cfe725b37
Bump version to v3.4.0.beta2 2024-10-07 12:14:09 +08:00
Penar Musaraj 92ac6be82a
SECURITY: prevent topic list filtering by hidden tags for unathorized users
This fixes an issue where unathorized users were able to filter topics
by tags that are hidden from them.
2024-10-07 11:48:55 +08:00
OsamaSayegh d7164d57ec
SECURITY: Block registrations for encoded emails that are invalid 2024-10-07 11:48:52 +08:00
Régis Hanol 34d04e7507
SECURITY: add pagination to post replies
When a post has some replies, and the user click on the button to show them, we would load ALL the replies. This could lead to DoS if there were a very large number of replies.

This adds support for pagination to these post replies.

Internal ref t/129773

FIX: Duplicated parent posts

DEV: Query refactor
2024-10-07 11:48:48 +08:00
Bianca Nenciu e081cc14fb
SECURITY: Use different anon cache keys for XHR requests
XHR requests are handled differently by the application and the
responses do not have any preloaded data so the cache key needs to
differntiate between those requests.
2024-10-07 11:48:45 +08:00
Jeff Wong 91ac382d83
DEV: update compile steps (#29031)
Remove emoji.clear cache calls as data.js.es6.erb hasn't existed in a while.
Emoji data is now compiled separately via javascript rake tasks.

Skip db and redis precompilation when no db is present
2024-10-05 09:50:33 +10:00
Loïc Guitaut 8eff922ace DEV: Output failing MF keys when compilation fails
Currently, when the MessageFormat compiler fails on some translations,
we just have the raw output from the compiler in the logs and that’s not
always very helpful.

Now, when there is an error, we iterate over the translation keys and
try to compile them one by one. When we detect one that is failing, it’s
added to a list that is now outputted in the logs. That way, it’s easier
to know which keys are not properly translated, and the problems can be
addressed quicker.
2024-10-04 23:38:27 +09:00
Loïc Guitaut ad8f46f4f1 DEV: Make params explicit for services in controllers 2024-10-03 16:56:39 +09:00
Jarek Radosz 53e6e32b2d
DEV: Attempt to fix `about` model flakes (#29057)
My theory is that there were nil entries (that we were filtering out) that then changed and we weren't resetting them properly.

(the failure no longer repro'd in 30 CI runs in this PR)
2024-10-03 03:19:34 +02:00
Keegan George 53d41d0727
DEV: Add missing x icon to `svg_sprite.rb` (#29059) 2024-10-02 16:20:24 -04:00
Loïc Guitaut fc1c5f6a8d DEV: Have `contract` take a block in services
Currently in services, the `contract` step is only used to define where
the contract will be called in the execution flow. Then, a `Contract`
class has to be defined with validations in it.

This patch allows the `contract` step to take a block containing
validations, attributes, etc. directly. No need to then open a
`Contract` class later in the service.

It also has a nice side effect, as it’s now easy to define multiples
contracts inside the same service. Before, we had the `class_name:`
option, but it wasn’t really useful as you had to redefine a complete
new contract class.
Now, when using a name for the contract other than `default`, a new
contract will be created automatically using the provided name.

Example:
```ruby
contract(:user) do
  attribute :user_id, :integer

  validates :user_id, presence: true
end
```
This will create a `UserContract` class and use it, also putting the
resulting contract in `context[:user_contract]`.
2024-10-02 17:00:01 +09:00
Sam baeca887d9
FEATURE: improve the suppression for admins when required (#29041)
Previously admins could still click on topics when `suppress_secured_categories_from_admin` was set

This change improves the block so admins without permission will not be allowed to click through till they add themselves to appropriate groups

Keep in mind this setting is a quality of life setting and not a SECURITY
setting, admins have an infinite way of bypassing visiblity limits
2024-10-02 10:52:02 +10:00
Daniel Waterworth c544686540
DEV: Add expires flag to enable_readonly (#29033)
... to control whether readonly mode expires or not.
2024-09-30 13:59:51 -05:00
Linca a1e5796ba1
FEAT: Allow admin delete user's associated accounts (#29018)
This commit introduces a feature that allows an admin to delete a user's
associated account. After deletion, a log will be recorded in staff
actions.

ref=t/136675
2024-09-27 20:08:05 +08:00
Loïc Guitaut 78d9f9fa40 DEV: Rename `ServiceRunner` to `Service::Runner` for consistency 2024-09-20 10:04:42 +02:00
Loïc Guitaut afdb1ac0a0 DEV: Disallow default params in service steps
With the current implementation, a service step can be written as:
```ruby
def my_step(a_default_value: 2)
  …
end
```
That’s a pattern we want to avoid as default values (if needed) should
be probably defined in a contract.

This patch makes a service raise an exception if a default value is
encountered.
2024-09-19 14:47:55 +02:00
Loïc Guitaut fe1098ebac FIX: Don’t raise on some search terms
Currently, when certain search terms are provided, this can lead to
`Search.need_segmenting?` raising an error because it makes `URI#path`
to return `nil` instead of a string.

This patch forces a cast to string so it won’t raise anymore.
2024-09-19 11:35:10 +02:00
Ted Johansson e68748318e
DEV: Remove FileStore#download deprecation warning (#28971)
At a point we split FileStore#download into two methods, one which returns nil (#download) and one which raises an exception (#download!) if there's a download error.

When we did this we added a deprecation warning to the #download method.

It has now been over a year and I have checked all relevant internal cases, so I think it's about time we remove this warning.
2024-09-19 11:08:26 +08:00
Loïc Guitaut 05b8ff436c DEV: Introduce a `Service::ActionBase` class for service actions
This will help to enforce a consistent pattern for creating service
actions.

This patch also namespaces actions and policies, making everything
related to a service available directly in
`app/services/<concept-name>`, making things more consistent at that
level too.
2024-09-18 17:02:46 +02:00
Linca 741e9d70ad
FIX: Don't show move topic for private messages for TL4 (#28871)
In TopicController, in addition to ensure_can_move_posts!, we also
checked if the topic is private message in this line:

```ruby
raise Discourse::InvalidAccess if params[:archetype] == "private_message" && !guardian.is_staff?
```

However, this was not present in `guardian.can_move_posts?`. As a result,
the frontend topic view got an incorrect serialized result, thinking
that TL4 could move the private message post. In fact, once they tried
to move it, they got the `InvalidAccess` error message.

This commit fixes that TL4 will no longer sees the "move to" option in
the "select post" panel for a private message.
2024-09-16 11:30:05 +08:00
Kris a914d3230b
DEV: remap all core icons for fontawesome 6 upgrade (#28715)
Followup to 7d8974d02f

Co-authored-by: David Taylor <david@taylorhq.com>
2024-09-13 16:50:52 +01:00
Osama Sayegh 64b67e0a4f
FIX: Exclude inactive and silenced users from /about page stats (#28877)
The user directory (`/u`) excludes inactive and silenced users from the list, so for the sake parity, it makes sense to also exclude those users from the /about page stats.

Internal topic: t/70928.
2024-09-12 22:28:49 +03:00
Loïc Guitaut b806dce13d DEV: Refactor suspend/silence user services
- fetch models inside services
- validate `user_id` in contracts
- use policy objects
- extract more logic to actions
- write specs for services and action
2024-09-12 10:28:48 +02:00
Jarek Radosz e87d0addf7
DEV: Run theme tests in random order (#28841)
that is: randomize the order in which theme test suites are scheduled
2024-09-11 10:45:12 +02:00
Bianca Nenciu 38592dc48e
PERF: Preload user options when status is enabled (#28827)
The user option 'hide_profile_and_presence' is necessary to figure out
if the user status has to be displayed or not. In order to avoid N+1s
generated by `include_status?` method, both `user_status` and
`user_option` relations have to be included.
2024-09-11 10:39:14 +03:00
Bianca Nenciu d63ffe22f4
DEV: Track SQL queries from MiniSql (#28824)
`track_sql_queries` only returned queries that were executed by
ActiveRecord. All queries executed through DB.exec, DB.query and others
were not returned.
2024-09-11 10:14:53 +03:00
Alan Guo Xiang Tan 279ffb3351
DEV: `JOBS` should be cap at 2 in `assets:precompile:build` (#28834)
This fixes a mistake I introduced in
b652f66378. I wanted to cap it at 2 not 4.
2024-09-11 08:18:42 +08:00
Alan Guo Xiang Tan b652f66378
DEV: Cap number of `thread-loader` workers in `assets:precompile:build` (#28830)
We were running into errors running `ember build` on machines with high
CPU counts. It was then noted that `thread-loader`, which embroider uses, defaults to spinning
up x workers where x is number of physical CPU cores - 1. That is
probably too much so we set out to find out an optimial count to set for
the `JOBS` env which embroider will use to set the number of
`thread-loader` workers.

I first built an image using the following Dockerfile.

```
FROM discourse/base:release

RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake plugin:install_all_official
RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake assets:precompile:prereqs
```

I then ran the following command on my M3 Max Macbook Pro that has 14
phyisal CPU cores.

```
for j in 1 2 4 8 14; do echo "JOBS=$j"; time docker run --rm -it -e JOBS=$j test:latest /bin/bash -c "su discourse -c 'cd /var/www/discourse && bundle exec rake assets:precompile:build'"; done
```

These are the results I got:

```
JOBS=1 0.04s user 0.03s system 0% cpu 1:01.92 total
JOBS=2 0.04s user 0.02s system 0% cpu 42.605 total
JOBS=4 0.04s user 0.02s system 0% cpu 37.012 total
JOBS=8 0.04s user 0.02s system 0% cpu 35.199 total
JOBs=14 0.04s user 0.02s system 0% cpu 37.941 total
```

We think JOBS=2 is a good default when the `JOBS` env has not been set.
Anything above just consumes more resources for little benefit.
2024-09-11 07:58:36 +08:00
David Taylor 9a36286920
FIX: Open file handles 'just in time' during s3 migration (#28806)
Previously we were opening the file handles, then putting them in a queue for upload. If that queue grows too large, we can hit a maximum open files limit.

This commit opens the file handle 'just in time', so the maximum number of open handles is equal to the upload concurrency (20).
2024-09-09 18:39:26 +01:00
Kris d6eb0f4d96
FIX: replace shield-blank with shield-halved for font awesome (#28804) 2024-09-09 10:48:09 -04:00
Jordan Vidrine 9ed9032150
FIX: Requested changes from dev (#28784)
* FIX: Requested changes from dev

* FIX: Add base-url to other links

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-09-09 08:55:42 -05:00
David Taylor 7d8974d02f
UX: Upgrade to fontawesome 6.6.0 (#28778)
This upgrade is designed to be fully backwards-compatible. Any icon names which have changed will be automatically remapped to the new name. For now, this will happen silently. In future, once core & official themes/plugins have been updated, we will start raising deprecation errors to help theme/plugin authors update their code.

Extracted from https://github.com/discourse/discourse/pull/28715

Announcement at https://meta.discourse.org/t/were-upgrading-our-icons-to-font-awesome-6/325349

Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
2024-09-09 14:40:56 +01:00
Sérgio Saquetim f8e3a90ab9
DEV: Forces custom search filter matchers to be case insensitive (#28785) 2024-09-06 12:30:51 -03:00
Jarek Radosz 02f8396cfc
DEV: Remove `register_color_scheme` api (#28777)
This was added 10 years ago, but currently there's not a single use in our public and private plugins and no reference in third-party plugins on github
2024-09-06 17:26:53 +02:00
Gabriel Grubba a98d3d40f2
FEATURE: Add user to topic_tags_changed event (#28714)
* FEATURE: Add user to topic_tags_changed event

Add user to topic_tags_changed event context
Update automation plugin with new arguments in event
Update tests for new arguments

relates to https://github.com/discourse/discourse-chat-integration/pull/214

* DEV: change variable name for better readability

changed `tags` to be payload and used `values_at` to get the values of the keys
2024-09-06 11:23:30 -03:00
Jarek Radosz 526a21730a
DEV: Remove `unsafe_eval` from test/theme-test CSP (#28774)
not needed anymore
2024-09-06 13:28:33 +02:00
Jarek Radosz 4792468c88
DEV: Drop qunit timeout args (#28775)
whatever was using that value is long gone
2024-09-06 13:08:42 +02:00
Natalie Tay 6bb8ac54cb
DEV: Include a basic oauth faraday formatter in core for usage in managed authenticators (#28758)
We currently have some occurrences of ____FaradayFormatter for OAuth logs. This commit creates a generic formatter so that any new authenticators can use it.
2024-09-05 22:29:29 +08:00
Loïc Guitaut e94707acdf DEV: Drop `WithServiceHelper`
This patch removes the `with_service` helper from the code base.
Instead, we can pass a block with actions directly to the `.call` method
of a service.

This simplifies how to use services:
- use `.call` without a block to run the service and get its result
  object.
- use `.call` with a block of actions to run the service and execute
  arbitrary code depending on the service outcome.

It also means a service is now “self-contained” and can be used anywhere
without having to include a helper or whatever.
2024-09-05 09:58:20 +02:00
Osama Sayegh a23773f83d
DEV: Deprecate `show_in_ui` argument of the `register_stat` API (#28727)
We need to start printing deprecation notices when the `show_in_ui` argument is used because it works only for the old about page which will be removed soon. For the new about page, we've introduced a new API `addAboutPageActivity` which is more flexible than a true/false argument on the server side.

Internal topic: t/136551.
2024-09-04 19:58:14 +03:00
Osama Sayegh 280adda09c
FEATURE: Support designating multiple groups as mods on category (#28655)
Currently, categories support designating only 1 group as a moderation group on the category. This commit removes the one group limitation and makes it possible to designate multiple groups as mods on a category.

Internal topic: t/124648.
2024-09-04 04:38:46 +03:00
Bianca Nenciu 5d7fdfb0a6
DEV: Use recorded runtime log for plugin specs (#28711)
The recorded runtime log was specified when running the plugin specs,
but it failed to match the entries from the runtime log and the passed
as command line arguments because the runtime log entries were not
prefixed with `./`.
2024-09-03 20:36:46 +03:00
Joffrey JAFFEUX e418f7056f
FIX: prevents PM to large groups (#28681)
This commit introduces a new hidden site setting: `group_pm_user_limit`, default to `1000` which will raise an error when attempting to create a PM target a large group.
2024-09-03 12:08:14 +02:00
David Taylor 80b9c280ba
DEV: Switch to pnpm for JS dependencies (#28671)
This will bring significant improvements to install speed & storage requirements. For information on how it may affect you, see https://meta.discourse.org/t/324521

This commit:
- removes the `yarn.lock` and replaces with `pnpm-lock.yaml`
- updates workspaces to pnpm format
- adjusts package dependencies to work with pnpm's stricter resolution strategy
- updates Rails app to load modules from more specific node_modules directories
- adds a `.pnpmfile` which automatically cleans up old yarn-managed `node_modules` directories
- updates various scripts to call `pnpm` instead of `yarn`
- updates patches to use pnpm's native patch system instead of patch-package
- adds a patch for licensee to support pnpm
2024-09-03 10:51:07 +01:00