Commit Graph

5888 Commits

Author SHA1 Message Date
Blake Erickson e2fbf4865a
DEV: Check if video thumbnails site setting is enabled (#21306) 2023-04-28 14:08:20 -06:00
Daniel Waterworth ae63d9ea09
PERF: Remove unnecessary queries from Theme#set_field (#21281) 2023-04-27 13:04:58 -05:00
Loïc Guitaut c4c2da83b4 DEV: Set limits for text fields in badges 2023-04-27 17:22:32 +02:00
Selase Krakani 552697e194
DEV: Add deprecation warning for user_badge_removed event (#21255)
`user_badge_removed` event has been replaced with `user_badge_revoked`
2023-04-27 13:07:02 +00:00
Loïc Guitaut a89b3c27aa DEV: set limits for text fields on groups 2023-04-27 13:58:46 +02:00
Sam 2ccc5fc66e
FEATURE: add support for figure and figcaption tags in embeddings (#21276)
Many blog posts use these to illustrate and images were previously omitted

Additionally strip superfluous HTML and BODY tags from embed HTML.

This was incorrectly returned from server.
2023-04-27 19:57:06 +10:00
Selase Krakani 37cc056c1b
FIX: Ensure group-filtered group user event webhooks fire (#21254)
Group user event webhooks filtered by group fail silently
because the `group_ids` job arg wasn't being passed into the job.

This change add's `group_ids` to the `EmitWebHookEvent` jobs queued for
`user_added_to_group` and `user_removed_from_group` events.
2023-04-26 22:38:28 +00:00
Isaac Janzen 366ff0e76b
FIX: Don't display destroy reviewable button on client (#21226)
# Context

https://meta.discourse.org/t/missing-translate-in-review-page/262604

![image](https://user-images.githubusercontent.com/50783505/234089049-72332040-e7d5-4081-824a-b0b36e37187a.png)

An additional button was added as a result of dd495a0e19 which was intended to grant access to deleting reviewable from the API. 

We were being too flexible by only checking if the user was an admin

012aaf0ba3/lib/guardian.rb (L237)

where it should instead by scoped to check if the request was an API call.

# Fix

https://github.com/discourse/discourse/pull/21226/files#diff-0a2548be4b18bd4ef2dffb3ef8e44984d2fef7f037b53e98f67abea52ef75aa2R237

# Additions

Added a new guard of `is_api?`

https://github.com/discourse/discourse/pull/21226/files#diff-0a2548be4b18bd4ef2dffb3ef8e44984d2fef7f037b53e98f67abea52ef75aa2R657-R660

In `app/models/reviewable.rb` we check if the user has the permissions to the destroy action via the `Guardian`. To do this we were instantiating a new `Guardian` class which then caused us to lose the context of the request. The request is a necessary component in the guard of `is_api?` so we needed to pass the already defined Guardian from the `app/controllers/reviewables_controller.rb` to the `#perform` method to ensure the request is present.
2023-04-24 20:22:37 -05:00
Selase Krakani cdf1589a85
FEATURE: Add support for user badge revocation webhook events (#21204)
Currently, only user badge grants emit webhook events. This change
extends the `user_badge` webhook to emit user badge revocation events.

A new `user_badge_revoked` event has been introduced instead of relying
on the existing `user_badge_removed` event. `user_badge_removed` emitted
just the `badge_id` and `user_id` which aren't helpful for generating a
meaningful webhook payload for revoked(deleted) user badges.

The new event emits  the user badge object.
2023-04-24 20:36:40 +00:00
Isaac Janzen 599979902e
FIX: Error when trying to bump a topic with no category (#21207)
When revising a post, if the topic that post belonged to did not have a category attached it would error with 

> NoMethodError (undefined method `read_restricted' for nil:NilClass)
2023-04-24 09:28:10 -05:00
Blake Erickson 6ae0c42c01
FIX: Do not overwrite existing thumbnails (#21199)
* FIX: Do not overwrite existing thumbnails

When auto generating video thumbnails they should not overwrite any
existing topic thumbnails.

This also addresses an issue with capitalized file extensions like .MOV
that were being excluded.

* Update app/models/post.rb

Remove comment

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>

---------

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-04-21 13:33:33 -06:00
Ted Johansson e002a24eca
FEATURE: Add new don't feed the trolls feature (#21001)
Responding to negative behaviour tends to solicit more of the same. Common wisdom states: "don't feed the trolls".

This change codifies that advice by introducing a new nudge when hitting the reply button on a flagged post. It will be shown if either the current user, or two other users (configurable via a site setting) have flagged the post.
2023-04-20 15:49:35 +08:00
Isaac Janzen a3693fec58
FEATURE: Allow drafts to be deleted via the API (#21148)
This PR adds the ability to destroy drafts for a passed user via the API. This was not possible before as this action was reserved for only your personal drafts.

If a user is an admin and calls the `#destroy` action from the API they are able to destroy a draft for a passed user. A user can be targeted by passed either their:
- username
- external_id (for SSO) 

to the request.

In the case you attempt to destroy a non-personal draft and
- You are not an admin
- You do not access the `#destroy` action via the API

you will raise a `Discourse::InvalidAccess` (403) and will not succeed in destroying the draft.
2023-04-19 14:41:45 -05:00
Ted Johansson f3f30d6865
SECURITY: Encode embed url (#21133)
The embed_url in "This is a companion discussion..." could be used for
XSS.

Co-authored-by: Blake Erickson <o.blakeerickson@gmail.com>
2023-04-18 15:05:29 +08:00
Loïc Guitaut a5235f7d16
DEV: Refactor STI/polymorphic associations in chat (#20789) 2023-04-17 15:41:56 +02:00
David Battersby 967010e545
FEATURE: Add an emoji deny list site setting (#20929)
This feature will allow sites to define which emoji are not allowed. Emoji in this list should be excluded from the set we show in the core emoji picker used in the composer for posts when emoji are enabled. And they should not be allowed to be chosen to be added to messages or as reactions in chat.

This feature prevents denied emoji from appearing in the following scenarios:
- topic title and page title
- private messages (topic title and body)
- inserting emojis into a chat
- reacting to chat messages
- using the emoji picker (composer, user status etc)
- using search within emoji picker

It also takes into account the various ways that emojis can be accessed, such as:
- emoji autocomplete suggestions
- emoji favourites (auto populates when adding to emoji deny list for example)
- emoji inline translations
- emoji skintones (ie. for certain hand gestures)
2023-04-13 15:38:54 +08:00
David Taylor 9238767f7e
FEATURE: Persist password hashing algorithm/params in database (#20980)
Previously, Discourse's password hashing was hard-coded to a specific algorithm and parameters. Any changes to the algorithm or parameters would essentially invalidate all existing user passwords.

This commit introduces a new `password_algorithm` column on the `users` table. This persists the algorithm/parameters which were use to generate the hash for a given user. All existing rows in the users table are assumed to be using Discourse's current algorithm/parameters. With this data stored per-user in the database, we'll be able to keep existing passwords working while adjusting the algorithm/parameters for newly hashed passwords.

Passwords which were hashed with an old algorithm will be automatically re-hashed with the new algorithm when the user next logs in.

Values in the `password_algorithm` column are based on the PHC string format (https://github.com/P-H-C/phc-string-format/blob/master/phc-sf-spec.md). Discourse's existing algorithm is described by the string `$pbkdf2-sha256$i=64000,l=32$`

To introduce a new algorithm and start using it, make sure it's implemented in the `PasswordHasher` library, then update `User::TARGET_PASSWORD_ALGORITHM`.
2023-04-11 10:16:28 +01:00
Krzysztof Kotlarek 63a0466548
FIX: improve performance of UserStat.ensure_consistency (#21044)
Optimize `UserStatpost_read_count` calculation.

In addition, tests were updated to fail when code is not evaluated. Creation of PostTiming was updating `post_read_count`. Count it has to be reset to ensure that ensure_consitency correctly calculates result.

Extracting users seen in the last hour to separate Common Table Expression reduces the amount of processed rows.

Before
```
Update on user_stats  (cost=267492.07..270822.95 rows=2900 width=174) (actual time=12606.121..12606.127 rows=0 loops=1)
  ->  Hash Join  (cost=267492.07..270822.95 rows=2900 width=174) (actual time=12561.814..12603.689 rows=10 loops=1)
        Hash Cond: (user_stats.user_id = x.user_id)
        Join Filter: (x.c <> user_stats.posts_read_count)
        Rows Removed by Join Filter: 67
        ->  Seq Scan on user_stats  (cost=0.00..3125.34 rows=75534 width=134) (actual time=0.014..39.173 rows=75534 loops=1)
        ->  Hash  (cost=267455.80..267455.80 rows=2901 width=48) (actual time=12558.613..12558.617 rows=77 loops=1)
              Buckets: 4096  Batches: 1  Memory Usage: 39kB
              ->  Subquery Scan on x  (cost=267376.03..267455.80 rows=2901 width=48) (actual time=12168.601..12558.572 rows=77 loops=1)
                    ->  GroupAggregate  (cost=267376.03..267426.79 rows=2901 width=12) (actual time=12168.595..12558.525 rows=77 loops=1)
                          Group Key: pt.user_id
                          ->  Sort  (cost=267376.03..267383.28 rows=2901 width=4) (actual time=12100.490..12352.106 rows=2072830 loops=1)
                                Sort Key: pt.user_id
                                Sort Method: external merge  Disk: 28488kB
                                ->  Nested Loop  (cost=1.28..267209.18 rows=2901 width=4) (actual time=0.040..11528.680 rows=2072830 loops=1)
                                      ->  Nested Loop  (cost=0.86..261390.02 rows=13159 width=8) (actual time=0.030..3492.887 rows=3581648 loops=1)
                                            ->  Index Scan using index_users_on_last_seen_at on users u  (cost=0.42..89.71 rows=28 width=4) (actual time=0.010..0.201 rows=78 loops=1)
                                                  Index Cond: (last_seen_at > '2023-04-11 00:22:49.555537'::timestamp without time zone)
                                            ->  Index Scan using index_post_timings_on_user_id on post_timings pt  (cost=0.44..9287.60 rows=4455 width=8) (actual time=0.081..38.542 rows=45919 loops=78)
                                                  Index Cond: (user_id = u.id)
                                      ->  Index Scan using forum_threads_pkey on topics t  (cost=0.42..0.44 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=3581648)
                                            Index Cond: (id = pt.topic_id)
                                            Filter: ((deleted_at IS NULL) AND ((archetype)::text = 'regular'::text))
                                            Rows Removed by Filter: 0
Planning Time: 0.692 ms
Execution Time: 12612.587 ms
```
After
```
Update on user_stats  (cost=9473.60..12804.30 rows=2828 width=174) (actual time=677.724..677.729 rows=0 loops=1)
  ->  Hash Join  (cost=9473.60..12804.30 rows=2828 width=174) (actual time=672.536..677.706 rows=1 loops=1)
        Hash Cond: (user_stats.user_id = x.user_id)
        Join Filter: (x.c <> user_stats.posts_read_count)
        Rows Removed by Join Filter: 54
        ->  Seq Scan on user_stats  (cost=0.00..3125.34 rows=75534 width=134) (actual time=0.012..23.977 rows=75534 loops=1)
        ->  Hash  (cost=9438.24..9438.24 rows=2829 width=48) (actual time=647.818..647.822 rows=55 loops=1)
              Buckets: 4096  Batches: 1  Memory Usage: 37kB
              ->  Subquery Scan on x  (cost=9381.66..9438.24 rows=2829 width=48) (actual time=647.409..647.805 rows=55 loops=1)
                    ->  HashAggregate  (cost=9381.66..9409.95 rows=2829 width=12) (actual time=647.403..647.786 rows=55 loops=1)
                          Group Key: pt.user_id
                          Batches: 1  Memory Usage: 121kB
                          ->  Nested Loop  (cost=1.86..9367.51 rows=2829 width=4) (actual time=0.056..625.245 rows=120022 loops=1)
                                ->  Nested Loop  (cost=1.44..3692.96 rows=12832 width=8) (actual time=0.047..171.754 rows=217440 loops=1)
                                      ->  Nested Loop  (cost=1.00..254.63 rows=25 width=12) (actual time=0.030..1.407 rows=56 loops=1)
                                            Join Filter: (u.id = user_stats_1.user_id)
                                            ->  Nested Loop  (cost=0.71..243.08 rows=25 width=8) (actual time=0.018..1.207 rows=87 loops=1)
                                                  ->  Index Scan using index_users_on_last_seen_at on users u  (cost=0.42..86.71 rows=27 width=4) (actual time=0.009..0.156 rows=87 loops=1)
                                                        Index Cond: (last_seen_at > '2023-04-11 00:47:07.437568'::timestamp without time zone)
                                                  ->  Index Only Scan using user_stats_pkey on user_stats us  (cost=0.29..5.79 rows=1 width=4) (actual time=0.011..0.011 rows=1 loops=87)
                                                        Index Cond: (user_id = u.id)
                                                        Heap Fetches: 87
                                            ->  Index Scan using user_stats_pkey on user_stats user_stats_1  (cost=0.29..0.45 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=87)
                                                  Index Cond: (user_id = us.user_id)
                                                  Filter: (posts_read_count < 10000)
                                                  Rows Removed by Filter: 0
                                      ->  Index Scan using index_post_timings_on_user_id on post_timings pt  (cost=0.44..92.98 rows=4455 width=8) (actual time=0.036..2.492 rows=3883 loops=56)
                                            Index Cond: (user_id = user_stats_1.user_id)
                                ->  Index Scan using forum_threads_pkey on topics t  (cost=0.42..0.44 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=217440)
                                      Index Cond: (id = pt.topic_id)
                                      Filter: ((deleted_at IS NULL) AND ((archetype)::text = 'regular'::text))
                                      Rows Removed by Filter: 0
Planning Time: 1.406 ms
Execution Time: 677.817 ms
```
2023-04-11 12:28:08 +10:00
Daniel Waterworth fcc73b441d
FIX: Make emoji cache store marshalled objects (#21029)
We only marshal arrays, hashes and sets, which meant that the emojis
here were just getting `to_s`ed.

This is a hack.
2023-04-10 10:44:11 -05:00
Krzysztof Kotlarek e586f6052f
FEATURE: public custom sidebar sections visible to anonymous (#20931)
Previously, public custom sections were only visible to logged-in users. In this PR, we are making them visible to anonymous as well.

The reason is that Community Section will be moved into custom section model to be easily editable by admins.
2023-04-06 08:55:47 +10:00
Krzysztof Kotlarek b72282123b
FIX: public sidebar sections belong to system user (#20972)
Before, public sidebar sections were belonging to admin. However, a better choice is system user.
2023-04-05 10:52:18 +10:00
David Taylor 2386ad12f2
Update default ga_version to v4 and add warning message for v3 (#20936)
Sites which are already using ga3 will stay on that version, and will be shown a warning in the admin panel until they update.

https://meta.discourse.org/t/upgrade-to-google-analytics-4-before-july-2023/260498
2023-04-04 13:14:20 +01:00
Alan Guo Xiang Tan e54359c135
DEV: Drop plugin related index from core (#20930)
The migration of the index has since been moved into the
discourse-akismet plugin where it should have always been.
2023-04-04 07:31:09 +08:00
Alan Guo Xiang Tan fd34032db2
DEV: Support filter for topics in specific subcategories on /filter (#20927)
This commit adds support for filtering for topics in specific
subcategories via the categories filter query language.

For example: `category:documentation:admins` will filter for topics and
subcategory topics in
the category with slug "admins" whose parent category has the slug
"documentation".

The `=` prefix can also be used such that
`=category:documentation:admins` will exclude subcategory topics of the
category with slug "admins" whose parent category has the slug
"documentation".
2023-04-03 18:36:59 +08:00
Krzysztof Kotlarek c86d772277
FIX: Drop internal URL validation for paths in sidebar (#20891)
`Rails.application.routes.recognize_path(value)` was not working for /admin paths because StaffConstraint.new requires user to check permission.

This validation is not bringing much value, and the easiest way is to drop it. In the worse case scenario, a user will have an incorrect link in their sidebar.

Bug reported: https://meta.discourse.org/t/custom-sidebar-sections-being-tested-on-meta/255303/66
2023-03-31 13:26:56 +08:00
Mark VanLandingham 9518e47204
FEATURE: ability to bulk_remove users from a group (#20876) 2023-03-30 08:06:36 -05:00
Martin Brennan 84ff96bd07
FIX: Do not validate email in TL promotion (#20892)
There is no need to validate the user's emails when
promoting/demoting their trust level, this can cause
issues in things like Jobs::Tl3Promotions, we don't
need to fail in that case when all we are doing is changing
trust level.
2023-03-30 13:52:10 +10:00
Blake Erickson aa09a78d14
DEV: Remove diffhtml_preview for video thumbnails (#20865)
Video thumbnail generation is no longer dependent on
`enable_diffhtml_preview` being enabled.
2023-03-28 12:38:50 -06:00
Vinoth Kannan 08ff6eebad
FIX: skip category preference update if already set by group. (#20823)
`default_categories_*` site settings will update the category preferences on user creation. But it shouldn't update the user's category preference if a group's setting already updated it for that user.
2023-03-28 19:43:01 +05:30
Sam 7038540af6
PERF: ensure sidebar section link index is correctly ordered (#20854)
We perform lookups on sidebar section links based on sidebar_section_id
totally ignoring user. This ensures we have an index to work with.

This removes the previous index `links_user_id_section_id_position` which
partially doubled up `idx_unique_sidebar_section_links`
2023-03-28 15:13:44 +11:00
Daniel Waterworth d563b73202
DEV: Make category topic_ids cache work like other caches (#20814) 2023-03-27 09:23:29 -05:00
Daniel Waterworth 3e80ab3143
DEV: Make category subcategory_ids cache work like other caches (#20815) 2023-03-27 09:23:23 -05:00
Daniel Waterworth b2554e1490
DEV: Make hex cache work like other caches (#20816) 2023-03-27 09:22:53 -05:00
Daniel Waterworth 0f4bb19524
DEV: make developer cache work like other caches (#20817) 2023-03-27 09:22:33 -05:00
Krzysztof Kotlarek 4047073292
FIX: display validation under custom sidebar fields (#20772)
Before, incorrectly filled fields were marked with red border. Now, additional information under the field is displayed to notify the user what is incorrect.

/t/93696
2023-03-27 13:03:16 +11:00
Daniel Waterworth 0728fa055b
DEV: Make theme cache work like other caches (#20813) 2023-03-24 12:37:16 -05:00
Mark VanLandingham ebada4a6b0
DEV: More specific API to including extra associations in CategoryList (#20790) 2023-03-23 12:39:38 -05:00
Mark VanLandingham 32aa821f12
DEV: Add preload API to CategoryList (#20778) 2023-03-22 15:12:08 -05:00
Martin Brennan 520d4f504b
FEATURE: Auto-remove users without permission from channel (#20344)
There are many situations that may cause users to lose permission to
send messages in a chat channel. Until now we have relied on security
checks in `Chat::ChatChannelFetcher` to remove channels which the
user may have a `UserChatChannelMembership` record for but which
they do not have access to.

This commit takes a more proactive approach. Now any of these following
`DiscourseEvent` triggers may cause `UserChatChannelMembership`
records to be deleted:

* `category_updated` - Permissions of the category changed
   (i.e. CategoryGroup records changed)
* `user_removed_from_group` - Means the user may not be able to access the
   channel based on `GroupUser` or also `chat_allowed_groups`
* `site_setting_changed` - The `chat_allowed_groups` was updated, some
   users may no longer be in groups that can access chat.
* `group_destroyed` - Means the user may not be able to access the
   channel based on `GroupUser` or also `chat_allowed_groups`

All of these are handled in a distinct service run in a background
job. Users removed are logged via `StaffActionLog` and then we
publish messages on a per-channel basis to users who had their
memberships deleted.

When the user has a channel they are kicked from open, we show
a dialog saying "You no longer have access to this channel".

When they click OK we redirect them either:

* To their first other public channel, if they have any followed
* The chat browse page if they don't

This is to save on tons of requests from kicked out users getting messages
from other channels.

When the user does not have the kicked channel open, we can just
silently yoink it out of their sidebar and turn off subscriptions.
2023-03-22 10:19:59 +10:00
Daniel Waterworth 7b01576c8d
DEV: Remove emoji cache dead code (#20764)
The cache is already not shared between app servers that have different
app_versions, so this check was redundant.
2023-03-21 12:33:12 -05:00
Daniel Waterworth a0a6f6d71b
FIX: Fix the emoji toned regexes (#20763) 2023-03-21 11:48:55 -05:00
Régis Hanol 37609897e8
FEATURE: log manual bounce reset (#20758)
DEV: rename the route "/admin/users/:id/reset_bounce_score" to use dashes instead of underscores
2023-03-21 15:26:26 +01:00
Krzysztof Kotlarek 1859025228
FIX: my links in sidebar section (#20754)
Links like `/my/preferences` were invalid in custom section. The reason is that `/my` links are just redirects from backend, and they are not recognized as valid Ember paths.

https://github.com/discourse/discourse/blob/main/config/routes.rb#L433

Therefore, regex match allowlist was added - similar to backend check:

https://github.com/discourse/discourse/blob/main/app/controllers/users_controller.rb#L471

/safe-mode is same case
2023-03-21 15:58:42 +11:00
Krzysztof Kotlarek db74e9484b
FEATURE: ability to reorder links in custom sidebar sections (#20626)
Drag and drop to reorder custom sidebar sections
2023-03-21 12:23:28 +11:00
Martin Brennan 2f6987a241
DEV: Minor refactor for bookmark polymorphic changes (#20733)
Followup to 184ce647ea,
this just implements Bianca's suggestion on the original
PR and catches the NameError, which was not necessary
before as we were not actually resolving any class from
bookmarkable_type.
2023-03-20 10:19:28 +01:00
Joffrey JAFFEUX 184ce647ea
FIX: correctly infer polymorphic class from bookmarkable type (#20719)
Prior to this change `registered_bookmarkable` would return `nil` as  `type` in `Bookmark.registered_bookmarkable_from_type(type)` would be `ChatMessage` and we registered a `Chat::Message` class.

This commit will now properly rely on each model `polymorphic_class_for(name)` to help us infer the proper type from a a `bookmarkable_type`.

Tests have also been added to ensure that creating/destroying chat message bookmarks is working correctly.

---

Longer explanation

Currently when you save a bookmark in the database, it's associated to another object through a polymorphic relationship, which will is represented by two columns: `bookmarkable_id` and `bookmarkable_type`. The `bookmarkable_id` contains the id of the relationship (a post ID for example) and the `bookmarkable_type` contains the type of the object as a string by default, (`"Post"` for example).

Chat plugin just started namespacing objects, as a result a model named `ChatMessage` is now named `Chat::Message`, to avoid complex and risky migrations we rely on methods provided by rails to alter the `bookmarkable_type` when we save it: we want to still save it as `"ChatMessage"` and not `"Chat::Message"`. And, to retrieve the correct model when we load the bookmark from the database: we want `"ChatMessage"` to load the `Chat::Message` model and not the `ChatMessage`model which doesn't exist anymore.

On top of this the bookmark codepath is allowing plugins to register types and will check against these types, so we alter this code path to be able to do a similar ChatMessage <-> Chat::Message dance and allow to check the type is valid. In the specific case of this commit, we were retrieving a `"ChatMessage"` bookmarkable_type from the DB and looking for it in the registered bookmarkable types which contain `Chat::Message` and not `ChatMessage`.
2023-03-17 17:20:24 +01:00
Joffrey JAFFEUX 12a18d4d55
DEV: properly namespace chat (#20690)
This commit main goal was to comply with Zeitwerk and properly rely on autoloading. To achieve this, most resources have been namespaced under the `Chat` module.

- Given all models are now namespaced with `Chat::` and would change the stored types in DB when using polymorphism or STI (single table inheritance), this commit uses various Rails methods to ensure proper class is loaded and the stored name in DB is unchanged, eg: `Chat::Message` model will be stored as `"ChatMessage"`, and `"ChatMessage"` will correctly load `Chat::Message` model.
- Jobs are now using constants only, eg: `Jobs::Chat::Foo` and should only be enqueued this way

Notes:
- This commit also used this opportunity to limit the number of registered css files in plugin.rb
- `discourse_dev` support has been removed within this commit and will be reintroduced later

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2023-03-17 14:24:38 +01:00
David Taylor 303f97ce89
PERF: Use native postgres upsert for ApplicationRequest (#20706)
Using `create_or_find_by!`, followed by `update_all!` requires two or three queries (two when the row doesn't already exist, three when it does). Instead, we can use postgres's native `INSERT ... ON CONFLICT ... DO UPDATE SET` feature to do the logic in a single atomic call.
2023-03-17 09:35:29 +00:00
Daniel Waterworth 84f590ab83
DEV: Store theme sprites in the DB (#20501)
Let's avoid fetching sprites from the CDN during page rendering.
2023-03-14 13:11:45 -05:00
Blake Erickson 943068a634
FIX: Welcome topic banner showing after general category is deleted (#20639)
If you happen to delete the general category before editing the welcome
topic, the banner will still display. This fix adds a after destroy hook
that will clear the entries for the welcome topic banner in the redis
cache.
2023-03-10 12:33:12 -07:00
Ted Johansson 87ec058b8b
FEATURE: Configurable auto-bump cooldown (#20507)
Currently the auto-bump cooldown is hard-coded to 24 hours.

This change makes the highlighted 24 hours part configurable (defaulting to 24 hours), and the rest of the process remains the same.

This uses the new CategorySetting model associated with Category. We decided to add this because we want to move away from custom fields due to the lack of type casting and validations, but we want to keep the loading of these optional as they are not needed for almost all of the flows.

Category settings will be back-filled to all categories as part of this change, and creating a new category will now also create a category setting.
2023-03-10 13:45:01 +08:00
Blake Erickson f144c64e13
Generate thumbnail images for video uploads (#19801)
* FEATURE: Generate thumbnail images for uploaded videos

Topics in Discourse have a topic thumbnail feature which allows themes
to show a preview image before viewing the actual Topic.

This PR allows for the ability to generate a thumbnail image from an
uploaded video that can be use for the topic preview.
2023-03-09 09:26:47 -07:00
Krzysztof Kotlarek 22bccef8f4
FIX: set external flag before validation (#20599)
Previously, `before_save` callback was used but `before_validation` has to be used to set external flag.
2023-03-09 10:44:54 +11:00
Roman Rizzi 910bf74c2e
FIX: Display a proper error when user already exists and email addresses are hidden. (#20585)
Follow-up to #16703. Returning an empty response leads to a bad UX since the user
has no feedback about what happened.
2023-03-08 12:38:58 -03:00
Loïc Guitaut 27f7cf18b1 FIX: Don’t email suspended users from group PM
Currently, when a suspended user belongs to a group PM (private message
with more than two people in it) and a staff member sends a message to
this group PM, then the suspended user will receive an email.
This happens because a suspended user can only receive emails from staff
members. But in this case, this can be seen as a bug as the expected
behavior would be instead to not send any email to the suspended user. A
staff member can participate in active discussions like any other
member and so their messages in this context shouldn’t be treated
differently than the ones from regular users.

This patch addresses this issue by checking if a suspended user receives
a message from a group PM or not. If that’s the case then an email won’t
be sent no matter if the post originated from a staff member or not.
2023-03-08 15:53:53 +01:00
Gerhard Schlager 12436d054d
DEV: Remove `badge_granted_title` column from `user_profiles` (#20476)
That column is obsolete since we added the `granted_title_badge_id` column in 2019 (56d3e29a69). Having both columns can lead to inconsistencies (mostly due to old data from before 2019).

For example, `BadgeGranter.revoke_ungranted_titles!` doesn't work correctly if `badge_granted_title` is `false` while `granted_title_badge_id` points to the badge that is used as title.
2023-03-08 13:37:20 +01:00
Martin Brennan 360d0dde65
DEV: Change Bookmarkable registration to DiscoursePluginRegistry (#20556)
Similar spirit to e195e6f614,
this moves the Bookmarkable registration to DiscoursePluginRegistry
so plugins which are not enabled do not register additional
bookmarkable classes.
2023-03-08 10:39:12 +10:00
Krzysztof Kotlarek a16ea24461
FEATURE: allow external links in custom sidebar sections (#20503)
Originally, only Discourse site links were available. After feedback, it was decided to extend this feature to external URLs.

/t/93491
2023-03-07 11:47:18 +11:00
Alan Guo Xiang Tan e3977f84a3
FIX: Incorrect topic tracking state count when a new category is created (#20506)
What is the problem?

We have a hidden site setting `show_category_definitions_in_topic_lists`
which is set to false by default. What this means is that category
definition topics are not shown in the topic list by default. Only the
category definition topic for the category being viewed will be shown.
However, we have a bug where we would show that a category has new
topics when a new child category along with its category definition
topic is created even though the topic list does not list the child
category's category definition topic.

What is the fix here?

This commit fixes the problem by shipping down an additional
`is_category_topic` attribute in `TopicTrackingStateItemSerializer` when
the `show_category_definitions_in_topic_lists` site setting has been set
to false. With the new attribute, we can then exclude counting child
categories' category definition topics when counting new and unread
counts for a category.
2023-03-06 10:13:10 +08:00
chapoi e52bbc1230
UX/DEV: Review queue redesign fixes (#20239)
* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* DEV: switch to selectKit

* UX: color approve/reject buttons in RQ

* DEV: regroup actions

* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* Join questions for flagged post with "or" with new I18n function
* Move ReviewableScores component out of context
* Add CSS classes to reviewable-item based on human type

* UX: add table header for scoring

* UX: don't display % score

* UX: prefix modifier class with dash

* UX: reviewQ flag table styling

* UX: consistent use of ignore icon

* DEV: only show context question on pending status

* UX: only show table headers on pending status

* DEV: reviewQ regroup actions for hidden posts

* UX: reviewQ > approve/reject buttons

* UX: reviewQ add fadeout

* UX: reviewQ styling

* DEV: move scores back into component

* UX: reviewQ mobile styling

* UX: score table on mobile

* UX: reviewQ > move meta info outside table

* UX: reviewQ > score layout fixes

* DEV: readd `agree_and_keep` and fix the spec tests.

* Fix the spec tests

* fix the quint test

* DEV: readd deleting replies

* UX: reviewQ copy tweaks

* DEV: readd test for ignore + delete replies

* Remove old

* FIX: Add perform_ignore back in for backwards compat

* DEV: add an action alias `ignore` for `ignore_and_do_nothing`.

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Vinoth Kannan <svkn.87@gmail.com>
2023-03-02 16:40:53 +01:00
David Battersby 96d03ea9c0
FIX: No small action created when a non-author removes itself from a PM (#20502)
Fixes a small issue where allowed user removes themselves from a private message before the post activity (small action) is created.

I also added some test coverage to prevent regression.

/t/92811
2023-03-02 13:47:54 +08:00
Martin Brennan e195e6f614
DEV: Move about_stat_groups to DiscoursePluginRegistry (#20496)
Follow up to 098ab29d41. Since
we just used a `cattr_reader` on `About` this was not safe
for multisite, since some sites could have the chat plugin
enabled and some may not. Using `DiscoursePluginRegistry` gets
around this issue, and makes it so the chat stats only show
for a site if `chat_enabled` is true.
2023-03-02 08:10:16 +10:00
David Battersby 8a2995f719
FIX: only show approved users in search_user results when site setting enabled (#20493)
Returns only approved users when using @ in composer (if must_approve_users site setting enabled).
2023-03-01 12:23:29 +08:00
Natalie Tay 44b7706a2b
UX: Skip applying link-type watched words to user custom fields (#20465)
We currently apply type: :link watched words to custom user fields. This makes the user card pretty ugly because we don't allow html / links there. Additionally, the admin UI also does not say that we apply this to custom user fields, but only words in posts.

So this PR is to remove the replacement of link-type watch words for custom user fields.
2023-03-01 10:43:34 +08:00
Martin Brennan d3a1b09361
FEATURE: Chat header icon indicator preference (#20474)
This commit allows the user to set their preference vis-a-vis
the chat icon in the header of the page. There are three options:

- All New (default) - This maintains the existing behaviour where
  all new messages in the channel show a blue dot on the icon
- Direct Messages and Mentions - Only show the green dot on the
  icon when you are directly messaged or mentioned, the blue dot
  is never shown
- Never - Never show any dot on the chat icon, for those who
  want tractor-beam-laser-focus
2023-03-01 11:01:44 +10:00
Keegan George def4133d59
DEV: Move `form_template_ids` serializer location (#20446) 2023-02-28 09:09:20 -08:00
Bianca Nenciu ccb345bd88
FEATURE: Update topic/comment embedding parameters (#20181)
This commit implements many changes to topic and comments embedding. It
deprecates the class_name field from EmbeddableHost and suggests using
the className parameter. discourse_username parameter has been
deprecated and it will fetch it from embedded site from the author or
discourse-username meta.

See the updated code sample from Admin > Customize > Embedding page.

* FEATURE: Add className parameter for Discourse embed

* DEV: Hide class_name from EmbeddableHost

* DEV: Deprecate class_name field of EmbeddableHost

* FEATURE: Use either author or discourse-username meta tag

* DEV: Deprecate discourse_username parameter

* DEV: Improve embed code sample
2023-02-28 14:31:59 +02:00
Blake Erickson 451ee71930
FEATURE: Add API scope for listing topics in a category (#20471)
Adding an api scope for `/c/{slug}/{id}.json`. Without this you had to
append `/l/latest` to the url in order for it to work with existing
scopes.
2023-02-27 16:21:39 -07:00
Osama Sayegh a509441148
DEV: Include unread topics in New topic lists and link to it in sidebar (#20432)
This commit introduces a few experimental changes to the New topics list and "Everything" link in the sidebar:

1. Make the New topics list include unread topics
2. Make the Everything section in the sidebar link to the New topics list (`/new`)
3. Remove "unread" or "new" text next to the count and keep the count
4. The count is a sum of new and unread topics counts

All of these of changes are behind an off-by-default feature flag. I've not written extensive tests for these changes because they're highly experimental.

Internal topic: t/77234.
2023-02-27 15:11:01 +03:00
Krzysztof Kotlarek d92fd30d23
FIX: do not escape slash for category text description (#20460)
Original solution to use `description` instead of `text_description` was wrong: https://github.com/discourse/discourse/pull/20436

Problem is that we have to escape HTML tags.

However, we would like to use escape method which is keep `/` intact.   Expected behavior is given by  ERB::Util.html_escape instead of Rack::Utils.escape_html

/t/92015
2023-02-27 12:48:48 +11:00
Keegan George 6108eee31d
DEV: Apply form template to categories (#20337) 2023-02-23 11:18:14 -08:00
Loïc Guitaut a9f2c6db64 SECURITY: Show only visible tags in metadata
Currently, the topic metadata show both public and private
tags whereas only visible ones should be exposed.
2023-02-23 17:22:20 +01:00
Blake Erickson 5dbdcb3f23
FEATURE: Adding some more api scopes (#20420)
Adds api scopes for

- deleting a topic
- deleting a post
- listing tags
2023-02-23 08:33:29 -07:00
Blake Erickson 51a7cd899e
FEATURE: Add API scopes for group endpoints (#20401)
Added two new api scopes for managing and administering groups.

See https://meta.discourse.org/t/249710
2023-02-22 09:06:49 -07:00
Krzysztof Kotlarek b9d037770c
DEV: configurable public sidebar sections (#20303)
Extension of https://github.com/discourse/discourse/pull/20057

Admin can create a public session visible to everyone. An additional checkbox is displayed for staff members.
2023-02-22 08:55:44 +11:00
Daniel Waterworth b96a23978a
PERF: Add index to speed up ReindexSearch job (#20151) 2023-02-21 13:16:43 -05:00
Loïc Guitaut f7c57fbc19 DEV: Enable `unless` cops
We discussed the use of `unless` internally and decided to enforce
available rules from rubocop to restrict its most problematic uses.
2023-02-21 10:30:48 +01:00
Gerhard Schlager 7ef482a292
REFACTOR: Fix pluralized strings in chat plugin (#20357)
* FIX: Use pluralized string

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Fix misuse of pluralized string

* DEV: Remove linting of `one` key in MessageFormat string, it doesn't work

* REFACTOR: Fix misuse of pluralized string

This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff. The string is quite complicated, so the best option was to switch to MessageFormat.

* REFACTOR: Fix misuse of pluralized string

* FIX: Use pluralized string

This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff.

* REFACTOR: Correctly pluralize reaction tooltips in chat

This also ensures that maximum 5 usernames are shown and fixes the number of "others" which was off by 1 if the current user reacted on a message.

* REFACTOR: Use translatable string as comma separator

* DEV: Add comment to translation to clarify the meaning of `%{identifier}`

* REFACTOR: Use translatable comma separator and use explicit interpolation keys

* REFACTOR: Don't interpolate lowercase channel status

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Don't interpolate channel status

* REFACTOR: Use %{count} interpolation key

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Correctly pluralize DM chat channel titles
2023-02-20 10:31:02 +01:00
Osama Sayegh f91631b625
DEV: Update javascript:update_constants rake task following template colocation (#20365)
As of ba3f62f576, handlebars templates are colocated with js files so the path to hbs templates referenced by this rake task is no longer valid. This commit fixes the path to hbs templates and updates a couple of files that are generated by the rake task.
2023-02-20 06:20:47 +03:00
Martin Brennan 7ad92b7ab9
FIX: Do not enqueue update_post_uploads_secure_status unnecessarily (#20366)
We call `post.update_uploads_secure_status` in both
`PostCreator` and `PostRevisor`. Only the former was checking
if `SiteSetting.secure_uploads?` was enabled, but the latter
was not. There is no need to enqueue the job
`UpdatePostUploadsSecureStatus` if secure_uploads is not
enabled for the site.
2023-02-20 11:21:54 +10:00
Leonardo Mosquera 6fa9a8990d
Add username template variables to Welcome User email template (#20352)
* Add username and name_or_username variables to SystemMessage defaults

* Allow username and name variables on welcome_user email template overrides

* Satisfy linting

* Add test
2023-02-17 18:38:21 -03:00
Sam f96ef33856
FIX: dominant color not working for 16bit images (#20300)
16 bit images were not returning the correct dominant color due truncation

The routine expected an 8bit color eg: #FFAA00, but ended up getting a 16bit one eg: #FFFAAA000. This caused a truncation, which leads to wildly off colors.
2023-02-15 12:41:04 +11:00
Ted Johansson 25a226279a
DEV: Replace #pluck_first freedom patch with AR #pick in core (#19893)
The #pluck_first freedom patch, first introduced by @danielwaterworth has served us well, and is used widely throughout both core and plugins. It seems to have been a common enough use case that Rails 6 introduced it's own method #pick with the exact same implementation. This allows us to retire the freedom patch and switch over to the built-in ActiveRecord method.

There is no replacement for #pluck_first!, but a quick search shows we are using this in a very limited capacity, and in some cases incorrectly (by assuming a nil return rather than an exception), which can quite easily be replaced with #pick plus some extra handling.
2023-02-13 12:39:45 +08:00
Ted Johansson a90ad52dff
DEV: Add dedicated category settings model - Part 1 (#20211)
This is the first in a multi-part change to move the custom fields to a new table. It includes:

- Adding a new CategorySetting model and corresponding table.
- Populating it with data from the category_custom_fields table.
2023-02-13 12:37:59 +08:00
Krzysztof Kotlarek cbd021db15
FIX: add index to sidebar_section_link (#20234)
Index on linkable_type and linkable_id should increase performance of this subquery https://github.com/discourse/discourse/blob/main/app/services/sidebar_site_settings_backfiller.rb#L86

Also, distinct is removing duplicates which are unnecessary.
2023-02-10 11:14:22 +11:00
Keegan George 871607a420
DEV: Create form templates (#20189) 2023-02-08 11:21:39 -08:00
Krzysztof Kotlarek 6e1f3e0023
FIX: improvements for user custom sections (#20190)
Improvements for this PR: https://github.com/discourse/discourse/pull/20057

What was fixed:
- [x] Use ember transitions instead of full reload
- [x] Link was inaccurately kept active
- [x] "+ save" renamed to just "save"
- [x] Render emojis in link name
- [x] UI to set icon
- [x] Delete link is trash icon instead of "x"
- [x] Add another link to on the left and rewording
- [x] Raname "link name" -> "name", "points to" ->  link
- [x] Add limits to fields
- [x] Move add section button to the bottom
2023-02-08 11:45:34 +11:00
Sérgio Saquetim 5d32db76dd
DEV: Added .only_deleted scope in the Trashable module (#20196) 2023-02-07 15:28:59 -03:00
Rafael dos Santos Silva 6e522e4aad
DEV: Move to Sass compilation to dart-sass (#19910)
This PR is a major change to Sass compilation in Discourse.

The new version of sass-ruby moves to dart-sass putting we back on the supported version of Sass. It does so while keeping compatibility with the existing method signatures, so minimal change is needed in Discourse for this change.

This moves us

From:
  - sassc 2.0.1 (Feb 2019)
  - libsass 3.5.2 (May 2018)

To:
  - dart-sass 1.58

This update applies the following breaking changes:

> 
> These breaking changes are coming soon or have recently been released:
> 
>  [Functions are stricter about which units they allow](https://sass-lang.com/documentation/breaking-changes/function-units) beginning in Dart Sass 1.32.0.
> 
>  [Selectors with invalid combinators are invalid](https://sass-lang.com/documentation/breaking-changes/bogus-combinators) beginning in Dart Sass 1.54.0.
> 
>  [/ is changing from a division operation to a list separator](https://sass-lang.com/documentation/breaking-changes/slash-div) beginning in Dart Sass 1.33.0.
> 
>  [Parsing the special syntax of @-moz-document will be invalid](https://sass-lang.com/documentation/breaking-changes/moz-document) beginning in Dart Sass 1.7.2.
> 
>  [Compound selectors could not be extended](https://sass-lang.com/documentation/breaking-changes/extend-compound) in Dart Sass 1.0.0 and Ruby Sass 4.0.0.


SCSS files have been migrated automatically using `sass-migrator division app/assets/stylesheets/**/*.scss`
2023-02-07 12:24:57 -03:00
Ted Johansson 676d5fadab
DEV: Limit and validate category settings inputs (#20135)
We recently had a bug which caused auto-bumping to "not work". The problem was that the value had been set to 0.5, which when coerced to an integer turned into 0. So the feature is "working as intended", but there's a possibility of misconfiguration.

When looking into this, I noticed that the inputs on the category settings page doesn't have any particular sanitisation in the front-end, and also one or two validations missing in the back-end.

This change:

- Takes an existing component, NumberField and enhances that by only allowing numeric input, essentially turning it into a managed input using the same approach as our PasswordField.
- Changes the numeric inputs on category settings page to use this component.
- Adds appropriate min constraints to the fields to disallow out-of-range values.
- Adds missing back-end validations to relevant fields.
2023-02-07 12:27:38 +08:00
Blake Erickson c540167982
FIX: Remove action buttons if post has already been reviewed (#20126)
* FIX: Remove action buttons if post has already been reviewed

* Change the approve to reject test to expect an error

* Adds a controller spec to ensure you can't edit a non-pending review item

* Remove unnessary conditional
2023-02-06 11:55:52 -07:00
David Taylor 754d1b71aa
FIX: Ensure ColorScheme#resolve falls back to base for missing color (#20186)
When a CUSTOM_SCHEME is missing a color (e.g. 'Dracula' is missing a 'highlight' color), we need to fallback to `ColorScheme.base_colors`. This regressed in 66256c15bd
2023-02-06 18:24:12 +00:00
Penar Musaraj a86112fc25
FEATURE: Allow embedded view to include a header (#20150)
This commits adds the ability to add a header to the embedded comments
view. One use case for this is to allow `postMessage` communication
between the comments iframe and the parent frame, for example, when
toggling the theme of the parent webpage.
2023-02-06 11:10:50 -05:00
Andrei Prigorshnev 84e13e9b1c
FIX: avoid race condition when setting user status (#19817)
We caught it in logs, race condition led to this error:

    ActiveRecord::RecordNotUnique 
    (PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "user_statuses_pkey"
    DETAIL:  Key (user_id)=(15) already exists.)


The reason the problem happened was that we were checking if a user has status and if not inserting status:

    if user_status
      ...
    else
      self.user_status = UserStatus.create!(status)
    end

The problem is that it's possible that another request will insert status just after we check if status exists and just before our request call `UserStatus.create!(status)`. Using `upsert` fixes the problem because under the hood `upsert` generates the only SQL request that uses "INSERT ... ON CONFLICT DO UPDATE". So we do everything in one SQL query, and that query takes care of resolving possible conflicts.
2023-02-06 18:56:28 +04:00
Gerhard Schlager e17c145e8d
FIX: Category hashtags weren't always found for sub-sub-categories (#20156)
The algorithm failed to find the correct category by slug when there are multiple sub-sub-categories with the same child-category name and the first child doesn't  have the correct grandchild.

So, searching for "child / grandchild" worked in the following case, it found (3):

- (1) parent 1
  - (2) child
    - (3) grandchild
- (4) parent 2
  - (5) child
    - (6) grandchild

But it failed to find the grandchild in the following case:

- (1) parent 1
  - (2) child
- (4) parent 2
  - (5) child
    - (6) grandchild

And this also fixes a flaky spec by forcing categories to always order by by `parent_category_id` and `id`.
This makes it possible to partly revert 60990aab55
2023-02-03 12:17:52 +01:00
Krzysztof Kotlarek 84a87a703c
DEV: configurable custom sidebar sections (#20057)
Allows users to configure their own custom sidebar sections with links withing Discourse instance. Links can be passed as relative path, for example "/tags" or full URL.

Only path is saved in DB, so when Discourse domain is changed, links will be still valid.

Feature is hidden behind SiteSetting.enable_custom_sidebar_sections. This hidden setting determines the group which members have access to this new feature.
2023-02-03 14:44:40 +11:00
Rafael dos Santos Silva e4fd3d9850
FIX: Better ordering of similar user search suggestions (#20142)
* FIX: Better ordering of similar user search suggestions
2023-02-02 14:39:44 -03:00
Rafael dos Santos Silva 14cf8eacf1
FEATURE: Use similarity in user search (#20112)
Currently, when doing `@mention` for users we have 0 tolerance for typos and misspellings.

With this patch, if a user search doesn't return enough results we go and use `pg_trgm` features to try and find more matches based on trigrams of usernames and names.

It also introduces GiST indexes on those fields in order to improve performance of this search, going from 130ms down to 15ms in my tests.

This is all gated in a feature flag and can be enabled by running  `SiteSetting.user_search_similar_results = true` in the rails console.
2023-02-02 13:35:04 -03:00
David Taylor 66256c15bd
UX: Calculate missing hover/selected colors from existing colors (#20105)
`--d-hover` is calculated to be equivalent to primary-100 in light mode, or primary-low in dark mode

`--d-selected` is calculated to be equivalent to primary-low in light mode, or primary-100 in dark mode

`lib/color_math` is introduced to provide some utilities for making these calculations.
2023-02-01 09:55:21 +00:00
Alan Guo Xiang Tan 07ef828db9
DEV: Improve MessageBus subscriptions for TopicTrackingState (#19767)
## Why do we need this change? 

When loading the ember app, [MessageBus does not start polling immediately](f31f0b70f8/app/assets/javascripts/discourse/app/initializers/message-bus.js (L71-L81)) and instead waits for `document.readyState` to be `complete`. What this means is that if there are new messages being created while we have yet to start polling, those messages will not be received by the client.

With sidebar being the default navigation menu, the counts derived from `topic-tracking-state.js` on the client side is prominently displayed on every page. Therefore, we want to ensure that we are not dropping any messages on the channels that `topic-tracking-state.js` subscribes to.  

## What does this change do? 

This includes the `MessageBus.last_id`s for the MessageBus channels which `topic-tracking-state.js` subscribes to as part of the preloaded data when loading a page. The last ids are then used when we subscribe the MessageBus channels so that messages which are published before MessageBus starts polling will not be missed.

## Review Notes

1. See https://github.com/discourse/message_bus#client-support for documentation about subscribing from a given message id.
2023-02-01 07:18:45 +08:00