Commit Graph

9306 Commits

Author SHA1 Message Date
Ted Johansson 437b73e322
SECURITY: Ensure site setting being updated is a configurable site setting (#21131) 2023-04-18 14:32:18 +08:00
Alan Guo Xiang Tan 1f0207ba06
DEV: Add support for more filters for `/filter` route (#21097)
* DEV: Support `likes-(min:max):<count>` on `/filter` route

This commit adds support for the following filters: 

1. `likes-min` 
2. `likes-max`
3. `views-min`
4. `views-max`
5. `likes-op-min`
6. `likes-op-max`

If the filter has an invalid value, i.e string that cannot be converted
into an integer, the filter will be ignored.

If either of each filter is specify multiple times, only the last
occurrence of each filter will be taken into consideration.
2023-04-14 10:21:04 +08:00
Alan Guo Xiang Tan 782b26d0eb
DEV: Support `posters-(min|max):<count>` on `/filter` route (#21095)
This commit adds support for the `posters-min:<count>` and
`posters-max:<count>` filters for the topics filtering query language.
`posters-min:1` will filter for topics with at least a one poster while
`posters-max:3` will filter for topics with a maximum of 3 posters.

If the filter has an invalid value, i.e string that cannot be converted
into an integer, the filter will be ignored.

If either of each filter is specify multiple times, only the last
occurence of each filter will be taken into consideration.
2023-04-14 07:48:38 +08:00
Alan Guo Xiang Tan bc4a9c50f2
DEV: Support `posts-min:<count>` and `posts-max:<count>` on `/filter` (#21090)
This commit adds support for the `posts-min:<count>` and
`posts-max:<count>` filters for the topics filtering query language.
`posts-min:1` will filter for topics with at least a one post while
`posts-max:3` will filter foor topics with a maximum of 3 posts.

If the filter has an invalid value, i.e string that cannot be converted
into an integer, the filter will be ignored.

If either of each filter is specify multiple times, only the last
occurence of each filter will be taken into consideration.
2023-04-14 06:05:55 +08:00
David Taylor e52f322cb5
UX: Use dominant color while loading onebox images (#21091)
When we "pull hotlinked images" on onebox images, they are added to the uploads table and their dominant color is calculated. This commit adds the data to the HTML so that it can be used by the client in the same way as non-onebox images. It also adds specific handling to the new `discourse-lazy-videos` plugin.
2023-04-13 12:04:46 +01:00
David Battersby 967010e545
FEATURE: Add an emoji deny list site setting (#20929)
This feature will allow sites to define which emoji are not allowed. Emoji in this list should be excluded from the set we show in the core emoji picker used in the composer for posts when emoji are enabled. And they should not be allowed to be chosen to be added to messages or as reactions in chat.

This feature prevents denied emoji from appearing in the following scenarios:
- topic title and page title
- private messages (topic title and body)
- inserting emojis into a chat
- reacting to chat messages
- using the emoji picker (composer, user status etc)
- using search within emoji picker

It also takes into account the various ways that emojis can be accessed, such as:
- emoji autocomplete suggestions
- emoji favourites (auto populates when adding to emoji deny list for example)
- emoji inline translations
- emoji skintones (ie. for certain hand gestures)
2023-04-13 15:38:54 +08:00
Alan Guo Xiang Tan 5b1306cb54
DEV: Refactor `TopicsFilter` (#21071)
Why this change?

Previously `TopicsFilter` was designed in such a way that we act on a
filter sequentially based on the order it was matched. However, this
made it hard to support filters composition where a similar filter may
be present further in the query string. Because of this limitation, I
previously introduced a private API `TopicsFilter.register_scope` which
allows us to act on a filter only after the entire query string has been
scanned. However, I felt that it made the code complicated and hard to
reason about.

In thie commit, I've changed it such that we scan through the entire
query string and group the values of each filter together. This allows
us to act on the values of a given filter in one go which I find easier
to reason about. This also opens up the possibility for us to ignore
certain filters when it has been specified multiple times.
2023-04-13 13:22:11 +08:00
David Taylor 121d5c6c6a
UX: Enable new notifications menu by default (#21060)
https://meta.discourse.org/t/260358
2023-04-12 09:45:29 +01:00
Alan Guo Xiang Tan a1524b84e2
DEV: Support `created-by:<username>` filter on `/filter` route (#21067)
This commit adds support for the `created-by:<username>` query filter
which will return topics created by the specified user. Multiple
usernames can be specified by comma seperating the usernames like so:
`created-by:username1,username2`. This will filter for topics created by
either of the specified users. Multiple `created-by:<username>` can also
be composed together. `created-by:username1 created-by:username2` is
equivalent to `created-by:username1,username2`.
2023-04-12 09:25:06 +08:00
Penar Musaraj 0ab3ba5f0d
SECURITY: strip `xlink:href` from uploaded SVGs (#21057)
This was inadvertently removed in 4c46c7e. In very specific scenarios,
this could be used execute arbitrary JavaScript.

Only affects instances where SVGs are allowed as uploads and CDN is not
configured.
2023-04-11 14:10:44 -04:00
David Taylor 9238767f7e
FEATURE: Persist password hashing algorithm/params in database (#20980)
Previously, Discourse's password hashing was hard-coded to a specific algorithm and parameters. Any changes to the algorithm or parameters would essentially invalidate all existing user passwords.

This commit introduces a new `password_algorithm` column on the `users` table. This persists the algorithm/parameters which were use to generate the hash for a given user. All existing rows in the users table are assumed to be using Discourse's current algorithm/parameters. With this data stored per-user in the database, we'll be able to keep existing passwords working while adjusting the algorithm/parameters for newly hashed passwords.

Passwords which were hashed with an old algorithm will be automatically re-hashed with the new algorithm when the user next logs in.

Values in the `password_algorithm` column are based on the PHC string format (https://github.com/P-H-C/phc-string-format/blob/master/phc-sf-spec.md). Discourse's existing algorithm is described by the string `$pbkdf2-sha256$i=64000,l=32$`

To introduce a new algorithm and start using it, make sure it's implemented in the `PasswordHasher` library, then update `User::TARGET_PASSWORD_ALGORITHM`.
2023-04-11 10:16:28 +01:00
Alan Guo Xiang Tan 2809d7ba8e
DEV: Support `in:<notification level>` filter on `/filter` route (#21038)
This commit adds support for the `in:<topic notification level>` query
filter. As an example, `in:tracking` will filter for topics that the
user is watching. Filtering for multiple topic notification levels can
be done by comma separating the topic notification level keys. For
example, `in:muted,tracking` or `in:muted,tracking,watching`.
Alternatively, the user can also compose multiple filters with `in:muted
in:tracking` which translates to the same behaviour as
`in:muted,tracking`.
2023-04-11 08:48:07 +08:00
Daniel Waterworth c68497159f
FIX: Use string for postgres recently readonly DistributedCache (#21040)
Since DistributedCaches don't marshal timestamps
2023-04-10 13:54:55 -05:00
Sérgio Saquetim f8fb7ee9f3
DEV: Introduced topic_query_create_list_topics modifier (#21016)
Introduced a modifier on topic_query to change list while they're created

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2023-04-07 13:01:42 -03:00
Alan Guo Xiang Tan b2a951e4a5
DEV: Support `in:bookmarked` filter for the `/filter` route (#21000)
This filters the topics list to the topics that the current user has bookmarks in.
2023-04-06 12:55:28 +08:00
Alan Guo Xiang Tan ab54a616c1
DEV: Introduce `in:pinned` filter for experimental `/filter` route (#20974)
This commit adds support for the `in:pinned` filter to the topics filtering
query language. When the filter is present, it will filter for topics
where `Topic#pinned_until` is greater than `Topic#pinned_at`.
2023-04-06 10:13:02 +08:00
Alan Guo Xiang Tan 5bec894a8c
PERF: Fix N+1 queries problem when listing topics list (#20971)
This performance regression was introduced in
7c6a8f1c74 where the preloading of tags in
`TopicQuery` was accidentally removed.
2023-04-06 06:58:35 +08:00
Mark VanLandingham 65f35e1ef2
FEATURE: SiteSetting for creation of small action on tag change (#20812)
This adds a SiteSetting, which when enabled, creates a small_action post for tag/category changes to the topic. It uses `topic.add_moderator_post, and passes raw text in, to describe the change.
2023-04-05 13:31:31 -05:00
David Taylor d3e5251704
PERF: Use `OpenSSL::KDF` for Pbkdf2 implementation (#20982)
This was introduced to the standard library in Ruby 2.4. In my testing, it produces the same result, and is around 8x faster than our pure-ruby implementation
2023-04-05 17:00:05 +01:00
David Taylor df849e51b7
DEV: Add plugin hook for transforming site setting defaults (#20941) 2023-04-05 12:28:16 +01:00
Alan Guo Xiang Tan 62696b9ee7
DEV: Properly support composing multiple category filters on `/filter` (#20953)
Before this commit, composing multiple category filters with a query such as category:category1 and category:category2 would not return any results. This is because we were filtering for topics that belonged to both category1 and category2, which is impossible since a topic can only belong to a single category.

With this commit, specifying a query like category:category1 category:category2 will now translate to filtering for topics that belong to either the category1 or category2 category.
2023-04-05 07:16:37 +08:00
Bianca Nenciu 9ff105973f
FEATURE: Allow invite only and Discourse connect (#20961)
Invite only and Discourse connect could not be enabled at the same time
because of some legacy reason. This is a follow up commit to ce04db8,
355d51a and 40f6ceb.
2023-04-04 19:52:11 +03:00
Jarek Radosz 29e2e3ff3b
DEV: Fix random typos (#20937) 2023-04-03 19:27:32 +02:00
Roman Rizzi ecac62dd6f
DEV: Make search results blurb non-pg headlines setting dependent (#20939)
Followup to #20915. If we're grouping search results that don't rely on core's search, we won't have access to pg headlines. This is now configurable via the constructor, defaulting to `SiteSetting.use_pg_headlines_for_excerpt`
2023-04-03 11:09:36 -03:00
David Taylor f3402be262
DEV: Introduce `Migration::Helpers` for new-site detection (#20934)
We use schema_migration_details to determine the age of a site in multiple migrations. This commit moves the logic into a dedicated `Migration::Helpers` module so that it doesn't need to be re-implemented every time.
2023-04-03 11:46:39 +01:00
Alan Guo Xiang Tan fd34032db2
DEV: Support filter for topics in specific subcategories on /filter (#20927)
This commit adds support for filtering for topics in specific
subcategories via the categories filter query language.

For example: `category:documentation:admins` will filter for topics and
subcategory topics in
the category with slug "admins" whose parent category has the slug
"documentation".

The `=` prefix can also be used such that
`=category:documentation:admins` will exclude subcategory topics of the
category with slug "admins" whose parent category has the slug
"documentation".
2023-04-03 18:36:59 +08:00
Alan Guo Xiang Tan 0162f0ccb0
DEV: Update experimental `/filter` route with categories support (#20911)
On the `/filter` route, the categories filtering query language is now
supported in the input per the example provided below:

```
category:bug => topics in the bug category AND all subcategories
=category:bug => topics in the bug category excluding subcategories
category:bug,feature => allow for categories either in bug or feature
=category:bug,feature => allow for exact categories match excluding sub cats
categories: => alias for category
```

Currently composing multiple category filters is not supported as we
have yet to determine what behaviour it should result in. For example,
`category:bug category:feature` would now return topics that are in both
the `bug` and `feature` category but it is not possible for a topic to
belong to two categories.
2023-03-31 14:32:12 +08:00
Sam 347681dd20
DEV: add topic_query_suggested_options modifier (#20893)
Add a modifier that will allow us to tune the results returned by suggested.

At the moment the modifier allows us to toggle including random results.
This was created for the discourse-ai module. It needs to switch off random
results when it returns related topics.

Longer term we can use it to toggle unread/new and other aspects.

This also demonstrates how to test the contract when adding modifiers.
2023-03-31 09:03:15 +11:00
Daniel Waterworth 2df0eca39a
DEV: Make postgres_readonly cache work like other caches (#20879)
We didn't have an authoritative source for this data previously, so now
it's stored in redis.
2023-03-30 09:14:59 -05:00
Renato Atilio 7afcb664fb
FIX: pretty-text shims - getURL's baseUri (#20822) 2023-03-30 08:35:06 -04:00
Alan Guo Xiang Tan a154d7207d
DEV: Improve test coverage for `/filters` route (#20890) 2023-03-30 12:57:23 +08:00
Martin Brennan e6feb3f136
DEV: Remove currentUser lookup in pretty-text (#20894)
This was added in d3f02a1270
for hashtags but later removed usage in
b2acc416e7. It was removed because
serializing the user does not include things like their
secure_categories.

It is not used by any other plugins or themes, and can cause
issues where it will error when operating on a null user. Better
to just pass in the user_id and use it to look up a user
directly in a PrettyText::Helper
2023-03-30 14:50:36 +10:00
Martin Brennan eea74e0e32
DEV: Add auto _map extension for simple/compact list SiteSettings (#20888)
Similar to the _map added for group_list SiteSettings in
e62e93f83a, this commit adds
the same extension for simple and compact `list` type SiteSettings,
so developers do not have to do the `.to_s.split("|")` dance
themselves all the time.

For example:

```
SiteSetting.markdown_linkify_tlds

=> "com|net|org|io|onion|co|tv|ru|cn|us|uk|me|de|fr|fi|gov|ddd"

SiteSetting.markdown_linkify_tlds_map

=> ["com", "net", "org", "io", "onion", "co", "tv", "ru", "cn", "us", "uk", "me", "de", "fr", "fi", "gov"]
```
2023-03-30 14:08:19 +10:00
Martin Brennan 84ff96bd07
FIX: Do not validate email in TL promotion (#20892)
There is no need to validate the user's emails when
promoting/demoting their trust level, this can cause
issues in things like Jobs::Tl3Promotions, we don't
need to fail in that case when all we are doing is changing
trust level.
2023-03-30 13:52:10 +10:00
Sam 795e6d72a4
FEATURE: modifier API for plugins (#20887)
Introduces a new API for plugin data modification without class-based extension overhead.

This commit introduces a new API that allows plugins to modify data in cases where they return different data rather than additional data, as is common with filtered_registers in DiscoursePluginRegistry. This API removes the need for defining class-based extension points.

When a plugin registers a modifier, it will automatically be called if the plugin is enabled. The core will then modify the parameter sent to it using the block registered by the plugin:
 
```ruby
DiscoursePluginRegistry.register_modifier(plugin_instance, :magic_sum_modifier) { |a, b| a + b }
sum = DiscoursePluginRegistry.apply_modifier(:magic_sum_filter, 1, 2)
expect(sum).to eq(3)
```

Key features of these modifiers:

- Operate in a stack (first registered, first called)
- Automatically disabled when the plugin is disabled
- Pass the cumulative result of all block invocations to the caller
2023-03-30 14:39:55 +11:00
Alan Guo Xiang Tan 4e11014693
DEV: Support `status:public` in topics filtering query language (#20889)
This commit adds support for the `status:public` filter which only
return topics that belong to public categories.
2023-03-30 10:57:26 +08:00
Alan Guo Xiang Tan 49e7e639cc
DEV: Update experimental `/filter` route with tags support (#20874)
The following are the changes being introduced in this commit:

1. Instead of mapping the query language to various query params on the
client side, we've decided that the benefits of having a more robust
query language far outweighs the benefits of having a more human readable query params in the URL.
As such, the `/filter` route will just accept a single `q` query param
and the query string will be parsed on the server side.

1. On the `/filter` route, the tags filtering query language is now
   supported in the input per the example provided below:

   ```
   tags:bug+feature tagged both bug and feature
   tags:bug,feature tagged either bug or feature
   -tags:bug+feature excluding topics tagged bug and feature
   -tags:bug,feature excluding topics tagged bug or feature
   ```

   The `tags` filter can also be specified multiple
times in the query string like so `tags:bug tags:feature` which will
filter topics that contain both the `bug` tag and `feature` tag. More
complex query like `tags:bug+feature -tags:experimental` will also work.
2023-03-30 09:00:42 +08:00
Jan Cernik afe3e36363
DEV: Remove lazy-yt and replace with lazy-videos (#20722)
- Refactors the old plugin to remove jquery usage
- Adds support for Vimeo videos (default on) and Tiktok (experimental and default off)
2023-03-29 11:54:25 -04:00
Sam ddec7bf6a3
DEV: allow API for list_suggested_for to exclude random (#20857)
This is needed so plugins can potentially create lists without random topics
2023-03-28 15:52:17 +11:00
Daniel Waterworth 9d34b73a5e
DEV: Make csp cache work like other caches (#20818) 2023-03-27 09:22:38 -05:00
Alan Guo Xiang Tan 4624cca00f
DEV: Fix `TopcisFilter#filter_tags` not working for a single tag (#20840)
Follow-up to dd88fdeabc
2023-03-27 16:58:40 +08:00
Alan Guo Xiang Tan dd88fdeabc
DEV: Introduce `TopicsFilter#filter_tags` method (#20839)
This change sets the ground work for allowing us to filter topics list
by tags in the following ways:

1. Filter for topics that matches all tags in a given set of tags
2. Filter for topics that matches any tags in a given set of tags
3. Exclude topics that matches all tags in a given set of tags
4. Exclude topics that matches any tags in a given set of tags
2023-03-27 14:16:53 +08:00
Krzysztof Kotlarek 4047073292
FIX: display validation under custom sidebar fields (#20772)
Before, incorrectly filled fields were marked with red border. Now, additional information under the field is displayed to notify the user what is incorrect.

/t/93696
2023-03-27 13:03:16 +11:00
Alan Guo Xiang Tan 56fbdde0e5
FIX: Broken `?status=(listed|unlisted)` query param support (#20834)
In 66c5054, the support for filtering a
topics list based on the visible attribute of a topic via the status query param
was accidentally removed.
2023-03-27 07:30:19 +08:00
Vinoth Kannan ed6f7b1e6d
FIX: use correct validation method name for `default_categories_normal`. (#20801)
When we renamed the `default_categories_regular` to `default_categories_normal` we missed a site setting validation method. It allowed the duplicate category ids in `default_categories_normal` site setting and caused the problem in user registration process.

5176c689e9
2023-03-24 11:40:37 +05:30
Alan Guo Xiang Tan 7c6a8f1c74
DEV: Refactor `TopicQuery#default_results` to improve readability (#20800)
This commit breaks down the code in TopicQuery#default_results into
smaller private methods to improve readability.
2023-03-24 11:17:55 +08:00
Martin Brennan 97f8f88cfe
FIX: ACL for OptimizedImage was using wrong path on multisite (#20784)
When setting the ACL for optimized images after setting the
ACL for the linked upload (e.g. via the SyncACLForUploads job),
we were using the optimized image path as the S3 key. This worked
for single sites, however it would fail silently for multisite
sites since the path would be incorrect, because the Discourse.store.upload_path
was not included.

For example, something like this:

somecluster1/optimized/2X/1/3478534853498753984_2_1380x300.png

Instead of:

somecluster1/uploads/somesite1/2X/1/3478534853498753984_2_1380x300.png

The silent failure is still intentional, since we don't want to
break other things because of ACL updates, but now we will update
the ACL correctly for optimized images on multisite sites.
2023-03-24 10:16:53 +10:00
David Taylor b81767c1b1
SECURITY: Limit URL length for theme remote (#20787) 2023-03-23 12:01:04 +00:00
Sam d87e78616d
FEATURE: allow site owners to disable impersonation (#20783)
Site owners can now disable impersonation using the global setting
`allow_impersonation` (Eg: DISCOURSE_ALLOW_IMPERSONATION: false)

see:

https://meta.discourse.org/t/thoughts-about-impersonate-user/258795
2023-03-23 15:16:05 +11:00
Jarek Radosz 627f69738f
DEV: Capture output in hashtags spec (#20773) 2023-03-23 11:47:14 +10:00