Commit Graph

10473 Commits

Author SHA1 Message Date
Sam 347681dd20
DEV: add topic_query_suggested_options modifier (#20893)
Add a modifier that will allow us to tune the results returned by suggested.

At the moment the modifier allows us to toggle including random results.
This was created for the discourse-ai module. It needs to switch off random
results when it returns related topics.

Longer term we can use it to toggle unread/new and other aspects.

This also demonstrates how to test the contract when adding modifiers.
2023-03-31 09:03:15 +11:00
Natalie Tay 068a36d354
UX: Improve error message when a topic cannot be moved due to category restrictions (#20900) 2023-03-31 02:18:57 +08:00
Daniel Waterworth 2df0eca39a
DEV: Make postgres_readonly cache work like other caches (#20879)
We didn't have an authoritative source for this data previously, so now
it's stored in redis.
2023-03-30 09:14:59 -05:00
Mark VanLandingham 9518e47204
FEATURE: ability to bulk_remove users from a group (#20876) 2023-03-30 08:06:36 -05:00
Renato Atilio 7afcb664fb
FIX: pretty-text shims - getURL's baseUri (#20822) 2023-03-30 08:35:06 -04:00
Sam 74312a6c52
DEV: stop getting topic.url in specs (#20895)
topic.url is a dangerous method to use, it is incompatible with our `sign_in`
helper.

This removes a bunch of places this was erroneously used and helps to avoid
future cargo culting.
2023-03-30 18:09:19 +11:00
Alan Guo Xiang Tan a154d7207d
DEV: Improve test coverage for `/filters` route (#20890) 2023-03-30 12:57:23 +08:00
Martin Brennan eea74e0e32
DEV: Add auto _map extension for simple/compact list SiteSettings (#20888)
Similar to the _map added for group_list SiteSettings in
e62e93f83a, this commit adds
the same extension for simple and compact `list` type SiteSettings,
so developers do not have to do the `.to_s.split("|")` dance
themselves all the time.

For example:

```
SiteSetting.markdown_linkify_tlds

=> "com|net|org|io|onion|co|tv|ru|cn|us|uk|me|de|fr|fi|gov|ddd"

SiteSetting.markdown_linkify_tlds_map

=> ["com", "net", "org", "io", "onion", "co", "tv", "ru", "cn", "us", "uk", "me", "de", "fr", "fi", "gov"]
```
2023-03-30 14:08:19 +10:00
Martin Brennan 84ff96bd07
FIX: Do not validate email in TL promotion (#20892)
There is no need to validate the user's emails when
promoting/demoting their trust level, this can cause
issues in things like Jobs::Tl3Promotions, we don't
need to fail in that case when all we are doing is changing
trust level.
2023-03-30 13:52:10 +10:00
Sam 795e6d72a4
FEATURE: modifier API for plugins (#20887)
Introduces a new API for plugin data modification without class-based extension overhead.

This commit introduces a new API that allows plugins to modify data in cases where they return different data rather than additional data, as is common with filtered_registers in DiscoursePluginRegistry. This API removes the need for defining class-based extension points.

When a plugin registers a modifier, it will automatically be called if the plugin is enabled. The core will then modify the parameter sent to it using the block registered by the plugin:
 
```ruby
DiscoursePluginRegistry.register_modifier(plugin_instance, :magic_sum_modifier) { |a, b| a + b }
sum = DiscoursePluginRegistry.apply_modifier(:magic_sum_filter, 1, 2)
expect(sum).to eq(3)
```

Key features of these modifiers:

- Operate in a stack (first registered, first called)
- Automatically disabled when the plugin is disabled
- Pass the cumulative result of all block invocations to the caller
2023-03-30 14:39:55 +11:00
Alan Guo Xiang Tan 4e11014693
DEV: Support `status:public` in topics filtering query language (#20889)
This commit adds support for the `status:public` filter which only
return topics that belong to public categories.
2023-03-30 10:57:26 +08:00
Alan Guo Xiang Tan 49e7e639cc
DEV: Update experimental `/filter` route with tags support (#20874)
The following are the changes being introduced in this commit:

1. Instead of mapping the query language to various query params on the
client side, we've decided that the benefits of having a more robust
query language far outweighs the benefits of having a more human readable query params in the URL.
As such, the `/filter` route will just accept a single `q` query param
and the query string will be parsed on the server side.

1. On the `/filter` route, the tags filtering query language is now
   supported in the input per the example provided below:

   ```
   tags:bug+feature tagged both bug and feature
   tags:bug,feature tagged either bug or feature
   -tags:bug+feature excluding topics tagged bug and feature
   -tags:bug,feature excluding topics tagged bug or feature
   ```

   The `tags` filter can also be specified multiple
times in the query string like so `tags:bug tags:feature` which will
filter topics that contain both the `bug` tag and `feature` tag. More
complex query like `tags:bug+feature -tags:experimental` will also work.
2023-03-30 09:00:42 +08:00
Jan Cernik afe3e36363
DEV: Remove lazy-yt and replace with lazy-videos (#20722)
- Refactors the old plugin to remove jquery usage
- Adds support for Vimeo videos (default on) and Tiktok (experimental and default off)
2023-03-29 11:54:25 -04:00
Blake Erickson aa09a78d14
DEV: Remove diffhtml_preview for video thumbnails (#20865)
Video thumbnail generation is no longer dependent on
`enable_diffhtml_preview` being enabled.
2023-03-28 12:38:50 -06:00
Rafael dos Santos Silva 2a7bdb2d66
FIX: Push notification delay should not be longer than specified (#20864)
When user.last_seen was less than push_notification_time_window_mins we
where delaying the notification for the whole
push_notification_time_window_mins PLUS the time the user was away from.

Originally reported in https://meta.discourse.org/t/-/259688
2023-03-28 13:22:54 -03:00
Vinoth Kannan 08ff6eebad
FIX: skip category preference update if already set by group. (#20823)
`default_categories_*` site settings will update the category preferences on user creation. But it shouldn't update the user's category preference if a group's setting already updated it for that user.
2023-03-28 19:43:01 +05:30
Sam ddec7bf6a3
DEV: allow API for list_suggested_for to exclude random (#20857)
This is needed so plugins can potentially create lists without random topics
2023-03-28 15:52:17 +11:00
Penar Musaraj daaa41ae07
WIP: Skip system specs with upload fabricators (#20821)
A shot in the dark to see if these are the culprits for the many random
spec failures in CI.
2023-03-27 13:11:17 -04:00
Alan Guo Xiang Tan 4624cca00f
DEV: Fix `TopcisFilter#filter_tags` not working for a single tag (#20840)
Follow-up to dd88fdeabc
2023-03-27 16:58:40 +08:00
Alan Guo Xiang Tan dd88fdeabc
DEV: Introduce `TopicsFilter#filter_tags` method (#20839)
This change sets the ground work for allowing us to filter topics list
by tags in the following ways:

1. Filter for topics that matches all tags in a given set of tags
2. Filter for topics that matches any tags in a given set of tags
3. Exclude topics that matches all tags in a given set of tags
4. Exclude topics that matches any tags in a given set of tags
2023-03-27 14:16:53 +08:00
Krzysztof Kotlarek 4047073292
FIX: display validation under custom sidebar fields (#20772)
Before, incorrectly filled fields were marked with red border. Now, additional information under the field is displayed to notify the user what is incorrect.

/t/93696
2023-03-27 13:03:16 +11:00
Alan Guo Xiang Tan db3d30af3a
DEV: Skip system tests timing out on CI (#20835) 2023-03-27 08:22:37 +08:00
Alan Guo Xiang Tan 56fbdde0e5
FIX: Broken `?status=(listed|unlisted)` query param support (#20834)
In 66c5054, the support for filtering a
topics list based on the visible attribute of a topic via the status query param
was accidentally removed.
2023-03-27 07:30:19 +08:00
Daniel Waterworth bc8ab33708
DEV: Disable test to see if this is what's causing ci failures (#20809) 2023-03-24 11:15:30 -05:00
Vinoth Kannan ed6f7b1e6d
FIX: use correct validation method name for `default_categories_normal`. (#20801)
When we renamed the `default_categories_regular` to `default_categories_normal` we missed a site setting validation method. It allowed the duplicate category ids in `default_categories_normal` site setting and caused the problem in user registration process.

5176c689e9
2023-03-24 11:40:37 +05:30
Krzysztof Kotlarek 4929288bdd
FIX: delay custom section reorder (#20799)
Reorder should start after 300ms.
In addition, pointer events should be blocked to not open link after reorder is finished.
2023-03-24 13:58:05 +11:00
Martin Brennan 97f8f88cfe
FIX: ACL for OptimizedImage was using wrong path on multisite (#20784)
When setting the ACL for optimized images after setting the
ACL for the linked upload (e.g. via the SyncACLForUploads job),
we were using the optimized image path as the S3 key. This worked
for single sites, however it would fail silently for multisite
sites since the path would be incorrect, because the Discourse.store.upload_path
was not included.

For example, something like this:

somecluster1/optimized/2X/1/3478534853498753984_2_1380x300.png

Instead of:

somecluster1/uploads/somesite1/2X/1/3478534853498753984_2_1380x300.png

The silent failure is still intentional, since we don't want to
break other things because of ACL updates, but now we will update
the ACL correctly for optimized images on multisite sites.
2023-03-24 10:16:53 +10:00
Mark VanLandingham ebada4a6b0
DEV: More specific API to including extra associations in CategoryList (#20790) 2023-03-23 12:39:38 -05:00
David Taylor b81767c1b1
SECURITY: Limit URL length for theme remote (#20787) 2023-03-23 12:01:04 +00:00
Alan Guo Xiang Tan 2d46824a87
DEV: Switch to data attributes to represent sidebar section name (#20771)
Data attributes are less restrictive than the class attribute.
2023-03-23 13:09:45 +08:00
Sam d87e78616d
FEATURE: allow site owners to disable impersonation (#20783)
Site owners can now disable impersonation using the global setting
`allow_impersonation` (Eg: DISCOURSE_ALLOW_IMPERSONATION: false)

see:

https://meta.discourse.org/t/thoughts-about-impersonate-user/258795
2023-03-23 15:16:05 +11:00
Jarek Radosz 627f69738f
DEV: Capture output in hashtags spec (#20773) 2023-03-23 11:47:14 +10:00
Alan Guo Xiang Tan 4cb79ce8be
UX: Styling adjustments for filters page (#20768)
Change styling of filter input & remove button.

This follows the same pattern of design we use for search. In the search dropdown we do not have a button to search. We rely on pressing enter. I've also provided an example of Github's PR filter UI at the bottom of this comment.

We also do not have buttons like this on any other topic-list header. On tag and category dropdowns, we also rely on pressing enter to filter the topic list by chosen categories & tags.

Co-authored-by: Jordan Vidrine <jordan@jordanvidrine.com>
2023-03-23 06:28:47 +08:00
Mark VanLandingham 32aa821f12
DEV: Add preload API to CategoryList (#20778) 2023-03-22 15:12:08 -05:00
Bianca Nenciu f12e77d500
FIX: Do not allow anonymous users to be anonymized (#20776) 2023-03-22 20:51:42 +02:00
David Taylor 7070f81596
DEV: Allow `DROP NOT NULL` in pre-deploy migrations (#20775)
Our SafeMigrate system is designed to prevent tables/columns being dropped in pre-deploy migrations. Its regex-based detection was triggering incorrectly on `ALTER COLUMN DROP NOT NULL`.
2023-03-22 14:43:32 +00:00
Alan Guo Xiang Tan b06e31f8e7
DEV: Remove experimental support for query string on `/filter` route (#20632) 2023-03-22 10:04:57 +08:00
Daniel Waterworth 7b01576c8d
DEV: Remove emoji cache dead code (#20764)
The cache is already not shared between app servers that have different
app_versions, so this check was redundant.
2023-03-21 12:33:12 -05:00
Daniel Waterworth a0a6f6d71b
FIX: Fix the emoji toned regexes (#20763) 2023-03-21 11:48:55 -05:00
Régis Hanol 37609897e8
FEATURE: log manual bounce reset (#20758)
DEV: rename the route "/admin/users/:id/reset_bounce_score" to use dashes instead of underscores
2023-03-21 15:26:26 +01:00
Krzysztof Kotlarek 1859025228
FIX: my links in sidebar section (#20754)
Links like `/my/preferences` were invalid in custom section. The reason is that `/my` links are just redirects from backend, and they are not recognized as valid Ember paths.

https://github.com/discourse/discourse/blob/main/config/routes.rb#L433

Therefore, regex match allowlist was added - similar to backend check:

https://github.com/discourse/discourse/blob/main/app/controllers/users_controller.rb#L471

/safe-mode is same case
2023-03-21 15:58:42 +11:00
Krzysztof Kotlarek db74e9484b
FEATURE: ability to reorder links in custom sidebar sections (#20626)
Drag and drop to reorder custom sidebar sections
2023-03-21 12:23:28 +11:00
Isaac Janzen ca4b73d20c
FIX: Support tag query param on `/tag/{name}` routes (#20742) 2023-03-20 13:51:39 -05:00
Daniel Waterworth da0d20d4a9
DEV: Refactor svg sprite parsing (#20727)
There was a lot of duplication in the svg parsing and coercion code. This reduces that duplication and causes svg sprite parsing to happen earlier so that more computation is cached.
2023-03-20 11:41:23 -05:00
Joffrey JAFFEUX 133ea4cfec
DEV: handles presence channel configured with everyone group (#20741)
This commit will allow any connected user to access a presence channel configured with the automatic group "everyone"
2023-03-20 16:56:43 +01:00
Sam 4a3c13a37b
FIX: search index failing on certain posts (#20736)
During search indexing we "stuff" the index with additional keywords for
entities that look like domain names.

This allows searches for `cnn` to find URLs for `www.cnn.com`

The search stuffing attempted to keep indexes aligned at the correct positions
by remapping the indexed terms. However under certain edge cases a single
word can stem into 2 different lexemes. If this happened we had an off by
one which caused the entire indexing to fail.

We work around this edge case (and carry incorrect index positions) for cases
like this. It is unlikely to impact search quality at all given index position
makes almost no difference in the search algorithm.
2023-03-20 15:43:08 +11:00
Ted Johansson 39c2f63b35 SECURITY: Add FinalDestination::FastImage that's SSRF safe 2023-03-16 15:27:09 -06:00
Alan Guo Xiang Tan fd16eade7f SECURITY: SSRF protection bypass with IPv4-mapped IPv6 addresses
As part of this commit, we've also expanded our list of private IP
ranges based on
https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
and https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml
2023-03-16 15:27:09 -06:00
Alan Guo Xiang Tan 52ef44f43b SECURITY: Monkey-patch web-push gem to use safer HTTP client
`FinalDestination::HTTP` is our patch of `Net::HTTP` which defend us
against SSRF and DNS rebinding attacks.
2023-03-16 15:27:09 -06:00
Loïc Guitaut 0bd64788d2 SECURITY: Rate limit the creation of backups 2023-03-16 16:09:22 +01:00