Commit Graph

51492 Commits

Author SHA1 Message Date
Ted Johansson e910dd0914 SECURITY: Prevent Onebox cache overflow by limiting downloads and URL lengths 2023-11-09 13:47:28 +11:00
Penar Musaraj 89a2e60706 SECURITY: Limit height of pre/svg elements
Ensures posts cannot have SVG or PRE elements that are too tall.
2023-11-09 13:47:26 +11:00
Martin Brennan 2c45b949ea SECURITY: Filter unread bookmark reminders the user cannot see
There is an edge case where the following occurs:

1. The user sets a bookmark reminder on a post/topic
2. The post/topic is changed to a PM before or after the reminder
   fires, and the notification remains unread by the user
3. The user opens their bookmark reminder notification list
   and they can still see the notification even though they cannot
   access the topic anymore

There is a very low chance for information leaking here, since
the only thing that could be exposed is the topic title if it
changes to something sensitive.

This commit filters the bookmark unread notifications by using
the bookmarkable can_see? methods and also prevents sending
reminder notifications for bookmarks the user can no longer see.
2023-11-09 13:47:24 +11:00
Roman Rizzi 628b293ff5 SECURITY: Onebox templates' HTML injections (stable).
The use of triple-curlies on Mustache templates opens the possibility for HTML injections.
2023-11-09 13:47:23 +11:00
Krzysztof Kotlarek 24cca10da7 SECURITY: SSRF vulnerability in TopicEmbed
Block redirects when making the final request in TopicEmbed to prevent Server Side Request Forgery (SSRF)
2023-11-09 13:47:21 +11:00
Régis Hanol 2ec2510517 SECURITY: escape display names
Ensure we escape the display names before passing it to the regexp used to update
quotes whenever a user change their display name.
2023-11-09 13:47:20 +11:00
Discourse Translator Bot 332f562703
Update translations (#24269) 2023-11-07 21:31:58 +01:00
Discourse Translator Bot f6ddfcb46f
Update translations (#24079) 2023-10-24 15:53:39 +02:00
Discourse Translator Bot ef3f415439
Update translations (#23956) 2023-10-17 11:10:09 -04:00
Penar Musaraj 808ed03993
Bump version to v3.1.2 2023-10-16 11:23:14 -04:00
Alan Guo Xiang Tan f2e52ed33c
SECURITY: Add a default limit as to when logs should be truncated
Why this change?

This ensures that malicious requests cannot end up causing the logs to
quickly fill up. The default chosen is sufficient for most legitimate
requests to the Discourse application.

When truncation happens, parsing of logs in supported format like
lograge may break down.
2023-10-16 10:51:31 -04:00
Kelvin Tan 628b320087
SECURITY: Prevent unauthorized access to grouped poll results
This adds access controls for the `/polls/grouped_poll_results`
endpoint, such that only users with appropriate permissions can read
the grouped results of a given poll.
2023-10-16 10:51:29 -04:00
Alan Guo Xiang Tan 0b84353162
SECURITY: Prevent arbitrary topic custom fields from being set
Why this change?

The `PostsController#create` action allows arbitrary topic custom fields
to be set by any user that can create a topic. Without any restrictions,
this opens us up to potential security issues where plugins may be using
topic custom fields in security sensitive areas.

What does this change do?

1. This change introduces the `register_editable_topic_custom_field` plugin
API which allows plugins to register topic custom fields that are
editable either by staff users only or all users. The registered
editable topic custom fields are stored in `DiscoursePluginRegistry` and
is called by a new method `Topic#editable_custom_fields` which is then
used in the `PostsController#create` controller action. When an unpermitted custom fields is present in the `meta_data` params,
a 400 response code is returned.

2. Removes all reference to `meta_data` on a topic as it is confusing
   since we actually mean topic custom fields instead.
2023-10-16 10:51:28 -04:00
David Taylor 157a321322
SECURITY: Correctly escape 'text' email preview (stable) 2023-10-16 10:51:26 -04:00
Bianca Nenciu c9888163d7
SECURITY: Hide user profiles from public
User profiles, including the summary, should be private to anonymous
users if hide_user_profiles_from_public is enabled.
2023-10-16 10:51:25 -04:00
Jan Cernik 265b3dbb4c
SECURITY: Add permissions to MessageBus in chat (stable)
Add spec

compact
2023-10-16 10:51:23 -04:00
Alan Guo Xiang Tan 5d7d607b5f DEV: Add hidden `cross_origin_opener_policy_header` site setting (#23346)
Why this change?

As part of our ongoing efforts to security harden the Discourse
application, we are adding the `cross_origin_opener_policy_header` site setting
which allows the `Cross-Origin-Opener-Policy` response header to be set on requests
that preloads the Discourse application. In more technical terms, only
GET requests that are not json or xhr will have the response header set.

The `cross_origin_opener_policy_header` site setting is hidden for now
for testing purposes and will either be released as a public site
setting or be remove if we decide to be opinionated and ship a default
for the `Cross-Origin-Opener-Policy` response header.
2023-10-11 14:51:28 -07:00
Discourse Translator Bot ec9911a48c
Update translations (#23874) 2023-10-11 11:18:08 +02:00
Discourse Translator Bot 4648e5855c
Update translations (#23758) 2023-10-04 09:54:27 +02:00
Discourse Translator Bot 0612f0d5b6
Update translations (#23628) 2023-09-27 11:03:06 +02:00
Roman Rizzi 61b51f9eb6
Bump version to v3.1.1 2023-09-12 18:50:58 -03:00
Bianca Nenciu 5dbe3b7b55
SECURITY: Add limits for themes and theme assets
This commit adds limits to themes and theme components on the:

- file size of about.json and .discourse-compatibility
- file size of theme assets
- number of files in a theme
2023-09-12 15:35:50 -03:00
Gerhard Schlager 2232e15020
SECURITY: Limit number of drafts per user and length of `draft_key`
The hidden site setting max_drafts_per_user defaults to 10_000 drafts per user.
The longest key should be "topic_<MAX_BIG_INT>" which is 25 characters.
2023-09-12 15:35:47 -03:00
Daniel Waterworth fed34a330b
SECURITY: Reduce maximum size of SVG sprite cache to prevent DoS
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-09-12 15:35:45 -03:00
Daniel Waterworth ce4c47e76e
PERF: Cache each theme field value once (#23192)
Previously, theme fields from components would be cached for each of
their parent themes.
2023-09-12 15:35:45 -03:00
OsamaSayegh 48316d75cd
SECURITY: Limit name field length of TOTP authenticators and security keys 2023-09-12 15:35:42 -03:00
Discourse Translator Bot d3c29c02b9
Update translations (#23539) 2023-09-12 15:27:53 +02:00
David Taylor 457b10e68a DEV: Make `navigateToTopic` more robust for themes/plugins (#22992)
This function was previously expecting multiple services to be injected on any class that uses it. This kind of hidden requirement leads to some very difficult-to-debug situations, so this commit updates the function to lookup all its required services inline.
2023-09-06 08:01:46 -07:00
Jeff Wong 28b632aceb Add plugin outlet for after-panel-body in user menu
Similar to panel-body-bottom but shows up outside the div, and
shows even during EmptyStateComponent is shown.
2023-09-05 18:21:58 -07:00
Jeff Wong ebbd0a3bc3 FIX: tests 2023-09-05 18:21:21 -07:00
Jeff Wong 24431f47a3 FEATURE: add silence reason dropdown to admin penalty reason
Adds dropdown list for pre-defined penalty options to silence to
mirror options on suspension list.
2023-09-05 18:21:21 -07:00
Discourse Translator Bot e74d727e2c
Update translations (#23409) 2023-09-05 15:42:40 +02:00
Discourse Translator Bot 714bae7f94
Update translations (#23310) 2023-08-29 15:50:57 +02:00
Alan Guo Xiang Tan f81ed652f0 DEV: Fix chromedriver binary errors when running system tests in parallel (#23122)
What is the problem here?

The `selenium-webdriver` gem is responsible for downloading the
right version of the `chromedriver` binary and it downloads it into the
`~/.cache/selenium` folder. THe problem here is that when a user runs `bin/turbo_rspec spec/system`
for the first time, all of the processes will try to download the
`chromedriver` binary to the same path at the same time and will lead
to concurrency errors.

What is the fix here?

Before running any RSpec suite, we first check if the `.cache/selenium`
folder is present. If it is not present, we use a file system lock to
download the `chromedriver` binary such that other processes that runs
after will not need to install the `chromedriver` binary.

The long term fix here is to get `selenium-manager` to download the `chromedriver` binary to a unique path for each
process but the `--cache-path` option for `selenium-manager` is currently not supported in `selenium-webdriver`.
2023-08-24 11:24:32 +08:00
Mark VanLandingham 4290036c0c DEV: Bump selenium-webdriver version to fix system spec running (#23117)
We can no long user Webdriver - SeleniumHQ/selenium#11066. Bumping selenium-webdriver did the trick, as well as manually setting the user_agent for mobile system specs. Unsure what changed to make this necessary, but it is necessary to get the app to boot in mobile view.
2023-08-24 11:24:32 +08:00
David Taylor 210f4ad3e7
FIX: Ensure service-worker cache is cleaned correctly (stable) (#23205)
By default, the workbox-expiration plugin will not expire cache entries which include a `Vary` header in the response. This means that cached entries can build up until the browser storage quota is hit.

This commit introduces the `ignoreVary: true` option, so that deletion is performed correctly. This will only apply going forward, so this commit also bumps the cache version and deletes the old caches.

Ref https://github.com/GoogleChrome/workbox/issues/2206
2023-08-23 14:58:28 +01:00
Discourse Translator Bot 45c4ce64c4
Update translations (#23182) 2023-08-23 09:29:44 +02:00
Penar Musaraj bf3e908f66
FIX: Compact tag picker input not focused in iOS (#22922) (#23090)
Should fix an iOS regression in f5e8e73. iOS does not pull up the keyboard if the `.focus()` call is delayed by a rendering timeout or an asynchronous ajax call. This PR adds earlier `.focus()` calls if the input element is present.
2023-08-18 11:23:06 -04:00
Discourse Translator Bot 14f6dcb4d0
Update translations (#23100) 2023-08-15 21:25:02 +02:00
Discourse Translator Bot 0467def2d2
Update translations (#23014) 2023-08-08 15:42:34 +02:00
Jarek Radosz 268efcccdd
FIX: Poll breakdown regressions (#22957)
Some related to the modal api change, some due to chart.js updates
2023-08-03 17:17:09 +02:00
David Taylor 29805c31b8
DEV: Fix theme error message (#22956) (#22958)
Since the refactoring in f822a933fa, the text of theme-related errors has been missing in the UI.
2023-08-03 16:10:44 +01:00
Jarek Radosz 73ee3c1499
FIX: Make poll breakdown modal closable again (#22953)
Regressed in https://github.com/discourse/discourse/pull/22164
2023-08-03 10:34:00 +02:00
Alan Guo Xiang Tan 93cc9276ea
PERF: Improve performance of queries when loading a topic list (#22949) (#22950)
What is the context of this change?

Before 7c6a8f1c74, we were using
`preload(:tags)` on the topics relation but that was accidentally
removed in the refactor. This was discovered and fixed in
5bec894a8c but insteadl of using
`preload(:tags)` we ended up using `includes(:tags)`. The problem here
is that `includes(:tags)` can either result in `preload(:tags)` or
`eager_load(:tags)` but for some reason ActiveRecord is deciding to
`eager_load(:tags)` resulting in a joins to the `topic_tags` and `tags`
table which is not necessarily and leads to more inefficient queries.

When `includes(:tags)` is used, listing the latest topics ended up
generating the following sample queries to fetch the list of topics to display.

```
SELECT DISTINCT "topics"."pinned_at" AS alias_0, "topics"."id" FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL)) ORDER BY "topics"."pinned_at" DESC LIMIT 30

SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54, "tags"."id" AS t2_r0, "tags"."name" AS t2_r1, "tags"."created_at" AS t2_r2, "tags"."updated_at" AS t2_r3, "tags"."pm_topic_count" AS t2_r4, "tags"."target_tag_id" AS t2_r5, "tags"."description" AS t2_r6, "tags"."public_topic_count" AS t2_r7, "tags"."staff_topic_count" AS t2_r8 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL)) AND "topics"."id" = 7 ORDER BY "topics"."pinned_at" DESC

SELECT DISTINCT topics.bumped_at AS alias_0, "topics"."id" FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (NOT ( pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL) )) ORDER BY topics.bumped_at DESC LIMIT 30

SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54, "tags"."id" AS t2_r0, "tags"."name" AS t2_r1, "tags"."created_at" AS t2_r2, "tags"."updated_at" AS t2_r3, "tags"."pm_topic_count" AS t2_r4, "tags"."target_tag_id" AS t2_r5, "tags"."description" AS t2_r6, "tags"."public_topic_count" AS t2_r7, "tags"."staff_topic_count" AS t2_r8 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (NOT ( pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL) )) AND "topics"."id" IN (477, 481, 480, 479, 478, 467, 466, 230, 209, 183, 173, 179, 168, 139, 102, 144, 150, 118, 126, 88, 63, 46, 117, 171, 45, 77, 154, 158, 43, 79) ORDER BY topics.bumped_at DESC
```

Note how there are two extra queries which has to select `DISTINCT
topics.pinned_at` and `DISTINCT topics.bumped_at` because of the
unnecessary left joins to the `topic_tags` and `tags` table result in
duplicated rows in the topic tables. As a result, PG is not able to
use our indexes to effectively execute the query.

Comparing this to the queries being executed when `preload(:tags)` is
used.

```
SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL)) ORDER BY "topics"."pinned_at" DESC LIMIT 30

SELECT "topic_tags".* FROM "topic_tags" WHERE "topic_tags"."topic_id" = 7

SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (NOT ( pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL) )) ORDER BY topics.bumped_at DESC LIMIT 30

SELECT "topic_tags".* FROM "topic_tags" WHERE "topic_tags"."topic_id" IN (477, 481, 480, 479, 478, 467, 466, 230, 209, 183, 173, 179, 168, 139, 102, 144, 150, 118, 126, 88, 63, 46, 117, 171, 45, 77, 154, 158, 43, 79)
SELECT "tags"."id", "tags"."name", "tags"."created_at", "tags"."updated_at", "tags"."pm_topic_count", "tags"."target_tag_id", "tags"."description", "tags"."public_topic_count", "tags"."staff_topic_count" FROM "tags" WHERE "tags"."id" IN (10, 20, 26, 7, 27, 28, 30, 19, 9, 4, 15, 29, 14, 18, 11, 25, 1, 21, 8, 22, 5, 32)
```

We end up with queries that are much more efficient as those queries can
effectively use the indexes.
2023-08-03 11:51:32 +08:00
Roman Rizzi 3e72cc2d7d
REVERT: suggested topic list tweaks stable (#22909)
We reverted these commits because they weren't stable enough. We'll keep working on them on `tests-passed`.

* Revert "UX: Use full width when displaying a single recommendations list. (#22896)"

This reverts commit dd8d89d9c8.

* Revert "UX:  Topic recommendations tweaks. (#22880)"

This reverts commit e7fb4be23e.
2023-08-01 12:02:34 -03:00
Discourse Translator Bot 2cf06171a5
Update translations (#22905) 2023-08-01 16:05:49 +02:00
David Taylor 0876f262e9
Bump version to v3.1.0 2023-08-01 10:00:12 +01:00
David Taylor 6a564109d1
Merge v3.1.0.beta8 into stable 2023-08-01 10:00:11 +01:00
David Taylor 9b339bcd2c
Bump version to v3.1.0.beta8 2023-08-01 09:50:45 +01:00
David Taylor bb217bbcc8
FIX: Ensure PresenceChannel does not raise error during readonly (#22899)
PresenceChannel configuration is cached using redis. That cache is used, and sometimes repopulated, during normal GET requests. When the primary redis server was readonly, that `redis.set` call would raise an error and cause the entire request to fail. Instead, we should ignore the failure and continue without populating the cache.
2023-08-01 09:34:57 +01:00