FEATURE: Only approved flags for post counters
* Why was this change necessary?
The counters for flagged posts in the user's profile and user index from
the admin view include flags that were rejected, ignored or pending
review. This introduces unnecessary noise. Also the flagged posts
counter in the user's profile includes custom flags which add further
noise to this signal.
* How does it address the problem?
* Modifying User#flags_received_count to return posts with only approved
standard flags
* Refactoring User#number_of_flagged_posts to alias to
User#flags_received_count
* Updating the flagged post staff counter hyperlink to navigate to a
filtered view of that user's approved flagged posts to maintain
consistency with the counter
* Adding system tests for the profile page to cover the flagged posts
staff counter
In the query generated by `TopicTrackingState.report`, there are two
subqueies being executed. The first subquery fetches all the topics
that are new for a given user while the second subquery fetches all the topics with
unread posts for a given user. For the second subquery, there is a
filter `topics.updated_at >= user_stats.first_unread_at` which is used
as a performance optimisation to reduce the number of rows that PG has
to scan through the `topics` table.
However, we started to notice in production that the PG planner doesn't
always execute the filter first to reduce the number of rows that it has
to scan through. Running the following query in one of our production
instance,
```
EXPLAIN ANALYZE
SELECT
DISTINCT topics.id as topic_id,
u.id as user_id,
topics.created_at,
topics.updated_at,
topics.highest_staff_post_number AS highest_post_number,
last_read_post_number,
c.id as category_id,
c.topic_id AS category_topic_id,
tu.notification_level,
us.first_unread_at,
GREATEST(
CASE
WHEN COALESCE(uo.new_topic_duration_minutes, 2880) = -1 THEN u.created_at
WHEN COALESCE(uo.new_topic_duration_minutes, 2880) = -2 THEN COALESCE(
u.previous_visit_at,u.created_at
)
ELSE ('2023-07-31 03:29:45.737630'::timestamp - INTERVAL '1 MINUTE' * COALESCE(uo.new_topic_duration_minutes, 2880))
END, u.created_at, '2023-07-25 15:06:44'
) AS treat_as_new_topic_start_date
FROM topics
JOIN users u on u.id = 13455
JOIN user_stats AS us ON us.user_id = u.id
JOIN user_options AS uo ON uo.user_id = u.id
JOIN categories c ON c.id = topics.category_id
LEFT JOIN topic_users tu ON tu.topic_id = topics.id AND tu.user_id = u.id
WHERE u.id = 13455 AND
topics.updated_at >= us.first_unread_at AND
topics.archetype <> 'private_message' AND
(("topics"."deleted_at" IS NULL AND (tu.last_read_post_number < topics.highest_staff_post_number) AND (COALESCE(tu.notification_level, 1) >= 2)) OR (1=0)) AND
NOT (
COALESCE((select array_agg(tag_id) from topic_tags where topic_tags.topic_id = topics.id), ARRAY[]::int[]) && ARRAY[451,452,453]
) AND
topics.deleted_at IS NULL AND
NOT (
last_read_post_number IS NULL AND
(
topics.category_id IN (SELECT "categories"."id" FROM "categories" LEFT JOIN categories categories2 ON categories2.id = categories.parent_category_id LEFT JOIN category_users ON category_users.category_id = categories.id AND category_users.user_id = 13455 LEFT JOIN category_users category_users2 ON category_users2.category_id = categories2.id AND category_users2.user_id = 13455 WHERE ((category_users.id IS NULL AND COALESCE(category_users2.notification_level, 1) = 0) OR COALESCE(category_users.notification_level, 1) = 0))
AND tu.notification_level <= 1
)
)
```
we get the following
```
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Unique (cost=201606.06..201608.15 rows=76 width=60) (actual time=91.279..91.294 rows=14 loops=1)
-> Sort (cost=201606.06..201606.25 rows=76 width=60) (actual time=91.278..91.284 rows=14 loops=1)
Sort Key: topics.id, topics.created_at, topics.updated_at, topics.highest_staff_post_number, tu.last_read_post_number, c.id, c.topic_id, tu.notification_level, us.first_unread_at, (GREATEST(CASE WHEN (COALESCE(uo.new_topic_duration_minutes, 2880) = '-1'::integer) THEN u.created_at WHEN (COALESCE(uo.new_topic_duration_minutes, 2880) = '-2'::integer) THEN COALESCE(u.previous_visit_at, u.created_at) ELSE ('2023-07-31 03:29:45.73763'::timestamp without time zone - ('00:01:00'::interval * (COALESCE(uo.new_topic_duration_minutes, 2880))::double precision)) END, u.created_at, '2023-07-25 15:06:44'::timestamp without time zone))
Sort Method: quicksort Memory: 26kB
-> Hash Join (cost=97519.51..201603.69 rows=76 width=60) (actual time=87.662..91.268 rows=14 loops=1)
Hash Cond: (topics.id = tu.topic_id)
Join Filter: ((tu.last_read_post_number < topics.highest_staff_post_number) AND ((tu.last_read_post_number IS NOT NULL) OR (NOT (hashed SubPlan 2)) OR (tu.notification_level > 1)))
Rows Removed by Join Filter: 10
-> Nested Loop (cost=1.54..104075.36 rows=3511 width=68) (actual time=0.055..3.609 rows=548 loops=1)
-> Nested Loop (cost=1.13..25.20 rows=1 width=32) (actual time=0.027..0.033 rows=1 loops=1)
-> Nested Loop (cost=0.71..16.76 rows=1 width=28) (actual time=0.020..0.023 rows=1 loops=1)
-> Index Scan using users_pkey on users u (cost=0.42..8.44 rows=1 width=20) (actual time=0.010..0.012 rows=1 loops=1)
Index Cond: (id = 13455)
-> Index Scan using user_stats_pkey on user_stats us (cost=0.29..8.31 rows=1 width=12) (actual time=0.008..0.010 rows=1 loops=1)
Index Cond: (user_id = 13455)
-> Index Scan using index_user_options_on_user_id_and_default_calendar on user_options uo (cost=0.42..8.44 rows=1 width=8) (actual time=0.007..0.008 rows=1 loops=1)
Index Cond: (user_id = 13455)
-> Nested Loop (cost=0.41..104015.12 rows=3504 width=36) (actual time=0.026..3.503 rows=548 loops=1)
-> Seq Scan on categories c (cost=0.00..13.73 rows=73 width=8) (actual time=0.003..0.039 rows=73 loops=1)
-> Index Only Scan using index_topics_on_updated_at_public on topics (cost=0.41..1424.20 rows=48 width=28) (actual time=0.012..0.046 rows=8 loops=73)
Index Cond: ((updated_at >= us.first_unread_at) AND (category_id = c.id))
Filter: (NOT (COALESCE((SubPlan 1), '{}'::integer[]) && '{451,452,453}'::integer[]))
Heap Fetches: 553
SubPlan 1
-> Aggregate (cost=4.31..4.32 rows=1 width=32) (actual time=0.002..0.002 rows=1 loops=548)
-> Index Only Scan using index_topic_tags_on_topic_id_and_tag_id on topic_tags (cost=0.29..4.31 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=548)
Index Cond: (topic_id = topics.id)
Heap Fetches: 178
-> Hash (cost=97222.14..97222.14 rows=19914 width=16) (actual time=87.545..87.546 rows=42884 loops=1)
Buckets: 65536 (originally 32768) Batches: 1 (originally 1) Memory Usage: 2387kB
-> Bitmap Heap Scan on topic_users tu (cost=1217.47..97222.14 rows=19914 width=16) (actual time=14.419..78.286 rows=42884 loops=1)
Recheck Cond: (user_id = 13455)
Filter: (COALESCE(notification_level, 1) >= 2)
Rows Removed by Filter: 15839
Heap Blocks: exact=45285
-> Bitmap Index Scan on index_topic_users_on_user_id_and_topic_id (cost=0.00..1212.49 rows=59741 width=0) (actual time=6.448..6.448 rows=58723 loops=1)
Index Cond: (user_id = 13455)
SubPlan 2
-> Nested Loop Left Join (cost=0.74..46.90 rows=1 width=4) (never executed)
Join Filter: (category_users2.category_id = categories2.id)
Filter: (((category_users.id IS NULL) AND (COALESCE(category_users2.notification_level, 1) = 0)) OR (COALESCE(category_users.notification_level, 1) = 0))
-> Nested Loop Left Join (cost=0.45..32.31 rows=73 width=16) (never executed)
Join Filter: (category_users.category_id = categories.id)
-> Nested Loop Left Join (cost=0.15..18.45 rows=73 width=8) (never executed)
-> Seq Scan on categories (cost=0.00..13.73 rows=73 width=8) (never executed)
-> Memoize (cost=0.15..0.28 rows=1 width=4) (never executed)
Cache Key: categories.parent_category_id
Cache Mode: logical
-> Index Only Scan using categories_pkey on categories categories2 (cost=0.14..0.27 rows=1 width=4) (never executed)
Index Cond: (id = categories.parent_category_id)
Heap Fetches: 0
-> Materialize (cost=0.29..11.69 rows=2 width=12) (never executed)
-> Index Scan using idx_category_users_user_id_category_id on category_users (cost=0.29..11.68 rows=2 width=12) (never executed)
Index Cond: (user_id = 13455)
-> Materialize (cost=0.29..11.69 rows=2 width=8) (never executed)
-> Index Scan using idx_category_users_user_id_category_id on category_users category_users2 (cost=0.29..11.68 rows=2 width=8) (never executed)
Index Cond: (user_id = 13455)
Planning Time: 1.740 ms
Execution Time: 91.414 ms
(59 rows)
```
From the execution plan, we can see the most of the time is spent
joining about 42888 rows in the `topics` table to the `topic_users` table.
However, we know that we only have to scan through a
subset of the `topics` table because the user's last unread at is '2023-07-20 11:33:05'.
If we filter the `topics` table with `topics.updated_at >= '2023-07-20 11:33:05'`, this would only
return about 1500 rows.
From our testing in production, the PG planner is able to execute a
better query plan when we avoid the unnecessary joins on `user_stats` just to be
able to get the user's `UserStat#first_unread_at`. Instead, we can just
pass the value of `UserStat#first_unread_at` directly as a query
parameter.
```
EXPLAIN ANALYZE
SELECT
DISTINCT topics.id as topic_id,
u.id as user_id,
topics.created_at,
topics.updated_at,
topics.highest_staff_post_number AS highest_post_number,
last_read_post_number,
c.id as category_id,
c.topic_id AS category_topic_id,
tu.notification_level,
GREATEST(
CASE
WHEN COALESCE(uo.new_topic_duration_minutes, 2880) = -1 THEN u.created_at
WHEN COALESCE(uo.new_topic_duration_minutes, 2880) = -2 THEN COALESCE(
u.previous_visit_at,u.created_at
)
ELSE ('2023-07-31 03:29:45.737630'::timestamp - INTERVAL '1 MINUTE' * COALESCE(uo.new_topic_duration_minutes, 2880))
END, u.created_at, '2023-07-25 15:06:44'
) AS treat_as_new_topic_start_date
FROM topics
JOIN users u on u.id = 13455
JOIN user_options AS uo ON uo.user_id = u.id
JOIN categories c ON c.id = topics.category_id
LEFT JOIN topic_users tu ON tu.topic_id = topics.id AND tu.user_id = u.id
WHERE u.id = 13455 AND
topics.updated_at >= '2023-07-20 11:33:05' AND
topics.archetype <> 'private_message' AND
(("topics"."deleted_at" IS NULL AND (tu.last_read_post_number < topics.highest_staff_post_number) AND (COALESCE(tu.notification_level, 1) >= 2)) OR (1=0)) AND
NOT (
COALESCE((select array_agg(tag_id) from topic_tags where topic_tags.topic_id = topics.id), ARRAY[]::int[]) && ARRAY[451,452,453]
) AND
topics.deleted_at IS NULL AND
NOT (
last_read_post_number IS NULL AND
(
topics.category_id IN (SELECT "categories"."id" FROM "categories" LEFT JOIN categories categories2 ON categories2.id = categories.parent_category_id LEFT JOIN category_users ON category_users.category_id = categories.id AND category_users.user_id = 13455 LEFT JOIN category_users category_users2 ON category_users2.category_id = categories2.id AND category_users2.user_id = 13455 WHERE ((category_users.id IS NULL AND COALESCE(category_users2.notification_level, 1) = 0) OR COALESCE(category_users.notification_level, 1) = 0))
AND tu.notification_level <= 1
)
);
```
Note how the filter is now `topics.updated_at >= '2023-07-20 11:33:05'`
instead of `topics.updated_at >= us.first_unread_at`. The modified query
above generates the following execution plan.
```
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Unique (cost=5189.86..5189.88 rows=1 width=52) (actual time=4.991..5.002 rows=14 loops=1)
-> Sort (cost=5189.86..5189.86 rows=1 width=52) (actual time=4.990..4.994 rows=14 loops=1)
Sort Key: topics.id, topics.created_at, topics.updated_at, topics.highest_staff_post_number, tu.last_read_post_number, c.id, c.topic_id, tu.notification_level, (GREATEST(CASE WHEN (COALESCE(uo.new_topic_duration_minutes, 2880) = '-1'::integer) THEN u.created_at WHEN (COALESCE(uo.new_topic_duration_minutes, 2880) = '-2'::integer) THEN COALESCE(u.previous_visit_at, u.created_at) ELSE ('2023-07-31 03:29:45.73763'::timestamp without time zone - ('00:01:00'::interval * (COALESCE(uo.new_topic_duration_minutes, 2880))::double precision)) END, u.created_at, '2023-07-25 15:06:44'::timestamp without time zone))
Sort Method: quicksort Memory: 26kB
-> Nested Loop (cost=52.11..5189.85 rows=1 width=52) (actual time=0.093..4.974 rows=14 loops=1)
-> Nested Loop (cost=51.70..5181.39 rows=1 width=60) (actual time=0.084..4.931 rows=14 loops=1)
-> Nested Loop (cost=51.28..5172.94 rows=1 width=44) (actual time=0.076..4.887 rows=14 loops=1)
-> Nested Loop (cost=0.41..1698.46 rows=59 width=36) (actual time=0.029..3.537 rows=548 loops=1)
-> Seq Scan on categories c (cost=0.00..13.73 rows=73 width=8) (actual time=0.005..0.039 rows=73 loops=1)
-> Index Only Scan using index_topics_on_updated_at_public on topics (cost=0.41..23.07 rows=1 width=28) (actual time=0.012..0.047 rows=8 loops=73)
Index Cond: ((updated_at >= '2023-07-20 11:33:05'::timestamp without time zone) AND (category_id = c.id))
Filter: (NOT (COALESCE((SubPlan 1), '{}'::integer[]) && '{451,452,453}'::integer[]))
Heap Fetches: 552
SubPlan 1
-> Aggregate (cost=4.31..4.32 rows=1 width=32) (actual time=0.002..0.002 rows=1 loops=548)
-> Index Only Scan using index_topic_tags_on_topic_id_and_tag_id on topic_tags (cost=0.29..4.31 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=548)
Index Cond: (topic_id = topics.id)
Heap Fetches: 178
-> Index Scan using index_topic_users_on_user_id_and_topic_id on topic_users tu (cost=50.86..58.88 rows=1 width=16) (actual time=0.002..0.002 rows=0 loops=548)
Index Cond: ((user_id = 13455) AND (topic_id = topics.id))
Filter: ((COALESCE(notification_level, 1) >= 2) AND (last_read_post_number < topics.highest_staff_post_number) AND ((last_read_post_number IS NOT NULL) OR (NOT (hashed SubPlan 2)) OR (notification_level > 1)))
Rows Removed by Filter: 0
SubPlan 2
-> Nested Loop Left Join (cost=0.74..50.43 rows=1 width=4) (never executed)
Join Filter: (category_users2.category_id = categories2.id)
Filter: (((category_users.id IS NULL) AND (COALESCE(category_users2.notification_level, 1) = 0)) OR (COALESCE(category_users.notification_level, 1) = 0))
-> Nested Loop Left Join (cost=0.45..35.84 rows=73 width=16) (never executed)
Join Filter: (category_users.category_id = categories.id)
-> Nested Loop Left Join (cost=0.15..21.97 rows=73 width=8) (never executed)
-> Seq Scan on categories (cost=0.00..13.73 rows=73 width=8) (never executed)
-> Memoize (cost=0.15..0.61 rows=1 width=4) (never executed)
Cache Key: categories.parent_category_id
Cache Mode: logical
-> Index Only Scan using categories_pkey on categories categories2 (cost=0.14..0.60 rows=1 width=4) (never executed)
Index Cond: (id = categories.parent_category_id)
Heap Fetches: 0
-> Materialize (cost=0.29..11.69 rows=2 width=12) (never executed)
-> Index Scan using idx_category_users_user_id_category_id on category_users (cost=0.29..11.68 rows=2 width=12) (never executed)
Index Cond: (user_id = 13455)
-> Materialize (cost=0.29..11.69 rows=2 width=8) (never executed)
-> Index Scan using idx_category_users_user_id_category_id on category_users category_users2 (cost=0.29..11.68 rows=2 width=8) (never executed)
Index Cond: (user_id = 13455)
-> Index Scan using users_pkey on users u (cost=0.42..8.44 rows=1 width=20) (actual time=0.003..0.003 rows=1 loops=14)
Index Cond: (id = 13455)
-> Index Scan using index_user_options_on_user_id_and_default_calendar on user_options uo (cost=0.42..8.44 rows=1 width=8) (actual time=0.002..0.002 rows=1 loops=14)
Index Cond: (user_id = 13455)
Planning Time: 1.281 ms
Execution Time: 5.092 ms
(48 rows)
```
With the new query, PG first does an index scan using the `index_topics_on_updated_at_public` index to filter away most of the topics making the subsequent joins much cheaper. Total query time has been reduced from ~90ms to ~5ms.
This optimisation will mostly affect users with very few/recent unread topics since a large `UserStat#firsts_unread_at` value will still mean scanning through a large portion of the `topics` table.
This is a similar fix to 32d4810e2b
Why this change?
Prior to this change, there is a bug in `TopicsController#bulk`
where it does not dismiss new unred posts in sub-subcategories when the
`category_id` and `include_subcategories=true` params are present. This
is because the controller did not account for sub-subcategories when
fetching the category ids of the new topics that should be dismissed.
This commit fixes the problem by relying on the `Category.subcategory_ids` class
method which accounts for sub-subcategories.
We currently are accumulating orphaned upload references whenever drafts are deleted.
This change deals with future cases by adding a dependent strategy of delete_all on the Draft#upload_references association. (We don't really need destroy strategy here, since UploadReference is a simple data bag and there are no validations or callbacks on the model.)
It deals with existing cases through a migration that deletes all existing, orphaned draft upload references.
We already handled 429 rate limit errors correctly. This commit adds backoff logic to other types of error to avoid requests being retried every second.
A previous change updated `ReviewableQueuedPost`'s `created_by`
to be consistent with other reviewable types. It assigns
the the creator of the post being queued to `target_created_by` and sets
the `created_by` to the creator of the reviewable itself.
This fix updates some of the `created_by` references missed during the
intial fix.
The store expects values for property names ending with `_id` to be a resource id
and `_ids` to be an array of resource ids.
This change ensures the store gracefully handles situations where an
embedded field with incompliant data structure sneaks its way to production.
By default, only 10 members are highlighted on group cards. However,
joining/leaving a big group via the buttons on the group card results in
up to 50 members being highlighted. For large groups, this causes the card
to move off-screen.
This happens because, while the initial render explicitly fetches only 10
members, we don't seem to apply the same limit as part of the member
reload performed when a user leaves/joins via the buttons on the card.
This PR fixes that by only making the first 10 users available for
highlight regardless of the number of members loaded in the store.
What is the problem here?
In multiple controllers, we are accepting a `limit` params but do not
impose any upper bound on the values being accepted. Without an upper
bound, we may be allowing arbituary users from generating DB queries
which may end up exhausing the resources on the server.
What is the fix here?
A new `fetch_limit_from_params` helper method is introduced in
`ApplicationController` that can be used by controller actions to safely
get the limit from the params as a default limit and maximum limit has
to be set. When an invalid limit params is encountered, the server will
respond with the 400 response code.
Using pinch-zoom on mobile devices with lightbox images can lead to scrolling of background content.
This change handles this by capturing the window.scrollY value when opening the lightbox, then when exiting we check if the scroll position has changed and reset it.
What is the context for this change?
Prior to this change, there is a bug in `TopicsController#reset_new`
where it does not dismiss new topics in sub-subcategories when the
`category_id` and `include_subcategories=true` params are present. This
is because the controller did not account for sub-subcategories when
fetching the category ids of the new topics that should be dismissed.
This commit fixes the problem by relying on the `Category.subcategory_ids` class
method which accounts for sub-subcategories.
This is happening because despite the user already existing in the forum, the `SingleSignOnRecord` doesn't exist and "require_activation" is set on the provider, causing us to skip looking for the email, and resulting in us creating a new User then seeing Validation failed: Primary email has already been taken when DiscourseConnect is attempting to make a new account.
provide the ability to edit theme settings in the json editor, and also copy them as a text file so they can be pasted into another instance.
Reference: /t/65023
Why this change?
In `PostDestroyer#make_previous_post_the_last_one` and
`Topic.reset_highest`, we have a query that looks something like this:
```
SELECT user_id FROM posts
WHERE topic_id = :topic_id AND
deleted_at IS NULL AND
post_type <> 4
#{post_type}
ORDER BY created_at desc
LIMIT 1
```
However, we currently don't have an index that caters directly to this
query. As a result, we have seen this query performing poorly on large
sites if the PG planner ends up using an index that is suboptimal for
the query.
This commit adds an index to the `posts` table on `topic_id` and then
`created_at`. For the query above, PG will be able to do a backwards
index scan efficiently.
Context of this change:
There are two site settings which an admin can configured to set the
default categories and tags that are shown for a new user. `default_navigation_menu_categories`
is used to determine the default categories while
`default_navigation_menu_tags` is used to determine the default tags.
Prior to this change when seeding the defaults, we will filter out the
categories/tags that the user do not have permission to see. However,
this means that when the user does eventually gain permission down the
line, the default categories and tags do not appear.
What does this change do?
With this commit, we have changed it such that all the categories and tags
configured in the `default_navigation_menu_categories` and
`default_navigation_menu_tags` site settings are seeded regardless of
whether the user's visibility of the categories or tags. During
serialization, we will then filter out the categories and tags which the
user does not have visibility of.
## Problem
History modal is flashing when changing revision versions
## Context
This was introduced in https://github.com/discourse/discourse/pull/22666
We need to have a conditional loading spinner for the initial paint of the history modal as we don't have the revision yet loaded, so this can cause some odd rendering issues. At the same time we don't want to display the loading spinner each time we toggle between the revision versions, because the loading spinner replaces the revision body causing the modal sizes to be drastically different resulting in _jumping_ or _flashing_.
## Fix
Render the loading spinner only on the first paint of the modal.
https://github.com/discourse/discourse/assets/50783505/8d19275e-86a5-4132-8a1f-af4b4f5301a6
Followup to f5e8e73.
This switches the placeholder label to the existing string "optional
tags" and only shows it if there are no items picked.
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
Change to drag move event handling. When position of mouse changed, we can assume it is not drag and drop, and we should keep default behaviour.
Otherwise, we stop propagation of the event to handle drag and drop correctly. s
New API to change sidebar mode. We defined two:
Separated - only sections belonging to specific panel are displayed, and buttons to switch the panel are available as well.
Combined - all sections are displayed together and switch panel buttons are not visible.
In addition, as a part of refactoring, a new service called SidebarState was introduced.
The `/u` route was broken when there were no directory columns because
its order parameter relied on the first column's name. This commit adds
a `likes_received` as the default order when there are no columns, which
results in a list of users being output without any additional columns.
For this very edge case, that's better than a JS error.
Fixes issue with scrolling background when lightbox is opened on mobile.
Since we rely on swiping for navigating lightbox galleries on mobile, we want to disable document scrolling.
We're seeing unhandled errors in production when web push notifications are failing with an SSL error. This is happening for a few users, but generating a large amount of log noise due to the sheer number of notifications.
This adds handling of SSL errors in two places:
1. In FinalDestination::HTTP, this is handled the same as a timeout error, and gives a chance to recover.
2. In PushNotificationPusher. This will cause the notification to retry a number of times, and if it keeps failing, disable push notifications for the user. (Existing behaviour.)
I wanted to wrap the SSL error in e.g. WebPush::RequestError, but the gem doesn't have request error handling, so didn't want to have the freedom patch diverge from the gem as well. Instead just propagating the raw SSL error.
Why this change?
We were verifying that a url for a section link in a custom sidebar
section is valid by passing the url string to `Router#recognize`.
If a `rootURL` has been set on the router, the url string that is passed
to `Router#recognize` has to start with the `rootURL`.
This commit fixes the problem by ensuring that `RouteInfoHelper` adds
the application subfolder path before calling `Router#recognize` on the
url string.
Why this change?
When setting up the `IntersectionObserver`, we did not account for the
top margin and padding causing no intersection event to fire when the
last tag is load into view. This commits fixes the problem by setting a
bottom margin using the `rootMargin` option when setting up the
`IntersectionObserver`.
This commit also improves the test coverage surrounding the loading of
more tags.
We recently replaced another ignore user modal with a Glimmer and DModal based component. This change makes use of that same component in the user card ignore modal by adding an enableSelection flag.
Interestingly, this missing parenthesis was silently repaired under Chrome/Firefox, but seems to have caused some issues on other browsers.
https://meta.discourse.org/t/272496
Why this change?
We're already displaying a category's description as the title attribute
on the category section link. We should do the same for tags as well.
Allow anonymous users (logged-in, but set to anonymous posting) to like posts
---------
Co-authored-by: Emmett Ling <eling@zendesk.com>
Co-authored-by: Nat <natalie.tay@discourse.org>
* Why was this change necessary?
The current logic in the user.hbs template file does not render the
trust level element for the user's info panel when the user is TL0,
because 0 is treated as falsey in the `if` conditional block.
Ref: https://meta.discourse.org/t/tl0-not-displayed-on-users-profile-pages/271779/10
* How does it address the problem?
This PR adds a predicate helper method local to the user controller that
includes an additional check which returns true if the trust_level of
the user is 0 on top of the existing logic. This allows TL0 users to
have their trust level rendered correctly in their profile's info panel.
In Safari, clicking any image in a lightbox gallery results in the first image loading (instead of the clicked image).
Previously we relied on document.activeElement to determine which lightbox image was clicked. However in Chrome the active element is the lightbox selector (a.lightbox), whereas in Safari the active element defaults to the body tag.
Currently the startingIndex that is calculated within processHTML() is used by lightbox to determine which image to load first. The starting index is currently achieved by checking each lightbox element within the gallery against the active element.
To fix this issue we can use the event.target to get the clicked image, then use the closest selector and pass that into the function to do the matching and return the correct startingIndex.
These methods were deprecated and marked for removal in 2.6. This change deletes them.
These deprecations use raise_error: true, so the fallbacks are at this point unreachable and can't be used anyway.
1. in the test, hiding is now done with css so if element gets rerendered it won't lose the styling
2. the skipping now allows for the `<article>` element itself being hidden
This has been proposed as the new default, and is currently in-use on many large ember apps without issue. It is already the default under Embroider. Testing locally, this seems to make incremental builds in development at least 2x faster.
https://github.com/ember-cli/ember-cli/issues/8681
- Convert `admin-incoming-email` modal to component-based API
- Testing that the modal was working in local development was extremely challenging due to the need for `rejected` and `bounced` emails. Something that is not easy to stub in a local dev environment. To make this process more smooth for future developers I have added a new rake task:
```
desc "Creates sample email logs"
task "email_logs:populate" => ["db:load_config"] do |_, args|
DiscourseDev::EmailLog.populate!
end
```
That will generate fully functional email logs in development to be toyed with.
<img width="787" alt="Screenshot 2023-07-20 at 3 27 04 PM" src="https://github.com/discourse/discourse/assets/50783505/47b3fe34-cd7e-49a5-8fe6-768c0fbd1aa2">
The gjs/gts formats are a new pattern for authoring Ember components. This commit introduces support for these patterns to our build pipeline for core/plugins, and converts a handful of components to use the new format. It also introduces relevant updates to our linting config, and to our sample vscode configuration.
Co-authored-by: Godfrey Chan <godfreykfc@gmail.com>
Co-authored-by: Krystan HuffMenne <kmenne+github@gmail.com>
Since 0fa92529ed, helpers can now be implemented as plain JS functions. This makes them much easier to write/read, and also makes them usable in `<template>` gjs files.
* FEATURE: allow sidebar section api to create external links
Right now, sidebar API allows creating sections with internal links. It should be extended to allow creating links to external URLs as well.
* FIX: after rebase
pass the extra public trees to `app.toTree()` to match:
0e00f2bf15/packages/test-setup/src/index.ts (L24-L27)
The ember-cli-terser addon now takes care of minifying all additional trees, so we can remove our custom terser-related logic
This brings them more in line with an idiomatic ember app looks
like, in particular, embroider really expects the CSS file to be
there.
As far as I can tell this is fairly harmless, since in production
the actual HTML is generated and served by Rails anyway.
Down the road, this may also be a good alternative to hacking the
build pipeline to bring in styles for tests.
Recently we started giving admins a notice in the advice panel when their translations have become outdated due to changes in core. However, we didn't include any additional information.
This PR adds more information about the outdated translation inside the site text edit page, together with an option to dismiss the warning.
* UX: Disclose AI model used and add animation to placeholder
* Move text into hbs template
DTooltip (weirdly) attaches to a sibling element, so we need something else to be rendered inside the RenderGlimmer wrapper div
---------
Co-authored-by: David Taylor <david@taylorhq.com>
e.g. the modernised share-topic modal will attempt to open the `create-invite` modal. Prior to this commit, this mixing of modern/legacy would fail silently, and the create-invite modal was never shown.
Initializing an EmberObject with a null object leads to an exception. This commit stops that from happening, and introduces an acceptance test for adding/removing banner topics via message-bus.
Co-authored-by: David Taylor <david@taylorhq.com>
011ba5b9 slightly changed the way the staff-action-log route is activated. It's now possible for `deserializeQueryParam` to be called with a null value, so we need to deal with that case.
This route is currently untested - we'll follow-up with another commit to add some.
This PR migrates the publish page modal to a Glimmer component and DModal.
Most of the code is lift-and-shift. However, the component state getters were implemented using meta-programming in the original controller. They have all been inlined here for clarity, searchability, etc.
Define new concept of panels in sidebar. Panels are wrappers around sidebar sections. In the future, it allows creating full focus mode by switching between panels.
A new API method called addSidebarPanel was added. Default main panel is already registered and by default all API sections are mounted to main.
When a type was disabled, the hashtag search _without_ a
term was erroring. This was because we weren't filtering
out the disabled types from types_in_priority_order first
like we were if there was a term provided.
This commit fixes that issue, and also makes it so
contexts_with_ordered_types and ordered_types_for_context
will only return hashtag types which are enabled.
This babel plugin is intended to supress the deprecation warnings
from building plugins, however, discourse-plugins does not actually
consume this plugin at all. Currently this happens to work due to
how the babel worker processes are shared and the timing/ordering
of the build, but it will stop working with the embroider build.
This commit extracts the plugin the a shared package so that it
can be properly consumed by discourse-plugins as well as core.
Performing a `Delete User`/`Delete and Block User` reviewable actions for a
queued post reviewable from the `review.show` route results in an error
popup even if the action completes successfully.
This happens because unlike other reviewable types, a user delete action
on a queued post reviewable results in the deletion of the reviewable
itself. A subsequent attempt to reload the reviewable record results in
404. The deletion happens as part of the call to `UserDestroyer` which
includes a step for destroying reviewables created by the user being
destroyed. At the root of this is the creator of the queued post
being set as the creator of the reviewable as instead of the system
user.
This change assigns the creator of the reviewable to the system user and
uses the more approapriate `target_created_by` column for the creator of the
post being queued.
- explicitly enables the jquery-integration. This was previously enabled by default, so no change in behavior for us
- enable template-only-glimmer-components. In core, we don't have any component templates under `templates/components`, so this flag has no effect. A shim, with associated tests, is introduced to preserve the old template-only 'classic component' behavior for themes and plugins.
We'd like to get this deprecation unsilenced before the 3.1 release so that theme/plugin developers see the messages and can make the necessary changes during the 3.2 release cycle. To avoid the remaining legacy core modals from creating overwhelming noise in the logs, deprecation messages for them are skipped.
When the app boots, Ember fires a `routeWillChange` event. This was causing us to set the `_trackView` flag in our ajax library, which would cause the next request to have the `Discourse-Track-View` header, despite not being relevant to the page view. Depending on the plugins/themes installed, this could lead to 'double counting' of pageviews. (because the initial HTML request is also counted as a page view)
This commit updates the the logic to ignore the first transition (by checking `transition.from`), and also introduces an acceptance test for the behaviour.
Co-authored-by: Régis Hanol <regis@hanol.fr>
Currently the dominant color attribute is only set for post images (not chat).
As a result, clicking lightbox images in chat will load the image within lightbox but also shows a JS error.
This change ensures that the dominant color is set before attempting to update the site theme color.
Adding a filter without a type parameter has been deprecated for the last three years, and was marked for removal in 2.9.0.
During this time we have had a few deprecation warnings in logs coming from Reports::Bookmarks.
The fallback was to set the type to the name of the filter. This change just passes the type (same as name) explicitly instead, and removes the deprecation fallback.
We have a number of raw comments indicating that certain methods and classes are deprecated and marked for removal. This change turn those comments into deprecation warnings so that we can 1) see them in the logs of our own hosting and 2) give some warning to self hosters.
Why this change?
The `legacy` navigation menu option for the `navigation_menu` site
setting will be removed shortly after the release of Discourse 3.1 in
the first beta release of Discourse 3.2. Therefore, we're adding an
admin dashboard warning to give sites on the `legacy` navigation menu a
heads up.
We need a nice way to only return some hashtag data
sources based on various site settings. This commit
adds an enabled? method that every hashtag data source
must implement. If this returns false the data source
will not be used at all for hashtag lookups or search.
What does this change do?
This commit removes the experimental label for a bunch of APIs that have
been used in production for quite some time at Discourse so that the
APIs can be released as part of Discourse 3.1
To decide to use flip behavior select-kit will check if it's located inside a modal as a modal will scroll if overflown, however, when locating the select-kit element in the footer or header this is not the case. This commit will deactivate `flip` modifier only when used inside modal body.
Fixes an issue where this.selector value was not binded at the time of adding the event listener. Therefore when someone opens a chat channel that has images, the value of selector would change. I also moved the callback to a named function (rather than the default handleEvent).
Why this change?
Prior to this change, we would only return tags that are used in at
least one public topic. However, this is confusing for users because the
tag could be used in a restricted category and that is not considered a
"public" topic. Instead, we will just display all the tags in the edit
tags navigation modal as long as it is visible to the user.
We recently introduced this advice to admins when some translation overrides are outdated or using unknown interpolation keys:
However we missed the case where the original translation key has been renamed or altogether removed. When this happens they are no longer visible in the admin interface, leading to the confusing situation where we say there are outdated translations, but none are shown.
Because we don't explicitly handle this case, some deleted translations were incorrectly marked as having unknown interpolation keys. (This is because I18n.t will return a string like "Translation missing: foo", which obviously has no interpolation keys inside.)
This change adds an additional status, deprecated for TranslationOverride, and the job that checks them will check for this status first, taking precedence over invalid_interpolation_keys. Since the advice only checks for the outdated and invalid_interpolation_keys statuses, this fixes the problem.
The Problem
Clicking on a large image opens lightbox, however the new lightbox currently waits for the first image to finish loading before it finishes loading the lightbox UI correctly (ie. background color). This makes the visual experience feel broken.
Because open() is waiting for the image to load, it doesn't trigger the onOpen callback, which appends a .has-lightbox class to the html tag. The lightbox background color requires that css class to be set for the styles to be applied correctly.
The Solution
This PR prevents blocking when loading loading the first image (image that was clicked) within the lightbox, and therefore allows the css class to be appended to the html tag correctly and as a result fixing the styling issues.
The #setCurrentItem function is async and awaits the loading of preloadItemImages already, so the image will load correctly when complete despite the rest of the UI loading in advance.
The primary motivation is to simplify `eagerLoadRawTemplateModules` which curently introspects the module dependencies (the `imports` at runtime). This is no longer supported in Embroider as the AMD shims do not have any dependencies (since it's managed internally with webpack).
Prior to this commit the `setSiteThemeColor` could mistakenly receive a color with a leading `#` which would cause an invalid color to be send to `postRNWebviewMessage` and would eventually cause a crash if we try to interpolate between this color and another.
The attribute Reviewable#post_options was deprecated (and replaced by #payload) four years ago, and marked for deletion in 2.9.0. This commit removes it.
Using the lastViewedTopicId indiscriminately can cause strange scrolling behavior when navigating to a **different** topic list after viewing a topic. We only want to refocus the topic when going 'back' to the same topic list which originally triggered the navigation.
Previously we had three query parameters to control which tests would be run. The default was to run all core/plugin tests together, which would almost always lead to errors and does not match the way we run tests in CI.
This commit removes the three old parameters (skip_core, skip_plugins and single_plugin), and introduces a new 'target' parameter. This can have a value of 'core', 'plugins', 'all', or a specific plugin name. The default is 'core'. Attempting to use the old parameters will raise an error.
Previously we were implementing scroll reset/memorization on a per-page basis. Many of these approaches relied on the `didInsertElement` hook, which is no longer appropriate since Discourse changed to use the 'loading slider' strategy for page transitions.
This commit rips out all of our custom scroll resetting/memorizing, and implements those things in a generic service. There are two features:
1. After every route transition, scroll to the top of the page
2. When using browser back/forward buttons, restore the last known scroll position for those routes
To opt-out of the behaviour, individual routes can add a scrollOnTransition boolean to their RouteInfo metadata using Ember's `buildRouteInfoMetadata` hook.
The new lightbox was missing the tracked property for items when it was launched earlier as experimental feature flag.
This PR should fix issues experienced when the user clicks between multiple galleries causing the carousel images not to be updated as they were previously not tracked.
Why this change?
Prior to this change, dismissing unreads posts did not publish the
changes across clients for the same user. As a result, users can end up
seeing an unread count being present but saw no topics being loaded when
visiting the `/unread` route.
Why this change?
Prior to this change, the ordering of the tags shown in the email subject
was non-deterministic as there was no specific order specified. This
problem was exposed by a flaky test which we had.
What is the fix?
This commit orders the tags used in the email subject first by the
`Tag#public_topic_count` column in descending order and then the `Tag#name`
column in ascending order.
Simplified query based on SiteSettings to join only relevant user_options rows.
In addition, index was added to 'watched_precedence_over_muted` column in `user_options` table to speed up query
Why this change?
Group mention notifications are currently placed in the "Others" tab
of the user menu which is odd considering that mentioned notifications
are in the reply tab. This commit changes it such that group mention
notifications are displayed in the reply tab as well.
This commit makes sure we don't load all data into memory when doing CSV exports.
The most important change here made to the recently introduced export of chat
messages (3ea31f4). We were loading all data into memory in the first version, with
this commit it's not the case anymore.
Speaking of old exports. Some of them already use find_each, and it worked as
expected, without loading all data into memory. And it will proceed working as
expected after this commit.
In general, I made sure this change didn't break other CSV exports, first manually, and
then by writing system specs for them. Sadly, I haven't managed yet to make those
specs stable, they work fine locally, but flaky in GitHub actions, so I've disabled them
for now.
I'll be making more changes to the CSV exports code soon, those system specs will be
very helpful. I'll be running them locally, and I hope I'll manage to make them stable
while doing that work.
Instead of having to remember every time, just always wait until the
current transaction (if it exists) has committed before clearing any
DistributedCache.
The only exception to this is caches that aren't caching things from
postgres.
This means we have to do the test setup after setting the test
transaction, because doing the test setup involves clearing caches.
Reapplying this - it now doesn't use after_commit if skip_db is set
* FEATURE: Inline topic summary. Cached version accessible to everyone.
Anons and non-members of the `custom_summarization_allowed_groups_map` groups can see cached summaries for any accessible topic. After the first 12 hours and if the posts to summarize have changed, allowed users clicking on the button will automatically re-generate it.
* Ensure chat summaries work and prevent model hallucinations when there are no messages.
In the past, widget implementors would have to subclass the MountWidget component and wire up `didUpdateAttrs` or an observer to trigger a re-render. If that wasn't done, then it could lead to weird behaviors, especially now that page transitions in Discourse do not de-render/re-render components by default.
This commit updates MountWidget so that it re-renders whenever any input arguments change.
Browser capabilities are inherently unconnected to the lifecycle of our app. Making them formally available outside of the service means that they can safely be used in non-app-linked functions without needing risky hacks like `helperContext()` or `discourse-common/lib/get-owner`.
One example of where the old hacks were problematic is the `translateModKey()` utility function. This is called in the root of the `discourse/components/modal/keyboard-shortcuts-help` es6 module. If anything (e.g. a theme/plugin) caused that es6 module to be `require()`d before the application was booted, a fatal error would occur.
Following this commit, `translateModKey()` can safely import and access `capabilities` without needing to worry about the app lifecycle.
The only potential downside to this approach is that the capabilities data now persists across tests. If any tests need to 'stub' capabilities, they will need to revert their changes at the end of the test (e.g. by using Sinon to stub a property).
This commit also updates some legacy references from `capabilities:main` to `service:capabilities`.
These avatar-related helper functions are used in pretty-text, which currently means we load the entire `discourse/lib/utilities` module into the mini-racer when running pretty-text on the server side. This stops us adding any logic or imports to discourse/lib/utilities which may depend on other `discourse/` namespace features.
This commit moves the avatar-related utils into a dedicated module in the `discourse-common` namespace, adds backwards-compatibility shims, and updates the pretty-text config accordingly.
- Unify the silencing methods, use a WeakMap to remember the seen objects
- Export a proper plugin and use the absolute path in the config, instead
of the proprietary config from `broccoli-babel-transpiler`
The latter causes problems in Embroider which doesn't use the broccoli
based babel pipeline.
Some themes were doing `require("i18n").t()`, which was never recommended, but did work prior to f8483295. This commit restores that functionality with a deprecation notice.
What does this commit do??
This commit introduces two changes:
1. As a follow up review comment to
cc463c3e9b, we remove the top level
recipientNames cache in composer message to be a property of the
`ComposerMessage` component instead. Across components, we're more
likely to get a cache miss than a hit since we're caching the entire
recipient array so we can just drop it. If we really need this
optimisation, we should probably use a map and cache the information for
each user instead. However, the request is fairly cheap so we avoid that
optimisation for now.
2. This commit adds a debounce to `_typeReply` as well since we were not
debouncing and the method was being called each time we received the
event.
Why is this change being made?
We've decided that the previous "community" section should look more
like a primary section that holds the most important navigation links
for the site and the word "community" doesn't quite fit that
description. Therefore, we've made the decision to drop the
section heading for the community section.
As part of removing the section heading, the following changes are made
as well:
1. Button to customize the section has been moved to the "footer" of the
"More..." section when `navigation_menu` site setting is set to `sidebar`.
When `navigation_menu` is set to `header dropdown`, a button to customize
the section is shown inline.
2. The section will no longer be collapsable.
3. The title of the section is no longer customisable as it is no longer
displayed. As a technical note, we have not dropped any previous
customisations of the section's title previously in case we have to
bring back the header in the future.
4. The new topic button that was previously present in the header has
been removed alongside the header. Admins can add a custom section
link to the `/new-topic` route if there would like to make it easier for
users to create a new topic in the sidebar.
Generally follows the same pattern as #22520
There are some changes here, notably it uses the addon's babel
settings rather than the app's, and it goes through the same
treatment as the rest of the addon code (which may include more
than just babel).
However, this probably brings us closer to the normal expectations
you have around developing addon code, and in any case, does not
seem to have any effect on the final output:
```
$ diff dist/assets/markdown-it-bundle.js /tmp/dist-before/assets/markdown-it-bundle.js
```
* FIX: Default parameter recipients to create new message via params must be a string
The default parameter recipients was defined as an empty array in:
- route:application#createNewMessageViaParams
- mixin:open-composer#openComposerWithMessageParams
However, in model:composer, targetRecipient is handled as a string as can be
verified due to the existence of the #targetRecipientsArray computed property.
Using the default parameter defined as an array was causing issues with
the `discourse-bcc` plugin when opening the composer using the route
/new-message.
* DEV: Added tests for the composer messages for private messages
* Fix test naming
Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
---------
Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
Currently, the admin/wizard build relies on the addon build getting
triggered first, so that its `treeForAddon()` hook will be called,
and then it can stash the result on the app's options, which is
super fragile. In Embroider the timing works differently so the
trees end up being `undefined`.
This inverts the logic so that it will be discourse core's build
calling these hooks at a specific timing and return the result
rather than coordinating through the options bag.
```
$ diff dist/assets/admin.js dist-after/assets/admin.js
$ diff dist/assets/wizard.js dist-after/assets/wizard.js
```
Currently the I18n module shim return an object. Per AMD/loader.js,
the properties on the object becomes named exports of the module,
i.e. `import { t } from 'I18n';`.
However, this is not how we actually consume this module. We always
do `import I18n from 'I18n';`.
The returned object from the shim (`window.I18n`) does NOT have a
`default` property on it. This is only working because loader.js
has a `makeDefaultExport` feature that defaults to true, which we
are relying on to synthesize the default export for us.
That feature has been noted as undesirable and may some day be
deprecated. In Embroider, it specifically disables the feature in
loader.js.
https://github.com/embroider-build/embroider/issues/539
chapoi
Loudghiri Ahmed
Kelv
Alan Guo Xiang Tan
Ted Johansson
dependabot[bot]
Kris
David Taylor
Jordan Vidrine
Carson Chang
Selase Krakani
Penar Musaraj
Ella E
Bianca Nenciu
Blake Erickson
OsamaSayegh
David Battersby
marstall
Isaac Janzen
Natalie Tay
Jarek Radosz
Joffrey JAFFEUX
Krzysztof Kotlarek
Mark VanLandingham
Emmett Ling
Roman Rizzi
Godfrey Chan
Sam
Martin Brennan
Juan David Martínez Cubillos
Jean
Faizaan Gagan
Jamie Dunstan
Joe
Andrei Prigorshnev
Daniel Waterworth
Bianca Nenciu
Sérgio Saquetim