Commit Graph

223 Commits

Author SHA1 Message Date
Sam Saffron 0c35b8b420 FEATURE: add suggested_topics_unread_max_days_old
This new site setting determines the maximum age of unread topics in
suggested. By default if you have any unread topics older than 90 days
they will be omitted from suggested.

This change was added for 2 reasons:

1. A performance safeguard, some users tend to collect a huge amount of
read state so it becomes super expensive to find unread

2. People who collect a large amount of unread are much more interested in
recent unread topics vs ancient unread topics, this makes suggested more
relevant

Also, this is a minor speed up for tests cause 3 expensive tests became 1.
2019-04-16 17:52:10 +10:00
Sam Saffron 0744d2ef73 PERF: speed up suggested unread
Previously we tried using a sub-query which has some terrible performance
implications, by adding 1 extra query we can eliminate the PG issue

A join to user_stats also is prone to the same slowdown
2019-04-05 15:55:29 +11:00
Sam Saffron 5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Sam 94b8ba4f8f FIX: remove slow platform detection from server side
Historically due to https://meta.discourse.org/t/why-is-discourse-so-slow-on-android/8823
we decreased page sizes of both home page and topic page on android by half.

This was done on the server side and as a side effect and caused page sizes on android
to mismatch between Android and non Android.

Unfortunately about a year ago googlebot started pretending it is Android,
this cause Google to start indexing pages as what android would see. So
it saw double the amount of pages in the index as what exists on desktop.
This in turn caused double the amount of indexing work and a large amount
of broken links on long topics.

This fix removes all special behavior which is no longer needed due to
other performance work in Discourse including raw handlebars on home page
and virtual dom on topic pages.

I tested we do not need this on Blu Advance 5.0 it has 1.3 GHZ mediatec mt6580
This phone retails for around $50 USD.

If we decide long term that we want any hacks like this we will shift them
to the client side. It can just hold data in memory without rendering.
2018-12-13 13:57:05 +11:00
David Taylor 6c71395bf6
FIX: Only hide shared draft topics from `latest` (#6737)
Previously we were hiding them from all topic lists, which can result in
topics being "stuck" in an unread state with no easy way to clear them.
2018-12-07 12:44:23 +00:00
David Taylor 0b1d660876
UX: Make shared drafts behaviour consistent for non-staff users (#6734)
This makes it easier to diagnose the problem when a public category
is set as the 'shared drafts category'. Doing this is not recommended.
2018-12-06 18:59:29 +00:00
Sam e17a13ce19 FEATURE: additional "related messages" section
This splits out previous message correspondence from suggeted and instead
has a dedicated section called "related messages"
2018-11-12 13:04:42 +11:00
Sam 5c86e2d749 tweaks to related message list generation
- exclude users in groups I am in from related message search
- correctly limit number of related messages
2018-10-29 16:09:58 +11:00
Sam 9933059426 FEATURE: push related PMs to take first 3 slots
Previously the related PMs were last meaning you would have to work through
all unread to see them.

Also amends it so it either asks for related by group OR user not both.
2018-10-29 10:47:59 +11:00
Arpit Jalan c0bb04d89d FIX: convert tag string to array when filtering topic list by tags 2018-10-08 08:56:25 +05:30
David Taylor 9bf522f227
FEATURE: Mixed case tagging (#6454)
- By default, behaviour is not changed: tags are made lowercase upon creation and edit.

- If force_lowercase_tags is disabled, then mixed case tags are allowed.

- Tags must remain case-insensitively unique. This is enforced by ActiveRecord and Postgres.

- A migration is added to provide a `UNIQUE` index on `lower(name)`. Migration includes a safety to correct any current tags that do not meet the criteria.

- A `where_name` scope is added to `models/tag.rb`, to allow easy case-insensitive lookups. This is used instead of `Tag.where(name: "blah")`.

- URLs remain lowercase. Mixed case URLs are functional, but have the lowercase equivalent as the canonical.
2018-10-05 10:23:52 +01:00
Gerhard Schlager 26082688d1 FIX: Zero is a valid value for the page parameter 2018-09-05 20:43:05 +02:00
Gerhard Schlager f33433bf9e Validation of params should restrict to max int (#6331)
* FIX: Validation of params should restrict to max int

* FIX: Send status 400 when "page" param isn't between 1 and max int
2018-09-03 14:45:32 +10:00
Gerhard Schlager 937ab3f213 FIX: Validation of min_posts and max_posts didn't work 2018-08-16 10:36:53 +02:00
Sam 38c10a3dc2 correct the validator 2018-08-15 14:56:24 +10:00
Sam 06f82a7d72 correct exception handling, always do to_i in array 2018-08-15 11:31:42 +10:00
Sam bc47148d35 add validation to exclude_category_ids 2018-08-15 09:53:28 +10:00
Régis Hanol 12bab65167 FIX: going from /categories to /latest on mobile might break infinite scrolling 2018-08-15 01:22:03 +02:00
Gerhard Schlager ba0e322fd0 FIX: Validation of topic params broke discourse-assign 2018-08-14 18:45:46 +02:00
Sam ad5f502332 FIX: add a basic validator for topic params
This cuts down on log noise when people try out sql injection
2018-08-14 17:01:04 +10:00
Guo Xiang Tan ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Robin Ward eab64710ff FIX: Shared draft performance fix + missing avatars 2018-03-28 16:11:43 -04:00
Robin Ward 4b5977aa6a Revert "PERF: Don't join on shared drafts unless you have to"
This reverts commit efedd9745f.
2018-03-28 15:35:13 -04:00
Robin Ward efedd9745f PERF: Don't join on shared drafts unless you have to 2018-03-28 13:57:39 -04:00
Robin Ward 2b161a2391 FIX: Don't include shared drafts in global latest 2018-03-26 10:43:55 -04:00
Robin Ward c686ae8d8f FIX: Ambiguous column name 2018-03-22 16:38:53 -04:00
Robin Ward 41fc8e32e2 FIX: N+1 query for shared drafts 2018-03-22 16:24:30 -04:00
Robin Ward b9abd7dc9e FEATURE: Shared Drafts
This feature can be enabled by choosing a destination for the
`shared drafts category` site setting.

* Staff members can create shared drafts, choosing a destination
category for the topic when it is published.

* Shared Drafts can be viewed in their category, or above the
topic list for the destination category where it will end up.

* When the shared draft is ready, it can be published to the
appropriate category by clicking a button on the topic view.

* When published, Drafts change their timestamps to the current
time, and any edits to the original post are removed.
2018-03-20 17:15:26 -04:00
Guo Xiang Tan ec57ca54b5 FEATURE: Admins should be able to view PMs of any group. 2018-03-19 14:12:01 +08:00
Guo Xiang Tan a35227918f UX: Display group topics in a topic list. 2018-03-15 11:37:55 +08:00
Arpit Jalan 24338fbbe8 FEATURE: replace PM tags dropdown with a dedicated tags page 2018-03-13 13:06:58 +05:30
Arpit Jalan c29660c8f1 FEATURE: filter personal messages by tags 2018-03-08 14:42:07 +05:30
Vinoth Kannan 7cbda949f1 REFACTOR: New spec tests and code improvement 2018-02-22 20:27:02 +05:30
Vinoth Kannan 2b509eaa91
Merge branch 'master' into pm-tags 2018-02-21 23:55:59 +05:30
Vinoth Kannan 84ce1acfef FEATURE: Allow staffs to tag PMs 2018-02-21 20:11:46 +05:30
Sam de5418a905 correct search in topic list 2018-02-20 20:47:44 +11:00
Arpit Jalan ff0376a80b rename 'enable_private_messages' to 'enable_personal_messages' 2018-02-01 13:25:29 +05:30
Robin Ward 17ebfd1715 FIX: Don't show suggested messages if private messages are disabled 2018-01-23 12:05:44 -05:00
Sam fcfce3e426 PERF: avoid expensive OR clause query info more efficiently 2018-01-15 16:38:58 +11:00
Sam 38c018a84b FIX: invalid cache for parent category with limit_suggested_to_category 2018-01-15 16:13:29 +11:00
Sam 41a604a764 PERF: improve perf of topic suggested query with limit_suggested_to_category 2018-01-15 15:32:25 +11:00
Guo Xiang Tan 385372e384 Revert "PERF: Reduce number of topics to filter while querying for unread."
This reverts commit c06b782cab.
2017-11-20 11:49:09 +08:00
Guo Xiang Tan c06b782cab PERF: Reduce number of topics to filter while querying for unread. 2017-09-26 12:36:52 +08:00
Guo Xiang Tan 77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan 1a9762a9c0 Fixed private group messages being exposed in suggested topics.
Revert "Revert "PERF: Avoid unnecessary expensive joins if possible.""

This reverts commit d9714c21c8.
2017-09-15 23:40:53 +08:00
Guo Xiang Tan d9714c21c8 Revert "PERF: Avoid unnecessary expensive joins if possible."
This reverts commit f3fadf41b7.

* This ended up exposing group pms to users that are not part
  of a group.
2017-09-15 22:22:07 +08:00
Guo Xiang Tan de46d59a94 Remove comment that is no longer relevant. 2017-09-14 19:57:50 +08:00
Guo Xiang Tan 195982fd41 PERF: Remove N+1 query when generating posters summary. 2017-09-14 12:07:35 +08:00
Guo Xiang Tan f3fadf41b7 PERF: Avoid unnecessary expensive joins if possible.
```
EXPLAIN ANALYZE SELECT  "topics".* FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (
  topics.id IN (
    SELECT topic_id
    FROM topic_allowed_groups tg
    JOIN group_users gu ON gu.user_id = 13455 AND gu.group_id =
tg.group_id
    WHERE gu.group_id IN (47)
  )
)
AND (
  topics.id IN (
    SELECT ta.topic_id
    FROM topic_allowed_users ta
    WHERE ta.user_id IN (32852,-10)
  )
  OR
  topics.id IN (
    SELECT tg.topic_id
    FROM topic_allowed_groups tg
    WHERE tg.group_id IN (-10)
  )
)
AND (topics.id NOT IN (69933,69995,69988,69984,69968,69973,69971,69952))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 3;
```

Planning time: 1.277 ms
Execution time: 71.577 ms

```
EXPLAIN ANALYZE SELECT  "topics".* FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
LEFT JOIN (
  SELECT * FROM topic_allowed_groups _tg
  LEFT JOIN group_users gu
  ON gu.user_id = 13455
  AND gu.group_id = _tg.group_id
  AND gu.group_id IN (47)
) tg ON topics.id = tg.topic_id
LEFT JOIN topic_allowed_users ta2 ON topics.id = ta2.topic_id AND
ta2.user_id IN (32852)
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (tg.topic_id IS NOT NULL)
AND (ta2.topic_id IS NOT NULL)
AND (topics.id NOT IN (69933,69995,69988,69984,69968,69973,69971,69952))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 3;
```

Planning time: 1.191 ms
Execution time: 0.129 ms
2017-09-14 11:18:58 +08:00
Guo Xiang Tan 28148197d6 PERF: Avoid `NOT IN (<subquery>>` which can get really slow.
```
EXPLAIN ANALYZE SELECT  "topics".*
FROM "topics" LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND
tu.user_id = 13455
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (
  topics.id IN (
    SELECT topic_id
    FROM topic_allowed_users
    WHERE user_id = 13455
   )
)
AND (
  topics.id IN (
    SELECT ta.topic_id
    FROM topic_allowed_users ta
    WHERE ta.user_id IN (2,1995,8307,17621,22980,-10)
  )
  OR
  topics.id IN (
    SELECT tg.topic_id
    FROM topic_allowed_groups tg
    WHERE tg.group_id IN (-10)
  )
)
AND (topics.id NOT IN (68559,60069,42145))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at
DESC LIMIT 5;
```

Planning time: 1.196 ms
Execution time: 21.176 ms

```
EXPLAIN ANALYZE SELECT  "topics".*
FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
LEFT JOIN topic_allowed_users ta ON topics.id = ta.topic_id AND
ta.user_id = 13455
LEFT JOIN topic_allowed_users ta2 ON topics.id = ta2.topic_id AND
ta2.user_id IN (2,1995,8307,17621,22980,-10)
LEFT JOIN topic_allowed_groups tg ON topics.id = tg.topic_id AND
tg.group_id IN (-10)
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (ta.topic_id IS NOT NULL)
AND (ta2.topic_id IS NOT NULL OR tg.topic_id IS NOT NULL)
AND (topics.id NOT IN (68559,60069,42145))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 5;
```

Planning time: 1.792 ms
Execution time: 2.546 ms
2017-09-13 22:28:33 +08:00