Commit Graph

36 Commits

Author SHA1 Message Date
Sam Saffron 884b9d4b78 PERF: optimize performance of new topics query (home page)
This ensures the bypass is always applied to query plan
also, add frozen strings
2018-02-06 12:37:23 -05:00
Guo Xiang Tan 77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam 29fac1ac18 PERF: improve performance of unread queries
Figuring out what unread topics a user has is a very expensive
operation over time.

Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)

When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.

This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.

We use pg functions, instead of nested queries here to assist the
planner.
2017-05-25 15:07:30 -04:00
Sam c04d4171ff FIX: whisper no longer experimental
- Regular users are not notified of whispers
- Regular users no longer have "stuck" topics in unread
- Additional tracking for staff highest post number
- Remove a bunch of unused columns in topics table
2016-12-02 17:03:31 +11:00
Sam fbdd9c0034 FIX: unread and new count not removing deleted topics on the fly
FIX: unread PMs interfering with unread count
2016-03-30 11:18:07 +11:00
Sam f0e942f647 PERF: move 3 more option columns out of the user table 2016-02-18 16:57:22 +11:00
Sam bc8c6d1a97 PERF: optimise new/unread query
This is part 1 of the work, need some internal cleanup to follow

new query seems to handle very large sets very efficiently
2015-09-29 11:55:09 +10:00
Sam f74c21d2e7 FIX: tracking of new/unread/latest in category/subcategory was broken 2015-09-21 10:36:20 +10:00
Sam 335be272ff FEATURE: implement capping of new/unread
We cap new and unread at 2/5th of SiteSetting.max_tracked_new_unread

This dynamic capping is applied under 2 conditions:

1. New capping is applied once every 15 minutes in the periodical job, this effectively ensures that usually even super active sites are capped at 200 new items

2. Unread capping is applied if a user hits max_tracked_new_unread,
  meaning if new + unread == 500, we defer a job that runs within 15 minutes that will cap user at 200 unread

This logic ensures that at worst case a user gets "bad" numbers for 15 minutes and then the system goes ahead and fixes itself up
2015-09-07 12:03:17 +10:00
Sam 00e59bdc62 FIX: display warning when user is tracking too many topics
FEATURE: allow admins to bump up number of tracked topics if needed
using max_tracked_new_unread
2015-08-25 15:40:50 +10:00
Régis Hanol bef80633b1 FEATURE: global admin override of most of the user preferences 2015-08-21 20:39:21 +02:00
Sam 04e9fdc610 make query less strange, no need for outer join
categories must exist for all topics
2015-07-21 22:45:20 +10:00
Sam 2847ec0af1 this join is brutal, hardcoding user id is way more efficient, but in pg 2015-07-21 22:26:51 +10:00
Sam 3b61b098ab PERF: optimise original query instead of huge rewrite
better perf for all cases
2015-07-21 21:53:54 +10:00
Sam 4491813d22 Revert "Revert "PERF: optimise query that gathers topic tracking state""
This reverts commit 909be09f1a.
2015-07-21 21:48:07 +10:00
Sam 909be09f1a Revert "PERF: optimise query that gathers topic tracking state"
This reverts commit 343e417a55.
2015-07-21 17:35:50 +10:00
Sam 343e417a55 PERF: optimise query that gathers topic tracking state
(this query runs on the front page to figure out new and unread topics)
2015-07-21 17:14:30 +10:00
Sam 803feefd54 MessageBus handles readonly redis now, no need to wrap it 2015-05-04 12:21:00 +10:00
Robin Ward 5b3f99aa50 Don't blow up if Redis switches to READONLY 2015-04-24 14:37:16 -04:00
Sam 832cb9fdc3 FIX: admin check missing from topic tracking state
FIX: handle muted categories correctly
2015-04-17 14:03:08 +10:00
Sam ca2100d012 PERF: work around LIMIT breaking query plan
(CTE is a optimisation boundary, so limit no longer has any effect on plan)
2014-09-10 22:19:49 +10:00
Sam 91dcc56fb3 Revert "PERF: avoid OR in complex query"
This reverts commit a8dc6daa38.
2014-09-10 17:10:57 +10:00
Sam a8dc6daa38 PERF: avoid OR in complex query
10x perf improvement on front page for sitepoint
2014-09-10 16:33:39 +10:00
Sam 4536f772c1 FIX: more reliable topic list counts
- unread was not incrementing when you read last post on topic
- new notifications were being inserted even if they existed in list
- terminology was all mixed up "1 new posts", split to 3 messages
- latest behaves as expected, updating count of new and updated topics
2014-08-05 13:27:34 +10:00
Robin Ward fb8dda7f42 FIX: We should use `category_id` instead of `category_name` to perform
operations, now that the subcategory names are not unique.
2014-07-16 15:40:35 -04:00
Sam fed8f7c6f2 BUGFIX: muted categories showing up in new via message bus 2014-06-18 11:21:40 +10:00
Sam 41b15d1ad1 BUGFIX: not notifying unread and new counts correctly 2014-06-06 15:17:02 +10:00
Robin Ward 4b61d6af28 BUGFIX: new date was incorrect 2014-03-03 16:11:59 -05:00
Robin Ward 3abe84941f FEATURE: Add `new_since` column to `UserStat` so we can reset the "New"
date.
2014-03-03 15:47:00 -05:00
Robin Ward fee9870427 FIX: New counts were taking into account muted categories 2014-02-27 12:35:33 -05:00
Robin Ward 9267c162a1 FIX: Showing Unread(1) when you weren't tracking the topic. Also
includes performance fix when having MANY new or unread topics.
2014-02-26 15:39:45 -05:00
Sam 6544f39bcc fix topic counts not updating automatically in various spots (top menu / categories page / drop down) 2013-11-25 17:37:51 +11:00
Sam ecf17cfebb work in progress, add fidelity to category group permissions (full, create posts, readonly) 2013-07-16 15:46:11 +10:00
Sam 6ad86aa305 read state is now published 2013-05-30 16:49:58 +10:00
Sam e93b7a3b20 more progress towards live unread and new counts, unread message implemented, still to implement delete messages 2013-05-30 16:49:57 +10:00