```
EXPLAIN ANALYZE SELECT "topics".* FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (
topics.id IN (
SELECT topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 13455 AND gu.group_id =
tg.group_id
WHERE gu.group_id IN (47)
)
)
AND (
topics.id IN (
SELECT ta.topic_id
FROM topic_allowed_users ta
WHERE ta.user_id IN (32852,-10)
)
OR
topics.id IN (
SELECT tg.topic_id
FROM topic_allowed_groups tg
WHERE tg.group_id IN (-10)
)
)
AND (topics.id NOT IN (69933,69995,69988,69984,69968,69973,69971,69952))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 3;
```
Planning time: 1.277 ms
Execution time: 71.577 ms
```
EXPLAIN ANALYZE SELECT "topics".* FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
LEFT JOIN (
SELECT * FROM topic_allowed_groups _tg
LEFT JOIN group_users gu
ON gu.user_id = 13455
AND gu.group_id = _tg.group_id
AND gu.group_id IN (47)
) tg ON topics.id = tg.topic_id
LEFT JOIN topic_allowed_users ta2 ON topics.id = ta2.topic_id AND
ta2.user_id IN (32852)
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (tg.topic_id IS NOT NULL)
AND (ta2.topic_id IS NOT NULL)
AND (topics.id NOT IN (69933,69995,69988,69984,69968,69973,69971,69952))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 3;
```
Planning time: 1.191 ms
Execution time: 0.129 ms
```
EXPLAIN ANALYZE SELECT "topics".*
FROM "topics" LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND
tu.user_id = 13455
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (
topics.id IN (
SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 13455
)
)
AND (
topics.id IN (
SELECT ta.topic_id
FROM topic_allowed_users ta
WHERE ta.user_id IN (2,1995,8307,17621,22980,-10)
)
OR
topics.id IN (
SELECT tg.topic_id
FROM topic_allowed_groups tg
WHERE tg.group_id IN (-10)
)
)
AND (topics.id NOT IN (68559,60069,42145))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at
DESC LIMIT 5;
```
Planning time: 1.196 ms
Execution time: 21.176 ms
```
EXPLAIN ANALYZE SELECT "topics".*
FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
LEFT JOIN topic_allowed_users ta ON topics.id = ta.topic_id AND
ta.user_id = 13455
LEFT JOIN topic_allowed_users ta2 ON topics.id = ta2.topic_id AND
ta2.user_id IN (2,1995,8307,17621,22980,-10)
LEFT JOIN topic_allowed_groups tg ON topics.id = tg.topic_id AND
tg.group_id IN (-10)
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (ta.topic_id IS NOT NULL)
AND (ta2.topic_id IS NOT NULL OR tg.topic_id IS NOT NULL)
AND (topics.id NOT IN (68559,60069,42145))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 5;
```
Planning time: 1.792 ms
Execution time: 2.546 ms
Figuring out what unread topics a user has is a very expensive
operation over time.
Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)
When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.
This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.
We use pg functions, instead of nested queries here to assist the
planner.
- Regular users are not notified of whispers
- Regular users no longer have "stuck" topics in unread
- Additional tracking for staff highest post number
- Remove a bunch of unused columns in topics table
FEATURE: clicking envelope takes you to inbox
Suggested messages works somewhat like suggested topics.
- New show up first (in either group inbox or inbox)
- Then unread (in either group inbox or inbox)
- Finally "related" which are messages with same participants as the current pm.
Messages are now in 3 buckets
- Inbox for all new messages
- Sent for all sent messages
- Archive for all messages you are done with
You can select messages from your Inbox or Sent and move them to your Archive,
you can move messages out of your Archive similarly
Similar concept applied to group messages, except that archiving and unarchiving
will apply to all group members
Old query used to scan the full topics table, on home page
Instead we now perform 2 queries, one for pinned and one for unpinned and merge
results in a 10x improvement on a 1 million topic DB
With this change, the EXPLAIN on the topic query is reduced in cost, and
script/bench.rb results are improved.
Explain topic query, cost at root level:
cost=641.84..641.92 --> cost=570.03..570.11
script/bench.rb result, 50th percentile:
BEFORE
---
categories_admin: 95
home_admin: 80
topic_admin: 33
categories: 71
home: 51
topic: 15
AFTER
---
categories_admin: 78
home_admin: 58
topic_admin: 49
categories: 40
home: 34
topic: 30
Sam
Arpit Jalan
Robin Ward
Guo Xiang Tan
Guo Xiang Tan
Bianca Nenciu
Vinoth Kannan
Neil Lalonde
James Kiesel
Régis Hanol
Joe Buhlig
Jeff Atwood
cpradio
riking