Commit Graph

31 Commits

Author SHA1 Message Date
Vinoth Kannan 2ae87a27fa make rubocop happy 2019-04-08 17:03:26 +05:30
Guo Xiang Tan f0f3deb32b PERF: Simplify query of `UserStat#update_topic_reply_count`.
For a user with alot of posts, we get a 25% speed up.
2019-04-08 17:48:19 +08:00
Guo Xiang Tan a7a5f90e20 Annotate models. 2019-04-05 17:13:12 +08:00
Sam Saffron 5b837f620b correct unread resetting to handle nulls
Note, to avoid race conditions we are setting last_unread to 10 minutes ago
if there is nothing unread.

This is safer in case of in progress transactions
we don't want to lose unread for any window of time.
2019-04-05 13:42:41 +11:00
Sam Saffron 5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Sam 236c755d62 FIX: do not store key tracking last seen time indefinitely
UserStat has some special logic to keep adding time read if repeat calls
are made in intervals less than 100 seconds. This is called regularly
when we update read timings on a topic.

We only need to cache this key in redis for 100 seconds, however previously
we would keep it forever, 1 key per user. This has potential of bloating
a very large amount of keys for no longer active users in redis.
2018-12-03 08:35:26 +11:00
Guo Xiang Tan 7534042427 DEV: Update annotations. 2018-11-07 11:11:19 +08:00
Arpit Jalan 72be638728 FEATURE: add external details to user fields 2018-09-20 08:10:51 +05:30
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Sam b7023da894 PERF: reduce queries required for post timings
- also freezes a bunch of strings
- bypass active record for an exists query
2018-01-17 15:50:41 +11:00
Neil Lalonde b37e40eea9 FEATURE: show read time in last 60 days 2017-11-16 15:46:51 -05:00
Neil Lalonde 24af9b7d97 FIX: when a topic is deleted, update the post count stats of all user who replied 2017-11-02 18:05:23 -04:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam 0aed2533ac Revert unread optimisation, has too many edge cases 2017-05-26 09:04:13 -04:00
Sam Saffron 7d59ff67b8 adjust qurey to include messages, once everything is read
then mark first_topic_unread_at to be current time
2017-05-25 18:40:32 -04:00
Sam 29fac1ac18 PERF: improve performance of unread queries
Figuring out what unread topics a user has is a very expensive
operation over time.

Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)

When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.

This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.

We use pg functions, instead of nested queries here to assist the
planner.
2017-05-25 15:07:30 -04:00
Régis Hanol cb99f59ec3 reset bounce score when email is successfully changed 2017-02-20 10:37:01 +01:00
Sam 089b1d164c annotate models
(reminder run RAILS_ENV=test bin/annotate once in a while)
2016-05-30 10:45:32 +10:00
Régis Hanol 8e611ec7a1 FEATURE: handle bounced emails 2016-05-02 23:15:32 +02:00
Sam d0ee32f3ce FIX: correct counts on user summary 2016-01-24 16:39:01 +11:00
Sam cd22b6158c PERF: stop mucking with user stats every 15 minutes
(pushed to twice daily)
2014-08-07 14:20:42 +10:00
Sam cb0ecd9ff1 PERF: store topic views in a topic view table
* cut down on storage of the work Topic, 3 times per row (in 2 indexes)
* only store one view per user per topic
* only store one view per ip per topic
2014-08-04 19:07:55 +10:00
Robin Ward 09d7a697bf OPTIMIZATION: Add index to `post_timings` and adjust the query 2014-08-01 13:14:00 -04:00
Sam e907cca62e annotate 2014-07-31 13:14:40 +10:00
Sam 0f9678fe49 FIX: faster update of all badges
Introduced badge triggers, introduced concept of badge that happens due to a post but has the post hidden

Delta badge grant happens once a minute, backed by redis
2014-07-23 11:46:07 +10:00
Sam 45ca83328d Annotate 2014-07-15 11:29:44 +10:00
Sam 6c1c8be794 Work in progress, keeping avatars locally
This introduces a new model to store the avatars and 3 uploads per user (gravatar, system and custom)

user can then pick which they want.
2014-05-27 10:08:03 +10:00
Sam 5c148faeb9 annotations were out of date 2014-03-20 15:35:51 +11:00
Sam 6befdceabf BUGFIX: UserStat spec was over ambitious with its mocking 2014-01-06 16:50:55 +11:00
Sam 5bf26ec34e large refactor, ship a few columns from the user table into user_stats 2013-10-07 15:04:59 +11:00
Robin Ward fcff4e80d1 New `user_stats` table to keep track of queried information on a user.
This is information that is not usually needed when representing a user
and is in a separate table with a has one relationship to avoid querying
it all the time.
2013-09-11 14:50:26 -04:00