Commit Graph

325 Commits

Author SHA1 Message Date
Neil Lalonde ced7e9a691 FEATURE: control which web crawlers can access using a whitelist or blacklist 2018-03-22 15:41:02 -04:00
Sam 86d12bd44b FEATURE: search within title using in:title
Also

- Significantly improved search ranking, title is treated most strongly
- Adds tag names to the index
- Run search re-indexer more aggressively
- Re-index topic and all posts on category change
2018-02-20 14:41:21 +11:00
Sam d8b4627fc8 we have to define this for tests to pass 2018-02-15 13:30:34 +11:00
Sam b5b866aab3 oops 2018-02-15 13:13:31 +11:00
Sam c89b42c488 PERF: only require the rss library if used
Before:

Total allocated: 257909321 bytes (2514134 objects)
Total retained:  39681579 bytes (343387 objects)

allocated memory by gem
-----------------------------------
  42875979  rss

retained memory by gem
-----------------------------------
   2080188  rss

retained objects by gem
-----------------------------------
     13052  rss

After:

Total allocated: 210562047 bytes (2252030 objects)
Total retained:  37433816 bytes (328635 objects)

----

So, 2 less megabytes on boot and 13000 objects stuck in ruby heaps forever.
2018-02-15 13:11:33 +11:00
Régis Hanol 7d501058d7 FIX: don't grant new user of the month badge to suspended users or based on deleted topics/posts 2018-02-13 17:32:38 +01:00
Neil Lalonde 76c309fe6b PERF: a faster way to count tags used per category 2018-02-12 15:16:47 -05:00
Neil Lalonde d7e90edeb5 PERF: Tl3Promotions job can limit the number of TL3 candidates by using some simple requirements in the query 2018-02-05 17:54:36 -05:00
Régis Hanol 130974d58d FIX: automatically unsilence users 2018-02-05 17:45:47 +01:00
Régis Hanol 77ac14d475 prevent users with a tiny number of posts to dominate the 'new user of the month' leaderboard 2018-01-29 12:09:17 +01:00
Neil Lalonde 2493648f9c PERF: calculate topic_counts for tags in an async job so tag queries that include counts are much faster 2018-01-12 11:03:03 -05:00
Robin Ward 69a90f31fb FEATURE: Allow Forums to disable the Backups feature 2017-12-21 15:22:04 -05:00
Neil Lalonde f5a3be750c PERF: add option to limit how many categories are processed in one call to CategoryFeaturedTopic.feature_topics (#5446) 2017-12-20 13:42:29 +11:00
scossar 11050e5d10 Don't override count value that has been set by query.count 2017-12-13 18:48:46 -08:00
Kyle Zhao 5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Robin Ward 77f90876d3 REFACTOR: Track manual locked user levels separately from groups 2017-11-27 11:23:44 -05:00
Régis Hanol 4addc5e329 Add missing contexts when destroying users 2017-11-22 15:43:54 +01:00
Régis Hanol c4d0eb802e FIX: delete upload records when sha is null 2017-11-21 10:20:42 +01:00
Régis Hanol 2d48caffdf FIX: be more lenient when deleting a custom emoji 2017-11-20 23:50:23 +01:00
Robin Ward 971e302ff2 FEATURE: Support an end date for user silencing 2017-11-14 13:20:19 -05:00
Régis Hanol ba2209f7d7 FIX: always clean up uploads with no sha1 2017-11-14 10:56:10 +01:00
Régis Hanol 7370adeae3 FIX: don't delete uploads referenced in drafts or queued posts when using the short_url 2017-11-13 15:01:31 +01:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Guo Xiang Tan 6def5a344a Let's be more aggressive with purging old WebHookEvent records. 2017-11-08 14:15:34 +08:00
Guo Xiang Tan 6090994cdf FEATURE: Retain the latest 30 days of WebHookEvent records by default. 2017-11-08 14:11:01 +08:00
Guo Xiang Tan 0abc5f90cd FIX: Broken link in new user of the month post. 2017-10-30 13:11:29 +08:00
Guo Xiang Tan d5e369d6cf Fix weird Rubocop error. 2017-10-23 12:30:28 +08:00
Sam 4ea87b5ab8 Merge branch 's3_refactor' 2017-10-09 10:27:52 +11:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan 4ba5e678d8 Require dependencies to enable live reload in dev for Sidekiq. 2017-10-06 11:39:00 +08:00
Kyle Zhao ac666ddf17 PollFeed: check 'content:encoded' for content first 2017-10-02 01:16:11 -04:00
Guo Xiang Tan d79fee9ff3 Fix undefined method for Nil class error. 2017-09-28 17:38:53 +08:00
Robin Ward fa41913ba5 FIX: Don't count private content in New User of the Month 2017-09-27 15:03:24 -04:00
Robin Ward d1ebc62065 The ability to display errors on flagging actions. 2017-09-25 12:28:01 -04:00
Robin Ward 677b016387 Send a suspension message via email to a user 2017-09-25 12:26:41 -04:00
Neil Lalonde 16fe7aa307 FEATURE: automatically handle flags and posts that have been waiting in a queue for a long time. Flags will be deferred. Posts waiting for approval will be rejected. Control how old the records need to be with the auto_handle_queued_age site setting. 2017-09-14 12:01:06 -04:00
Neil Lalonde beea5cac48 FIX: send the queued posts reminder as a message to moderators instead of an email to the contact_email 2017-09-12 18:00:51 -04:00
Neil Lalonde 2c56f8df7c FEATURE: show tags in search results 2017-08-25 11:52:59 -04:00
Sam Saffron 56f7b4e01e PERF: reindex search data without loading large post counts 2017-08-16 08:18:59 -04:00
Erick Guan 6e59149a77 FIX: rebuild index when engine replaced (#5021) 2017-08-16 07:38:34 -04:00
Leo McArdle 836dee1120 FIX: add additional email to tests and clean up resulting mess 2017-07-31 22:27:29 +00:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Robin Ward 6b6ad9391b Clean up job for search logs 2017-07-14 14:30:58 -04:00
Guo Xiang Tan 1ab60d83df PERF: Terminate scheduled job earlier if badge is not enabled. 2017-06-26 10:13:58 +09:00
Guo Xiang Tan 6cc174ca82 PERF: Fetch users in batches in grant anniversary badge job. 2017-06-23 16:54:35 +09:00
Robin Ward d92c938491 FIX: Don't raise an error cleaning up uploads if a path is nil 2017-06-08 14:31:58 -04:00
Régis Hanol e8f0771dc9 FIX: the 'clean_up_uploads' jobs would delete images used in site settings
when they were entered using absolute URLs, with the CDN or simple a different format than the one used in the database
2017-06-07 22:53:15 +02:00
Sam 0aed2533ac Revert unread optimisation, has too many edge cases 2017-05-26 09:04:13 -04:00
Sam 29fac1ac18 PERF: improve performance of unread queries
Figuring out what unread topics a user has is a very expensive
operation over time.

Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)

When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.

This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.

We use pg functions, instead of nested queries here to assist the
planner.
2017-05-25 15:07:30 -04:00
Robin Ward b78b010757 FIX: Fixes to grant new user of the month badge 2017-05-24 11:36:24 -04:00