Commit Graph

475 Commits

Author SHA1 Message Date
Robin Ward 14f9d40e48 FEATURE: Clarify Reviewable User Actions
"Approve" is now "Approve User" and "Delete" is a dropdown with a choice
that allows you to block.
2019-04-17 11:44:17 -04:00
Dan Ungureanu e23c87c442
FIX: Prevent exception in rescue block.
> Job exception: undefined method `id' for nil:NilClass
> clean_up_inactive_users.rb:25:in `rescue in block in execute'
2019-04-16 01:27:39 +03:00
Robin Ward f7ebfb1acc FIX: Only consider pending queued posts for cleaning up uploads 2019-04-12 14:39:32 -04:00
Régis Hanol ab79060468 FIX: don't send activation reminder to staged users 2019-04-10 17:29:08 +02:00
Bianca Nenciu 3d545d66df FEATURE: Send user activation reminders. (#7280) 2019-04-10 16:53:52 +02:00
Tarek Khalil 442fb2facb FEATURE: Remove ignore feature SiteSetting and enable ignore by default (#7349) 2019-04-10 12:54:59 +02:00
Sam Saffron ec1c3559da PERF: correct clean up inactive so it does not clog scheduler
also add a hard limit of 1000 users per job run so we do not clog the
scheduler

destroyer.destroy has a transaction and this can have some serious complications
with the open record set find_each has going
2019-04-09 22:25:05 +10:00
Guo Xiang Tan 6d0e8821f8 FIX: Optimize query and avoid bloating memory in `Jobs::MigrateUploadScheme`. 2019-04-09 15:45:30 +08:00
Vinoth Kannan 914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Guo Xiang Tan 108c231d1c FIX: Clean up `topic_search_data` of trashed topics.
This keeps the index and table smaller.
2019-04-08 16:53:39 +08:00
Guo Xiang Tan c4997ce85f FIX: Don't try and reindex posts that have been trashed. 2019-04-08 16:38:43 +08:00
Robin Ward c9ddc6ce30 FIX: Make sure the site setting works and fix build 2019-04-04 17:44:10 -04:00
Vinoth Kannan 2c7a50860f DEV: disable missing post uploads check by default. 2019-04-05 02:29:10 +05:30
Vinoth Kannan a385f6c48a REFACTOR: create custom field even when missing post uploads not found 2019-04-04 21:49:44 +05:30
Vinoth Kannan 44e87372da FEATURE: Add scheduled job to ensure s3 uploads existence 2019-04-04 20:37:44 +05:30
Robin Ward 588f61f717 FIX: Linting 2019-04-03 18:08:07 -04:00
Vinoth Kannan 3b581de3b8 remove unrelated code
already added in a separate job
2019-04-04 02:26:28 +05:30
Vinoth Kannan b3fb0a7039 FEATURE: ensure consistency of post uploads in cooked content 2019-04-04 02:23:28 +05:30
Robin Ward c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Guo Xiang Tan d151425353
PERF: Delete search data of posts from trashed topics periodically. (#7302)
This keeps both the index and table smaller.
2019-04-03 10:10:41 +08:00
Guo Xiang Tan feb731bffd FIX: Regenerate optimized images instead of migrating from old scheme.
`OptimizedImage.migrate_to_new_scheme` was optimizing optimized images
which we don't need to do. Regnerating the optimized image is way easier.
2019-04-03 09:45:02 +08:00
Guo Xiang Tan d8704c11ca PERF: Better use of index when queueing a topci for search reindex.
Also move `Search::INDEX_VERSION` to `SearchIndexer` which is where the
version is actually being used.
2019-04-02 09:53:37 +08:00
Guo Xiang Tan aa2311a7b0 FIX: Don't reindex posts belonging to a deleted topic for search.
Posts belonging to a deleted topic can't be index for search so we need
to avoid loading those post ids.
2019-04-02 07:36:53 +08:00
Guo Xiang Tan 3fc5dbb045 FIX: Don't attempt to reindex posts that have an empty raw.
If the post ids keep loading, we might end up in a situations where
we're always loading the same post ids over and over again without
indexing anything new.

Follow up to daeda80ada.
2019-04-02 07:13:33 +08:00
Robin Ward 76669bb5a6 FIX: Don't refer to pending review items as flags
They could be queued posts or users, and the notice should reflect that
properly.
2019-04-01 14:46:56 -04:00
Guo Xiang Tan daeda80ada
FIX: Don't index posts with empty `Post#raw` for search. (#7263)
* DEV: Remove unnecessary join in `Jobs::ReindexSearch`.

* FIX: Don't index posts with empty `Post#raw` for search.
2019-04-01 10:06:27 +08:00
Tarek Khalil b1cb95fc23
FEATURE: Introduce ignore duration selection (#7266)
* FEATURE: Introducing new UI for tracking User's ignored or muted states
2019-03-29 10:14:53 +00:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Guo Xiang Tan b58c965aad FIX: Destroy optimized image if attempting to migrate to new scheme fails. 2019-03-26 14:37:34 +08:00
Tarek Khalil fed2dd9148 FEATURE: Add scheduled job to purge expired ignored users (#7211) 2019-03-20 11:01:43 +01:00
Guo Xiang Tan 4020c87680 DEV: Refactor tests for `Jobs::CleanUpInactiveUsers`.
* Remove use of 0 in favor of `TrustLevel.levels[:newuser]`.
* Consolidate two tests into a single one.
* Test that disabling the feature works.
* Avoid loading full ActiveRecord object in test when we only need to
know the existence of the record.
2019-03-19 09:57:21 +08:00
Bianca Nenciu 2347661a74 FEATURE: Clean up inactive users. (#7172) 2019-03-18 16:25:15 +01:00
Tarek Khalil bd6d31c9ec
FEATURE: Add `IgnoredUsersSummary` daily job (#7144)
* FEATURE: Add `IgnoredUsersSummary` daily job

## Why?

This is part of the [Ability to ignore a user feature](https://meta.discourse.org/t/ability-to-ignore-a-user/110254/8).

We want to:

1. Send an automatic group PM that goes out to moderators
2. When {x} users have Ignored the same user, threshold defined by a site setting, default of 5
3. Only send this message every X days which is defined by another site setting
2019-03-14 22:51:43 +00:00
Guo Xiang Tan b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
Dan Ungureanu 32bae48fd3 DEV: Use User#human? User#bot? (#7140) 2019-03-12 07:58:14 +08:00
Gerhard Schlager 1121514799 UX: Localize date format in "new user of the month" message 2019-03-06 21:58:25 +01:00
Robin Ward 78ddc82952 FIX: Respect min_flags_staff_visibility for new flags too
There was a situation where if:

* There were new flags to review that met the visibility threshold

AND

* There were old flags that *didn't* meet the threshold

THEN

a pending flags notification would be sent out. This fixes that case.
Staff should not be notified of flags if they do not meet the threshold
and are old.
2019-01-25 11:27:43 -05:00
Robin Ward 96b2585a91 REFACTOR: Remove unncessary stubs from pending flags reminder
They seem to be calculated fine by the application, and stubbing
makes the tests more brittle and prone to regression.
2019-01-24 13:45:58 -05:00
Robin Ward f32de88dfc FIX: Don't notify of pending flags if min_flags_staff_visibility not met 2019-01-22 11:01:18 -05:00
Guo Xiang Tan 95e3841974 FIX: Remove old reference and use `MiniScheduler::Stat`. 2019-01-18 16:36:11 +08:00
Sam 384135845b FEATURE: introduce ultra_low priority queue
This commit introduces an ultra low priority queue for post rebakes. This
way rebakes can never interfere with regular sidekiq processing for cases
where we perform a large scale rebake.

Additionally it allows Post.rebake_old to be run with rate_limiter: false
to avoid triggering the limiter when rebaking. This is handy for cases
where you want to just force the full rebake and not wait for it to trickle
2019-01-17 14:53:19 +11:00
Saurabh Patel b63b399799 DEV: remove uploaded_meta_id column from category (#6725)
* DEV: remove uploaded_meta_id column from category

* remove uploaded_meta part
2019-01-10 09:37:21 +08:00
Sam e08a3f719c FEATURE: push post rebake regular task to low priority queue
This allows us to run regular rebakes without starving the normal queue.

It additionally adds the ability to specify queue with `Jobs.enqueue` so
we can specifically queue a job with lower priority using the `queue` arg.
2019-01-09 08:57:20 +11:00
Sam f74ef71130 DEV: add transaction and active check to invalidate job
Follows up on 3b76f19668 this per @tgxworld a transaction makes sense here

Very minor
2018-12-13 14:59:56 +11:00
Neil Lalonde 3b76f19668 FIX: invalidating inactive admin emails should mark them as not active 2018-12-12 17:07:49 -05:00
Neil Lalonde a1db15fead FEATURE: require admins to re-validate their email addresses if they haven't been seen for a number of days, configurable with the invalidate_inactive_admin_email_after_days site setting. Social logins are also revoked. Default is 365 days. 2018-12-12 15:32:38 -05:00
David Taylor 9db829134c
FIX: Use database to persist metadata during social registration (#6750)
Previously was using the cookie_store, which is limited to 4kb. This caused issues for providers sending large volumes of metadata about a user.
2018-12-10 15:10:06 +00:00
Saurabh Patel 9e3143445b DEV:add uploaded_meta option in category for category meta image (#6724) 2018-12-07 16:24:07 +01:00
Sam 8e307e633e FIX: posts would not auto rebake unless gravatar download was enabled 2018-12-07 17:03:22 +11:00
Guo Xiang Tan eecd1a7d8c FIX: `Jobs::CleanUpUploads` fails when value of upload data_type is an empty string. 2018-11-30 10:46:39 +08:00
Guo Xiang Tan 81b3bdaabd FIX: Remove site settings override for deprecated url site settings. 2018-11-20 11:42:39 +08:00
Guo Xiang Tan 9e86b425bc FIX: Job to clean up old URL settings when new setting has been set.
Related to 44391ee8ab
2018-11-16 09:33:31 +08:00
Guo Xiang Tan 5b320e6652 DEV: Don't warn about using deprecated site settings in clean up uploads job. 2018-11-14 20:58:06 +08:00
Guo Xiang Tan 44391ee8ab
FEATURE: Upload Site Settings. (#6573) 2018-11-14 15:03:02 +08:00
Guo Xiang Tan 7b44339529 FIX: Prevent uploads used in site settings from being deleted. 2018-11-13 09:15:16 +08:00
Gerhard Schlager 24e5be3f0c FIX: Relative links in translations should work with subfolder 2018-11-08 23:31:05 +00:00
Bianca Nenciu 2070edf889 FIX: Clarify User.group_locked_trust_level.
* Rename User.group_locked_trust_level to User.group_granted_trust_level.

* Remove the column from users table.
2018-11-07 10:27:44 +08:00
Guo Xiang Tan 84d4c81a26 FEATURE: Support backup uploads/downloads directly to/from S3.
This reverts commit 3c59106bac.
2018-10-15 09:43:31 +08:00
Guo Xiang Tan 3c59106bac Revert "FEATURE: Support backup uploads/downloads directly to/from S3."
This reverts commit c29a4dddc1.

We're doing a beta bump soon so un-revert this after that is done.
2018-10-11 11:08:23 +08:00
Gerhard Schlager c29a4dddc1 FEATURE: Support backup uploads/downloads directly to/from S3. 2018-10-11 10:38:43 +08:00
Gerhard Schlager 3a6ab02563 FEATURE: Notify admins when scheduling of backup fails 2018-10-02 15:48:16 +02:00
Sam e1975e293f FIX: when uploads are destroyed clear up avatar refs in user table
This also auto corrects twice daily when we ensure consistency
2018-08-31 14:46:42 +10:00
Guo Xiang Tan 36a7028f19 FEATURE: Clean up `PostReplyKey` records.
* Default retention of 90 days.
2018-08-23 10:40:02 +08:00
Bianca Nenciu 860c1c3dcd FEATURE: Automatically expire keys if not used for a configurable amount of time. (#6264) 2018-08-20 17:36:14 +02:00
Gerhard Schlager b73950692b FIX: Parsing non-existent feed should not fail 2018-08-10 18:37:14 +02:00
Simon Cossar 3cd4dc0f5f Allow users with group_locked_trust_level to be promoted to tl3 (#6249) 2018-08-10 10:42:23 +10:00
Osama Sayegh 880462a41c FEATURE: display out of date themes on admin dashboard
* FEATURE: disaply out of date themes on admin dashboard

* Update copy
2018-08-03 09:53:48 +10:00
Gerhard Schlager a115aae45f Use rchardet instead of charlock_holmes gem 2018-08-01 10:41:20 +02:00
Gerhard Schlager 5d421fb946 FIX: Try respecting charset in HTTP header of RSS feed 2018-08-01 10:41:20 +02:00
Gerhard Schlager ff942ed2f3 FIX: Try detecting encoding of RSS feed 2018-08-01 10:41:20 +02:00
Guo Xiang Tan 87537b679c Drop `reply_key`, `skipped` and `skipped_reason` from `email_logs`. 2018-07-30 11:39:28 +08:00
Guo Xiang Tan fad9c2b971 PERF: Move `EmailLog#reply_key` into new `post_reply_keys` table. 2018-07-24 13:51:53 +08:00
Guo Xiang Tan ae8b0a517f PERF: Split skipped email logs into a seperate table. 2018-07-24 13:14:37 +08:00
Joffrey JAFFEUX 06323f9c89
FIX: takes old dashboard out of caching job 2018-07-21 12:42:03 -04:00
Joffrey JAFFEUX 0972dd2552
FIX: takes dashboard-next out of caching job 2018-07-21 11:13:15 -04:00
Joffrey JAFFEUX 1d5096eb46 FIX: lazy load more reports in dashboard 2018-07-20 23:35:53 -04:00
Joffrey JAFFEUX 1a78e12f4e
FEATURE: part 2 of dashboard improvements
- moderation tab
- sorting/pagination
- improved third party reports support
- trending charts
- better perf
- many fixes
- refactoring
- new reports

Co-Authored-By: Simon Cossar <scossar@users.noreply.github.com>
2018-07-19 14:33:11 -04:00
Régis Hanol 6d6e026e3c FEATURE: selectable avatars 2018-07-18 12:57:43 +02:00
Sam 91266cdabb correct auto bump topic logic 2018-07-17 09:33:33 +10:00
Sam ac0053f491 FEATURE: navigate to first post and auto bump category settings
### navigate_to_first_post_after_read setting for categories

When enabled on categories logged on users will return to OP after
reading the entire category. (useful for documentation categories)

### num_auto_bump_daily

Set a number of topics that will automatically bump daily on a category.

- Every 15 minutes we will check if any category has this setting
- Categories with the setting are shuffled
- We exclude pinned, closed, category description and archived topics
- Maximum of 1 topic for the list of categories is bumped till limit reached per category
- We always try to bump oldest first
- Limit is elastic using a RateLimiter that ensures that we only bump N per day

Also some minor organisation on category settings

Froze strings on category.rb
2018-07-16 18:10:35 +10:00
Guo Xiang Tan 4172e1dd52
FIX: Rename `User#usernames` that clashes with `Group#name`. (#6069) 2018-07-09 16:54:57 +08:00
Guo Xiang Tan f7322c05ad FIX: Avoid `destroy_all` in `Jobs::CleanUpUploads`.
`destroy_all` loads all the relation into memory as once.

See https://github.com/rails/rails/issues/22510
2018-07-02 12:44:36 +08:00
Arpit Jalan 7550e9ff95 FIX: purge unactivated users with a message from non-human users 2018-06-29 13:03:04 +05:30
Ernesto Serrano c5e24da5dd Update poll_mailbox.rb 2018-06-25 16:18:07 +10:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Sam 12fd0b827e PERF: release results in a couple of cases 2018-06-13 15:11:43 +10:00
Guo Xiang Tan d600e71b3d FIX: Clean up stale `UserExport` records daily.
* Add tests for `UserExport.remove_old_exports`
2018-06-05 09:43:40 +08:00
Régis Hanol 6024fb84cc make rubocop happy 2018-06-04 19:06:52 +02:00
Régis Hanol ab8cdd6b5b
Upload.url can't be NULL 2018-06-04 18:43:00 +02:00
Régis Hanol 085eaaf18d
FIX: always delete invalid upload records 2018-06-04 18:40:57 +02:00
Arpit Jalan 1841dd48dc FIX: revert utf-8 encode changes 2018-05-20 17:50:36 +05:30
Arpit Jalan 9512796ef6 FIX: check for blank response when polling feed 2018-05-18 17:00:44 +05:30
Régis Hanol 39aceed63d FIX: don't purge unactivated users with a message 2018-05-16 18:24:11 +02:00
Arpit Jalan 146f634c8f FIX: UTF-8 encode feed response body 2018-05-16 12:59:23 +05:30
Sam 6332d5040d UX: switch dashboard to be the new dashboard
Also:
- add pageviews
- add problems and version sections
2018-05-14 13:07:59 +10:00
Sam bc9e0d46af PERF: use cached reports for dashboard if available 2018-05-14 12:01:44 +10:00
Arpit Jalan 91bf10bd12 FIX: create upload record for exported csv files 2018-04-20 00:27:49 +05:30
Joffrey JAFFEUX 0e414d0890
dashboard next: trending search report
This commit also improves how data is loaded sync and async
2018-04-19 18:19:21 +02:00
Joffrey JAFFEUX 01c061d20d
dashboard next: perf and UI tweaks
* cache CORE reports
* adds backups/uploads section
* few css tweaks
2018-04-18 21:30:41 +02:00
Guo Xiang Tan 7bf9650e96 Remove comment that is no longer accurate. 2018-04-18 14:22:18 +08:00