Commit Graph

429 Commits

Author SHA1 Message Date
Neil Lalonde 399e937a38 FIX: prevent sending multiple summary emails due to Sidekiq delays 2019-03-22 12:34:34 -04:00
David Taylor a9d5ffbe3d FIX: Prevent critical emails bypassing disable, and improve email test logic
- The test_email job is removed, because it was always being run synchronously (not in sidekiq)
- 34b29f62 added a bypass for critical emails, to match the spec. This removes the bypass, and removes the spec.
- This adapts the specs for 72ffabf6, so that they check for emails being sent
- This reimplements c2797921, allowing test emails to be sent even when emails are disabled
2019-03-22 17:28:43 +08:00
Tarek Khalil a31a35b334 FEATURE: Ignored user notification behaviour should be as a muted user (#7227) 2019-03-21 12:15:34 +01:00
Tarek Khalil 3b59ff0d02 [FEATURE] Disallow ignoring self, admins or moderators users (#7202) 2019-03-20 11:18:46 +01:00
Tarek Khalil fed2dd9148 FEATURE: Add scheduled job to purge expired ignored users (#7211) 2019-03-20 11:01:43 +01:00
Guo Xiang Tan 4020c87680 DEV: Refactor tests for `Jobs::CleanUpInactiveUsers`.
* Remove use of 0 in favor of `TrustLevel.levels[:newuser]`.
* Consolidate two tests into a single one.
* Test that disabling the feature works.
* Avoid loading full ActiveRecord object in test when we only need to
know the existence of the record.
2019-03-19 09:57:21 +08:00
Bianca Nenciu 2347661a74 FEATURE: Clean up inactive users. (#7172) 2019-03-18 16:25:15 +01:00
Penar Musaraj 9334d2f4f7
FEATURE: add more granular user option levels for email notifications (#7143)
Migrates email user options to a new data structure, where `email_always`, `email_direct` and `email_private_messages` are replace by

* `email_messages_level`, with options: `always`, `only_when_away` and `never` (defaults to `always`)
* `email_level`, with options: `always`, `only_when_away` and `never` (defaults to `only_when_away`)
2019-03-15 10:55:11 -04:00
Tarek Khalil bd6d31c9ec
FEATURE: Add `IgnoredUsersSummary` daily job (#7144)
* FEATURE: Add `IgnoredUsersSummary` daily job

## Why?

This is part of the [Ability to ignore a user feature](https://meta.discourse.org/t/ability-to-ignore-a-user/110254/8).

We want to:

1. Send an automatic group PM that goes out to moderators
2. When {x} users have Ignored the same user, threshold defined by a site setting, default of 5
3. Only send this message every X days which is defined by another site setting
2019-03-14 22:51:43 +00:00
Robin Ward fa5a158683 REFACTOR: Move `queue_jobs` out of `SiteSetting`
It is not a setting, and only relevant in specs. The new API is:

```
Jobs.run_later!        # jobs will be thrown on the queue
Jobs.run_immediately!  # jobs will run right away, avoid the queue
```
2019-03-14 10:47:38 -04:00
Guo Xiang Tan b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
Penar Musaraj 95532814df DEV: Make Rubocop happy 2019-03-11 22:33:24 -04:00
Guo Xiang Tan 34b29f62db DEV: Remove the use of stubs and mocks in `Jobs::UserEmail` tests.
We can only be sure that an email is sent when we get a mailer in
`ActionMailer::Deliveries`. A couple of tests were actually incorrect
because it didn't flow through our email sender where there are more
conditions in determining whether an email is sent or not.
2019-03-12 09:39:16 +08:00
Robin Ward d1d9a4f128 Add new `run_jobs_synchronously!` helper for tests
Previously if you wanted to have jobs execute in test mode, you'd have
to do `SiteSetting.queue_jobs = false`, because the opposite of queue
is to execute.

I found this very confusing, so I created a test helper called
`run_jobs_synchronously!` which is much more clear about what it does.
2019-03-11 16:58:35 -04:00
David Taylor fc7938f7e0
REFACTOR: Migrate GoogleOAuth2Authenticator to use ManagedAuthenticator (#7120)
https://meta.discourse.org/t/future-social-authentication-improvements/94691/3
2019-03-07 11:31:04 +00:00
Joffrey JAFFEUX 703c724cf3
REFACTOR: Migrate InstagramAuthenticator to use ManagedAuthenticator (#7081) 2019-03-04 14:54:28 +01:00
Régis Hanol 326d892f5e Aadd 'secondary_emails' field in users export
FIX: escape_comma wasn't working in CSV exports
FIX: group_names field wasn't properly serialized
2019-02-27 10:12:20 +01:00
Vinoth Kannan 484bd82278 FIX: Add onceoff job to remove double quotes from s3 etags 2019-02-14 05:19:41 +05:30
Vinoth Kannan ba9cc83d4c FIX: Destination prefix in S3 inventory configuration is incorrect 2019-02-06 20:51:28 +05:30
Vinoth Kannan ff12c4b2d4 FIX: Bucket name is missing in S3 inventory data path 2019-02-06 19:16:08 +05:30
Vinoth Kannan b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30
Robin Ward 78ddc82952 FIX: Respect min_flags_staff_visibility for new flags too
There was a situation where if:

* There were new flags to review that met the visibility threshold

AND

* There were old flags that *didn't* meet the threshold

THEN

a pending flags notification would be sent out. This fixes that case.
Staff should not be notified of flags if they do not meet the threshold
and are old.
2019-01-25 11:27:43 -05:00
Robin Ward 96b2585a91 REFACTOR: Remove unncessary stubs from pending flags reminder
They seem to be calculated fine by the application, and stubbing
makes the tests more brittle and prone to regression.
2019-01-24 13:45:58 -05:00
Robin Ward f32de88dfc FIX: Don't notify of pending flags if min_flags_staff_visibility not met 2019-01-22 11:01:18 -05:00
Sam 384135845b FEATURE: introduce ultra_low priority queue
This commit introduces an ultra low priority queue for post rebakes. This
way rebakes can never interfere with regular sidekiq processing for cases
where we perform a large scale rebake.

Additionally it allows Post.rebake_old to be run with rate_limiter: false
to avoid triggering the limiter when rebaking. This is handy for cases
where you want to just force the full rebake and not wait for it to trickle
2019-01-17 14:53:19 +11:00
Sam 824c3420e9 DEV: make Jobs.enqueue tests less fragile
Previously we depended on non Sidekiq specific mocking which is not the
official way of testing Sidekiq, this made these tests very fragile

New testing is more robust and complete
2019-01-09 09:51:11 +11:00
Sam e08a3f719c FEATURE: push post rebake regular task to low priority queue
This allows us to run regular rebakes without starving the normal queue.

It additionally adds the ability to specify queue with `Jobs.enqueue` so
we can specifically queue a job with lower priority using the `queue` arg.
2019-01-09 08:57:20 +11:00
David Taylor 5bf16d7d10 FEATURE: Topic timer for bumping a topic in the future 2019-01-04 13:08:04 +00:00
David Taylor 1960236822
FIX: Suspicious login detection (#6772) 2018-12-14 16:30:34 +00:00
Neil Lalonde 3b76f19668 FIX: invalidating inactive admin emails should mark them as not active 2018-12-12 17:07:49 -05:00
Neil Lalonde a1db15fead FEATURE: require admins to re-validate their email addresses if they haven't been seen for a number of days, configurable with the invalidate_inactive_admin_email_after_days site setting. Social logins are also revoked. Default is 365 days. 2018-12-12 15:32:38 -05:00
David Taylor 3fedb2ad20 DEV: Style and performance improvements
Follow-up from 9db8291
2018-12-11 09:58:20 +00:00
David Taylor 9db829134c
FIX: Use database to persist metadata during social registration (#6750)
Previously was using the cookie_store, which is limited to 4kb. This caused issues for providers sending large volumes of metadata about a user.
2018-12-10 15:10:06 +00:00
Rishabh da41a515cd Add test to check if posts are auto rebaked by PeriodicalUpdates job when gravatar download is disabled
follow-up for 8e307e633e
2018-12-10 11:10:57 +11:00
Vinoth Kannan d33d031742
FEATURE: Filter topic and post web hook events by tags (#6726)
* FEATURE: Filter topic and post web hook events by tags

* Add a spec test with unmatched tags
2018-12-05 14:44:06 +05:30
Guo Xiang Tan eecd1a7d8c FIX: `Jobs::CleanUpUploads` fails when value of upload data_type is an empty string. 2018-11-30 10:46:39 +08:00
Penar Musaraj 03deda2147
Upgrade to FontAwesome 5 (take two) (#6673)
* Add missing icons to set

* Revert FA5 revert

 This reverts commit 42572ff

* use new SVG syntax in locales

* Noscript page changes (remove login button, center "powered by" footer text)

* Cast wider net for SVG icons in settings

- include any _icon setting for SVG registry (offers better support for plugin settings)

- let themes store multiple pipe-delimited icons in a setting

- also replaces broken onebox image icon with SVG reference in cooked post processor

* interpolate icons in locales

* Fix composer whisper icon alignment

* Add support for stacked icons

* SECURITY: enforce hostname to match discourse hostname

This ensures that the hostname rails uses for various helpers always matches
the Discourse hostname

* load SVG sprite with pre-initializers

* FIX: enable caching on SVG sprites

* PERF: use JSONP for SVG sprites so they are served from CDN

This avoids needing to deal with CORS for loading of the SVG

Note, added the svg- prefix to the filename so we can quickly tell in
dev tools what the file is

* Add missing SVG sprite JSONP script to CSP

* Upgrade to FA 5.5.0

* Add support for all FA4.7 icons

- adds complete frontend and backend for renamed FA4.7 icons

- improves performance of SvgSprite.bundle and SvgSprite.all_icons

* Fix group avatar flair preview

- adds an endpoint at /svg-sprites/search/:keyword

- adds frontend ajax call that pulls icon in avatar flair preview even when it is not in subset

* Remove FA 4.7 font files
2018-11-26 16:49:57 -05:00
Guo Xiang Tan 81b3bdaabd FIX: Remove site settings override for deprecated url site settings. 2018-11-20 11:42:39 +08:00
Guo Xiang Tan 9e86b425bc FIX: Job to clean up old URL settings when new setting has been set.
Related to 44391ee8ab
2018-11-16 09:33:31 +08:00
Guo Xiang Tan df111259fe More URL site settings into a onceoff job.
* Doing it in a post migration was a bad idea
  because the migration will fail if the site
  is down while trying to download uploads
  which points to the instance. This mainly
  affects self-hosters using `discourse_docker`
  where `./launcher rebuild` will take the
  existing container down.
2018-11-14 20:29:20 +08:00
Bianca Nenciu fce0a0ccc8 FEATURE: Compute distance between logins to generate login alerts. (#6562) 2018-11-14 13:26:47 +01:00
Guo Xiang Tan 44391ee8ab
FEATURE: Upload Site Settings. (#6573) 2018-11-14 15:03:02 +08:00
Guo Xiang Tan 7b44339529 FIX: Prevent uploads used in site settings from being deleted. 2018-11-13 09:15:16 +08:00
Gerhard Schlager 24e5be3f0c FIX: Relative links in translations should work with subfolder 2018-11-08 23:31:05 +00:00
Bianca Nenciu 2070edf889 FIX: Clarify User.group_locked_trust_level.
* Rename User.group_locked_trust_level to User.group_granted_trust_level.

* Remove the column from users table.
2018-11-07 10:27:44 +08:00
Guo Xiang Tan bdb8e9efdb DEV: Remove mocks from specs. 2018-11-07 09:55:58 +08:00
Bianca Nenciu e0ccd36dbe FEATURE: Suspicious logins report. (#6544) 2018-10-30 22:51:58 +00:00
Régis Hanol d17c8df926 Only check for suspicious login for staff members 2018-10-26 00:29:28 +02:00
Bianca Nenciu 6a3767cde7 FEATURE: Warn users via email about suspicious logins. (#6520)
* FEATURE: Warn users via email about suspicious logins.

* DEV: Move suspicious login check to a job.
2018-10-25 09:45:31 +00:00
Kyle Zhao 0f1afad6da FIX: extracted theme JavaScripts for multisite (#6502)
* FIX: extracted theme javascripts for multisite

* onceoff to rebake all theme fields
2018-10-18 17:05:34 +11:00
Neil Lalonde 12f132736b FIX: error looking at users in admin when tl3_promotion_min_duration is set to a very high value 2018-10-11 15:11:48 -04:00
Guo Xiang Tan 1b1ef21481 DEV: Improve `Jobs::CleanUpEmailLogs` specs. 2018-10-08 11:11:20 +08:00
Vinoth Kannan 8430ea927e FIX: Generate webhook payloads before destroy events (#6325) 2018-10-05 16:53:59 +08:00
Guo Xiang Tan cfa7173da3 FIX: Onceoff job to fix missing user profile backgrounds. 2018-10-01 16:26:40 +08:00
Robin Ward 02da022c70
PERF: Quit out of the email job quickly if disabled (#6423)
This prevents sidekiq from doing a bunch of queries when email is
disabled.

Critical emails are a special case and will be sent.
2018-10-01 01:15:45 +08:00
Guo Xiang Tan 767f27929d Rename `Jobs::RecoverPostUploads` to rerun the job take 2. 2018-09-19 22:40:32 +08:00
Guo Xiang Tan aa1af9fc22 FIX: Onceoff job to recover missing post uploads.
This fixes the regression due to 1f636c445b
2018-09-14 09:04:01 +08:00
Sam d9c0dc8687 correct prev commit
s3. did not exists it is s3-
2018-09-05 16:11:44 +10:00
Sam 83e1315e42 FIX: correct urls in uploads table to point at dualstack
Last week we added support for dual stack urls but did not remap the
the old records in the uploads and optimized images table

This caused a few minor edge cases worst was that if you rebaked old
images S3 CDN was not repopulated.
2018-09-05 15:58:04 +10:00
Sam 1866a8e8da correct invalid spec 2018-08-31 15:06:30 +10:00
Gerhard Schlager b2cf725700 FIX: Don't try to send invite email when invite was deleted 2018-08-29 12:43:12 +02:00
Sam 44cf3cf975 FIX: queue heartbeats in readonly modes
If sidekiq is paused or Discourse is in readonly continue to queue
heartbeats

If we do not do that then a master process can end up reaping sidekiq
workers and causing various badness

This also impacts restore which can do weird stuff TM in cases like this
2018-08-29 12:36:59 +10:00
Sam 740308675b FEATURE: erode bounce score every time an email is sent
Introduces a hidden setting (default is 0.1) that erodes bounce score
every time we send an email. This means that erratic failures are less
painful cause system auto corrects
2018-08-28 17:02:12 +10:00
Kyle Zhao e25a6e085e FIX: drop title updates through RSS feeds
can create an update loop
2018-08-28 16:25:04 +10:00
Guo Xiang Tan 36a7028f19 FEATURE: Clean up `PostReplyKey` records.
* Default retention of 90 days.
2018-08-23 10:40:02 +08:00
Guo Xiang Tan 8bdf14834b PERF: Restrict number of skipped email log for `Jobs::UserEmail`. 2018-08-21 11:14:43 +08:00
Guo Xiang Tan 2c70d3f443 Take 2 on ba6f11c521. 2018-08-21 10:06:36 +08:00
Bianca Nenciu 860c1c3dcd FEATURE: Automatically expire keys if not used for a configurable amount of time. (#6264) 2018-08-20 17:36:14 +02:00
Guo Xiang Tan 16c0ebe8a8 Fix the build. 2018-08-17 16:53:07 +08:00
Joffrey JAFFEUX 10a3499d68
uses emoji versions for specs (#6276) 2018-08-16 13:45:30 +02:00
Gerhard Schlager b73950692b FIX: Parsing non-existent feed should not fail 2018-08-10 18:37:14 +02:00
Simon Cossar 3cd4dc0f5f Allow users with group_locked_trust_level to be promoted to tl3 (#6249) 2018-08-10 10:42:23 +10:00
Guo Xiang Tan ba6f11c521 PERF: Only log the first skipped email when user exceeds daily limit.
https://meta.discourse.org/t/cleaning-up-e-mail-logs/39132
2018-08-08 16:25:00 +08:00
Guo Xiang Tan 4e11811321 FIX: `UserAvatar#update_gravatar!` does not update `User#uploaded_avatar`.
https://meta.discourse.org/t/missing-user-profile-pictures/93844/4
2018-08-06 13:29:44 +08:00
Gerhard Schlager 5d421fb946 FIX: Try respecting charset in HTTP header of RSS feed 2018-08-01 10:41:20 +02:00
Gerhard Schlager ff942ed2f3 FIX: Try detecting encoding of RSS feed 2018-08-01 10:41:20 +02:00
Guo Xiang Tan 87537b679c Drop `reply_key`, `skipped` and `skipped_reason` from `email_logs`. 2018-07-30 11:39:28 +08:00
Guo Xiang Tan fad9c2b971 PERF: Move `EmailLog#reply_key` into new `post_reply_keys` table. 2018-07-24 13:51:53 +08:00
Guo Xiang Tan ae8b0a517f PERF: Split skipped email logs into a seperate table. 2018-07-24 13:14:37 +08:00
Vinoth Kannan f8e9190617 FEATURE: Retry web hook when it is failed 2018-07-23 10:12:04 +08:00
Joffrey JAFFEUX 06323f9c89
FIX: takes old dashboard out of caching job 2018-07-21 12:42:03 -04:00
Régis Hanol 6d6e026e3c FEATURE: selectable avatars 2018-07-18 12:57:43 +02:00
Guo Xiang Tan 202b839208 Fix the build. 2018-07-17 17:20:57 +08:00
Guo Xiang Tan 4172e1dd52
FIX: Rename `User#usernames` that clashes with `Group#name`. (#6069) 2018-07-09 16:54:57 +08:00
Gerhard Schlager 50192de046 Fix the specs 2018-06-25 16:18:07 +10:00
Ernesto Serrano 93e60a59f5 Added rspec test 2018-06-25 16:18:07 +10:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Sam 1d9fdcd9d3 Attempt to make spec more robust 2018-06-14 16:10:30 +10:00
Sam 89ad2b5900 DEV: Rails 5.2 upgrade and global gem upgrade
This updates tests to use latest rails 5 practice
and updates ALL dependencies that could be updated

Performance testing shows that performance has not regressed
if anything it is marginally faster now.
2018-06-07 14:21:33 +10:00
Guo Xiang Tan aaea24448e DEV: Add specs for 085eaaf18d. 2018-06-05 10:03:26 +08:00
Guo Xiang Tan dcba320bf6 Remove spec that isn't testing for anything. 2018-06-05 09:30:02 +08:00
Gerhard Schlager c0bab84dc1 Fix random build error
Spec usually failed when there was an UpdateGravatar job for the system user.
2018-06-01 17:34:19 +02:00
Guo Xiang Tan 7fc8a36529 DEV: Take 2 Queue jobs in tests by default.
On my machine this cuts the time taken to run our test suite
from ~11mins to ~9mins.
2018-05-31 16:23:23 +08:00
Guo Xiang Tan 56e9ff6853 Revert "DEV: Queue jobs in tests by default."
Too risky for now

This reverts commit be28154d3b.
2018-05-31 15:34:46 +08:00
Guo Xiang Tan be28154d3b DEV: Queue jobs in tests by default. 2018-05-31 14:45:47 +08:00
Guo Xiang Tan 58fbe74b6f Remove assertion `to_not raise_error`.
If an error is raised the test would fail anyway.
2018-05-25 09:57:08 +08:00
Guo Xiang Tan 43f7cb05c9 FIX: Broken ping event for web hooks due to missing payload. 2018-05-24 15:16:52 +08:00
Guo Xiang Tan bf84037f79 FIX: Payload for webhooks should be current as of the time the event was triggered.
https://meta.discourse.org/t/group-category-tag-user-deleted-webhooks-not-firing/87752
2018-05-21 17:29:58 +08:00
Sam 21e0b7c818 avoid async report pattern and replace with simpler hijack 2018-05-16 16:05:03 +10:00
Régis Hanol a28c58feb1 FIX: automatic group membership when using SSO 2018-05-15 01:48:30 +02:00