Commit Graph

478 Commits

Author SHA1 Message Date
Guo Xiang Tan 8165ceb320 Make rubocop happy. 2019-05-13 09:55:44 +08:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan f04518fdf9 DEV: Reduce number of jobs enqueued.
Apply code review suggestion from
405ba00c08.
2019-05-08 15:58:08 +08:00
Guo Xiang Tan 405ba00c08 FEATURE: Create notifications on wiki edits for watching users.
* Moves creation of notification into background job.
2019-05-08 15:31:10 +08:00
Guo Xiang Tan 451f7842ff DEV: More `send` -> `public_send`. 2019-05-07 10:05:58 +08:00
Guo Xiang Tan 152238b4cf DEV: Prefer `public_send` over `send`. 2019-05-07 09:33:21 +08:00
Sam Saffron 9be70a22cd DEV: introduce new API to look up dynamic site setting
This removes all uses of both `send` and `public_send` from consumers of
SiteSetting and instead introduces a `get` helper for dynamic lookup

This leads to much cleaner and safer code long term as we are always explicit
to test that a site setting is really there before sending an arbitrary
string to the class

It also removes a couple of risky stubs from the auth provider test
2019-05-07 11:00:30 +10:00
Guo Xiang Tan ebca588fd0 DEV: Remove unused line of code. 2019-05-02 16:54:10 +08:00
Vinoth Kannan 9c78c18256 Check both site and global setting for s3 enabled value. 2019-05-01 10:49:53 +05:30
Robin Ward 404b35bd04 FEATURE: Category Reviewable by Group
Allow a group to review content in a particular category.
2019-04-30 15:23:06 -04:00
Robin Ward fca3f53e9c FIX: Safer scoring with concurrency 2019-04-23 15:53:37 -04:00
Robin Ward 70097966ed FIX: Don't add the reviewable score twice 2019-04-23 15:46:18 -04:00
Gerhard Schlager a7bc1ecbae FEATURE: Add support for Unicode usernames and group names
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2019-04-23 13:00:27 +02:00
Michał Frąckiewicz 8f7cfc29db FIX: Detect SNS notifications for SES correctly (#7284)
* Fix handling SNS notifications for AWS SES

This fixes detection of email bounce by:
- removing hard requirement for email ID, ID in webhook msg never equals this in email_log
- gets bounce_score from user stats instead of nonexistent field in webhook msg

* Remove empty line

* Prettify access to EmailLog for parsing SNS notification

Co-Authored-By: SystemZ <SystemZ@users.noreply.github.com>
2019-04-23 13:03:31 +10:00
Guo Xiang Tan 0210b0aabc REFACTOR: Prefer accessing instance variables directly.
* No need to wrap instance variables with another method to read it.
* Remove memoization that wasn't really memozing anything.
2019-04-20 09:39:25 +08:00
Tarek Khalil 6e46197bc8
FIX: Disable webhooks on 410 and 404 HTTP responses (#7392)
FIX: Disable webhooks on 410 and 404 HTTP responses (#7392)
2019-04-18 12:36:37 +01:00
Tarek Khalil 02a9429c38
REFACTOR: Quick refactor of the webhook event emitter job (#7385)
* REFACTOR: Quick refactor of the webhook event emitter job
2019-04-17 10:03:23 +01:00
Robin Ward ba6d4b2a8d FIX: Better handling for toggling `must_approve_users`
If you turn it on now, default all users to approved since they were
previously. Also support approving a user that doesn't have a reviewable
record (it will be created first.)

This also includes a refactor to move class method calls to
`DiscourseEvent` into an initializer. Otherwise the load order of
classes makes a difference in the test environment and some settings
might be triggered and others not, randomly.
2019-04-16 15:56:35 -04:00
Guo Xiang Tan d6e45864ce Fix failing spec. 2019-04-16 09:59:35 +08:00
Guo Xiang Tan ce4c8e957b PERF: Avoid looking up the same group multiple times during bulk invite.
Also cache the guardian check because it does a query to check if the
user is an owner of the group.
2019-04-16 09:42:25 +08:00
Guo Xiang Tan 25514550f0 FIX: Bulk invite should skip invite and add existing users to groups. 2019-04-16 09:42:25 +08:00
Guo Xiang Tan 6e6cdfaf80 FIX: Bulk invite should not add users to automatic groups.
* Also checks that the user creating the bulk invite has permission.
2019-04-16 09:42:25 +08:00
Arpit Jalan 594674703c
FIX: properly log webhook errors in UI on rescue (#7376) 2019-04-15 12:19:48 +05:30
Robin Ward c0c236f36e FIX: Better to put a ReviewableUser back into pending than silently fail 2019-04-11 16:36:47 -04:00
Robin Ward d21dd521d2 FIX: Don't try to create a reviewable user twice 2019-04-11 12:30:54 -04:00
Robin Ward 331a809738 FEATURE: Display the reason for many reviewable items
Queued Posts and Users will now display a reason why they are in the
review queue.
2019-04-11 12:04:45 -04:00
Maja Komel ca33d091b3 FIX: don't trigger notifications when changing category/tags of unlisted topics (#7323) 2019-04-05 15:06:38 +02:00
Arpit Jalan f87b35e6f4 FIX: do not raise exception if the file is missing 2019-04-04 15:36:51 +05:30
Robin Ward 111a502231 FIX: Deleting Users should work nicely with Reviewable Users
"Rejecting" a user in the queue is equivalent to deleting them, which
would then making it impossible to review rejected users. Now we store
information about the user in the payload so if they are deleted things
still display in the Rejected view.

Secondly, if a user is destroyed outside of the review queue, it will
now automatically "Reject" that queue item.
2019-04-03 16:42:39 -04:00
Robin Ward 82bddcbe51 FIX: Don't create two reviewable scores for a user 2019-04-03 16:03:32 -04:00
Robin Ward c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Guo Xiang Tan 8794d940d3 Revert `update_columns` -> `update!` when rebaking/post-processing post.
`update!` goes through validation which means old posts that doesn't
adhere to the existing validations will raise an error.
2019-04-01 16:29:00 +08:00
Guo Xiang Tan cfd507822f
PERF: Improve quality of `PostSearchData#raw_data`. (#7275)
This commit fixes the follow quality issue with `PostSearchData#raw_data`:

1. URLs are being tokenized and links with similar href and characters
are being duplicated in the raw data.

`Post#cooked`:

```
<p><a href=\"https://meta.discourse.org/some.png\" class=\"onebox\" target=\"_blank\" rel=\"nofollow noopener\">https://meta.discourse.org/some.png</a></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png discourse org/some png https://meta.discourse.org/some.png discourse org/some png
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png meta discourse org
```

2. Ligthbox being included in search pollutes the
`PostSearchData#raw_data` unncessarily.

From 28 March 2018 to 28 March 2019, searches for the term `image` on
`meta.discourse.org` had a click through rate of 2.1%. Non-lightboxed images are not included in indexing for search yet we were indexing content within a lightbox. Also, search for terms like `image` was affected we were using `Pasted image` as the filename for
uploads that were pasted.

`Post#cooked`

```
<p>Let me see how I can fix this image<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://meta.discourse.org/some.png\" title=\"some.png\" rel=\"nofollow noopener\"><img src=\"https://meta.discourse.org/some.png\" width=\"275\" height=\"299\"><div class=\"meta\">\n<svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#far-image\"></use></svg><span class=\"filename\">some.png</span><span class=\"informations\">1750×2000</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#discourse-expand\"></use></svg>\n</div></a></div></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image some.png png https://meta.discourse.org/some.png discourse org/some png some.png png 1750×2000
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image
```

In terms of indexing performance, we now have to parse the given HTML
through nokogiri twice. However performance is not a huge worry here since a string length of 194170 takes only 30ms
to scrub plus the indexing takes place in a background job.
2019-04-01 10:14:29 +08:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
David Taylor 95d5819218 FIX: Re-download hotlinked optimized images (#7249)
* FIX: Download local images, even if download remote is disabled
2019-03-27 21:31:12 +01:00
Neil Lalonde 399e937a38 FIX: prevent sending multiple summary emails due to Sidekiq delays 2019-03-22 12:34:34 -04:00
David Taylor a9d5ffbe3d FIX: Prevent critical emails bypassing disable, and improve email test logic
- The test_email job is removed, because it was always being run synchronously (not in sidekiq)
- 34b29f62 added a bypass for critical emails, to match the spec. This removes the bypass, and removes the spec.
- This adapts the specs for 72ffabf6, so that they check for emails being sent
- This reimplements c2797921, allowing test emails to be sent even when emails are disabled
2019-03-22 17:28:43 +08:00
Vinoth Kannan 1e3cb7575d DEV: Update webhook event attributes even when an error raised 2019-03-21 20:45:21 +05:30
Vinoth Kannan 4c6bfb9b39 DEV: Don't destroy webhook in case of error 2019-03-21 18:34:54 +05:30
Tarek Khalil a31a35b334 FEATURE: Ignored user notification behaviour should be as a muted user (#7227) 2019-03-21 12:15:34 +01:00
Régis Hanol 31e06dbcd2 FIX: SES webhook wasn't parsing the message 2019-03-19 11:40:19 +01:00
Bianca Nenciu 5114ef958a FIX: Do not trigger post alerts for empty posts. (#7138) 2019-03-15 17:58:43 +01:00
Penar Musaraj 9334d2f4f7
FEATURE: add more granular user option levels for email notifications (#7143)
Migrates email user options to a new data structure, where `email_always`, `email_direct` and `email_private_messages` are replace by

* `email_messages_level`, with options: `always`, `only_when_away` and `never` (defaults to `always`)
* `email_level`, with options: `always`, `only_when_away` and `never` (defaults to `only_when_away`)
2019-03-15 10:55:11 -04:00
Bianca Nenciu c6ed86220e FIX: Notify on tag change. (#7119) 2019-03-12 18:09:34 +01:00
Guo Xiang Tan 34b29f62db DEV: Remove the use of stubs and mocks in `Jobs::UserEmail` tests.
We can only be sure that an email is sent when we get a mailer in
`ActionMailer::Deliveries`. A couple of tests were actually incorrect
because it didn't flow through our email sender where there are more
conditions in determining whether an email is sent or not.
2019-03-12 09:39:16 +08:00
Régis Hanol 326d892f5e Aadd 'secondary_emails' field in users export
FIX: escape_comma wasn't working in CSV exports
FIX: group_names field wasn't properly serialized
2019-02-27 10:12:20 +01:00
Régis Hanol e3a23116d2
FIX: don't update gravatar if the user has no email 2019-02-20 22:34:43 +01:00
Régis Hanol fc14847c14 PERF: only require aws-sdk-sns gem when it's being used 2019-02-14 11:08:21 +01:00
Régis Hanol 4d674acc25 FEATURE: AWS SNS bounce notifications webhooks 2019-02-13 21:26:40 +01:00
Vinoth Kannan b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30