Commit Graph

123 Commits

Author SHA1 Message Date
Blake Erickson 12f041de5d
FIX: Tag watching for everyone tag groups (#15622)
* FIX: Tag watching for everyone tag groups

Tags in tag groups that have permissions set to everyone were not able
to be saved correctly. A user on their preferences page would mark the
tags that they wanted to save, but the watched_tags in the response
would be empty. This did not apply to admins, just regular users. Even
though the watched tags were being saved in the db, the user serializer
response was filtering them out. When a user refreshed their preferences
pages it would show zero watched tags.

This appears to be a regression introduced by:

0f598ca51e

The issue that needed to be fixed is that we don't track the "everyone"
group (which has an id of 0) in the group_users table. This is because
everyone has access to it, so why fill a row for every single user, that
would be a lot. The fix was to update the query to include tag groups
that had permissions set to the "everyone" group (group_id 0).

I also added another check to the existing spec for updating
watched tags for tags that aren't in a tag group so that it checks the
response body. I then added a new spec which updates watched tags for
tags in a tag group which has permissions set to everyone.

* Resolve failing tests

Improve SQL query syntax for including the "everyone" group with the id
of 0.

This commit also fixes a few failing tests that were introduced. It
turns out that the Fabrication of the Tag Group Permissions was faulty.
What happens when creating the tag groups without any permissions is
that it sets the permission to "everyone". If we then follow up with
fabricating a tag group permission on the tag group instead of having a
single permission it will have 2 (everyone + the group specified)! We
don't want this. To fix it I removed the fabrication of tag group
permissions and just set the permissions directly when creating the tag
group.

* Use response.parsed_body instead of JSON.parse
2022-01-18 15:02:29 -07:00
Roman Rizzi e005e3f153
DEV: Create post actions without creating a notification and store custom data. (#15397)
I plan to use this in an upcoming discourse-reactions PR, where I want to like a post without notifying the user, so I can instead create a reaction notification.

Additionally, we decouple the a11y attributes from the icon itself, which will let us extend the widget's icon without losing them.
2021-12-27 11:25:37 -03:00
Daniel Waterworth 102fa71ef3
PERF: Speed up the tests by pre-fabricating more things (#15318) 2021-12-15 11:41:14 -06:00
Roman Rizzi 0719531bd3
FIX: Notify group members when someone quotes or replies to their post. (#15295)
When a member set a group PM notification level to Normal, we didn't notify group members if someone quoted or replied to one of their posts.
2021-12-15 09:07:39 -03:00
Roman Rizzi b7b61d4b56
FEATURE: A notification consolidation plan for keeping the latest one. (#15249)
We previously used ConsolidateNotifications with a threshold of 1 to re-use an existing notification and bump it to the top instead of creating a new one. It produces some jumpiness in the user notification list, and it relies on updating the `created_at` attribute, which is a bit hacky.

As a better alternative, we're introducing a new plan that deletes all the previous versions of the notification, then creates a new one.
2021-12-10 10:32:15 -03:00
Roman Rizzi 43903f8dfe
FIX: Updating a consolidated notification should bump it to the top. (#15199)
In the future, it would be better to have a consolidated_at timestamp instead of updating created_at.
2021-12-06 12:31:44 -03:00
Natalie Tay 0f598ca51e
SECURITY: Only show tags to users with permission (#15148) 2021-12-01 10:26:56 +08:00
Roman Rizzi 1fc06520bd
REFACTOR: Improve support for consolidating notifications. (#14904)
* REFACTOR: Improve support for consolidating notifications.

Before this commit, we didn't have a single way of consolidating notifications. For notifications like group summaries, we manually removed old ones before creating a new one. On the other hand, we used an after_create callback for likes and group membership requests, which caused unnecessary work, as we need to delete the record we created to replace it with a consolidated one.

We now have all the consolidation rules centralized in a single place: the consolidation planner class. Other parts of the app looking to create a consolidable notification can do so by calling Notification#consolidate_or_save!, instead of the default Notification#create! method.

Finally, we added two more rules: one for re-using existing group summaries and another for deleting duplicated dashboard problems PMs notifications when the user is tracking the moderator's inbox. Setting the threshold to one forces the planner to apply this rule every time.

I plan to add plugin support for adding custom rules in another PR to keep this one relatively small.

* DEV: Introduces a plugin API for consolidating notifications.

This commit removes the `Notification#filter_by_consolidation_data` scope since plugins could have to define their criteria. The Plan class now receives two blocks, one to query for an already consolidated notification, which we'll try to update, and another to query for existing ones to consolidate.

It also receives a consolidation window, which accepts an ActiveSupport::Duration object, and filter notifications created since that value.
2021-11-30 13:36:14 -03:00
Bianca Nenciu 73760c77d9
FEATURE: Mention @here to notify users in topic (#14900)
Use @here to mention all users that were allowed to topic directly or
through group, who liked topics or read the topic. Only first 10 users
will be notified.
2021-11-23 22:25:54 +02:00
Martin Brennan 9f8ee8f137
FIX: Don't publish notifications to MessageBus for inactive users (#15035)
We are pushing /notification-alert/#{user_id} and /notification/#{user_id}
messages to MessageBus from both PostAlerter and User#publish_notification_state.
This can cause memory issues on large sites with many users. This commit
stems the bleeding by only sending these alert messages if the user
in question has been seen in the last 30 days, which eliminates a large
chunk of users on some sites.
2021-11-22 14:38:49 +11:00
Martin Brennan 515acb8fc4
FIX: Error when SMTP enabled in PostAlerter with no incoming email (#14981)
When 31035010af
was done it failed to take into account the case where the smtp_enabled
site setting was true, but the topic had no allowed groups / no
incoming email record, which caused errors for topics even with
nothing to do with group SMTP.
2021-11-17 09:24:17 +10:00
Martin Brennan 31035010af
FIX: Use correct group out of multiple for SMTP sender (#14957)
When there are multiple groups on a topic, we were selecting
the first from the topic allowed groups to act as the sender
email address when sending group SMTP replies via PostAlerter.
However, this was not ordered, and since there is no created_at
column on TopicAllowedGroup we cannot order this nicely, which
caused just a random group to be used (based on whatever postgres
decided it felt like that morning).

This commit changes the group used for SMTP sending to be the
group using the email_username of the to address of the first
incoming email for the topic, if there are more than one allowed
groups on the topic. Otherwise it just uses the only SMTP enabled
group.
2021-11-16 10:21:49 +10:00
Mark VanLandingham 67265a5045
DEV: Plugin instance method for push_notification_filters (#14787) 2021-11-03 12:21:33 -05:00
Bianca Nenciu 8b99a7f73d
FIX: Move check if user is suspended later (#14566)
Calling create_notification_alert could still send a notification to a
suspended user. This just moves the check if user is suspended right
before sending the notification.
2021-10-11 20:55:18 +03:00
Alan Guo Xiang Tan 37b8ce79c9
FEATURE: Add last visit indication to topic view page. (#13471)
This PR also removes grey old unread bubble from the topic badges by
dropping `TopicUser#highest_seen_post_number`.
2021-07-05 14:17:31 +08:00
Martin Brennan 100c3d6d62
FIX: Respect personal_email_time_window_seconds in group SMTP (#13630)
For other private messages we have the site setting
personal_email_time_window_seconds (default 20s) which allows
people to edit their post etc. before the email is sent.

This PR makes the Jobs::GroupSmtpEmail enqueuer in the
PostAlerter use the same delay.

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2021-07-05 10:09:16 +10:00
Martin Brennan 87684f7c5e
FEATURE: Use group SMTP job and mailer instead of UserNotifications change (#13489)
This PR backtracks a fair bit on this one https://github.com/discourse/discourse/pull/13220/files.

Instead of sending the group SMTP email for each user via `UserNotifications`, we are changing to send only one email with the existing `Jobs::GroupSmtpEmail` job and `GroupSmtpMailer`. We are changing this job and mailer along with `PostAlerter` to make the first topic allowed user the `to_address` for the email and any other `topic_allowed_users` to be the CC address on the email. This is to cut down on emails sent via SMTP, which is subject to daily limits from providers such as Gmail. We log these details in the `EmailLog` table now.

In addition to this, we have changed `PostAlerter` to no longer rely on incoming email email addresses for sending the `GroupSmtpEmail` job. This was unreliable as a user's email could have changed in the meantime. Also it was a little overcomplicated to use the incoming email records -- it is far simpler to reason about to just use topic allowed users.

This also adds a fix to include cc_addresses in the EmailLog.addressed_to_user scope.
2021-06-28 08:55:13 +10:00
Martin Brennan b463a80cbf
FIX: Do not enqueue :group_smtp_email job if IMAP disabled for the group (#13307)
When a group only has SMTP enabled and not IMAP, we do not
want to enqueue the :group_smtp_email job because using the group's
SMTP credentials for sending user_private_message emails is
handled by the UserNotifications class.

We do not want the :group_smtp_email job to be enqueued because
that uses a reply key instead of the group.email_username
for the reply-to address which is not what we want for SMTP
only, and also creates an IncomingEmail record to prevent IMAP
double syncing which we do not need either.

There is an open question about what happens when IMAP is
enabled after SMTP has been enabled for a while, and also questions
around whether we could do away with :group_smtp_email altogether
and handle everything via EmailLog and UserNotifications, adding
additional columns to the former and modifying the Imap::Sync
class to take this into account...a lot more further testing
for IMAP needs to be done to answer those questions.

For now, this fix should be sufficient to get the correct
reply-to address for user_private_response messages sent in
response to emails sent directly to the group's
email_username SMTP address.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2021-06-07 14:17:35 +10:00
Martin Brennan 964da21817
FEATURE: Improve group email settings UI (#13083)
This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory.

### UI

Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP.

You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator.

If there is an issue with the configuration or credentials a meaningful error message should be shown.

IMAP settings must also be validated when IMAP is enabled, before saving.

When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved):

### Database & Backend

This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group.

For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous.

There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-28 09:28:18 +10:00
Josh Soref 59097b207f
DEV: Correct typos and spelling mistakes (#12812)
Over the years we accrued many spelling mistakes in the code base. 

This PR attempts to fix spelling mistakes and typos in all areas of the code that are extremely safe to change 

- comments
- test descriptions
- other low risk areas
2021-05-21 11:43:47 +10:00
Dan Ungureanu d1d9f83304
FIX: Link notification to first unread post (#12868)
* FIX: Link notification to first unread post

If a topic with a few posts was posted in a watched category or with a
watched tag, the created notification would always point to the last
post, instead of pointing to the first one.

The root cause is that the query that fetched the first unread post
uses 'TopicUser' records and those are not created by default for
user watching a category or tag. In this case, it should use the
'CategoryUser' or 'TagUser' records.

* DEV: Use named bind variables
2021-05-04 13:03:00 +10:00
Krzysztof Kotlarek 146775072a
FIX: mention notification takes precedence over group (#12407)
When a specific user is mentioned and group to which this user belongs, a direct mention notification should take precedence.
2021-03-17 08:20:41 +11:00
Blake Erickson 2123561125
FIX: Mobile app notification urls w/ subfolder (#12282)
The urls that we generate for mobile post notifications don't take into
account the subfolder url if a site happens to have one configured. When
this happens when you tap on a new mobile notification it takes you to
a url that doesn't work because it is missing the subfolder portion.

I honestly think this should be handled in the Post model like we do
with the Topic model. `Post.url` should know how to handle subfolder
installs, but that seemed like a very risky change because there are
lots of other places in the codebase where we tack on the base_path and
I didn't want to risk duplicating it.

I also found a small typo in the topics controller spec.
2021-03-04 07:07:37 -07:00
Martin Brennan 44f15d4281
FIX: Do not email group user with user_private_message notification (#11754)
There was an issue that occurred with this order of operations:

* An IMAP topic was created by emailing a group
* A second user was invited to the topic (not the OP and not the group)
* A user with access to the group replies to the topic
* The second user receives a user_private_message notification email because of their involvement in the topic
* The second user replies to the email via email

This new reply would then go and notify the other group PM users, except for those who emailed the group topic directly, which is handled via the group SMTP mailer. However because the new post already has an incoming email because it is parsed via the Email::Receiver via POP3 the group SMTP section of the post alerter is skipped, and the group's email address is not ignored for the user_private_message notification.

This PR fixes it so the group is not ever sent an email via the PM notification. This is important because any new emails in the group's IMAP inbox will be picked up by the Imap::Sync code and created as a new topic which is not at all desirable.

Also in this PR I split up the specs a bit more for group SMTP in the post alerter to make them easier to read and they each only test one thing.
2021-01-20 10:53:08 +10:00
Martin Brennan 6c155dba77
FIX: Stop incorrect emailing of group email from PostAlerter (#11723)
Fixes bug introduced by bd25627198

What happens is we send notifications to everyone involved in the group inbox topic about new posts, however we pass the param `skip_send_email_to: email_addresses`. In the above commit I removed the group email address from this `email_addresses` array. This breaks the IMAP inbox because we email the group with the reply, and the IMAP sync tool finds this email and opens a new unrelated topic with it.
2021-01-15 14:31:59 +10:00
Martin Brennan bd25627198
FIX: IMAP post alerter race condition and code improvements (#11711)
This PR fixes a race condition with the IMAP notification code. In the `Email::Receiver` we call the `NewPostManager` to create the post and enqueue jobs and sends alerts via `PostAlerter`. However, if the post alerter reaches the `notify_pm_users` and the `group_notifying_via_smtp` method _before_ the incoming email is updated with the post and topic, we unnecessarily send a notification to the person who just posted. The result of this is that the IMAP syncer re-imports the email sent to the user about their own post, which looks like this in the group inbox:

To fix this, we skip the jobs enqueued by `NewPostManager` and only enqueue them with `PostJobsEnqueuer` manually _after_ the incoming email record has been updated with the post and topic.

Other improvements:

* Moved code to calculate email addresses from `IncomingEmail` records into the topic, with a group passed in, for easier testing and debugging. It is not the responsibility of the post alerter to figure this stuff out.
* Add shortcut methods on `IncomingEmail` to split or provide an empty array for to and cc addresses to avoid repetition.
2021-01-15 10:54:46 +10:00
Mark VanLandingham 649ed24bb4
FEATURE: Do not disturb (#11484) 2020-12-18 09:03:51 -06:00
David Taylor 5140ec9acf
DEV: Cleanup ignored user logic (#11107)
- IgnoredUser records should all now have an expiring_at value. This commit enforces that in the DB, and fixes any corrupt rows
- Changes to the ignored user list are now handled by the `/u/{username}/notification_level` endpoint. This allows setting expiration dates on the ignore. This commit removes the old logic for saving a list of usernames in the user preferences.
- Many specs were calling `IgnoredUser.create`. This commit changes them to use `Fabricate(:ignored_user)` for consistency
2020-11-03 12:38:54 +00:00
David Taylor 1ba9b34b03
DEV: Move UserApiKey scopes to dedicated table (#10704)
This has no functional impact yet, but it is the first step in adding more granular scopes to UserApiKeys
2020-09-29 10:57:48 +01:00
Penar Musaraj f7ca93b2fa
FIX: Do not downcase group name in notification payload
Followup to 3d39b4bb
2020-09-21 13:33:29 -04:00
Guo Xiang Tan 31652c1ad3
DEV: Improve assertions for spec.
This was showing up as a flaky spec.
2020-08-31 11:08:18 +08:00
Krzysztof Kotlarek 7194b31443
FEATURE: don't notify about changed tags for a private message (#10408)
* FEATURE: don't notify about changed tags for a private message

Only staff members observing specific tag should receive a notification

* FIX: remove other category which is not used

* FIX: improved specs to ensure that revise was succesful
2020-08-13 17:22:34 +10:00
Dan Ungureanu c72bc27888
FEATURE: Implement support for IMAP and SMTP email protocols. (#8301)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-07-10 12:05:55 +03:00
Bianca Nenciu 052c91770f
FIX: Reply notifications should not appear as edited (#9965) 2020-06-08 15:23:33 +02:00
Krzysztof Kotlarek 05799538e6
FIX: correct edit notification username for PMs (#9649)
When the tag is added to PM by admin, notification has an incorrect username (post author username instead of admin username)
2020-05-07 07:52:21 +10:00
Sam Saffron 3877ef2cfa
FEATURE: Unconditionally notify on edits at least once a day
This ensures that at a minimum you are notified once a day of
repeat edits by the same user.

Long term we may consider winding this down to say 1 hour or
making it configurable.
2020-05-04 17:55:00 +10:00
Sam Saffron 3d0ccf8642
FIX: missing edit notifications in some rare cases
Due to a refactor in e90f9e5cc4 we stopped notifying on edits if
a user liked a post and then edited.

The like could have happened a long time ago so this gets extra
confusing.

This change makes the suppression more deliberate. We only want
to suppress quite/link/mention if the user already got a reply
notification.

We can expand this suppression if it is not enough.
2020-05-04 17:40:56 +10:00
Dan Ungureanu 4e5f9d4cd1
DEV: Drop 'key' column from user_api_keys (#9388) 2020-04-22 12:13:19 +03:00
Krzysztof Kotlarek e90f9e5cc4
FIX: when unread reply notification exists don't create new (#8921)
* FIX: when unread reply notification exists don't create new

From time to time, the user is creating a reply post and then they want to add additional details. They edit an existing post and for example, add a quote from a previous one.

In that situation, if the user to whom reply was directed to already have the unread notification, we should not create the new one.

That behaviour was mentioned here: https://meta.discourse.org/t/reply-then-edit-to-add-quote-notification-redundancy/138358

* FIX: dont create new notification if already exists
2020-02-14 16:41:42 +11:00
Krzysztof Kotlarek 5b03f35614
FIX: suppress notification flood when post is edited (#8838) 2020-02-03 11:27:18 +11:00
mintsaxon 4b54791bcc FIX: Notify staged users about private categories (#8765)
group membership and `CategoryUser` notification level should be
respected to determine whether to notify staged users about activity in
private categories, instead of only ever generating notifications for staged
users' own topics (which has been the behaviour since
0c4ac2a7bc)
2020-01-22 16:33:25 -03:00
Roman Rizzi 0c4ac2a7bc
Merge pull request from GHSA-569c-22ff-pj3x 2020-01-16 15:17:16 -03:00
Krzysztof Kotlarek 17366d3bcc
FEATURE: notify tag watchers when tag was added to post (#8299)
Issue was mentioned in this [meta topic](https://meta.discourse.org/t/send-a-notification-to-watching-users-when-adding-tag/125314)

It is working well when category is changed because NotifyCategoryChange job already got that code:
```
if post&.topic&.visible?
  post_alerter = PostAlerter.new
  post_alerter.notify_post_users(post, User.where(id: args[:notified_user_ids]))
  post_alerter.notify_first_post_watchers(post, post_alerter.category_watchers(post.topic))
end
```

For NotifyTagChange job notify post users were missing so it worked only when your notification was set to `watching first post`
2019-11-07 08:20:15 +11:00
Daniel Waterworth 55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Penar Musaraj 7b0517895e FEATURE: Add "Group owners" to posting options for groups
Context: https://meta.discourse.org/t/121589

This new setting option lets group owners message/mention large groups
without granting that privilege to all members.
2019-07-08 17:14:11 -04:00
Penar Musaraj 27387b0859 Do not collapse quote notifications
This prevents an issue where edits to a post with a quote would trigger push notifications indefinitely.
2019-06-27 18:20:43 -04:00
Bianca Nenciu 934adb14d2
FIX: On tag change notify only users watching the tag. (#7707) 2019-06-11 18:06:54 +03:00
Robin Ward 89b84651c3 Migrate score settings to use sensitivities
We hide scores so these settings no longer made sense.
2019-05-24 15:44:24 -04:00
Penar Musaraj fc5bb39096 FIX: Do not send duplicate alerts for the same post (#7476) 2019-05-15 18:47:36 +02:00
Guo Xiang Tan f530048973 Fix broken spec in 405ba00 take 2. 2019-05-08 16:07:14 +08:00