Commit Graph

281 Commits

Author SHA1 Message Date
Roman Rizzi e0ba35350e
FEATURE: Custom unsubscribe options (#17090)
With this change, plugins can create custom unsubscribe keys, extend the unsubscribe view with custom preferences, and decide how they are updated.
2022-06-21 15:49:47 -03:00
Martin Brennan 4d3c1ceb44
FEATURE: Log the SMTP response in EmailLog (#17056)
When sending emails with delivery_method_options -> return_response
set to true, the SMTP sending code inside Mail will return the SMTP
response when calling deliver! for mail within the app. This commit
ensures that Email::Sender captures this response if it is returned
and stores it against the EmailLog created for the sent email.

A follow up PR will make this visible within the admin email UI.
2022-06-15 10:28:30 +10:00
Andrei Prigorshnev 25e4095c9c
FIX: respect user timezone in emails about silencing and suspending (#16918) 2022-05-27 13:58:54 +04:00
Martin Brennan a6be4972a8
FIX: Use our header value instead of custom header on duplicates (#16711)
When we build and send emails using MessageBuilder and Email::Sender
we add custom headers defined in SiteSetting.email_custom_headers.
However this was causing errors in cases where the custom headers
defined a header that we already specify in outbound emails (e.g.
the Precedence: list header for topic/post emails).

This commit makes it so we always use the header value defined in Discourse
core if there is a duplicate, discarding the custom header value
from the site setting.

cf. https://meta.discourse.org/t/email-notifications-fail-if-duplicate-headers-exist/222960/14
2022-05-11 13:47:12 +10:00
Loïc Guitaut 008b700a3f DEV: Upgrade to Rails 7
This patch upgrades Rails to version 7.0.2.4.
2022-04-28 11:51:03 +02:00
Gerhard Schlager 1a56ce3674 FEATURE: Site setting to cap the recipient list in notification emails
* Adds a hidden site setting: `max_participant_names`
* Replaces duplicate code in `GroupSmtpMailer` and `UserNotifications`
* Groups are sorted by the number of users (decreasing)
* Replaces the query to count users of each group with `Group#user_count`)
* Users are sorted by their last reply in the topic (most recent first)
* Adds lots of tests
2022-04-21 10:43:13 +02:00
Gerhard Schlager 87c872823b DEV: Remove unused code and rename interpolation key 2022-04-21 10:43:13 +02:00
Gerhard Schlager b7230d14a3 REFACTOR: Add `full_url` to `Group`
Lets stop writing the same code over and over again.
2022-04-14 11:53:57 +02:00
Gerhard Schlager b3cda195b8 REFACTOR: Add `full_url` and `display_name` to `User`
Lets stop writing the same code over and over again.
2022-04-14 11:53:57 +02:00
Martin Brennan 0a738bd5bc
FEATURE: Allow sending group SMTP emails with from alias (#15687)
This commit allows group SMTP emails to be sent with a
different from email address that has been set up as an
alias in the email provider. Emails from the alias will
be grouped correctly using Message-IDs in the mail client,
and replies to the alias go into the correct group inbox.
2022-02-07 13:52:01 +10:00
Dan Ungureanu fa8cd629f1
DEV: Hash tokens stored from email_tokens (#14493)
This commit adds token_hash and scopes columns to email_tokens table.
token_hash is a replacement for the token column to avoid storing email
tokens in plaintext as it can pose a security risk. The new scope column
ensures that email tokens cannot be used to perform a different action
than the one intended.

To sum up, this commit:

* Adds token_hash and scope to email_tokens

* Reuses code that schedules critical_user_email

* Refactors EmailToken.confirm and EmailToken.atomic_confirm methods

* Periodically cleans old, unconfirmed or expired email tokens
2021-11-25 09:34:39 +02:00
Martin Brennan d3678f6930
FIX: Do not show recipient user in email participants list (#14642)
This commit removes the recipient's username from the
respond to / participants list that is shown at the bottom
of user notification emails. For example if the recipient's
username was jsmith, and there were participants ljones and
bmiller, we currently show this:

> "reply to this email to respond to jsmith, ljones, bmiller"

or

> "Participants: jsmith, ljones, bmiller"

However this is a bit redundant, as you are not replying to
yourself here if you are the recipient user. So we omit the
recipient user's username from this list, which is only used
in the text of the email and not elsewhere.
2021-10-19 15:26:22 +10:00
Andrei Prigorshnev 1a8c949900
UX: suspend forever time period messages (#13776)
When the Forever option is selected for suspending a user, the user is suspended for 1000 years. Without customizing the site’s text, this time period is displayed to the user in the suspension email that is sent to the user, and if the user attempts to log back into the site. Telling someone that they have been suspended for 1000 years seems likely to come across as a bad attempt at humour.

This PR special case messages when a user suspended or silenced forever.
2021-07-20 14:42:08 +04:00
Martin Brennan 0dadd61d27
FIX: Change email from to not have via site_name for group SMTP (#13788)
We now use the group's full name in group SMTP emails, so we are dropping the via #{site_name}. If group owners still want this they can just change the full name of the group.
2021-07-20 11:56:04 +10:00
Martin Brennan 4d0178deab
FIX: Do not show In Reply To for group SMTP emails (#13541)
We do not want to show the In Reply To section of the
group SMTP email template, it is similar to Context Posts
which we removed and is unnecessary.

This PR also removes the link to staged user profiles in
the email; their email addresses will just be converted
to regular mailto: links.
2021-06-28 13:19:17 +10:00
Martin Brennan d3e27cabf6
FIX: Improve participant display in group SMTP emails (#13539)
This PR makes several changes to the group SMTP email contents to make it look more like a support inbox message.

* Remove the context posts, they only add clutter to the email and replies
* Display email addresses of staged users instead of odd generated usernames
* Add a "please reply above this line" message to sent emails
2021-06-28 10:42:06 +10:00
Martin Brennan 87684f7c5e
FEATURE: Use group SMTP job and mailer instead of UserNotifications change (#13489)
This PR backtracks a fair bit on this one https://github.com/discourse/discourse/pull/13220/files.

Instead of sending the group SMTP email for each user via `UserNotifications`, we are changing to send only one email with the existing `Jobs::GroupSmtpEmail` job and `GroupSmtpMailer`. We are changing this job and mailer along with `PostAlerter` to make the first topic allowed user the `to_address` for the email and any other `topic_allowed_users` to be the CC address on the email. This is to cut down on emails sent via SMTP, which is subject to daily limits from providers such as Gmail. We log these details in the `EmailLog` table now.

In addition to this, we have changed `PostAlerter` to no longer rely on incoming email email addresses for sending the `GroupSmtpEmail` job. This was unreliable as a user's email could have changed in the meantime. Also it was a little overcomplicated to use the incoming email records -- it is far simpler to reason about to just use topic allowed users.

This also adds a fix to include cc_addresses in the EmailLog.addressed_to_user scope.
2021-06-28 08:55:13 +10:00
Jarek Radosz 61472d6aaa
DEV: Rename `hilight` to `highlight` (#13526) 2021-06-25 18:05:50 +02:00
Martin Brennan 1fdef0dc5b
FIX: Remove duplicate add_unsubscribe_link hash key (#13467)
This double key was introduced in
f0c10edd28
2021-06-22 11:25:39 +10:00
Martin Brennan f0c10edd28
FIX: Remove List-Unsubscribe header if using group SMTP (#13448)
The other mailing list headers were removed if using
group SMTP in ff6114d83f
2021-06-21 09:33:32 +10:00
Martin Brennan 7fca7fb7ff
DEV: Add SMTP group ID to EmailLog (#13381)
Adds a new `smtp_group_id` column to `EmailLog` which is filled in if the mail `from_address` matches a group's `email_username`. This is for easier debugging, so we know which emails have been sent via group SMTP.
2021-06-15 11:29:46 +10:00
Martin Brennan eb2c399445
FEATURE: Use group SMTP settings for sending user notification emails (initial) (#13220)
This PR changes the `UserNotification` class to send outbound `user_private_message` using the group's SMTP settings, but only if:

* The first allowed_group on the topic has SMTP configured and enabled
* SiteSetting.enable_smtp is true
* The group does not have IMAP enabled, if this is enabled the `GroupSMTPMailer` handles things

The email is sent using the group's `email_username` as both the `from` and `reply-to` address, so when the user replies from their email it will go through the group's SMTP inbox, which needs to have email forwarding set up to send the message on to a location (such as a hosted site email address like meta@discoursemail.com) where it can be POSTed into discourse's handle_mail route.

Also includes a fix to `EmailReceiver#group_incoming_emails_regex` to include the `group.email_username` so the group does not get a staged user created and invited to the topic (which was a problem for IMAP), as well as updating `Group.find_by_email` to find using the `email_username` as well for inbound emails with that as the TO address.

#### Note

This is safe to merge without impacting anyone seriously. If people had SMTP enabled for a group they would have IMAP enabled too currently, and that is a very small amount of users because IMAP is an alpha product, and also because the UserNotification change has a guard to make sure it is not used if IMAP is enabled for the group. The existing IMAP tests work, and I tested this functionality by manually POSTing replies to the SMTP address into my local discourse.

There will probably be more work needed on this, but it needs to be tested further in a real hosted environment to continue.
2021-06-03 14:47:32 +10:00
Martin Brennan 964da21817
FEATURE: Improve group email settings UI (#13083)
This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory.

### UI

Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP.

You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator.

If there is an issue with the configuration or credentials a meaningful error message should be shown.

IMAP settings must also be validated when IMAP is enabled, before saving.

When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved):

### Database & Backend

This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group.

For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous.

There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-28 09:28:18 +10:00
Dan Ungureanu a5273b37f7
FIX: Show inviter name in email's from field (#13141)
'From' field of the email contained the name of the user who posted the
shared post. Instead, it should contain the name of the inviter.
2021-05-26 12:55:07 +10:00
Josh Soref 13d40ead97
DEV: Correct spelling mistakes in comments 2021-05-21 13:37:17 +10:00
Dan Ungureanu 528cfea079
FEATURE: Auto-activate users invited by email (#12675)
When invited by email, users will receive an invite URL which contains
a token. If that token is present when the invite is redeemed, their
account will be automatically activated.
2021-04-14 12:15:56 +03:00
Roman Rizzi 958fbfb719
FEATURE: Send an email notification when a post is approved. (#12665)
We now send an email when a queued post is approved, and we create a notification.
2021-04-12 12:08:23 -03:00
Dan Ungureanu fb19ee9eee
FIX: Correctly use invite to topic email templates (#12411)
It was used both when inviting from a topic page and when creating
invites with "Send to topic on first login", while it should be used
only in the former case.
2021-03-16 17:08:54 +02:00
Martin Brennan 6b4d066834
FIX: Skip sending PM email for user silence (#12240)
We were sending 2 emails for user silencing if a message was provided in the UI. Also always send email for user silence and user suspend with reason regardless of whether message provided.
2021-03-02 09:18:09 +10:00
Krzysztof Kotlarek 06b7c44593
FEATURE: reason to reject user signup (#11700)
Feature for `Must Approve Users` setup. When a user is rejected, a staff member can optionally set a reason for audit purposes. In addition, feedback email can be sent to the user.

Meta: https://meta.discourse.org/t/account-rejection-email/103112/8
2021-01-15 09:43:26 +11:00
Martin Brennan 632942e697
FIX: Ensure group SMTP and message builder always uses from address for Reply-To when IMAP is enabled (#11037)
There is a site setting reply_by_email_enabled which when combined with reply_by_email_address creates a Reply-To header in emails in the format "test+%{reply_key}@test.com" along with a PostReplyKey record, so when replying Discourse knows where to route the reply.

However this conflicts with the IMAP implementation. Since we are sending the email for a group via SMTP and from their actual email account, we want all replys to go to that email account as well so the IMAP sync job can pick them up and put them in the correct place. So if the group has IMAP enabled and configured, then the reply-to header will be correct.

This PR also makes a further fix to 64b0b50 by using the correct recipient user for the PostReplyKey record. If the post user is used we encounter this error:

if destination.user_id != user.id && !forwarded_reply_key?(destination, user)
  raise ReplyUserNotMatchingError, "post_reply_key.user_id => #{destination.user_id.inspect}, user.id => #{user.id.inspect}"
end
This is because the user above is found from the from_address, but the destination which is the PostReplyKey is made by the post.user, which will be different people.
2020-10-28 07:01:58 +10:00
Martin Brennan 6e2be3e60b
FIX: When admin changes an email for the user the user must confirm the change (#10830)
See https://meta.discourse.org/t/changing-a-users-email/164512 for additional context.

Previously when an admin user changed a user's email we assumed that they would need a password reset too because they likely did not have access to their account. This proved to be incorrect, as there are other reasons a user needs admin to change their email. This PR:

* Changes the admin change email for user flow so the user is sent an email to confirm the change
* We now record who the email change request was requested by
* If the requested by user is admin and not the user we note this in the email sent to the user
* We also make the confirm change email route open to anonymous users, so it can be clicked by the user even if they do not have access to their account. If there is a logged in user we make sure the confirmation matches the current user.
2020-10-07 13:02:24 +10:00
Martin Brennan dede942007
FEATURE: Allow email image embed with secure media (#10563)
This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced:

* `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them.
* `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`.

`Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients.

All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`.

Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions.

![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-10 09:50:16 +10:00
Guo Xiang Tan 255b0e9f14
PERF: Replace video and audio links in search blurb while indexing.
In the near future, we will be swtiching to PG headlines to generate the
search blurb. As such, we need to replace audio and video links in the
raw data used for headline generation. This also means that we avoid
replacing links each time we need to generate the blurb.
2020-08-06 12:25:03 +08:00
Martin Brennan 0e78cd6e3a
FIX: Add strip_secure_urls method to GroupSmtpMailer
* this mailer needs some more cleanup and specs;
  this commit just adds the missing method so the
  mailer does not error completely in secure media
  environments
2020-07-24 13:55:07 +10:00
Dan Ungureanu c72bc27888
FEATURE: Implement support for IMAP and SMTP email protocols. (#8301)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-07-10 12:05:55 +03:00
Bianca Nenciu 75151f0457
FIX: Use correct URL for unsubscribe (#10077) 2020-06-24 09:31:20 +02:00
Penar Musaraj 76b05ef8ad
DEV: Use short_date helper for email post template (#10063) 2020-06-17 11:29:37 -04:00
Dan Ungureanu 5bfe1ee4f1
FEATURE: Improve UX support for multiple email addresses (#9691) 2020-06-10 19:11:49 +03:00
Penar Musaraj fa6aa7f627
FIX: Digest frequency issue on user creation
If `default email digest frequency` was set to "Never", users would get
a `digest_after_minutes` set to `nil` which triggered this error
in the logs if/when the site eventually changed that setting and
enabled digests:

```
NoMethodError (undefined method `>=' for nil:NilClass)
/var/www/discourse/app/mailers/user_notifications.rb:227:in `digest'
```
2020-06-01 17:39:16 -04:00
Martin Brennan 4239ca1f8d
FIX: Change user digest email total unread notification calculation (#9648)
We now add together unread notifications (which are low-priority only) and unread high priority notifications (PMs and bookmark reminder notifications), and removed the separate "X unread high priority notifications" count in the user digest email.
2020-05-06 14:23:13 +10:00
Martin Brennan b79ea986ac
FEATURE: High priority bookmark reminder notifications (#9290)
Introduce the concept of "high priority notifications" which include PM and bookmark reminder notifications. Now bookmark reminder notifications act in the same way as PM notifications (float to top of recent list, show in the green bubble) and most instances of unread_private_messages in the UI have been replaced with unread_high_priority_notifications.

The user email digest is changed to just have a section about unread high priority notifications, the unread PM section has been removed.

A high_priority boolean column has been added to the Notification table and relevant indices added to account for it.

unread_private_messages has been kept on the User model purely for backwards compat, but now just returns unread_high_priority_notifications count so this may cause some inconsistencies in the UI.
2020-04-01 09:09:20 +10:00
Martin Brennan edbc356593
FIX: Replace deprecated URI.encode, URI.escape, URI.unescape and URI.unencode (#8528)
The following methods have long been deprecated in ruby due to flaws in their implementation per http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/29293?29179-31097:

URI.escape
URI.unescape
URI.encode
URI.unencode
escape/encode are just aliases for one another. This PR uses the Addressable gem to replace these methods with its own encode, unencode, and encode_component methods where appropriate.

I have put all references to Addressable::URI here into the UrlHelper to keep them corralled in one place to make changes to this implementation easier.

Addressable is now also an explicit gem dependency.
2019-12-12 12:49:21 +10:00
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Penar Musaraj 102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Arpit Jalan b7327d2c34 UX: show user email address on "grant admin access" email and UI 2019-11-04 14:47:00 +05:30
Neil Lalonde 4c2d6e19ba PERF: cache new users counts in summary emails
The query to count how many new users there are since a given date
is expensive. It's the least personalized stat and the one we fallback
to last when no better number can be found for the target user.
Give up accuracy so we can aggressively cache the user counts
that appear in this email.
2019-10-25 16:33:36 -04:00
Daniel Waterworth 55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Arpit Jalan c596d7df77 FIX: respect private_email setting for user invited notification email 2019-10-16 12:50:30 +05:30
Arpit Jalan 600233482f FIX: include topic link when inviting existing users to a topic/PM
FEATURE: allow staff to use topic_url for customizing email template
2019-10-16 12:36:16 +05:30