Commit Graph

448 Commits

Author SHA1 Message Date
Régis Hanol bf322281e3 Improve handling of inlined images in incoming emails 2017-05-03 23:02:02 +02:00
Guo Xiang Tan 90cd35c496 PERF: Reduce allocated memory when generating mailing list email.
Before: 10590714 bytes
After: 5303394 bytes
2017-05-03 16:59:44 +08:00
Claus Strasburger e9bb9a167b Fix reply-by-mail for 8-bit transfer encodings
The mail class seems to handle mails sent with Content-Transfer-Encoding: 8bit
somewhat weirdly: It decodes them (to utf-8), changes the raw source to base64,
and does not modify the Content-Type:charset= header.

This leads to Discourse trying the message encoding (in my example ISO-8859-1)
first, and if that does not contain any unparseable characters, it uses that.
Sadly, in ISO-8859-1, every byte sequence is valid.

Fix this by always trying to decode as UTF-8 first. The probability of someone
using another encoding that cleanly (but wrongly) decodes as UTF-8 should be
fairly low.
2017-04-30 23:30:40 +02:00
Régis Hanol aba76bace6 add support to keep img tags when converting to html 2017-04-28 22:14:46 +02:00
Régis Hanol 0ec15af970 restore the 'incoming_email_prefer_html' site setting 2017-04-27 14:31:11 +02:00
Robin Ward bf9c4a7828 FEATURE: secure_email site setting to prevent data going out in email 2017-04-26 13:05:56 -04:00
Régis Hanol b76674f640 FEATURE: convert incoming emails in HTML to markdown
- remove incoming_email_prefer_html site setting
- remove HtmlCleaner class
2017-04-26 16:49:06 +02:00
Guo Xiang Tan 423f2ab228 FIX: Processing incoming email should be done in a background job. 2017-04-24 13:57:28 +08:00
Ryan C. Gordon a51c191a66 Make Email::Receiver.check_address() into a class method. 2017-04-05 23:10:36 -04:00
Arpit Jalan 9f930125f5 FIX: replace site_name in email subject with site title 2017-03-22 23:38:46 +05:30
Arpit Jalan 1853a4852c FIX: use email prefix only in subject 2017-03-21 20:29:57 +05:30
Arpit Jalan 521c88fe58 FIX: enqueue activation email for invited user that has password set 2017-03-20 17:13:21 +05:30
Régis Hanol ee9d621d9c FIX: surround the FROM alias with " in order to support the @ character 2017-03-07 23:37:21 +01:00
Neil Lalonde e634b37f9a FIX: from field of emails should be including email_site_title or site title settings 2017-02-27 14:23:07 -05:00
Régis Hanol d88562e72b Revert "use RFC-complient signature separator dash-dash-space" 2017-02-14 14:58:08 +01:00
Robert Riemann bcd4513963 use RFC-complient signature separator dash-dash-space
currently, Discourse uses '---' in its notifications to
separate the signature with unsubscribe links etc. from
the body of the message.

The RFC standard defines '-- '.
https://www.ietf.org/rfc/rfc3676.txt (4.3)

The problem has been discussed in:

https://meta.discourse.org/t/previous-replies-separator-is-not-rfc-compliant/39410

And an incomplete fix has been added a year ago:
86819f08c3

The separator is important, because some mail clients strip off the
signature automatically in replies if the signature is recognised as such.
2017-02-10 11:46:02 +01:00
Régis Hanol 91d09ebc08 post ids are always an integer 2017-02-08 23:46:11 +01:00
Régis Hanol e9e88a010f FIX: find replied-to post even when incoming email doesn't have a reply key 2017-02-08 21:38:52 +01:00
Régis Hanol 82555ca761 FIX: mail threading wasn't working properly in Mac Mail 2017-02-01 23:02:41 +01:00
Régis Hanol c725225f69 FIX: log message when revoking email with too many bounces 2017-02-01 16:53:24 +01:00
Régis Hanol 8fc7420f83 FIX: prevent huge custom emojis in emails 2017-01-30 18:06:48 +01:00
Matt Palmer 04ae3539d0 FEATURE: Better error message when incoming e-mail is missing a Date: header 2017-01-13 11:05:00 +11:00
Neil Lalonde 42c39ab38e Don't display email addresses in staff action logs for revoked email 2017-01-10 17:51:22 -05:00
Régis Hanol 185dcb2ca1 handle emails with localized headers 😠 2017-01-09 22:59:30 +01:00
Régis Hanol 98c62bccb5 FIX: mark forwarded email as read by the forwarder
FIX: 'Re:' prefix is mostly used for replies and not forwarded emails
2017-01-06 15:33:55 +01:00
Jeff Atwood 15a0f3cb14 add vertical align to email blockquote avatar img 2017-01-02 13:49:00 -08:00
Arpit Jalan 495a511862 simplify quote markup in emails 2017-01-02 21:37:01 +05:30
Neil Lalonde a65281d5ea FIX: better support for featured link topics in summary emails 2016-12-19 17:05:49 -05:00
Neil Lalonde 3256620d5d FIX: some blank topics and posts in summary email because they're images 2016-12-19 16:21:31 -05:00
Robin Ward e03d5e2140 Reapply Ember 2.10 for good this time!
This reverts commit ddd299f4aa.
2016-12-19 11:19:10 -05:00
Robin Ward ddd299f4aa Revert "Revert "Revert Ember 2.10+ for a short while""
This reverts commit 76bbc481cb.
2016-12-16 10:29:30 -05:00
Robin Ward 76bbc481cb Revert "Revert Ember 2.10+ for a short while"
This reverts commit 21682fd60b.
2016-12-16 09:52:29 -05:00
Robin Ward 21682fd60b Revert Ember 2.10+ for a short while 2016-12-15 16:43:38 -05:00
Neil Lalonde f01f95d62d FEATURE: new settings to customize some colors in emails 2016-12-15 14:43:53 -05:00
Erick Guan 52763f5115
FEATURE: Allow posting a link with topics 2016-12-05 17:20:54 +01:00
Régis Hanol 951ef0d949 UX: fix onebox styling in emails 2016-12-05 12:00:04 +01:00
Régis Hanol eb453d0f82 the note in a FWed email should be a whisper only in PM and when the author is member of the group 2016-12-01 18:43:56 +01:00
Régis Hanol 62763f025c FIX: wasn't able to parse FROM email in the embedded email 2016-12-01 18:34:47 +01:00
Régis Hanol a03287f2ee FIX: 'In-Reply-To' header should default to topic_message_id 2016-11-28 14:18:02 +01:00
Régis Hanol 74b6fe8739 FIX: respect RFCs when setting 'In-Reply-To' and 'References' email headers 2016-11-25 23:25:39 +01:00
Régis Hanol 2125a630d9 FIX: encoding issues with forwarded emails 2016-11-17 12:44:39 +01:00
Régis Hanol a69f45d0da FEATURE: new 'always_show_trimmed_content' site setting 2016-11-16 22:06:07 +01:00
Régis Hanol 17f2be9f88 FEATURE: new 'enable_forwarded_email' site setting 2016-11-16 19:42:11 +01:00
Régis Hanol 17169b3037 FIX: issues with 'X-MSYS-API' custom header 2016-11-03 02:26:12 +01:00
Régis Hanol f8caae0be7 FIX: don't overwrite custom email headers when using mandrill/sparkpost 2016-10-30 11:38:55 +01:00
Régis Hanol 71f940d478 FIX: use metadata to hold the message_id with sparkpost 2016-10-27 19:35:50 +02:00
Régis Hanol 2a61cc8c88 FIX: email styling with blacklisted iframes 2016-10-21 12:37:03 +02:00
Régis Hanol 3949c24f80 FIX: sparkpost webhooks support 2016-10-17 11:26:49 +02:00
Vibol Hou 34af73c7cb FEATURE: sparkpost webhook 2016-09-26 22:13:34 -07:00
Régis Hanol 51322a46b3 FEATURE: retry processing incoming emails on rate limit 2016-08-08 22:28:27 +02:00
Régis Hanol 5c06076b5c FIX: strip blacklisted attachments before checking for empty email body 2016-08-08 22:20:33 +02:00
Régis Hanol e92f5e4fbf FEATURE: new email attachment blacklists site settings 2016-08-03 17:55:54 +02:00
Régis Hanol 323bd555c0 FIX: process all incoming email's destinations instead of only the first 2016-08-03 15:57:37 +02:00
Régis Hanol b08ab829b8 added 'X-Auto-Response-Suppress' email header (props to elijah) 2016-08-03 11:02:07 +02:00
Régis Hanol ca4c939e9a incoming mail is considered autogenerated when the subject match a list of 'out of office' keywords 2016-08-02 00:04:59 +02:00
Régis Hanol 7d0dc7cb7a FIX: don't create staged user(s) when receiving a bounce (mailer-deamon begone) 2016-08-01 23:37:59 +02:00
Régis Hanol b0f7e4ba00 FEATURE: deactive users after too many bounces 2016-07-25 18:57:06 +02:00
Régis Hanol 7648916314 new 'reset_bounce_score_after_days' site setting 2016-07-25 17:29:54 +02:00
Régis Hanol d2e22ab215 extract bounce scores into site settings 2016-07-25 17:27:28 +02:00
Régis Hanol 6ed03bf3d9 FIX: always update bounce_score even when email was not auto_generated 2016-07-15 18:00:40 +02:00
Régis Hanol 8d2c878b34 FIX: only send 1 rejection email per day to prevent loops 2016-07-15 13:18:55 +02:00
Régis Hanol 926c021125 set 'List-Unsubscribe' email header to new unsubscribe url instead of user preferences url 2016-07-13 22:32:46 +02:00
Jeff Atwood 15a46d419f tweak in-reply-to email CSS 2016-07-12 16:26:44 -07:00
James Kiesel 3588780ac3 Don't reject likes by email for closed topics (#4311) 2016-07-05 17:33:08 +02:00
James Kiesel 949be591da FEATURE: Allow <3 and ❤ to trigger like via email (#4310) 2016-07-05 15:59:23 +02:00
Felix Freiberger 8053bde1a2 Generate proper URIs for iframes in emails (#4042) 2016-07-04 11:29:12 +02:00
Régis Hanol 9bf34ce49b FIX: support 'nil' alternative reply by email address 2016-07-02 11:09:06 +02:00
Matt Palmer 22bbc3ea73 Log backtrace when we get mystery meat errors in e-mail processing 2016-07-02 15:03:13 +10:00
Régis Hanol 214f5bff5c don't send more than 1 reply per day to auto-generated emails 2016-06-28 16:42:05 +02:00
Régis Hanol 376881845c always strip s/mime signatures in incoming emails 2016-06-27 22:26:05 +02:00
Régis Hanol 589bae5c03 try to fix badly encoded emails 2016-06-26 13:27:34 +02:00
Régis Hanol 8c51d34100 FIX: receiving a bounce from a deleted user 2016-06-22 20:41:21 +02:00
Régis Hanol d0a51df4d0 use standard rails logger 2016-06-22 18:09:11 +02:00
Régis Hanol 2ecd0da59f REFACTOR: use same code path for handling emails via API and POP 2016-06-22 15:50:49 +02:00
Arpit Jalan a4a53d6500 FIX: apply style formatting on post notification emails 2016-06-17 15:09:48 +05:30
Jeff Atwood 4f88f84053 attempt to fix styling error in mail post quotes 2016-06-17 00:56:25 -07:00
Matt Palmer fde5ef4902 Merge pull request #4273 from mpalmer/log-useful-info-replyusernotmatchingerror
Log some useful info when raising ReplyUserNotMatchingError
2016-06-17 12:06:03 +10:00
Matt Palmer eaf111f3e1 Log some useful info when raising ReplyUserNotMatchingError 2016-06-17 12:01:08 +10:00
Régis Hanol 49f8a2baa7 FEATURE: support for mandrill webhooks 2016-06-13 12:32:14 +02:00
James Cook c0e25b5a9a Replace certain uses of 'gsub' with 'tr' or 'chomp' for a speed
improvement
2016-06-10 22:08:37 -05:00
Régis Hanol dffe50a2e6 new alternative reply by email addresses 2016-06-10 16:14:42 +02:00
Régis Hanol fe595f1653 FEATURE: mailjet webhook 2016-06-06 19:47:45 +02:00
Régis Hanol ef8b652d1f FIX: elided part of incoming email was dismissed when creating a new topic 2016-06-06 10:30:04 +02:00
Régis Hanol 99ad251731 different email footer when mailing_list_mode is enabled 2016-06-03 15:48:54 +02:00
Régis Hanol 9704603fab FEATURE: sendgrid webhooks 2016-06-01 21:48:06 +02:00
Régis Hanol 116efffdaa FEATURE: webhooks support for mailgun 2016-05-30 17:11:17 +02:00
David McClure a92fd9d701 Add Site Setting to use HTML from incoming email when available. (#4236) 2016-05-26 10:13:01 +02:00
Régis Hanol addd58d961 fix emoji size in email notifications 2016-05-24 15:00:25 +02:00
Régis Hanol c012b18601 FIX: sending email wasn't working anymore when a plugin used 'reduce_cooked' (cc @gdpelican) 2016-05-21 20:13:00 +02:00
James Kiesel feffe23cc5 FEATURE: More granular mailing list mode (#4068)
* Rearrange frontend to account for mailing list mode

* Allow update of user preference for mailing list frequency

* Add mailing list frequency estimate

* Simplify frequency estimate; disable activity summary for mailing list mode

* Remove combined updates

* Add specs for enqueue mailing list mode job

* Write mailing list method for mailer

* Fix linting error

* Account for stale topics

* Add translations for default mailing list setting

* One query for mailing list topics

* Fix failing spec

* WIP

* Flesh out html template

* First pass at text-based mailing list summary

* Add user avatar

* Properly format posts for mailing list

* Move make_all_links_absolute into Email::Styles

* Apply first_seen_at to user

* Send mailing list email summary hourly based on first_seen_at

* Branch and test cleanup

* Use existing mailing list mode estimate

* Fix failing specs
2016-05-21 15:17:54 +02:00
Régis Hanol c75d58ab21 FEATURE: new 'ignore_by_title' site setting 2016-05-18 23:07:01 +02:00
Régis Hanol 5f76287b18 FEATURE: cap number of staged users (defaults to 10) created per incoming email 2016-05-16 21:45:34 +02:00
Régis Hanol 341d5e10b3 FIX: auto-reply to VERP should be considered as hard boucne 2016-05-16 09:01:15 +02:00
Régis Hanol d224966a0e FIX: retry sending an email in 1 hour when SMTP server is busy 2016-05-09 20:37:33 +02:00
Régis Hanol 1e57bbf5c8 Lots bounce emails related fixes
- Show bounce score on user admin page
- Added reset bounce score button on user admin page
- Only whitelisted email types are sent to emails with high bounce score
- FIX: properly detect bounces even when there is no TO: header in the email
- Don't desactivate a user when reaching the bounce threshold
2016-05-06 19:34:33 +02:00
Régis Hanol 8e611ec7a1 FEATURE: handle bounced emails 2016-05-02 23:15:32 +02:00
Régis Hanol a244c33023 temporarily disable processing email replies to VERP 2016-04-29 18:30:05 +02:00
Régis Hanol 84d4ebd910 use 'reply_by_email_address' setting for handling bounce emails using VERP 2016-04-25 20:06:45 +02:00
Régis Hanol 4960b62110 FEATURE: new 'block_auto_generated_emails' site setting 2016-04-20 21:29:27 +02:00
Régis Hanol 7d9f2265b9 FIX: improve support for handling emails coming from screened email addresses 2016-04-18 23:01:54 +02:00
Sam fad017d842 FEATURE: add support for bounce emails
We now optionally add a Variable Email Return Path to every email we send.
This allows us to cleanly handle email bounces, which in turn will improve
deliverability.
2016-04-18 17:14:01 +10:00
Régis Hanol 4d9c81fde7 Merge pull request #4148 from tgxworld/dont_reply_to_emails_that_are_autogenerated
FIX: Don't send rejection mailer to bounced emails.
2016-04-13 15:36:14 +02:00
Régis Hanol 56089c4856 FEATURE: new 'auto_generated_whitelist' site setting 2016-04-11 22:47:34 +02:00
Régis Hanol a359a973e2 remove useless content when sending email to a staged used 2016-04-11 19:06:10 +02:00
Régis Hanol e4980392da improve receiver specs to account for posts approval 2016-04-11 18:20:26 +02:00
Arpit Jalan c5794ab0f6 allow magic login route emails even when emails are disabled 2016-04-08 11:20:09 +05:30
Guo Xiang Tan 5734c7f3f3 FIX: Don't send rejection mailer to bounced emails. 2016-04-07 22:21:17 +08:00
Régis Hanol 2c508f205a FEATURE: properly set 'In-Reply-To' header when sending an email that is a reply to an incoming email 2016-04-06 21:05:10 +02:00
Régis Hanol 79329c8e85 FIX: check we used the right encoding when parsing incoming emails 2016-03-30 19:54:38 +02:00
Régis Hanol 6c82fbebf5 detect more auto-generated emails 2016-03-30 18:41:09 +02:00
Régis Hanol 39863953cd new 'enable_staged_users' site setting 2016-03-23 18:56:03 +01:00
Régis Hanol 03a1aa0000 SECURITY: only add elided part of email in PM 2016-03-17 23:10:46 +01:00
Régis Hanol e09e807027 FIX: skip validation when replying via email for staged users 2016-03-14 22:21:18 +01:00
Régis Hanol 4a3cb4a000 FIX: use MD5 of the email_string when there's no 'Message-Id' 2016-03-14 18:18:58 +01:00
Régis Hanol 841cec1bc1 FIX: support emails in with weird encoding 2016-03-11 18:51:53 +01:00
Régis Hanol 6d84a8a1b3 FIX: don't send out elided message in email notifications
UX: improved details tag for elided messages
2016-03-11 17:51:16 +01:00
Régis Hanol 2747e14b4c FEATURE: hide elided part of incoming emails behind a [details] tag 2016-03-09 18:51:54 +01:00
Régis Hanol 622d804d46 FEATURE: Add rejection message on rejected IncomingEmail
FIX: Better RateLimit description in rejected IncomingEmail
FEATURE: Send email when hitting a rate limit
2016-03-07 16:56:17 +01:00
scossar a20a52b25f add user locale 2016-03-03 17:21:50 -08:00
Régis Hanol be5a54d67d FEATURE: new 'allow_all_attachments_for_group_messages' site setting 2016-02-29 22:39:24 +01:00
Sam Saffron 89d550dac5 UX: In Reply / Previous Replies some times purple in gmail 2016-02-27 19:07:15 +11:00
Régis Hanol 21b51090bf FIX: don't tell staged users they can visit topics in email notifications 2016-02-26 23:56:56 +01:00
Régis Hanol f7432d8ec9 FEATURE: add support for multiple incoming emails for groups & categories 2016-02-24 19:47:58 +01:00
Régis Hanol 54262cc9b2 FIX: properly handle invalid from header 2016-02-24 17:40:57 +01:00
Régis Hanol c3a393a11f let's try with 'normal' linebreaks 2016-02-24 15:16:29 +01:00
Régis Hanol 999e7ac4bc FIX: newlines around attachments in received emails 2016-02-24 11:38:12 +01:00
Régis Hanol 5415b2666d FIX: staged users couldn't create new topic in a private category via email 2016-02-22 19:57:53 +01:00
Régis Hanol 52a6682690 FIX: don't create an EmailLog when we can't send a digest 2016-02-17 17:31:46 +01:00
Régis Hanol 4ad5660615 add slightly more logs when skipping email notifications 2016-02-15 17:53:07 +01:00
Régis Hanol 86819f08c3 FIX: use RFC-compliant previous replies separator 2016-02-11 18:48:09 +01:00
Régis Hanol cad7fc1062 FIX: don't allow blocked user to send emails in 2016-02-11 10:39:57 +01:00
Régis Hanol 91bb38626c FEATURE: new incoming email details modal 2016-02-10 22:00:27 +01:00
Arpit Jalan 82a75c00c0 UX: change 'Visit Topic' to 'Visit Message' for message notification email 2016-02-09 13:39:10 +05:30
Régis Hanol 7d992cb4c5 FIX: sending emails to mailing list subscribers wasn't working 2016-02-03 19:27:58 +01:00
Régis Hanol cf4c256b17 FEATURE: new 'raw email' modal when listing rejected emails 2016-02-01 21:41:49 +01:00
Régis Hanol 49d1f88f6e FIX: support incoming emails with no subject 2016-02-01 12:16:15 +01:00
Régis Hanol 27aaed0aa5 FIX: don't use 'Return-Path' header to determine whether the email was auto generated 2016-02-01 10:21:58 +01:00
Régis Hanol 46b6c55197 FIX: replace 'discourse_email_parser' with 'email_reply_trimmer' to better trim replies from plain text emails
FIX: undefined method `number_to_human_size' when email contains attachments
2016-01-30 01:29:31 +01:00
Régis Hanol 96380bfd38 FIX: only create 1 email_log when an email is sent 2016-01-29 16:49:49 +01:00
Régis Hanol 378b7f964c Revert "FEATURE (WIP): add max-width and center email notifications" 2016-01-29 11:13:59 +01:00
scossar 77167f12ad move styles to Styles 2016-01-27 19:07:21 -08:00
scossar 3f09ec2aca add layout for notifications 2016-01-27 11:27:32 -08:00
Arpit Jalan f0b04ac730 Merge pull request #3970 from gdpelican/unsubscribe-via-email
Hotfix for unsubscribe via email
2016-01-21 14:59:17 +05:30
James Kiesel b92e2b33eb Hotfix for unsubscribe via email 2016-01-21 21:46:15 +13:00
Régis Hanol 4a17cdc1e4 FIX: don't invite users with emails configured as 'incoming' (reply, group our category) 2016-01-20 23:08:27 +01:00
Régis Hanol f145310cd5 FIX: associates email replies using both 'In-Reply-To' and 'References' headers 2016-01-20 22:52:08 +01:00
Jeff Atwood f5c0972210 Merge pull request #3966 from scossar/simple-email-button
FIX: email button padding for Outlook
2016-01-20 12:25:01 -08:00
scossar 1fb709362a fix border widths 2016-01-20 11:03:13 -08:00
scossar e2b265eb98 reduce border 2016-01-20 10:19:13 -08:00
James Kiesel c7283751a3 Unsubscribe via email 2016-01-20 22:25:25 +13:00