Commit Graph

241 Commits

Author SHA1 Message Date
Régis Hanol fe32733a57 extract signatures from emails sent using Zimbra 2018-04-13 19:04:27 +02:00
Gerhard Schlager f2d00e5eff FEATURE: Use Message-ID for detecting email replies to group
Ignores the site setting "find_related_post_with_key" and always tries to honor the `In-Reply-To` and `References` header for emails sent to a group.

The senders email address must be included in the `To` or `CC` header of a previous email sent to the group and the `Message-ID` of that email must be included in the current email's `In-Reply-To` or `References` header.
2018-04-05 11:00:38 +02:00
Guo Xiang Tan 142571bba0 Remove use of `rescue nil`.
* `rescue nil` is a really bad pattern to use in our code base.
  We should rescue errors that we expect the code to throw and
  not rescue everything because we're unsure of what errors the
  code would throw. This would reduce the amount of pain we face
  when debugging why something isn't working as expexted. I've
  been bitten countless of times by errors being swallowed as a
  result during debugging sessions.
2018-04-02 13:52:51 +08:00
Régis Hanol a0386655a6 Add support to elide content from ProtonMail emails 2018-03-30 10:41:32 +02:00
Gerhard Schlager fcd352e089 FIX: Try fixing unparsable email addresses
The mail gem returns `UnstructuredField` when it fails to parse email addresses, but the `Receiver` always expects an `AddressList`.
2018-03-27 18:28:54 +02:00
Régis Hanol 20ba54d536
FIX: extracting mail content from exchange emails 2018-03-14 22:02:43 +01:00
Guo Xiang Tan a9713ca20b REFACTOR: Always prefer to raise an error if record fails to save. 2018-03-13 12:01:01 +08:00
Gerhard Schlager 832f0a9c4d FIX: Typo prevented extraction of email signatures 2018-03-06 11:34:47 +01:00
Régis Hanol 482c615ef8 FEATURE: extract signatures from most popular email services/software 2018-03-02 01:51:15 +01:00
Régis Hanol 3c430a3949 FEATURE: begone gmail signatures! 2018-02-27 15:19:34 +01:00
Régis Hanol 73ee62f55f FEATURE: automatically elide forwarded emails and signature from outlook 2018-02-27 15:00:50 +01:00
Régis Hanol fd33090646 FEATURE: automatically elides gmail quotes 2018-02-26 23:54:02 +01:00
Régis Hanol 61930e092a FIX: support incoming emails with just an attachment 2018-02-16 18:14:56 +01:00
Régis Hanol 361030d8ad Also try popular Windows-1252 encoding when decoding incoming email 2018-01-30 23:45:04 +01:00
Gerhard Schlager bb54eb1192 Improvements to mbox importer
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
2018-01-17 12:04:57 +01:00
Gerhard Schlager e0d73a957d FEATURE: Allow posting via email to read-only mailing list mirror category 2018-01-05 11:21:53 +01:00
Gerhard Schlager d7cd7e4dc7 FIX: Never mark emails sent to mailing list mirror as auto-generated 2018-01-05 11:21:53 +01:00
Gerhard Schlager ceb7590bcb FIX: bounced email can contain multiple status codes 2018-01-03 17:59:20 +01:00
Gerhard Schlager 16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
Gerhard Schlager 1a3ab7c02e ignore some site settings for emails sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager 9207dee69a FEATURE: escape HTML when cooking plaintext emails 2017-11-15 20:22:11 +01:00
Gerhard Schlager d3baae5365 removes whitespaces and uses scope 2017-11-13 15:23:24 +01:00
Gerhard Schlager 5210e3e744 FEATURE: accept incoming email with reply_key mismatch when original email was forwarded 2017-11-12 23:44:22 +01:00
Gerhard Schlager 4dc4bc70c8 FIX: ignore_by_title should match case-insensitive 2017-11-12 01:43:18 +01:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Régis Hanol be0c7609f1 FIX: validates attachments against current authorized extensions 2017-11-07 19:17:33 +01:00
Gerhard Schlager 880d154381 FIX: deleting staged user of rejected email shouldn't delete incoming email 2017-10-31 15:13:23 +01:00
Gerhard Schlager c0bb97b5cb FIX: delete staged users when the incoming email is rejected 2017-10-11 16:17:01 +02:00
Régis Hanol 3bdd8f57c1 FIX: invited staged users would sometimes not get notified of replies 2017-10-06 16:37:28 +02:00
Gerhard Schlager 1477a0e910 Adds a rake task for refreshing posts received via email
This is useful when the email_reply_trimmer gem was updated and you want to apply those changes to existing posts.
2017-10-06 14:29:11 +02:00
Gerhard Schlager 7f50380221 FIX: respect email domain whitelist/blacklist when creating staged users 2017-10-03 16:36:08 +02:00
Gerhard Schlager 76706f9144 FIX: don't create staged users when incoming email is rejected
FIX: don't send subscription mail to new users
2017-10-03 16:36:08 +02:00
Gerhard Schlager 6e04f05173 that shouldn't have been there... 2017-09-15 17:33:31 +02:00
Gerhard Schlager d51eee4dbc FIX: don't try to send a rejection message when the sender was not detected 2017-09-15 17:30:02 +02:00
Gerhard Schlager a2187b0acd FIX: better error handling for incoming emails 2017-09-15 17:30:02 +02:00
Gerhard Schlager 31ecb4fecf FIX: Handle incoming emails without email address in From header (#5177) 2017-09-12 22:35:24 +02:00
Leo McArdle 99527af38a FIX: show rejected emails with unrecognized errors (#5026)
Although 407a23663d will send rejection
messages for unrecognized errors, sometimes processing the email will
raise an error which has a blank message.

This commit:

1. Shows rejected emails which have already been processed and contain
   a blank error in /admin/email/rejected

2. Replaces new blank error messages with the error type
2017-08-04 16:20:44 +02:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Leo McArdle d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
David Taylor 9294826f31 Allow inclusion of elided text from emails when sending to a category… (#4844)
* Allow inclusion of elided text from emails when sending to a category address
2017-06-29 13:03:14 +09:00
Leo McArdle 5e0efb3410 FEATURE: setting to only use the key when finding the related post of an email reply
this fixes email-in threading problems when using a SMTP server which modifies the message_id
header, like Amazon SES
2017-06-19 12:22:44 +01:00
Régis Hanol 54e8fb0d89 FEATURE: new 'allow_staff_to_upload_any_file_in_pm' site setting 2017-06-12 22:41:29 +02:00
Robin Ward 602e4d0da4 FIX: Return `nil` when the `reply_key` is missing 2017-06-08 14:28:48 -04:00
Gerhard Schlager 8299e7e8c3
Add new, experimental version of mbox importer 2017-05-29 20:59:18 +02:00
Régis Hanol 9dddb81cf6 FIX: remove memoization on class method used in a job 2017-05-22 23:35:41 +02:00
Régis Hanol 4fb335f1f0 FIX: don't process the same incoming email more than once 2017-05-18 16:43:07 +02:00
Régis Hanol 80435cf2af Add mutex around incoming email processing 2017-05-18 01:09:51 +02:00
Régis Hanol 9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
Régis Hanol bf322281e3 Improve handling of inlined images in incoming emails 2017-05-03 23:02:02 +02:00
Claus Strasburger e9bb9a167b Fix reply-by-mail for 8-bit transfer encodings
The mail class seems to handle mails sent with Content-Transfer-Encoding: 8bit
somewhat weirdly: It decodes them (to utf-8), changes the raw source to base64,
and does not modify the Content-Type:charset= header.

This leads to Discourse trying the message encoding (in my example ISO-8859-1)
first, and if that does not contain any unparseable characters, it uses that.
Sadly, in ISO-8859-1, every byte sequence is valid.

Fix this by always trying to decode as UTF-8 first. The probability of someone
using another encoding that cleanly (but wrongly) decodes as UTF-8 should be
fairly low.
2017-04-30 23:30:40 +02:00
Régis Hanol aba76bace6 add support to keep img tags when converting to html 2017-04-28 22:14:46 +02:00
Régis Hanol 0ec15af970 restore the 'incoming_email_prefer_html' site setting 2017-04-27 14:31:11 +02:00
Régis Hanol b76674f640 FEATURE: convert incoming emails in HTML to markdown
- remove incoming_email_prefer_html site setting
- remove HtmlCleaner class
2017-04-26 16:49:06 +02:00
Guo Xiang Tan 423f2ab228 FIX: Processing incoming email should be done in a background job. 2017-04-24 13:57:28 +08:00
Ryan C. Gordon a51c191a66 Make Email::Receiver.check_address() into a class method. 2017-04-05 23:10:36 -04:00
Régis Hanol 91d09ebc08 post ids are always an integer 2017-02-08 23:46:11 +01:00
Régis Hanol e9e88a010f FIX: find replied-to post even when incoming email doesn't have a reply key 2017-02-08 21:38:52 +01:00
Régis Hanol c725225f69 FIX: log message when revoking email with too many bounces 2017-02-01 16:53:24 +01:00
Matt Palmer 04ae3539d0 FEATURE: Better error message when incoming e-mail is missing a Date: header 2017-01-13 11:05:00 +11:00
Neil Lalonde 42c39ab38e Don't display email addresses in staff action logs for revoked email 2017-01-10 17:51:22 -05:00
Régis Hanol 185dcb2ca1 handle emails with localized headers 😠 2017-01-09 22:59:30 +01:00
Régis Hanol 98c62bccb5 FIX: mark forwarded email as read by the forwarder
FIX: 'Re:' prefix is mostly used for replies and not forwarded emails
2017-01-06 15:33:55 +01:00
Régis Hanol eb453d0f82 the note in a FWed email should be a whisper only in PM and when the author is member of the group 2016-12-01 18:43:56 +01:00
Régis Hanol 62763f025c FIX: wasn't able to parse FROM email in the embedded email 2016-12-01 18:34:47 +01:00
Régis Hanol 2125a630d9 FIX: encoding issues with forwarded emails 2016-11-17 12:44:39 +01:00
Régis Hanol a69f45d0da FEATURE: new 'always_show_trimmed_content' site setting 2016-11-16 22:06:07 +01:00
Régis Hanol 17f2be9f88 FEATURE: new 'enable_forwarded_email' site setting 2016-11-16 19:42:11 +01:00
Régis Hanol 5c06076b5c FIX: strip blacklisted attachments before checking for empty email body 2016-08-08 22:20:33 +02:00
Régis Hanol e92f5e4fbf FEATURE: new email attachment blacklists site settings 2016-08-03 17:55:54 +02:00
Régis Hanol 323bd555c0 FIX: process all incoming email's destinations instead of only the first 2016-08-03 15:57:37 +02:00
Régis Hanol ca4c939e9a incoming mail is considered autogenerated when the subject match a list of 'out of office' keywords 2016-08-02 00:04:59 +02:00
Régis Hanol 7d0dc7cb7a FIX: don't create staged user(s) when receiving a bounce (mailer-deamon begone) 2016-08-01 23:37:59 +02:00
Régis Hanol b0f7e4ba00 FEATURE: deactive users after too many bounces 2016-07-25 18:57:06 +02:00
Régis Hanol 7648916314 new 'reset_bounce_score_after_days' site setting 2016-07-25 17:29:54 +02:00
Régis Hanol d2e22ab215 extract bounce scores into site settings 2016-07-25 17:27:28 +02:00
Régis Hanol 6ed03bf3d9 FIX: always update bounce_score even when email was not auto_generated 2016-07-15 18:00:40 +02:00
James Kiesel 3588780ac3 Don't reject likes by email for closed topics (#4311) 2016-07-05 17:33:08 +02:00
James Kiesel 949be591da FEATURE: Allow <3 and ❤ to trigger like via email (#4310) 2016-07-05 15:59:23 +02:00
Régis Hanol 9bf34ce49b FIX: support 'nil' alternative reply by email address 2016-07-02 11:09:06 +02:00
Régis Hanol 214f5bff5c don't send more than 1 reply per day to auto-generated emails 2016-06-28 16:42:05 +02:00
Régis Hanol 376881845c always strip s/mime signatures in incoming emails 2016-06-27 22:26:05 +02:00
Régis Hanol 589bae5c03 try to fix badly encoded emails 2016-06-26 13:27:34 +02:00
Régis Hanol 8c51d34100 FIX: receiving a bounce from a deleted user 2016-06-22 20:41:21 +02:00
Matt Palmer fde5ef4902 Merge pull request #4273 from mpalmer/log-useful-info-replyusernotmatchingerror
Log some useful info when raising ReplyUserNotMatchingError
2016-06-17 12:06:03 +10:00
Matt Palmer eaf111f3e1 Log some useful info when raising ReplyUserNotMatchingError 2016-06-17 12:01:08 +10:00
Régis Hanol dffe50a2e6 new alternative reply by email addresses 2016-06-10 16:14:42 +02:00
Régis Hanol ef8b652d1f FIX: elided part of incoming email was dismissed when creating a new topic 2016-06-06 10:30:04 +02:00
Régis Hanol 116efffdaa FEATURE: webhooks support for mailgun 2016-05-30 17:11:17 +02:00
David McClure a92fd9d701 Add Site Setting to use HTML from incoming email when available. (#4236) 2016-05-26 10:13:01 +02:00
Régis Hanol c75d58ab21 FEATURE: new 'ignore_by_title' site setting 2016-05-18 23:07:01 +02:00
Régis Hanol 5f76287b18 FEATURE: cap number of staged users (defaults to 10) created per incoming email 2016-05-16 21:45:34 +02:00
Régis Hanol 341d5e10b3 FIX: auto-reply to VERP should be considered as hard boucne 2016-05-16 09:01:15 +02:00
Régis Hanol 1e57bbf5c8 Lots bounce emails related fixes
- Show bounce score on user admin page
- Added reset bounce score button on user admin page
- Only whitelisted email types are sent to emails with high bounce score
- FIX: properly detect bounces even when there is no TO: header in the email
- Don't desactivate a user when reaching the bounce threshold
2016-05-06 19:34:33 +02:00
Régis Hanol 8e611ec7a1 FEATURE: handle bounced emails 2016-05-02 23:15:32 +02:00
Régis Hanol a244c33023 temporarily disable processing email replies to VERP 2016-04-29 18:30:05 +02:00
Régis Hanol 4960b62110 FEATURE: new 'block_auto_generated_emails' site setting 2016-04-20 21:29:27 +02:00
Régis Hanol 7d9f2265b9 FIX: improve support for handling emails coming from screened email addresses 2016-04-18 23:01:54 +02:00
Régis Hanol 4d9c81fde7 Merge pull request #4148 from tgxworld/dont_reply_to_emails_that_are_autogenerated
FIX: Don't send rejection mailer to bounced emails.
2016-04-13 15:36:14 +02:00
Régis Hanol 56089c4856 FEATURE: new 'auto_generated_whitelist' site setting 2016-04-11 22:47:34 +02:00
Régis Hanol e4980392da improve receiver specs to account for posts approval 2016-04-11 18:20:26 +02:00