Commit Graph

222 Commits

Author SHA1 Message Date
Sam 766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Kyle Zhao 488fba3c5f
FEATURE: allow plugins and themes to extend the default CSP (#6704)
* FEATURE: allow plugins and themes to extend the default CSP

For plugins:

```
extend_content_security_policy(
  script_src: ['https://domain.com/script.js', 'https://your-cdn.com/'],
  style_src: ['https://domain.com/style.css']
)
```

For themes and components:

```
extend_content_security_policy:
  type: list
  default: "script_src:https://domain.com/|style_src:https://domain.com"
```

* clear CSP base url before each test

we have a test that stubs `Rails.env.development?` to true

* Only allow extending directives that core includes, for now
2018-11-30 09:51:45 -05:00
Vinoth Kannan cedd2118c4
FEATURE: If PM email bounced for staged user then alert in whisper reply (#6648) 2018-11-27 00:29:37 +05:30
Sam 86255faa08 FEATURE: do not switch to JPEG unless you meet 75k byte savings
This also adjusts the algorithm to expect

- 30% saving for JPEG conversion

AND

- Minimum of 75K bytes saved

The reasoning for increase of saving requirements is cause PNG may have been
uploaded unoptimized, 30% saving on PNG is very possible
2018-11-21 11:01:08 +11:00
Régis Hanol 4459665dee
REFACTOR: use tables instead of custom fields for polls (#6359)
Co-authored-by: Guo Xiang Tan <tgx_world@hotmail.com>
2018-11-19 14:50:00 +01:00
Guo Xiang Tan 44391ee8ab
FEATURE: Upload Site Settings. (#6573) 2018-11-14 15:03:02 +08:00
Maja Komel 1ac3e5473a FIX: don't strip eml attachments from received emails 2018-11-05 09:35:22 +01:00
Bianca Nenciu effbef7d0b UX: Use user locale for locations. (#6527)
* UX: Use user locale for locations.

* DEV: Added MaxMindDB test data and fixed test.
2018-10-25 10:54:01 +00:00
David Taylor 7ac08f936e
FEATURE: Upload tags from CSV (#6484) 2018-10-15 09:12:54 +01:00
Gerhard Schlager 7a41a783a4 FIX: Don't reply to Unsubscribe email sent to mailing list mirror 2018-10-11 16:09:22 +02:00
Guo Xiang Tan 5039a6c3f1 FIX: Strip null bytes in mail subjects. 2018-10-11 09:46:32 +08:00
Guo Xiang Tan 40fa96777d
FEATURE: Post deployment migrations. (#6406)
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.

The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.

```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```

To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
2018-10-08 15:47:38 +08:00
Penar Musaraj da9eee5262 FIX: Force enable user PM emails option when user posts to a group by email. 2018-10-02 12:38:10 +08:00
Gerhard Schlager e2770bc1c4 FIX: async reload of locales could result in missing translations 2018-10-01 17:14:36 +02:00
Bianca Nenciu 5407036ef9 DEV: Run prettier. (#6420) 2018-09-21 11:02:23 +00:00
Guo Xiang Tan 4a92c5b2d6 `UploadRecovery` should recover attachments too. 2018-09-19 10:44:36 +08:00
Guo Xiang Tan ce6a0a5e9e FIX: Moving upload to tombstone should update modification time.
A upload created a long time ago will be nuked from the tombstone
immediately if it gets deleted.
2018-09-18 10:48:29 +08:00
Guo Xiang Tan 71caf7521d Drop `tif`, `tiff`, `webp` and `bmp` from supported images.
https://meta.discourse.org/t/cr2-raw-files-are-being-treated-as-tiff-files/96775/3?u=tgxworld
2018-09-12 09:29:54 +08:00
Kris faf09bb8c8 Replacing default brown category color 2018-08-24 14:18:14 -04:00
Guo Xiang Tan 36a7028f19 FEATURE: Clean up `PostReplyKey` records.
* Default retention of 90 days.
2018-08-23 10:40:02 +08:00
Sam d7b1919ead correct specs 2018-08-20 12:46:14 +10:00
Sam 8b5e42ea16 FIX: always test and coerce to image on upload
In the past the filename of the origin was used as the source
for the extension of the file when optimizing on upload.

We now use the actual calculated extension based on upload data.
2018-08-20 12:18:59 +10:00
Guo Xiang Tan 2b57239389 FIX: Upload's content is the only source of truth for the file type. 2018-08-07 13:15:00 +08:00
David Taylor 812add18bd REFACTOR: Serve auth provider information in the site serializer.
At the moment core providers are hard-coded in Javascript, and plugin providers get added to the JS payload at compile time. This refactor means that we only ship enabled providers to the client.
2018-08-06 09:25:48 +01:00
Joffrey JAFFEUX 066010db7d
FEATURE: introduces list/compact_list components 2018-08-03 16:41:37 -04:00
Gerhard Schlager 5d421fb946 FIX: Try respecting charset in HTTP header of RSS feed 2018-08-01 10:41:20 +02:00
Gerhard Schlager ff942ed2f3 FIX: Try detecting encoding of RSS feed 2018-08-01 10:41:20 +02:00
Guo Xiang Tan 30242972d3 Add spec for 4bf3bf6786. 2018-07-26 09:16:14 +08:00
David Taylor eda1462b3b
FEATURE: List, revoke and reconnect associated accounts. Phase 1 (#6099)
Listing connections is supported for all built-in auth providers. Revoke and reconnect is currently only implemented for Facebook.
2018-07-23 16:51:57 +01:00
Guo Xiang Tan 1d74ccaaf8 Add compatibility for ImageMagick7. 2018-07-17 15:50:58 +08:00
Patrick Gansterer 28dd7fb562 FEATURE: Create hidden posts for received spam emails (#6010)
* Add possibility to add hidden posts with PostCreator

* FEATURE: Create hidden posts for received spam emails

Spamchecker usually have 3 results: HAM, SPAM and PROBABLY_SPAM
SPAM gets usually directly rejected and needs no further handling.
HAM is good message and usually gets passed unmodified.
PROBABLY_SPAM gets an additional header to allow further processing.
This change addes processing capabilities for such headers and marks
new posts created as hidden when received via email.
2018-07-05 11:07:46 +02:00
Régis Hanol 8a53941fe0 FIX: less aggressive gmail eliding 2018-07-04 20:04:46 +02:00
OsamaSayegh e58ed247f2 REFACTOR: uploads controller specs to requests (#5907) 2018-06-04 11:13:52 +08:00
Ryan Mulligan fac4bf2f85 ignore emails that are from the reply by email addresses (#5843) 2018-05-23 10:04:45 +02:00
Régis Hanol 86eb3528ec FEATURE: clearer error message when receiving a reply to an old notification 2018-05-09 18:51:01 +02:00
Régis Hanol ddb092f397 FIX: update mail gem to fix UTF-8 parsing issue 2018-04-25 21:53:37 +02:00
Gerhard Schlager fcd352e089 FIX: Try fixing unparsable email addresses
The mail gem returns `UnstructuredField` when it fails to parse email addresses, but the `Receiver` always expects an `AddressList`.
2018-03-27 18:28:54 +02:00
Gerhard Schlager 19c5afc69d Protect against accidental table renames 2018-03-27 13:18:13 +11:00
Sam 6a3c8fe69c FEATURE: protect against accidental column or table drops
Often we need to amend our schema, it is tempting to use
drop_table, rename_column and drop_column to amned schema
trouble though is that existing code that is running in production
can depend on the existance of previous schema leading to application
breaking until new code base is deployed.

The commit enforces new rules to ensure we can never drop tables or
columns in migrations and instead use Migration::ColumnDropper and
Migration::TableDropper to defer drop the db objects
2018-03-21 15:43:32 +11:00
Sam b57d4c87a4 FEATURE: update CommonMark to version 0.28 2018-03-08 12:19:53 +11:00
OsamaSayegh 282f53f0cd FEATURE: Theme settings (2) (#5611)
Allows theme authors to specify custom theme settings for the theme. 

Centralizes the theme/site settings into a single construct
2018-03-04 19:04:23 -05:00
Régis Hanol fd33090646 FEATURE: automatically elides gmail quotes 2018-02-26 23:54:02 +01:00
Sam 720e1965e3 FEATURE: add category suppress from latest
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.

New setting explicitly only removes from latest list but leaves the
category list alond
2018-02-22 09:56:35 +11:00
Guo Xiang Tan c7c8f38eac Use proper encoding for email fixtures. 2018-02-21 17:06:35 +08:00
Régis Hanol 61930e092a FIX: support incoming emails with just an attachment 2018-02-16 18:14:56 +01:00
Sam e456ae1b6c Revert "Upgrade mail gem to drop dependency on `mime-types`."
This reverts commit 498cc15aa9.

Noticed a certain amount of extra allocations due to this upgrade,
reverting to confirm if this is the case
2018-02-16 07:04:22 +11:00
Guo Xiang Tan 498cc15aa9 Upgrade mail gem to drop dependency on `mime-types`.
Boot Memory Usage

```
Total allocated: 225695379 bytes (1990663 objects)
Total retained:  32795116 bytes (291607 objects)
```

```
Total allocated: 218342892 bytes (1897350 objects)
Total retained:  30123426 bytes (261198 objects)
```
2018-02-15 16:24:26 +08:00
Gerhard Schlager eb52c5469e FEATURE: Allow plugins to register a new locale 2018-01-25 14:57:41 +01:00
Gerhard Schlager 9f7ae908d8 Add specs to check email domain whitelist/blacklist for To and Cc 2018-01-10 16:57:26 +01:00
Arpit Jalan 6c1ebbb95c add test case for csv BOM handling 2018-01-09 15:49:41 +05:30
Gerhard Schlager f086d28b30 FIX: Do not validate messages sent to mailing list mirror 2018-01-05 11:21:53 +01:00
Gerhard Schlager ceb7590bcb FIX: bounced email can contain multiple status codes 2018-01-03 17:59:20 +01:00
Kyle Zhao 5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Gerhard Schlager 1a3ab7c02e ignore some site settings for emails sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager 5210e3e744 FEATURE: accept incoming email with reply_key mismatch when original email was forwarded 2017-11-12 23:44:22 +01:00
Gerhard Schlager 4dc4bc70c8 FIX: ignore_by_title should match case-insensitive 2017-11-12 01:43:18 +01:00
Robin Ward 4270c93666 FIX: Missing yml file 2017-11-10 14:29:36 -05:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Régis Hanol be0c7609f1 FIX: validates attachments against current authorized extensions 2017-11-07 19:17:33 +01:00
Vinoth Kannan 59529ea54a generalize spec fixture 2017-11-04 11:47:26 +05:30
Vinoth Kannan a00af4d85a FEATURE: Rake task to export and import category structure 2017-11-01 17:17:05 +05:30
Gerhard Schlager c0bb97b5cb FIX: delete staged users when the incoming email is rejected 2017-10-11 16:17:01 +02:00
Gerhard Schlager 7f50380221 FIX: respect email domain whitelist/blacklist when creating staged users 2017-10-03 16:36:08 +02:00
Gerhard Schlager 76706f9144 FIX: don't create staged users when incoming email is rejected
FIX: don't send subscription mail to new users
2017-10-03 16:36:08 +02:00
Kyle Zhao 15cd3b78ae integration test for PollFeed job 2017-10-02 01:16:11 -04:00
Gerhard Schlager 31ecb4fecf FIX: Handle incoming emails without email address in From header (#5177) 2017-09-12 22:35:24 +02:00
Erick Guan 1646bc0031 FIX: fails loud if default setting is not set
Noted:
- `push_api_secret_key` is set in initializer. Shimed with ''
- `default_theme_key` is set in seeding. Shimed with ''
2017-08-15 12:07:25 +02:00
Guo Xiang Tan 2c39743d5d Introduce multisite tests for better coverage. 2017-08-08 12:58:22 +09:00
Guo Xiang Tan 3f24ed2b3e Can't revert due to incompatibility of new site setting types.
Revert "Revert "FEATURE: Site settings defaults per locale""

This reverts commit 439fe8ba24.
2017-08-07 10:43:09 +09:00
Guo Xiang Tan 439fe8ba24 Revert "FEATURE: Site settings defaults per locale"
This reverts commit 468a8fcd20.
2017-08-07 10:31:50 +09:00
Erick Guan 468a8fcd20 FEATURE: Site settings defaults per locale
This change-set allows setting different defaults for different locales. 

It also:

- Adds extensive testing around site setting validation

- raises deprecation error if site setting has the default property based on env

- relocated site settings for dev and tests in the initializer

- deprecated client_setting in the site setting's loading process

- ensure it raises when a enum site setting being set

- default_locale is promoted to `required` category.

- fixes incorrect default setting and validation

- fixes ensure type check for site settings

- creates a benchmark for site setting

- sets reasonable defaults for Chinese
2017-08-02 12:24:19 -04:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan 0ab0c82f11 Reenable skipped `Email::Receiver` spec. 2017-07-24 22:27:24 +09:00
Sam fc90dd3592 Work in progress, CommonMark spec 2017-07-20 17:47:16 -04:00
David Taylor 9294826f31 Allow inclusion of elided text from emails when sending to a category… (#4844)
* Allow inclusion of elided text from emails when sending to a category address
2017-06-29 13:03:14 +09:00
Sam 6a1f579c6e FIX: don't search for plugins in nested subdirectories 2017-05-16 17:28:56 -04:00
Sam bc0b9af576 FEATURE: support uploads for themes
This allows themes to bundle various assets
2017-05-10 15:47:11 -04:00
Arpit Jalan e89d0a6b20 FIX: importing a theme via file was broken 2017-05-08 12:03:24 +05:30
Régis Hanol bf322281e3 Improve handling of inlined images in incoming emails 2017-05-03 23:02:02 +02:00
Claus Strasburger e9bb9a167b Fix reply-by-mail for 8-bit transfer encodings
The mail class seems to handle mails sent with Content-Transfer-Encoding: 8bit
somewhat weirdly: It decodes them (to utf-8), changes the raw source to base64,
and does not modify the Content-Type:charset= header.

This leads to Discourse trying the message encoding (in my example ISO-8859-1)
first, and if that does not contain any unparseable characters, it uses that.
Sadly, in ISO-8859-1, every byte sequence is valid.

Fix this by always trying to decode as UTF-8 first. The probability of someone
using another encoding that cleanly (but wrongly) decodes as UTF-8 should be
fairly low.
2017-04-30 23:30:40 +02:00
Régis Hanol 887e9af84f FEATURE: new 'max_image_megapixels' site setting 2017-01-11 23:37:12 +01:00
Régis Hanol 185dcb2ca1 handle emails with localized headers 😠 2017-01-09 22:59:30 +01:00
Régis Hanol 98c62bccb5 FIX: mark forwarded email as read by the forwarder
FIX: 'Re:' prefix is mostly used for replies and not forwarded emails
2017-01-06 15:33:55 +01:00
Régis Hanol 62763f025c FIX: wasn't able to parse FROM email in the embedded email 2016-12-01 18:34:47 +01:00
Régis Hanol 214f5bff5c don't send more than 1 reply per day to auto-generated emails 2016-06-28 16:42:05 +02:00
Régis Hanol 800081f606 FIX: staged users weren't able to reply in restricted categories 2016-06-26 19:25:45 +02:00
Régis Hanol dffe50a2e6 new alternative reply by email addresses 2016-06-10 16:14:42 +02:00
Régis Hanol c75d58ab21 FEATURE: new 'ignore_by_title' site setting 2016-05-18 23:07:01 +02:00
Régis Hanol 8e611ec7a1 FEATURE: handle bounced emails 2016-05-02 23:15:32 +02:00
Régis Hanol 4960b62110 FEATURE: new 'block_auto_generated_emails' site setting 2016-04-20 21:29:27 +02:00
Régis Hanol 7d9f2265b9 FIX: improve support for handling emails coming from screened email addresses 2016-04-18 23:01:54 +02:00
Régis Hanol 4d9c81fde7 Merge pull request #4148 from tgxworld/dont_reply_to_emails_that_are_autogenerated
FIX: Don't send rejection mailer to bounced emails.
2016-04-13 15:36:14 +02:00
Régis Hanol 56089c4856 FEATURE: new 'auto_generated_whitelist' site setting 2016-04-11 22:47:34 +02:00
Régis Hanol e4980392da improve receiver specs to account for posts approval 2016-04-11 18:20:26 +02:00
Guo Xiang Tan 5734c7f3f3 FIX: Don't send rejection mailer to bounced emails. 2016-04-07 22:21:17 +08:00
Régis Hanol 39863953cd new 'enable_staged_users' site setting 2016-03-23 18:56:03 +01:00
Régis Hanol 4a3cb4a000 FIX: use MD5 of the email_string when there's no 'Message-Id' 2016-03-14 18:18:58 +01:00
Régis Hanol 841cec1bc1 FIX: support emails in with weird encoding 2016-03-11 18:51:53 +01:00
Régis Hanol be5a54d67d FEATURE: new 'allow_all_attachments_for_group_messages' site setting 2016-02-29 22:39:24 +01:00
Régis Hanol f7432d8ec9 FEATURE: add support for multiple incoming emails for groups & categories 2016-02-24 19:47:58 +01:00