Commit Graph

7879 Commits

Author SHA1 Message Date
Krzysztof Kotlarek b2481adb40
FIX: persist secure image width and height if is given (#10994)
`max-width: 50%; max-height: 400px;` is a good fallback, however, if width and height are given and are smaller than fallback -  we should persist that smaller size.
2020-10-22 13:25:09 +11:00
Martin Brennan 64b0b50ac0
FIX: Pass user to Email::Sender to avoid broken reply key for group_smtp email (#10978)
Our Email::Sender class accepts an optional user argument, which is used to create a PostReplyKey record when present. This record is used to sub out the %{reply_key} placeholder in the Reply-To mail header, so if we do not pass in the user we get a broken Reply-To header.

This is especially problematic in the IMAP group SMTP situation, because these emails go to customers that we are replying to, and when they reply to us the email bounces! This fixes the issue by passing user to the Email::Sender when sending a group_smtp email but there is still more to do in another PR.

This Email::Sender optional user is a bit of a footgun IMO, especially because most of the time we use it there is a user we can source. I would like to do another PR for this after this one to make the parameter not optional, so we don't end up with these reply issues down the line again.
2020-10-22 10:49:08 +10:00
Daniel Waterworth 48c3172913
DEV: Remove preserve_email_structure_when_styling setting (#10956)
This was made adjustable to allow rolling back quickly if problems came
up. The new behaviour was made default in 93137066 and no problems with
this have been reported.
2020-10-19 12:19:30 +01:00
David Taylor b7c680853d DEV: Introduce plugin API to contribute user api key scopes 2020-10-19 10:40:55 +01:00
David Taylor 1cec333f48 REFACTOR: Introduce RouteMatcher class
This consolidates logic used to match routes in ApiKey, UserApiKey and DefaultCurrentUserProvider. This reduces duplicated logic, and will allow UserApiKeysScope to easily re-use the parameter matching logic from ApiKeyScope
2020-10-19 10:40:55 +01:00
Krzysztof Kotlarek 2ad4fc39b6
FIX: calculate page if page param is not given to TopicView (#10953)
Currently, when page param is not given to TopicView we calculate page for canonical_path, however, it is skipped for next_path.

We should use the same calculation to define page, so next page URL will be accurate. Currently if you [view source of meta post](view-source:https://meta.discourse.org/t/post-rate-limit-trigger-for-a-topic-thats-heating-up/98294/46) you will see:

```
<link rel="canonical" href="https://meta.discourse.org/t/post-rate-limit-trigger-for-a-topic-thats-heating-up/98294?page=3" />
<link rel="next" href="/t/post-rate-limit-trigger-for-a-topic-thats-heating-up/98294?page=2">
```
2020-10-19 17:11:49 +11:00
Roman Rizzi 21c53ed249
FEATURE: Topic slow mode. (#10904)
Adds a new slow mode for topics that are heating up. Users will have to wait for a period of time before being able to post again.

We store this interval inside the topics table and track the last time a user posted using the last_posted_at datetime in the TopicUser relation.
2020-10-16 16:24:38 -03:00
Gerhard Schlager d5ef6188ed
PERF: Disable Sidekiq only during database restore (#10857)
It pauses Sidekiq, clears Redis (namespaced to the current site), clears Sidekiq jobs for the current site, restores the database and unpauses Sidekiq. Previously it stayed paused until the end of the restore.

Redis is cleared because we don't want any old data lying around (e.g. old Sidekiq jobs). Most data in Redis is prefixed with the name of the multisite, but Sidekiq jobs in a multisite are all stored in the same keys. So, deleting those jobs requires a little bit more logic.
2020-10-16 15:19:02 +02:00
Bianca Nenciu 43e52a7dc1
DEV: Remove gifsicle dependency (#10357)
Dependency on gifsicle, allow_animated_avatars and allow_animated_thumbnails
site settings were all removed. Animated GIF images are still allowed, but
the generated optimized images are no longer animated for those (which were
used for avatars and thumbnails).

The added 'animated' is populated by extracting information using FastImage.
This field was used to selectively reoptimize old animations. This process
happens in the background.
2020-10-16 13:41:27 +03:00
Neil Lalonde d77e30a011
DEV: generated release notes shouldn't include squashed commits (#10936)
Limit git log output to the first line of the commit message
when generating the list of commits in a release. Some commit messages
are including the names of all commits that were squashed, resulting
in duplicate and confusing lines in the release notes.
2020-10-15 16:20:32 -04:00
Penar Musaraj 5763309953
FEATURE: WCAG compliant color schemes (#10882)
Co-authored-by: Kris <kris.aubuchon@discourse.org>
2020-10-15 14:05:48 -04:00
Neil Lalonde bb165ac8b1
Version bump to v2.6.0.beta4 (#10933) 2020-10-15 12:59:38 -04:00
Justin DiRose f4f2e081d7
FIX: Resolve issues running qunit via rake (#10922)
discourse-perspective-api was not successfully running tests via the
qunit:test rake task due to inconsistent naming between core and the
repo. As a result we no longer need the mapping in the plugin rake task, too.
2020-10-14 14:05:10 -05:00
jbrw 099bf97dca
Tag groups can belong to groups (#10854) 2020-10-14 13:15:54 -04:00
Penar Musaraj caa58a4bd1
DEV: Retry when Net::HTTP throws EOFError (#10921)
Might fix an inconsistent issue when running tests in CI.
2020-10-14 11:55:26 -04:00
Penar Musaraj 74de7a49f5
Simplify theme and color scheme seeding (#10872)
Now that we have support for user-selectable color schemes, it makes sense
to simplify seeding and theme updates in the wizard. 

We now:

- seed only one theme, named "Default" (previously "Light")
- seed a user-selectable Dark color scheme
- rename the "Themes" wizard step to "Colors"
- update the default theme's color scheme if a default is set
(a new theme is created if there is no default)
2020-10-14 10:18:02 -04:00
Gerhard Schlager 57095f0bb7 FIX: Killing a Unicorn worker shouldn't kill a running backup or restore process
By spawning and forking the backup and restore, the process owner changes from 🦄 to the init process.
2020-10-13 19:48:53 +02:00
Gerhard Schlager 1febf11362 FIX: Backup didn't work anymore after a running backup was canceled 2020-10-13 19:48:53 +02:00
Justin DiRose 8c77b84aac
Revert "FEATURE: Upgrade analytics.js to gtag.js (#10893)" (#10910)
Reverting due to a few unforseen issues with customizations.
2020-10-13 12:20:41 -05:00
Justin DiRose f4034226c2
FEATURE: Upgrade analytics.js to gtag.js (#10893)
Per Google, sites are encouraged to upgrade from `analytics.js` to `gtag.js` for Google Analytics tracking. This commit updates core Discourse to use the new `gtag.js` API Google is asking sites to use. This API has feature parity with `analytics.js` but does not use trackers.
2020-10-13 11:24:06 -05:00
Roman Rizzi 108414e47c
DEV: Users must be able to see a topic to moderate it. (#10906)
Follows-up a8c47e7c. It makes more sense to check if the user can see the topic inside the `can_moderate?` method instead of doing it separately.
2020-10-13 13:03:14 -03:00
Joffrey JAFFEUX e54c8a998b
Revert "DEV: makes babel configuration consistent in all cases (#10884)" (#10907)
This reverts commit c39dc9157c.
2020-10-13 15:58:08 +02:00
Joffrey JAFFEUX c39dc9157c
DEV: makes babel configuration consistent in all cases (#10884)
Creates a BabelHelper builder using a default list of plugins, to ensure the transpiled code is always using the same plugins instead of differents plugins in different cases.
2020-10-13 15:33:29 +02:00
Bianca Nenciu 25b8ed740b
DEV: Make site setting type uploaded_image_list use upload IDs (#10401)
It used to be a list of concatenated upload URLs which was prone to
break.
2020-10-13 16:17:06 +03:00
Sam a6d9adf346
DEV: ensure queue_time and background_requests are floats (#10901)
GlobalSetting can end up with a String and we expect a Float
2020-10-13 18:08:38 +11:00
Sam 32393f72b1
PERF: backoff background requests when overloaded (#10888)
When the server gets overloaded and lots of requests start queuing server
will attempt to shed load by returning 429 errors on background requests.

The client can flag a request as background by setting the header:
`Discourse-Background` to `true`

Out-of-the-box we shed load when the queue time goes above 0.5 seconds.

The only request we shed at the moment is the request to load up a new post
when someone posts to a topic.

We can extend this as we go with a more general pattern on the client.

Previous to this change, rate limiting would "break" the post stream which
would make suggested topics vanish and users would have to scroll the page
to see more posts in the topic.

Server needs this protection for cases where tons of clients are navigated
to a topic and a new post is made. This can lead to a self inflicted denial
of service if enough clients are viewing the topic.

Due to the internal security design of Discourse it is hard for a large
number of clients to share a channel where we would pass the full post body
via the message bus.

It also renames (and deprecates) triggerNewPostInStream to triggerNewPostsInStream

This allows us to load a batch of new posts cleanly, so the controller can
keep track of a backlog

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-10-13 16:56:03 +11:00
Jarek Radosz 6932a373a3
FIX: Handle .discourse-compatibility syntax errors (#10891)
Previously, any errors in those files would e.g. blow up the update process in docker_manager.
Now it prints out an error and proceeds as if there was no compatibility file.

Includes:

* DEV: Extract setup_git_repo
* DEV: Use `Dir.mktmpdir`
* DEV: Default to `main` branch (The latest versions of git already do this, so to avoid problems do this by default)
2020-10-12 18:25:06 +02:00
tshenry 7afb5be5f9
DEV: Add discourse-teambuild to official plugins (#10874)
Discourse-teambuild is [marked official on its Meta topic](https://meta.discourse.org/t/discourse-teambuild-run-your-own-team-building-activity/134907) and is included in the [all-the-plugins](1193bcdc0c/.gitmodules (L520-L522)) repo.
2020-10-09 14:50:00 -07:00
jbrw ac31fe8321
FEATURE - SiteSetting to disable user option to hide their profiles and presences (#10885)
* FEATURE - SiteSetting to disable user option to hide their profiles and presences
2020-10-09 17:18:44 -04:00
Arpit Jalan 4326827a4e
FIX: second factor cannot be enabled if SSO is enabled (#10880)
* FIX: second factor cannot be enabled if SSO is enabled

If `enable_sso` setting is enabled then admin should not be able to
enable `enforce_second_factor` setting as that will lock users out.

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-10-09 22:36:38 +05:30
Daniel Waterworth 721ee36425
Replace `base_uri` with `base_path` (#10879)
DEV: Replace instances of Discourse.base_uri with Discourse.base_path

This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
Robin Ward 4a8f5dbfbd REFACTOR: Remove `.erb` file from javascript tests
We can't use erb in Ember CLI (since it does not have Ruby) so this has
been ported to use our `javascript:update_constants` rake test instead.

Note we don't have to run this every time a notification type as it's
only used by fixtures to fill in some specific types we test against.
2020-10-08 18:36:09 -04:00
Martin Brennan f63da1c146
FIX: Confirm new email not sent for staff if email disabled with "non-staff" option (#10794)
See https://meta.discourse.org/t/email-address-change-confirmation-email-not-sent-but-every-other-notification-emails-are/165358

In short: with disable emails set to non-staff, email address change confirmation emails (those sent to the new address) are not sent for staff or admin members.

This was happening because we were looking up the staff user with the to_address of the email, but the to address was the new email address because we are sending a confirm email change email, and thus the user could not be found. We didn't need to do this anyway because we are passing the user into the Email::Sender class anyway.
2020-10-08 13:52:17 +10:00
Guo Xiang Tan 2607bb602e Fix broken spec.
Follow-up to 3c678df942
2020-10-08 10:52:46 +08:00
Sam 3c678df942
PERF: avoid lookbehinds when indexing search (#10862)
* PERF: avoid lookbehinds when indexing search

Previously we used a `EmailCook.url_regexp` this regex used lookbehinds

Unfortunately certain strings could lead to pathological behavior causing
CPU to skyrocket and regex replace to take a very very long time.

EmailCook still needs a fix, but it is less urgent cause it already splits
to single lines. That said we will correct that as well in a seperate PR.

New implementation is far more naive and relies on the extra spaces search
indexer inserts.
2020-10-08 11:40:13 +11:00
jbrw a1918801a4
FIX - downcase strings consistently (#10861)
ruby and postgres can treat certain characters differently when downcasing them. So do all the downcasing in ruby so that we get consistent results.
2020-10-08 11:28:07 +11:00
Penar Musaraj ddd6c990f6
FIX: Respect show_category_definitions_in_topic_lists in category lists (#10853)
When that site setting is enabled, the category counts (new/unread)
include the subcategory definition topics, but the topics aren't included
in the list. This fixes that discrepancy.
2020-10-07 14:19:48 -04:00
Penar Musaraj 2ad7d98990
FIX: Include topics from subcategories in tracked list (#10850) 2020-10-07 12:15:28 -04:00
Joffrey JAFFEUX 82031aaf16
DEV: makes discourse-styleguide core (styleguide) (#10847)
This plugin is only useful for developers, however, making it core allows us to centralize any component modification in one commit.

This integration also adds a new site_setting: `styleguide_admin_only` which allows to enable a styleguide on a live site while restricting visibility to admins only.

By default, styleguide is disabled.
2020-10-07 14:48:38 +02:00
Martin Brennan 6e2be3e60b
FIX: When admin changes an email for the user the user must confirm the change (#10830)
See https://meta.discourse.org/t/changing-a-users-email/164512 for additional context.

Previously when an admin user changed a user's email we assumed that they would need a password reset too because they likely did not have access to their account. This proved to be incorrect, as there are other reasons a user needs admin to change their email. This PR:

* Changes the admin change email for user flow so the user is sent an email to confirm the change
* We now record who the email change request was requested by
* If the requested by user is admin and not the user we note this in the email sent to the user
* We also make the confirm change email route open to anonymous users, so it can be clicked by the user even if they do not have access to their account. If there is a logged in user we make sure the confirmation matches the current user.
2020-10-07 13:02:24 +10:00
Arpit Jalan f4c7c7bff3
FEATURE: bulk remove tags (#10831)
https://meta.discourse.org/t/bulk-actions-remove-tags-in-addition-to-change-tags/52145
2020-10-06 22:41:55 +05:30
Penar Musaraj 4efbf8dc4f
Remove "further reading" step from wizard (#10825)
Instead, add link to top themes on meta in wizard's last step.
2020-10-05 15:06:05 -04:00
Robin Ward 00afd308c1 FIX: Escaped `mailto` URLs would raise an exception
This prevents exceptions from being raised if a URL has an invalid
component.
2020-10-05 14:42:44 -04:00
Penar Musaraj a4356b99af
FEATURE: Separate base and heading font site_settings (#10807)
Allows site administrators to pick different fonts for headings in the wizard and in their site settings. Also correctly displays the header logos in wizard previews.
2020-10-05 13:40:41 -04:00
Bianca Nenciu 318efa8093
FIX: Always let moderators to see their group PMs (#10813)
When enable_personal_messages was disabled, moderators could not see
the private messages for the "moderators" group. The link was displayed
on the client side, but the checks on the server side did not allow it.
2020-10-05 19:38:08 +03:00
Robin Ward 23f24bfb51 REFACTOR: Move javascript tests inside discourse app
This is where they should be as far as ember is concerned. Note this is
a huge commit and we should be really careful everything continues to
work properly.
2020-10-02 11:29:36 -04:00
Gerhard Schlager 99181bb3b8
FIX: Stripping lines from incoming email shouldn't fail for blank body (#10800) 2020-10-02 15:44:35 +02:00
Jarek Radosz 891987a284
DEV: Recover missing files of existing uploads (#10757)
UploadRecovery only worked on missing Upload records. Now it also works with existing ones that have an invalid_etag status.

The specs (first that test the S3 path) are a bit of stub-a-palooza, but that's how much this class interacts with the the outside world. 🤷‍♂️
2020-10-01 14:54:45 +02:00
Martin Brennan 39b2fb8649
FIX: Invalid URLs could raise exceptions when calling UrlHelper.rails_route_from_url (#10782)
Upload.secure_media_url? raised an exceptions when the URL was invalid,
which was a issue in some situations where secure media URLs must be
removed.

For example, sending digests used PrettyText.strip_secure_media,
which used Upload.secure_media_url? to replace secure media with
placeholders. If the URL was invalid, then an exception would be raised
and left unhandled.

Now instead in UrlHelper.rails_route_from_url we return nil if there is something wrong with the URL.

Co-authored-by: Bianca Nenciu <nenciu.bianca@gmail.com>
2020-09-30 15:20:00 +10:00
David Taylor 1ba9b34b03
DEV: Move UserApiKey scopes to dedicated table (#10704)
This has no functional impact yet, but it is the first step in adding more granular scopes to UserApiKeys
2020-09-29 10:57:48 +01:00
Martin Brennan a8ed0b4612
FIX: Correct corrupt encoding in emails containing attachments 2020-09-29 14:10:57 +10:00
Martin Brennan 4193eb0419
FIX: Respect force download when downloading secure media via lightbox (#10769)
The download link on the lightbox for images was not downloading the image if the upload was marked secure, because the code in the upload controller route was not respecting the dl=1 param for force download.

This PR fixes this so the download link works for secure images as well as regular ligthboxed images.
2020-09-29 12:12:03 +10:00
Martin Brennan 3cd601dcc9
FIX: Admin change email for user process improvements and fixes (#10755)
See https://meta.discourse.org/t/changing-a-users-email/164512 for context.

When admin changes an email for a user, we were incorrectly sending the password reset email to the user's old address. Also the new email does not come into effect until the reset password process is done, so this PR adds some notes to the admin to make this clearer.
2020-09-29 09:45:45 +10:00
Arpit Jalan f7940b1d20
FEATURE: advanced search option for max posts count (#10761)
This commit adds an option to search for max posts count and updates
the UI for posts count search to show a min/max range in single line.
2020-09-28 21:34:16 +05:30
Arpit Jalan 4498c59085 FEATURE: add alias for min_post_count search filter 2020-09-28 16:07:44 +05:30
Krzysztof Kotlarek e7c72cd1e4
FIX: deprecate whitelist constants (#10716)
Deprecation of:
WHITELISTED_REDIRECT_HOSTNAMES
CUSTOM_INTERPOLATION_KEYS_WHITELIST
WHITELISTED_SVG_ELEMENTS
2020-09-28 13:52:05 +10:00
Mark VanLandingham b8015ab654
FIX: Dismiss unread respects tracked query param (#10714)
* WIP:  'dismiss...' respectes tracked query param

* Address review comments

* Dismiss new respects query params

* Remove comment

* Better variable name

* remove self
2020-09-25 12:39:37 -07:00
Penar Musaraj b1692422b1
FEATURE: Support using system font in wizard font step (#10742)
Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
2020-09-25 09:39:14 -04:00
Neil Lalonde 2a268bd89b
Version bump to v2.6.0.beta3 2020-09-24 16:05:52 -04:00
Arpit Jalan cdf45f4fe6 Update regex for views search filter. 2020-09-24 17:05:55 +05:30
Arpit Jalan 0c5cd0d1ef FEATURE: advanced search filters for view count 2020-09-24 15:22:18 +05:30
Bianca Nenciu 214b4c3910
FIX: Remove category id from category class name (#10712)
The generated class name included ID (parent_slug-child_slug-child_id),
but the client side did not expect it (parent_slug-child_slug).
2020-09-23 19:22:07 +03:00
Mark VanLandingham 9f73e8779d
FIX: Only staff can edit topic details when first post is locked (#10729) 2020-09-23 11:13:18 -05:00
Bianca Nenciu eb891778ff
DEV: Do not translate font names (#10723)
Use the names as provided by discourse-fonts and remove the
translated strings.

It also ensures that the selected font is present in case a font will
be removed in the future.
2020-09-23 12:00:07 +03:00
Bianca Nenciu 4abbe3d361
FEATURE: Make search filters case insensitive (#10715) 2020-09-23 11:59:42 +03:00
Vinoth Kannan 18f0cef72e DEV: skip topic deletion staff log when user skip new user tips.
Previously, every time when a user skipped the "new user tips" it will add a staff action log since the discobot's welcome post is destroyed.
2020-09-23 12:25:39 +05:30
Robin Ward ce3fe2f4c4 REFACTOR: Support bundling our `admin` section as an ember addon 2020-09-22 15:14:29 -04:00
Krzysztof Kotlarek 0bb51dcbfa
FIX: TL2 promotion message and advance training (#10679)
This is a little bit of refactoring. Core Discourse should have default promotion message for TL2.

In addition, when the Discobot plugin is enabled, the user is invited to advanced training
2020-09-22 10:17:52 +10:00
jbrw 8dac8abdb4
DEV - write JS files to versioned subdirectories (#10707)
* DEV - write JS files to versioned subdirectories

* Cleaner JS in cacheBuster
2020-09-21 15:06:56 -04:00
Penar Musaraj 577293c438
FIX: respect moderator group permissions in guardian (#10713)
Since 9e4ed03, moderators can view groups with visibility level set to "Group owners, members and moderators".

This fixes an issue where moderators can see the group in /g but then get a 404 when clicking on individual groups.
2020-09-21 12:32:43 -04:00
Neil Lalonde f1743ff69c
FIX: error "unknown attribute verified" in uploads rake tasks
The new column is verification_status.
verified: nil is now verification_status: unchecked
2020-09-21 10:47:52 -04:00
Arpit Jalan 376255ed84 UX: add width and height to GitHub onebox inline avatar in emails. 2020-09-21 11:47:09 +05:30
David Taylor fe932eb644
FIX: Ensure multisite DB is set correctly when spawning backup process
Followup to 76477a1c8b
2020-09-18 20:06:49 +01:00
Penar Musaraj c664ba8fa8
DEV: Fix flaky UploadSecurity spec 2020-09-18 14:36:41 -04:00
Anthony Dillon 01afcf30cc
FIX: Whitelist marker elements when uploading SVGs (#10703) 2020-09-18 14:13:07 -04:00
Roman Rizzi f85f73be88
FEATURE: Review posts with media. (#10693)
To check if a post contains any embedded media, we look if the "image_sizes" attribute is present in the new post manager arguments.

We want to see one boxed links, but we only store the raw content of the post. To work around this, I extracted the onebox logic from the composer editor into a module.
2020-09-18 12:45:09 -03:00
Gerhard Schlager 76477a1c8b FIX: Forking prevented notifications from being sent after backup
This is a workaround for https://github.com/rubyjs/mini_racer/issues/175
2020-09-18 17:35:17 +02:00
Martin Brennan 14b324e5ed
FIX: Provide better API for registering custom upload public types (#10697)
With secure media and the UploadSecurity class, we need a nice way for plugins to register custom upload types that should be considered public and never secure.
2020-09-18 11:54:33 +10:00
Martin Brennan 80268357e7
DEV: Change upload verified column to be integer (#10643)
Per review https://review.discourse.org/t/dev-add-verified-to-uploads-and-fill-in-s3-inventory-10406/14180

Change the verified column for Upload to a verified_status integer column, to avoid having NULL as a weird implicit status.
2020-09-17 13:35:29 +10:00
Penar Musaraj 6610576deb
FIX: Clear stylesheet cache on base font change
Previously, we were not clearing the cache on themes using the default
color scheme.
2020-09-15 13:13:09 -04:00
Robin Ward e7abba9bbe FIX: Missing files from previous commit 2020-09-15 10:59:41 -04:00
Penar Musaraj 273db57d6e
FEATURE: Allow admins to delete user SSO records in the UI (#10669)
Also displays the user's last payload in the admin UI to help with debugging SSO issues.
2020-09-15 10:00:10 -04:00
Joffrey JAFFEUX bbddce4d3a
DEV: updates js transpiler to use babel 7 (#10627)
Updates our js transpiler code to use Babel 7.11.6

List of changes in this commit:

- Updates plugins, babel plugins all have a new version which doesn't contain -es2015- anymore
- Drops [transform-es2015-classes](https://babeljs.io/docs/en/babel-plugin-transform-classes) this plugin shouldn't be needed now that we don't support IE
- Drops check-es2015-constants, checking constants is now part of babel and the check-constants plugin is deprecated. As a result the behavior slightly changed, and is now wrapping every const call in a readOnlyError function which would throw if assigned a new value. This explains the modified spec.
- Adds [proposal-optional-chaining](https://babeljs.io/docs/en/babel-plugin-proposal-optional-chaining)

```javascript
const obj = {
  foo: {
    bar: {
      baz: 42,
    },
  },
};

const baz = obj?.foo?.bar?.baz; // 42
```

- Adds [proposal-json-strings](https://babeljs.io/docs/en/babel-plugin-proposal-json-strings)

```javascript
// IN
const ex = "before
after";
//                ^ There's a U+2028 char between 'before' and 'after'


// OUT
const ex = "before\u2028after";
//                ^ There's a U+2028 char between 'before' and 'after'
```

- Adds [proposal-nullish-coalescing-operator](https://babeljs.io/docs/en/babel-plugin-proposal-nullish-coalescing-operator)

```javascript
var object = {};
var foo = object.foo ?? "default"; // default
```

- Adds [proposal-logical-assignment-operators](https://babeljs.io/docs/en/babel-plugin-proposal-logical-assignment-operators)

```javascript
let a;
let b = 2;
a ||= b; // 2
```

- Adds [proposal-numeric-separator](https://babeljs.io/docs/en/babel-plugin-proposal-numeric-separator)

```javascript
let budget = 1_000_000_000_000;
console.log(budget === 10 ** 12); // true
```

- Adds proposal-object-rest-spread https://babeljs.io/docs/en/babel-plugin-proposal-object-rest-spread

```javascript
let { x, y, ...z } = { x: 1, y: 2, a: 3, b: 4 };
console.log(x); // 1
console.log(y); // 2
console.log(z); // { a: 3, b: 4 }
```

- Adds proposal-optional-catch-binding https://babeljs.io/docs/en/babel-plugin-proposal-optional-catch-binding

```javascript
try {

} catch {

} finally {
  // ensures finally is available in every browsers
}
```

- Adds improved regex support for firefox through (transform-dotall-regex](https://babeljs.io/docs/en/next/babel-plugin-transform-dotall-regex.html) and (proposal-unicode-property-regex](https://babeljs.io/docs/en/babel-plugin-proposal-unicode-property-regex)

- Drops async/generator stuff, the browser we target should allow to use this (excepts iterable async)
2020-09-15 09:26:33 +02:00
Guo Xiang Tan 2ff16b3650
FIX: `TopicQuery.list_private_messages_unread` ignore notification level 2020-09-15 13:33:11 +08:00
Penar Musaraj 06b4ca5dc7
FIX: Mark only uploads as verified/unverified in S3 inventory 2020-09-14 10:21:34 -04:00
David Taylor 66eda8c9df
DEV: Add include_pms option to TopicQuery (#10647)
This is intended for use by plugins which are building their own topic lists, and want to include PMs alongside regular topics (e.g. discourse-assign). It does not get used directly in core.
2020-09-14 12:07:35 +01:00
Krzysztof Kotlarek cb58cbbc2c
FEATURE: allow to extend topic_eager_loads in Search (#10625)
This additional interface is required by encrypt plugin
2020-09-14 11:58:28 +10:00
Krzysztof Kotlarek 333ddd4011 SECURITY: return error on oversized images 2020-09-14 10:45:11 +10:00
Justin DiRose 450c731b54
DEV: Add discourse-algolia to official plugins (#10657) 2020-09-11 15:44:06 -05:00
jbrw 033cebf978
DEV - versions of JS files written to a JS file to be included by loa… (#10649)
* DEV - versions of JS files written to a JS file to be included by load-script and appended as params to URLs

* Formatting

* Incorporate feedback from PR

* Update filename of public-js-versions
2020-09-11 13:53:56 -04:00
Gerhard Schlager 81395be4c1 FIX: Ensure uploads are linked to post when revising a post
Editing a post didn't update the `post_uploads` right away. Instead it relied on the `CookedPostProcessor`. This can lead to an inconsistent state if uploads are added or removed during an edit and, for some reason, the `ProcessPost` job doesn't run (successfully). This inconsistency leads to missing uploads, because the newly added uploads appear to be unused and will be deleted by the `CleanUpUploads` job. In addition to that, uploads, which got removed during the edit, appear to be still in use and won't be deleted by the background job.

This commit ensures that the `post_uploads` are updated during the edit without relying on a background job.
2020-09-11 11:49:15 +02:00
Guo Xiang Tan dbc630f45b
PERF: Fix N+1 queries on private messages route. 2020-09-11 15:20:27 +08:00
Sam Saffron 44fba9463b
FEATURE: Add support for not persistent sessions
In some cases Discourse admins may opt for sessions not to persist when a
browser is closed.

This is particularly useful in healthcare and education settings where
computers are shared among multiple workers.

By default `persistent_sessions` site setting is enabled, to opt out you
must disable the site setting.
2020-09-11 15:11:24 +10:00
Gerhard Schlager ac70c48be4 FIX: Prevent "uploads are missing in S3" alerts after restoring a backup
After restoring a backup it takes up to 48 hours for uploads stored on S3 to appear in the S3 inventory. This change prevents alerts about missing uploads by preventing the EnsureS3UploadsExistence job from running in the first 48 hours after a restore. During the restore it  deletes the count of missing uploads from the PluginStore, so that an alert isn't triggered by an old number.
2020-09-10 21:37:48 +02:00
Joffrey JAFFEUX a60ba75ac2
FIX: prevents eslint failure on unmatched patterns in plugins (#10648)
The recent upgrade of eslint fails if no files is found for the defined patterns.
2020-09-10 20:17:40 +02:00
Ahmed Gagan ddab7cc239
DEV: Add plugin api to permit bulk_action parameters (#10638) 2020-09-10 17:18:45 +01:00
Roman Rizzi efb9fd6ac0
FIX: Make sure rel attributes are correctly set. (#10645)
We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 12:59:51 -03:00
David Taylor 2f96474155
FIX: Ignore empty search terms in topic queries
Previously an empty search term would cause an invalid tsquery, and led to a 500 error. Now an empty string will be ignored.
2020-09-10 15:49:11 +01:00
Guo Xiang Tan 5732e4288e
DEV: Remove redundant variable. 2020-09-10 14:35:15 +08:00
Martin Brennan 7f2f87bf59
DEV: Review fixes (#10641)
See comments in https://review.discourse.org/t/dev-imap-log-to-database-10435/14337/6 for context.
2020-09-10 13:41:46 +10:00
Guo Xiang Tan e6ca1b4326
FIX: Admin search for PMs should only search own PMs.
In c6ceda8c, a bug was introduced where an admin searching for his own
private messages will actually end up searching through all private
messages on the site.

Follow-up to c6ceda8c4e
2020-09-10 11:37:18 +08:00
Martin Brennan dede942007
FEATURE: Allow email image embed with secure media (#10563)
This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced:

* `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them.
* `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`.

`Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients.

All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`.

Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions.

![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-10 09:50:16 +10:00
Krzysztof Kotlarek d260e42c8a
FIX: topic_creator accepts participant_count in import mode (#10632)
The issue mentioned here: https://meta.discourse.org/t/imported-private-discussion-doesnt-appear-in-the-author-inbox/163252

`participant_count` is important to attribute for private messages. If they are imported, we should allow them to set that attribute.

A workaround would be evaluating `update_statistics` method on each Topic but that is less performant.
2020-09-10 08:16:57 +10:00
Roman Rizzi cac64a95aa
FIX: We should check for watched words first even if the user is a fast typer. (#10630) 2020-09-09 14:36:22 -03:00
Robin Ward add3d46bc0 FIX: `approve post count` did not take new topics into account
This means if a user created a new topic, and it was approved, it would not
count towards their approval count. Replies worked correctly.
2020-09-09 12:47:37 -04:00
Penar Musaraj b409954398
DEV: Improve theme support for color definitions (#10634)
- Lets child components extend color definitions
- Includes default theme color definitions
- Fails gracefully on color stylesheet SCSS errors
- Includes theme variables when extending colors
2020-09-09 11:43:34 -04:00
Guo Xiang Tan 9b75d95fc6 PERF: Keep track of first unread PM and first unread group PM for user.
This optimization helps to filter away topics so that the joins on
related tables when querying for unread messages is not expensive.
2020-09-09 14:05:41 +08:00
Guo Xiang Tan 0398271f87
FIX: Admin should always be able to view group messages.
Follow-up to b0f22f2523.

Before this fix, `SiteSetting.enable_personal_messages = false` will
prevent admin from viewing group messages.
2020-09-09 08:58:21 +08:00
Penar Musaraj d4a7058cb1
DEV: Strip unicode from color scheme stylesheet filenames (#10628) 2020-09-08 15:00:16 -04:00
Guo Xiang Tan a1d135f12a
DEV: Correct use of `sanitize_sql_array` in `TopicQuery`. 2020-09-08 12:30:09 +08:00
Guo Xiang Tan d3ebaa41ce
DEV: Address review comments for 5ed84d9885. 2020-09-08 11:17:35 +08:00
Guo Xiang Tan 5ed84d9885
SECURITY: Don't allow moderators to list PMs of all groups.
* Also return 404 when a user is trying to list PMs of a group that
cannot be accessed by the user.
2020-09-08 10:37:00 +08:00
Guo Xiang Tan 0d3239bf21
Revert "SECURITY: Don't allow moderators to view the admins inbox"
Superseeded by d9a5280f5665d12bf46efd8cdcc6200da2cdedd8

This reverts commit 18d35bf64a.
2020-09-08 10:36:49 +08:00
Daniel Waterworth 18d35bf64a SECURITY: Don't allow moderators to view the admins inbox 2020-09-07 18:02:41 +01:00
Martin Brennan 431bd84dec
FIX: Make deleted topic post bookmarks more resilient (#10619)
This PR ensures that new bookmarks cannot be created for deleted posts and topics, and also makes sure that if a bookmark was created and then the topic deleted that the show topic page does not error from trying to retrieve the bookmark reminder at.
2020-09-07 14:52:14 +10:00
Jordan Vidrine e73ff68f75
FEATURE: Add css classes to stylesheet link elements (#10598)
Adding these classes to the stylesheet link elements in order to toggle dark/light schemes via this theme-component. Eventually this theme-component could possible be merged into core.
2020-09-04 15:40:40 -05:00
Roman Rizzi 726bae8f0c
DEV: Pass the for_export flag to the before_upload_creation event (#10599) 2020-09-04 17:24:40 -03:00
Bianca Nenciu 58b97ace23
DEV: Use a special import to declare font faces (#10583)
Update discourse-fonts to v0.0.3.

Follow-up to 7b7357147e.
2020-09-04 16:25:50 +03:00
Penar Musaraj 7a0d1cadc5
DEV: Add option to precompile default multisite stylesheets 2020-09-03 13:00:09 -04:00
Robin Ward c4079780be REFACTOR: Remove `_.merge` 2020-09-03 10:03:33 -04:00
Daniel Waterworth 5bb30d8016 FIX: whoops, linting 2020-09-03 12:10:11 +01:00
Daniel Waterworth 6518cac5cf DEV: Add log rake task
Now, if you do `rake log db:migrate`, it'll run migrations, but with
logging enabled, so that you can see what is actually happening.
2020-09-03 11:43:30 +01:00
Joffrey JAFFEUX 02f0637892
FIX: updates highlightjs to latest version to avoid a freeze (#10578)
Example repro of the freeze:

```
```css
[
```
2020-09-02 15:14:48 +02:00
Dan Ungureanu 38c9c87128
FIX: Add to tags result set only visible tags (#10580) 2020-09-02 13:24:40 +03:00
Jeff Wong d49e96c6a3
DEV: add plugin hooks for silence message parameters (#10538)
DEV: add plugin hooks for silence message parameters

Allows plugins to add, and update extra silence message params for custom
i18n vars

Allows plugins to override system messages via `message_title` and
`message_raw` parameters. We can later expose these params where necessary via event
hooks. Expose the parameter for the on user_silenced trigger.
2020-09-01 17:25:24 -07:00
Vinoth Kannan 3b55de90e5 FIX: skip pm view action log while generating webhook payload.
Currently, while generating webhook payloads for a topic it's accidentally adding a personal message view log in 'system' user's history.
2020-09-02 05:40:42 +05:30
Penar Musaraj 899b841554
DEV: Faster SCSS refresh locally for plugins
Only updates the targeted stylesheet when available
2020-09-01 13:56:40 -04:00
Jay Pfaffman 5e66d7d082
FIX: emails.rake use authentication (#10572)
It looks like support got added for DISCOURSE_SMTP_AUTHENTICATION in Discourse, but not in this rake task (or perhaps it's always been broken?).

N.B.: I have not tested this code, but it reportedly works and looks safe to me.

https://meta.discourse.org/t/smtp-unrecognized-authentication-type-with-office-365/145830/6?u=pfaffman
2020-09-01 11:08:18 -04:00
Daniel Waterworth a3580fa292 DEV: Show plugin versions in CI 2020-09-01 16:05:07 +01:00
Blake Erickson 43ffd4d28f
DEV: Switch our fast_xor gem for xorcist (#10565)
* DEV: Switch our fast_xor gem for xorcist

We use the `xor` function as part of password hashing and we want to use
a faster version than the native ruby xor'ing feature so we use a gem
for this.

fast_xor has been abandoned, and xorcist fixed our initial holdup for
switching in https://github.com/fny/xorcist/issues/4

xorcist also has jruby support so we can remove our jruby fallback
logic.

* Move using statement inside of class
2020-08-31 13:20:44 -06:00
Bianca Nenciu f2e14a3946
FEATURE: Add site setting and wizard step to set base font (#10250)
Co-authored-by: Neil Lalonde <neillalonde@gmail.com>
2020-08-31 13:14:09 +03:00
Jarek Radosz 7290ca1f4a DEV: Remove jquery.cookie
A follow up to c172f2068d

cc: @eviltrout
2020-08-31 11:53:01 +02:00
Faizaan Gagan 73c750a1db
FEATURE: allow specifying platform name for platform specific gems (#10559) 2020-08-31 12:36:04 +10:00
Krzysztof Kotlarek 94152e4640
FIX: dont error when bookmark topic is nil (#10555) 2020-08-31 09:15:36 +10:00
Gerhard Schlager ce1620f2ad
FIX: Pluralized translation overrides didn't work for en_US
"en_US" doesn't contain most of the translations, so it falls back to "en". But that behavior stopped translation overrides to work for pluralized strings in "en_US", because it relies on existing translations. This fixes it by looking up the existing translation in all fallback locales.
2020-08-29 00:11:46 +02:00
jbrw e1896f7e71
FIX - rails_route_from_url would fail if path contained unicode characters 2020-08-28 15:10:10 -04:00
Sam Saffron 80e17f5f9e
DEV: Remove staff bypass on fix missing
After thinking about it, I worry that this will potentially leave a site
setting set when people hit ctrl-c ... feels a tiny bit risky, so leaving
it out.
2020-08-28 12:35:35 +10:00
Sam Saffron ed9323043f
DEV: Allow all uploads when fixing missing s3
If we do not do this we can not properly re-download files
2020-08-28 12:28:59 +10:00
Martin Brennan 2352f4bfc7
DEV: Replace SECURE_MEDIA_ROUTE const with other methods (#10545)
This is so if the route changes this const won't be around to bite us, use the Rails route methods instead.
2020-08-28 11:28:11 +10:00
David Taylor a3577435f7
FEATURE: Additional control of iframes in oneboxes (#10523)
This commit adds a new site setting "allowed_onebox_iframes". By default, all onebox iframes are allowed. When the list of domains is restricted, Onebox will automatically skip engines which require those domains, and use a fallback engine.
2020-08-27 20:12:13 +01:00
Gerhard Schlager c0cd69d280
DEV: Restoring backups didn't work on macOS
This is a follow-up for f51ccea028 because specs were failing on macOS. macOS uses bsdtar by default and the --transform option isn't supported by bsdtar. But the -s option is and it works the same.
2020-08-27 19:08:47 +02:00
Jarek Radosz 188d224303
DEV: Restore UploadRecovery specs (#10543)
Originally disabled in 0c0192e. Upload specs now use separate paths for each spec worker.

Fixes an issue in UploadRecovery#recover_from_local – it didn't take into account the testing infix (e.g. test_0) in the uploads/tombstone paths.
2020-08-27 15:57:10 +02:00
Guo Xiang Tan 40c6d90df3 PERF: Create a partial regular post_search_data index on large sites.
With the addition of `PostSearchData#private_message`, a partial
index consisting of only search data from regular posts can be created.
The partial index helps to speed up searches on large sites since PG
will not have to do an index scan on the entire search data index which
has shown to be a bottle neck.
2020-08-27 13:42:00 +08:00
Sam Saffron f7314f8ab4
DEV: Improve s3 upload image analysis
- Introduces uploads:delete_missing_s3 which can be used to "give up" and
delete broken records from the database

- Fixes a bug in fix_missing_s3 - crashing on deleted posts

- Adds more info to analyze_missing_s3
2020-08-27 11:50:07 +10:00
Vinoth Kannan 618a7ecb35 FIX: `default_tags_muted` site setting won't have tag ids.
Instead it only have list of tag names separated by comma.
89fcb75af2
2020-08-26 23:05:29 +05:30
Sam Saffron b60c39fdf0
DEV: search more carefully for missing uploads
rake uploads:analyze_missing_s3 was not looking at all places, amend it so
it looks in all the places where uploads could exist.
2020-08-26 17:48:42 +10:00
Robin Ward 2523a21fc3 FIX: Server side rendering was broken with new `helperContext()` 2020-08-25 09:16:32 -04:00
siriwatknp 1a2800ad07 fix: 🐛 category & tag search regex to support thai character 2020-08-25 16:12:26 +08:00
David Taylor 629ee5494d
FEATURE: Allow plugins to register parameter-based API routes (#10505)
Example usage:

```
add_api_parameter_route(
  method: :get,
  route: "users#bookmarks",
  format: :ics
)
```
2020-08-24 10:24:52 +01:00
Martin Brennan e8a842ab8c
FIX: Stop category logo + background being marked secure (#10513)
Meta topic: https://meta.discourse.org/t/secure-media-uploads-breaks-category-logos/161693

Category backgrounds and logos are public uploads and should not be marked as secure.

I also discovered that a lot of the UploadSecurity specs for public types were returning false positives; this has been fixed.
2020-08-24 17:12:28 +10:00
Guo Xiang Tan 05174df5c0
FIX: Restrict `personal_messages:` advanced search filter to admin.
The filter noops if an incorrect username is passed. This filter is not
exposed as part of the UI but is only used when an admin transitions
from a search within a user's personal messages to the full page search.

Follow-up to 4b30799054.
2020-08-24 13:53:48 +08:00
Guo Xiang Tan c6ceda8c4e
PERF: Avoid extra subquery when searching within PMs for normal user.
Note the following query being generated where the filter for a user's
private messages is executed twice.

```sql
SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id", (TS_RANK_CD(
  post_search_data.search_data,
  TO_TSQUERY('english', '''test'':*ABCD'),
  0|32
)
 * (
  CASE categories.search_priority
  WHEN 2
  THEN 0.6
  WHEN 3
  THEN 0.8
  WHEN 4
  THEN 1.2
  WHEN 5
  THEN 1.4
  ELSE
    CASE WHEN topics.closed
    THEN 0.9
    ELSE 1
    END
  END
)
) rank, topics.bumped_at topic_bumped_at FROM "posts" INNER JOIN "post_search_data" ON "post_search_data"."post_id" = "posts"."id" INNER JOIN "topics" ON "topics"."id" = "posts"."topic_id" AND ("topics"."deleted_at" IS NULL) LEFT JOIN categories ON categories.id = topics.category_id WHERE ("posts"."deleted_at" IS NULL) AND "posts"."post_type" IN (1, 2, 3) AND (topics.visible) AND (topics.archetype = 'private_message' AND post_search_data.private_message) AND (posts.topic_id IN (SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 99999
UNION ALL
SELECT tg.topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 99999 AND gu.group_id = tg.group_id
)) AND (post_search_data.search_data @@ TO_TSQUERY('english', '''test'':*ABCD')) AND (posts.topic_id IN (SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 99999
UNION ALL
SELECT tg.topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 99999 AND gu.group_id = tg.group_id
)) AND ((categories.id IS NULL) OR (NOT categories.read_restricted) OR (categories.id IN (999999))) ORDER BY rank DESC, topic_bumped_at DESC
```
2020-08-24 13:49:43 +08:00
Guo Xiang Tan 2f043dc89a
Fix lint. 2020-08-24 12:38:46 +08:00
Guo Xiang Tan 4b30799054
FIX: Correct `personal_messages:<username>` advanced search filter.
Renamed from `private_messages` to `personal_messages` without
deprecation because the `private_messages` advanced search filter never
worked in the first place when it was implemented.
2020-08-24 11:54:30 +08:00
Gerhard Schlager f51ccea028
FIX: Backups should use relative paths for local uploads
This also ensures that restoring a backup works when it was created with the wrong upload paths in the time between ab4c0a4970 (shortly after v2.6.0.beta1) and this fix.
2020-08-21 15:22:28 +02:00
Guo Xiang Tan 106a2f58a2
DEV: Drop support for deprecated `in:private` search filter. 2020-08-21 17:18:39 +08:00
Guo Xiang Tan 0684118008
DEV: Remove array_agg from search orders that does not need it. 2020-08-21 14:39:07 +08:00
Neil Lalonde 713e6f1bb6
Version bump to v2.6.0.beta2 2020-08-20 16:36:25 -04:00
Gerhard Schlager 208d85aaff DEV: Ensure prettier uses the same patterns everywhere 2020-08-20 16:27:32 +02:00
Gerhard Schlager 11647b79f7 FIX: SiteSettings::LocalProcessProvider didn't work on multisite
It always used "test" as current site.
2020-08-20 11:15:20 +02:00
Vinoth Kannan 89fcb75af2 FIX: default_tags_muted setting should work for anonymous users too. 2020-08-20 10:40:03 +05:30
Blake Erickson 4395e4d165 FIX: Sending a PM through a flag on a deleted post
Because we allow all the other flag types on a deleted post we should be
able to send a pm to the user letting them know why we deleted their
post.

Bug report:

https://meta.discourse.org/t/-/161156
2020-08-19 17:36:52 -06:00
Vinoth Kannan 8348a41124
FEATURE: add `regular_categories` field in site setting & user option. (#10477)
Like "default watching" and "default tracking" categories option now the "regular" categories support is added. It will be useful for sites that are muted by default. The user option will be displayed only if `mute_all_categories_by_default` site setting is enabled.
2020-08-20 00:35:04 +05:30
Mark VanLandingham a3c0d4a8b5
FIX: Error message when setting enforce 2fa with social logins (#10479) 2020-08-19 13:16:31 -05:00
Blake Erickson 367de2594d
FIX: Unlike own posts on ownership transfer (#10446)
* FIX: Unlike own posts on ownership transfer

If a user has liked a post that has passed the
`post_undo_action_window_mins` system setting window and you transfer ownership
of that post to that user you will be the owner of a post that you have
liked, but cannot unlike resulting in a weird UI behavior. This commit
fixes this issue.

The existing tests didn't check for the timeout window for unliking
posts so I added that in.

I couldn't find a good way to do this logic inside of the guardian class
so rather than duplicating behavior of the `PostActionDestroyer` class
inside of the `PostOwnerChanger` I decided to pass in a "bypass"
variable that could be used to check if the calling class is the
'post_owner_changer' and bypass the guardian instead. I went this route
because the guardian `can_delete_post_action` method has no way of
distinguishing how to allow a user to be able to unlike their own posts
after the timeout window but only on a post owner change.

* use an options hash instead
2020-08-19 09:21:02 -06:00
jbrw aa1fc01307
FEATURE - Moderators can create and manage groups (#10432)
Enabling the moderators_manage_categories_and_groups site setting will allow moderator users to create/manage groups.

* show New Group form to moderators

* Allow moderators to update groups and read logs, where appropriate

* Rename site setting from create -> manage

* improved tests

* Migration should rename old log entries

* Log group changes, even if those changes mean you can no longer see the group

* Slight reshuffle

* RouteTo /g if they no longer have permissions to view group
2020-08-19 10:41:40 -04:00
Penar Musaraj 882b0aac19
DEV: Let themes extend color definitions (#10429)
Themes can now declare custom colors that get compiled in core's color definitions stylesheet, thus allowing themes to better support dark/light color schemes. 

For example, if you need your theme to use tertiary for an element in a light color scheme and quaternary in a dark scheme, you can add the following SCSS to your theme's `color_definitions.scss` file: 

```
:root {
  --mytheme-tertiary-or-quaternary: #{dark-light-choose($tertiary, $quaternary)};
}
```

And then use the `--mytheme-tertiary-or-quaternary` variable as the color property of that element. You can also use this file to add color variables that use SCSS color transformation functions (lighten, darken, saturate, etc.) without compromising your theme's compatibility with different color schemes.
2020-08-18 13:02:13 -04:00
Sam Saffron 2a7490149c
DEV: don't fail if in uploads:fix_missing_s3 when fix fails
Previously a single error on a file like invalid extension could fail the
entire rake task
2020-08-18 17:55:49 +10:00
Guo Xiang Tan 92b7fe4c62
PERF: Add partial index for non-pm search. 2020-08-18 15:55:08 +08:00
David Taylor 33c21c5921
DEV: Skip readonly tables when remapping content (#10461) 2020-08-18 08:53:12 +01:00
Sam Saffron 24fe08230f
FEATURE: ensure posts are rebaked when missing is fixed
This ensures any corrupt optimized images are removed and re-created
2020-08-18 15:37:24 +10:00
David Taylor 645b65adfa
FIX: Add playsinline to twitter GIFs 2020-08-18 00:26:51 +01:00
David Taylor eb981b8fbb
FEATURE: Autoplay oneboxed twitter GIF media 2020-08-17 23:56:41 +01:00
David Taylor a895e4a15b
FIX: Skip rendering twitter video if matching format not found
Followup to d5a4318a
2020-08-17 20:53:14 +01:00
David Taylor d5a4318ac1
FIX: Replace iframe with <video> for twitter videos
This avoids the samesite cookie related error on chrome. It also adds support for twitter 'GIF' content, and allows videos to resize smoothly for narrow devices.
2020-08-17 20:37:36 +01:00
jahan-ggn 65649eaef0
User card settings (#10302)
* settings implemented

* prettier

* settings updated

* rubocop

* prettier

* Revert "rubocop"

This reverts commit 7805145a7d.

* Revert "prettier"

This reverts commit 2c53f4fa12.

* settings updated and changed

* rubocop

* changes applied

* final changes done

* Server side feature added

* spec changed

* changed user_updater and profile file

* Fix user card specs

* web hook serializer solved

* site-setting changed

Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
2020-08-17 12:37:45 -04:00
David Taylor 0a5376084d
FIX: Ensure auto close notice is posted with system locale
Previously it was created with the locale of the user who created the final post in the topic
2020-08-17 15:40:47 +01:00
Guo Xiang Tan 248bebb8cd
PERF: Remove extra subquery in search.
I also noticed that removing the subquery helps the planner to plan
better.
2020-08-17 13:52:12 +08:00
David Taylor 8ac85f54fb
DEV: Remove blob: workers from CSP (#10440)
Ace editor is reconfigured to load workers directly from their JS URL. Workers must be on the same origin as the site, so they will not use the CDN.
2020-08-14 18:15:30 +01:00
Vinoth Kannan 476d26159a
FEATURE: add new user option `skip_new_user_tips`. (#10437)
And add new site setting `default_other_skip_new_user_tips` in user preferences category.
2020-08-14 19:10:56 +05:30
David Taylor 3cf93e9a8f
Revert "DEV: Disable ACE editor worker blobs"
This reverts commit d5463d2a4d.

With S3 assets, CORS prevents loading worker assets directly.
2020-08-14 14:12:11 +01:00
David Taylor a5608025aa
FIX: Add script asset locations to worker-src CSP directives
We no longer need :blob worker src since d5463d2a. But we do want to allow workers to be loaded from all our existing script-src options.
2020-08-14 12:31:50 +01:00
Martin Brennan 4670b62969
DEV: IMAP log to database (#10435)
Convert all IMAP logging to write to a database table for easier inspection. These logs are cleaned up daily if they are > 5 days old.

Logs can easily be watched in dev by setting DISCOURSE_DEV_LOG_LEVEL=\"debug\" and running tail -f development.log | grep IMAP
2020-08-14 12:01:31 +10:00
Penar Musaraj eae8b0465c
DEV: Upgrade ACE Editor to 1.4.12, remove extra ACE files (#10431) 2020-08-13 21:53:27 -04:00
Martin Brennan ffb31b8d2b
FIX: Do not require tagging to be enabled for IMAP archive and delete (#10426)
Previously we did an early return if either SiteSetting.tagging_enabled or SiteSetting.allow_staff_to_tag_pms was false when updating the email on the IMAP server -- however this also stopped us from archiving or deleting emails if either of these were disabled.
2020-08-13 14:04:40 +10:00
Sam Saffron 5be26c7d24
DEV: add error handling in case download fails 2020-08-13 13:48:23 +10:00
Jeff Wong 787ad7d84d FIX: allow plugin pinning to fetch missing commits
Add update for fetching git commits if they do not exist, eg with
clone --depth 1 - only can fetch via git fetch --depth 1 {remote} {ref}
the ref needs to be a full, non-ambiguous reference.
2020-08-12 20:03:37 -07:00
Sam Saffron 5011435ec7
DEV: do not correct sha when correctly uploads 2020-08-13 11:52:57 +10:00
Sam Saffron f48fa30ecd
DEV: fix_missing_s3 attempts to re-download if missing
unverified uploads get re-downloaded and corrected if they exist in the task
2020-08-13 11:22:14 +10:00
David Taylor bd0a7553c4
DEV: Detect when s3 inventory failure is caused by etag difference (#10427) 2020-08-13 09:30:28 +10:00
David Taylor 451b9b245f
DEV: Rename new upload rake tasks
These tasks are s3-specific, so update the names to make that very clear
2020-08-12 23:26:13 +01:00
Michael Brown c25dc43f54 FIX: AWS S3 errors don't necessarily include a message
* If the error doesn't have a message, the class name will help
* example:
  before: "Failed to download #{filename} because "
  after: "Failed to download #{filename} because Aws::S3::Errors::NotFound"
2020-08-12 17:00:09 -04:00
Rafael dos Santos Silva 28669dfeb2
PERF: Faster TL3 promotion replies needed calculation (#10416)
Removing the LIMIT makes PostgreSQL use index_posts_on_user_id_and_created_at
which is much faster overall.

Before: 22 seconds
After: 100 ms
2020-08-12 11:28:34 -03:00
Penar Musaraj 6dd9f2eca2
FIX: color scheme selection with non-default theme
This fixes an issue where a non-default theme set to use the base color
scheme (i.e. the theme had an empty `color_scheme_id`) was loading the
default theme's color scheme instead.
2020-08-12 08:49:13 -04:00
Guo Xiang Tan 93f8396b4b
FIX: Limit PG headline based search blurb generation to 200 characters.
* Recovers omission characters '...' in blurb as well.
2020-08-12 15:34:27 +08:00
Sam Saffron ec173a72d9
DEV: add more counts to analyze missing 2020-08-12 17:28:58 +10:00
Sam Saffron 97fe68e980
DEV: look for avatars when analyzing missing uploads 2020-08-12 14:04:21 +10:00
Sam Saffron 990cd8c2e4
FEATURE: introduce tasks for dealing with legacy broken uploads
`rake uploads:analyze_missing` can be used get rich information regarding
uploads missing from s3 (where verified is false)

`rake uploads:fix_missing` is a work in progress task for automatically
correcting certain historic issues. At the moment it simply rebakes all
posts with missing uploads, but it will improve over time
2020-08-12 13:33:04 +10:00
Martin Brennan 95b71b35d6
FEATURE: IMAP delete email sync for group inboxes (#10392)
Adds functionality to reflect topic delete in Discourse to IMAP inbox (Gmail only for now) and reflecting Gmail deletes in Discourse.

Adding lots of tests, various refactors and code improvements.

When Discourse topic is destroyed in PostDestroyer mark the topic incoming email as imap_sync: true, and do the opposite when post is recovered.
2020-08-12 10:16:26 +10:00
Penar Musaraj c05aced094
FIX: Invalidate cache when updating color scheme colors (#10417) 2020-08-11 16:28:59 -04:00
Martin Brennan b950b3fb3f
DEV: Add verified to uploads and fill in S3 inventory (#10406)
When we run the S3 inventory, mark uploads that exist as verified true, those that don't as verified false, and uploads not included in the check / not yet checked as verified nil.
2020-08-11 14:43:51 +10:00
Blake Erickson 2032c11f78
FIX: Return 422 when creating topics with tags w/out permission (#10400)
The UI prevents users from trying to create tags on topics when they
don't have permission, but if you are trying to add tags to a topic via
the API and you don't have permission before this change it would
silently succeed in creating the topic, but it wouldn't have any tags.

Now a 422 error will be returned with an error message when trying to
create a topic with tags when tagging is disabled or you don't have
enough trust level to add tags to a topic.

Bug report: https://meta.discourse.org/t/-/70525/14
2020-08-10 16:14:15 -06:00
jbrw d67f7a7984
Allows mod posts to be created for category group moderators on closed/archived topics (#10399) 2020-08-10 15:21:01 -04:00
David Taylor 4476680d4b
UX: Set silence_reason using the system locale
Previously we would use the user's locale, which can be confusing for moderators.
2020-08-10 18:51:54 +01:00
Penar Musaraj f179510a68
FIX: include both name and id in color scheme stylesheet filename slugs (#10397) 2020-08-07 13:43:45 -04:00
jbrw 1399e87cdc
Add recently renamed category group moderation setting to DeprecatedSettings 2020-08-07 13:21:25 -04:00
Régis Hanol bc63232d2e
FIX: sync reviewable count when opening the hamburger menu (#10368)
When a tab is open but left unattended for a while, the red, green, and blue
pills tend to go out of sync.

So whevener we open the notifications menu, we sync up the notification count
(eg. blue and green pills) with the server.

However, the reviewable count (eg. the red pill) is not a notification and
is located in the hamburger menu. This commit adds a new route on the server
side to retrieve the reviewable count for the current user and a ping
(refreshReviewableCount) from the client side to sync the reviewable count
whenever they open the hamburger menu.

REFACTOR: I also refactored the hamburger-menu widget code to prevent repetitive uses
of "this.".

PERF: I improved the performance of the 'notify_reviewable' job by doing only 1 query
to the database to retrieve all the pending reviewables and then tallying based on the
various rights.
2020-08-07 18:13:02 +02:00
jbrw 3593e582a3
FIX - limit number of embedded media items in a post (#10391)
* FIX - limit number of embedded media items in a post

* Add renamed settings to DeprecatedSettings
2020-08-07 12:08:59 -04:00
Daniel Waterworth abebc4e05e DEV: Use verbose mode when running tests in CI 2020-08-07 17:04:55 +01:00
Daniel Waterworth e55578681e DEV: Add seed explicitly in turbo_rspec so that runs can be reproduced 2020-08-07 16:51:53 +01:00
Penar Musaraj 86cb5803ba
Convert HEIC uploads to JPG by default (#10395) 2020-08-07 11:17:50 -04:00
Régis Hanol c5c5bb4fad DEV: prevent already initialized constant warning 2020-08-07 16:28:43 +02:00
Guo Xiang Tan 053cbe3112
PERF: Limit characters used to generate headline for search blurb.
We determined using the following benchmark script that limiting to 2500 chars would mean a maximum of
25ms spent generating headlines.

```
require 'benchmark/ips'

string = <<~STRING
Far far away, behind the word mountains...
STRING

def sql_excerpt(string, l = 1000000)
  DB.query_single(<<~SQL)
  SELECT TS_HEADLINE('english', left('#{string}', #{l}), PLAINTO_TSQUERY('mountains'))
  SQL
end

def ruby_excerpt(string)
  output = DB.query_single("SELECT '#{string}'")[0]
  Search::GroupedSearchResults::TextHelper.excerpt(output, 'mountains', radius: 100)
end

puts "Ruby Excerpt: #{ruby_excerpt(string)}"
puts "SQL Excerpt: #{sql_excerpt(string)}"
puts

Benchmark.ips do |x|
  x.time = 10

  [1000, 2500, 5000, 10000, 20000, 50000].each do |l|
    short_string = string[0..l]

    x.report("ts_headline excerpt #{l}") do
      sql_excerpt(short_string, l)
    end

    x.report("actionview excerpt #{l}") do
      ruby_excerpt(short_string)
    end
  end

  x.compare!
end
```

```
actionview excerpt 1000:    20570.7 i/s
actionview excerpt 2500:    17863.1 i/s - 1.15x  (± 0.00) slower
actionview excerpt 5000:    14228.9 i/s - 1.45x  (± 0.00) slower
actionview excerpt 10000:    10906.2 i/s - 1.89x  (± 0.00) slower
actionview excerpt 20000:     6255.0 i/s - 3.29x  (± 0.00) slower
ts_headline excerpt 1000:     4337.5 i/s - 4.74x  (± 0.00) slower
actionview excerpt 50000:     3222.7 i/s - 6.38x  (± 0.00) slower
ts_headline excerpt 2500:     2240.4 i/s - 9.18x  (± 0.00) slower
ts_headline excerpt 5000:     1258.7 i/s - 16.34x  (± 0.00) slower
ts_headline excerpt 10000:      667.2 i/s - 30.83x  (± 0.00) slower
ts_headline excerpt 20000:      348.7 i/s - 58.98x  (± 0.00) slower
ts_headline excerpt 50000:      131.9 i/s - 155.91x  (± 0.00) slower
```
2020-08-07 14:36:52 +08:00
Guo Xiang Tan e60c74d3c1
FEATURE: Use PG `ts_headline` for highlighting topic title in search. 2020-08-07 12:43:09 +08:00
Krzysztof Kotlarek 12a00d6dc5
FEATURE: add advanced order to search (#10385)
Similar to `advanced_filter` I introduced `advanced_order`.

I needed a new option because default orders are evaluated after advanced_filter so I couldn't use it.

Also, that part is a little bit more generic
```
elsif word =~ /order:\w+/
  @order = word.gsub('order:', '').to_sym
nil
```

After those changes, I can use them in plugins in this way:
```
Search.advanced_order(:votes) do |posts|
  posts.reorder("COALESCE((SELECT dvvc.counter FROM discourse_voting_vote_counters dvvc WHERE dvvc.topic_id = subquery.topic_id), 0) DESC")
end
```
2020-08-07 12:47:00 +10:00
Guo Xiang Tan ab2b6f8dea
FIX: Specify config when generating tsquery using `ts_headline`. 2020-08-07 10:21:14 +08:00
Robin Ward 7df57b35da REFACTOR: Remove `Discourse.__widget_helpers`
It's now a variable in the context where the templates are created.
2020-08-06 14:35:46 -04:00
Jarek Radosz cd4f251891
FEATURE: Poll breakdown 2.0 (#10345)
The poll breakdown modal replaces the grouped pie charts feature.

Includes:

* MODAL: Untangle `onSelectPanel`
Previously modal-tab component would call on click the onSelectPanel callback with itself (modal-tab) as `this` which severely limited its usefulness. Now showModal binds the callback to its controller.

"The PR includes a fix/change to d-modal (b7f6ec6) that hasn't been extracted to a separate PR because it's not currently possible to test a change like this in abstract, i.e. with dynamically created controllers/components in tests. The percentage/count toggle test for the poll breakdown feature is essentially a test for that d-modal modification."
2020-08-06 17:57:06 +02:00
David Taylor 76c02cac65
DEV: Ensure the correct plugin stylesheets are reloaded in development
If a plugin name contained the name of another plugin, the wrong stylesheets would be reloaded. For example, working on discourse-prometheus-alert-receiver would cause discourse-prometheus stylesheets to be reloaded.
2020-08-06 16:10:57 +01:00
Penar Musaraj 87e2c9de24
DEV: Plugins can extend color definitions (#10383) 2020-08-06 09:46:17 -04:00
Penar Musaraj 6fdc711b4a
FEATURE: Allow users to opt out of automatic dark mode (#10377) 2020-08-06 09:45:37 -04:00
David Taylor 8c03868808
DEV: Correctly render data- attributes in widget hbs templates (#10376)
In virtualdom, element 'properties' are not completely synonymous with element 'attributes'. In particular, `data-` properties will not be rendered as attributes. To ensure all attributes are passed through, we need to include them under an `attributes` key. For more info, see https://github.com/Matt-Esch/virtual-dom/blob/master/docs/vnode.md#custom-attributes-data-
2020-08-06 14:33:09 +01:00
Guo Xiang Tan 2193d02433
PERF: Use PG headlines for blurb generation and highlighting for search. 2020-08-06 14:56:29 +08:00
Sam Saffron ba482c251c
FIX: stop sync of tracking state when list is filtered
This stops sync of tracking state when list is filtered, in the past this
would cause the tracking state to go off wack.

Additionally this introduces an alias for "filter=tracking", called "f=tracking"

This was done cause the term "filter" is used internally in 2 different ways
the main way is for /unread /new filtering.

Trying to also call a query param "filter" causes enormous amounts of
internal pain, this circumvents the issue.
2020-08-06 16:34:02 +10:00
Guo Xiang Tan 255b0e9f14
PERF: Replace video and audio links in search blurb while indexing.
In the near future, we will be swtiching to PG headlines to generate the
search blurb. As such, we need to replace audio and video links in the
raw data used for headline generation. This also means that we avoid
replacing links each time we need to generate the blurb.
2020-08-06 12:25:03 +08:00
Guo Xiang Tan 06ef87da51
DEV: Make rubocop happy. 2020-08-06 10:11:07 +08:00
Guo Xiang Tan ee5d8fba0c
PERF: Optimize `ActionView::Helpers::TextHelper#excerpt`. 2020-08-06 09:59:20 +08:00
jbrw 67e8bc5342
FEATURE - allow category group moderators to split/merge topics (#10351) 2020-08-05 10:33:25 -04:00
David Taylor cb12a721c4
REFACTOR: Refactor pull_hotlinked_images job
This commit should cause no functional change
- Split into functions to avoid deep nesting
- Register custom field type, and remove manual json parse/serialize
- Recover from deleted upload records

Also adds a test to ensure pull_hotlinked_images redownloads secure images only once
2020-08-05 12:14:59 +01:00
Andrew Schleifer aee3c2c34d FIX: attempt to output a useful error message
currently AWS problems will just dump a stack trace
2020-08-05 17:49:42 +08:00
Guo Xiang Tan 850d5696f2
FIX: Disable concurrent migration for `multisite:migrate`.
Rails migration is not thread safe.
2020-08-05 15:12:56 +08:00
Guo Xiang Tan 1d281353b6
FIX: Don't run seeds if multisite migration fails. 2020-08-05 15:12:56 +08:00
Sam Saffron e8328968bb
DEV: allow qunit:test to run concurrently
Previously we were fighting on pid file which makes it hard to run multiple
2020-08-05 16:57:12 +10:00
Kane York 13feb300a8
FIX: Topic map was incorrectly counting assign actions (#10360)
The assign plugin is one of two situations where a post can be both a whisper and a small-action. Check the action_code field to filter out small-actions.
2020-08-05 11:51:28 +10:00
Penar Musaraj d0953dcda9
DEV: Fix watching plugin stylesheet changes locally
I broke this in e6dbb4fcf5
2020-08-04 14:10:25 -04:00
Penar Musaraj 266c0c50d7
FIX: Load base color scheme when default theme is not set
Followup to c937afc
2020-08-04 12:15:07 -04:00
Régis Hanol ef92b72ff6 DEV: ensure 'posts:reorder_posts' rake task work on single topic
when the topic_id argument is provided.
2020-08-04 14:57:49 +02:00
Joffrey JAFFEUX 8a0478b97d
DEV: adds plugin api to add custom recipients of a post revision (#10367)
* DEV: adds plugin api to add custom recipients of a post revision

Usage:

```
add_post_revision_notifier_recipients do |post_revision|
  [78]
end
```
2020-08-04 11:57:33 +02:00
Régis Hanol 712ab33ff8 DEV: fix 'posts:reorder' rake task
Running the reorder rake task was triggering the following error:

PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "post_timings_unique"

I re-worked the queries and refactored to use the same couple of queries for all similar tables/columns.
2020-08-04 11:07:07 +02:00
Guo Xiang Tan 4c03a944f6
PERF: Move URI regexp in `GroupSearchResults.blurb_for` into constant
No need to generate the huge regexp over and over again.
2020-08-04 14:38:43 +08:00
Guo Xiang Tan 3b08b15855
PERF: Remove one extra call to Redis when searching. 2020-08-04 14:02:02 +08:00
Martin Brennan 5a3494b1e1
FIX: IMAP archive fix and group list mailbox code unification (#10355)
* Fixed an issue I introduced in the last PR where I am just archiving everything regardless of whether it is actually archived in Discourse man_facepalming
* Refactor group list_mailboxes IMAP code to use providers, add specs, and add provider code to get the correct prodivder
2020-08-04 14:19:57 +10:00
Penar Musaraj c937afc75e
FEATURE: automatic dark mode (#10341)
A first step to adding automatic dark mode color scheme switching. Adds a new SCSS file at `color_definitions.scss` that serves to output all SCSS color variables as CSS custom properties. And replaces all SCSS color variables with the new CSS custom properties throughout the stylesheets. 

This is an alpha feature at this point, can only be enabled via console using the `default_dark_mode_color_scheme_id` site setting.
2020-08-03 22:57:10 -04:00
Gerhard Schlager 92b2448500 DEV: Fix i18n deprecation warnings 2020-08-03 15:57:29 +02:00
Guo Xiang Tan fe83baa9b3
FIX: Exclude `DELETE` methods from invalid request with payload.
Follow-up 105d560177

Our client side code is sending params as part of the request payload so
that is going to be tricky to fix.
2020-08-03 17:05:11 +08:00
Guo Xiang Tan 105d560177
SECURITY: 413 for GET, HEAD or DELETE requests with payload. 2020-08-03 14:21:33 +08:00
Guo Xiang Tan 0d8c49c385
DEV: Remove unnecessary rack patches.
PRs have all been merged.
2020-08-03 14:17:10 +08:00
Martin Brennan ed68ac054d
DEV: Cover other case for generated message ID in Email::Receiver 2020-08-03 13:37:06 +10:00
Martin Brennan 7f55ed1a4a
DEV: Clarify Email::Receiver existing incoming email behaviour 2020-08-03 13:31:34 +10:00
Martin Brennan 2920988b3a
FIX: IMAP sync email update uniqueness across groups and minor improvements (#10332)
Adds a imap_group_id column to IncomingEmail to deal with an issue where we were trying to update emails in the mailbox, calling IncomingEmail.where(imap_sync: true). However UID and UIDVALIDITY could be the same across accounts. So if group A used IMAP details for Gmail account A, and group B used IMAP details for Gmail account B, and both tried to sync changes to an email with UID of 3 (e.g. changing Labels), one account could affect the other. This even applied to Archiving!

Also in this PR:

* Fix error occurring if we do a uid_fetch and no emails are returned
* Allow for creating labels within the target mailbox (previously we would not do this, only use existing labels)
* Improve consistency for log messages
* Add specs for generic IMAP provider (Gmail specs still to come)
* Add custom archiving support for Gmail
* Only use Message-ID for uniqueness of IncomingEmail if it was generated by us
* Various refactors and improvements
2020-08-03 13:10:17 +10:00