Commit Graph

8496 Commits

Author SHA1 Message Date
Krzysztof Kotlarek 7f56abac90
FIX: remove post-action when a post is permanently deleted (#11242)
Followup of https://github.com/discourse/discourse/pull/11115

When we permanently remove the post, we should remove related post-actions as well.
2020-11-17 07:40:36 +11:00
David Taylor 475b4892e3
DEV: Remove 'diff local changes' when updating remote themes (#11247)
Since 65e123498b, it is now impossible to make local changes to remote themes, so this warning is not needed.
2020-11-16 19:28:12 +00:00
Roman Rizzi 689623ffc0
FEATURE: Add an API key scopes for handling incoming email. (#11245)
Admins need to create a global API key if they want to use the mail-receiver. Let's add a scope for that.
2020-11-16 14:14:12 -03:00
Martin Brennan 879e4a9e29
FIX: Inline avatar style for onebox when embedding secure images (#11229)
When embedding secure images that are inline-avatars for oneboxes we weren't applying the correct sizing/style.
2020-11-16 09:58:40 +10:00
Bianca Nenciu 5ca0fbc423
FIX: Show read indicator only for group PMs (#11224)
It used to show for PMs converted to public topics.
2020-11-13 19:13:37 +02:00
Justin DiRose 65e123498b
FIX: Disallow editing of remote themes (#11189)
Allowing the editing of remote themes has been something Discourse has advised against for some time. This commit removes the ability to edit or upload files to remote themes from Admin > Customize to enforce the recommended practice.
2020-11-13 09:57:49 -06:00
David Taylor dc005c593e
DEV: Introduce `:before_auth` DiscourseEvent (#11233)
This is useful for plugins to manipulate the auth hash from OmniAuth before it is read by the Authenticator class
2020-11-13 14:41:54 +00:00
Roman Rizzi f2bef7ea8f
FIX: Store Reviewable's force_review as a boolean. (#11219)
* FIX: Store Reviewable's force_review as a boolean.

Using the `force_review` flag raises the score to hit the minimum visibility threshold. This strategy turned out to be ineffective on sites with a high number of flags, where these values could rapidly fluctuate.

This change adds a `force_review` column on the reviewables table and modifies the `Reviewable#list_for` method to show these items when passing the `status: :pending` option, even if the score is not high enough. ReviewableQueuedPosts and ReviewableUsers are always created using this option.
2020-11-13 08:19:01 -03:00
Bianca Nenciu e98c7b15d6
FIX: Do not optimize animated images in cooked posts (#11214)
CookedPostProcessor replaces all large images with their optimized
versions, but for GIF images the optimized version is limited to first
frame only. This caused animations it cooked posts to require a click
to show up the lightbox and start playing.
2020-11-12 21:47:30 +02:00
tshenry e1fbd56f6f
UX: Use appropriate logo on static pages (#11211)
Now that we have dark logo settings in core, we can relatively easily ensure that static pages (such as the 404 page) use a logo that is appropriate for the given light or dark color scheme.
2020-11-12 10:50:55 -08:00
Gerhard Schlager 6ff07bb73f
FEATURE: Create revision when bulk moving topics (#10802)
This behavior can be configured with the new "create_revision_on_bulk_topic_moves" site setting. It's enabled by default.
2020-11-12 13:57:12 +01:00
Dan Ungureanu 74a118c13a
FIX: Ensure staff can see /tag/none (#11208)
TopicQueryParams#build_topic_list_options changes params which could
lead to options[:no_tags] and options[:tags] be set simultaneously.
2020-11-12 12:21:53 +11:00
Bianca Nenciu 2910996feb
FIX: Do not optimize uploaded custom emoji (#11203)
Animated emojis were converted to static images. This commit moves the
responsability on site admins to optimize their animated emojis before
uploading them (gifsicle is no longer used).
2020-11-12 11:22:38 +11:00
David Taylor 803b8933fa
DEV: Ensure DiscourseEvent handlers cleaned up during specs (#11205) 2020-11-11 19:46:13 +00:00
David Taylor 255633578c
DEV: Move logout redirect logic to server and add plugin hook (#11199)
This will allow authentication plugins to provide single-logout functionality by redirect users to the identity provider after logout.
2020-11-11 15:47:42 +00:00
Mark VanLandingham be07853cc1
DEV: Add plugins client/server translation yml file priority structure (#11194)
Plugin client.en.yml and server.en.yml can now be client/server-(1-100).en.yml. 1 is the lowest priority, and 100 is the highest priority. This allows plugins to set their priority higher than other plugins, so that they can override eachothers' translations.
2020-11-11 09:44:01 -06:00
Bianca Nenciu a48f7ba61c
FEATURE: Improve errors when title is invalid (#11149)
It used to simply say "title is invalid" without giving any hint what
the problem could be. This commit adds different errors messages for
all caps titles, low entropy titles or titles with very long words.
2020-11-11 15:11:36 +02:00
Dan Ungureanu ab314218d3
FEATURE: Implement edit functionality for post notices (#11140)
All post notice related custom fields were moved to a single one.
2020-11-11 14:49:53 +02:00
Rafael dos Santos Silva cdb7ddc258
DEV: Update MomentJS and highlightJS
We updated version of moment and moment-timezone as our current versions are outdated making Discourse Dates broken on places where timezone had updates, like here in Brazil.

This also update highlightJS to the latest version and corrected a test that relied on a no longer supported locale in
moment.
2020-11-11 12:34:26 +11:00
Arpit Jalan fec9d6e578
FIX: when creating linked topics make sure they belong to same category (#11188)
This PR fixes a bug where the newly created linked topic was being
assigned "uncategorized" category. Now we'll assign linked topics the
same category as that of parent topic.

Meta reference: https://meta.discourse.org/t/linked-topics-splitting-and-managing-megatopics/168992/10?u=techapj
2020-11-11 00:44:27 +05:30
Arpit Jalan 00b41437b0
FIX: hide sso email behind a button click and log views (#11186) 2020-11-11 00:42:44 +05:30
David Taylor a7adf30357
FEATURE: Allow /u/by-external to work for all managed authenticators (#11168)
Previously, `/u/by-external/{id}` would only work for 'Discourse SSO' systems. This commit adds a new 'provider' parameter to the URL: `/u/by-external/{provider}/{id}`

This is compatible with all auth methods which have migrated to the 'ManagedAuthenticator' pattern. That includes all core providers, and also popular plugins such as discourse-oauth2-basic and discourse-openid-connect.

The new route is admin-only, since some authenticators use sensitive information like email addresses as the external id.
2020-11-10 10:41:46 +00:00
David Taylor cf21de0e7a
DEV: Migrate Github authentication to ManagedAuthenticator (#11170)
This commit adds an additional find_user_by_email hook to ManagedAuthenticator so that GitHub login can continue to support secondary email addresses

The github_user_infos table will be dropped in a follow-up commit.

This is the last core authenticator to be migrated to ManagedAuthenticator 🎉
2020-11-10 10:09:15 +00:00
Krzysztof Kotlarek 586c8efbd8
FEATURE: the ability to permanently destroy the private message (#11115)
PostDestroyer should accept the option to permanently destroy post from the database. In addition, when the first post is destroyed it destroys the whole topic.

Currently, that feature is limited to private messages and creator of the post. It will be used by discourse-encrypt to explode encrypted private messages.
2020-11-10 15:40:48 +11:00
Martin Brennan 27e94f2f98
FIX: Make secure image onebox check more robust (#11179)
When embedding secure images which have been oneboxed, we checked to see if the image's parent's parent had the class onebox-body. This was not always effective as if the image does not get resized/optimized then it does not have the aspect-image div wrapping it. This would cause the image to embed in the email but be huge.

This PR changes the check to see if any of the image's ancestors have the class onebox-body, or if the image has the onebox-avatar class to account for variations in HTML structure.
2020-11-10 12:55:18 +10:00
Guo Xiang Tan f70b330e7a DEV: Fix the build.
Follow-up to 650da7b626
2020-11-09 14:25:14 +08:00
Guo Xiang Tan 650da7b626 PERF: Update index for category in a background job.
Search indexing can get expensive and there is no need for us to block
the entire request just to wait for index to finish.
2020-11-09 13:51:26 +08:00
Justin DiRose 09b8a61f65
FEATURE: Add Google Universal Analytics v4 as an option (#11123)
Per Google, sites are encouraged to upgrade from Universal Analytics v3 `analytics.js` to v4 `gtag.js` for Google Analytics tracking. We're giving admins the option to stay on the v3 API or migrate to v4. Admins can change the implementation they're using via the `ga_version` site setting. Eventually Google will deprecate v3, but our implementation gives admins the choice on what to use for now.

We chose this implementation to make the change less error prone, as many site admins are using custom events via the v3 UA API. With the site stetting defaulted to `v3_analytics`, site analytics won't break until the admin is ready to make the migration.

Additionally, in the v4 implementation, we do not enable automatic pageview tracking (on by default in the v4 API). Instead we rely on Discourse's page change API to report pageviews on transition to avoid double-tracking.
2020-11-06 14:15:36 -06:00
Bianca Nenciu 0863c36221
FIX: Improve errors when invite to topic fails (#11133)
It used to simply say "not allowed" without giving any hint what the
problem could be. This commit refactors the code and tries to improve
readability.
2020-11-06 16:58:10 +02:00
Martin Brennan 00c8f520e9
FIX: Do not enable published page if secure media enabled (#11131)
There are issues around displaying images on published pages when secure media is enabled. This PR temporarily makes it appear as if published pages are enabled if secure media is also enabled.
2020-11-06 10:33:19 +10:00
jbrw bba73fc15e
FEATURE: Allow category group moderators to delete topics (#11069)
* FEATURE - allow category group moderators to delete topics

* Allow individual posts to be deleted

* DEV - refactor for new `can_moderate_topic?` method
2020-11-05 12:18:26 -05:00
Arpit Jalan 436bd48512
UX: update topic small action post to add link for new linked topic URL (#11132)
https://meta.discourse.org/t/linked-topics-splitting-and-managing-megatopics/168992/4?u=techapj
2020-11-05 22:39:21 +05:30
Penar Musaraj 9f6c4ad71a
FIX: inconsistency in S3 inventory config (#11112)
Ensures it matches S3 inventory config generation in our hosting.
2020-11-05 08:39:40 -05:00
Sam 2686d14b9a
PERF: introduce aggressive rate limiting for anonymous (#11129)
Previous to this change our anonymous rate limits acted as a throttle.
New implementation means we now also consider rate limited requests towards
the limit.

This means that if an anonymous user is hammering the server it will not be
able to get any requests through until it subsides with traffic.
2020-11-05 16:36:17 +11:00
tshenry d778d99b55
FIX: limit_topics_per_day should only apply to regular topics (#11127)
PMs were being evaluted by both the limit_topics_per_day and limit_private_messages_per_day rate limiters when it should only be the latter.
2020-11-04 16:23:49 -08:00
Penar Musaraj c1f3bd6a1c
FIX: secure_media stripping on lightboxes, non-image links (#11121)
- Fixes stripping of lightboxes with empty srcset attribute
- Does not fail when email has links with secure media URLs but no child image elements
2020-11-04 15:45:50 -05:00
jbrw d85fc96193
DEV - Test for an integer padded with spaces (#11111) 2020-11-04 11:05:31 -05:00
Arpit Jalan 24d1b1f159
UX: when creating linked topic use URL and let onebox work its magic (#11118)
When the linked topic is created we'll not hardcode the topic title and
let onebox work its magic instead so that the title can be updated
automatically.
2020-11-04 17:19:09 +05:30
Rafael dos Santos Silva b70b3e867c
FIX: Safely skip secure_media steps when it's not enabled (#11110)
* FIX: Safely skip secure_media steps when it's not enabled

* DEV: Secure media tests should enable secure media
2020-11-03 13:53:15 -03:00
Bianca Nenciu d2116f0029
FIX: Show error message if user is already silenced or suspended (#10988)
Users could be silenced or suspended by two staff members at the same time and
would not be aware of it. This commit shows an error message if another penalty
has been applied.
2020-11-03 17:38:56 +02:00
Dan Ungureanu 3c51647872
DEV: Remove legacy tag and category routes (#10338)
* DEV: Remove client-side legacy tag and category routes

* DEV: Remove server-side legacy tag routes

* DEV: Refactor ListController#set_category

* FIX: Remove reference to discovery.parentCategory

* FIX: Refactor TagsController#set_category_from_params

* FIX: Build correct canonical URL for tags and categories

* DEV: Fix deprecation notice in Ruby 2.7

* DEV: Replace use of removed legacy tag route

* DEV: Add deprecation notices for old routes and controllers
2020-11-03 16:57:58 +02:00
David Taylor 5140ec9acf
DEV: Cleanup ignored user logic (#11107)
- IgnoredUser records should all now have an expiring_at value. This commit enforces that in the DB, and fixes any corrupt rows
- Changes to the ignored user list are now handled by the `/u/{username}/notification_level` endpoint. This allows setting expiration dates on the ignore. This commit removes the old logic for saving a list of usernames in the user preferences.
- Many specs were calling `IgnoredUser.create`. This commit changes them to use `Fabricate(:ignored_user)` for consistency
2020-11-03 12:38:54 +00:00
Kane York 789e3775df
FIX: Make all email subject vars available in notification subjects (#11064)
A site owner attempting to use both the email_subject site setting and translation overrides for normal post notification
email subjects would find themselves frusturated at the lack of template argument parity.
Make all the variables available for translation overrides by adding the subject variables to the custom interpolation keys list and applying them.

Reported at https://meta.discourse.org/t/customize-subject-format-for-standard-emails/20801/47?u=riking
2020-11-02 20:00:11 -08:00
jbrw 9ff7f25106
DEV - handle malformed `page` param (#11093)
* DEV - handle malformed page params
2020-11-02 12:40:54 -05:00
Arpit Jalan 1476e17c35
FEATURE: new setting to create a linked topic on autoclosing mega topics (#11001)
This commit adds a site setting `auto_close_topics_create_linked_topic`
which when enabled works in conjunction with `auto_close_topics_post_count`
setting and creates a new linked topic for the topic just closed.

The auto-created new topic contains a link for all the previous topics
and the topic titles are appended with `(Part {n})`.

The setting is enabled by default.
2020-11-02 12:18:48 +05:30
Martin Brennan 3655062c60
FIX: Ensure oneboxed secure images which are optimized and also lightboxed optimized images are embedded in email (#11061)
We had an issue where onebox thumbnail was too large and thus was optimized, and we are using the image URLs in post to redact and re-embed, based on the sha1 in the URL. Optimized image URLs have extra stuff on the end like _99x99 so we were not parsing out the sha1 correctly. Another issue I found was for posts that have giant images, the original was being used to embed in the email and thus would basically never get included because it is huge.

For example the URL 787b17ea61_2_690x335.jpeg was not parsed correctly; we would end up with 787b17ea6140f4f022eb7f1509a692f2873cfe35_2_690x335.jpeg as the sha1 which would not find the image to re-embed that was already attached to the email.

This fix will use the first optimized image of the detected upload when we are redacting and then re-embedding to make sure we are not sending giant things in email. Also, I detect if it is a onebox thumbnail or the site icon and force appropriate sizes and styles.
2020-11-02 09:52:21 +10:00
Vinoth Kannan af4938baf1
Revert "DEV: enable cors to all cdn get requests from workbox. (#10684)" (#11076)
This reverts commit e3de45359f.

We need to improve out strategy by adding a cache breaker with this change ... some assets on CDNs and clients may have incorrect CORS headers which can cause stuff to break.
2020-10-30 16:05:35 +11:00
Vinoth Kannan 347423007a
DEV: remove instagram login site settings and auth classes. (#11073)
Instagram removed the support for login and should use Facebook login instead.
2020-10-30 09:09:56 +05:30
Vinoth Kannan 72810853ea
FIX: strip the trailing slash (/) of cors origins. (#10996)
Strips trailing `/` from global settings
Provides a validation for site settings to ensure a trailing `/` is not added
2020-10-29 13:01:06 +11:00
Roman Rizzi 2f32336081
FIX: Staff can create and edit posts even if a topic is in slow mode. (#11057)
Additionally, ninja edits are no longer restricted.
2020-10-28 16:47:50 -03:00
Vinoth Kannan e3de45359f
DEV: enable cors to all cdn get requests from workbox. (#10685)
Now all external requests from the service worker will be in CORS mode without credentials.
2020-10-28 23:36:19 +05:30
Martin Brennan 632942e697
FIX: Ensure group SMTP and message builder always uses from address for Reply-To when IMAP is enabled (#11037)
There is a site setting reply_by_email_enabled which when combined with reply_by_email_address creates a Reply-To header in emails in the format "test+%{reply_key}@test.com" along with a PostReplyKey record, so when replying Discourse knows where to route the reply.

However this conflicts with the IMAP implementation. Since we are sending the email for a group via SMTP and from their actual email account, we want all replys to go to that email account as well so the IMAP sync job can pick them up and put them in the correct place. So if the group has IMAP enabled and configured, then the reply-to header will be correct.

This PR also makes a further fix to 64b0b50 by using the correct recipient user for the PostReplyKey record. If the post user is used we encounter this error:

if destination.user_id != user.id && !forwarded_reply_key?(destination, user)
  raise ReplyUserNotMatchingError, "post_reply_key.user_id => #{destination.user_id.inspect}, user.id => #{user.id.inspect}"
end
This is because the user above is found from the from_address, but the destination which is the PostReplyKey is made by the post.user, which will be different people.
2020-10-28 07:01:58 +10:00
jbrw 586dd064c6
FIX - don't hide tags if user has correct permissions (#11046) 2020-10-27 14:17:13 -04:00
Kane York e35fcd3340
FEATURE: Include rejected queued posts in the user archive export (#10859)
Requested at https://meta.discourse.org/t/where-can-a-user-find-the-post-that-was-rejected-by-the-moderator/165671?u=riking

Field whitelisting is applied to the json field using Hash#slice, which was activesupport until Ruby 2.5.
2020-10-27 07:48:48 -07:00
Krzysztof Kotlarek 8253f8fc5c
PERF: we don't need to use a huge image to test thumbnails (#11025)
* PERF: we don't need to use a huge image to test thumbnails

Generating images with 5000x5000 dimensions is an expensive operation.

Using smaller images reduce the time of model spec from 11s to 3s and integration spec from 6s to 2s.
2020-10-27 12:39:52 +11:00
jbrw aeb24bd4b5
FIX - don't attempt to optimized animated images (#11031)
* FIX - don't attempt to optimized animated images

* ensure_safe_paths before calling ImageMagick
2020-10-26 15:10:19 -04:00
Blake Erickson c9515ede2e
DEV: Add api docs for invite and group update endpoint (#11019)
Documenting a few more endpoints so that our api docs can be
automatically generated. Made a couple other minor changes, like
including the "OK" example for our default success response.
2020-10-26 08:55:22 -06:00
Dan Ungureanu 43557143fe
FIX: Reset invite when resending it (#11013)
Resending an invite moved the expire date in the future, but did not
invalidate it. For example, if an invite was sent to an email,
invalidated and then resent, it would still be left invalidated.
2020-10-26 12:26:43 +02:00
Arpit Jalan 9aa2723d57
UX: improve error message for already logged in users (#11020) 2020-10-24 21:21:01 +05:30
jbrw 2bcca46cc5
FEATURE - ImageMagick jpeg quality (#11004)
* FEATURE - Add SiteSettings to control JPEG image quality

`recompress_original_jpg_quality` - the maximum quality of a newly
uploaded file.

`image_preview_jpg_quality` - the maximum quality of OptimizedImages
2020-10-23 12:38:28 -04:00
jbrw ce76553010
FEATURE: allow category group moderators to edit posts (#11005)
* FEATURE: allow category group moderators to edit posts

If the `enable_category_group_moderation` SiteSetting is enabled, posts should be editable by those belonging to the appropraite groups.
2020-10-23 12:37:44 -04:00
Roman Rizzi c0848a5cc4
FIX: Only include last_posted_at if there's a topic_user object. (#11011)
Trying to include this attribute when topic_user is nil causes an error when visiting a topic as anon. Additionally, we don't display the slow mode banner for these users.
2020-10-23 11:31:59 -03:00
Guo Xiang Tan 68fc2a18b1 FIX: Properly handle quotes and backslash in `Search.set_tsquery_weight_filter` 2020-10-23 08:43:34 +08:00
Jahan Gagan cce03de616 EDIT: specs and translation updated 2020-10-22 10:42:17 -04:00
Jahan Gagan fab67fafc1 FIX: invite users with sufficient trust level 2020-10-22 10:42:17 -04:00
Krzysztof Kotlarek b2481adb40
FIX: persist secure image width and height if is given (#10994)
`max-width: 50%; max-height: 400px;` is a good fallback, however, if width and height are given and are smaller than fallback -  we should persist that smaller size.
2020-10-22 13:25:09 +11:00
Martin Brennan 64b0b50ac0
FIX: Pass user to Email::Sender to avoid broken reply key for group_smtp email (#10978)
Our Email::Sender class accepts an optional user argument, which is used to create a PostReplyKey record when present. This record is used to sub out the %{reply_key} placeholder in the Reply-To mail header, so if we do not pass in the user we get a broken Reply-To header.

This is especially problematic in the IMAP group SMTP situation, because these emails go to customers that we are replying to, and when they reply to us the email bounces! This fixes the issue by passing user to the Email::Sender when sending a group_smtp email but there is still more to do in another PR.

This Email::Sender optional user is a bit of a footgun IMO, especially because most of the time we use it there is a user we can source. I would like to do another PR for this after this one to make the parameter not optional, so we don't end up with these reply issues down the line again.
2020-10-22 10:49:08 +10:00
Angus McLeod 516e7e392b
FEATURE: Add non_automatic group type param to groups index API 2020-10-21 09:46:45 +11:00
Bianca Nenciu be5efc9410
FIX: Ensure old uploads can have animated field updated (#10963)
If admins decreased the maximum filesize limit the ActiveRecord
validations would fail.
2020-10-20 19:11:43 +03:00
jbrw a74805d3f8
FIX: moderators can add/remove group owners (#10960)
If `SiteSetting.moderators_manage_categories_and_groups` is enabled, a moderator shoud be able to add/remove group owners.
2020-10-19 16:30:21 -04:00
David Taylor b7c680853d DEV: Introduce plugin API to contribute user api key scopes 2020-10-19 10:40:55 +01:00
David Taylor 23e5c605f6 DEV: Add support for allowed parameters in user api key scopes
Initially, this feature is only intended for use in core/plugins, so there is no API for requesting a parameter-scoped key. That may change in future.
2020-10-19 10:40:55 +01:00
David Taylor 1cec333f48 REFACTOR: Introduce RouteMatcher class
This consolidates logic used to match routes in ApiKey, UserApiKey and DefaultCurrentUserProvider. This reduces duplicated logic, and will allow UserApiKeysScope to easily re-use the parameter matching logic from ApiKeyScope
2020-10-19 10:40:55 +01:00
Krzysztof Kotlarek 2ad4fc39b6
FIX: calculate page if page param is not given to TopicView (#10953)
Currently, when page param is not given to TopicView we calculate page for canonical_path, however, it is skipped for next_path.

We should use the same calculation to define page, so next page URL will be accurate. Currently if you [view source of meta post](view-source:https://meta.discourse.org/t/post-rate-limit-trigger-for-a-topic-thats-heating-up/98294/46) you will see:

```
<link rel="canonical" href="https://meta.discourse.org/t/post-rate-limit-trigger-for-a-topic-thats-heating-up/98294?page=3" />
<link rel="next" href="/t/post-rate-limit-trigger-for-a-topic-thats-heating-up/98294?page=2">
```
2020-10-19 17:11:49 +11:00
Gerhard Schlager 918d1d8363
DEV: Fix heisentest (#10946)
This should fix the following sporadic spec failure:

```
  1) JsLocaleHelper performs fallbacks to English if a translation is not available
     Failure/Error: expect(ctx.eval('I18n.translations.uk.js').keys).to contain_exactly("all_three", "english_and_user", "only_user", "site_and_user")

       expected collection contained:  ["all_three", "english_and_user", "only_user", "site_and_user"]
       actual collection contained:    ["about", "action_codes", "activity", "admin", "admin_title", "adplugin", "age", "akismet", "all_time..."voting", "week", "week_desc", "weekly", "wizard_required", "year", "year_desc", "yes_value", "you"]
       the missing elements were:      ["all_three", "english_and_user", "only_user", "site_and_user"]
       the extra elements were:        ["about", "action_codes", "activity", "admin", "admin_title", "adplugin", "age", "akismet", "all_time..."voting", "week", "week_desc", "weekly", "wizard_required", "year", "year_desc", "yes_value", "you"]
     # ./spec/components/js_locale_helper_spec.rb:182:in `block (2 levels) in <main>'
     # ./bundle/ruby/2.6.0/gems/webmock-3.9.2/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
```
2020-10-18 12:00:35 +02:00
Roman Rizzi 21c53ed249
FEATURE: Topic slow mode. (#10904)
Adds a new slow mode for topics that are heating up. Users will have to wait for a period of time before being able to post again.

We store this interval inside the topics table and track the last time a user posted using the last_posted_at datetime in the TopicUser relation.
2020-10-16 16:24:38 -03:00
Gerhard Schlager d5ef6188ed
PERF: Disable Sidekiq only during database restore (#10857)
It pauses Sidekiq, clears Redis (namespaced to the current site), clears Sidekiq jobs for the current site, restores the database and unpauses Sidekiq. Previously it stayed paused until the end of the restore.

Redis is cleared because we don't want any old data lying around (e.g. old Sidekiq jobs). Most data in Redis is prefixed with the name of the multisite, but Sidekiq jobs in a multisite are all stored in the same keys. So, deleting those jobs requires a little bit more logic.
2020-10-16 15:19:02 +02:00
Bianca Nenciu 43e52a7dc1
DEV: Remove gifsicle dependency (#10357)
Dependency on gifsicle, allow_animated_avatars and allow_animated_thumbnails
site settings were all removed. Animated GIF images are still allowed, but
the generated optimized images are no longer animated for those (which were
used for avatars and thumbnails).

The added 'animated' is populated by extracting information using FastImage.
This field was used to selectively reoptimize old animations. This process
happens in the background.
2020-10-16 13:41:27 +03:00
Penar Musaraj 5763309953
FEATURE: WCAG compliant color schemes (#10882)
Co-authored-by: Kris <kris.aubuchon@discourse.org>
2020-10-15 14:05:48 -04:00
jbrw 099bf97dca
Tag groups can belong to groups (#10854) 2020-10-14 13:15:54 -04:00
Penar Musaraj 74de7a49f5
Simplify theme and color scheme seeding (#10872)
Now that we have support for user-selectable color schemes, it makes sense
to simplify seeding and theme updates in the wizard. 

We now:

- seed only one theme, named "Default" (previously "Light")
- seed a user-selectable Dark color scheme
- rename the "Themes" wizard step to "Colors"
- update the default theme's color scheme if a default is set
(a new theme is created if there is no default)
2020-10-14 10:18:02 -04:00
Martin Brennan c3cede697d
FEATURE: Add weekly bookmark cleanup code (#10899)
When posts or topics are deleted we don't want to immediately delete associated bookmarks, so we have a grace period to recover them and their reminders if the post or topic is un-deleted. This PR adds a task to the Weekly scheduled job to go and delete bookmarks attached to posts or topics deleted > 3 days ago.
2020-10-14 09:38:57 +10:00
Gerhard Schlager 1febf11362 FIX: Backup didn't work anymore after a running backup was canceled 2020-10-13 19:48:53 +02:00
Justin DiRose 8c77b84aac
Revert "FEATURE: Upgrade analytics.js to gtag.js (#10893)" (#10910)
Reverting due to a few unforseen issues with customizations.
2020-10-13 12:20:41 -05:00
Justin DiRose f4034226c2
FEATURE: Upgrade analytics.js to gtag.js (#10893)
Per Google, sites are encouraged to upgrade from `analytics.js` to `gtag.js` for Google Analytics tracking. This commit updates core Discourse to use the new `gtag.js` API Google is asking sites to use. This API has feature parity with `analytics.js` but does not use trackers.
2020-10-13 11:24:06 -05:00
Joffrey JAFFEUX e54c8a998b
Revert "DEV: makes babel configuration consistent in all cases (#10884)" (#10907)
This reverts commit c39dc9157c.
2020-10-13 15:58:08 +02:00
Joffrey JAFFEUX c39dc9157c
DEV: makes babel configuration consistent in all cases (#10884)
Creates a BabelHelper builder using a default list of plugins, to ensure the transpiled code is always using the same plugins instead of differents plugins in different cases.
2020-10-13 15:33:29 +02:00
Bianca Nenciu 25b8ed740b
DEV: Make site setting type uploaded_image_list use upload IDs (#10401)
It used to be a list of concatenated upload URLs which was prone to
break.
2020-10-13 16:17:06 +03:00
Sam a6d9adf346
DEV: ensure queue_time and background_requests are floats (#10901)
GlobalSetting can end up with a String and we expect a Float
2020-10-13 18:08:38 +11:00
Sam 32393f72b1
PERF: backoff background requests when overloaded (#10888)
When the server gets overloaded and lots of requests start queuing server
will attempt to shed load by returning 429 errors on background requests.

The client can flag a request as background by setting the header:
`Discourse-Background` to `true`

Out-of-the-box we shed load when the queue time goes above 0.5 seconds.

The only request we shed at the moment is the request to load up a new post
when someone posts to a topic.

We can extend this as we go with a more general pattern on the client.

Previous to this change, rate limiting would "break" the post stream which
would make suggested topics vanish and users would have to scroll the page
to see more posts in the topic.

Server needs this protection for cases where tons of clients are navigated
to a topic and a new post is made. This can lead to a self inflicted denial
of service if enough clients are viewing the topic.

Due to the internal security design of Discourse it is hard for a large
number of clients to share a channel where we would pass the full post body
via the message bus.

It also renames (and deprecates) triggerNewPostInStream to triggerNewPostsInStream

This allows us to load a batch of new posts cleanly, so the controller can
keep track of a backlog

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-10-13 16:56:03 +11:00
Krzysztof Kotlarek 6be60b0ae5
FEATURE: respect tags_sort_alphabetically setting when display tags (#10889)
Currently, tag labels are displayed in random order.

They should be displayed in alphabetical or popularity order based on SiteSetting (tags_sort_alphabetically)

Meta: https://meta.discourse.org/t/how-to-apply-tag-sorts-by-popularity-to-topic-list-currently-it-seems-only-apply-to-tag-page/163186/7
2020-10-13 08:23:04 +11:00
Jarek Radosz acf5a26058
DEV: Fix an apparently "too modern" git command (#10894)
A follow-up to a follow-up. (6932a373a3 and 572da7a57b)
Our `discourse_test` Docker image uses git 2.20.1 released on Dec 15, 2018. It does not support `git branch --show-current`. (it was added in 2.22.0)
2020-10-12 22:54:56 +02:00
Jarek Radosz 572da7a57b
DEV: Fix a spec incompatibility with pre-2.28 git (#10892)
Regression introduced in 6932a373a3
2020-10-12 19:59:54 +02:00
Jarek Radosz 6932a373a3
FIX: Handle .discourse-compatibility syntax errors (#10891)
Previously, any errors in those files would e.g. blow up the update process in docker_manager.
Now it prints out an error and proceeds as if there was no compatibility file.

Includes:

* DEV: Extract setup_git_repo
* DEV: Use `Dir.mktmpdir`
* DEV: Default to `main` branch (The latest versions of git already do this, so to avoid problems do this by default)
2020-10-12 18:25:06 +02:00
jbrw ac31fe8321
FEATURE - SiteSetting to disable user option to hide their profiles and presences (#10885)
* FEATURE - SiteSetting to disable user option to hide their profiles and presences
2020-10-09 17:18:44 -04:00
Arpit Jalan 4326827a4e
FIX: second factor cannot be enabled if SSO is enabled (#10880)
* FIX: second factor cannot be enabled if SSO is enabled

If `enable_sso` setting is enabled then admin should not be able to
enable `enforce_second_factor` setting as that will lock users out.

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-10-09 22:36:38 +05:30
Daniel Waterworth 721ee36425
Replace `base_uri` with `base_path` (#10879)
DEV: Replace instances of Discourse.base_uri with Discourse.base_path

This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
Penar Musaraj 5130b4d674
Ignore disabled out-of-date remote themes (#10870)
No need to alert in admin dashboard about out of date remote themes
that are disabled.
2020-10-08 13:48:16 -04:00
Martin Brennan f63da1c146
FIX: Confirm new email not sent for staff if email disabled with "non-staff" option (#10794)
See https://meta.discourse.org/t/email-address-change-confirmation-email-not-sent-but-every-other-notification-emails-are/165358

In short: with disable emails set to non-staff, email address change confirmation emails (those sent to the new address) are not sent for staff or admin members.

This was happening because we were looking up the staff user with the to_address of the email, but the to address was the new email address because we are sending a confirm email change email, and thus the user could not be found. We didn't need to do this anyway because we are passing the user into the Email::Sender class anyway.
2020-10-08 13:52:17 +10:00
Sam 3c678df942
PERF: avoid lookbehinds when indexing search (#10862)
* PERF: avoid lookbehinds when indexing search

Previously we used a `EmailCook.url_regexp` this regex used lookbehinds

Unfortunately certain strings could lead to pathological behavior causing
CPU to skyrocket and regex replace to take a very very long time.

EmailCook still needs a fix, but it is less urgent cause it already splits
to single lines. That said we will correct that as well in a seperate PR.

New implementation is far more naive and relies on the extra spaces search
indexer inserts.
2020-10-08 11:40:13 +11:00
jbrw a1918801a4
FIX - downcase strings consistently (#10861)
ruby and postgres can treat certain characters differently when downcasing them. So do all the downcasing in ruby so that we get consistent results.
2020-10-08 11:28:07 +11:00
Penar Musaraj ddd6c990f6
FIX: Respect show_category_definitions_in_topic_lists in category lists (#10853)
When that site setting is enabled, the category counts (new/unread)
include the subcategory definition topics, but the topics aren't included
in the list. This fixes that discrepancy.
2020-10-07 14:19:48 -04:00
Penar Musaraj 2ad7d98990
FIX: Include topics from subcategories in tracked list (#10850) 2020-10-07 12:15:28 -04:00
David Taylor c0293339b8
PERF: Do not enqueue digest emails when attempted recently (#10849)
Previously, Jobs::EnqueueDigestEmails would enqueue a digest job for every user, even if there are no topics to send. The digest job would exit, no email would send, and last_emailed_at would not change. 30 minutes later, Jobs::EnqueueDigestEmails would run again and re-enqueue jobs for the same users.

120fa8ad introduced a temporary mitigation for this issue, by randomly selecting a subset of those users each time.

This commit adds a new `digest_attempted_at` column to the `user_stats` table. This column is updated every time a digest job completes for a user. Using this, we can avoid scheduling digest jobs for the same user every 30 minutes. This also removes the random user selection in 120fa8ad, and instead prioritizes users who had digests attempted the longest time ago.
2020-10-07 15:30:38 +01:00
Sam 120fa8ad2f
PERF: Introduce absolute limit of digests per 30 minutes (#10845)
To avoid blocking the sidekiq queue a limit of 10,000 digests per 30 minutes
is introduced.

This acts as a safety measure that makes sure we don't keep pouring oil on
a fire.

On multisites it is recommended to set the number way lower so sites do not
dominate the backlog. A reasonable default for multisites may be 100-500.

This can be controlled with the environment var

DISCOURSE_MAX_DIGESTS_ENQUEUED_PER_30_MINS_PER_SITE
2020-10-07 17:30:15 +11:00
Martin Brennan 6e2be3e60b
FIX: When admin changes an email for the user the user must confirm the change (#10830)
See https://meta.discourse.org/t/changing-a-users-email/164512 for additional context.

Previously when an admin user changed a user's email we assumed that they would need a password reset too because they likely did not have access to their account. This proved to be incorrect, as there are other reasons a user needs admin to change their email. This PR:

* Changes the admin change email for user flow so the user is sent an email to confirm the change
* We now record who the email change request was requested by
* If the requested by user is admin and not the user we note this in the email sent to the user
* We also make the confirm change email route open to anonymous users, so it can be clicked by the user even if they do not have access to their account. If there is a logged in user we make sure the confirmation matches the current user.
2020-10-07 13:02:24 +10:00
Kane York 68e87bb58e
User export: profile as json, export auth token logs (#10819)
* FEATURE: Export the entire user profile as json, not just bio/website

* FEATURE: Add session log information to user export

Even though the columns are named 'auth_token' etc, the content is not actually usable to log into the forum with. Despite all that, it is still truncated for export, to avoid any 'token hash cracking' situations.
2020-10-06 15:51:53 -07:00
Roman Rizzi a8c47e7c7f
SECURITY: Ensure users can see the topic before setting a topic timer. (#10841) 2020-10-06 16:49:06 -03:00
Arpit Jalan f4c7c7bff3
FEATURE: bulk remove tags (#10831)
https://meta.discourse.org/t/bulk-actions-remove-tags-in-addition-to-change-tags/52145
2020-10-06 22:41:55 +05:30
Robin Ward 00afd308c1 FIX: Escaped `mailto` URLs would raise an exception
This prevents exceptions from being raised if a URL has an invalid
component.
2020-10-05 14:42:44 -04:00
Penar Musaraj a4356b99af
FEATURE: Separate base and heading font site_settings (#10807)
Allows site administrators to pick different fonts for headings in the wizard and in their site settings. Also correctly displays the header logos in wizard previews.
2020-10-05 13:40:41 -04:00
Bianca Nenciu 318efa8093
FIX: Always let moderators to see their group PMs (#10813)
When enable_personal_messages was disabled, moderators could not see
the private messages for the "moderators" group. The link was displayed
on the client side, but the checks on the server side did not allow it.
2020-10-05 19:38:08 +03:00
Robin Ward 80a5482f28 Embedded topics are now unlisted by default
Previously this site setting `embed unlisted` defaulted to false and
empty topics would be generated for embed, but those topics tend to take
up a lot of room on the topic lists.

This new default creates invisible topics by default until they receive
their first reply.
2020-10-05 12:09:20 -04:00
Jarek Radosz cf44cdb082
FIX: Validate category name on parent change (#10815)
Previously, moving a category into another one, that already had a child category of that name (but with a non-conflicting slug) would cause a 500 error:

```
# PG::UniqueViolation:
#   ERROR:  duplicate key value violates unique constraint "unique_index_categories_on_name"
#   DETAIL:  Key (COALESCE(parent_category_id, '-1'::integer), name)=(5662, Amazing Category 0) already exists.
```

It now returns 422, and shows the same message as when you're renaming a category: "Category Name has already been taken".
2020-10-05 11:50:05 +02:00
Arpit Jalan 9b45391cf9
FEATURE: explain why invites are disabled to staff users (#10810)
Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-10-03 13:05:26 +05:30
Gerhard Schlager 99181bb3b8
FIX: Stripping lines from incoming email shouldn't fail for blank body (#10800) 2020-10-02 15:44:35 +02:00
Krzysztof Kotlarek 5cf411c3ae
FIX: move hp request from /users to /token (#10795)
`hp` is a valid username and we should not prevent users from registering it.
2020-10-02 09:01:40 +10:00
Jarek Radosz 891987a284
DEV: Recover missing files of existing uploads (#10757)
UploadRecovery only worked on missing Upload records. Now it also works with existing ones that have an invalid_etag status.

The specs (first that test the S3 path) are a bit of stub-a-palooza, but that's how much this class interacts with the the outside world. 🤷‍♂️
2020-10-01 14:54:45 +02:00
Gerhard Schlager f5c4594f6d DEV: Fix heisentest 2020-09-30 17:20:56 +02:00
Arpit Jalan ff44515a18
FIX: better error message if invite is expired (#10783)
https://meta.discourse.org/t/invite-token-is-invalid/165270/5?u=techapj
2020-09-30 20:32:33 +05:30
Martin Brennan 39b2fb8649
FIX: Invalid URLs could raise exceptions when calling UrlHelper.rails_route_from_url (#10782)
Upload.secure_media_url? raised an exceptions when the URL was invalid,
which was a issue in some situations where secure media URLs must be
removed.

For example, sending digests used PrettyText.strip_secure_media,
which used Upload.secure_media_url? to replace secure media with
placeholders. If the URL was invalid, then an exception would be raised
and left unhandled.

Now instead in UrlHelper.rails_route_from_url we return nil if there is something wrong with the URL.

Co-authored-by: Bianca Nenciu <nenciu.bianca@gmail.com>
2020-09-30 15:20:00 +10:00
Gerhard Schlager 9d4009b0e8
FIX: Use correct locale for error messages (#10776)
Error messages for exceeded rate limits and invalid parameters always used the English locale instead of the default locale or the current user's locale.
2020-09-29 21:42:45 +02:00
Arpit Jalan 025490ecb9
FEATURE: composer option to reload page and force save draft (#10773) 2020-09-29 22:29:03 +05:30
David Taylor 1ba9b34b03
DEV: Move UserApiKey scopes to dedicated table (#10704)
This has no functional impact yet, but it is the first step in adding more granular scopes to UserApiKeys
2020-09-29 10:57:48 +01:00
Martin Brennan a8ed0b4612
FIX: Correct corrupt encoding in emails containing attachments 2020-09-29 14:10:57 +10:00
Martin Brennan 4193eb0419
FIX: Respect force download when downloading secure media via lightbox (#10769)
The download link on the lightbox for images was not downloading the image if the upload was marked secure, because the code in the upload controller route was not respecting the dl=1 param for force download.

This PR fixes this so the download link works for secure images as well as regular ligthboxed images.
2020-09-29 12:12:03 +10:00
Martin Brennan 3cd601dcc9
FIX: Admin change email for user process improvements and fixes (#10755)
See https://meta.discourse.org/t/changing-a-users-email/164512 for context.

When admin changes an email for a user, we were incorrectly sending the password reset email to the user's old address. Also the new email does not come into effect until the reset password process is done, so this PR adds some notes to the admin to make this clearer.
2020-09-29 09:45:45 +10:00
Arpit Jalan f7940b1d20
FEATURE: advanced search option for max posts count (#10761)
This commit adds an option to search for max posts count and updates
the UI for posts count search to show a min/max range in single line.
2020-09-28 21:34:16 +05:30
Joffrey JAFFEUX cad223c012
DEV: fix broken backend test due to #10749 (#10758) 2020-09-28 14:05:01 +02:00
Jarek Radosz e00abbe1b7 DEV: Clean up S3 specs, stubs, and helpers
Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-28 12:02:25 +01:00
Arpit Jalan 4498c59085 FEATURE: add alias for min_post_count search filter 2020-09-28 16:07:44 +05:30
Krzysztof Kotlarek e7c72cd1e4
FIX: deprecate whitelist constants (#10716)
Deprecation of:
WHITELISTED_REDIRECT_HOSTNAMES
CUSTOM_INTERPOLATION_KEYS_WHITELIST
WHITELISTED_SVG_ELEMENTS
2020-09-28 13:52:05 +10:00
Mark VanLandingham b8015ab654
FIX: Dismiss unread respects tracked query param (#10714)
* WIP:  'dismiss...' respectes tracked query param

* Address review comments

* Dismiss new respects query params

* Remove comment

* Better variable name

* remove self
2020-09-25 12:39:37 -07:00
Justin DiRose 82a18f125f
FIX: Don't skip the new user badge (#10743)
The NewUserOfTheMonth badge is part of the Badges::GettingStarted group. This group is skipped in BadgeGranter if the user skips the new user tips. However, the NewUserOfTheMonth badge granter job does not account for this. Instead, it notifies the user they've received the badge even if they did not.

This commit introduces a simple fix to allow granting of this badge even to users who skipped the new user tips.
2020-09-24 12:58:31 -05:00
David Taylor f1d64bbbe5
FEATURE: Add a site setting to control automatic auth redirect (#10732)
This allows administrators to stop automatic redirect to an external authenticator. It only takes effect when there is a single authentication method, and the site is login_required
2020-09-24 17:06:07 +01:00
Arpit Jalan cdf45f4fe6 Update regex for views search filter. 2020-09-24 17:05:55 +05:30
Arpit Jalan 0c5cd0d1ef FEATURE: advanced search filters for view count 2020-09-24 15:22:18 +05:30
Guo Xiang Tan 911d47934a
DEV: Prefer `update!` over `update` when return value is not checked. 2020-09-24 09:56:09 +08:00
Robin Ward 1976306539 Add extra locales to bootstrap.json
This allows an app (such as Ember CLI) to get the full list of locales
for a user, including admin and overrides.
2020-09-23 14:48:52 -04:00
Arpit Jalan 3684337e4a FIX: email always settings were not being respected
https://meta.discourse.org/t/email-notification-for-messages/163937/10?u=techapj
2020-09-23 22:00:15 +05:30
Bianca Nenciu 214b4c3910
FIX: Remove category id from category class name (#10712)
The generated class name included ID (parent_slug-child_slug-child_id),
but the client side did not expect it (parent_slug-child_slug).
2020-09-23 19:22:07 +03:00
Mark VanLandingham 9f73e8779d
FIX: Only staff can edit topic details when first post is locked (#10729) 2020-09-23 11:13:18 -05:00
Bianca Nenciu 4abbe3d361
FEATURE: Make search filters case insensitive (#10715) 2020-09-23 11:59:42 +03:00
Krzysztof Kotlarek 0bb51dcbfa
FIX: TL2 promotion message and advance training (#10679)
This is a little bit of refactoring. Core Discourse should have default promotion message for TL2.

In addition, when the Discobot plugin is enabled, the user is invited to advanced training
2020-09-22 10:17:52 +10:00
Krzysztof Kotlarek c934a0f759
FIX: don't error Topic#similar_to when prepared raw is blank (#10711)
If raw contains incorrect URL, `prepare_data` returns empty string:

https://github.com/discourse/discourse/blob/master/lib/search.rb#L91

Therefore we should not only check if the cooked post is not blank but also if prepared data is not blank.
2020-09-22 07:53:12 +10:00
Penar Musaraj f7ca93b2fa
FIX: Do not downcase group name in notification payload
Followup to 3d39b4bb
2020-09-21 13:33:29 -04:00
Penar Musaraj 577293c438
FIX: respect moderator group permissions in guardian (#10713)
Since 9e4ed03, moderators can view groups with visibility level set to "Group owners, members and moderators".

This fixes an issue where moderators can see the group in /g but then get a 404 when clicking on individual groups.
2020-09-21 12:32:43 -04:00
Vinoth Kannan 6b818fb855 FIX: use normal logo in published pages if small not available.
It was returning an error if small logo is not available. If both logos are not available then it will be hidden now.
2020-09-21 09:20:39 +05:30
Sam Saffron cba5baa427
FIX: compensate for typographer
typographer can change " to ” leading to breakages in parser

At least codify this. Longer term we want to re-prioritize typographer so
it always runs after bbcode parsing.
2020-09-21 09:44:37 +10:00
Sam Saffron a91ee45de9
FIX: Nested quotes in BBCode
Previously attributes such as `[test a='a"a' b="a'a"]` were not correctly
handled.

This amends the regex parser to ensure it correctly parses attributes
without breaking incorrectly on the first nested quote
2020-09-21 08:56:25 +10:00
Penar Musaraj c664ba8fa8
DEV: Fix flaky UploadSecurity spec 2020-09-18 14:36:41 -04:00
Roman Rizzi f85f73be88
FEATURE: Review posts with media. (#10693)
To check if a post contains any embedded media, we look if the "image_sizes" attribute is present in the new post manager arguments.

We want to see one boxed links, but we only store the raw content of the post. To work around this, I extracted the onebox logic from the composer editor into a module.
2020-09-18 12:45:09 -03:00
Martin Brennan 14b324e5ed
FIX: Provide better API for registering custom upload public types (#10697)
With secure media and the UploadSecurity class, we need a nice way for plugins to register custom upload types that should be considered public and never secure.
2020-09-18 11:54:33 +10:00
Penar Musaraj 3d39b4bbb5
FIX: Do not downcase group name in current user serializer 2020-09-17 13:03:42 -04:00
Guo Xiang Tan b47b640598
FEATURE: Hidden `SiteSetting.keep_old_ip_address_count` to track IP history. 2020-09-17 12:50:39 +08:00
Martin Brennan 49bd066037
FEATURE: Default allow embed secure images in email to true (#10688)
We are making the changes from the PR #10563 the default behaviour. Now, if secure media is enabled, secure images will be embedded in emails by default instead of redacting them and displaying a message. This will be a nicer overall experience by default, and for forums that want to be super strict with redaction this setting can always be disabled.
2020-09-17 14:15:02 +10:00
Martin Brennan 80268357e7
DEV: Change upload verified column to be integer (#10643)
Per review https://review.discourse.org/t/dev-add-verified-to-uploads-and-fill-in-s3-inventory-10406/14180

Change the verified column for Upload to a verified_status integer column, to avoid having NULL as a weird implicit status.
2020-09-17 13:35:29 +10:00
Neil Lalonde b207842720
FIX: group tag notifications included for all users
Regression from my last commit
2020-09-16 16:02:28 -04:00
Neil Lalonde 8333872e88
FIX: N+1 for admins viewing groups page
Groups page was loading fields that are only used on the group show
page, so move those fields to the GroupShowSerializer.
Also only fetch the default category and tag notifications once.
2020-09-16 14:58:52 -04:00
Arpit Jalan 32d6286bea FIX: category slug route was not working for subfolder setup
https://meta.discourse.org/t/relative-url-root-issues-incorrect-latest-link-incorrect-redirect/163266

URLs like `/forum/c/staff` (subfolder setup) were landing on
`/forum/forum/c/staff/3`. Note the extra "/forum". This commit
strips the redundant subfolder path from category URL.
2020-09-16 12:09:16 +05:30
Penar Musaraj 6610576deb
FIX: Clear stylesheet cache on base font change
Previously, we were not clearing the cache on themes using the default
color scheme.
2020-09-15 13:13:09 -04:00
Penar Musaraj 273db57d6e
FEATURE: Allow admins to delete user SSO records in the UI (#10669)
Also displays the user's last payload in the admin UI to help with debugging SSO issues.
2020-09-15 10:00:10 -04:00
Joffrey JAFFEUX bbddce4d3a
DEV: updates js transpiler to use babel 7 (#10627)
Updates our js transpiler code to use Babel 7.11.6

List of changes in this commit:

- Updates plugins, babel plugins all have a new version which doesn't contain -es2015- anymore
- Drops [transform-es2015-classes](https://babeljs.io/docs/en/babel-plugin-transform-classes) this plugin shouldn't be needed now that we don't support IE
- Drops check-es2015-constants, checking constants is now part of babel and the check-constants plugin is deprecated. As a result the behavior slightly changed, and is now wrapping every const call in a readOnlyError function which would throw if assigned a new value. This explains the modified spec.
- Adds [proposal-optional-chaining](https://babeljs.io/docs/en/babel-plugin-proposal-optional-chaining)

```javascript
const obj = {
  foo: {
    bar: {
      baz: 42,
    },
  },
};

const baz = obj?.foo?.bar?.baz; // 42
```

- Adds [proposal-json-strings](https://babeljs.io/docs/en/babel-plugin-proposal-json-strings)

```javascript
// IN
const ex = "before
after";
//                ^ There's a U+2028 char between 'before' and 'after'


// OUT
const ex = "before\u2028after";
//                ^ There's a U+2028 char between 'before' and 'after'
```

- Adds [proposal-nullish-coalescing-operator](https://babeljs.io/docs/en/babel-plugin-proposal-nullish-coalescing-operator)

```javascript
var object = {};
var foo = object.foo ?? "default"; // default
```

- Adds [proposal-logical-assignment-operators](https://babeljs.io/docs/en/babel-plugin-proposal-logical-assignment-operators)

```javascript
let a;
let b = 2;
a ||= b; // 2
```

- Adds [proposal-numeric-separator](https://babeljs.io/docs/en/babel-plugin-proposal-numeric-separator)

```javascript
let budget = 1_000_000_000_000;
console.log(budget === 10 ** 12); // true
```

- Adds proposal-object-rest-spread https://babeljs.io/docs/en/babel-plugin-proposal-object-rest-spread

```javascript
let { x, y, ...z } = { x: 1, y: 2, a: 3, b: 4 };
console.log(x); // 1
console.log(y); // 2
console.log(z); // { a: 3, b: 4 }
```

- Adds proposal-optional-catch-binding https://babeljs.io/docs/en/babel-plugin-proposal-optional-catch-binding

```javascript
try {

} catch {

} finally {
  // ensures finally is available in every browsers
}
```

- Adds improved regex support for firefox through (transform-dotall-regex](https://babeljs.io/docs/en/next/babel-plugin-transform-dotall-regex.html) and (proposal-unicode-property-regex](https://babeljs.io/docs/en/babel-plugin-proposal-unicode-property-regex)

- Drops async/generator stuff, the browser we target should allow to use this (excepts iterable async)
2020-09-15 09:26:33 +02:00
Guo Xiang Tan 2ff16b3650
FIX: `TopicQuery.list_private_messages_unread` ignore notification level 2020-09-15 13:33:11 +08:00
Krzysztof Kotlarek d83e3f9ce8 FIX: improvements after code review 2020-09-15 13:29:35 +08:00
Krzysztof Kotlarek 145814d29c FIX: spec for oversized images security fix
Spec to cover solution presented here - 333ddd4011
2020-09-15 13:29:35 +08:00
David Taylor 66eda8c9df
DEV: Add include_pms option to TopicQuery (#10647)
This is intended for use by plugins which are building their own topic lists, and want to include PMs alongside regular topics (e.g. discourse-assign). It does not get used directly in core.
2020-09-14 12:07:35 +01:00
Krzysztof Kotlarek cb58cbbc2c
FEATURE: allow to extend topic_eager_loads in Search (#10625)
This additional interface is required by encrypt plugin
2020-09-14 11:58:28 +10:00
Sam Saffron 15f7fa801d
DEV: remove stubs from silencer spec
Overall stubs lead to long term instability, this helps stabilize them.

Also fixed flaky spec around plugin hooks.

It was relying you `.posts` on system_user which could be loaded already
and invalid. Instead we now load it by hand.
2020-09-14 11:48:48 +10:00
Martin Brennan 5268568d23
FEATURE: Remove user topic timers and migrate to bookmarks with reminders (#10474)
This PR removes the user reminder topic timers, because that system has been supplanted and improved by bookmark reminders. The option is removed from the UI and all existing user reminder topic timers are migrated to bookmark reminders.

Migration does this:

* Get all topic_timers with status_type 5 (reminders)
* Gets all bookmarks where the user ID and topic ID match
* Loops through the found topic timers
  * If there is no bookmark for the OP of the topic, then we just create a bookmark with a reminder
  * If there is a bookmark for the OP of the topic and it does **not** have a reminder set, then just 
update it with the topic timer reminder
  * If there is a bookmark for the OP of the topic with a reminder then just discard the topic timer
* Cancels all outstanding user reminder topic timers
* **Trashes (not deletes) all user reminder topic timers**

Notes:

* For now I have left the user reminder topic timer job class in place; this is so the jobs can be cancelled in the migration. It and the specs will be deleted in the next PR.
* At a later date I will write a migration to delete all trashed user topic timers. They are not deleted here in case there are data issues and they need to be recovered.
* A future PR will change the UI of the topic timer modal to make it look more like the bookmark modal.
2020-09-14 11:11:55 +10:00
Krzysztof Kotlarek 333ddd4011 SECURITY: return error on oversized images 2020-09-14 10:45:11 +10:00
Joffrey JAFFEUX 539eb63904
DEV: heisentest cause by a post not created yet (#10656) 2020-09-12 08:47:06 +02:00
Gerhard Schlager 81395be4c1 FIX: Ensure uploads are linked to post when revising a post
Editing a post didn't update the `post_uploads` right away. Instead it relied on the `CookedPostProcessor`. This can lead to an inconsistent state if uploads are added or removed during an edit and, for some reason, the `ProcessPost` job doesn't run (successfully). This inconsistency leads to missing uploads, because the newly added uploads appear to be unused and will be deleted by the `CleanUpUploads` job. In addition to that, uploads, which got removed during the edit, appear to be still in use and won't be deleted by the background job.

This commit ensures that the `post_uploads` are updated during the edit without relying on a background job.
2020-09-11 11:49:15 +02:00
Guo Xiang Tan 543e972fec
PERF: Remove category_user lookup when loading private messages.
Private messages do not belong to categories so the query is unnecessary
overhead.
2020-09-11 16:26:38 +08:00
Sam Saffron 44fba9463b
FEATURE: Add support for not persistent sessions
In some cases Discourse admins may opt for sessions not to persist when a
browser is closed.

This is particularly useful in healthcare and education settings where
computers are shared among multiple workers.

By default `persistent_sessions` site setting is enabled, to opt out you
must disable the site setting.
2020-09-11 15:11:24 +10:00
Vinoth Kannan 9e4ed03b8f FEATURE: moderators allowed to view groups which members can see.
Currently, if a group's visibility is set to "Group owners, members" then the mods can't view those group pages. The same rule is applied for members visibility setting too.

This reverts commit 7fc7090. And fixed the spec test fails.
2020-09-11 08:20:13 +05:30
Guo Xiang Tan 245d29e5a3
SECURITY: Mod should not see `group_users` and `second_factor_enabled`.
Moderators should not be able to see `UserSerializer#group_users` and `UserSerializer#second_factor_enabled` of other users.

Impact of leaking this is low because the information leaked is not
exploitable.
2020-09-11 10:23:35 +08:00
Guo Xiang Tan 7fc70900d2
Revert "FEATURE: moderators allowed to view groups which members can see."
Build is broken.

This reverts commit 2bf0c4188b.
2020-09-11 10:04:54 +08:00
Kane York e0a0928420
FEATURE: Add bookmarks to the user export (#10591) 2020-09-11 11:03:22 +10:00
Vinoth Kannan 2bf0c4188b FEATURE: moderators allowed to view groups which members can see.
Currently, if a group's visibility is set to "Group owners, members" then the mods can't view those group pages. The same rule is applied for members visibility setting too.
2020-09-11 03:33:44 +05:30
Gerhard Schlager ac70c48be4 FIX: Prevent "uploads are missing in S3" alerts after restoring a backup
After restoring a backup it takes up to 48 hours for uploads stored on S3 to appear in the S3 inventory. This change prevents alerts about missing uploads by preventing the EnsureS3UploadsExistence job from running in the first 48 hours after a restore. During the restore it  deletes the count of missing uploads from the PluginStore, so that an alert isn't triggered by an old number.
2020-09-10 21:37:48 +02:00
Vinoth Kannan ba2f3119c0 UX: display moderators group name in the group dropdown menu. 2020-09-11 00:06:40 +05:30
Roman Rizzi efb9fd6ac0
FIX: Make sure rel attributes are correctly set. (#10645)
We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 12:59:51 -03:00
David Taylor 2f96474155
FIX: Ignore empty search terms in topic queries
Previously an empty search term would cause an invalid tsquery, and led to a 500 error. Now an empty string will be ignored.
2020-09-10 15:49:11 +01:00
Joffrey JAFFEUX e4fbe404b4
linting (#10644) 2020-09-10 11:34:48 +02:00
Guo Xiang Tan f27de87bf3
FIX: Update first_pm_unread_at of user's groups without unread.
If a user always read all group messages, we will never update the
`first_pm_unread_at` column since the previous query will not return the
group_user. Instead, we should update `first_pm_unread_at` to the
current timestamp if the user has read everything.

Follow-up to 9b75d95fc6
2020-09-10 17:19:38 +08:00
Martin Brennan 7f2f87bf59
DEV: Review fixes (#10641)
See comments in https://review.discourse.org/t/dev-imap-log-to-database-10435/14337/6 for context.
2020-09-10 13:41:46 +10:00
Guo Xiang Tan e6ca1b4326
FIX: Admin search for PMs should only search own PMs.
In c6ceda8c, a bug was introduced where an admin searching for his own
private messages will actually end up searching through all private
messages on the site.

Follow-up to c6ceda8c4e
2020-09-10 11:37:18 +08:00
Martin Brennan dede942007
FEATURE: Allow email image embed with secure media (#10563)
This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced:

* `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them.
* `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`.

`Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients.

All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`.

Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions.

![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-10 09:50:16 +10:00
Krzysztof Kotlarek d260e42c8a
FIX: topic_creator accepts participant_count in import mode (#10632)
The issue mentioned here: https://meta.discourse.org/t/imported-private-discussion-doesnt-appear-in-the-author-inbox/163252

`participant_count` is important to attribute for private messages. If they are imported, we should allow them to set that attribute.

A workaround would be evaluating `update_statistics` method on each Topic but that is less performant.
2020-09-10 08:16:57 +10:00
Roman Rizzi cac64a95aa
FIX: We should check for watched words first even if the user is a fast typer. (#10630) 2020-09-09 14:36:22 -03:00
Robin Ward add3d46bc0 FIX: `approve post count` did not take new topics into account
This means if a user created a new topic, and it was approved, it would not
count towards their approval count. Replies worked correctly.
2020-09-09 12:47:37 -04:00
Penar Musaraj b409954398
DEV: Improve theme support for color definitions (#10634)
- Lets child components extend color definitions
- Includes default theme color definitions
- Fails gracefully on color stylesheet SCSS errors
- Includes theme variables when extending colors
2020-09-09 11:43:34 -04:00
Guo Xiang Tan 611d9f61fd
Fix the build. 2020-09-09 15:43:38 +08:00
Guo Xiang Tan 9b75d95fc6 PERF: Keep track of first unread PM and first unread group PM for user.
This optimization helps to filter away topics so that the joins on
related tables when querying for unread messages is not expensive.
2020-09-09 14:05:41 +08:00
Guo Xiang Tan 0398271f87
FIX: Admin should always be able to view group messages.
Follow-up to b0f22f2523.

Before this fix, `SiteSetting.enable_personal_messages = false` will
prevent admin from viewing group messages.
2020-09-09 08:58:21 +08:00
Blake Erickson 4dfdfeaafd
DEV: Resolve this child category pending request spec (#10604)
There is a request spec that was ignored with the `xit` flag almost a
year ago and every time you generate the api docs with

```
rake rswag:specs:swaggerize
```

it shows the output of this pending test and I guess I finally got sick
of looking at it, so here is a fix for it.

Original Commit: d84c34ad75
2020-09-08 13:23:11 -06:00
Penar Musaraj d4a7058cb1
DEV: Strip unicode from color scheme stylesheet filenames (#10628) 2020-09-08 15:00:16 -04:00
Guo Xiang Tan d3ebaa41ce
DEV: Address review comments for 5ed84d9885. 2020-09-08 11:17:35 +08:00
Guo Xiang Tan b0f22f2523
SECURITY: Remove indication that a group exists if user can't see it.
Minor security fix but we should not leak any hints that a group exists
even if a user does not have access to the group.
2020-09-08 10:53:48 +08:00