Commit Graph

5339 Commits

Author SHA1 Message Date
Dan Ungureanu da2889a7a8
DEV: Add more verbose logging for image uploads (#13270)
Image optimization fails randomly (very rare) without a trace and it is
near impossible to find culprit image, reproduce the issue and attempt
to fix.
2021-06-04 15:13:58 +03:00
Alan Guo Xiang Tan 982eaab9b0 PERF: Defer setting of distributed cache in category.
See follow up commit for rational.

Follow-up to 8cfe203
2021-06-04 10:49:54 +08:00
Alan Guo Xiang Tan 3c1f4d5771 FIX: Clear post action types application serializer fragment cache.
The bug was introduced in dc10bdee3d
2021-06-04 09:14:49 +08:00
Alan Guo Xiang Tan a8667b5454 PERF: Defer setting of distributed cache in more spots.
See follow up commit for rational.

Follow-up to 8cfe203383
2021-06-04 09:13:18 +08:00
Martin Brennan eb2c399445
FEATURE: Use group SMTP settings for sending user notification emails (initial) (#13220)
This PR changes the `UserNotification` class to send outbound `user_private_message` using the group's SMTP settings, but only if:

* The first allowed_group on the topic has SMTP configured and enabled
* SiteSetting.enable_smtp is true
* The group does not have IMAP enabled, if this is enabled the `GroupSMTPMailer` handles things

The email is sent using the group's `email_username` as both the `from` and `reply-to` address, so when the user replies from their email it will go through the group's SMTP inbox, which needs to have email forwarding set up to send the message on to a location (such as a hosted site email address like meta@discoursemail.com) where it can be POSTed into discourse's handle_mail route.

Also includes a fix to `EmailReceiver#group_incoming_emails_regex` to include the `group.email_username` so the group does not get a staged user created and invited to the topic (which was a problem for IMAP), as well as updating `Group.find_by_email` to find using the `email_username` as well for inbound emails with that as the TO address.

#### Note

This is safe to merge without impacting anyone seriously. If people had SMTP enabled for a group they would have IMAP enabled too currently, and that is a very small amount of users because IMAP is an alpha product, and also because the UserNotification change has a guard to make sure it is not used if IMAP is enabled for the group. The existing IMAP tests work, and I tested this functionality by manually POSTing replies to the SMTP address into my local discourse.

There will probably be more work needed on this, but it needs to be tested further in a real hosted environment to continue.
2021-06-03 14:47:32 +10:00
Bianca Nenciu 648d2fd793
DEV: Add test for link watched words (#13251) 2021-06-03 11:36:07 +10:00
Alan Guo Xiang Tan 8cfe203383 PERF: Defer setting of distributed cache in performance critical paths.
Setting a key/value pair in DistributedCache involves waiting on the
write to Redis to finish. In most cases, we don't need to wait on the
setting of the cache to finish. We just need to take our return value
and move on.
2021-06-03 09:30:52 +08:00
Martin Brennan 83211cff25
FIX: Change order of topic_tracking_state SELECT SQL (#13259)
This allows us to do DISTINCT on the topic_id to remove
duplicates (e.g. in extensions to the report SQL), and
also introduces an additional_join_sql string to allow
extensions to JOIN additional tables.
2021-06-03 11:21:33 +10:00
Andrei Prigorshnev 2672358b72
FIX: active record annotation of topic model (#13250)
I've removed constraint from the excerpt column in 00300b1, but forgot to change annotation
2021-06-02 18:16:03 +04:00
Andrei Prigorshnev 932a2fe419
FIX: PG::StringDataRightTruncation when linking posts (#13134)
Users who use encoded slugs on their sites sometimes run into 500 error when pasting a link to another topic in a post. The problem happens when generating a backward "reflection" link that would appear in a linked topic. Link URL restricted on the database level to 500 chars in length. At first glance, it should work since we have a restriction on topic title length.

But it doesn't work when a site uses encoded slugs, like here (take a look at the URL). The link to a topic, in this case, can be much longer than 500 characters.

By the way, an error happens only when generating a "reflection" link and doesn't happen with a direct link, we truncate that link. It works because, in this case, the original long link is still present in the post body and can be used for navigation. But we can't do the same for backward "reflection" links (without rewriting their implementation), the whole link must be saved to the database.

The simplest and cleanest solution will be just to remove the restriction on the database level. Abuse is impossible here since we are already protected by the restriction on topic title length. There aren’t performance benefits in using length-constrained columns in Postgres, in fact, length-constrained columns need a few extra CPU cycles to check the length when storing data.
2021-06-02 15:27:04 +04:00
Andrei Prigorshnev b7b8f5e6f3
FIX: Moderator actions and small actions shouldn't prevent fully merged topics from closing (#13200)
When a topic is fully merged into another topic we close it and schedule its deleting. But, because of a bug, if the merged topic contains some moderator actions or small actions it won't be merged. This change fixes this problem.

An important note: in general, we don't want to close a topic after moving posts if it still contains some regular posts or whispers. But when we are moving posts to a private message we don't want the notice about it to be publicly visible. So we use whispers with action_code == 'split_topic' instead of small_actions in such cases and we should ignore this specific kind of whispers when decide if we should close the merged topic.
2021-06-02 13:42:03 +04:00
Bianca Nenciu d9484db718
FIX: Split link watched words from replace (#13196)
It was not clear that replace watched words can be used to replace text
with URLs. This introduces a new watched word type that makes it easier
to understand.
2021-06-02 15:36:49 +10:00
Martin Brennan e15c86e8c5
DEV: Topic tracking state improvements (#13218)
I merged this PR in yesterday, finally thinking this was done https://github.com/discourse/discourse/pull/12958 but then a wild performance regression occurred. These are the problem methods:

1aa20bd681/app/serializers/topic_tracking_state_serializer.rb (L13-L21)

Turns out date comparison is super expensive on the backend _as well as_ the frontend.

The fix was to just move the `treat_as_new_topic_start_date` into the SQL query rather than using the slower `UserOption#treat_as_new_topic_start_date` method in ruby. After this change, 1% of the total time is spent with the `created_in_new_period` comparison instead of ~20%.

----

History:

Original PR which had to be reverted **https://github.com/discourse/discourse/pull/12555**. See the description there for what this PR is achieving, plus below.

The issue with the original PR is addressed in 92ef54f402

If you went to the `x unread` link for a tag Chrome would freeze up and possibly crash, or eventually unfreeze after nearly 10 mins. Other routes for unread/new were similarly slow. From profiling the issue was the `sync` function of `topic-tracking-state.js`, which calls down to `isNew` which in turn calls `moment`, a change I had made in the PR above. The time it takes locally with ~1400 topics in the tracking state is 2.3 seconds.

To solve this issue, I have moved these calculations for "created in new period" and "unread not too old" into the tracking state serializer.

When I was looking at the profiler I also noticed this issue which was just compounding the problem. Every time we modify topic tracking state we recalculate the sidebar tracking/everything/tag counts. However this calls `forEachTracked` and `countTags` which can be quite expensive as they go through the whole tracking state (and were also calling the removed moment functions).

I added some logs and this was being called 30 times when navigating to a new /unread route because  `sync` is being called from `build-topic-route` (one for each topic loaded due to pagination). So I just added a debounce here and it makes things even faster.

Finally, I changed topic tracking state to use a Map so our counts of the state keys is faster (Maps have .size whereas objects you have to do Object.keys(obj) which is O(n).)

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2021-06-02 09:06:29 +10:00
Gerhard Schlager 41ee5b7c86
FIX: Don't store translated trust level names in anonymous cache (#13224)
Refactors `TrustLevel` and moves translations from server to client

Additional changes:
  * "staff" and "admin" wasn't translatable in site settings
  * it replaces a concatenated string with a translation
  * uses translation for trust levels in users_by_trust_level report
  * adds a DB migration to rename keys of translation overrides affected by this commit
2021-06-01 22:11:48 +02:00
Penar Musaraj 4d4c3fe1e4
FIX: Delete internal links when moderator deletes a post (#13233) 2021-06-01 14:02:53 -04:00
Joffrey JAFFEUX 624edac3bb
DEV: calls user_added_to_group on group/TL enforcement (#13222) 2021-06-01 10:34:41 +02:00
Joffrey JAFFEUX 1cd0424ccd
FEATURE: lets users favorite 2 badges to show on user-card (#13151) 2021-06-01 10:33:40 +02:00
Osama Sayegh b81b24dea2
Revert "DEV: Topic tracking state improvements (#12958)" (#13209)
This reverts commit 002c676344.

Perf regression, we will redo it.
2021-05-31 17:47:42 +10:00
Martin Brennan 002c676344
DEV: Topic tracking state improvements (#12958)
Original PR which had to be reverted **https://github.com/discourse/discourse/pull/12555**. See the description there for what this PR is achieving, plus below.

The issue with the original PR is addressed in 92ef54f402

If you went to the `x unread` link for a tag Chrome would freeze up and possibly crash, or eventually unfreeze after nearly 10 mins. Other routes for unread/new were similarly slow. From profiling the issue was the `sync` function of `topic-tracking-state.js`, which calls down to `isNew` which in turn calls `moment`, a change I had made in the PR above. The time it takes locally with ~1400 topics in the tracking state is 2.3 seconds.

To solve this issue, I have moved these calculations for "created in new period" and "unread not too old" into the tracking state serializer.

When I was looking at the profiler I also noticed this issue which was just compounding the problem. Every time we modify topic tracking state we recalculate the sidebar tracking/everything/tag counts. However this calls `forEachTracked` and `countTags` which can be quite expensive as they go through the whole tracking state (and were also calling the removed moment functions).

I added some logs and this was being called 30 times when navigating to a new /unread route because  `sync` is being called from `build-topic-route` (one for each topic loaded due to pagination). So I just added a debounce here and it makes things even faster.

Finally, I changed topic tracking state to use a Map so our counts of the state keys is faster (Maps have .size whereas objects you have to do Object.keys(obj) which is O(n).)
2021-05-31 09:22:28 +10:00
Andrei Prigorshnev 74f7150324
FEATURE: Automatically timed delete stub topics after entire topic is merged into another topic (#13187)
When a topic is fully merged into another topic we close it. Now we want also to set a timer for deleting this topic. By default, stub topics will be deleted in 7 days. Users can change this period or disable auto-deleting by setting the period to 0.
2021-05-28 17:33:10 +04:00
Martin Brennan 501de809da
FIX: Do not mark badge image uploads as secure (#13193)
* FIX: Do not mark badge image uploads as secure

We do not need badge_image upload types to be marked as secure.
Post migration is the same as
https://github.com/discourse/discourse/pull/12081.

See
https://meta.discourse.org/t/secure-media-uploads/140017/122?u=martin
2021-05-28 12:35:52 +10:00
Martin Brennan 964da21817
FEATURE: Improve group email settings UI (#13083)
This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory.

### UI

Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP.

You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator.

If there is an issue with the configuration or credentials a meaningful error message should be shown.

IMAP settings must also be validated when IMAP is enabled, before saving.

When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved):

### Database & Backend

This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group.

For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous.

There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-28 09:28:18 +10:00
Vinoth Kannan 3358ab6b59
FIX: don't update `allow_title` column of existing badges in seed. (#13190)
The default `allow_title` column value is "true" for regular and leader badges. After we disable it in admin side the seed method enabling it again while upgrading. So we shouldn't do it for existing badges.
2021-05-28 00:30:57 +05:30
Bianca Nenciu 571ee4537a
FEATURE: Silence watched word (#13160)
This is a new type of watched word to replace auto_silence_first_post_
regex site setting.
2021-05-27 19:19:58 +03:00
Sam d45682716b
FIX: automatically expire bad push channels (#13156)
Previously we would retry push notifications indefinitely for all errors
except for ExpiredSubscription

Under certain conditions other persistent errors may arise such as a persistent
rate limit.

If we track more than 3 errors in a period of time longer than a day we will
delete the subscription

Also performs a bit of internal cleanup to ensure protected methods really
are private.
2021-05-27 06:49:20 +10:00
Roman Rizzi 83dd47f0b4
FEATURE: Navigate to an approved queued topic from the review queue. (#12841)
Admins can visit an approved queued topic from the review queue by clicking their title. We no longer store the created post and topic ids in the reviewable's payload object. Instead, we set the `topic_id` and `target_id` attributes.
2021-05-26 15:43:18 -03:00
David Taylor ea61bcaf13
DEV: Define theme test modules under a `/test` subdirectory (#13158) 2021-05-26 11:49:51 +01:00
Dan Ungureanu a5273b37f7
FIX: Show inviter name in email's from field (#13141)
'From' field of the email contained the name of the user who posted the
shared post. Instead, it should contain the name of the inviter.
2021-05-26 12:55:07 +10:00
Dan Ungureanu 197e3f24ce
FEATURE: Show stale reviewable to other clients (#13114)
The previous commits removed reviewables leading to a bad user
experience. This commit updates the status, replaces actions with a
message and greys out the reviewable.
2021-05-26 09:47:35 +10:00
Andrei Prigorshnev 0df6b0bc47
FIX: slow mode dialog doesn't remember Enabled Until value (#13076)
If reload a page after enabling slow mode and open the slow mode dialog again it would show a slow mode interval but wouldn't show Enabled Until value. This PR fixes it.
2021-05-21 18:13:14 +04:00
Josh Soref 13d40ead97
DEV: Correct spelling mistakes in comments 2021-05-21 13:37:17 +10:00
Josh Soref 59097b207f
DEV: Correct typos and spelling mistakes (#12812)
Over the years we accrued many spelling mistakes in the code base. 

This PR attempts to fix spelling mistakes and typos in all areas of the code that are extremely safe to change 

- comments
- test descriptions
- other low risk areas
2021-05-21 11:43:47 +10:00
Sam 058b5310c1
FIX: Correctly publish messages unconditionally to admins (#13053)
Under certain conditions admins would miss messages when posting action in
topics where they have permission.

This also fixes an error where we would sometimes explode when publishing to
an empty group.
2021-05-20 16:58:27 +10:00
Martin Brennan 38742bc208
FIX: Wrong scope used for notification levels user serializer (#13039)
This is a recent regression introduced by https://github.com/discourse/discourse/pull/12937 which makes it so that when looking at a user profile that is not your own, specifically the category and tag notification settings, you would see your own settings instead of the target user. This is only a problem for admins because regular users cannot see these details for other users.

The issue was that we were using `scope` in the serializer, which refers to the current user, rather than using a scope for the target user via `Guardian.new(user)`.

However, on further inspection the `notification_levels_for` method for `TagUser` and `CategoryUser` did not actually need to be accepting an instance of Guardian, all that it was using it for was to check guardian.anonymous? which is just a fancy way of saying user.blank?. Changed this method to just accept a user instead and send the user in from the serializer.
2021-05-14 09:45:14 +10:00
Blake Erickson ee880c4c89
DEV: Extract out compound conditional (#13008) 2021-05-10 11:11:51 -06:00
Roman Rizzi d4b5a81b05
FIX: Recalculate scores only when approving or transitioning to pending. (#13009)
Recalculating a ReviewableFlaggedPost's score after rejecting or ignoring it sets the score as 0, which means that we can't find them after reviewing. They don't surpass the minimum priority threshold and are hidden.

Additionally, we only want to use agreed flags when calculating the different priority thresholds.
2021-05-10 14:09:04 -03:00
Jeff Wong 11fe13b45e
Color scheme optional defer publish (#12972)
* DEV: add a method of skipping publishing stylesheets afer color scheme save

allows a method to publish all stylesheets if we make changes to many
stylesheets at once

* use after_save_commit for stylesheet change callbacks

This may be more reliable for picking up new stylesheet changes via messagebus
as after_save does not guarantee the updates exists in the DB yet.

* add skip_publish option for create_from_base
2021-05-06 11:26:58 -07:00
Martin Brennan 72648dd576
FIX: Base topic details message on current category and tag tracking state (#12937)
The user may have changed their category or tag tracking settings since a topic was tracked/watched based on those settings in the past. In that case we need to alter the reason message we show them otherwise it is very confusing for the end user to be told they are tracking a topic because of a category, when they are no longer tracking that category.

For example: "You will see a count of new replies because you are tracking this category." becomes: "You will see a count of new replies because you were tracking this category in the past."

To do this, it was necessary to add tag and category tracking info to current user serializer. I improved the serializer code so it only does 3 SQL queries instead of 9 to get the tracking information for tags and categories for the current user.
2021-05-06 09:14:07 +10:00
Joffrey JAFFEUX d14a7f1965
DEV: removes comment about disable_jump_reply (#12939)
This has been removed in 40fa96777d
2021-05-04 11:44:09 +02:00
Bianca Nenciu 77c92fd674
FIX: Hide Uncategorized unless allow_uncategorized_topics (#12889)
Uncategorized was sometimes visible even if allow_uncategorized_topics
was false. This was especially happening on mobile, if at least one
topic was uncategorized.
2021-05-04 13:05:08 +10:00
Jeff Wong 75e159f0ed
FEATURE: add support for like webhooks (#12917)
* FEATURE: add support for like webhooks

Add support for like webhooks. Webhook events only send on user membership
in the defined webhook group filters.

This also fixes group webhook events, as before this was never used, and
the logic was not correct.
2021-04-30 17:08:38 -07:00
Joffrey JAFFEUX ed818a4a19
FIX: prevents malformed href to crash TopicEmbed (#12910)
If the associated page of a remote url passed to `TopicEmber.new(remote_url)` contained a malformed link like: `<a href="(http://foo.bar)">Baz</a>` it would raise an uncaught exception:

```
Job exception: Invalid scheme format: (http
```
2021-04-30 11:10:19 +02:00
Jeff Wong 0abb272289
DEV: Cleanup after remote update check (#12887)
Checking for remote should cleanup after itself. Currently each check litters
the /tmp filesystem with checkouts. This patch ensures that update checks
keep the system a bit tidier.
2021-04-28 17:07:27 -07:00
Osama Sayegh 4f88f2eb15
FEATURE: Allow theme tests to be run in production (take 2) (#12845)
This commit allows site admins to run theme tests in production via a new `/theme-qunit` route. When you visit `/theme-qunit`, you'll see a list of the themes/components installed on your site that have tests, and from there you can select a theme or component that you run its tests.

We also have a new rake task `themes:install_and_test` that can be used to install a list of themes/components on a temporary database and run the tests of the themes/components that are installed. This rake task can be useful when upgrading/deploying a Discourse instance to make sure that the installed themes/components are compatible with the new Discourse version being deployed, and if the tests fail you can abort the build/deploy process so you don't end up with a broken site.
2021-04-28 23:12:08 +03:00
Martin Brennan 6d53005e8b
Revert "DEV: Improving topic tracking state code (#12555)" (#12864)
This reverts commit 45df579db0.

This was causing huge browser freezes and crashes.
2021-04-28 11:29:54 +10:00
Martin Brennan 45df579db0
DEV: Improving topic tracking state code (#12555)
The aim of this PR is to improve the topic tracking state JavaScript code and test coverage so further modifications can be made in plugins and in core. This is focused on making topic tracking state changes easier to respond to with callbacks, and changing it so all state modifications go through a single method instead of modifying `this.state` all over the place. I have also tried to improve documentation, make the code clearer and easier to follow, and make it clear what are public and private methods.

The changes I have made here should not break backwards compatibility, though there is no way to tell for sure if other plugin/theme authors are using tracking state methods that are essentially private methods. Any name changes made in the tracking-state.js code have been reflected in core.

----

We now have a `_trackedTopicLimit` in the tracking state. Previously, if a topic was neither new nor unread it was removed from the tracking state; now it is only removed if we are tracking more than `_trackedTopicLimit` topics (which is set to 4000). This is so plugins/themes adding topics with `TopicTrackingState.register_refine_method` can add topics to track that aren't necessarily new or unread, e.g. for totals counts.

Anywhere where we were doing `tracker.states["t" + data.topic_id] = newObject` has now been changed to flow through central `modifyState` and `modifyStateProp` methods. This is so state objects are not modified until they need to be (e.g. sometimes properties are set based on certain conditions) and also so we can run callback functions when the state is modified.

I added `onStateChange` and `onMessageIncrement` methods to register callbacks that are called when the state is changed and when the message count is incremented, respectively. This was done so we no longer need to do things like `@observes("trackingState.states")` in other Ember classes.

I split up giant functions like `sync` and `establishChannels` into smaller functions for readability and testability, and renamed many small functions to _functionName to designate them as private functions which not be called by consumers of `topicTrackingState`. Public functions are now all documented (well...at least ones that are not immediately obvious).

----

On the backend side, I have changed the MessageBus publish events for TopicTrackingState to send back tags and tag IDs for more channels, and done some extra code cleanup and refactoring. Plugins may override `TopicTrackingState.report` so I have made its footprint as small as possible and externalised the main parts of it into other methods.
2021-04-28 09:54:45 +10:00
Roman Rizzi cdbdb04909
UX: The Site's logo is the selected option when changing the system's user avatar. (#12861)
If the "use_site_small_logo_as_system_avatar" setting is enabled, the site's small logo is displayed as the selected option by the avatar-selector. Choosing a different avatar disables the setting.
2021-04-27 17:28:15 -03:00
Andrei Prigorshnev d3c0b6bfe1
FEATURE: include avatar flair on the avatars listed in a user summary’s “Most…” sections (#12858) 2021-04-27 23:09:32 +04:00
David Taylor 1fd8f6df5f
PERF: Improve theme stylesheet compilation performance (#12850)
When building the `scss_load_paths`, we were creating a full export of the theme (including uploads), and not cleaning it up. With many uploads, this can be extremely slow (because it downloads every upload from S3), and the lack of cleanup could cause a disk to fill up over time.

This commit updates the ZipExporter to provide a `with_export_dir` API, which takes care of cleanup. It also adds a kwarg which allows exporting only extra_scss fields. This should make things much faster for themes with many uploads.
2021-04-27 14:33:43 +01:00
David Taylor 657dff3544
PERF: Remove N+1s from ThemeController#update and #show (#12842)
These endpoints only return one `Theme` row, but the one-many relations were not being preloaded efficiently. This commit moves the `includes` statement to a scope, and makes use of it in `#index`, `#show`, and `#update`.
2021-04-27 12:30:29 +01:00
Krzysztof Kotlarek e29605b79f
FEATURE: the ability to search users by custom fields (#12762)
When the admin creates a new custom field they can specify if that field should be searchable or not.

That setting is taken into consideration for quick search results.
2021-04-27 15:52:45 +10:00
Jeff Wong fa393b2956
FEATURE: add reviewable score updated webhook (#12846)
Adds a webhook to notify when a reviewable score is updated.

This is different from created or status changed as additional flags can
roll in and update the score without updating status. Useful for applications
looking to integrate in with Discourse's scores
2021-04-26 17:40:32 -07:00
Osama Sayegh a169dc6832
Revert "FEATURE: Allow theme tests to be run in production (#12815)" (#12840)
This reverts commit 7217dcb67a.

https://meta.discourse.org/t/failed-to-bootstrap-due-to-out-of-memory-killer/188141/18?u=osama

Precompiling test_helper.js is so expensive that it can make bootstrap
fail on servers with limited resources (2GB RAM). We will find another
way that doesn't require much resources.
2021-04-26 23:05:58 +03:00
Osama Sayegh 7217dcb67a
FEATURE: Allow theme tests to be run in production (#12815)
This commit allows site admins to run theme tests in production via a new `/theme-qunit` route. When you visit `/theme-qunit`, you'll see a list of the themes/components installed on your site that have tests, and from there you can select a theme or component that you run its tests.

We also have a new rake task `themes:install_and_test` that can be used to install a list of themes/components on a temporary database and run the tests of the themes/components that are installed. This rake task can be useful when upgrading/deploying a Discourse instance to make sure that the installed themes/components are compatible with the new Discourse version being deployed, and if the tests fail you can abort the build/deploy process so you don't end up with a broken site.
2021-04-26 12:56:45 +03:00
Andrei Prigorshnev f7aeb257ee
FIX: Skip notifications about beginner badges (#12819) 2021-04-26 11:41:51 +04:00
Roman Rizzi 60059a7190
FEATURE: A low priority filter for the review queue. (#12822)
This filter hides reviewables with a score lower than the "reviewable_low_priority_threshold" setting. We only use reviewables that already met this threshold to calculate the Medium and High priority filters.
2021-04-23 15:34:24 -03:00
Dan Ungureanu cfee2728ce
FEATURE: New share topic modal (#12804)
The old share modal used to host both share and invite functionality,
under two tabs. The new "Share Topic" modal can be used only for
sharing, but has a link to the invite modal.

Among the sharing methods, there is also "Notify" which points out
that existing users will simply be notified (this was not clear
before). Staff members can notify as many users as they want, but
regular users are restricted to one at a time, no more than
max_topic_invitations_per_day. The user will not receive another
notification if they have been notified of the same topic in past hour.

The "Create Invite" modal also suffered some changes: the two radio
boxes for selecting the type (invite or email) have been replaced by a
single checkbox (is email?) and then the two labels about emails have
been replaced by a single one, some fields were reordered and the
advanced options toggle was moved to the bottom right of the modal.
2021-04-23 19:18:23 +03:00
Arpit Jalan 271a372e3d
FIX: when a flag is handled archive message for moderator group (#12798)
When a user flags a post with the “Something Else” option, a PM between
the user and the moderators group is created. If no moderators reply to
the PM, when the flag is handled at /review, an auto-reply is created
for the PM. However, the PM is not archived, it stays in the inbox.

This commit ensures that the PM is archived for moderator group when no
moderator has replied to that PM.
2021-04-22 15:53:44 +05:30
Roman Rizzi 6b613e3076
FEATURE: Review every post using the review queue. (#12734)
* FEATURE: Review every post using the review queue.

If the `review_every_post` setting is enabled, posts created and edited by regular uses are sent to the review queue so staff can review them. We'll skip PMs and posts created or edited by TL4 or staff users.

Staff can choose  to:

- Approve the post (nothing happens)
- Approve and restore the post (if deleted)
- Approve and unhide the post (if hidden)
- Reject and delete it
- Reject and keep deleted (if deleted)
- Reject and suspend the user
- Reject and silence the user

* Update config/locales/server.en.yml

Co-authored-by: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2021-04-21 08:41:36 -03:00
Osama Sayegh 45ccadeeeb
DEV: Upgrade Rails to 6.1.3.1 (#12688)
Rails 6.1.3.1 deprecates a few API and has some internal changes that break our tests suite, so this commit fixes all the deprecations and errors and now Discourse should be fully compatible with Rails 6.1.3.1. We also have a new release of the rails_failover gem that's compatible with Rails 6.1.3.1.
2021-04-21 12:36:32 +03:00
Penar Musaraj c47e6a2004
FIX: Use CDN urls for theme settings of type upload (#12773) 2021-04-20 18:42:02 -04:00
Penar Musaraj d44deb45f3
FIX: Use CDN urls for theme uploads (#12769) 2021-04-20 13:25:35 -04:00
Dan Ungureanu 85d4b60a45
FIX: Improve error messages for invites (#12714)
The error messages used to include an unnecessary 'Validation failed:
Email' prefix which was removed.
2021-04-15 14:46:32 +03:00
David Taylor c60668a052
FIX: Ensure the top 6 categories are shown in the user summary (#12691)
Previously it would pluck 6 categories which the user had posted in, **then** order them. To select the **top 6** categories, we need to perform the ordering in the SQL query before the LIMIT
2021-04-15 11:05:03 +01:00
Michael Brown dda1cd6a38 FEATURE: allow setting postgres connection variables via environment 2021-04-14 13:31:32 -04:00
Dan Ungureanu 99dadb2129
PERF: Async notify users after inviting group (#12697)
Inviting a group generates a notification for each member. If this
happens synchronously it may take a while, leading to a poor user
experience.
2021-04-14 19:30:51 +03:00
Dan Ungureanu 8c24a848e0
FEATURE: Reduce invite key length (#12692)
We used to generate invite keys that were 32-characters long which were
not very friendly and lead to very long links. This commit changes the
generation method to use almost all alphanumeric characters to produce
a 10-character long invite key.

This commit also introduces a rate limit for redeeming invites because
the probability of guessing an invite key has increased.
2021-04-14 19:22:16 +03:00
Dan Ungureanu 528cfea079
FEATURE: Auto-activate users invited by email (#12675)
When invited by email, users will receive an invite URL which contains
a token. If that token is present when the invite is redeemed, their
account will be automatically activated.
2021-04-14 12:15:56 +03:00
Bianca Nenciu e4e2c7c66f
FIX: Improve anchor links (#12683)
* FIX: Use theme color for anchor icon

* FIX: Do not count anchor links

* FIX: Do not count hashtags links either

* DEV: Add tests for link_count

* FIX: Disable anchors in quotes and preview

* FIX: Try building some anchor slugs for unicode

* DEV: Fix tests
2021-04-14 10:27:07 +03:00
Martin Brennan eeaecd4fd2
FEATURE: Category setting to allow unlimited first post edits by the owner of the topic (#12690)
This PR adds a new category setting which is a column in the `categories` table, `allow_unlimited_owner_edits_on_first_post`.

What this does is:

* Inside the `can_edit_post?` method of `PostGuardian`, if the current user editing a post is the owner of the post, it is the first post, and the topic's category has `allow_unlimited_owner_edits_on_first_post`, then we bypass the check for `LimitedEdit#edit_time_limit_expired?` on that post.
* Also, similar to wiki topics, in `PostActionNotifier#after_create_post_revision` we send a notification to all users watching a topic when the OP is edited in a topic with the category setting `allow_unlimited_owner_edits_on_first_post` enabled.

This is useful for forums where there is a Marketplace or similar category, where topics are created and then updated indefinitely by the OP rather than the OP making new topics or additional replies. In a way this acts similar to a wiki that only one person can edit.
2021-04-14 15:54:09 +10:00
Martin Brennan 66d17fdd6b
FIX: Topic user bookmarked column is out of sync after post moves (#12612)
When posts are moved from one topic to another, the `topic_user.bookmarked` column for all users in the new and the old topic needs to be resynced, for example because a user bookmarks post 12 in topic 1, then it is moved to topic 2, the topic_user record for topic 1 should no longer be bookmarked. A background job has been added to sync the column for a specified topic, or for no topic at all, which does it for all topics like the migration.

Also includes a migration that we have run in the past to fix bad data.

----

This has been addressed in other places in the past:

https://github.com/discourse/discourse/pull/10211
https://github.com/discourse/discourse/pull/10188
2021-04-14 09:10:53 +10:00
Roman Rizzi 958fbfb719
FEATURE: Send an email notification when a post is approved. (#12665)
We now send an email when a queued post is approved, and we create a notification.
2021-04-12 12:08:23 -03:00
Osama Sayegh cd24eff5d9
FEATURE: Introduce theme/component QUnit tests (take 2) (#12661)
This commit allows themes and theme components to have QUnit tests. To add tests to your theme/component, create a top-level directory in your theme and name it `test`, and Discourse will save all the files in that directory (and its sub-directories) as "tests files" in the database. While tests files/directories are not required to be organized in a specific way, we recommend that you follow Discourse core's tests [structure](https://github.com/discourse/discourse/tree/master/app/assets/javascripts/discourse/tests).

Writing theme tests should be identical to writing plugins or core tests; all the `import` statements and APIs that you see in core (or plugins) to define/setup tests should just work in themes.

You do need a working Discourse install to run theme tests, and you have 2 ways to run theme tests:

* In the browser at the `/qunit` route. `/qunit` will run tests of all active themes/components as well as core and plugins. The `/qunit` now accepts a `theme_name` or `theme_url` params that you can use to run tests of a specific theme/component like so: `/qunit?theme_name=<your_theme_name>`.

* In the command line using the `themes:qunit` rake task. This take is meant to run tests of a single theme/component so you need to provide it with a theme name or URL like so: `bundle exec rake themes:qunit[name=<theme_name>]` or `bundle exec rake themes:qunit[url=<theme_url>]`.

There are some refactors to how Discourse processes JavaScript that comes with themes/components, and these refactors may break your JS customizations; see https://meta.discourse.org/t/upcoming-core-changes-that-may-break-some-themes-components-april-12/186252?u=osama for details on how you can check if your themes/components are affected and what you need to do to fix them.

This commit also improves theme error handling in Discourse. We will now be able to catch errors that occur when theme initializers are run and prevent them from breaking the site and other themes/components.
2021-04-12 15:02:58 +03:00
Sam 5deda5ef3e
FIX: automatically timeout long running image magick commands (#12670)
Previously certain images may lead to convert / identify to run for unreasonable
amounts of time

This adds a maximum amount of time these commands can run prior to forcing
them to stop
2021-04-12 13:55:54 +10:00
Joffrey JAFFEUX b6337b72f1
FEATURE: adds last day to about page stats (#12663)
* FEATURE: adds last day to about page stats

* make it clear it's last 24 hours

* applies same copy fix to days
2021-04-12 12:50:33 +10:00
wilson29thid d5b30b9b7b
FEATURE: Add user_confirmed_email to user event webhook (#12539) 2021-04-12 12:48:42 +10:00
Martin Brennan 1ba5ccd8af
FIX: When user has already hit bookmark limit, do not error for clear_reminder! or other updates (#12658)
We introduced a cap on the number of bookmarks the user can add in be145ccf2f. However this has caused unintended side effects; when the `jobs/scheduled/bookmark_reminder_notifications.rb` runs we get this error for users who already had more bookmarks than the limit:

> Job exception: Validation failed: Sorry, you have too many bookmarks, visit #{url}/my/activity/bookmarks to remove some.

This is because the `clear_reminder!` call was triggering a bookmark validation, which raised an error because the user already had to many, holding up other reminders.

This PR also adds `max_bookmarks_per_user` hidden site setting (default 2000). This replaces the BOOKMARK_LIMIT const so we can raise it for certain sites.
2021-04-09 13:06:35 +10:00
Vinoth Kannan 26d7eedf4c
FEATURE: trigger webhook when a user added/removed in a group. (#12653)
Whenever a group is added or removed from a group a webhook event will get triggered if it's active.
2021-04-08 21:16:34 +05:30
David Taylor 38e7fe2770
FIX: Ensure group flair upload is present when deciding type (#12650)
Previously, if the upload_id was present, but the upload was missing, the entire site would give a server error.

We have no foreign keys on this relation, so we have to be able to cope with the situation where the upload_id is present, but the actual upload has been deleted.

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2021-04-08 15:54:09 +01:00
Joffrey JAFFEUX 081ada090c
UX: shows a hint when there are more tags than displayed (#12649) 2021-04-08 15:51:31 +02:00
Osama Sayegh 2b9ab3a0d9
Revert "FEATURE: Introduce theme/component QUnit tests (#12517)" (#12632)
This reverts commit a53d8d3e61 and 105634435f.

Reverted because the change broke some components. Will be added back in a few days.
2021-04-07 17:45:49 +03:00
Osama Sayegh 105634435f
FIX: Prevent double slashes in Ember templates paths (#12630)
Follow-up to https://github.com/discourse/discourse/pull/12517
2021-04-07 14:08:29 +03:00
Osama Sayegh a53d8d3e61
FEATURE: Introduce theme/component QUnit tests (#12517)
This commit allows themes and theme components to have QUnit tests. To add tests to your theme/component, create a top-level directory in your theme and name it `test`, and Discourse will save all the files in that directory (and its sub-directories) as "tests files" in the database. While tests files/directories are not required to be organized in a specific way, we recommend that you follow Discourse core's tests [structure](https://github.com/discourse/discourse/tree/master/app/assets/javascripts/discourse/tests).

Writing theme tests should be identical to writing plugins or core tests; all the `import` statements and APIs that you see in core (or plugins) to define/setup tests should just work in themes.

You do need a working Discourse install to run theme tests, and you have 2 ways to run theme tests:

* In the browser at the `/qunit` route. `/qunit` will run tests of all active themes/components as well as core and plugins. The `/qunit` now accepts a `theme_name` or `theme_url` params that you can use to run tests of a specific theme/component like so: `/qunit?theme_name=<your_theme_name>`.

* In the command line using the `themes:qunit` rake task. This take is meant to run tests of a single theme/component so you need to provide it with a theme name or URL like so: `bundle exec rake themes:qunit[name=<theme_name>]` or `bundle exec rake themes:qunit[url=<theme_url>]`.

There are some refactors to internal code that's responsible for processing themes/components in Discourse, most notably:

* `<script type="text/discourse-plugin">` tags are automatically converted to modules.

* The `theme-settings` service is removed in favor of a simple `lib` file responsible for managing theme settings. This was done to allow us to register/lookup theme settings very early in our Ember app lifecycle and because there was no reason for it to be an Ember service.

These refactors should 100% backward compatible and invisible to theme developers.
2021-04-07 10:39:57 +03:00
Bianca Nenciu 1682827f67
FIX: Include watched topics from muted categories in digests (#12602)
Topic that are muted or from muted categories are not included in
digests, but non-muted topics from muted categories should be included.
2021-04-07 00:01:15 +03:00
Arpit Jalan 3db08c073b
FIX: "confirm new email" emails were failing for EmailChangeRequest records with blank requested_by_user_id field (#12579) 2021-04-01 16:39:28 +05:30
Régis Hanol c847f5e8a1
DEV: small refactor of the category_moderators method (#12550)
* DEV: small refactor of the category_moderators method

Used `index_by(&:id)` instead of `map { |u| [u.id, u] }.to_h` thanks to @cvx's recommendation.

Also renamed the `moderators` variable to not clash with method of the same name.
2021-03-30 23:12:53 +02:00
Penar Musaraj 65ad8750c7
DEV: Remove draft attributes from topic lists (#12525) 2021-03-30 11:42:26 -04:00
Daniel Waterworth 70970bb791
FIX: Show error messages when adding permalinks in the admin UI (#12545)
Also, check for uniqueness of permalinks before attempting to save.
2021-03-29 13:36:59 -05:00
Dan Ungureanu 8335c8dc1a
FEATURE: Allow admins to pre-populate user fields (#12361)
Admins can use bulk invites to pre-populate user fields. The imported
CSV file must have a header with "email" column (first position) and
names of the user fields (exact match).

Under the hood, the bulk invite will create staged users and populate
the user fields of those.
2021-03-29 14:03:19 +03:00
Martin Brennan 2d686191b5
FIX: Bookmark topics were not being updated when the post moved (#12542)
Because bookmarks have both topic and post ID, when the post was moved into another topic the bookmark was still attached to the post but did not show in the UI. This PR makes it so the all topic IDs for bookmarks attached to a post are updated when a post is moved.

Also included is a migration to fix affected records (e.g. on Meta there are 20 affected records).

See: https://meta.discourse.org/t/improved-bookmarks-with-reminders/144542/203
2021-03-29 11:25:48 +10:00
Régis Hanol e8cad4bbf3
PERF: improve category moderators query (#12538)
In the about page, we list a certain number of category moderators.

This rewrites the SQL query used to retrieve the most recent category moderators in order
to perform better with a large number of users/categories/category moderators.

TIL: you can ORDER BY inside an ARRAY_AGG in postgres
TIL: you can slide ARRAYS in postgres
2021-03-28 10:25:30 +02:00
Neil Lalonde eb25d3a954
FIX: add all the keys that should be allowed in user_notifications (#12526) 2021-03-25 14:00:05 -04:00
Dan Ungureanu dffc3a2f8e
DEV: Add tests for invite system (#12524) 2021-03-25 18:26:22 +02:00
Neil Lalonde ca4bc9b8c7
FIX: error overriding user notification string with valid keys (#12510)
When overriding the translation for i18n keys used in user notifications
like user_notifications.reply_by_email, errors were returned for
valid interpolation keys. Keys like topic_title_url_encoded are
supported, so no error should be raised.

https://meta.discourse.org/t/-/50305/7
2021-03-24 15:58:24 -04:00
Mark VanLandingham 371afc45e0
DEV: API for plugins to add post update params and handlers (#12505) 2021-03-24 10:22:16 -05:00
Arpit Jalan c449bf77b3
FIX: new-topic route with sub-category and tags were broken (#12503)
https://meta.discourse.org/t/complex-url-for-creating-new-topics-results-in-server-side-error/184067/4?u=techapj
2021-03-24 19:54:29 +05:30
Dan Ungureanu 2a4ddc621d
FIX: Add migration to set correct redemption_count (#12491)
Redeeming email invites did not increase the redemption_count which let
those invites in a weird state were they were both pending and redeemed.
2021-03-23 18:57:39 +02:00
Bianca Nenciu 437c9a554b
FEATURE: Import and export watched word (#12444)
Find & Replace and Autotag watched words were not completely exported
and import did not work with these either. This commit changes the
input and output format to CSV, which allows for a secondary column.

This change is backwards compatible because a CSV file with only one
column has one value per line.
2021-03-22 22:32:18 +02:00
lhkjacky b7556fca80
DEV: Add ap-east-1 S3 region (#12457)
Asia Pacific (Hong Kong) ap-east-1
2021-03-22 10:22:29 +01:00
Martin Brennan 49f4c548ef
FEATURE: Bookmark pinning (#12431)
Users can now pin bookmarks from their bookmark list. This will anchor the bookmark to the top of the list, and show a pin icon next to it. This also applies in the nav bookmarks panel. If there are multiple pinned bookmarks they sort by last updated order.
2021-03-22 09:50:22 +10:00
Penar Musaraj d470e4fade
FEATURE: Allow users to save draft and close composer (#12439)
We previously included this option conditionally when users were replying
or creating a new topic while they had content already in the composer.

This makes the dialog always include three buttons:
  - Close and discard
  - Close and save draft for later
  - Keed editing

This also changes how the backend notifies the frontend when there is
a current draft topic. This is now sent via the `has_topic_draft`
property in the current user serializer.
2021-03-19 09:19:15 -04:00
Martin Brennan 355d51afde
FEATURE: Allow using invites when DiscourseConnect SSO is enabled (#12419)
This PR allows invitations to be used when the DiscourseConnect SSO is enabled for a site (`enable_discourse_connect`) and local logins are disabled. Previously invites could not be accepted with SSO enabled simply because we did not have the code paths to handle that logic.

The invitation methods that are supported include:

* Inviting people to groups via email address
* Inviting people to topics via email address
* Using invitation links generated by the Invite Users UI in the /my/invited/pending route

The flow works like this:

1. User visits an invite URL
2. The normal invitation validations (redemptions/expiry) happen at that point
3. We store the invite key in a secure session
4. The user clicks "Accept Invitation and Continue" (see below)
5. The user is redirected to /session/sso then to the SSO provider URL then back to /session/sso_login
6. We retrieve the invite based on the invite key in secure session. We revalidate the invitation. We show an error to the user if it is not valid. An additional check here for invites with an email specified is to check the SSO email matches the invite email
7. If the invite is OK we create the user via the normal SSO methods
8. We redeem the invite and activate the user. We clear the invite key in secure session.
9. If the invite had a topic we redirect the user there, otherwise we redirect to /

Note that we decided for SSO-based invites the `must_approve_users` site setting is ignored, because the invite is a form of pre-approval, and because regular non-staff users cannot send out email invites or generally invite to the forum in this case.

Also deletes some group invite checks as per https://github.com/discourse/discourse/pull/12353
2021-03-19 10:20:10 +10:00
Krzysztof Kotlarek c5a116859d
FIX: delete post action from permanently deleted posts (#12309)
When Post is permanently deleted, we should delete correlated PostAction as well.
2021-03-18 15:22:41 +11:00
Osama Sayegh a23d0f9961
UX: Add image uploader widget for uploading badge images (#12377)
Currently the process of adding a custom image to badge is quite clunky; you have to upload your image to a topic, and then copy the image URL and pasting it in a text field. Besides being clucky, if the topic or post that contains the image is deleted, the image will be garbage-collected in a few days and the badge will lose the image because the application is not that the image is referenced by a badge.

This commit improves that by adding a proper image uploader widget for badge images.
2021-03-17 08:55:23 +03:00
Dan Ungureanu fb19ee9eee
FIX: Correctly use invite to topic email templates (#12411)
It was used both when inviting from a topic page and when creating
invites with "Send to topic on first login", while it should be used
only in the former case.
2021-03-16 17:08:54 +02:00
Martin Brennan 139a547f22
DEV: Change Bookmark.clear_reminder! update to update! (#12398)
* Change update to update! for bookmark clear_reminder!

This way the update will not fail silently
2021-03-15 16:10:40 +11:00
Daniel Waterworth 27386ba714
FIX: Kernel.open is deprecated (#12387) 2021-03-12 12:42:32 -06:00
Martin Brennan cf1a80dea6
FIX: Make sure tag-based topic list uses its own preload key (#12354)
When transitioning from a tag topic list e.g. /tag/alerts
to the / route the topic list was not reloaded because the
same preload key was used for both lists (topic_list_latest).
The topic list was only reloaded when clicking on the / route
a second time because then it is forced to reload.

In the topic list adapter, we call `PreloadStore.getAndRemove` to
get the topic lists:

534777f5fd/app/assets/javascripts/discourse/app/adapters/topic-list.js (L34-L41)

Now instead of both / and /tag/alerts sharing the same preload
key of `topic_list_latest`, the tag has a key of `topic_list_tag/alerts/l/latest`
2021-03-12 09:06:21 +10:00
Dan Ungureanu 9c93a62b97
DEV: Update API documentation for invites (#12360) 2021-03-11 18:19:32 +02:00
Roman Rizzi 8fcad73b36
FEATURE: Admins can flag posts so they can review them later. (#12311)
Staff can send a post to the review queue by clicking the "Flag Post" button next to "Take Action...". Clicking it flags the post using the "Notify moderators" score type and hides it. A custom message will be sent to the user.
2021-03-11 08:21:24 -03:00
David Taylor 0902e56162
FEATURE: Flag to disable DiscourseConnect CSRF protection (#12356)
This is not recommended. But if you have other protections in place for CSRF mitigation, you may wish to disable Discourse's implementation. This site setting is not visible in the UI, and must be changed via the console.
2021-03-11 10:38:34 +00:00
Kris 83519e7f3a
FEATURE: Always show tag dropdown when tagging is enabled (#12209) 2021-03-10 21:13:15 -05:00
Penar Musaraj 10780d2448
DEV: support json_schema in theme settings (#12294) 2021-03-10 20:15:04 -05:00
Bianca Nenciu 92ad2182f5
FIX: Validate user email only once (#12327)
It was validated twice: once by validates_associated and once
by has_many.
2021-03-10 14:49:13 +02:00
Alan Guo Xiang Tan ce04db8610 FEATURE: Allow invites redemption with Omniauth providers. 2021-03-09 09:27:18 +08:00
Dan Ungureanu fecf3e20d9
FEATURE: Various improvements to invite system (#12314)
* FEATURE: Do not delete invite if link was copied

* FIX: Show error to user if invite redeeming fails

The error was only displayed to console.

* UX: Better placement of bulk buttons

Destroy all expired invites should be on the expired tab, not pending.

* FIX: Ensure invited_groups is unique per invite and group

* FIX: Do not refresh topic list if title unchanged

* FIX: Do not close modal on enter

This intereferes with the group and topic chooser.
Wrapping everything in a form disables this behavior.

* FIX: Move link and email options outside advanced section

* FIX: Do not close modal if saving a link invite

User may still want to copy the link.
2021-03-09 00:15:14 +02:00
Dan Ungureanu 7f3240ea31
FEATURE: Various improvements to invite system (#12298)
* FIX: Do not show expired invites under Pending tab

* DEV: Controller action was renamed in previous commit

* FEATURE: Add 'Expired' tab to invites

* FEATURE: Refresh model after removing expired invites

* FEATURE: Do not immediately add invite to the list

Opening the 'create-invite' modal used to automatically generate an
invite to reserve an invite link. If the user did not save it and
closed the modal, the invite would be destroyed. This operations caused
the invite list to change in the background and confuse users.

* FEATURE: Sort redeemed users by creation time

* UX: Improve show / hide advanced options link

* FIX: Show redeemed users even if invites were trashed

* UX: Change modal title when editing invite

* UX: Remove Get Link button

Users can get it from the edit modal

* FEATURE: Add limit for invite links generated by regular users

* FEATURE: Add option to skip email

* UX: Show better error messages

* FIX: Show "Invited by" even if invite was trashed

Follow up to 1fdfa13a099d8e46edd0c481b3aaaafe40455ced.

* FEATURE: Add button to save without sending email

Follow up to c86379a465f28a3cc64a4a8c939cf32cf2931659.

* DEV: Use a buffer to hold all changed data

* FEATURE: Close modal after save

* FEATURE: Rate limit resend invite email

* FEATURE: Make the save buttons smarter

* FEATURE: Do not always send email even for new invites
2021-03-06 13:29:35 +02:00
Daniel Waterworth fae2fc0b5e
FIX: Race condition in draft sequence updates (#12299)
This was causing RecordNotUnique exceptions
2021-03-05 13:40:00 -06:00
Krzysztof Kotlarek e076506135
FIX: new topic indicator on the mobile categories page (#12271)
Regression with new dismiss button. We need the same solution to indicate if the topic was seen for category_list as done for topic_list:
https://github.com/discourse/discourse/blob/master/app/models/topic_list.rb#L123

Meta: https://meta.discourse.org/t/dismissed-new-topics-still-show-blue-dots-in-categories-view/181596
2021-03-05 09:04:19 +11:00
David Taylor 5a4d3e7576
FIX: Ensure UserField changes are reflected instantly in webhooks (#12291)
The Guardian object memoizes a list of allowed user fields. Normally this is fine because Guardian objects only persist for a single request. However, the WebHook class was memoizing a guardian at the class level. This meant that an app restart was required for changes to be reflected. Plus, the Guardian was being shared across all sites in a multisite instance.

Initializing a guardian is cheap, so we can manage without memoization here.
2021-03-04 21:41:57 +00:00
Krzysztof Kotlarek ae3839580e
FIX: message bus when group private message (#12269)
When the private message was addressed to the group.
Group members didn't receive MessageBus messages about new posts.
To see content, they needed to refresh the page.

Meta: https://meta.discourse.org/t/group-private-message-message-bus-issue/181009/7
2021-03-04 08:33:58 +11:00
Dan Ungureanu c047640ad4
FEATURE: Various improvements to invite system (#12023)
The user interface has been reorganized to show email and link invites
in the same screen. Staff has more control over creating and updating
invites. Bulk invite has also been improved with better explanations.

On the server side, many code paths for email and link invites have
been merged to avoid duplicated logic. The API returns better responses
with more appropriate HTTP status codes.
2021-03-03 11:45:29 +02:00
Bianca Nenciu b49b455e47
FEATURE: Autotag watched words (#12244)
New topics with be matched against a set of watched words and be
tagged accordingly.
2021-03-03 10:53:38 +02:00
Rafael dos Santos Silva 83f332b5a5
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180) 2021-03-02 16:04:16 -03:00
Gerhard Schlager 11ccea5c9a
FIX: `Topic#invite_group` failed to notify users on newly created topic (#12255)
When `PostCreator` creates a new topic it loads the `allowed_groups` of the topic. `Fabricate` doesn't do that and that's why the existing spec worked even though it should have failed, because `PostAlerter#notify_group_summary` didn't create a notification for a non-fabricated topic.

`Topic#invite_group` added a new `TopicAllowedGroup` record without reloading `Topic.allowed_groups`. A subsequent call to `PostAlerter#notify_group_summary` didn't work because it didn't find the invited group in the topic's `allowed_groups` association.
2021-03-02 15:46:50 +01:00
Osama Sayegh c0e2fdd200
FIX: Components mobile-specific CSS was missing (#12259)
Fix for: https://meta.discourse.org/t/our-components-stop-working/181580?u=osama.

This fixes an old hidden bug that was exposed in cf0192018e. The bug is that we call the `Stylesheet::Manager.stylesheet_details` method with the `target` arg as `:mobile_theme` when we want to retrieve a theme component's mobile CSS. The problem is that this `target` value will at some point be looked up in the `Theme.targets` enum which doesn't have a `:mobile_theme` key, instead it has `:mobile` key.

This commit adds a step that removes the `_theme` suffix in the `Theme.list_baked_fields` method to fix this problem.
2021-03-02 17:20:43 +03:00
Bianca Nenciu 6f0397fba0
DEV: Remove useless ORDER clause (#12247)
These results are combined with others and then ordered in Ruby.
2021-03-02 11:55:07 +11:00
jbrw a9b6f4d829
FIX - use ImageMagick to determine size of svg images (#12230)
SVG files can have dimensions expressed in inches, centimeters, etc., which may lead to the dimensions being misinterpreted (e.g. “8in” ends up as 8 pixels).

If the file type is `svg`, ask ImageMagick to work out what size the SVG file should be rendered on screen.

NOTE: The `pencil.svg` file was obtained from https://freesvg.org/1534028868, which has placed the file in to the public domain.
2021-03-01 11:44:00 -05:00
Penar Musaraj bfa3e24e48
FIX: Update digest when updating color definitions in theme component (#12233) 2021-03-01 09:14:58 -05:00
Penar Musaraj cf0192018e
FIX: Do not output empty style tags for components (#12229) 2021-03-01 09:14:25 -05:00
Blake Erickson f53546c03d
FIX: Exclude users with posts from purge_unactivated query (#12231)
Unactivated users that have posts cannot be deleted so we shouldn't
include them in the initial query to try and purge them. Otherwise we
are just loading up sidekiq with pointless work to be doing every day.

Without this change if there are 201 unactivated users to purge, but the
first 200 have posts, the 201st user will never be deleted even though
it is the only user that doesn't have a post and is actually the one
that should be deleted.
2021-03-01 16:46:28 +11:00
Penar Musaraj aa1442fdc3
DEV: Use separate files for theme component stylesheets (take 2) (#12225)
This switches to outputting a separate file for each theme component CSS
asset. We have separate CSS plugin files, separate JS files
(for plugins/themes/components), it makes sense to do the same for
component CSS assets.

Benefits:
- easier debugging
- fixes a regression with theme component sourcemaps
- changes to theme components are updated individually

With HTTP/2, there is also no performance downside to having additional
files in the initial request.
2021-02-26 12:30:23 -05:00
Penar Musaraj e74bdfdf8e
Revert "DEV: Use separate files for theme component stylesheets (#12214)" (#12224)
This reverts commit f57a49c2f9.

This had some unexpected side effects, needs some more work.
2021-02-26 08:20:39 -05:00
Penar Musaraj 45c5fd2172
DEV: Remove JoyPixels emoji option (#12197)
- removes the option from site settings
- deletes the site setting on existing sites that have it
- marks posts using emojis as requiring a rebake

Note that the actual image files are not removed here, the plan is to
remove them in a few weeks/months (when presumably the rebaking of old
posts has been completed).
2021-02-26 07:44:52 -05:00
Penar Musaraj f57a49c2f9
DEV: Use separate files for theme component stylesheets (#12214)
This switches to outputting a separate file for each theme component CSS
asset. We have separate CSS plugin files, separate JS files
(for plugins/themes/components), it makes sense to do the same for
component CSS assets.

Benefits:
- easier debugging
- fixes a regression with theme component sourcemaps
- changes to theme components are updated individually

With HTTP/2, there is also no performance downside to having additional
files in the initial request.
2021-02-26 07:44:15 -05:00
Roman Rizzi bb3d5e9758
FEATURE: Filter reviewables by id. (#12213)
The API now accepts an array called "ids" to select specific items. This parameter is not present on the UI.

Example usage: "yoursite.com/review.json?ids[]=1&ids[]=2"
2021-02-26 07:56:14 -03:00
Arpit Jalan 93a0a906b5
FIX: allow adding user to PM when inviter is in allowed list (even (#12212)
though other participants are not in allowed list)

If you create an allowlist of users who can PM you, and use the function
“Only specific users can send me private messages”, then you can’t be
added to group messages unless everyone in that message is already in
your allow list.

This commit allows user to be added to a group message even when other
participants are not in allowed list
2021-02-25 22:56:49 +05:30
Bianca Nenciu 533800a87b
Add watched words of type "replace" (#12020)
This commit includes other various improvements to watched words.

auto_silence_first_post_regex site setting was removed because it overlapped
with 'require approval' watched words.
2021-02-25 14:00:58 +02:00
Joffrey JAFFEUX de32fa299d
DEV: adds DiscourseEvent - topic_first_visited_by_user (#12195)
This event would mostly allow plugins to create workflows once users have visited a specific topic.
2021-02-24 10:50:02 +01:00
Roman Rizzi 95d3877709
FIX: Don't update `posts_read_count` when the post is from a PM. (#12131)
We don't want TL0 users doing the discobot tutorial to increase their read count.
2021-02-23 11:36:00 -03:00
Roman Rizzi e485e95792
FIX: Adding a custom scope should not modify the original ones. (#12178)
Default scopes are stored inside a class variable, which shouldn't be modified when a custom scope is added. If this happens, we're no longer to remove the scope when the plugin is disabled.
2021-02-22 20:10:53 -03:00
Arpit Jalan f75e1867ff
FIX: do not send rejection emails to auto-deleted reviewable users (#12160)
FIX: add context when user is deleted via auto handle queued reviewable
FIX: do not delete email_log when a user is deleted
2021-02-22 18:37:47 +05:30
David Taylor ef19431e44
DEV: Improve User#email= behavior (#11338)
- Only apply the change after `save` is called on the record
- Automatically remove matching secondary emails
2021-02-22 11:42:37 +00:00
Penar Musaraj 5604ce70d4
DEV: More refactoring of SCSS importers (#12143) 2021-02-19 11:22:24 -05:00
Roman Rizzi aac9e6cb0a
FIX: Don't require a rejection reason if the user is a spammer. (#12141) 2021-02-19 12:57:01 -03:00
Rafael dos Santos Silva d86cb7d1c3
FIX: Use the S3 CDN for the Group flair (#12130) 2021-02-18 17:24:44 -03:00
Osama Sayegh 69017298e8
FIX: Limit post read time to the max integer value (#12126)
Some users somehow manage to keep a topic open for a very long time that it causes the post read time to exceed the max integer value (2^31 - 1) which causes errors when we try to update the read time in the database to values above the integer limit.

This PR will cap posts read time at 2^31 - 1 to prevent these errors.
2021-02-18 17:48:15 +03:00
David Taylor 13d2a1f82c
SECURITY: Attach DiscourseConnect (SSO) nonce to current session (#12124) 2021-02-18 10:35:10 +00:00
Martin Brennan e0f0fe5624
FIX: Bump max topic timer duration to 20 years (#12107)
This way it has some sort of cap, even if it seems pretty
high, and we don't have to worry about requests for increasing
it from 2 to 5 to 10 etc.
2021-02-18 11:52:30 +10:00
Roman Rizzi 07cf0f9460
FIX: Allow plugins to correctly extend API key scopes. (#12113)
Adding a scope from a plugin was broken. This commit fixes it and adds a test.

It also documents the instance method and renames the serialized "id" attribute to "scope_id" to avoid a conflict when the scope also has a parameter with the same name.
2021-02-17 14:42:44 -03:00
David Taylor 9d7dc89d79
DEV: Replace deprecated sso site settings with discourse_connect (#12099) 2021-02-17 09:31:20 +00:00
Martin Brennan fb83757edb
FIX: Auto close topic from category settings based on topic created_at (#12082)
Previously when inheriting category auto-close settings for a topic, those settings were disrupted if another topic timer was assigned or if a topic was closed then manually re-opened.

This PR makes it so that when a topic is manually re-opened the topic auto-close settings are inherited from the category. However, they will now be based on the topic created_at date. As an example, for a topic with a category auto close hours setting of 72 (3 days):

* Topic was created on 2021-02-15 08:00
* Topic was closed on 2021-02-16 10:00
* Topic was opened again on 2021-02-17 06:00

Now, the topic will inherit the auto close timer again and will close automatically at **2021-02-18 08:00**, which is based on the creation date. If the current date and time is greater than the original auto-close time (e.g. we were at 2021-02-20 13:45) then no auto-close timer is created.

Note, this will not happen if the topic category auto-close setting is "based on last post".
2021-02-17 07:51:39 +10:00
Bianca Nenciu fad1fac196
FIX: Update topic_count when updating visibility (#11946)
Updating a topic's visibility did not increase or decrease the
topic_count of a category, but Category.update_stats does ignore
unlisted topics which resulted in inconsistencies when deleting
such topics.
2021-02-16 17:45:12 +02:00
Krzysztof Kotlarek ad3ec5809f
FIX: Dismiss new with better migration (#12062)
Original PR was reverted because of broken migration https://github.com/discourse/discourse/pull/12058

I fixed it by adding this line
```
          AND topics.id IN(SELECT id FROM topics ORDER BY created_at DESC LIMIT :max_new_topics)
```

This time it is left joining a limited amount of topics. I tested it on few databases and it worked quite smooth
2021-02-15 08:50:33 +11:00
Martin Brennan ff1ac57feb
FIX: Validate duration minutes values for topic timer (#12040)
Add server and client side validations to ensure topic timer durations cannot exceed 2 years and cannot be less than or equal to 0.
2021-02-12 09:05:14 +10:00
Krzysztof Kotlarek a696cc07d2
Revert "FEATURE: Ability to dismiss all new topics (#12018)" (#12058)
This reverts commits 7426764af4 and f5b18e2a31
2021-02-12 08:50:25 +11:00
Blake Erickson 395a903cf6
DEV: Show warning message when using ember css selectors (#12036)
* DEV: Show warning message when using ember css selectors

When editing the theme css via the admin UI a warning message
will be displayed if it detects that the `#emberXXX` or `.ember-view`
css selectors are being used. These are dynamic selectors that ember
generates, but they can change so they should not be used.

* Update error message text to be more helpful

* Display a warning instead of erroring out

This allows the theme to still be saved, but a warning is displayed.

Updated the tests to check for the error message.

Updated the pre tags css so that it wraps for long messages.
2021-02-11 13:48:57 -07:00
Krzysztof Kotlarek f5b18e2a31
FEATURE: Ability to dismiss all new topics (#12018)
Follow up https://github.com/discourse/discourse/pull/11968

Dismiss all new topics using the same DismissTopicService. In addition, MessageBus receives exact topic ids which should be marked as `seen`.
2021-02-11 13:35:09 +11:00
Krzysztof Kotlarek 354ec6694a
FEATURE: Ability to dismiss new topics in a specific tag (#11968)
* FEATURE: Ability to dismiss new topics in a specific tag

Follow up of https://github.com/discourse/discourse/pull/11927

Using the same mechanism to disable new topics in a tag.

* FIX: respect when category and tag is selected
2021-02-09 10:39:30 +11:00
David Taylor 821bb1e8cb
FEATURE: Rename 'Discourse SSO' to DiscourseConnect (#11978)
The 'Discourse SSO' protocol is being rebranded to DiscourseConnect. This should help to reduce confusion when 'SSO' is used in the generic sense.

This commit aims to:
- Rename `sso_` site settings. DiscourseConnect specific ones are prefixed `discourse_connect_`. Generic settings are prefixed `auth_`
- Add (server-side-only) backwards compatibility for the old setting names, with deprecation notices
- Copy `site_settings` database records to the new names
- Rename relevant translation keys
- Update relevant translations

This commit does **not** aim to:
- Rename any Ruby classes or methods. This might be done in a future commit
- Change any URLs. This would break existing integrations
- Make any changes to the protocol. This would break existing integrations
- Change any functionality. Further normalization across DiscourseConnect and other auth methods will be done separately

The risks are:
- There is no backwards compatibility for site settings on the client-side. Accessing auth-related site settings in Javascript is fairly rare, and an error on the client side would not be security-critical.
- If a plugin is monkey-patching parts of the auth process, changes to locale keys could cause broken error messages. This should also be unlikely. The old site setting names remain functional, so security-related overrides will remain working.

A follow-up commit will be made with a post-deploy migration to delete the old `site_settings` rows.
2021-02-08 10:04:33 +00:00
Martin Brennan 4af77f1e38
FEATURE: Allow durations < 1 hour and < 1 day for topic timers where duration is specified (auto delete replies, close based on last post) (#11961)
This PR allows entering a float value for topic timers e.g. 0.5 for 30 minutes when entering hours, 0.5 for 12 hours when entering days. This is achieved by adding a new column to store the duration of a topic timer in minutes instead of the ambiguous both hours and days that it could be before.

This PR has ommitted the post migration to delete the duration column in topic timers; it will be done in a subsequent PR to ensure that no data is lost if the UPDATE query to set duration_mintues fails.

I have to keep the old keyword of duration in set_or_create_topic_timer for backwards compat, will remove at a later date after plugins are updated.
2021-02-05 10:12:56 +10:00
Penar Musaraj 12ffba771c
FIX: Improve SCSS handling in components (#11963)
- ignores errors when including component SCSS in parent theme
- adds support for SCSS `@import`s in components' `color_definitions.scss` files
2021-02-04 08:51:18 -05:00
Krzysztof Kotlarek f39e7fe81d
FEATURE: New way to dismiss new topics (#11927)
This is a try to simplify logic around dismiss new topics to have one solution to work in all places - dismiss all-new, dismiss new in a specific category or even in a specific tag.
2021-02-04 11:27:34 +11:00
Jarek Radosz 8ad5284cf7
FIX: Don't create email invites when SSO is on or local logins are off (#11951)
A more general, lower-level change in addition to #11950.

Most code paths already check if SSO is enabled or if local logins are disabled before trying to create an email invite.
This is a safety net to ensure no invalid invites sneak by. 

Also includes:
FIX: Don't allow to bulk invite when SSO is on (or when local logins are disabled)
This mirrors can_invite_to_forum? and other email invite code paths.
2021-02-03 19:01:23 +01:00
Jarek Radosz 39a9651847
DEV: Remove `update_mail_receiver` and `deprecated_api_usage` (#11947)
`update_mail_receiver`-related code was removed in 269ec70ba8 and `deprecated_api_usage`-related code was removed in d04ba4b3b2.
2021-02-03 17:41:47 +01:00
Penar Musaraj e8b82724fd
DEV: Refactor theme SCSS compilation (#11919) 2021-02-02 13:09:41 -05:00
Gerhard Schlager d055552994
Fix i18n issues reported on Crowdin (#11747)
* Pluralize `groups.errors.adding_too_many_users`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/248/en-ar#53882

* Pluralize `js.composer.error.title_too_short`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#41172

* Pluralize `js.composer.error.title_too_long`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#41174

* Pluralize `js.composer.error.post_length`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#41178

* Pluralize `js.topic.progress.jump_prompt_of`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#41958

* Use translations to join strings about posters
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/248/en-ar#49334
  It also makes some changes to the crawler view:
    * Removes `poster.moreCount` which is only available on the client for PMs
    * CSS class names are actually stored in `poster.extras` instead of `poster.extraClasses`

* Stop concatenating category stats
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#40740

* Pluralize `js.summary.description`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#40782

* Pluralize `js.summary.description_time_MF`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#40784

* Use translation to join list of tags
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#43372

* Pluralize `admin_js.admin.groups.manage.membership.automatic_membership_user_count`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#43720

* Pluralize `js.post.controls.delete_topic_confirm_modal`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#54804

* Stop concatenating `js.post.last_edited_on`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#42358

* Stop concatenating `js.post.wiki_last_edited_on`
  This fixes https://discourse.crowdin.com/translate/f3230e7607a36bb0a2f97fd90605a44e/246/en-ar#42356
  It also fixes a regression because `js.post.wiki_last_edited_on` wasn't used anymore since 2017.
2021-02-02 10:50:04 +01:00
romanrizzi 736b4137cc FIX: Return the CDN's URL when using the site's small logo as the system's user avatar.
Using "UrlHelper#absolute" returns the S3 URL, which is fine for the client because it modifies it to use the CDN instead. On the other hand, this replacement doesn't happen when the URL is server-side rendered, returning a 403 for the system's avatar.
2021-02-01 10:31:36 +08:00
Martin Brennan 4af4d36175
FIX: IMAP allow unknown senders to reply to group topics via email (#11877)
Adds a new column/setting to groups, allow_unknown_sender_topic_replies, which is default false. When enabled, this scenario is allowed via IMAP:

* OP sends an email to the support email address which is synced to a group inbox via IMAP, creating a group topic
* Group user replies to the group topic
* An email notification is sent to the OP of the topic via GroupSMTPMailer
* The OP has several email accounts and the reply is sent to all of them, or they forward their reply to another email account
* The OP replies from a different email address than the OP (gloria@gmail.com instead of gloria@hey.com for example)
* The a new staged user is created, the new reply is accepted and added to the topic, and the staged user is added to the topic allowed users

Without allow_unknown_sender_topic_replies enabled the new reply creates an entirely new topic (because the email address it is sent from is not previously part of the topic email chain).
2021-01-29 09:59:10 +10:00
Martin Brennan f49e3e5731
DEV: Add security_last_changed_at and security_last_changed_reason to uploads (#11860)
This PR adds security_last_changed_at and security_last_changed_reason to uploads. This has been done to make it easier to track down why an upload's secure column has changed and when. This necessitated a refactor of the UploadSecurity class to provide reasons why the upload security would have changed.

As well as this, a source is now provided from the location which called for the upload's security status to be updated as they are several (e.g. post creator, topic security updater, rake tasks, manual change).
2021-01-29 09:03:44 +10:00
Mark VanLandingham 809274fe0d
DEV: Replace 'processed' column on notifications with new table (#11864) 2021-01-27 10:29:24 -06:00
David Taylor 1ccb6583b1
FIX: Do not cache translated trust level names in site settings (#11863)
There's no real need to cache these, and the caching can introduce
problems when different sites/users are using different locales.
2021-01-27 14:31:19 +00:00
Régis Hanol f421d9bdd6
FIX: only de-prioritise exact matches in mentions (#11843)
Not when doing a site-wide search like we do in the Directory.

This solves the following specfailure:

  1) DirectoryItemsController with data finds user by name
     Failure/Error: expect(json['directory_items'].length).to eq(1)

       expected: 1
            got: 0

       (compared using ==)
     # ./spec/requests/directory_items_controller_spec.rb:88:in `block (3 levels) in <main>'
     # ./spec/rails_helper.rb:271:in `block (2 levels) in <top (required)>'
     # ./bundle/ruby/2.7.0/gems/webmock-3.11.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
2021-01-25 21:27:15 +01:00
Régis Hanol 27656f5c84
FIX: un-prioritise inactive users in user search (#11838)
When doing a user search (eg. when mentioning a user) we will not prioritie
users who hasn't been seen in over a year.

REFACTOR the user-search specs to be more precise regarding the ordering
2021-01-25 20:33:11 +01:00
Régis Hanol aa1138ff71
FIX: reindex_search job should work on model with no search data (#11819)
Lots of changes but it's mostly a refactoring.

The interesting part that was fix are the 'load_problem_<model>_ids' methods.
They will now return records with no search data associated so they can be properly indexed for the search.
This "bad" state usually happens after a migration.
2021-01-25 11:23:36 +01:00
Mark VanLandingham 56294b4fba
FIX: Remove scheduled DND timings when schedule is disabed (#11814) 2021-01-22 13:02:11 -06:00
Bianca Nenciu d2cf43a7d5
FIX: Update categories without full page refresh (#11793)
Creating or moving a category required a full page refresh until it
showed up correctly.
2021-01-22 10:21:09 +02:00
Vinoth Kannan 872f3e6934
UX: warn about messages to be orphaned while deleting a group. (#11727)
Currently, after destroying a group its messages are inaccessible to everyone. Only admins can access using direct URLs.
2021-01-22 03:29:34 +05:30
Martin Brennan 9ee8a01c3a
FIX: Change default for IncomingEmail#created_via to 0 (unknown) and make NOT NULL (#11782)
Follow up to https://review.discourse.org/t/dev-add-created-via-column-to-incomingemail-pr-11751/18366/6
2021-01-21 12:59:50 +10:00
Martin Brennan f34fa999a2
DEV: IMAP debugging improvements (#11784)
Improvements to make console access to IncomingEmail more pleasant, and stopping certain IMAP logs from landing in the DB because they just create too much noise,
2021-01-21 11:37:47 +10:00
Gerhard Schlager 3b2f6e129a
FEATURE: Add English (UK) as locale (#11768)
* "English" gets renamed into "English (US)"
* "English (UK)" replaces "English"

@discourse-translator-bot keep_translations_and_approvals
2021-01-20 21:32:22 +01:00
Mark VanLandingham 1a7922bea2
FEATURE: Create notification schedule to automatically set do not disturb time (#11665)
This adds a new table UserNotificationSchedules which stores monday-friday start and ends times that each user would like to receive notifications (with a Boolean enabled to remove the use of the schedule). There is then a background job that runs every day and creates do_not_disturb_timings for each user with an enabled notification schedule. The job schedules timings 2 days in advance. The job is designed so that it can be run at any point in time, and it will not create duplicate records.

When a users saves their notification schedule, the schedule processing service will run and schedule do_not_disturb_timings. If the user should be in DND due to their schedule, the user will immediately be put in DND (message bus publishes this state).

The UI for a user's notification schedule is in user -> preferences -> notifications. By default every day is 8am - 5pm when first enabled.
2021-01-20 10:31:52 -06:00
Bianca Nenciu 15da528870
FIX: Support for single use invite links (#11719)
The "invite link" checked if the number of uses was greater than 1. This
is not always true as single use invite links are perfectly valid.
2021-01-20 10:50:02 +02:00
Martin Brennan fb184fed06
DEV: Add created_via column to IncomingEmail (#11751)
This should make it easier to track down how the incoming email was created, which is one of four locations:

The POP3 poller (which picks up reply via email replies)
The admin email controller #handle_mail (which is where hosted mail is sent)
The IMAP sync tool
The group SMTP mailer, which sends emails when replying to IMAP topics, pre-emptively creating IncomingEmail records to avoid double syncing
2021-01-20 13:22:41 +10:00
Martin Brennan 0034cbda8a
DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698)
Moves the topic timer jobs from being scheduled ahead of time with enqueue_at to a 5 minute scheduled run like bookmark reminders, in a new job called Jobs::EnqueueTopicTimers. Backwards compatibility is maintained by checking if an existing topic timer job is enqueued in sidekiq for the timer, and if it is not running it inside the new job.

The functionality to close/open a topic if it is in the opposite state still remains in the after_save block of TopicTimer, with further commentary, which is used for Open/Close Temporarily.

This also removes the ensure_consistency! functionality of topic timers as it is no longer needed; the new job will always pick up the timers because they are not stored in a fragile state of sidekiq.
2021-01-19 13:30:58 +10:00
Martin Brennan be145ccf2f
FIX: Add bookmark limits (#11725)
Adds a bookmark search per page limit, a total bookmark creation limit, and a rate limit per day for bookmark creation.
2021-01-19 08:53:49 +10:00
Roman Rizzi 7ac9a4d2ec
FIX: Allow users to opt-out of using the site's small logo as the system user avatar. (#11744) 2021-01-18 14:09:07 -03:00
Martin Brennan bd25627198
FIX: IMAP post alerter race condition and code improvements (#11711)
This PR fixes a race condition with the IMAP notification code. In the `Email::Receiver` we call the `NewPostManager` to create the post and enqueue jobs and sends alerts via `PostAlerter`. However, if the post alerter reaches the `notify_pm_users` and the `group_notifying_via_smtp` method _before_ the incoming email is updated with the post and topic, we unnecessarily send a notification to the person who just posted. The result of this is that the IMAP syncer re-imports the email sent to the user about their own post, which looks like this in the group inbox:

To fix this, we skip the jobs enqueued by `NewPostManager` and only enqueue them with `PostJobsEnqueuer` manually _after_ the incoming email record has been updated with the post and topic.

Other improvements:

* Moved code to calculate email addresses from `IncomingEmail` records into the topic, with a group passed in, for easier testing and debugging. It is not the responsibility of the post alerter to figure this stuff out.
* Add shortcut methods on `IncomingEmail` to split or provide an empty array for to and cc addresses to avoid repetition.
2021-01-15 10:54:46 +10:00
Krzysztof Kotlarek 06b7c44593
FEATURE: reason to reject user signup (#11700)
Feature for `Must Approve Users` setup. When a user is rejected, a staff member can optionally set a reason for audit purposes. In addition, feedback email can be sent to the user.

Meta: https://meta.discourse.org/t/account-rejection-email/103112/8
2021-01-15 09:43:26 +11:00
Martin Brennan 2404fa7a23
DEV: Split toggle topic close job (#11679)
Splits the `ToggleTopicClosed` job into two distinct `OpenTopic` and `CloseTopic` jobs to make the code clearer. The old job cannot be deleted yet because of outstanding sidekiq schedules, so a todo has been added to do so later this year.

Also replaced mentions of `topic_status_update` with `topic_timer` in some files, because the `topic_status_update` model is obsolete and replaced by topic timer.

Added some shortcut methods for checking if a topic is open/whether a user can change an open topic.
2021-01-13 08:49:29 +10:00
Bianca Nenciu ec0212e56b
FIX: Make category slugs lowercase (#11277)
Admins could specify category slug with upper case characters and same slug,
but with different cases could be used simultaneously.
2021-01-12 17:28:33 +02:00
Mark VanLandingham 3eae4d3669
FIX: Publish DND ends_at updates in httpdate format (#11683) 2021-01-11 14:11:26 -06:00
Robin Ward 49b753eee0
FIX: Hiding a post should not trigger the duplicate check (#11680)
If for some reason a post was allowed to be duplicated (probably via
staff edit) hiding it should be permitted.
2021-01-11 14:56:08 -05:00
Krzysztof Kotlarek efaa63bd1d
FIX: always truncate uncomplete emojis in excerpts (#11678)
Additional fix after https://github.com/discourse/discourse/pull/11667

Always truncate "broken" emojis from excerpts.
2021-01-11 13:43:11 +11:00
Krzysztof Kotlarek 7f78b6ec10
FIX: broken emojis in topic excerpt (#11667)
When a post is truncated into the excerpt, sometimes we are breaking code of emoji
2021-01-11 10:40:41 +11:00
Roman Rizzi caa17386ee
FEATURE: Use the site's small logo as the system user's avatar. (#11661) 2021-01-08 10:40:00 -03:00
Mark VanLandingham 4601f3be7e
FEATURE: Send notification emails when users leave do not disturb mode (#11643) 2021-01-07 10:49:49 -06:00
Penar Musaraj c819284660
UX: Improve color scheme choices in user prefs (#11656) 2021-01-07 11:15:38 -05:00
Martin Brennan 64ba5b1d21
FIX: Group SMTP email improvements (#11633)
Fixes a rare race condition causing the `Imap::Sync` class to create an incoming email and associated post/topic, which then kicks off the PostAlerter to notify others in the PM about a reply in the topic, but for the OP which is not necessary (because the person emailing the IMAP inbox already knows about the OP). Basically, we should never be sending the group SMTP email for the first post in a topic.

Also in this PR:

* Custom attribute accessors for the to/from/cc addresses on `IncomingEmail`, to parse them from an array to a joined string so the logic for this is only in one place.
* Store extra detail against the `IncomingEmail` created in `GroupSmtpMailer`
* regex test Mail header Reply-To as string instead of Field, which fixes `warning: deprecated Object#=~ is called on Mail::Field; it always returns nil`
* Add DEBUG_IMAP to log all IMAP logs as warnings for easier debugging
* Changed the Rails logging to `ImapSyncLog` in the `GroupSmtpMailer`
2021-01-05 15:32:04 +10:00
David Taylor 13e39d8b9f
PERF: Improve cook_url performance for topic thumbnails (#11609)
- Only initialize the S3Helper when needed
- Skip initializing the S3Helper for S3Store#cdn_url
- Allow cook_url to be passed a `local` hint to skip unnecessary checks
2020-12-30 18:13:13 +00:00
Rafael dos Santos Silva 8da0b4979b
FIX: S3 CDN warning was showing wrong warning message (#11603) 2020-12-29 14:09:55 -03:00
Sam 53f4d54f23
PERF: add indexes to speed up profile pages (#11598)
These 2 indexes optimise performance on profile pages.

The summary page displays:

1. A list of "Top Link" - links sorted by number of clicks posted by user
2. A list of "Top Replies" - replies made by a user that go the most hearts

These two areas could devolve into full index or table scans, new indexes are there to avoid this cost on large dbs

One minor downside is that storage requirements go a tiny bit up to maintain the new indexes
2020-12-29 15:54:05 +11:00
Rafael dos Santos Silva 330ef3700f
FEATURE: Add dash warning for using S3 without CDN (#11561)
Over the years we had some admins complaining that most of their costs
running Discourse is on S3 uploads.

That only happens when they are using S3 without a CDN, which is widely
considered a bad pratice for public websites, as egress bandwidth can
get expensive quite fast.

Related:

https://meta.discourse.org/t/download-avatars-locally-from-s3-shutting-down-s3-in-my-site/174001/3?u=falco

https://meta.discourse.org/t/use-webtorrent-to-load-media-objects/155873/2?u=falco
2020-12-28 12:43:48 -03:00
Krzysztof Kotlarek f84ff26aa9
FIX: use Redis replica host and port (#11566)
Introduce Redis `replica` config and deprecate `slave`
2020-12-23 13:14:19 +11:00
Krzysztof Kotlarek 07bf7a91f4
Revert "FIX: use Redis replica host and port (#11543)" (#11552)
This reverts commit b0e1210b0c.
2020-12-22 16:16:50 +11:00
Krzysztof Kotlarek b0e1210b0c
FIX: use Redis replica host and port (#11543)
* FIX: use Redis replica host and port

Introduce Redis `replica` config and deprecate `slave`

* FIX: move deprecations to separate file
2020-12-22 15:52:00 +11:00
David Taylor d25fd34b44
DEV: Remove with_deleted workarounds for old Rails version (#11550)
* DEV: Remove with_deleted workarounds for old Rails version

These workarounds using private APIs are no longer required in the latest version of Rails. The referenced issue (https://github.com/rails/rails/issues/4306) was closed in 2013. The acts_as_paranoid workaround which this was based on was removed for rails > 5.

Switching to using a scope also allows us to use it within a `belongs_to` relation (e.g. in the Poll model). This avoids issues which can be caused by unscoping all `where` clauses.

Predicates are not necessarily strings, so calling `.join(" AND ")` can sometimes cause weird errors. If we use `WhereClause#ast`, and then `.to_sql` we achieve the same thing with fully public APIs, and it will work successfully for all predicates.
2020-12-22 10:38:59 +11:00
Rafael dos Santos Silva 32e7ee4867
FEATURE: Adds support for the WEBP image format (#11532)
Adds support for both still and animated webp images.
Animated images won't be optimized or get thumbnails.

Depends on https://github.com/discourse/discourse_docker/pull/506
and https://github.com/discourse/discourse_docker/pull/507
2020-12-21 20:09:00 -03:00
Mark VanLandingham 649ed24bb4
FEATURE: Do not disturb (#11484) 2020-12-18 09:03:51 -06:00
Bianca Nenciu 806f05f851
FIX: Make find_by_slug_path work with default slugs (#11501)
Default slugs are generated by adding '-category' to category ID.
2020-12-18 16:05:01 +02:00
David Taylor c54217f33d
FIX: Clear in-process theme cache after clearing DB cache (#11517)
If we clear the in-process cache first, it might get re-filled from the
DB before we clear the DB cache. This would be more likely on high-traffic
sites.
2020-12-17 11:02:41 +11:00
Bmorrical e7b837e7e1 Have doubled the size of the Watched Words Per Action Limit 2020-12-16 13:37:10 -05:00
Krzysztof Kotlarek 6dd3f986b2
FIX: no email when the category is muted and mailing list is on (#11490)
When `mute_all_categories_by_default` setting is enabled we should not send mailing list until category, tag or topic is explicitly watched.
2020-12-16 09:30:21 +11:00
Roman Rizzi c7b9f044a4
FEATURE: Automatically disable slow mode. (#11461)
Staff and TL4 users can decide the slow mode duration. We'll internally set a topic timer to disable it.
2020-12-14 14:06:50 -03:00
Robin Ward a51a06115a FIX: Posts with Staff Colors were excluded from the group activity
Now they are included, with the correct color applied.
2020-12-11 11:52:03 -05:00
Bianca Nenciu df26d2e72a
FIX: Build correct topic list filter (#11473)
* FIX: 'false' value was treated as a truthy value

For example, latest.json?no_subcategories=false used to have set
no_subcategories to the string value of 'false', which is not false.

* DEV: Remove dead code

* FIX: Redirect to /none under the right conditions

These conditions are:
 - neither /all or /none present
 - only for default filter

* FIX: Build correct topic list filter

/none was never added to the topic list filter

* FIX: Do not show count for subcategories if 'none' category

* FIX: preload_key must contain /none if no_subcategories
2020-12-11 14:20:48 +02:00
Krzysztof Kotlarek 3ea4f36f26
FIX: use sql_fragment instead of sanitize_sql_array (#11460)
This is a follow up to comment under this PR https://github.com/discourse/discourse/pull/11441

Sam suggested using sql_fragment instead of sanitize_sql_array
2020-12-11 10:56:26 +11:00
Krzysztof Kotlarek da2a61e36c
FIX: correct tracking when mute all categories (#11441)
Currently, we have a solution for muted topics. Basically, when a post is created first we send a `muted` message to users who muted that specific topic:

https://github.com/discourse/discourse/blob/master/app/models/topic_tracking_state.rb#L91

Later, topic tracking state filters if the topic is muted or not before update state:

https://github.com/discourse/discourse/blob/master/app/assets/javascripts/discourse/app/models/topic-tracking-state.js#L58:L67

That solution works quite well.

I wanted to extend it to handle `mute all categories by default` setting as well.

In that case, we should only inform the user about new topic/post when they explicitly want to.

If that setting is enabled, we would send "unmuted" message to a user who watches specific category, topic or tag. In all other cases, don't inform user about new topic as all categories are muted by default.

Meta: https://meta.discourse.org/t/threads-muted-by-mute-all-by-default-are-showing-up-as-new-but-not-visible/168324
2020-12-10 16:49:05 +11:00
Gerhard Schlager 538e1142e7
FIX: Prevent error when primary email is missing (#11446) 2020-12-09 18:14:45 +01:00
Mark VanLandingham 521934f163
FIX: Only cache reports with exceptions for 1 minute (#11447) 2020-12-09 10:54:41 -06:00
David Taylor 942888a10c
FEATURE: Increase maximum watched word length from 50 to 100 chars (#11437)
This is useful for more complex regex watched words

https://meta.discourse.org/t/166249
2020-12-09 17:45:34 +11:00
tshenry a446e0fde1
FEATURE: Link to text customization when editing system badges (#11345)
Being that system badges ship with every instance of Discourse, we've opted to define the name, description, and long description in our locales files to promote translation into other languages. When an admin visited the overview page of a system badge in their admin panel, they were met with disabled inputs for these text properties. The problem is that we failed to educate the admin that the text needs to be managed via the site text customization settings. 

This change adds a small "Customize Text" link under theses inputs that takes the admin to the specific site text customization where they can make desired changes.
2020-12-08 11:55:49 -08:00
Mark VanLandingham 9f786306bd
FIX: Actually error when topic timer time is in the past (#11434) 2020-12-08 11:13:45 -06:00
Bianca Nenciu ed52577e1c
FIX: Group#flair_url must be a real URL (#11400)
It used to be a short URL, but that did not work with the lightbox
in {{image-uploader}}.
2020-12-07 13:35:41 +02:00
Joffrey JAFFEUX 54519a2976
FIX: do not replace smiling_face by slight_smile (#11412) 2020-12-06 18:28:39 +01:00
Roman Rizzi 4d9686c494
FEATURE: Reviewed by filter. (#11405)
New reviewable filter to only display reviewables handled by a specific user.
2020-12-04 14:09:05 -03:00
Krzysztof Kotlarek e4d51e5b0a
FIX: correct link in the notification about moved post (#11399)
Notification is created by a job. If the job is evaluated before changes are committed to a database, a notification will have an incorrect URL.

Therefore, the job should be lodged in enqueue_jobs method which is triggered after the transaction:

```ruby
Topic.transaction do
  move_posts_to topic
end
add_allowed_users(participants) if participants.present? && @move_to_pm
enqueue_jobs(topic)
```

I improved a little bit specs to ensure that the destination topic_id is set. However, that tests are passing even without code improvements. I couldn't find an easy way to "delay" database transaction.

Meta: https://meta.discourse.org/t/bug-with-notifications-for-moved-posts/168937
2020-12-04 08:43:42 +11:00
Krzysztof Kotlarek 9c5ee4923b
FEATURE: silently close topic (#11392)
New TopicTimer to silently close topic. It will be used by discourse-solved plugin

Meta: https://meta.discourse.org/t/allow-auto-close-for-solved-to-do-so-silently/169300
2020-12-03 10:43:19 +11:00
Jordan Vidrine 2b66a4364d
Update header_primary in shades of blue (#11391)
This commit changes header_primary to a shade of blue, fixing contrast issues with some text colors in the header.
2020-12-02 09:22:30 -06:00
Arpit Jalan d627cc6233
Fix typo. (#11395) 2020-12-02 19:08:59 +05:30
Arpit Jalan a78c64db35
FIX: on invite redemption only update pending ReviewableUser record (#11394)
When the invite was being redeemed and the ReviewableUser record status
for the invited user was not pending an error was being raised.

This commit makes sure that we are only looking for ReviewableUser
record with status pending and updates that to approved.
2020-12-02 18:38:26 +05:30
Arpit Jalan 86feaec9be
FIX: URL encode tag name (#11393)
Tags with emoji in the name were failing to redirect via permalink
handling. This commit percent encodes the emoji name which fixes
the issue.

https://meta.discourse.org/t/permalinks-not-working-in-unicode-tags/171757/
2020-12-02 12:36:41 +05:30
David Taylor f56c4b78da
FEATURE: Add users:log_out API key scope (#11359) 2020-11-26 10:39:38 +00:00
Sam 3e1b94c227
PERF: avoid checking card background and user background when not supplied (#11299)
Previously we were performing a bunch of needless work when SSO failed
to supply profile and card backgrounds.
2020-11-25 10:53:44 +11:00
Bianca Nenciu f9ff51870b
FIX: Rebake theme fields if upload changes (#11341)
Updating SVG sprites of a theme did not take effect immediately because
the cache was not cleared.
2020-11-25 10:49:12 +11:00
David Taylor 0c685a46de
FEATURE: Add user update, anonymize and delete API scopes (#11335) 2020-11-24 12:54:24 +00:00
David Taylor 475b4892e3
DEV: Remove 'diff local changes' when updating remote themes (#11247)
Since 65e123498b, it is now impossible to make local changes to remote themes, so this warning is not needed.
2020-11-16 19:28:12 +00:00
Blake Erickson 06084fb8be
FIX: IndexCategoryForSearch category_id job exception (#11239)
On category create an exception will be thrown on this job because the
save transaction hasn't completed yet and the job cannot find the
category id. To prevent this we can use the rails 6 `after_save_commit`
hook that will fire after the category save transaction has finished for
both update and create actions.
2020-11-16 10:18:25 -07:00
Roman Rizzi 689623ffc0
FEATURE: Add an API key scopes for handling incoming email. (#11245)
Admins need to create a global API key if they want to use the mail-receiver. Let's add a scope for that.
2020-11-16 14:14:12 -03:00
David Taylor 86ffa3ba4f
PERF: Preload topic thumbnails for all topic lists (#11238)
Previously thumbnails were only preloaded for queries using `TopicQuery#default_results`, which meant that requests for PM topic lists would lead to N+1 queries.

This commit moves the preloading into TopicList#load_topics, along with other similar preloads (e.g. plugin custom fields)

The direct call to `ActiveRecord::Associations::Preloader#preload` is necessary because `@topics` can be an array, not an `ActiveRecord::Relation`
2020-11-16 13:23:49 +00:00
Dan Ungureanu bc8423a1bf
FEATURE: Add auto update field to themes (#11102)
Themes marked for auto update will be automatically updated when
Discourse is updated. This is triggered by discourse_docker or
docker_manager running Rake task 'themes:update'.
2020-11-16 14:44:09 +02:00
Roman Rizzi f2bef7ea8f
FIX: Store Reviewable's force_review as a boolean. (#11219)
* FIX: Store Reviewable's force_review as a boolean.

Using the `force_review` flag raises the score to hit the minimum visibility threshold. This strategy turned out to be ineffective on sites with a high number of flags, where these values could rapidly fluctuate.

This change adds a `force_review` column on the reviewables table and modifies the `Reviewable#list_for` method to show these items when passing the `status: :pending` option, even if the score is not high enough. ReviewableQueuedPosts and ReviewableUsers are always created using this option.
2020-11-13 08:19:01 -03:00
Dan Ungureanu ab314218d3
FEATURE: Implement edit functionality for post notices (#11140)
All post notice related custom fields were moved to a single one.
2020-11-11 14:49:53 +02:00
Sam ffc3da35a6
FIX: handle rapid concurrent SSO attempts more gracefully (#11180)
Rapid concurrent SSO attempts is something that happens quite frequently
in the wild at large enough scale.

When this happens conditions such as adding a user to a group could possibly
fire concurrently causing a user to be added to the same group twice and
erroring out.

To avoid all concurrency issues here we protect with a coarse distributed
mutex. This heavily mitigates the risk around concurrent group additions and
concurrent updates to user related records.
2020-11-10 10:40:41 +00:00
David Taylor cf21de0e7a
DEV: Migrate Github authentication to ManagedAuthenticator (#11170)
This commit adds an additional find_user_by_email hook to ManagedAuthenticator so that GitHub login can continue to support secondary email addresses

The github_user_infos table will be dropped in a follow-up commit.

This is the last core authenticator to be migrated to ManagedAuthenticator 🎉
2020-11-10 10:09:15 +00:00
Krzysztof Kotlarek 586c8efbd8
FEATURE: the ability to permanently destroy the private message (#11115)
PostDestroyer should accept the option to permanently destroy post from the database. In addition, when the first post is destroyed it destroys the whole topic.

Currently, that feature is limited to private messages and creator of the post. It will be used by discourse-encrypt to explode encrypted private messages.
2020-11-10 15:40:48 +11:00
Arpit Jalan 394f37cf66
FIX: do not use "max_tags_in_filter_list" setting for showing PM tags (#11146)
Ensure we do not respect max_tags_in_filter_list when showing the list of PM tags.

This filter is used on a full page view and there is not point limiting it to a small number.

The expectation is that PM tags are very rarely used, so a hard limit of 1000 should be safe for now.
2020-11-10 08:09:59 +11:00
Guo Xiang Tan 650da7b626 PERF: Update index for category in a background job.
Search indexing can get expensive and there is no need for us to block
the entire request just to wait for index to finish.
2020-11-09 13:51:26 +08:00
Alan Guo Xiang Tan 0e4d966a74
DEV: Remove stale ignored_columns. (#11160) 2020-11-09 15:57:59 +11:00
Bianca Nenciu 0863c36221
FIX: Improve errors when invite to topic fails (#11133)
It used to simply say "not allowed" without giving any hint what the
problem could be. This commit refactors the code and tries to improve
readability.
2020-11-06 16:58:10 +02:00