Commit Graph

5162 Commits

Author SHA1 Message Date
Vinoth Kannan 872f3e6934
UX: warn about messages to be orphaned while deleting a group. (#11727)
Currently, after destroying a group its messages are inaccessible to everyone. Only admins can access using direct URLs.
2021-01-22 03:29:34 +05:30
Martin Brennan 9ee8a01c3a
FIX: Change default for IncomingEmail#created_via to 0 (unknown) and make NOT NULL (#11782)
Follow up to https://review.discourse.org/t/dev-add-created-via-column-to-incomingemail-pr-11751/18366/6
2021-01-21 12:59:50 +10:00
Martin Brennan f34fa999a2
DEV: IMAP debugging improvements (#11784)
Improvements to make console access to IncomingEmail more pleasant, and stopping certain IMAP logs from landing in the DB because they just create too much noise,
2021-01-21 11:37:47 +10:00
Gerhard Schlager 3b2f6e129a
FEATURE: Add English (UK) as locale (#11768)
* "English" gets renamed into "English (US)"
* "English (UK)" replaces "English"

@discourse-translator-bot keep_translations_and_approvals
2021-01-20 21:32:22 +01:00
Mark VanLandingham 1a7922bea2
FEATURE: Create notification schedule to automatically set do not disturb time (#11665)
This adds a new table UserNotificationSchedules which stores monday-friday start and ends times that each user would like to receive notifications (with a Boolean enabled to remove the use of the schedule). There is then a background job that runs every day and creates do_not_disturb_timings for each user with an enabled notification schedule. The job schedules timings 2 days in advance. The job is designed so that it can be run at any point in time, and it will not create duplicate records.

When a users saves their notification schedule, the schedule processing service will run and schedule do_not_disturb_timings. If the user should be in DND due to their schedule, the user will immediately be put in DND (message bus publishes this state).

The UI for a user's notification schedule is in user -> preferences -> notifications. By default every day is 8am - 5pm when first enabled.
2021-01-20 10:31:52 -06:00
Bianca Nenciu 15da528870
FIX: Support for single use invite links (#11719)
The "invite link" checked if the number of uses was greater than 1. This
is not always true as single use invite links are perfectly valid.
2021-01-20 10:50:02 +02:00
Martin Brennan fb184fed06
DEV: Add created_via column to IncomingEmail (#11751)
This should make it easier to track down how the incoming email was created, which is one of four locations:

The POP3 poller (which picks up reply via email replies)
The admin email controller #handle_mail (which is where hosted mail is sent)
The IMAP sync tool
The group SMTP mailer, which sends emails when replying to IMAP topics, pre-emptively creating IncomingEmail records to avoid double syncing
2021-01-20 13:22:41 +10:00
Martin Brennan 0034cbda8a
DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698)
Moves the topic timer jobs from being scheduled ahead of time with enqueue_at to a 5 minute scheduled run like bookmark reminders, in a new job called Jobs::EnqueueTopicTimers. Backwards compatibility is maintained by checking if an existing topic timer job is enqueued in sidekiq for the timer, and if it is not running it inside the new job.

The functionality to close/open a topic if it is in the opposite state still remains in the after_save block of TopicTimer, with further commentary, which is used for Open/Close Temporarily.

This also removes the ensure_consistency! functionality of topic timers as it is no longer needed; the new job will always pick up the timers because they are not stored in a fragile state of sidekiq.
2021-01-19 13:30:58 +10:00
Martin Brennan be145ccf2f
FIX: Add bookmark limits (#11725)
Adds a bookmark search per page limit, a total bookmark creation limit, and a rate limit per day for bookmark creation.
2021-01-19 08:53:49 +10:00
Roman Rizzi 7ac9a4d2ec
FIX: Allow users to opt-out of using the site's small logo as the system user avatar. (#11744) 2021-01-18 14:09:07 -03:00
Martin Brennan bd25627198
FIX: IMAP post alerter race condition and code improvements (#11711)
This PR fixes a race condition with the IMAP notification code. In the `Email::Receiver` we call the `NewPostManager` to create the post and enqueue jobs and sends alerts via `PostAlerter`. However, if the post alerter reaches the `notify_pm_users` and the `group_notifying_via_smtp` method _before_ the incoming email is updated with the post and topic, we unnecessarily send a notification to the person who just posted. The result of this is that the IMAP syncer re-imports the email sent to the user about their own post, which looks like this in the group inbox:

To fix this, we skip the jobs enqueued by `NewPostManager` and only enqueue them with `PostJobsEnqueuer` manually _after_ the incoming email record has been updated with the post and topic.

Other improvements:

* Moved code to calculate email addresses from `IncomingEmail` records into the topic, with a group passed in, for easier testing and debugging. It is not the responsibility of the post alerter to figure this stuff out.
* Add shortcut methods on `IncomingEmail` to split or provide an empty array for to and cc addresses to avoid repetition.
2021-01-15 10:54:46 +10:00
Krzysztof Kotlarek 06b7c44593
FEATURE: reason to reject user signup (#11700)
Feature for `Must Approve Users` setup. When a user is rejected, a staff member can optionally set a reason for audit purposes. In addition, feedback email can be sent to the user.

Meta: https://meta.discourse.org/t/account-rejection-email/103112/8
2021-01-15 09:43:26 +11:00
Martin Brennan 2404fa7a23
DEV: Split toggle topic close job (#11679)
Splits the `ToggleTopicClosed` job into two distinct `OpenTopic` and `CloseTopic` jobs to make the code clearer. The old job cannot be deleted yet because of outstanding sidekiq schedules, so a todo has been added to do so later this year.

Also replaced mentions of `topic_status_update` with `topic_timer` in some files, because the `topic_status_update` model is obsolete and replaced by topic timer.

Added some shortcut methods for checking if a topic is open/whether a user can change an open topic.
2021-01-13 08:49:29 +10:00
Bianca Nenciu ec0212e56b
FIX: Make category slugs lowercase (#11277)
Admins could specify category slug with upper case characters and same slug,
but with different cases could be used simultaneously.
2021-01-12 17:28:33 +02:00
Mark VanLandingham 3eae4d3669
FIX: Publish DND ends_at updates in httpdate format (#11683) 2021-01-11 14:11:26 -06:00
Robin Ward 49b753eee0
FIX: Hiding a post should not trigger the duplicate check (#11680)
If for some reason a post was allowed to be duplicated (probably via
staff edit) hiding it should be permitted.
2021-01-11 14:56:08 -05:00
Krzysztof Kotlarek efaa63bd1d
FIX: always truncate uncomplete emojis in excerpts (#11678)
Additional fix after https://github.com/discourse/discourse/pull/11667

Always truncate "broken" emojis from excerpts.
2021-01-11 13:43:11 +11:00
Krzysztof Kotlarek 7f78b6ec10
FIX: broken emojis in topic excerpt (#11667)
When a post is truncated into the excerpt, sometimes we are breaking code of emoji
2021-01-11 10:40:41 +11:00
Roman Rizzi caa17386ee
FEATURE: Use the site's small logo as the system user's avatar. (#11661) 2021-01-08 10:40:00 -03:00
Mark VanLandingham 4601f3be7e
FEATURE: Send notification emails when users leave do not disturb mode (#11643) 2021-01-07 10:49:49 -06:00
Penar Musaraj c819284660
UX: Improve color scheme choices in user prefs (#11656) 2021-01-07 11:15:38 -05:00
Martin Brennan 64ba5b1d21
FIX: Group SMTP email improvements (#11633)
Fixes a rare race condition causing the `Imap::Sync` class to create an incoming email and associated post/topic, which then kicks off the PostAlerter to notify others in the PM about a reply in the topic, but for the OP which is not necessary (because the person emailing the IMAP inbox already knows about the OP). Basically, we should never be sending the group SMTP email for the first post in a topic.

Also in this PR:

* Custom attribute accessors for the to/from/cc addresses on `IncomingEmail`, to parse them from an array to a joined string so the logic for this is only in one place.
* Store extra detail against the `IncomingEmail` created in `GroupSmtpMailer`
* regex test Mail header Reply-To as string instead of Field, which fixes `warning: deprecated Object#=~ is called on Mail::Field; it always returns nil`
* Add DEBUG_IMAP to log all IMAP logs as warnings for easier debugging
* Changed the Rails logging to `ImapSyncLog` in the `GroupSmtpMailer`
2021-01-05 15:32:04 +10:00
David Taylor 13e39d8b9f
PERF: Improve cook_url performance for topic thumbnails (#11609)
- Only initialize the S3Helper when needed
- Skip initializing the S3Helper for S3Store#cdn_url
- Allow cook_url to be passed a `local` hint to skip unnecessary checks
2020-12-30 18:13:13 +00:00
Rafael dos Santos Silva 8da0b4979b
FIX: S3 CDN warning was showing wrong warning message (#11603) 2020-12-29 14:09:55 -03:00
Sam 53f4d54f23
PERF: add indexes to speed up profile pages (#11598)
These 2 indexes optimise performance on profile pages.

The summary page displays:

1. A list of "Top Link" - links sorted by number of clicks posted by user
2. A list of "Top Replies" - replies made by a user that go the most hearts

These two areas could devolve into full index or table scans, new indexes are there to avoid this cost on large dbs

One minor downside is that storage requirements go a tiny bit up to maintain the new indexes
2020-12-29 15:54:05 +11:00
Rafael dos Santos Silva 330ef3700f
FEATURE: Add dash warning for using S3 without CDN (#11561)
Over the years we had some admins complaining that most of their costs
running Discourse is on S3 uploads.

That only happens when they are using S3 without a CDN, which is widely
considered a bad pratice for public websites, as egress bandwidth can
get expensive quite fast.

Related:

https://meta.discourse.org/t/download-avatars-locally-from-s3-shutting-down-s3-in-my-site/174001/3?u=falco

https://meta.discourse.org/t/use-webtorrent-to-load-media-objects/155873/2?u=falco
2020-12-28 12:43:48 -03:00
Krzysztof Kotlarek f84ff26aa9
FIX: use Redis replica host and port (#11566)
Introduce Redis `replica` config and deprecate `slave`
2020-12-23 13:14:19 +11:00
Krzysztof Kotlarek 07bf7a91f4
Revert "FIX: use Redis replica host and port (#11543)" (#11552)
This reverts commit b0e1210b0c.
2020-12-22 16:16:50 +11:00
Krzysztof Kotlarek b0e1210b0c
FIX: use Redis replica host and port (#11543)
* FIX: use Redis replica host and port

Introduce Redis `replica` config and deprecate `slave`

* FIX: move deprecations to separate file
2020-12-22 15:52:00 +11:00
David Taylor d25fd34b44
DEV: Remove with_deleted workarounds for old Rails version (#11550)
* DEV: Remove with_deleted workarounds for old Rails version

These workarounds using private APIs are no longer required in the latest version of Rails. The referenced issue (https://github.com/rails/rails/issues/4306) was closed in 2013. The acts_as_paranoid workaround which this was based on was removed for rails > 5.

Switching to using a scope also allows us to use it within a `belongs_to` relation (e.g. in the Poll model). This avoids issues which can be caused by unscoping all `where` clauses.

Predicates are not necessarily strings, so calling `.join(" AND ")` can sometimes cause weird errors. If we use `WhereClause#ast`, and then `.to_sql` we achieve the same thing with fully public APIs, and it will work successfully for all predicates.
2020-12-22 10:38:59 +11:00
Rafael dos Santos Silva 32e7ee4867
FEATURE: Adds support for the WEBP image format (#11532)
Adds support for both still and animated webp images.
Animated images won't be optimized or get thumbnails.

Depends on https://github.com/discourse/discourse_docker/pull/506
and https://github.com/discourse/discourse_docker/pull/507
2020-12-21 20:09:00 -03:00
Mark VanLandingham 649ed24bb4
FEATURE: Do not disturb (#11484) 2020-12-18 09:03:51 -06:00
Bianca Nenciu 806f05f851
FIX: Make find_by_slug_path work with default slugs (#11501)
Default slugs are generated by adding '-category' to category ID.
2020-12-18 16:05:01 +02:00
David Taylor c54217f33d
FIX: Clear in-process theme cache after clearing DB cache (#11517)
If we clear the in-process cache first, it might get re-filled from the
DB before we clear the DB cache. This would be more likely on high-traffic
sites.
2020-12-17 11:02:41 +11:00
Bmorrical e7b837e7e1 Have doubled the size of the Watched Words Per Action Limit 2020-12-16 13:37:10 -05:00
Krzysztof Kotlarek 6dd3f986b2
FIX: no email when the category is muted and mailing list is on (#11490)
When `mute_all_categories_by_default` setting is enabled we should not send mailing list until category, tag or topic is explicitly watched.
2020-12-16 09:30:21 +11:00
Roman Rizzi c7b9f044a4
FEATURE: Automatically disable slow mode. (#11461)
Staff and TL4 users can decide the slow mode duration. We'll internally set a topic timer to disable it.
2020-12-14 14:06:50 -03:00
Robin Ward a51a06115a FIX: Posts with Staff Colors were excluded from the group activity
Now they are included, with the correct color applied.
2020-12-11 11:52:03 -05:00
Bianca Nenciu df26d2e72a
FIX: Build correct topic list filter (#11473)
* FIX: 'false' value was treated as a truthy value

For example, latest.json?no_subcategories=false used to have set
no_subcategories to the string value of 'false', which is not false.

* DEV: Remove dead code

* FIX: Redirect to /none under the right conditions

These conditions are:
 - neither /all or /none present
 - only for default filter

* FIX: Build correct topic list filter

/none was never added to the topic list filter

* FIX: Do not show count for subcategories if 'none' category

* FIX: preload_key must contain /none if no_subcategories
2020-12-11 14:20:48 +02:00
Krzysztof Kotlarek 3ea4f36f26
FIX: use sql_fragment instead of sanitize_sql_array (#11460)
This is a follow up to comment under this PR https://github.com/discourse/discourse/pull/11441

Sam suggested using sql_fragment instead of sanitize_sql_array
2020-12-11 10:56:26 +11:00
Krzysztof Kotlarek da2a61e36c
FIX: correct tracking when mute all categories (#11441)
Currently, we have a solution for muted topics. Basically, when a post is created first we send a `muted` message to users who muted that specific topic:

https://github.com/discourse/discourse/blob/master/app/models/topic_tracking_state.rb#L91

Later, topic tracking state filters if the topic is muted or not before update state:

https://github.com/discourse/discourse/blob/master/app/assets/javascripts/discourse/app/models/topic-tracking-state.js#L58:L67

That solution works quite well.

I wanted to extend it to handle `mute all categories by default` setting as well.

In that case, we should only inform the user about new topic/post when they explicitly want to.

If that setting is enabled, we would send "unmuted" message to a user who watches specific category, topic or tag. In all other cases, don't inform user about new topic as all categories are muted by default.

Meta: https://meta.discourse.org/t/threads-muted-by-mute-all-by-default-are-showing-up-as-new-but-not-visible/168324
2020-12-10 16:49:05 +11:00
Gerhard Schlager 538e1142e7
FIX: Prevent error when primary email is missing (#11446) 2020-12-09 18:14:45 +01:00
Mark VanLandingham 521934f163
FIX: Only cache reports with exceptions for 1 minute (#11447) 2020-12-09 10:54:41 -06:00
David Taylor 942888a10c
FEATURE: Increase maximum watched word length from 50 to 100 chars (#11437)
This is useful for more complex regex watched words

https://meta.discourse.org/t/166249
2020-12-09 17:45:34 +11:00
tshenry a446e0fde1
FEATURE: Link to text customization when editing system badges (#11345)
Being that system badges ship with every instance of Discourse, we've opted to define the name, description, and long description in our locales files to promote translation into other languages. When an admin visited the overview page of a system badge in their admin panel, they were met with disabled inputs for these text properties. The problem is that we failed to educate the admin that the text needs to be managed via the site text customization settings. 

This change adds a small "Customize Text" link under theses inputs that takes the admin to the specific site text customization where they can make desired changes.
2020-12-08 11:55:49 -08:00
Mark VanLandingham 9f786306bd
FIX: Actually error when topic timer time is in the past (#11434) 2020-12-08 11:13:45 -06:00
Bianca Nenciu ed52577e1c
FIX: Group#flair_url must be a real URL (#11400)
It used to be a short URL, but that did not work with the lightbox
in {{image-uploader}}.
2020-12-07 13:35:41 +02:00
Joffrey JAFFEUX 54519a2976
FIX: do not replace smiling_face by slight_smile (#11412) 2020-12-06 18:28:39 +01:00
Roman Rizzi 4d9686c494
FEATURE: Reviewed by filter. (#11405)
New reviewable filter to only display reviewables handled by a specific user.
2020-12-04 14:09:05 -03:00
Krzysztof Kotlarek e4d51e5b0a
FIX: correct link in the notification about moved post (#11399)
Notification is created by a job. If the job is evaluated before changes are committed to a database, a notification will have an incorrect URL.

Therefore, the job should be lodged in enqueue_jobs method which is triggered after the transaction:

```ruby
Topic.transaction do
  move_posts_to topic
end
add_allowed_users(participants) if participants.present? && @move_to_pm
enqueue_jobs(topic)
```

I improved a little bit specs to ensure that the destination topic_id is set. However, that tests are passing even without code improvements. I couldn't find an easy way to "delay" database transaction.

Meta: https://meta.discourse.org/t/bug-with-notifications-for-moved-posts/168937
2020-12-04 08:43:42 +11:00
Krzysztof Kotlarek 9c5ee4923b
FEATURE: silently close topic (#11392)
New TopicTimer to silently close topic. It will be used by discourse-solved plugin

Meta: https://meta.discourse.org/t/allow-auto-close-for-solved-to-do-so-silently/169300
2020-12-03 10:43:19 +11:00
Jordan Vidrine 2b66a4364d
Update header_primary in shades of blue (#11391)
This commit changes header_primary to a shade of blue, fixing contrast issues with some text colors in the header.
2020-12-02 09:22:30 -06:00
Arpit Jalan d627cc6233
Fix typo. (#11395) 2020-12-02 19:08:59 +05:30
Arpit Jalan a78c64db35
FIX: on invite redemption only update pending ReviewableUser record (#11394)
When the invite was being redeemed and the ReviewableUser record status
for the invited user was not pending an error was being raised.

This commit makes sure that we are only looking for ReviewableUser
record with status pending and updates that to approved.
2020-12-02 18:38:26 +05:30
Arpit Jalan 86feaec9be
FIX: URL encode tag name (#11393)
Tags with emoji in the name were failing to redirect via permalink
handling. This commit percent encodes the emoji name which fixes
the issue.

https://meta.discourse.org/t/permalinks-not-working-in-unicode-tags/171757/
2020-12-02 12:36:41 +05:30
David Taylor f56c4b78da
FEATURE: Add users:log_out API key scope (#11359) 2020-11-26 10:39:38 +00:00
Sam 3e1b94c227
PERF: avoid checking card background and user background when not supplied (#11299)
Previously we were performing a bunch of needless work when SSO failed
to supply profile and card backgrounds.
2020-11-25 10:53:44 +11:00
Bianca Nenciu f9ff51870b
FIX: Rebake theme fields if upload changes (#11341)
Updating SVG sprites of a theme did not take effect immediately because
the cache was not cleared.
2020-11-25 10:49:12 +11:00
David Taylor 0c685a46de
FEATURE: Add user update, anonymize and delete API scopes (#11335) 2020-11-24 12:54:24 +00:00
David Taylor 475b4892e3
DEV: Remove 'diff local changes' when updating remote themes (#11247)
Since 65e123498b, it is now impossible to make local changes to remote themes, so this warning is not needed.
2020-11-16 19:28:12 +00:00
Blake Erickson 06084fb8be
FIX: IndexCategoryForSearch category_id job exception (#11239)
On category create an exception will be thrown on this job because the
save transaction hasn't completed yet and the job cannot find the
category id. To prevent this we can use the rails 6 `after_save_commit`
hook that will fire after the category save transaction has finished for
both update and create actions.
2020-11-16 10:18:25 -07:00
Roman Rizzi 689623ffc0
FEATURE: Add an API key scopes for handling incoming email. (#11245)
Admins need to create a global API key if they want to use the mail-receiver. Let's add a scope for that.
2020-11-16 14:14:12 -03:00
David Taylor 86ffa3ba4f
PERF: Preload topic thumbnails for all topic lists (#11238)
Previously thumbnails were only preloaded for queries using `TopicQuery#default_results`, which meant that requests for PM topic lists would lead to N+1 queries.

This commit moves the preloading into TopicList#load_topics, along with other similar preloads (e.g. plugin custom fields)

The direct call to `ActiveRecord::Associations::Preloader#preload` is necessary because `@topics` can be an array, not an `ActiveRecord::Relation`
2020-11-16 13:23:49 +00:00
Dan Ungureanu bc8423a1bf
FEATURE: Add auto update field to themes (#11102)
Themes marked for auto update will be automatically updated when
Discourse is updated. This is triggered by discourse_docker or
docker_manager running Rake task 'themes:update'.
2020-11-16 14:44:09 +02:00
Roman Rizzi f2bef7ea8f
FIX: Store Reviewable's force_review as a boolean. (#11219)
* FIX: Store Reviewable's force_review as a boolean.

Using the `force_review` flag raises the score to hit the minimum visibility threshold. This strategy turned out to be ineffective on sites with a high number of flags, where these values could rapidly fluctuate.

This change adds a `force_review` column on the reviewables table and modifies the `Reviewable#list_for` method to show these items when passing the `status: :pending` option, even if the score is not high enough. ReviewableQueuedPosts and ReviewableUsers are always created using this option.
2020-11-13 08:19:01 -03:00
Dan Ungureanu ab314218d3
FEATURE: Implement edit functionality for post notices (#11140)
All post notice related custom fields were moved to a single one.
2020-11-11 14:49:53 +02:00
Sam ffc3da35a6
FIX: handle rapid concurrent SSO attempts more gracefully (#11180)
Rapid concurrent SSO attempts is something that happens quite frequently
in the wild at large enough scale.

When this happens conditions such as adding a user to a group could possibly
fire concurrently causing a user to be added to the same group twice and
erroring out.

To avoid all concurrency issues here we protect with a coarse distributed
mutex. This heavily mitigates the risk around concurrent group additions and
concurrent updates to user related records.
2020-11-10 10:40:41 +00:00
David Taylor cf21de0e7a
DEV: Migrate Github authentication to ManagedAuthenticator (#11170)
This commit adds an additional find_user_by_email hook to ManagedAuthenticator so that GitHub login can continue to support secondary email addresses

The github_user_infos table will be dropped in a follow-up commit.

This is the last core authenticator to be migrated to ManagedAuthenticator 🎉
2020-11-10 10:09:15 +00:00
Krzysztof Kotlarek 586c8efbd8
FEATURE: the ability to permanently destroy the private message (#11115)
PostDestroyer should accept the option to permanently destroy post from the database. In addition, when the first post is destroyed it destroys the whole topic.

Currently, that feature is limited to private messages and creator of the post. It will be used by discourse-encrypt to explode encrypted private messages.
2020-11-10 15:40:48 +11:00
Arpit Jalan 394f37cf66
FIX: do not use "max_tags_in_filter_list" setting for showing PM tags (#11146)
Ensure we do not respect max_tags_in_filter_list when showing the list of PM tags.

This filter is used on a full page view and there is not point limiting it to a small number.

The expectation is that PM tags are very rarely used, so a hard limit of 1000 should be safe for now.
2020-11-10 08:09:59 +11:00
Guo Xiang Tan 650da7b626 PERF: Update index for category in a background job.
Search indexing can get expensive and there is no need for us to block
the entire request just to wait for index to finish.
2020-11-09 13:51:26 +08:00
Alan Guo Xiang Tan 0e4d966a74
DEV: Remove stale ignored_columns. (#11160) 2020-11-09 15:57:59 +11:00
Bianca Nenciu 0863c36221
FIX: Improve errors when invite to topic fails (#11133)
It used to simply say "not allowed" without giving any hint what the
problem could be. This commit refactors the code and tries to improve
readability.
2020-11-06 16:58:10 +02:00
tshenry d778d99b55
FIX: limit_topics_per_day should only apply to regular topics (#11127)
PMs were being evaluted by both the limit_topics_per_day and limit_private_messages_per_day rate limiters when it should only be the latter.
2020-11-04 16:23:49 -08:00
Dan Ungureanu 3c51647872
DEV: Remove legacy tag and category routes (#10338)
* DEV: Remove client-side legacy tag and category routes

* DEV: Remove server-side legacy tag routes

* DEV: Refactor ListController#set_category

* FIX: Remove reference to discovery.parentCategory

* FIX: Refactor TagsController#set_category_from_params

* FIX: Build correct canonical URL for tags and categories

* DEV: Fix deprecation notice in Ruby 2.7

* DEV: Replace use of removed legacy tag route

* DEV: Add deprecation notices for old routes and controllers
2020-11-03 16:57:58 +02:00
David Taylor 5140ec9acf
DEV: Cleanup ignored user logic (#11107)
- IgnoredUser records should all now have an expiring_at value. This commit enforces that in the DB, and fixes any corrupt rows
- Changes to the ignored user list are now handled by the `/u/{username}/notification_level` endpoint. This allows setting expiration dates on the ignore. This commit removes the old logic for saving a list of usernames in the user preferences.
- Many specs were calling `IgnoredUser.create`. This commit changes them to use `Fabricate(:ignored_user)` for consistency
2020-11-03 12:38:54 +00:00
Kane York 789e3775df
FIX: Make all email subject vars available in notification subjects (#11064)
A site owner attempting to use both the email_subject site setting and translation overrides for normal post notification
email subjects would find themselves frusturated at the lack of template argument parity.
Make all the variables available for translation overrides by adding the subject variables to the custom interpolation keys list and applying them.

Reported at https://meta.discourse.org/t/customize-subject-format-for-standard-emails/20801/47?u=riking
2020-11-02 20:00:11 -08:00
Arpit Jalan 1476e17c35
FEATURE: new setting to create a linked topic on autoclosing mega topics (#11001)
This commit adds a site setting `auto_close_topics_create_linked_topic`
which when enabled works in conjunction with `auto_close_topics_post_count`
setting and creates a new linked topic for the topic just closed.

The auto-created new topic contains a link for all the previous topics
and the topic titles are appended with `(Part {n})`.

The setting is enabled by default.
2020-11-02 12:18:48 +05:30
Jarek Radosz 2f4a1ff61b
DEV: Update rubocop-discourse from 2.3.2 to 2.4.0 (#11079)
Also fixes whitespace related issues raised by rubocop.
2020-10-30 15:04:29 +01:00
Mark VanLandingham c750a02f05
DEV: Move reports to concerns for autoloading (#11066) 2020-10-29 11:49:09 -05:00
Martin Brennan 632942e697
FIX: Ensure group SMTP and message builder always uses from address for Reply-To when IMAP is enabled (#11037)
There is a site setting reply_by_email_enabled which when combined with reply_by_email_address creates a Reply-To header in emails in the format "test+%{reply_key}@test.com" along with a PostReplyKey record, so when replying Discourse knows where to route the reply.

However this conflicts with the IMAP implementation. Since we are sending the email for a group via SMTP and from their actual email account, we want all replys to go to that email account as well so the IMAP sync job can pick them up and put them in the correct place. So if the group has IMAP enabled and configured, then the reply-to header will be correct.

This PR also makes a further fix to 64b0b50 by using the correct recipient user for the PostReplyKey record. If the post user is used we encounter this error:

if destination.user_id != user.id && !forwarded_reply_key?(destination, user)
  raise ReplyUserNotMatchingError, "post_reply_key.user_id => #{destination.user_id.inspect}, user.id => #{user.id.inspect}"
end
This is because the user above is found from the from_address, but the destination which is the PostReplyKey is made by the post.user, which will be different people.
2020-10-28 07:01:58 +10:00
Arpit Jalan c6bf70c870
DEV: annotate models (#11047) 2020-10-27 23:42:33 +05:30
Mark VanLandingham 554a617b3d
DEV: Better custom field preload error (#11044) 2020-10-27 10:05:07 -05:00
Dan Ungureanu 43557143fe
FIX: Reset invite when resending it (#11013)
Resending an invite moved the expire date in the future, but did not
invalidate it. For example, if an invite was sent to an email,
invalidated and then resent, it would still be left invalidated.
2020-10-26 12:26:43 +02:00
jbrw 2bcca46cc5
FEATURE - ImageMagick jpeg quality (#11004)
* FEATURE - Add SiteSettings to control JPEG image quality

`recompress_original_jpg_quality` - the maximum quality of a newly
uploaded file.

`image_preview_jpg_quality` - the maximum quality of OptimizedImages
2020-10-23 12:38:28 -04:00
David Taylor abb00c3780
FIX: Restore users#topic_tracking_state route to api session_info scope (#10992)
This route was inadvertently removed in 1cec333f, and is required for showing new/unread counts in Discourse mobile apps
2020-10-21 19:44:34 +01:00
David Taylor b7c680853d DEV: Introduce plugin API to contribute user api key scopes 2020-10-19 10:40:55 +01:00
David Taylor 23e5c605f6 DEV: Add support for allowed parameters in user api key scopes
Initially, this feature is only intended for use in core/plugins, so there is no API for requesting a parameter-scoped key. That may change in future.
2020-10-19 10:40:55 +01:00
David Taylor 1cec333f48 REFACTOR: Introduce RouteMatcher class
This consolidates logic used to match routes in ApiKey, UserApiKey and DefaultCurrentUserProvider. This reduces duplicated logic, and will allow UserApiKeysScope to easily re-use the parameter matching logic from ApiKeyScope
2020-10-19 10:40:55 +01:00
Roman Rizzi 21c53ed249
FEATURE: Topic slow mode. (#10904)
Adds a new slow mode for topics that are heating up. Users will have to wait for a period of time before being able to post again.

We store this interval inside the topics table and track the last time a user posted using the last_posted_at datetime in the TopicUser relation.
2020-10-16 16:24:38 -03:00
Bianca Nenciu 43e52a7dc1
DEV: Remove gifsicle dependency (#10357)
Dependency on gifsicle, allow_animated_avatars and allow_animated_thumbnails
site settings were all removed. Animated GIF images are still allowed, but
the generated optimized images are no longer animated for those (which were
used for avatars and thumbnails).

The added 'animated' is populated by extracting information using FastImage.
This field was used to selectively reoptimize old animations. This process
happens in the background.
2020-10-16 13:41:27 +03:00
Penar Musaraj 5763309953
FEATURE: WCAG compliant color schemes (#10882)
Co-authored-by: Kris <kris.aubuchon@discourse.org>
2020-10-15 14:05:48 -04:00
Penar Musaraj 74de7a49f5
Simplify theme and color scheme seeding (#10872)
Now that we have support for user-selectable color schemes, it makes sense
to simplify seeding and theme updates in the wizard. 

We now:

- seed only one theme, named "Default" (previously "Light")
- seed a user-selectable Dark color scheme
- rename the "Themes" wizard step to "Colors"
- update the default theme's color scheme if a default is set
(a new theme is created if there is no default)
2020-10-14 10:18:02 -04:00
Martin Brennan c3cede697d
FEATURE: Add weekly bookmark cleanup code (#10899)
When posts or topics are deleted we don't want to immediately delete associated bookmarks, so we have a grace period to recover them and their reminders if the post or topic is un-deleted. This PR adds a task to the Weekly scheduled job to go and delete bookmarks attached to posts or topics deleted > 3 days ago.
2020-10-14 09:38:57 +10:00
Bianca Nenciu 25b8ed740b
DEV: Make site setting type uploaded_image_list use upload IDs (#10401)
It used to be a list of concatenated upload URLs which was prone to
break.
2020-10-13 16:17:06 +03:00
Daniel Waterworth 721ee36425
Replace `base_uri` with `base_path` (#10879)
DEV: Replace instances of Discourse.base_uri with Discourse.base_path

This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
Penar Musaraj 5130b4d674
Ignore disabled out-of-date remote themes (#10870)
No need to alert in admin dashboard about out of date remote themes
that are disabled.
2020-10-08 13:48:16 -04:00
David Taylor c0293339b8
PERF: Do not enqueue digest emails when attempted recently (#10849)
Previously, Jobs::EnqueueDigestEmails would enqueue a digest job for every user, even if there are no topics to send. The digest job would exit, no email would send, and last_emailed_at would not change. 30 minutes later, Jobs::EnqueueDigestEmails would run again and re-enqueue jobs for the same users.

120fa8ad introduced a temporary mitigation for this issue, by randomly selecting a subset of those users each time.

This commit adds a new `digest_attempted_at` column to the `user_stats` table. This column is updated every time a digest job completes for a user. Using this, we can avoid scheduling digest jobs for the same user every 30 minutes. This also removes the random user selection in 120fa8ad, and instead prioritizes users who had digests attempted the longest time ago.
2020-10-07 15:30:38 +01:00
Martin Brennan 6e2be3e60b
FIX: When admin changes an email for the user the user must confirm the change (#10830)
See https://meta.discourse.org/t/changing-a-users-email/164512 for additional context.

Previously when an admin user changed a user's email we assumed that they would need a password reset too because they likely did not have access to their account. This proved to be incorrect, as there are other reasons a user needs admin to change their email. This PR:

* Changes the admin change email for user flow so the user is sent an email to confirm the change
* We now record who the email change request was requested by
* If the requested by user is admin and not the user we note this in the email sent to the user
* We also make the confirm change email route open to anonymous users, so it can be clicked by the user even if they do not have access to their account. If there is a logged in user we make sure the confirmation matches the current user.
2020-10-07 13:02:24 +10:00
Jarek Radosz cf44cdb082
FIX: Validate category name on parent change (#10815)
Previously, moving a category into another one, that already had a child category of that name (but with a non-conflicting slug) would cause a 500 error:

```
# PG::UniqueViolation:
#   ERROR:  duplicate key value violates unique constraint "unique_index_categories_on_name"
#   DETAIL:  Key (COALESCE(parent_category_id, '-1'::integer), name)=(5662, Amazing Category 0) already exists.
```

It now returns 422, and shows the same message as when you're renaming a category: "Category Name has already been taken".
2020-10-05 11:50:05 +02:00
Martin Brennan 39b2fb8649
FIX: Invalid URLs could raise exceptions when calling UrlHelper.rails_route_from_url (#10782)
Upload.secure_media_url? raised an exceptions when the URL was invalid,
which was a issue in some situations where secure media URLs must be
removed.

For example, sending digests used PrettyText.strip_secure_media,
which used Upload.secure_media_url? to replace secure media with
placeholders. If the URL was invalid, then an exception would be raised
and left unhandled.

Now instead in UrlHelper.rails_route_from_url we return nil if there is something wrong with the URL.

Co-authored-by: Bianca Nenciu <nenciu.bianca@gmail.com>
2020-09-30 15:20:00 +10:00
Arpit Jalan 025490ecb9
FEATURE: composer option to reload page and force save draft (#10773) 2020-09-29 22:29:03 +05:30
David Taylor 1ba9b34b03
DEV: Move UserApiKey scopes to dedicated table (#10704)
This has no functional impact yet, but it is the first step in adding more granular scopes to UserApiKeys
2020-09-29 10:57:48 +01:00
Krzysztof Kotlarek e7c72cd1e4
FIX: deprecate whitelist constants (#10716)
Deprecation of:
WHITELISTED_REDIRECT_HOSTNAMES
CUSTOM_INTERPOLATION_KEYS_WHITELIST
WHITELISTED_SVG_ELEMENTS
2020-09-28 13:52:05 +10:00
Osama Sayegh d745568633
DEV: Stop polluting `Class` with deprecated settings methods (#10736)
`self.class` here evaluates to `Class` and then we're calling `define_method` on it which means all classes will have those methods defined in them. For example:

```
~/discourse(master*) » rails c
Loading development environment (Rails 6.0.3.3)
[1] pry(main)> Integer.methods
=> [:sqrt,
 :yaml_tag,
 :email_domains_blacklist=,
 :email_domains_whitelist=,
 :unicode_username_character_whitelist=,
 :user_website_domains_whitelist=,
 :whitelisted_link_domains=,
 :email_domains_blacklist,
 :email_domains_whitelist,
 :unicode_username_character_whitelist,
...
...
```

Fix here is to use `self.define_singleton_method`.
2020-09-24 01:58:53 +03:00
Bianca Nenciu eb891778ff
DEV: Do not translate font names (#10723)
Use the names as provided by discourse-fonts and remove the
translated strings.

It also ensures that the selected font is present in case a font will
be removed in the future.
2020-09-23 12:00:07 +03:00
Krzysztof Kotlarek 0bb51dcbfa
FIX: TL2 promotion message and advance training (#10679)
This is a little bit of refactoring. Core Discourse should have default promotion message for TL2.

In addition, when the Discobot plugin is enabled, the user is invited to advanced training
2020-09-22 10:17:52 +10:00
Krzysztof Kotlarek c934a0f759
FIX: don't error Topic#similar_to when prepared raw is blank (#10711)
If raw contains incorrect URL, `prepare_data` returns empty string:

https://github.com/discourse/discourse/blob/master/lib/search.rb#L91

Therefore we should not only check if the cooked post is not blank but also if prepared data is not blank.
2020-09-22 07:53:12 +10:00
Daniel Waterworth 6e69c25134 FIX: Use asset_hostnames in rails_multisite
We want to respond to the x-forwarded-for host from the CDN too.
2020-09-17 10:32:43 +01:00
Guo Xiang Tan b47b640598
FEATURE: Hidden `SiteSetting.keep_old_ip_address_count` to track IP history. 2020-09-17 12:50:39 +08:00
Martin Brennan 80268357e7
DEV: Change upload verified column to be integer (#10643)
Per review https://review.discourse.org/t/dev-add-verified-to-uploads-and-fill-in-s3-inventory-10406/14180

Change the verified column for Upload to a verified_status integer column, to avoid having NULL as a weird implicit status.
2020-09-17 13:35:29 +10:00
Guo Xiang Tan 49f53bdc50
DEV: Fix another `MessageBus::InvalidMessageTarget` error.
Follow-up to c2f2b57a9b.
2020-09-15 14:25:10 +08:00
Guo Xiang Tan f7438fc64b
DEV: Fix `MessageBus::InvalidMessageTarget` error after upgrading mb.
Follow-up to c2f2b57a9b
2020-09-15 14:15:42 +08:00
Martin Brennan 5268568d23
FEATURE: Remove user topic timers and migrate to bookmarks with reminders (#10474)
This PR removes the user reminder topic timers, because that system has been supplanted and improved by bookmark reminders. The option is removed from the UI and all existing user reminder topic timers are migrated to bookmark reminders.

Migration does this:

* Get all topic_timers with status_type 5 (reminders)
* Gets all bookmarks where the user ID and topic ID match
* Loops through the found topic timers
  * If there is no bookmark for the OP of the topic, then we just create a bookmark with a reminder
  * If there is a bookmark for the OP of the topic and it does **not** have a reminder set, then just 
update it with the topic timer reminder
  * If there is a bookmark for the OP of the topic with a reminder then just discard the topic timer
* Cancels all outstanding user reminder topic timers
* **Trashes (not deletes) all user reminder topic timers**

Notes:

* For now I have left the user reminder topic timer job class in place; this is so the jobs can be cancelled in the migration. It and the specs will be deleted in the next PR.
* At a later date I will write a migration to delete all trashed user topic timers. They are not deleted here in case there are data issues and they need to be recovered.
* A future PR will change the UI of the topic timer modal to make it look more like the bookmark modal.
2020-09-14 11:11:55 +10:00
Guo Xiang Tan 543e972fec
PERF: Remove category_user lookup when loading private messages.
Private messages do not belong to categories so the query is unnecessary
overhead.
2020-09-11 16:26:38 +08:00
Vinoth Kannan 9e4ed03b8f FEATURE: moderators allowed to view groups which members can see.
Currently, if a group's visibility is set to "Group owners, members" then the mods can't view those group pages. The same rule is applied for members visibility setting too.

This reverts commit 7fc7090. And fixed the spec test fails.
2020-09-11 08:20:13 +05:30
Guo Xiang Tan 7fc70900d2
Revert "FEATURE: moderators allowed to view groups which members can see."
Build is broken.

This reverts commit 2bf0c4188b.
2020-09-11 10:04:54 +08:00
Vinoth Kannan 2bf0c4188b FEATURE: moderators allowed to view groups which members can see.
Currently, if a group's visibility is set to "Group owners, members" then the mods can't view those group pages. The same rule is applied for members visibility setting too.
2020-09-11 03:33:44 +05:30
Gerhard Schlager ac70c48be4 FIX: Prevent "uploads are missing in S3" alerts after restoring a backup
After restoring a backup it takes up to 48 hours for uploads stored on S3 to appear in the S3 inventory. This change prevents alerts about missing uploads by preventing the EnsureS3UploadsExistence job from running in the first 48 hours after a restore. During the restore it  deletes the count of missing uploads from the PluginStore, so that an alert isn't triggered by an old number.
2020-09-10 21:37:48 +02:00
Robert dee451605b
FEATURE: add user opt to select bookmarks as home (#10597)
Admins can currently add the bookmarks discovery route link
to the homepage interface, but users can't presently select
that as their default home view.  This change facilitates that,
adding the option to the existing Default Home Page dropdown on
the User Preferences Interface page.
2020-09-10 11:13:12 -04:00
Guo Xiang Tan f27de87bf3
FIX: Update first_pm_unread_at of user's groups without unread.
If a user always read all group messages, we will never update the
`first_pm_unread_at` column since the previous query will not return the
group_user. Instead, we should update `first_pm_unread_at` to the
current timestamp if the user has read everything.

Follow-up to 9b75d95fc6
2020-09-10 17:19:38 +08:00
Martin Brennan 7f2f87bf59
DEV: Review fixes (#10641)
See comments in https://review.discourse.org/t/dev-imap-log-to-database-10435/14337/6 for context.
2020-09-10 13:41:46 +10:00
Guo Xiang Tan 9b75d95fc6 PERF: Keep track of first unread PM and first unread group PM for user.
This optimization helps to filter away topics so that the joins on
related tables when querying for unread messages is not expensive.
2020-09-09 14:05:41 +08:00
Robin Ward 4dd07843c6 FIX: We have changed the way `__widget_helpers` are resolved 2020-09-08 12:10:59 -04:00
Blake Erickson 67dec38f31 FIX: Gravatar download attempt if user is missing their email
It is possible that a user could exist without an email, if so we should
not enqueue a job to download their gravatar.

This commit resolves this error that can occur:

```
Job exception: undefined method `email' for nil:NilClass
/var/www/discourse/app/models/user.rb:1204:in `email'
/var/www/discourse/app/jobs/regular/update_gravatar.rb:12:in `execute'
```

This commit also fixes the original spec which actually was wrong. The
job never enqueued in the original spec and so the gravatar was never
actually updated and the test was checking if the two values were the
same, but they were both null and never updated, so of course they were
the same!

A new test has also been added to make sure the gravatar job isn't
enqueued when a user's email is missing.
2020-09-02 20:19:46 -06:00
Gerhard Schlager 31c1d57228
FIX: System message wasn't fully translatable 2020-09-02 15:24:25 +02:00
Arpit Jalan 34478760b2 Update annotations. 2020-09-01 13:41:04 +05:30
Arpit Jalan bb2e830010 Remove out-of-date / non-core annotations. 2020-09-01 12:40:57 +05:30
Arpit Jalan e66eefe179 Annotate models. 2020-09-01 10:22:32 +05:30
Bianca Nenciu f2e14a3946
FEATURE: Add site setting and wizard step to set base font (#10250)
Co-authored-by: Neil Lalonde <neillalonde@gmail.com>
2020-08-31 13:14:09 +03:00
Sam Saffron b31da92ede
DEV: clear last seen cache consistently
Previously in some cases the test suite could fail due to a bad entry in
redis from previous tests

This ensures the correct cache is expired when needed

Additionally improves performance of the redis check
2020-08-31 08:54:42 +10:00
Sam Saffron 8ca8a7edba
DEV: Lint file
Ruby in warning mode complains about `foo *[]` instead `foo(*[])` is
preferred
2020-08-31 08:52:01 +10:00
Kane York 84dba3c1c7
FIX: UserAction did not have a types enum (#10553)
This is important for Data Explorer to provide proper help text.
2020-08-28 11:49:19 -07:00
Penar Musaraj b7cfc9e861
FEATURE: User selectable color schemes (#10544) 2020-08-28 10:36:52 -04:00
Martin Brennan 2352f4bfc7
DEV: Replace SECURE_MEDIA_ROUTE const with other methods (#10545)
This is so if the route changes this const won't be around to bite us, use the Rails route methods instead.
2020-08-28 11:28:11 +10:00
Guo Xiang Tan dbfb2a1e11
Minor follow-up bug fix for 5ff6c10. 2020-08-26 17:25:28 +08:00
Guo Xiang Tan 5ff6c10320
FIX: Ensure that topic timers belonging to trashed topics are destroyed. 2020-08-26 16:11:10 +08:00
Guo Xiang Tan 1e8f216e17
FIX: Ensure autoclose based on last post is executed by system user. 2020-08-26 12:08:46 +08:00
Guo Xiang Tan ffc5bb7278
Revert "FIX: Topic Timer auto opening closed topics (#10524)"
This reverts commit 7cfd5f87ff.
2020-08-26 09:32:11 +08:00
Blake Erickson 7cfd5f87ff
FIX: Topic Timer auto opening closed topics (#10524)
This commit is addressing an issue where it is possible that there could
be multiple topic timer jobs running to close a topic or a weird race
condition state causing a topic that was just closed to be re-opened.

By removing the logic from the Topic Timer model into the Topic Timer
controller endpoint we isolate the code that is used for setting an
auto-open or an auto-close timer to just that functionality making the
topic timer background jobs safer if multiple are running.

Possibly in the future if we would like this logic back in the model a
refactor will be needed where we actually pass in the auto-close and
auto-open action instead of mixing it with the close and open
action that is currently being passed to the controller.
2020-08-26 11:17:12 +10:00
fzngagan c363189858 Accounted for the change while reading the fields added specs to confirm working 2020-08-25 09:52:18 -04:00
fzngagan 352ad826c1 FIX: symbols and strings should be treated similarly while saving custom fields 2020-08-25 09:52:18 -04:00
Roman Rizzi dd13304b81
FEATURE: More API scopes (#10493) 2020-08-24 12:15:08 -03:00
Guo Xiang Tan daddad7fd6
DEV: Update annotations. 2020-08-21 11:36:53 +08:00
Guo Xiang Tan aae9e6e5fd
FIX: `Topic.similar_to` results in error if cooked raw is blank. 2020-08-21 10:54:08 +08:00
Kane York ab0b034404
FIX: Wizard could not send custom color schemes to the client correctly (#10484)
This was likely introduced with the refactor to make ColorSchemeColor a database object. Add a test so this doesn't happen again.

Also test other basics of the WizardSerializer.

For some reason, the .as_json left Ruby objects in; I solved this with a round trip through JSON during the test.
2020-08-20 17:10:33 -07:00
Guo Xiang Tan 337f062f0f
PERF: Defer indexing post for search when saving a post.
Indexing a post for search is slow and there is no reason for us to have
to block saving a post due to search indexing.
2020-08-21 07:52:43 +08:00
Penar Musaraj 3c06dd9b99
FIX: Detect dark scheme server-side for better dark logo support (#10490)
* FIX: Use dark logo when dark scheme is default

* Small refactor
2020-08-20 14:23:18 -04:00
Vinoth Kannan 8348a41124
FEATURE: add `regular_categories` field in site setting & user option. (#10477)
Like "default watching" and "default tracking" categories option now the "regular" categories support is added. It will be useful for sites that are muted by default. The user option will be displayed only if `mute_all_categories_by_default` site setting is enabled.
2020-08-20 00:35:04 +05:30
Neil Lalonde 298ed5d021
FIX: delete unused tags shouldn't delete tags belonging to tag groups 2020-08-19 12:17:49 -04:00
Joffrey JAFFEUX dde3d63b33
DEV: adds event_reminder (27) and event_invitation (28) (#10473)
Note that these events are also added to the list of events sent to push notifications.
2020-08-19 12:07:51 +02:00
Roman Rizzi 390615fbcd
UX: Help users understand the meaning of each scope. (#10468) 2020-08-18 15:12:04 -03:00
Penar Musaraj 882b0aac19
DEV: Let themes extend color definitions (#10429)
Themes can now declare custom colors that get compiled in core's color definitions stylesheet, thus allowing themes to better support dark/light color schemes. 

For example, if you need your theme to use tertiary for an element in a light color scheme and quaternary in a dark scheme, you can add the following SCSS to your theme's `color_definitions.scss` file: 

```
:root {
  --mytheme-tertiary-or-quaternary: #{dark-light-choose($tertiary, $quaternary)};
}
```

And then use the `--mytheme-tertiary-or-quaternary` variable as the color property of that element. You can also use this file to add color variables that use SCSS color transformation functions (lighten, darken, saturate, etc.) without compromising your theme's compatibility with different color schemes.
2020-08-18 13:02:13 -04:00
Vinoth Kannan 562180dd9a
FEATURE: add option to skip new user tips in first notification. (#10462) 2020-08-18 13:43:40 +05:30
Sam Saffron 38e7b1a049
FIX: when destroying uploads clear card and profile background
There is an fk to user_profile that can make destroying uploads fail
if they happen to be set as user profile.

This ensures we clear this information when destroying uploads.

There are more relationships, but this makes some more progress.
2020-08-18 10:55:16 +10:00
Penar Musaraj 3745f2bb86
FEATURE: Dark mode alternative logos (#10441) 2020-08-17 15:43:20 -04:00
Joffrey JAFFEUX 6d0eb7178d
FIX: prevents errors on /tags when a tag `constructor` exists (#10449)
This is due to js objects having a constructor property:

```
const obj = {};
obj['constructor'] // return [native code] and not undefined
```
2020-08-17 17:40:34 +02:00
Guo Xiang Tan 872ef82f4d
Revert "PERF: Prefer joins over subquery for `User#private_posts_for_user`."
This reverts commit 05b43e5ae4.
2020-08-17 14:58:47 +08:00
Guo Xiang Tan 05b43e5ae4
PERF: Prefer joins over subquery for `User#private_posts_for_user`.
The subquery here prevents the planner from optimizing the query. As
such, we prefer joining against the requried tables instead.
2020-08-17 14:47:48 +08:00
Vinoth Kannan 476d26159a
FEATURE: add new user option `skip_new_user_tips`. (#10437)
And add new site setting `default_other_skip_new_user_tips` in user preferences category.
2020-08-14 19:10:56 +05:30
Martin Brennan 4670b62969
DEV: IMAP log to database (#10435)
Convert all IMAP logging to write to a database table for easier inspection. These logs are cleaned up daily if they are > 5 days old.

Logs can easily be watched in dev by setting DISCOURSE_DEV_LOG_LEVEL=\"debug\" and running tail -f development.log | grep IMAP
2020-08-14 12:01:31 +10:00
Blake Erickson c2f3c0dc44 FIX: generate_topic_thumbnails job infinitely running for corrupted images
It's possible that the original topic image is broken in some form, so
we shouldn't try and generate a topic thumbnail for it. The fix will
prevent the generate_topic_thumbnails job being enqueued every time the
topic is viewed.
2020-08-13 17:08:32 -06:00
Neil Lalonde d65a839577
FEATURE: allow group membership to unmute categories and tags
For sites that are configured to mute some or all categories and tags
for users by default, groups can now be configured to set members'
notification level to normal from the group manage UI.
2020-08-13 17:20:53 -04:00
Vinoth Kannan 310952fd6a FIX: generate topic excerpt when moving posts to new topic.
Currently, it's not generating the excerpt by default. We have to trigger the "Rebuild HTML" action to do it.
2020-08-13 11:30:14 +05:30
Rafael dos Santos Silva 28669dfeb2
PERF: Faster TL3 promotion replies needed calculation (#10416)
Removing the LIMIT makes PostgreSQL use index_posts_on_user_id_and_created_at
which is much faster overall.

Before: 22 seconds
After: 100 ms
2020-08-12 11:28:34 -03:00
Penar Musaraj c05aced094
FIX: Invalidate cache when updating color scheme colors (#10417) 2020-08-11 16:28:59 -04:00
David Taylor fe7a7ecf6c
FIX: Include secure media URLs when linking post uploads (#10404)
Normally, secure media urls are linked like `/secure-media-uploads/...`. In this case, uploads were already being linked correctly.

But sometimes (e.g. when pulling hotlinked onebox images) secure media is referenced with a full domain name (`//example.com/secure-media-uploads`). This commit ensures that those uploads are also linked correctly.
2020-08-10 17:59:29 +01:00
jbrw 3593e582a3
FIX - limit number of embedded media items in a post (#10391)
* FIX - limit number of embedded media items in a post

* Add renamed settings to DeprecatedSettings
2020-08-07 12:08:59 -04:00
Penar Musaraj 9c9aa21726
Add site setting to pick dark mode color scheme (#10390)
Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-08-07 08:52:47 -04:00
Krzysztof Kotlarek fb9c2e5f89
FEATURE: notification for vote plugin (#10384)
This must exist in core as we want to see a correct notification after disable of the voting plugin.
2020-08-07 09:51:16 +10:00
Neil Lalonde 1ca81fbb95
FEATURE: set notification levels when added to a group (#10378)
* FEATURE: set notification levels when added to a group

This feature allows admins and group owners to define default
category and tag tracking levels that will be applied to user
preferences automatically at the time when users are added to the
group. Users are free to change those preferences afterwards.
When removed from a group, the user's notification preferences aren't
changed.
2020-08-06 12:27:27 -04:00
David Taylor ceb858c70a
PERF: Release post_upload records when downloaded image is removed (#10379)
Previously we would unconditionally keep all images downloaded via pull_hotlinked_images, even if they are later removed from the post. This commit removes that logic, and relies on the existing link_post_uploads process to pick up the downloaded images in `cooked`. Specs are added to ensure this is working correctly for regular hotlinked images, and for oneboxes.
2020-08-06 10:06:34 +10:00
David Taylor cb12a721c4
REFACTOR: Refactor pull_hotlinked_images job
This commit should cause no functional change
- Split into functions to avoid deep nesting
- Register custom field type, and remove manual json parse/serialize
- Recover from deleted upload records

Also adds a test to ensure pull_hotlinked_images redownloads secure images only once
2020-08-05 12:14:59 +01:00
Robin Ward a1ec711373 FIX: Previous fix comitted the wrong thing and was broken :( 2020-08-04 13:46:26 -04:00
Robin Ward 6da7a97eee FIX: Exclude shared drafts from digests 2020-08-04 13:35:48 -04:00
Mark VanLandingham b76731d722
FEATURE: Invite emails to groups from add member modal (#10308) 2020-08-04 10:02:01 -05:00
Martin Brennan 5a3494b1e1
FIX: IMAP archive fix and group list mailbox code unification (#10355)
* Fixed an issue I introduced in the last PR where I am just archiving everything regardless of whether it is actually archived in Discourse man_facepalming
* Refactor group list_mailboxes IMAP code to use providers, add specs, and add provider code to get the correct prodivder
2020-08-04 14:19:57 +10:00
Bianca Nenciu 2682da81ad
FIX: Get correct selectable avatar from URL (#10339)
The URL for selectable avatars was 'cooked' which means that the find_by
method was not enough.
2020-08-03 17:15:41 +03:00
Gerhard Schlager 957e851ffe Revert "FIX: Regularly reset unknown extension of uploads"
This reverts commit cc7b24b88b as it shouldn't be needed anymore for new uploads.
2020-08-03 13:37:32 +02:00
Martin Brennan 2920988b3a
FIX: IMAP sync email update uniqueness across groups and minor improvements (#10332)
Adds a imap_group_id column to IncomingEmail to deal with an issue where we were trying to update emails in the mailbox, calling IncomingEmail.where(imap_sync: true). However UID and UIDVALIDITY could be the same across accounts. So if group A used IMAP details for Gmail account A, and group B used IMAP details for Gmail account B, and both tried to sync changes to an email with UID of 3 (e.g. changing Labels), one account could affect the other. This even applied to Archiving!

Also in this PR:

* Fix error occurring if we do a uid_fetch and no emails are returned
* Allow for creating labels within the target mailbox (previously we would not do this, only use existing labels)
* Improve consistency for log messages
* Add specs for generic IMAP provider (Gmail specs still to come)
* Add custom archiving support for Gmail
* Only use Message-ID for uniqueness of IncomingEmail if it was generated by us
* Various refactors and improvements
2020-08-03 13:10:17 +10:00
Blake Erickson 8de635fe92
DEV: Show message when cannot invite user to PM (#10336)
* DEV: Show message when cannot invite user to PM

When inviting a user to a PM return a message that says, "Sorry, this
user can't be invited." if they have been muted or are not in a users
allowed pm users list.

* Minor refactor & improved some text
2020-07-31 09:52:19 -06:00
Simon Cossar 3d9eb3e085
Rename options passed to Readability::Document back to whitelist and blacklist (#10340) 2020-07-30 12:56:48 -07:00
Bianca Nenciu d2380a01b6
FIX: Allow 'c' as a tag (#10305)
The new tag routes allow the use of tag 'c'.
2020-07-29 14:47:12 +03:00
Martin Brennan 36c11cb62b
FIX: Rename delete_when_reminder_sent? bookmark method to avoid conflict with AR (#10333)
I added delete_when_reminder_sent to ignored_columns because it no longer exists and added a shortcut method delete_when_reminder_sent? to the Bookmark model. However I have been seeing some weird errors like:

> Job exception: unknown attribute 'delete_when_reminder_sent' for Bookmark.

So I am very suspicious. I am just renaming the method to auto_delete_when_reminder_sent? to avoid any potential conflicts.

Also found include_bookmark_delete_on_owner_reply? in PostSerializer which is used for nothing; I must have forgotten to delete it before.
2020-07-29 17:02:36 +10:00
Martin Brennan 9e5b213089
FIX: Ensure topic user bookmarked synced on bookmark auto-delete (#10323)
For the following conditions, the TopicUser.bookmarked column was not updated correctly:

* When a bookmark was auto-deleted because the reminder was sent
* When a bookmark was auto-deleted because the owner of the bookmark replied to the topic

This adds another migration to fix the out-of-sync column and also some refactors to BookmarkManager to allow for more of these delete cases. BookmarkManager is used instead of directly destroying the bookmark in PostCreator and BookmarkReminderNotificationHandler.
2020-07-29 09:43:32 +10:00
Guo Xiang Tan 4f8262e0d5
FIX: Cooked snippet of raw in `Topic.similar_to`.
If we don't cook the raw, we end up trying to match uncooked raw against
`TopicSearchData#search_data` which consists of cooked raw.
2020-07-28 15:20:18 +08:00
Guo Xiang Tan 4b21b5aac1
FIX: Handle case where `Post#raw` is blank in `Topic.similar_to`. 2020-07-28 13:23:53 +08:00
Guo Xiang Tan dcfe765dac
Make rubocop happy. 2020-07-28 12:06:38 +08:00
Guo Xiang Tan 597d542c33
FIX: Improve `Topic.similar_to` with better `Topic#title` matches.
This changes PG text search to only match the given title against
lexemes that are formed from the title. Likewise, the given raw will
only be matched against lexemes that are formed from the post's raw.
2020-07-28 12:00:27 +08:00
David Taylor 407bb96a22
FIX: Avoid validation error when deleting users with locked trust level 2020-07-27 17:40:10 +01:00
Krzysztof Kotlarek f2342c2d32
FIX: reserve id for reaction notifications (#10315)
Use id 25 for reaction notifications
2020-07-27 11:39:50 +10:00
Krzysztof Kotlarek e0d9232259
FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
Guo Xiang Tan b979579c1b
DEV: Refactor draft attributes for `CategoryList` and `TopicList`.
Avoid repeating the same logic in a bunch of places which will allow us
to make changes to the draft attributes easier in the future.
2020-07-24 10:11:30 +08:00
jbrw 2aec92d0b4
FEATURE - allow Group Moderators to edit category description (#10292)
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2020-07-23 09:50:00 -04:00
Krzysztof Kotlarek 4b053462c0
FIX: display correct status on unsubscribe page (#10294)
There was a bug that even when `email_digest` was set to false but
`digest_after_minutes` was positive, we were not displaying correct
status.

In addition, the message is improved when the user is unsubscribed +
unsubscribe from all is hidden.
2020-07-23 16:20:10 +10:00
Martin Brennan e027acd367
FIX: Move consts and translations for bookmark auto delete prefs (#10295) 2020-07-23 13:12:28 +10:00
Arpit Jalan db4e310376 DEV: annotate models 2020-07-22 16:02:31 +05:30
Martin Brennan 8dc3543134
FIX: Ignore removed delete_when_reminder_sent bookmarks column 2020-07-22 10:17:21 +10:00
jbrw 0ed784b4fc
FEATURE: Create logs for Group Moderator changes (#10271) 2020-07-21 14:29:02 -04:00
Guo Xiang Tan cebd8da299
FIX: Don't raise error when adding to cache fails in readonly mode. 2020-07-21 12:20:23 +08:00
Martin Brennan 41b43a2a25
FEATURE: Add "delete on owner reply" bookmark functionality (#10231)
This adds an option to "delete on owner reply" to bookmarks. If you select this option in the modal, then reply to the topic the bookmark is in, the bookmark will be deleted on reply.

This PR also changes the checkboxes for these additional bookmark options to an Integer column in the DB with a combobox to select the option you want.

The use cases are:

* Sometimes I will bookmark the topics to read it later. In this case we definitely don’t need to keep the bookmark after I replied to it.
* Sometimes I will read the topic in mobile and I will prefer to reply in PC later. Or I may have to do some research before reply. So I will bookmark it for reply later.
2020-07-21 10:00:39 +10:00
Blake Erickson 690f17bcbe
FEATURE: Allow List for PMs (#10270)
* FEATURE: Allow List for PMs

This feature adds a new user setting that is disabled by default that
allows them to specify a list of users that are allowed to send them
private messages. This way they don't have to maintain a large list of
users they don't want to here from and instead just list the people they
know they do want. Staff will still always be able to send messages to
the user.

* Update PR based on feedback
2020-07-20 15:23:49 -06:00
David Taylor fab8b8649e
PERF: Combine avatar_lookup and primary_group_lookup into user_lookup (#10253)
These two classes were running very similar queries, which could be expensive on large topics
2020-07-17 10:48:08 +01:00
Guo Xiang Tan ff7678e210
FIX: Reindex posts when `Topic#title` or `Category#name` changes. 2020-07-17 11:12:31 +08:00
Blake Erickson cc8540701e FIX: Be sure to use same units when comparing thumbnail size 2020-07-16 14:30:29 -06:00
Roman Rizzi f13ec11c64
FEATURE: Add scopes to API keys (#9844)
* Added scopes UI

* Create scopes when creating a new API key

* Show scopes on the API key show route

* Apply scopes on API requests

* Extend scopes from plugins

* Add missing scopes. A mapping can be associated with multiple controller actions

* Only send scopes if the use global key option is disabled. Use the discourse plugin registry to add new scopes

* Add not null validations and index for api_key_id

* Annotate model

* DEV: Move default mappings to ApiKeyScope

* Remove unused attribute and improve UI for existing keys

* Support multiple parameters separated by a comma
2020-07-16 15:51:24 -03:00
Vinoth Kannan 3252cb847c FIX: : trigger `user_updated` event only if email changed after user creation.
Follow-up to 1460d7957c
2020-07-16 18:21:30 +05:30
Vinoth Kannan 1460d7957c FEATURE: trigger `user_updated` event if email changed.
It will trigger `user_updated` Discourse event if a user_email record is created or updated or destroyed.
2020-07-16 14:20:41 +05:30
Blake Erickson 1e9ce51151
FIX: Prevent thumbnail gen if image too large (#10247)
It's possible through an import or other means to have images larger
than the current max allowed image size in the db.

If this happens the thumbnail generation job will keep running
indefinitely trying to download a new copy of the original but
discarding it because it is larger than the max_file_size eventually
causing this error

`Job exception: undefined method `path' for nil:NilClass`

because the newly downloaded image is now nil.

This fix stops the enqueuing of the `GenerateTopicThumbnails` job for
all images that happen to be larger than the max image size.
2020-07-16 11:15:53 +10:00
jbrw 06073fe8c6
FEATURE: Allow group moderators to close/archive topics
* FEATURE: Allow group moderators to close/archive topics
2020-07-14 12:36:19 -04:00
David Taylor 3d65678a13
DEV: Add timestamp columns to optimized_images table (#10199)
This allows us to filter by created/updated date when comparing to an S3 inventory.
2020-07-14 11:50:33 +01:00
Rafael dos Santos Silva a4d5d1f51d PERF: Add user_id condition so we can use another index in the query 2020-07-13 15:26:56 -03:00
Arpit Jalan ba57dc57cc DEV: use group_ids instead of group_names in invite params 2020-07-13 17:37:04 +05:30
Bianca Nenciu 4492718864
FIX: Skip whisper posts when updating topic like count (#10157) 2020-07-13 16:30:00 +10:00
Dan Ungureanu c72bc27888
FEATURE: Implement support for IMAP and SMTP email protocols. (#8301)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-07-10 12:05:55 +03:00
Guo Xiang Tan cbe1dd8ec7
Revert "FIX: Delete related search data when record has been deleted."
This reverts commit ecc799ab56.

This commit does not fix anything because we've always been deleting
records in `Searchable`.
2020-07-09 10:08:35 +08:00
Arpit Jalan 78beb4368a FIX: 'resend all invite' button was not working as expected 2020-07-08 15:52:44 +05:30
Martin Brennan 6be7a66ba7
FIX: Cap bookmark name at 100 chars and truncate existing names (#10189)
We have a couple of examples of enormous amounts of text being entered in the name column of bookmarks. This is not desirable...it is just meant to be a short note / reminder of why you bookmarked this.

This PR caps the column at 100 characters and truncates existing names in the database to 100 characters.
2020-07-08 17:19:01 +10:00
Martin Brennan 07ad243603
FIX: Stop updating bookmarked column from TopicUser.update_post_action_cache (#10188)
* This is causing issues where sometimes bookmarked is out of sync with what is in the Bookmark table. The BookmarkManager handles updating this column now.
* Add migration to fix bookmarked column that is incorrectly marked false when a Bookmark record exists.
2020-07-08 15:27:42 +10:00
Kris 66257ca8b6 FEATURE: Add "smallest" option to user text size preferences 2020-07-07 13:08:19 -04:00
Dan Ungureanu e08b860e88
FIX: Improve category hashtag lookup (#10133)
* FIX: Improve category hashtag lookup

This commit improves support for sub-sub-categories and does not include
the ID of the category in the slug, which fixes the composer preview.

* FIX: Sub-sub-categories can be mentioned using only two levels

* FIX: Remove support for three-level hashtags

* DEV: Simplify code
2020-07-07 10:19:01 +10:00
Blake Erickson 2e13ed061d FIX: Generate Topic Thumbnails nil to Array error
If any value, including nil, is passed in as an argument the default
won't be set, so we need to handle when a non-Array value is passed in
to the `generate_thumbnails!` method.
2020-07-06 15:35:33 -06:00
Jarek Radosz 64ce12a758
FIX: `OptimizedImage#filesize` (#10095)
`OptimizedImage#filesize` calls `Discourse.store.download` with an OptimizedImage as an argument. It would in turn attempt to call `#original_filename` and `#secure?` on that object. Both would fail as these methods do not exist on OptimizedImage, only on Upload. We didn't know about these issues because:
1. `#calculate_filesize` is not called often, because the filesize is saved on OptimizedImage creation, so it's used mostly for manual filesize recalculation
2. we were using `rescue nil` which swallows all errors
2020-07-06 17:01:29 +02:00
David Taylor cb1b472a0f
PERF: Do not include thumbnail information in default topic list payload (#10163)
Now it is only included when a theme/plugin has requested it.
2020-07-06 10:59:21 +01:00
Régis Hanol 48b4ed41f5 FIX: uploading an existing image as a site setting
The previous fix (f43c0a5d85) wasn't working for images that were already uploaded.
The "metadata" (eg. 'for_*' and 'secure' attributes) were not added to existing uploads.

Also used 'Upload.get_from_url' is the admin/site_setting controller to properly retrieve
an upload from its URL.

Fixed the Upload::URL_REGEX to use the \h (hexadecimal) for the SHA

Follow-up-to: f43c0a5d85
2020-07-03 19:16:54 +02:00
Roman Rizzi 2df388ffd7
DEV: Plugins can extend ReviewableScore types. (#10156) 2020-07-02 11:47:43 -03:00
Guo Xiang Tan ecc799ab56
FIX: Delete related search data when record has been deleted. 2020-07-02 12:24:59 +08:00
Régis Hanol 860deeb072 FIX: identify slug-less topic urls everywhere
In 91c89df6, I fixed the onebox to support local topics with a slug-less URL.
This commit fixes all the other spots (search, topic links and user badges) where we look up for a local topic.

Follow-up-to: 91c89df6
2020-06-29 12:31:20 +02:00
Régis Hanol 7109d94ee7 FIX: properly invalidate inline oneboxes when rebaking
When rebaking a post we were invalidating _regular_ oneboxes but not inline oneboxes.

DEV: also renamed 'InlineOneboxer.purge' to 'InlineOneboxer.invalidate' to keep
the API consistent with 'Oneboxer.invalidate'
2020-06-24 11:54:54 +02:00
Kane York 52278ce6fd
FIX: Use Discourse.system_user when we need a placeholder admin (#9781) 2020-06-24 15:51:30 +10:00
Jarek Radosz f28ea4751b FIX: A variable name typo
Not that this whole method is used much anymore.
2020-06-19 19:29:19 +02:00
Jarek Radosz 58a5293d9b
FIX: Delete PostUploads on Post deletion (#10090)
Related uploads will then be removed by Jobs::CleanUpUploads
2020-06-19 17:45:08 +02:00
Vinoth Kannan ae47bcb269
DEV: move user references deletion code to `before_destroy`. (#10085)
Moving the `delete_source_user_references` method code from user merger service to user model.
2020-06-18 17:42:39 -04:00
Cassius d13d5e2b99
FIX: Allow multiple hyphens in host (#10078)
* FIX: Allow multiple hyphens in host

* FIX: added spec for multiple hyphens
2020-06-18 13:58:47 -04:00
Bernhard Suttner e31471585a
DEV: allow to have duplicate topic titles if categegory is different (#10034)
Co-authored-by: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-06-18 11:19:47 -04:00
Dan Ungureanu d21a08c284
DEV: Deprecate Category#url_with_id in favor of Category#url (#9972) 2020-06-18 11:32:14 +03:00
David Taylor 6caad5c083
FIX: Do not send staff welcome message if user already has role 2020-06-17 12:12:55 +01:00
Joffrey JAFFEUX 9da3a7f436
FEATURE: allows published pages to be public (#10053) 2020-06-17 12:42:20 +02:00
David Taylor a2f80670e1
FIX: Do not count youtube thumbnail when counting post images (#10049) 2020-06-15 20:25:30 +01:00
Arpit Jalan a94387c088 FIX: check for inviter group permissions at the time of redeeming invite 2020-06-15 14:44:35 +05:30
Guo Xiang Tan 402b80f306
DEV: Make rails_failover compatible with `SKIP_DB_AND_REDIS` env. 2020-06-15 16:23:24 +08:00
Guo Xiang Tan 58e52c0e4f
DEV: Use rails_failover gem for ActiveRecord and Redis failover handling 2020-06-15 15:47:07 +08:00
Guo Xiang Tan d8cd912769
DEV: Switch to db config to disable advisory locks. 2020-06-15 14:33:41 +08:00
Sam Saffron 7211394e4d
FEATURE: include tags in unread/new messages
We need this information to maintain counts on tracked/watched
tags.
2020-06-12 16:20:30 +10:00
Guo Xiang Tan 2fbd4eb11f
DEV: Remove double query in `Group#remove_review_group`.
Probably a mistake during commit.
2020-06-12 10:14:45 +08:00
Dan Ungureanu b7e70850e4
FIX: Allow users to add emails which were deleted before 2020-06-11 14:54:11 +03:00
Sam Saffron a26b490047
FIX: present correct new/unread counts when filtered by tag
Previously we would incorrectly ignore tags.

This ensures tracking state is properly shipped to client if
show_filter_by_tag is enabled.
2020-06-11 16:47:57 +10:00
Dan Ungureanu 5bfe1ee4f1
FEATURE: Improve UX support for multiple email addresses (#9691) 2020-06-10 19:11:49 +03:00
Dan Ungureanu 3a7ca97c36
FIX: Use include-subcategories filter in report export (#10007)
Some filters were renamed and the conversion of the filter names and arguments
was removed.
2020-06-10 18:57:39 +03:00
David Taylor ae6c4cd237
FIX: Allow merging users when email domains are restricted 2020-06-09 18:51:59 +01:00