Commit Graph

966 Commits

Author SHA1 Message Date
David Taylor 68b4fe4cf8
SECURITY: Expand and improve SSRF Protections (#18815)
See https://github.com/discourse/discourse/security/advisories/GHSA-rcc5-28r3-23rr

Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
Co-authored-by: Daniel Waterworth <me@danielwaterworth.com>
2022-11-01 16:33:17 +00:00
Osama Sayegh fa9e708cb7
FIX: Don't notify topic author about small action posts (#18789) 2022-10-31 06:26:20 +08:00
Alan Guo Xiang Tan 4244b1c57d
FIX: Ignore unique conflicts when backfilling sidebar defaults (#18785)
`insert_all!` raises an error when the insertion violates any unique
constraints which is not what we want here.

Follow-up to 1b56a55f50
2022-10-28 07:47:41 +08:00
Alan Guo Xiang Tan 1b56a55f50
DEV: Sidebar default tags and categories are determined at user creation (#18620)
The previous sidebar default tags and categories implementation did not
allow for a user to configure their sidebar to have no categories or
tags. This commit changes how the defaults are applied. When a user is being created,
we create the SidebarSectionLink records based on the `default_sidebar_categories` and
`default_sidebar_tags` site settings. SidebarSectionLink records are
only created for categories and tags which the user has visibility on at
the point of user creation.

With this change, we're also adding the ability for admins to apply
changes to the `default_sidebar_categories` and `default_sidebar_tags`
site settings historically when changing their site setting. When a new
category/tag has been added to the default, the new category/tag will be
added to the sidebar for all users if the admin elects to apply the changes historically.
Like wise when a tag/category is removed, the tag/category will be
removed from the sidebar for all users if the admin elects to apply the
changes historically.

Internal Ref: /t/73500
2022-10-27 06:38:50 +08:00
Osama Sayegh 43ef44127c
UX: Send notification of type `replied` to topic author if they're watching the topic (#18684)
Related to aeee7ed.

Before the change in aeee7ed, notifications for direct replies to your posts and notifications for replies in watched topics looked the same in the notifications menu -- they both used the arrow icon.

We decided in aeee7ed to distinguish them by changing "watched topics" notifications to use the bell icon because it was confusing for users who watch topics to see the same icon for direct replies and "watched topics". However, that change also means that non-power/new users who receive replies to topics _they create_ will get notifications with the bell icon because technically they're watching the topic, but the arrow icon is more appropriate for this case because we use it throughout the app to indicate "replies".

This commit adds a special-case so that if a user is watching a topic AND the topic is created by them, they receive notifications with the arrow icon (type `replied`) instead of the bell icon (type `posted`) for new posts in the topic.

Internal topic: t/79051.
2022-10-25 11:53:35 +03:00
Martin Brennan 0730a56ce7
FEATURE: Generic hashtag autocomplete sorting (#18718)
Adds sorting for the HashtagAutocompleteService to
sort the results by case-insensitive text _within_
the type sort order specified by the params. This
should fix some flaky specs as well.
2022-10-25 08:59:17 +10:00
Martin Brennan 7c25597da2
FEATURE: Generic hashtag autocomplete part 1 (#18592)
This commit adds a new `/hashtag/search` endpoint and both
relevant JS and ruby plugin APIs to handle plugins adding their
own data sources and priority orders for types of things to search
when `#` is pressed.

A `context` param is added to `setupHashtagAutocomplete` which
a corresponding chat PR https://github.com/discourse/discourse-chat/pull/1302
will now use.

The UI calls `registerHashtagSearchParam` for each context that will
require a `#` search (e.g. the topic composer), for each type of record that
the context needs to search for, as well as a priority order for that type. Core
uses this call to add the `category` and `tag` data sources to the topic composer.

The `register_hashtag_data_source` ruby plugin API call is for plugins to
add a new data source for the hashtag searching endpoint, e.g. discourse-chat
may add a `channel` data source.

This functionality is hidden behind the `enable_experimental_hashtag_autocomplete`
flag, except for the change to `setupHashtagAutocomplete` since only core and
discourse-chat are using that function. Note this PR does **not** include required
changes for hashtag lookup or new styling.
2022-10-19 14:03:57 +10:00
Krzysztof Kotlarek 243efa8931
FEATURE: allow user to set preferred sidebar list destination (#18594)
User can choose between latest or new/unread and that preference will affect behavior of sidebar links.
2022-10-18 13:21:52 +11:00
Andrei Prigorshnev 0fe111e492
FEATURE: add user status to user preferences (#18532) 2022-10-12 23:35:25 +04:00
Bianca Nenciu 7611fec0da
FEATURE: Implement new onboarding popups (#18362)
This commit introduces a new framework for building user tutorials as
popups using the Tippy JS library. Currently, the new framework is used
to replace the old notification spotlight and tips and show a new one
related to the topic timeline.

All popups follow the same structure and have a title, a description and
two buttons for either dismissing just the current tip or all of them
at once.

The state of all seen popups is stored in a user option. Updating
skip_new_user_tips will automatically update the list of seen popups
accordingly.
2022-10-12 18:38:45 +03:00
Loïc Guitaut 26fe047724 DEV: Use AR enums in reviewables related code
This is the first patch of many to replace our custom enums in Ruby by
the ones provided by `ActiveRecord`.
2022-09-22 14:44:27 +02:00
Roman Rizzi 08cb9ecca4
FIX: Don't delete previous messages when we're inside the `sent_recently` window. (#18239)
`delete_previous!` deletes existing topics even when we cannot send a new one due to the `limit_once_per` option. The dashboard problems PM gets deleted the next time the job runs (30 minutes), so the inbox could be empty when
admins click on the summary notification.
2022-09-13 12:43:24 -03:00
Bianca Nenciu 0f70eae7b0
FIX: Create single notification per post and user (#18091)
A user could receive more than a notification for the same post if they
watched both the categories and tags at different levels. This commit
makes sure that only the watching notification is created.

* Add DiscourseEvent before post notifications are created
2022-09-01 16:07:18 +03:00
Bianca Nenciu 6564d04e0d
FIX: Do not notify admins watching PM tags (#18103)
Admins received notifications if a PM was tagged with a tag they
watched even if they were not invited to the PM.
2022-08-29 16:11:11 +03:00
Bianca Nenciu 0d8ecab362
FIX: Restore trust level when leaving group (#17954)
If a user was granted a trust level, joined a group that granted a trust
level and left the group, the trust level was reset. This commit tries
to restore the last known trust level before joining the group by
looking into staff logs.

This commit also migrates old :change_trust_level user history records
to use previous_value and new_value fields.
2022-08-29 13:00:48 +03:00
Martin Brennan 49a70a37f1
FIX: Remove last_unread_post excerpt logic for bookmarks (#17979)
The logic to determine what post excerpt to show for
a topic-level bookmark based on the last unread post
was complex and slow, so we decided to remove it and
always just use the first post excerpt.

This commit also fixes an issue where a couple of
instances of for_topic were missed when doing the
Bookmarkable refactors, so:

1. Clicking the topic bookmark link was not taking
   the user to the last unread post
2. When replying to a topic where there was a topic
   level bookmark with the auto delete preference
   of "on owner reply", we were not removing the
   bookmark from the UI correctly.

A test has been added for the former, the latter would
be quite time-consuming to test and not really worth
it considering it's quite an edge case UI bug.
2022-08-19 09:35:25 +10:00
David Taylor 913db5d546
PERF: Only load the current user's topic_user for bookmarks list (#17873)
Previously, for every bookmarked topic, all topic_user records were being preloaded. Only the current user's record is actually required.

This commit introduces a new `perform_custom_preload!` API which bookmarkables can use to add custom preloading logic. We use this in topic_bookmarkable to load just the topic_user data we need (in the same way as `topic_list.rb`).

Co-authored-by: Blake Erickson <o.blakeerickson@gmail.com>
2022-08-17 09:40:24 +08:00
Bianca Nenciu 2db076f9c8
FIX: Don't notify editor when category or tag change (#17833)
When a user was editing a topic they were also receiving a notification
if they were watching any of the new category or tags.
2022-08-10 18:55:29 +03:00
Bianca Nenciu d5dc4ca0e9
FIX: Make word watcher work with nil strings (#17830)
Censoring or replacing nil strings raised an error.
2022-08-08 16:34:51 -03:00
David Taylor 381365facc
FIX: Update word_watcher cache key following schema change (#17755)
862007fb18 introduced a change to the format that watched words are cached in Redis. Newly-deployed versions of the app were attempting to load the old-format data from Redis, leading to a server error. This commit introduces a CACHE_VERSION constant which we can easily bump when making changes to the cache schema.
2022-08-02 12:11:08 +01:00
Selase Krakani 862007fb18
FEATURE: Add support for case-sensitive Watched Words (#17445)
* FEATURE: Add case-sensitivity flag to watched_words

Currently, all watched words are matched case-insensitively. This flag
allows a watched word to be flagged for case-sensitive matching.
To allow allow for backwards compatibility the flag is set to false by
default.

* FEATURE: Support case-sensitive creation of Watched Words via API

Extend admin creation and upload of Watched Words to support case
sensitive flag. This lays the ground work for supporting
case-insensitive matching of Watched Words.

Support for an extra column has also been introduced for the Watched
Words upload CSV file. The new column structure is as follows:

 word,replacement,case_sentive

* FEATURE: Enable case-sensitive matching of Watched Words

WordWatcher's word_matcher_regexp now returns a list of regular
expressions instead of one case-insensitive regular expression.

With the ability to flag a Watched Word as case-sensitive, an action
can have words of both sensitivities.This makes the use of the global
Regexp::IGNORECASE flag added to all words problematic.

To get around platform limitations around the use of subexpression level
switches/flags, a list of regular expressions is returned instead, one for each
case sensitivity.

Word matching has also been updated to use this list of regular expressions
instead of one.

* FEATURE: Use case-sensitive regular expressions for Watched Words

Update Watched Words regular expressions matching and processing to handle
the extra metadata which comes along with the introduction of
case-sensitive Watched Words.

This allows case-sensitive Watched Words to matched as such.

* DEV: Simplify type casting of case-sensitive flag from uploads

Use builtin semantics instead of a custom method for converting
string case flags in uploaded Watched Words to boolean.

* UX: Add case-sensitivity details to Admin Watched Words UI

Update Watched Word form to include a toggle for case-sensitivity.
This also adds support for, case-sensitive testing and matching of  Watched Word
in the admin UI.

* DEV: Code improvements from review feedback

 - Extract watched word regex creation out to a utility function
 - Make JS array presence check more explicit and readable

* DEV: Extract Watched Word regex creation to utility function

Clean-up work from review feedback. Reduce code duplication.

* DEV: Rename word_matcher_regexp to word_matcher_regexp_list

Since a list is returned now instead of a single regular expression,
change `word_matcher_regexp` to `word_matcher_regexp_list` to better communicate
this change.

* DEV:  Incorporate WordWatcher updates from upstream

Resolve conflicts and ensure apply_to_text does not remove non-word characters in matches
that aren't at the beginning of the line.
2022-08-02 10:06:03 +02:00
Alan Guo Xiang Tan 3bd5f2d411
DEV: Introduce SiteSetting to enable/disable Sidebar. (#17662)
This commit removes the ability to enable/disable the Sidebar on a per
user basis and introduces a site wide setting. For testing purposes, sidebar can be enabled/disabled via the `enable_sidebar=1` or `enable_sidebar=0` query param.
2022-07-27 13:42:26 +08:00
Bianca Nenciu 5f13ca5e54
FIX: Don't cook user fields to apply watched words (#17590)
The previous method for reused the PrettyText logic which applied the
watched word logic, but had the unwanted effect of cooking the text too.
This meant that regular text values were converted to HTML.

Follow up to commit 5a4c35f627.
2022-07-26 18:15:42 +03:00
Alan Guo Xiang Tan 78427e0797
DEV: Refactor user_badge_granted DiscourseEvent logic (#17579)
Follow-up to 02ce9b8a62
2022-07-22 09:06:02 +08:00
Joffrey JAFFEUX 02ce9b8a62
FIX: badge backfilling triggers :user_badge_granted DiscourseEvent (#17514) 2022-07-20 09:33:07 +08:00
Martin Brennan 0ca1152c1c
DEV: Add bookmark_id to bookmark reminder_handler notifications (#17547)
This is so we can join the Notification table onto the
Bookmark table. A slight refactor was needed to ensure
that the required values are always included and the
consumer does not need to think about this.

The discourse-chat and discourse-data-explorer plugins
will be updated to take advantage of this commit.
2022-07-18 12:51:57 +10:00
Martin Brennan 6b2ea1b47b
FIX: Change UserCommScreener to use user_ids (#17489)
It makes more sense to use user_ids for the UserCommScreener
introduced in fa5f3e228c since
in most cases the ID will be available, not the username. This
was discovered while starting work on a plugin that will
use this. In the cases where only usernames are available
the extra query is negligble.
2022-07-14 15:23:09 +10:00
Martin Brennan fa5f3e228c
DEV: Refactor user mute/ignore/disallow PM checks into central class (#17366)
The idea behind this refactor is to centralise all of the user ignoring / muting / disallow PM checks in a single place, so they can be used consistently in core as well as for plugins like chat, while improving the main bulk of the checks to run in a single fast non-AR query.

Also fixed up the invite error when someone is muting/ignoring the user that is trying to invite them to the topic.
2022-07-13 13:58:01 +10:00
David Taylor 2d5d15b4bb
FIX: Ensure pull-hotlinked can rewrite lone oneboxes (#17354)
Mutating the `raw` variable like this would cause issues upstream, meaning that the modification is not persisted. Instead, we should allocate a new string like the other replacement methods.
2022-07-06 11:46:33 +01:00
David Taylor 78e03649ab
FIX: Replace onebox markdown when pulling hotlinked image (#17328)
If an image is oneboxed directly, then we should replace the onebox URL with a markdown image tag. This ensures that the wrapper link points to the downloaded version rather than the original.

This regressed in bf6f8299
2022-07-05 10:47:10 +08:00
Alan Guo Xiang Tan 3266350e80
FEATURE: Decouple category/tag presence in sidebar from notifi level (#17273) 2022-06-30 14:54:20 +08:00
Jarek Radosz 20e34b5da6
DEV: Stabilize watched words order (#17215)
Fixes a flaky spec:

```
  1) WordWatcher.word_matcher_regexp format of the result regexp is correct when watched_words_regular_expressions = true
     Failure/Error: expect(regexp.inspect).to eq("/(#{word1})|(#{word2})/i")

       expected: "/(word35)|(word36)/i"
            got: "/(word36)|(word35)/i"

       (compared using ==)
     # ./spec/services/word_watcher_spec.rb:19:in `block (4 levels) in <main>'
```
2022-06-23 15:38:12 +02:00
Martin Brennan a176b57be0
FIX: Use bookmarkable pattern for bookmark cleanup (#17202)
We have a `cleanup!` class method on bookmarks that deletes
bookmarks X days after their related record (post/topic) are
deleted. This commit changes this method to use the
registered_bookmarkables for this instead, and each bookmarkable
type can delete related bookmarks in their own way.
2022-06-23 14:09:39 +10:00
Isaac Janzen 6ae761604a
FIX: Send quote notifications to correct users when prioritizing full names (#17030) 2022-06-09 11:52:28 -05:00
Sam 7b4e338c0e
FIX: fallback to default push notification icon if none exists (#16961)
Due to some changes we started notifying via push notifications on other
families of notifications. There are a total of about 30 or so possible
notification you could get, some can be pushed.

This fallback means that if for any reason we are unable to find an icon
for a push notification we just fallback to the Discourse logo.

Also go with a simple reply icon for watching first post.

Note, that in production `image_url` can return an exception if an image is
missing. This is not the case in test / development.
2022-06-01 12:00:05 +10:00
Bianca Nenciu 7328a2bfb0
FIX: Apply censored words to inline onebox (#16873)
Censored watched words were not censored inside the title of an inline
oneboxes. Malicious users could exploit this behaviour to insert bad
words. The same issue has been fixed for regular Oneboxes in commit
d184fe59ca.
2022-05-25 14:51:47 +03:00
Penar Musaraj ebdfc536dd
Revert "FEATURE: Include participants in PN search data (#16855)" (#16904)
This reverts commit 71c74a262d.
2022-05-25 15:08:36 +10:00
Alan Guo Xiang Tan 6edf101d5f
DEV: Minor improvements to WordWatcher (#16735)
Follow-up to fd1dc91eed
2022-05-24 10:23:54 +08:00
David Taylor bf6f8299a7 FEATURE: Pull hotlinked images immediately after posting
Previously, with the default `editing_grace_period`, hotlinked images were pulled 5 minutes after a post is created. This delay was added to reduce the chance of automated edits clashing with user edits.

This commit refactors things so that we can pull hotlinked images immediately. URLs are immediately updated in the post's `cooked` HTML. The post's raw markdown is updated later, after the `editing_grace_period`.

This involves a number of behind-the-scenes changes including:

- Schedule Jobs::PullHotlinkedImages immediately after Jobs::ProcessPost. Move scheduling to after the `update_column` call to avoid race conditions

- Move raw changes into a separate job, which is delayed until after the ninja-edit window

- Move disable_if_low_on_disk_space logic into the `pull_hotlinked_images` job

- Move raw-parsing/replacing logic into `InlineUpload` so it can be easily be shared between `UpdateHotlinkedRaw` and `PullUserProfileHotlinkedImages`
2022-05-23 14:28:02 +01:00
Gerhard Schlager 1e1b85c214
FIX: Site setting changes for boolean should be logged as true/false (#16888)
Previously true/false sometimes was logged as t or f
2022-05-23 11:23:10 +08:00
Penar Musaraj 71c74a262d
FEATURE: Include participants in PN search data (#16855)
This makes it easier to find PMs involving a particular user, for
example by searching for `in:messages thisUser` (previously, that query
would only return results in posts where `thisUser` was in the post body).
2022-05-18 10:34:01 -04:00
Alan Guo Xiang Tan fd1dc91eed
DEV: Don't cache watched words in test env (#16731)
The cache was causing state to leak between tests since the `WatchedWord` record in the DB would have been rolled back but `WordWatcher` still had the word in the cache.
2022-05-12 14:45:05 +08:00
David Taylor 67b23c0e22
PERF: Update all user_histories with one query in UserDestroyer (#16717)
7a284164 previously switched the UserDestroyer to use find_each when iterating over UserHistory records. Unfortunately, since this logic is wrapped in a transaction, this didn't actually solve the memory usage problem. ActiveRecord maintains references to all modified models within a transaction.

This commit updates the logic to use a single SQL query, rather than updating models one-by-one
2022-05-11 13:39:31 +01:00
Arpit Jalan 977da1da77
FIX: do not log category custom fields changes if the value is unchanged (#16712) 2022-05-11 13:51:51 +05:30
Martin Brennan 907adce1cb
FIX: Use registered bookmarkables for BookmarkManager (#16695)
These validate/after_create/after_destroy methods were added
back in b8828d4a2d before
the RegisteredBookmarkable API and pattern was nailed down.
This commit updates BookmarkManager to call out to the
relevant bookmarkable for these and bookmark_metadata for
consistency.
2022-05-11 09:51:03 +10:00
Martin Brennan fbcc35b417
DEV: Remove PostAction/UserAction bookmark refs (#16681)
We have not used anything related to bookmarks for PostAction
or UserAction records since 2020, bookmarks are their own thing
now. Deleting all this is just cleaning up old cruft.
2022-05-10 10:42:18 +10:00
Sam 2df3c65ba9
FIX: add support for pipelined and multi redis commands (#16682)
Latest redis interoduces a block form of multi / pipelined, this was incorrectly
passed through and not namespaced.

Fix also updates logster, we held off on upgrading it due to missing functions
2022-05-10 08:19:02 +10:00
Roman Rizzi 6bdcd7afb2
FEATURE: Promote the "delete group" staff action log. (#16656)
We used to log group deletion as custom, which means we can't later search for them. Include group ID in the details.
2022-05-09 12:12:52 -03:00
Martin Brennan 222c8d9b6a
FEATURE: Polymorphic bookmarks pt. 3 (reminders, imports, exports, refactors) (#16591)
A bit of a mixed bag, this addresses several edge areas of bookmarks and makes them compatible with polymorphic bookmarks (hidden behind the `use_polymorphic_bookmarks` site setting). The main ones are:

* ExportUserArchive compatibility
* SyncTopicUserBookmarked job compatibility
* Sending different notifications for the bookmark reminders based on the bookmarkable type
* Import scripts compatibility
* BookmarkReminderNotificationHandler compatibility

This PR also refactors the `register_bookmarkable` API so it accepts a class descended from a `BaseBookmarkable` class instead. This was done because we kept having to add more and more lambdas/properties inline and it was very messy, so a factory pattern is cleaner. The classes can be tested independently as well.

Some later PRs will address some other areas like the discourse narrative bot, advanced search, reports, and the .ics endpoint for bookmarks.
2022-05-09 09:37:23 +10:00
Joffrey JAFFEUX 81580e559c
FIX: prevents double user_badge_granted event (#16654)
This bug was causing double events to be fired as :user_badge_granted is already called when a `user_badge` is created. More over the signature of the block in the UserBadge code is `badge_id, user_id` not `badge, user_id`.
2022-05-05 15:09:41 +02:00