Commit Graph

916 Commits

Author SHA1 Message Date
Loïc Guitaut 008b700a3f DEV: Upgrade to Rails 7
This patch upgrades Rails to version 7.0.2.4.
2022-04-28 11:51:03 +02:00
Alan Guo Xiang Tan 98c49acad5
DEV: Setup experimental sidebar skeleton (#16575)
* hidden siteSetting to enable experimental sidebar
* user preference to enable experimental sidebar
* `experimental_sidebar_enabled` attribute for current user
* Empty glimmer component for Sidebar
2022-04-28 15:27:06 +08:00
Martin Brennan 3e4621c2cb
FEATURE: Polymorphic bookmarks pt. 2 (lists, search) (#16335)
This pull request follows on from https://github.com/discourse/discourse/pull/16308. This one does the following:

* Changes `BookmarkQuery` to allow for querying more than just Post and Topic bookmarkables
* Introduces a `Bookmark.register_bookmarkable` method which requires a model, serializer, fields and preload includes for searching. These registered `Bookmarkable` types are then used when validating new bookmarks, and also when determining which serializer to use for the bookmark list. The `Post` and `Topic` bookmarkables are registered by default.
* Adds new specific types for Post and Topic bookmark serializers along with preloading of associations in `UserBookmarkList`
* Changes to the user bookmark list template to allow for more generic bookmarkable types alongside the Post and Topic ones which need to display in a particular way

All of these changes are gated behind the `use_polymorphic_bookmarks` site setting, apart from the .hbs changes where I have updated the original `UserBookmarkSerializer` with some stub methods.

Following this PR will be several plugin PRs (for assign, chat, encrypt) that will register their own bookmarkable types or otherwise alter the bookmark serializers in their own way, also gated behind `use_polymorphic_bookmarks`.

This commit also removes `BookmarkQuery.preloaded_custom_fields` and the functionality surrounding it. It was added in 0cd502a558 but only used by one plugin (discourse-assign) where it has since been removed, and is now used by no plugins. We don't need it anymore.
2022-04-22 08:23:42 +10:00
Alan Guo Xiang Tan 6fc52d95de DEV: Remove scoped order warning in PostDestroyer.
`Scoped order is ignored, it's forced to be batch order.`

`find_each` ignores the `order` scope and triggers a warning in
production which is noisy.

Follow-up to 7a284164ce
2022-04-13 09:34:36 +08:00
Bianca Nenciu 86c7e07428
FEATURE: Improve screened IPs roll up and extend for IPv6 (#15585)
This commit improves the logic for rolling up IPv4 screened IP
addresses and extending it for IPv6. IPv4 addresses will roll up only
up to /24. IPv6 can rollup to /48 at most. The log message that is
generated contains the list of original IPs and new subnet.
2022-04-12 21:07:37 +03:00
Penar Musaraj df10a27067
FIX: Exclude automatic anchors from search index (#16396) 2022-04-06 16:06:45 -04:00
Daniel Waterworth 6e9a068e44
FIX: Limit max word length in search index (#16380)
Long words bloat the index for little benefit.
2022-04-06 12:23:30 -05:00
Sam 1598e6b489
FIX: users watching tags in open tag groups not notified (#16384)
All users are members of the EVERYONE group, but this group is special and
is omitted from the group_users table. When checking permission we need to
make sure we also add a bypass.

This also fixes a very buggy test in post_alerter, it was confirming the
broken behavior due to fabricator flow.

When it defined the tag group the everyone group automatically had full access
then the additional permission fabricated just added one more group. After
fix was made to code the test started failing. Fabricators can be risky.
2022-04-06 11:43:57 +10:00
Martin Brennan b982992ef7
FIX: Auto-generated emails causing group SMTP email storm (#16372)
When emailing a group inbox and including other support-type
emails (or even just regular ones with autoresponders) in the
CC field, each automated reply to the group inbox triggered
more emails to be sent out to all CC addresses to notify them
of the new reply, which in turn caused more automated emails
to be sent to the group inbox.

This commit fixes the issue by preventing any emails being sent
by the PostAlerter when the new post has an incoming email record
which is_auto_generated, which we detect in Email::Receiver.
2022-04-05 13:18:49 +10:00
Faizaan Gagan 1da4b9eeb3
FIX: update 'posted' column on post owner change (#16367)
Fixes the issue where making a user x as owner of a post doesn't
cause the concerned topic to be listed in new owner's `My Posts`
top menu filter

per https://meta.discourse.org/t/199369
2022-04-04 22:42:38 +05:30
David Taylor 7a284164ce
PERF: Update UserDestroyer to fetch histories and actions in batch (#16316)
This is much more memory-efficient when there are 1000s of rows
2022-03-29 16:56:55 +01:00
David Taylor b2a8dc4c0f
FIX: Maintain HTML `<img` when downloading remote images (#16278)
Under some conditions, replacing an `<img` with `![]()` can break rendering, and make the image disappear.

Context at https://meta.discourse.org/t/152801
2022-03-29 10:55:10 +01:00
Penar Musaraj 94750c81fa
FIX: Update group inbox notifications on archive/unarchive (#16152) 2022-03-11 11:57:47 +01:00
Bianca Nenciu 34b4b53bac
FEATURE: Use Postgres unaccent to ignore accents (#16100)
The search_ignore_accents site setting can be used to make the search
indexer remove the accents before indexing the content. The unaccent
function from PostgreSQL is better than Ruby's unicode_normalize(:nfkd).
2022-03-07 23:03:10 +02:00
Dan Ungureanu 39ab14531a
FEATURE: API to create user's associated account (#15737)
Discourse users and associated accounts are created or updated when a
user logins or connects the account using their account preferences.
This new API can be used to create associated accounts and users too,
if necessary.
2022-03-03 18:17:02 +02:00
Jarek Radosz 2fc70c5572
DEV: Correctly tag heredocs (#16061)
This allows text editors to use correct syntax coloring for the heredoc sections.

Heredoc tag names we use:

languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS
other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 20:50:55 +01:00
Jarek Radosz 6f6406ea03
DEV: Fix random typos (#16066) 2022-02-28 10:20:58 +08:00
Martin Brennan 2d30dd439f
DEV: Add chat_quoted notification type (#15968)
This is needed for the notification sent when quoting
chat messages inside a post.
2022-02-16 15:22:08 +10:00
Alan Guo Xiang Tan 6c374cf158
DEV: Improve ArgumenError raised in PostOwnerChanger. (#15907)
Currently, it doesn't provide any context about the arguement which
caused the error.
2022-02-16 12:52:20 +11:00
Martin Brennan f9ec2b90a0
DEV: Drop user_stats count column constraints (#15949)
We added this constraint in 5bd55acf83
but it is causing problems in hosted sites and is catching the
issue too far down the line. This commit removes the constraint
for now, and also fixes an issue found with PostDestroyer
which wasn't using the UserStatCountUpdater when updating post_count
and thus was causing negative numbers to occur.
2022-02-16 12:49:11 +11:00
Alan Guo Xiang Tan 81791a821c
DEV: Fix small typo. (#15904) 2022-02-11 11:51:51 +08:00
Alan Guo Xiang Tan b876ff6281
FIX: Update user stat counts when post/topic visibility changes. (#15883)
Breakdown of fixes in this commit:

* `UserStat#topic_count` was not updated when visibility of
the topic changed.

* `UserStat#post_count` was not updated when post was hidden or
unhidden.

* `TopicConverter` was only incrementing or decrementing the counts by 1
even if a user has multiple posts in the topic.

* The commit turns off the verbose logging by default as it is just
noise to normal users who are not debugging this problem.
2022-02-11 09:00:58 +08:00
Alan Guo Xiang Tan ae0625323a
FIX: Avoid errors when updating post and topic count user stats. (#15876)
In ab5361d69a, we rescue from the PG error
but the transaction is already aborted causing any DB query after to
fail. As such, we avoid triggering the error in the first place by
checking that we would not be insertin a negative number into the
counter cache.

Follow-up to ab5361d69a
2022-02-09 21:48:18 +08:00
Alan Guo Xiang Tan ab5361d69a
FIX: Avoid raising error when updating post and topic count user stats. (#15873)
There are still spots in the code base which results in us trying to turn the post and topic count negative. However,
we have a job that runs on a daily basis which will correct the count. Therefore, avoid raising an error for now
and log the exception instead.
2022-02-09 12:51:13 +08:00
Dan Ungureanu 820fea835c
FIX: Further reduce the input of to_tsvector (#15716)
Random strings can result into much longer tsvectors. For example
parsing a Base64 string of ~600kb can result in a tsvector of over 1MB,
which is the maximum size of a tsvector.

Follow-up-to: 823c3f09d4
2022-02-07 23:03:01 +02:00
Alan Guo Xiang Tan 5bd55acf83
FIX: Add DB constraints for post & topic counter cache for `UserStat` (#15626)
Ensures that `UserStat#post_count` and `UserStat#topic_count` does not
go below 0. When it does like it did now, we tend to have bugs in our
code since we're usually coding with the assumption that the count isn't
negative.

In order to support the constraints, our post and topic fabricators in
tests will now automatically increment the count for the respective
user's `UserStat` as well. We have to do this because our fabricators
bypasss `PostCreator` which holds the responsibility of updating `UserStat#post_count` and
`UserStat#topic_count`.
2022-02-07 11:23:34 +08:00
David Taylor 863262a5de
DEV: Update more Jobs#enqueue calls to use strings (#15823)
Symbols are converted to strings anyway, so there is no change in behaviour. The latest version of sidekiq introduced a warning for this.
2022-02-04 19:58:34 +00:00
Gerhard Schlager 1fef96a2e7
FIX: Prevent "integer out of range" when merging post timings (#15723) 2022-01-26 23:34:28 +01:00
Alan Guo Xiang Tan 77137c5d29 FIX: Single line emojis has emoji metadata indexed twice.
This commit fixes a bug where we our `HTMLScrubber` was only searching
for emoji img tags which contains only the "emoji" class. However, our emoji image tags
may contain more than just the "emoji" class like "only-emoji" when an
emoji exists by itself on a single line.
2022-01-24 14:03:17 +08:00
Jarek Radosz 5a50f18c0c
DEV: Avoid `$` globals (#15453)
Also:
* Remove an unused method (#fill_email)
* Replace a method that was used just once (#generate_username) with `SecureRandom.alphanumeric`
* Remove an obsolete dev puma `tmp/restart` file logic
2022-01-08 23:39:46 +01:00
Peter Zhu c5fd8c42db
DEV: Fix methods removed in Ruby 3.2 (#15459)
* File.exists? is deprecated and removed in Ruby 3.2 in favor of
File.exist?
* Dir.exists? is deprecated and removed in Ruby 3.2 in favor of
Dir.exist?
2022-01-05 18:45:08 +01:00
Roman Rizzi 23b75d8a2b
FEATURE: Before consolidation callbacks. (#15428)
You can add callbacks that get called before updating an already consolidated notification or creating a consolidated one.

Instances of this rule can add callbacks to access the old notifications about to be destroyed or the consolidated one and add additional data inside the data hash versus having to execute extra queries when adding this logic inside the `set_mutations` block.
2021-12-30 15:40:16 -03:00
Roman Rizzi e005e3f153
DEV: Create post actions without creating a notification and store custom data. (#15397)
I plan to use this in an upcoming discourse-reactions PR, where I want to like a post without notifying the user, so I can instead create a reaction notification.

Additionally, we decouple the a11y attributes from the icon itself, which will let us extend the widget's icon without losing them.
2021-12-27 11:25:37 -03:00
Roman Rizzi 1ad47030fe
REFACTOR: Use a consolidation rule for collapsing likes. (#15393)
This PR moves the behavior from the PostAlerter. We delete an existing liked notification and set the `username2` attribute to the previous `display_username`. We repeat this process unless the last one is old enough or it's not in the most recent ones.
2021-12-23 09:44:03 -03:00
Andrei Prigorshnev c202252190
FEATURE: when suggesting usernames skip input that consist entirely of disallowed characters (#15368) 2021-12-21 21:13:05 +04:00
Andrei Prigorshnev 952bebc5a1
FIX: fallback to `anonymous` not `user` when suggesting usernames for anonymous users (#15354) 2021-12-21 21:09:55 +04:00
Roman Rizzi 0719531bd3
FIX: Notify group members when someone quotes or replies to their post. (#15295)
When a member set a group PM notification level to Normal, we didn't notify group members if someone quoted or replied to one of their posts.
2021-12-15 09:07:39 -03:00
Roman Rizzi b7b61d4b56
FEATURE: A notification consolidation plan for keeping the latest one. (#15249)
We previously used ConsolidateNotifications with a threshold of 1 to re-use an existing notification and bump it to the top instead of creating a new one. It produces some jumpiness in the user notification list, and it relies on updating the `created_at` attribute, which is a bit hacky.

As a better alternative, we're introducing a new plan that deletes all the previous versions of the notification, then creates a new one.
2021-12-10 10:32:15 -03:00
Roman Rizzi 3602f83cf4
FEATURE: Delete previous reviewable reminders. (#15250)
We send the reminder using the GroupMessage class, which supports removing previous messages. We can't match them by raw because they could mention different moderators. Also, I had to change the subject to remove dynamically generated values, which is necessary for finding them.
2021-12-10 10:17:39 -03:00
Angus McLeod df3886d6e5
FEATURE: Experimental support for group membership via google auth (#14835)
This commit introduces a new site setting "google_oauth2_hd_groups". If enabled, group information will be fetched from Google during authentication, and stored in the Discourse database. These 'associated groups' can be connected to a Discourse group via the "Membership" tab of the group preferences UI. 

The majority of the implementation is generic, so we will be able to add support to more authentication methods in the near future.

https://meta.discourse.org/t/managing-group-membership-via-authentication/175950
2021-12-09 12:30:27 +00:00
Andrei Prigorshnev f3508065a3
FIX: auth incorrectly handles duplicate usernames (#15197) 2021-12-06 20:49:04 +04:00
Roman Rizzi 43903f8dfe
FIX: Updating a consolidated notification should bump it to the top. (#15199)
In the future, it would be better to have a consolidated_at timestamp instead of updating created_at.
2021-12-06 12:31:44 -03:00
Andrei Prigorshnev 1c0022c195
FIX: extract and fix overriding of usernames by external auth (#14637) 2021-12-02 17:42:23 +04:00
Natalie Tay 0f598ca51e
SECURITY: Only show tags to users with permission (#15148) 2021-12-01 10:26:56 +08:00
Roman Rizzi 1fc06520bd
REFACTOR: Improve support for consolidating notifications. (#14904)
* REFACTOR: Improve support for consolidating notifications.

Before this commit, we didn't have a single way of consolidating notifications. For notifications like group summaries, we manually removed old ones before creating a new one. On the other hand, we used an after_create callback for likes and group membership requests, which caused unnecessary work, as we need to delete the record we created to replace it with a consolidated one.

We now have all the consolidation rules centralized in a single place: the consolidation planner class. Other parts of the app looking to create a consolidable notification can do so by calling Notification#consolidate_or_save!, instead of the default Notification#create! method.

Finally, we added two more rules: one for re-using existing group summaries and another for deleting duplicated dashboard problems PMs notifications when the user is tracking the moderator's inbox. Setting the threshold to one forces the planner to apply this rule every time.

I plan to add plugin support for adding custom rules in another PR to keep this one relatively small.

* DEV: Introduces a plugin API for consolidating notifications.

This commit removes the `Notification#filter_by_consolidation_data` scope since plugins could have to define their criteria. The Plan class now receives two blocks, one to query for an already consolidated notification, which we'll try to update, and another to query for existing ones to consolidate.

It also receives a consolidation window, which accepts an ActiveSupport::Duration object, and filter notifications created since that value.
2021-11-30 13:36:14 -03:00
David Taylor 03998e0a29
FIX: Use CDN URL for internal onebox avatars (#15077)
This commit will also trigger a background rebake for all existing posts with internal oneboxes
2021-11-25 12:07:34 +00:00
Dan Ungureanu fa8cd629f1
DEV: Hash tokens stored from email_tokens (#14493)
This commit adds token_hash and scopes columns to email_tokens table.
token_hash is a replacement for the token column to avoid storing email
tokens in plaintext as it can pose a security risk. The new scope column
ensures that email tokens cannot be used to perform a different action
than the one intended.

To sum up, this commit:

* Adds token_hash and scope to email_tokens

* Reuses code that schedules critical_user_email

* Refactors EmailToken.confirm and EmailToken.atomic_confirm methods

* Periodically cleans old, unconfirmed or expired email tokens
2021-11-25 09:34:39 +02:00
Bianca Nenciu 73760c77d9
FEATURE: Mention @here to notify users in topic (#14900)
Use @here to mention all users that were allowed to topic directly or
through group, who liked topics or read the topic. Only first 10 users
will be notified.
2021-11-23 22:25:54 +02:00
Jarek Radosz 68b0fdd2b3
DEV: Fix silence user context message (#15040) 2021-11-22 19:44:20 +01:00
Martin Brennan 9f8ee8f137
FIX: Don't publish notifications to MessageBus for inactive users (#15035)
We are pushing /notification-alert/#{user_id} and /notification/#{user_id}
messages to MessageBus from both PostAlerter and User#publish_notification_state.
This can cause memory issues on large sites with many users. This commit
stems the bleeding by only sending these alert messages if the user
in question has been seen in the last 30 days, which eliminates a large
chunk of users on some sites.
2021-11-22 14:38:49 +11:00