Commit Graph

769 Commits

Author SHA1 Message Date
Martin Brennan b583872eed
DEV: Introduce enabled? API to hashtag data sources (#22632)
We need a nice way to only return some hashtag data
sources based on various site settings. This commit
adds an enabled? method that every hashtag data source
must implement. If this returns false the data source
will not be used at all for hashtag lookups or search.
2023-07-18 09:39:01 +10:00
Roman Rizzi 61aeb2da90
FEATURE: Inline topic summary. Cached version accessible to everyone. (#22551)
* FEATURE:  Inline topic summary. Cached version accessible to everyone.

Anons and non-members of the `custom_summarization_allowed_groups_map` groups can see cached summaries for any accessible topic. After the first 12 hours and if the posts to summarize have changed, allowed users clicking on the button will automatically re-generate it.

* Ensure chat summaries work and prevent model hallucinations when there are no messages.
2023-07-12 11:21:51 -03:00
Krzysztof Kotlarek 0744d242c6
FIX: post alerter notification when topic directly watched (#22433)
In previous PR https://github.com/discourse/discourse/pull/22340 bug was introduced. Notifications were blocked when, even if topic was watched directly. New query is taking TopicUser into consideration.

In addition, in user interface, when `watched_precedence_over_muted` is not set, then value from SiteSetting should be displayed.
2023-07-06 11:27:23 +10:00
Krzysztof Kotlarek 134dcdd63a
FEATURE: allow user to override watched_precedence_over_muted setting (#22340)
Recently, site setting watched_precedence_over_muted was introduced - https://github.com/discourse/discourse/pull/22252

In this PR, we are allowing users to override it. The option is only displayed when the user has watched categories and muted tags, or vice versa.
2023-07-04 15:08:29 +10:00
Roman Rizzi 71ff38bab6
DEV: Fix lint and flaky summarization spec (#22303) 2023-06-27 13:18:10 -03:00
Roman Rizzi f4e7a80600
DEV: Cache summarization strategy results. (#22230)
Updates the interface for implementing summarization strategies and adds a cache layer to summarize topics once.

The cache stores the final summary and each chunk used to build it, which will be useful when we have to extend or rebuild it.
2023-06-27 11:44:34 -03:00
Krzysztof Kotlarek 9cf981f1f1
FEATURE: new watched_precedence_over_muted setting (#22252)
New setting which allow admin to define behavior when topic is in watched category and muted topic and vice versa.

If watched_precedence_over_muted setting is true, that topic is still visible in list of topics and notification is created.

If watched_precedence_over_muted setting is false, that topic is not still visible in list of topics and notification is skipped as well.
2023-06-27 14:49:34 +10:00
Loïc Guitaut 0f4beab0fb DEV: Update the rubocop-discourse gem
This enables cops related to RSpec `subject`.

See https://github.com/discourse/rubocop-discourse/pull/32
2023-06-26 11:41:52 +02:00
Mark VanLandingham 114a9a10b7
DEV: Add spec for notification data modifier (#22223) 2023-06-21 08:57:25 -05:00
Krzysztof Kotlarek 2effcaa0f9
FIX: Update sidebar to be navigation menu (#22101)
Communities can use sidebar or header dropdown, therefore navigation menu is a better name settings in 2 places:

- Old user sidebar preferences;
- Site setting about default tags and categories.
2023-06-15 09:31:28 +10:00
Ted Johansson a674c6c4c2
DEV: Update username in new quote format - Part 1 (#22032)
When we introduced the new quote format with full-name display name:

```
[quote="Ted Johansson, post:1, topic:2, username:ted"]
we overlooked the code responsible for rewriting quotes when a user's name is changed.
```

The functional part of this change adds support for the new quote format in the code that updates quotes when a user's username changes. See the test case in `spec/services/username_changer_spec.rb` for the details.

In addition, this change adds a regression test for PrettyText to cover the new quote format, and extracts the code responsible for rewriting raw and cooked quotes into its own `QuoteRewriter` class. The functionality of the latter is tested through the tests in `spec/services/username_changer_spec.rb`.
2023-06-14 16:14:11 +08:00
Régis Hanol 4cb3412a56
PERF: improve `findAllMatches` speed (#22083)
When we introduced unicode support in the regular expressions used in watched words (9a27803) we didn't realize the cost adding the `u` flag would be.

Turns out, it's pretty bad when you have lots of regular expressions to test. A customer had slightly less than 200 watched words, and it would freeze the browser for about 2s on the first check of those regular expressions (roughly 10ms per regular expression).

This commit introduces a new field (`word`) to the serialized watched words which is then converted to a very fast and cheap regular expression on the client-side. We use that regexp to quicly check whether a matcher is even worth trying so that we don't incure the cost of compiling the expensive unicode regexp.

This commit also busts the `WordWatcher` cache since we added a new field to be serialized.

One nice side effect of using `matchAll` instead of a `while / exec` loop is that the likeliness of having a bad regexp matching infinitely is vastly reduced 🙌
2023-06-13 18:34:28 +02:00
Selase Krakani c45eb8a618
FIX: Create new PM notifications for `watching_first_post` groups (#21997)
At the moment, PMs to groups with default notification level set to
`watching_first_post` do not generate "emailable" notifications. This happens
because, topic user notification level which is indirectly derived
from the group's default notification level is set to `tracking` if the
group's notification level happens to be `watching_first_post`.

This leads to a `group_message_summary` notification being created
instead of a `private_message` notification which results in no email
alerts being sent when a  topic is created.

As this `watching_first_post` --> `tracking` switcheroo appears to be
intentional instead being a bug, this change extends `PostAlerter`'s
`notify_pm_users` method to create a `private_message` notification for
first posts created in a `watching_first_post` group even if the topic
user notification level is set to `tracking`
2023-06-08 17:41:44 +00:00
Sam c2332d7505
FEATURE: reduce avatar sizes to 6 from 20 (#21319)
* FEATURE: reduce avatar sizes to 6 from 20

This PR introduces 3 changes:

1. SiteSetting.avatar_sizes, now does what is says on the tin.
previously it would introduce a large number of extra sizes, to allow for
various DPIs. Instead we now trust the admin with the size list.

2. When `avatar_sizes` changes, we ensure consistency and remove resized
avatars that are not longer allowed per site setting. This happens on the
12 hourly job and limited out of the box to 20k cleanups per cycle, given
this may reach out to AWS 20k times to remove things.

3.Our default avatar sizes are now "24|48|72|96|144|288" these sizes were
very specifically picked to limit amount of bluriness introduced by webkit.
Our avatars are already blurry due to 1px border, so this corrects old blur.

This change heavily reduces storage required by forums which simplifies
site moves and more.

Co-authored-by: David Taylor <david@taylorhq.com>
2023-06-01 10:00:01 +10:00
Osama Sayegh bb3c05ba0e
DEV: Allow plugins to hook into user preferences update process on the server (#21737)
This commit introduces a new `within_user_updater_transaction` event that's triggered inside the transaction that saves user updates in `UserUpdater`. Plugins can hook into the transaction using the event to include custom changes in the transaction. Callbacks for this event receive 2 arguments:

1. the user being saved
2. the changed attributes that are passed to `UserUpdater`.

There's also new modifier in this commit called `users_controller_update_user_params` to allow plugins to allowlist custom params in the `UsersController` which eventually end up getting passed as attributes to the `UserUpdater` and the new `within_user_updater_transaction` event where they can be used to perform additional updates using the custom params.

-----

New API is used in https://github.com/discourse/discourse-mailinglist-integration/pull/1.
2023-05-26 03:26:38 +03:00
Martin Brennan 0b3cf83e3c
FIX: Do not cook icon with hashtags (#21676)
This commit makes some fundamental changes to how hashtag cooking and
icon generation works in the new experimental hashtag autocomplete mode.
Previously we cooked the appropriate SVG icon with the cooked hashtag,
though this has proved inflexible especially for theming purposes.

Instead, we now cook a data-ID attribute with the hashtag and add a new
span as an icon placeholder. This is replaced on the client side with an
icon (or a square span in the case of categories) on the client side via
the decorateCooked API for posts and chat messages.

This client side logic uses the generated hashtag, category, and channel
CSS classes added in a previous commit.

This is missing changes to the sidebar to use the new generated CSS
classes and also colors and the split square for categories in the
hashtag autocomplete menu -- I will tackle this in a separate PR so it
is clearer.
2023-05-23 09:33:55 +02:00
Natalie Tay 07061410d8
FIX: Anonymizing a user clears their user status too (#21673) 2023-05-22 13:18:09 +08:00
Bianca Nenciu 9a2780397f
FIX: Handle all UTF-8 characters (#21344)
Watched words were converted to regular expressions containing \W, which
handled only ASCII characters. Using [^[:word]] instead ensures that
UTF-8 characters are also handled correctly.
2023-05-15 12:45:04 +03:00
Martin Brennan 86204fa4f0
FIX: Hashtag subcategory ref incorrect when not highest-ranked type (#21163)
This commit fixes the following scenario:

1. The user is searching for hashtags in chat, where the subcategory
   type is not highest-ranked in priority order.
2. There can, but doesn't have to be, a higher-ranked matching chat
   channel that has the same slug as the subcategory.
3. Since it is not the highest-ranked type, the subcategory, which
   normally has a ref of parent:child, has its ref changed to
   child::category, which does not work

This was happening because whenever a hashtag type was not highest
ranked, if _any_ other hashtag results conflicted slugs, we would
append the ::type suffix. Now, we only append this suffix if a
higher-ranked type conflicts with the hashtag, and we use the current ref
to build the new typed ref to preserve this parent:child format as well,
it's more accurate.
2023-04-20 09:03:55 +10:00
Jarek Radosz 29e2e3ff3b
DEV: Fix random typos (#20937) 2023-04-03 19:27:32 +02:00
Jan Cernik afe3e36363
DEV: Remove lazy-yt and replace with lazy-videos (#20722)
- Refactors the old plugin to remove jquery usage
- Adds support for Vimeo videos (default on) and Tiktok (experimental and default off)
2023-03-29 11:54:25 -04:00
Rafael dos Santos Silva 2a7bdb2d66
FIX: Push notification delay should not be longer than specified (#20864)
When user.last_seen was less than push_notification_time_window_mins we
where delaying the notification for the whole
push_notification_time_window_mins PLUS the time the user was away from.

Originally reported in https://meta.discourse.org/t/-/259688
2023-03-28 13:22:54 -03:00
Sam 4a3c13a37b
FIX: search index failing on certain posts (#20736)
During search indexing we "stuff" the index with additional keywords for
entities that look like domain names.

This allows searches for `cnn` to find URLs for `www.cnn.com`

The search stuffing attempted to keep indexes aligned at the correct positions
by remapping the indexed terms. However under certain edge cases a single
word can stem into 2 different lexemes. If this happened we had an off by
one which caused the entire indexing to fail.

We work around this edge case (and carry incorrect index positions) for cases
like this. It is unlikely to impact search quality at all given index position
makes almost no difference in the search algorithm.
2023-03-20 15:43:08 +11:00
Ted Johansson 39c2f63b35 SECURITY: Add FinalDestination::FastImage that's SSRF safe 2023-03-16 15:27:09 -06:00
Gerhard Schlager 12436d054d
DEV: Remove `badge_granted_title` column from `user_profiles` (#20476)
That column is obsolete since we added the `granted_title_badge_id` column in 2019 (56d3e29a69). Having both columns can lead to inconsistencies (mostly due to old data from before 2019).

For example, `BadgeGranter.revoke_ungranted_titles!` doesn't work correctly if `badge_granted_title` is `false` while `granted_title_badge_id` points to the badge that is used as title.
2023-03-08 13:37:20 +01:00
Natalie Tay 44b7706a2b
UX: Skip applying link-type watched words to user custom fields (#20465)
We currently apply type: :link watched words to custom user fields. This makes the user card pretty ugly because we don't allow html / links there. Additionally, the admin UI also does not say that we apply this to custom user fields, but only words in posts.

So this PR is to remove the replacement of link-type watch words for custom user fields.
2023-03-01 10:43:34 +08:00
Sam e636abeb0d
FIX: do not notify admins on suppressed categories (#20238)
* FIX: do not notify admins on suppressed categories

Avoid notifying admins on categories where they are not explicitly members
in cases where SiteSetting.suppress_secured_categories_from_admin is
enabled.

This helps keep notification stream clean and avoids admins mistakenly
being invited to discussions that should be suppressed
2023-02-14 16:45:06 +11:00
Ted Johansson 25a226279a
DEV: Replace #pluck_first freedom patch with AR #pick in core (#19893)
The #pluck_first freedom patch, first introduced by @danielwaterworth has served us well, and is used widely throughout both core and plugins. It seems to have been a common enough use case that Rails 6 introduced it's own method #pick with the exact same implementation. This allows us to retire the freedom patch and switch over to the built-in ActiveRecord method.

There is no replacement for #pluck_first!, but a quick search shows we are using this in a very limited capacity, and in some cases incorrectly (by assuming a nil return rather than an exception), which can quite easily be replaced with #pick plus some extra handling.
2023-02-13 12:39:45 +08:00
Rafael dos Santos Silva 6e522e4aad
DEV: Move to Sass compilation to dart-sass (#19910)
This PR is a major change to Sass compilation in Discourse.

The new version of sass-ruby moves to dart-sass putting we back on the supported version of Sass. It does so while keeping compatibility with the existing method signatures, so minimal change is needed in Discourse for this change.

This moves us

From:
  - sassc 2.0.1 (Feb 2019)
  - libsass 3.5.2 (May 2018)

To:
  - dart-sass 1.58

This update applies the following breaking changes:

> 
> These breaking changes are coming soon or have recently been released:
> 
>  [Functions are stricter about which units they allow](https://sass-lang.com/documentation/breaking-changes/function-units) beginning in Dart Sass 1.32.0.
> 
>  [Selectors with invalid combinators are invalid](https://sass-lang.com/documentation/breaking-changes/bogus-combinators) beginning in Dart Sass 1.54.0.
> 
>  [/ is changing from a division operation to a list separator](https://sass-lang.com/documentation/breaking-changes/slash-div) beginning in Dart Sass 1.33.0.
> 
>  [Parsing the special syntax of @-moz-document will be invalid](https://sass-lang.com/documentation/breaking-changes/moz-document) beginning in Dart Sass 1.7.2.
> 
>  [Compound selectors could not be extended](https://sass-lang.com/documentation/breaking-changes/extend-compound) in Dart Sass 1.0.0 and Ruby Sass 4.0.0.


SCSS files have been migrated automatically using `sass-migrator division app/assets/stylesheets/**/*.scss`
2023-02-07 12:24:57 -03:00
Gerhard Schlager e17c145e8d
FIX: Category hashtags weren't always found for sub-sub-categories (#20156)
The algorithm failed to find the correct category by slug when there are multiple sub-sub-categories with the same child-category name and the first child doesn't  have the correct grandchild.

So, searching for "child / grandchild" worked in the following case, it found (3):

- (1) parent 1
  - (2) child
    - (3) grandchild
- (4) parent 2
  - (5) child
    - (6) grandchild

But it failed to find the grandchild in the following case:

- (1) parent 1
  - (2) child
- (4) parent 2
  - (5) child
    - (6) grandchild

And this also fixes a flaky spec by forcing categories to always order by by `parent_category_id` and `id`.
This makes it possible to partly revert 60990aab55
2023-02-03 12:17:52 +01:00
Sam 651476e89e
FIX: domain searches not working properly for URLs (#20136)
If a post contains domain with a word that stems to a non prefix single
words will not match it.

For example: in happy.com, `happy` stems to `happi`. Thus searches for happy
will not find URLs with it included.

This bloats the index a tiny bit, but impact is limited.

Will require a full reindex of search to take effect. 

When we are done refining search we can consider a full version bump.
2023-02-03 09:55:28 +11:00
Selase Krakani 2e78045af1
FIX: Extend username updates to self-mentions (#20071)
Posts with self-mentions aren't updated with username updates. This happens
because mention `UserAction` entries aren't logged for self-mentions.

This change updates the lookup of `Post` and `PostRevision` with mentions to bypass
`UserAction` entries.
2023-02-02 12:33:42 +00:00
Sam 4570118a63
FIX: search index duplicate parser matching is too restrictive (#20129)
Previous regex did not allow for cases where a lexeme contains a : (colon)

This can happen when parsing URLs. New algorithm allows for this.
Test was amended to more clearly call out index problems
2023-02-02 12:17:19 +11:00
Jan Cernik 06817bd94f
FIX: Category permission change not creating a log (#20027)
It didn't create a log if the category was public { "everyone" => 1 }
2023-01-31 10:15:17 -03:00
Sam 07679888c8
FEATURE: allow restricting duplication in search index (#20062)
* FEATURE: allow restricting duplication in search index

This introduces the site setting `max_duplicate_search_index_terms`.
Using this number we limit the amount of duplication in our search index.

This allows us to more correctly weight title searches, so bloated posts
don't unfairly bump to the top of search results.

This feature is completely disabled by default and behind a site setting

We will experiment with it first. Note entire search index must be rebuilt
for it to take effect.


---------

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2023-01-31 12:41:31 +11:00
David Taylor 79bea9464c
PERF: Move user-tips and narrative to per-user messagebus channels (#19773)
Using a shared channel with per-message permissions means that every client is updated with the channel's 'last_id', even if there are no messages available to them. Per-user channel names avoid this problem - the last_id will only be incremented when there is a message for the given user.
2023-01-30 11:48:09 +00:00
Martin Brennan 63fdb6dd65
FIX: Do not add empty use/svg tags in ExcerptParser (#19969)
There was an issue where if hashtag-cooked HTML was sent
to the ExcerptParser without the keep_svg option, we would
end up with empty </use> and </svg> tags on the parts of the
excerpt where the hashtag was, in this case when a post
push notification was sent.

Fixed this, and also added a way to only display a plaintext
version of the hashtag for cases like this via PrettyText#excerpt.
2023-01-24 14:40:24 +10:00
Alan Guo Xiang Tan f122f24b35
SECURITY: Default tags to show count of topics in unrestricted categories (#19916)
Currently, `Tag#topic_count` is a count of all regular topics regardless of whether the topic is in a read restricted category or not. As a result, any users can technically poll a sensitive tag to determine if a new topic is created in a category which the user has not excess to. We classify this as a minor leak in sensitive information.

The following changes are introduced in this commit:

1. Introduce `Tag#public_topic_count` which only count topics which have been tagged with a given tag in public categories.
2. Rename `Tag#topic_count` to `Tag#staff_topic_count` which counts the same way as `Tag#topic_count`. In other words, it counts all topics tagged with a given tag regardless of the category the topic is in. The rename is also done so that we indicate that this column contains sensitive information. 
3. Change all previous spots which relied on `Topic#topic_count` to rely on `Tag.topic_column_count(guardian)` which will return the right "topic count" column to use based on the current scope. 
4. Introduce `SiteSetting.include_secure_categories_in_tag_counts` site setting to allow site administrators to always display the tag topics count using `Tag#staff_topic_count` instead.
2023-01-20 09:50:24 +08:00
Ted Johansson 421fbfd1c7
FIX: Fix flaky test resulting from PostAlerter keyword arguments (#19826)
We've been doing some work to support new keyword argument semantics in Ruby 3. As part of that we made some changes to `DiscourseEvent::TestHelper`. The backwards compatibility fix doesn't work if the method is called with an empty hash as the final argument. This fix adds a valid option to the final hash in the particular test.
2023-01-12 09:46:50 +08:00
David Taylor cb932d6ee1
DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
Alan Guo Xiang Tan 673089a6b4
FIX: Error condition in SidebarSiteSettingsBackfiller (#19787) 2023-01-09 07:01:58 +08:00
Gerhard Schlager 7e33cb3665
FIX: Add missing email template for `user_watching_category_or_tag` (#19653)
Adds a spec to hopefully prevent this in the future.

Follow-up to aa3a9b6fea
2022-12-29 15:36:53 +01:00
Rafael dos Santos Silva 7b53973bd8
DEV: Use WebPush fork for OpenSSL 3 compat (#19627)
* DEV: Use WebPush fork for OpenSSL 3 compat

* add some context on gemfile changes
2022-12-27 15:28:13 -03:00
Martin Brennan e15b382666
FIX: Include tags inside tag groups for hashtag search (#19539)
We were using the `for_input: true` param when calling
DiscourseTagging, which is really meant for selecting tags
in the UI, which often need a parent tag selected first
before the child tags in tag group will show. We just
want to show all tags regardless of grouping in hashtag
search.`
2022-12-21 15:14:50 +10:00
Martin Brennan 1b49a90e5b
FIX: Do not return muted child categories termless hashtag search (#19526)
We generally do not return muted child categories to the user
if they have muted the parent category, this commit respects that
rule for CategoryHashtagDataSource
2022-12-20 13:45:03 +10:00
Martin Brennan c31113c257
DEV: Remove flaky hashtag test (#19524)
Not sure why this is flaky, either way it doesn't really
add anything so removing it.
2022-12-20 08:21:58 +08:00
Selase Krakani 7ba115769a
DEV: Skip push notifications for active online users (#19502)
* DEV: Skip push notifications for active online users

Currently, users with active push subscriptions get push notifications
regardless of their "presence" on the site.

This change introduces a `push_notification_time_window_mins`
site setting which is used in conjunction with a user's `last_seen_at` to
determine if push notifications should be sent. A user is considered to
be actively online if their `last_seen_at` is within `push_notification_time_window_mins`
minutes. `push_notification_time_window_mins` is set to 10 by default.

* DEV: Remove client param for push_notification_time_window_mins site setting

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
2022-12-19 20:17:40 +00:00
chapoi 8db1f1892d
UX: Hashtag autocomplete styling (#19426)
* UX: added fadeout + hashtag styling

UX: add full name to autocomplete

UX: autocomplete mentions styling

UX: emoji styling user status

UX: autocomplete emoji

* DEV: Move hashtag tag counts into new secondary_text prop

* FIX: Add is-online style to mention users via chat

UX: make is-online avatar styling globally available

* DEV: Fix specs

* DEV: Test fix

Co-authored-by: Martin Brennan <martin@discourse.org>
2022-12-19 12:31:45 +01:00
Martin Brennan 6b9c0ee554
DEV: Change HashtagAutocompleteService to use DiscoursePluginRegistry (#19491)
Follow up to a review in #18937, this commit changes the HashtagAutocompleteService to no longer use class variables to register hashtag data sources or types in context priority order. This is to address multisite concerns, where one site could e.g. have chat disabled and another might not. The filtered plugin registers I added will not be included if the plugin is disabled.
2022-12-19 13:46:17 +10:00
Martin Brennan ec9ec1e04e
FEATURE: Sort hashtags starting with term higher priority (#19463)
This introduces another "section" of queries to the
hashtag autocomplete search, which returns results for
each type that start with the search term. So now results
will be in this order, and within these sections ordered
by the types in priority order:

1. Exact matches sorted by type
2. "starts with" sorted by type
3. Everything else sorted by type then name within type
2022-12-15 13:01:44 +10:00