In certain cases, like when `SiteSetting.slug_generation_method`
is set to `none` with certain locales, the autogenerated chat
channel slugs will end up blank. This was causing errors in
unrelated jobs calling `update!` on the channel. Instead, we
should just copy Category behaviour, which does not error
if the autogenerated slug ends up blank. We already allow
for this with chat channel URLs, using `-` in place of the
missing slug.
The problem here was that if your input has an Enter
listener (such as the chat message input) and the
`fill_in(with: str)` string has a `\n` at the end, this
is treated as an Enter keypress, so this `fill_in` was
submitting the chat message.
This commit fixes an issue where the chat message bookmarks
did not respect the user's `bookmark_auto_delete_preference`
which they select in their user preference page.
Also, it changes the default for that value to "keep bookmark and clear reminder"
rather than "never", which ends up leaving a lot of expired bookmark
reminders around which are a pain to clean up.
`last_message_sent_at` has a `NOT_NULL` constraint in the DB so it should be safe to use for sorting.
This was causing two flakeys:
```
1) UserNotifications.chat_summary with public channel email subject with regular mentions includes both channel titles when there are exactly two with unread mentions
Failure/Error: example.run
expected: "[Discourse] New message in Random 62 and Test channel"
got: "[Discourse] New message in Test channel and Random 62"
(compared using ==)
# ./plugins/chat/spec/mailers/user_notifications_spec.rb:203:in `block (6 levels) in <main>'
# ./spec/rails_helper.rb:356:in `block (2 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
2) UserNotifications.chat_summary with public channel email subject with regular mentions displays a count when there are more than two channels with unread mentions
Failure/Error: example.run
expected: "[Discourse] New message in Random 62 and 2 others"
got: "[Discourse] New message in Test channel 0 and 2 others"
(compared using ==)
# ./plugins/chat/spec/mailers/user_notifications_spec.rb:236:in `block (6 levels) in <main>'
# ./spec/rails_helper.rb:356:in `block (2 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
```
This commit is a series of fixes to improve stability of system tests following the use of threadsafe:
* Jobs.run_immediately in before block was causing issues
* During test a js error could be caused by an undefined this.details in chat-live-pane
* Apply the chat composer click trick everywhere when sending a message, it ensures we are not hiding anything with autocomplete
* There was another case not using send_message yet
This PR removes the limit added to max_users_notified_per_group_mention during #19034 and improve the performance when expanding mentions for large channel or groups by removing some N+1 queries and making the whole process async.
* Fully async chat message notifications
* Remove mention setting limit and get rid of N+1 queries
It should fix flakeys we have due to using_session. This commit is also fixing tests which were failing constantly with treadsafe enabled.
A test has also bene skipped as the issue couldn't be found so far.
More info: https://github.com/teamcapybara/capybara#threadsafe-mode
- improves UI by displaying channel status on it's own line
- ensures channel status is correctly updated right after the request on frontend
- adds status on info page
Recent changes surfaced the various issues with this codepath:
- we were not correctly reseting `messageLookup` leading to us trying to scroll to a non existing message in the view
- we were calling markAsRead which would scroll to the bottom, even if we had a target message
- we were not debouncing fetchMessages, which could cause multiple reload of the messages when loading it with a targetMessageId: first fetch from last read and then immediately fetch from targetMessageId
- other naming inconsistencies
- not handling drawer
This commit also adds tests for classic scenarios related to this use case.
I could repro the same failure by doing: `page.driver.browser.network_conditions = { offline: false, latency: 3000, throughput: 0 }`
Wait shouldn't be needed as we wait for selector, but I couldn't find a better solution on this case for now.
We are all in on system specs, so this commit moves all the chat quoting acceptance tests (some of which have been skipped for a while) into system specs.
The way our markdown raw_html hoisting worked, we only
supported one level of hoisting the HTML content. However
when nesting [chat] transcript BBCode we need to allow
for multiple levels of it. This commit changes opts.discourse.hoisted
to be more constant, and the GUID keys that have the hoisted
content are only deleted by unhoistForCooked rather than
the cook function itself, which prematurely deletes them
when they are needed further down the line.
Honestly seems like it's being in some weird loop for
discourse/hashtag_autocomplete_spec.rb for this:
```ruby
within topic_page.post_by_number(2) do
cooked_hashtags = page.all(".hashtag-cooked", count: 2)
expect(cooked_hashtags[0]["outerHTML"]).to eq(<<~HTML.chomp)
<a class=\"hashtag-cooked\" href=\"#{category.url}\" data-type=\"category\" data-slug=\"cool-cat\"><svg class=\"fa d-icon d-icon-folder svg-icon svg-node\"><use href=\"#folder\"></use></svg><span>Cool Category</span></a>
HTML
expect(cooked_hashtags[1]["outerHTML"]).to eq(<<~HTML.chomp)
<a class=\"hashtag-cooked\" href=\"#{tag.url}\" data-type=\"tag\" data-slug=\"cooltag\"><svg class=\"fa d-icon d-icon-tag svg-icon svg-node\"><use href=\"#tag\"></use></svg><span>cooltag</span></a>
HTML
end
```
I see this many times in the full logs with `SELENIUM_VERBOSE_DRIVER_LOGS=1`:
```
COMMAND FindElements {
"using": "css selector",
"value": "#post_2"
}
Followed by:
COMMAND FindChildElements {
"id": "26dfe542-659b-46cc-ac8c-a6c2d9cbdf0a",
"using": "css selector",
"value": ".hashtag-cooked"
}
```
Over and over and over, there are 58 such occurrences. I am beginning to
think `within` is just poison that should be avoided.
Previously, calling `sign_in` would cause the browser to be redirected to `/`, and would cause the Ember app to boot. We would then call `visit()`, causing the app to boot for a second time.
This commit adds a `redirect=false` option to the `/session/username/become` route. This avoids the unnecessary boot of the app, and leads to significantly faster system spec run times.
In local testing, this takes the full system-spec suite for chat from ~6min to ~4min.
1. `test()` and `render()` instead of `componentTest()`
2. Angle brackets
3. `strictEqual()`/`true()`/`false()` assertions
This removes all remaining uses of `componentTest` from core
Following the removal of user in current_user_membership we were now doing: `User.create(null)`.
I don't think it has any impact but this is just wasteful and could lead to issues if used.
The client already has all the information about the current user so
there is no need for us to be serializing the current `User` object each
time per channel that is preloaded.
In production, profiling shows that this unneeded serializing
adds a roughly 5% overhead to a request.
- chat-message is not using chat-api yet and the `jsonMode` shouldn't have been added
- correctly error on `getChannel` not found
- adds/correct relevant system tests
Note this is a very large PR, and some of it could have been splited, but keeping it one chunk made it to merge conflicts and to revert if necessary. Actual new code logic is also not that much, as most of the changes are removing js tests, adding system specs or moving things around.
To make it possible this commit is doing the following changes:
- converting (and adding new) existing js acceptances tests into system tests. This change was necessary to ensure as little regressions as possible while changing paradigm
- moving away from store. Using glimmer and tracked properties requires to have class objects everywhere and as a result works well with models. However store/adapters are suffering from many bugs and limitations. As a workaround the `chat-api` and `chat-channels-manager` are an answer to this problem by encapsulating backend calls and frontend storage logic; while still using js models.
- dropping `appEvents` as much as possible. Using tracked properties and a better local storage of channel models, allows to be much more reactive and doesn’t require arbitrary manual updates everywhere in the app.
- while working on replacing store, the existing work of a chat api (backend) has been continued to support more cases.
- removing code from the `chat` service to separate concerns, `chat-subscriptions-manager` and `chat-channels-manager`, being the largest examples of where the code has been rewritten/moved.
Future wok:
- improve behavior when closing/deleting a channel, it's already slightly buggy on live, it's rare enough that it's not a big issue, but should be improved
- improve page objects used in chat
- move more endpoints to the API
- finish temporarily skipped tests
- extract more code from the `chat` service
- use glimmer for `chat-messages`
- separate concerns in `chat-live-pane`
- eventually add js tests for `chat-api`, `chat-channels-manager` and `chat-subscriptions-manager`, they are indirectly heavy tested through system tests but it would be nice to at least test the public API
<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
Prior to this change, each request executed 2 Redis calls per chat channel
that was loaded. The number of Redis calls quickly adds up once a user
is following multiple channels.
This commit adds an index for the query which the chat plugin executes
multiple times when preloading user data in `Chat::ChatChannelFetcher.unread_counts`.
Sample query plan from a query I grabbed from one of our production
instance.
Before:
```
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GroupAggregate (cost=10.77..696.67 rows=7 width=16) (actual time=7.735..7.736 rows=0 loops=1)
Group Key: cc.id
-> Nested Loop (cost=10.77..696.54 rows=12 width=8) (actual time=7.734..7.735 rows=0 loops=1)
Join Filter: (cc.id = cm.chat_channel_id)
-> Nested Loop (cost=0.56..76.44 rows=1 width=16) (actual time=0.011..0.037 rows=7 loops=1)
-> Index Only Scan using chat_channels_pkey on chat_channels cc (cost=0.28..22.08 rows=7 width=8) (actual time=0.004..0.014 rows=7 loops=1)
Index Cond: (id = ANY ('{192,300,228,727,8,612,1633}'::bigint[]))
Heap Fetches: 0
-> Index Scan using user_chat_channel_unique_memberships on user_chat_channel_memberships uccm (cost=0.28..7.73 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=7)
Index Cond: ((user_id = 1338) AND (chat_channel_id = cc.id))
-> Bitmap Heap Scan on chat_messages cm (cost=10.21..618.98 rows=89 width=12) (actual time=1.096..1.097 rows=0 loops=7)
Recheck Cond: (chat_channel_id = uccm.chat_channel_id)
Filter: ((deleted_at IS NULL) AND (user_id <> 1338) AND (id > COALESCE(uccm.last_read_message_id, 0)))
Rows Removed by Filter: 2085
Heap Blocks: exact=7106
-> Bitmap Index Scan on index_chat_messages_on_chat_channel_id_and_created_at (cost=0.00..10.19 rows=270 width=0) (actual time=0.114..0.114 rows=2085 loops=7)
Index Cond: (chat_channel_id = uccm.chat_channel_id)
Planning Time: 0.408 ms
Execution Time: 7.762 ms
(19 rows)
```
After:
```
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GroupAggregate (cost=5.84..367.39 rows=7 width=16) (actual time=0.130..0.131 rows=0 loops=1)
Group Key: cc.id
-> Nested Loop (cost=5.84..367.26 rows=12 width=8) (actual time=0.129..0.130 rows=0 loops=1)
Join Filter: (cc.id = cm.chat_channel_id)
-> Nested Loop (cost=0.56..76.44 rows=1 width=16) (actual time=0.038..0.069 rows=7 loops=1)
-> Index Only Scan using chat_channels_pkey on chat_channels cc (cost=0.28..22.08 rows=7 width=8) (actual time=0.011..0.022 rows=7 loops=1)
Index Cond: (id = ANY ('{192,300,228,727,8,612,1633}'::bigint[]))
Heap Fetches: 0
-> Index Scan using user_chat_channel_unique_memberships on user_chat_channel_memberships uccm (cost=0.28..7.73 rows=1 width=8) (actual time=0.006..0.006 rows=1 loops=7)
Index Cond: ((user_id = 1338) AND (chat_channel_id = cc.id))
-> Bitmap Heap Scan on chat_messages cm (cost=5.28..289.71 rows=89 width=12) (actual time=0.008..0.008 rows=0 loops=7)
Recheck Cond: ((chat_channel_id = uccm.chat_channel_id) AND (id > COALESCE(uccm.last_read_message_id, 0)) AND (deleted_at IS NULL))
Filter: (user_id <> 1338)
-> Bitmap Index Scan on index_chat_messages_on_chat_channel_id_and_id (cost=0.00..5.26 rows=90 width=0) (actual time=0.008..0.008 rows=0 loops=7)
Index Cond: ((chat_channel_id = uccm.chat_channel_id) AND (id > COALESCE(uccm.last_read_message_id, 0)))
Planning Time: 1.217 ms
Execution Time: 0.188 ms
(17 rows)
```
* UX: added fadeout + hashtag styling
UX: add full name to autocomplete
UX: autocomplete mentions styling
UX: emoji styling user status
UX: autocomplete emoji
* DEV: Move hashtag tag counts into new secondary_text prop
* FIX: Add is-online style to mention users via chat
UX: make is-online avatar styling globally available
* DEV: Fix specs
* DEV: Test fix
Co-authored-by: Martin Brennan <martin@discourse.org>
Use `Chat::ChatChannelFetcher.secured_public_channel_search` directly
checking for existence instead of running through
`Chat::ChatChannelFetcher.secured_public_channels` which executes 7 more
DB queries.
Follow up to a review in #18937, this commit changes the HashtagAutocompleteService to no longer use class variables to register hashtag data sources or types in context priority order. This is to address multisite concerns, where one site could e.g. have chat disabled and another might not. The filtered plugin registers I added will not be included if the plugin is disabled.
In both ChatMessage#rebake! and in ChatMessageProcessor
when we were calling ChatMessage.cook we were missing the
user_id to cook with, which causes missed hashtag cooks
because of missing permissions.
Previously, restricted category chat channel was available for all groups - even `readonly`. From now on, only user who belong to group with `create_post` or `full` permissions can access that chat channel.
* DEV: Remove enable_whispers site setting
Whispers are enabled as long as there is at least one group allowed to
whisper, see whispers_allowed_groups site setting.
* DEV: Always enable whispers for admins if at least one group is allowed.
In #15474 we introduced dedicated support for date ranges. As part of that
change we added a fallback of "magic" date ranges, which treats dates in
any paragraph with exactly two dates as a range. There were discussions
about migrating all such paragraphs to use the new date range element, but
it was ultimately decided against.
This change removes the fallback and, as a bonus, adds support for multiple
date ranges in the same paragraph.
This introduces another "section" of queries to the
hashtag autocomplete search, which returns results for
each type that start with the search term. So now results
will be in this order, and within these sections ordered
by the types in priority order:
1. Exact matches sorted by type
2. "starts with" sorted by type
3. Everything else sorted by type then name within type
There is no need to duplicate check chat messages when they are being
edited but not having their message text changed. This was leading to
a validation error when adding/removing an upload but not changing the
message text.
FEATURE: Chat and Sidebar are now on by default
- Set the sidebar site setting to be enabled by default
- Set the chat site setting to be enabled by default
- Updated existing specs that assumed the original default
- Use a migration to keep old defaults for existing sites
Instead of passing `user` to `guardian.can_chat?`, we
can just use the inner `@user` that is part of the guardian
instance already to determine whether that user can chat,
since this is how it works for all other usages of guardian
even within chat.
1. "What Goes Up Must Come Down" – if you subscribe to message bus, make sure you also unsubscribe
2. When you unsubscribe - remove only your subscription, not **all** subscriptions on given channel
Attempt #2. The first attempt tried to extend a core `@bound` method in new-user-narrative plugin which did not work. I reworked that plugin in the meantime. This new PR also cleans up message bus subscriptions in now core-merged chat plugin.
We must set `treatAsTextarea` to true when using autocomplete
in the chat composer, since it is at the bottom of the screen
we always want to show it above the composer. This fixes the
issue where the hashtag autocomplete results went behind the
keyboard on mobile (which was not happening for mentions).
Previously with this experimental feature a user would be
able to search for public channels for public categories
using the new #hashtag system even if they couldn't chat.
This commit fixes the hole.
This commit allows us to type # in the UI and present autocomplete
results immediately with the following logic for the topic composer,
and reversed for the chat composer:
* Categories the user can access and has not muted sorted by `topic_count`
* Tags the user can access and has not muted sorted by `topic_count`
* Chat channels the user is a member of sorted by `messages_count`
So in effect, we allow searching for hashtags without a search term.
To do this we add a new `search_without_term` to each data source so
each one can define how it wants to handle this logic.
This new site setting replaces the
`enable_experimental_sidebar_hamburger` and `enable_sidebar` site
settings as the sidebar feature exits the experimental phase.
Note that we're replacing this without depreciation since the previous
site setting was considered experimental.
Internal Ref: /t/86563
* FEATURE: Enforce mention limits for chat messages
The first part of these changes adds a new setting called `max_mentions_per_chat_message`, which skips notifications when the message contains too many mentions. It also respects the `max_users_notified_per_group_mention` setting
and skips notifications if expanding a group mention would exceed it.
We also include a new component to display JIT warning for these limits to the user while composing a message.
* Simplify ignoring/muting filter in chat_notifier
* Post-send warnings for unsent warnings
* Improve pluralization
* Address review feedback
* Fix test
* Address second feedback round
* Third round of feedback
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
This commit adds the messages_count column for ChatChannel messages,
which is the number of not-deleted messages in the channel.
This is not updated every time a message is created or deleted in a
channel, so it should not be displayed in the UI.
It is updated eventually via Jobs::ChatPeriodicalUpdates, which
will have additional functions in future after being introduced
here.
Also update these counts for existing channels in a post migration.
User options were serialized at the root level of CurrentUserSerializer,
but UserSerializer has a user_option field. This inconsistency caused
issues in the past because user_option fields had to be duplicated on
the frontend.
The settings tab of each category channel should now present the option to allow or disallow channel wide mentions: @here and @all.
When disallowed, using these mentions in the channel should have no effect.
1. Originally the feature did "Scroll to new posts when user is near bottom of PM" (74e1889924)
2. Then that feature was limited to "Only scroll to posts that are not your own in PMs." (4a26561927)
3. It was limited further to "Only scroll PMs on new message" (eaf7746ec9)
4. And later to "only scroll to bottom for discobot" (267d129f38)
5. And the code was relegated to new-user-narrative plugin (48b7696dbc)
I don't think it's worth it to keep this scrolling code just for this very small specific case.
This did potentially confict with other post scrolling code, and also using `modifyClass` is something we'd like to avoid.
Martin Brennan
Kris
Joffrey JAFFEUX
Alan Guo Xiang Tan
Roman Rizzi
Discourse Translator Bot
David Taylor
Krzysztof Kotlarek
Jarek Radosz
Rafael dos Santos Silva
Bianca Nenciu
Daniel Waterworth
chapoi
dependabot[bot]
Ted Johansson
Jordan Vidrine
Jan Cernik
Andrei Prigorshnev
Blake Erickson
Gerhard Schlager