Commit Graph

368 Commits

Author SHA1 Message Date
Martin Brennan 9953a6edd9
DEV: Rearchitect chat tracking state (#21550)
This moves chat tracking state calculation for channels
and threads into a central Chat::TrackingStateManager service, that
serves a similar purpose to the TopicTrackingState model
in core.

This service calls down to these query classes:

* ThreadUnreadsQuery
* ChannelUnreadsQuery

To get the unread_count and mention_count for the appropriate
channels and threads.

As well as this, this commit refactors the client-side chat
tracking state.

Now, there is a central ChatTrackingStateManager Ember Service
so all tracking is accessible and can be counted from one place,
which can also initialize tracking from an initial payload.

The actual tracking counts are now maintained in a ChatTrackingState
class that is initialized on the `.tracking` property of both channel and
thread objects.

This removes the attributes on UserChatChannelMembership and decoration
of said membership from ChannelFetcher, preferring instead to have an additional
object for tracking in the JSON.
2023-05-16 14:51:13 +02:00
Andrei Prigorshnev f4fde4e49b
DEV: When deleting a chat message, do not delete mention records (#21486)
A chat message may be restored later, so we shouldn't be deleting `chat_mentions` records for it.

But we still have to remove notifications (see 082cd139).
2023-05-11 20:05:59 +04:00
Martin Brennan 26f9ccd8bb
FEATURE: Create and update thread memberships (#21501)
When the user sends a message in a thread, we want to
create a membership for them in the background (default
to notification level of Watching) so we can track whether
they have read the thread.

Then, for now since we don't have granular message reading/
scrolling in the thread panel, we just update the thread
last_read_message_id for the user to the latest reply in the
thread when they open the thread panel. This at least will
mark the thread as read.

In future PRs we want to show the blue dot indicator in various
places in the UI for unread threads which will also require
some MessageBus functionality.

This takes into account the same issue fixed for channels
in ae3231e140
2023-05-11 14:35:26 +02:00
Martin Brennan 616885895a
FIX: Chat NotificationLevels extension breaking in prod (#21484)
When setting DISCOURSE_ZEITWERK_EAGER_LOAD=1 to enable
eager loading the previous solution to adding chat_levels
to the core NotificationLevels would break with a module
loading error (c.f. cc2570fce3)

We don't actually _need_ to extend the core class, we can just
make our own for chat, let's do this instead.
2023-05-10 18:46:06 +02:00
Martin Brennan 79812db7d3
DEV: Fix double Chat:: module usage in chat models (#21483) 2023-05-10 17:51:46 +02:00
Martin Brennan cc2570fce3
DEV: Create UserChatThreadMembership table and model (#21481)
This will enable us to begin work on user tracking
state for a thread so we can show thread-specific
unreads and mentions indicators. In this case are following
the core notification_level paradigm rather than the solution
UserChatChannelMembership went with, and eventually we
will want to refactor the other table to match this as well.

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2023-05-10 17:19:48 +02:00
Martin Brennan 91c5658e9b
FIX: Handle deleted original message for thread index (#21470)
Since we have channel message retention which deletes
messages, we can end up with cases where the thread
is still around but the message is deleted. We will
handle the cascade delete in a different commit --
for now we will ensure the thread list lookup handles
this case and doesn't error.
2023-05-10 13:58:15 +02:00
Jan Cernik cbbaeb55b5
FIX: Don't autojoin users when they have ready-only permissions (#20213)
After this change, in order to join a chat channel, a user needs to be in a group with at least “Reply” permission for the category. If the user only has “See” permission, they are able to preview the channel, but not join it or send messages. The auto-join function also follows this new restriction.

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
2023-05-10 08:45:13 -03:00
Joffrey JAFFEUX c6b43ce68b
FEATURE: Thread list initial UI (#21412)
This commit adds an initial thread list UI. There are several limitations
with this that will be addressed in future PRs:

* There is no MessageBus reactivity, so e.g. if someone edits the original
   message of the thread it will not be reflected in the list. However if
   the thread title is updated the original message indicator will be updated.
* There is no unread functionality for threads in the list, if new messages
   come into the thread there is no indicator in the UI.
* There is no unread indicator on the actual button to open the thread list.
* No pagination.

In saying that, this is the functionality so far:

* We show a list of the 50 threads that the user has most recently participated
   in (i.e. sent a message) for the channel in descending order.
* Each thread we show a rich excerpt, the title, and the user who is the OM creator.
* The title is editable by staff and by the OM creator.
* Thread indicators show a title. We also replace emojis in the titles.
* Thread list works in the drawer/mobile.
2023-05-10 11:42:32 +02:00
Joffrey JAFFEUX 22521d3428
DEV: converts models to native classes (#21418)
- `ChatChannel`
- `UserChatChannelMembership`

Also creates a new `chat-direct-message` model used as the object for the`chatable` property of the `ChatChannel` when the `ChatChannel` is a direct message channel. When the chatable is a category a real `Category` object will now be returned.

Archive state of a `ChatChannel` is now hold in a `ChatChannelArchive` object.
2023-05-08 18:24:41 +02:00
Martin Brennan ae3231e140
FIX: Incorrect unread count shown in channel when message deleted (#21410)
When we were deleting messages in chat, we would find all of
the UserChatChannelMembership records that had a matching
last_read_message_id and set that column to NULL.

This became an issue when multiple users had that deleted message
set to their last_read_message_id. When we called ChannelUnreadsQuery
to get the unread count for each of the user's channels, we were
COALESCing the last_read_message_id and returning 0 if it was NULL,
which meant that the unread count for the channel would be the total
count of the messages not sent by the user in that channel.

This was particularly noticeable for DM channels since we show
the count with the indicator in the header. This issue would disappear
as soon as the user opened the problem channel, because we would then
set the last_read_message_id to an actual ID.

To circumvent this, instead of NULLifying the last_read_message_id in
most cases, it makes more sense to just set it to the most recent
non-deleted chat message ID for the channel. The only time it will
be set to NULL now is when there are no more other messages in the
channel.
2023-05-05 15:28:48 +02:00
Andrei Prigorshnev 35a414bb38
DEV: Create and update chat message mentions earlier (#21388)
We need to create and update `chat_mentions` records for messages earlier. They should be created or updated before we  call `Chat::Publisher.publish_new!` `Chat::Publisher.publish_edit!` to send the message to message bus subscribers).

This logic is covered with tests in `message_creator_spec.rb`, `message_updater_spec.rb`, `notifier_spec.rb` and `notify_mentioned_spec.rb`.

See the commits history for steps of refactoring.
2023-05-05 15:47:07 +04:00
Joffrey JAFFEUX 187b59d376
UX: implements draft threads (#21361)
This commit implements all the necessary logic to create thread seamlessly. For this it relies on the same logic used for messages and generates a `staged-id`(using the format: `staged-thread-CHANNEL_ID-MESSAGE_ID` which is used to re-conciliate state client sides once the thread has been persisted on the backend.

Part of this change the client side is now always using real thread and channel objects instead of sometimes relying on a flat `threadId` or `channelId`.

This PR also brings three UX changes:
- thread starts from top
- number of buttons on message actions is dependent of the width of the enclosing container
- <kbd>shift + ArrowUp</kbd> will reply to the last message
2023-05-05 08:55:55 +02:00
Martin Brennan b2a727336e
FIX: Thread mention read state and notification links (#21385)
* FIX: Link to thread for mentions inside thread

When mentioning a user in a thread, when we send the
notification and display it in the UI we want the URL
of the notification to point to the thread URL to open
the panel, rather than the main channel which is confusing.

For now, we don't have a way to highlight the linked-to message
in the thread, we can revisit this later.

* FIX: Mark mention notifications read when thread opens

Since we have no scrolling/message visibility/thread membership
for now, when a user opens the thread panel we just want to mark
all mention notifications relating to messages in the thread
for the user as read.
2023-05-04 17:28:51 +02:00
Martin Brennan 896707c7ea
FIX: Do not delete old chat messages if chat disabled (#21214)
Further followup to 24ec06ff85,
where I prevented other chat scheduled jobs from running if
chat was disabled. We should not be running any plugin scheduled
jobs if that plugin is disabled, it can cause unexpected
behaviour.
2023-04-24 14:42:50 +10:00
Martin Brennan 24ec06ff85
FEATURE: Reintroduce better thread reply counter cache (#21197)
This was reverted in 38cebd3ed5.
The issue was that I was using Discourse.redis.delete_prefixed
which does a slow redis KEYS lookup, which is not advised in
production. This commit removes that, and also ensures the periodical
thread count update only happens if threading is enabled.

I changed to use a redis INCR/DECR for reply count
cache. This avoids a round trip to redis to GET the current
count, and also avoids multi-process issues, where
if there's two processes trying to increment at the
same time, they may both receive the same value, add one
to it, then both write the same value back.
Then, it's only n+1 instead of n+2.

This also prevents almost all chat scheduled jobs from
running if chat is disabled, the only one remaining is
the message retention job.
2023-04-24 09:32:04 +10:00
Martin Brennan 21f93731a3
DEV: Move channel creation for category into service (#21167)
This commit moves the category channel creation out
of the Chat::Api::Channel controller and into a
dedicated CreateCategoryChannel service. A follow up
commit will move the DM channel creation out of
the old DirectMessageChannelCreator service.

Also includes a new on_model_errors helper
for chat service class usage, that collects model
validation errors to present in a nice way.

---------

Co-authored-by: Loïc Guitaut <loic@discourse.org>
2023-04-24 09:15:16 +10:00
Daniel Waterworth 38cebd3ed5
Revert "FEATURE: Better thread reply counter cache (#21108)" (#21192)
This reverts commit 180e3e11d1.

Per internal discussions, this is a temporary revert, to investigate if this is causing a performance regression.
2023-04-20 15:09:47 -05:00
Martin Brennan 351e3ccd98
FIX: Chat publisher publishing to thread when threading disabled (#21179)
Followup to bd5c5c4b5f, a
bug was introduced there for any channel that did not have
threading enabled or sites with the experimental threading
disabled. When the user replied to another chat message,
since this is always a thread in the background, we weren't
sending any MessageBus messages to the main channel, since
the message was a thread reply.

However in the UI these messages still show in the main stream
of the channel if threading is turned off, so the UI was not
reacting to these things happening in the backend. The worst
issue was that new clients would not see new replies sent in
reply to other messages in the channel.
2023-04-20 14:38:00 +10:00
Martin Brennan a8cf8e57b4
FIX: Do not count thread messages for channel unreads (#21126)
We currently don't have a nice UI to show unread messages for the thread,
and it will take some time to create one. For now, this commit makes it so
new messages inside a thread do not count towards a chat channel's unread
counts, and new messages sent in a thread do not update a user's `last_read_message_id`
for a channel.

In addition, this PR refactors the `Chat::ChannelFetcher` to use the `Chat::ChannelUnreadsQuery`
query class for consistency, and made said class able to return zeroed-out records
for channels the user is not a member of.

Finally, a small bug is fixed here where if a user's `last_read_message_id` for
a channel was a thread's OM ID, then the thread OM would not show in the
main channel stream for them until another reply to the channel was posted.
2023-04-19 08:53:51 +10:00
Martin Brennan 180e3e11d1
FEATURE: Better thread reply counter cache (#21108)
This commit introduces a redis cache over the top of the thread
replies_count DB cache, so that we can quickly and accurately
increment/decrement the reply count for all users and not have
to constantly update the database-level count. This is done so
the UI can have a count that is displayed to the users on each
thread indicator, that appears to live update on each chat
message create/trash/recover inside the thread.

This commit also introduces the `Chat::RestoreMessage` service
and moves the restore endpoint into the `Api::ChannelMessages`
controller as part of incremental migrations to move things out
of ChatController.

Finally, this commit refactors `Chat::Publisher` to be less repetitive
with its `MessageBus` sending code.
2023-04-18 14:01:01 +10:00
Martin Brennan 1e85de36e2
FEATURE: Hook up chat bulk delete for threads (#21109)
Followup to bd5c5c4b5f,
this commit hooks up the bulk delete events for chat
messages inside the thread panel, by fanning out the
deleted message IDs based on whether they belong to
a thread or not.

Also adds a system spec to cover this case, as previously
the bulk delete event would have been broken with an incorrect
`typ` rather than `type` hash key.
2023-04-18 08:28:20 +10:00
Loïc Guitaut a5235f7d16
DEV: Refactor STI/polymorphic associations in chat (#20789) 2023-04-17 15:41:56 +02:00
Martin Brennan 9a15eab3ab
FIX: Do not serialize thread data if threading disabled (#21107)
Followup to ba11cf4767,
this commit makes it so that none of the chat message
thread data is serialized if threading_enabled is false
for the channel or if enable_experimental_chat_threaded_discussions
is false, there is no need to serialize the data in this
case.
2023-04-17 14:29:06 +10:00
Joffrey JAFFEUX 274820d247
FIX: prevents exception when publishing processed (#21104)
This regression happened in bd5c5c4b5f and is due to `message_bus_targets = calculate_publish_targets(chat_channel, chat_message)` expecting a `chat_channel` which was only defined after.

Example exception in logs:

```
Job exception: undefined local variable or method `chat_channel' for Chat::Publisher:Module

/var/www/discourse/plugins/chat/app/services/chat/publisher.rb:91:in `publish_processed!' 
/var/www/discourse/plugins/chat/app/jobs/regular/chat/process_message.rb:21:in `block in execute' 
/var/www/discourse/lib/distributed_mutex.rb:53:in `block in synchronize' 
/var/www/discourse/lib/distributed_mutex.rb:49:in `synchronize' 
/var/www/discourse/lib/distributed_mutex.rb:49:in `synchronize' 
/var/www/discourse/lib/distributed_mutex.rb:34:in `synchronize' 
/var/www/discourse/plugins/chat/app/jobs/regular/chat/process_message.rb:7:in `execute' 
/var/www/discourse/app/jobs/base.rb:249:in `block (2 levels) in perform' 
```

This commit also:
- adds a spec to ensure oneboxing is not regressing anymore
- increment the version on message processed to ensure callbacks are correctly ran

Note we should also have more tests in `Chat::Publisher`, this will be done when we move it to a proper service.

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2023-04-16 10:30:33 +02:00
Joffrey JAFFEUX ba11cf4767
FIX: prevents exception if thread is not found (#21099)
It is yet to investigate the exact reasons leading to this, but probably due to some delete operation or migration, it seems possible to have a message with a thread_id leading to a non existing thread row. This is only a temporary solution to prevent the crash. We should also probably be more defensive here and not include any of this if threading is not enabled.
2023-04-14 23:03:09 +02:00
Martin Brennan bd5c5c4b5f
FEATURE: Reacting to MessageBus in chat thread panel (#21070)
This commit introduces a ChatChannelPaneSubscriptionsManager
and a ChatChannelThreadPaneSubscriptionsManager that inherits
from the first service that handle MessageBus subscriptions
for the main channel and the thread panel respectively.

This necessitated a change to Chat::Publisher to be able to
send MessageBus messages to multiple channels based on whether
a message was an OM for a thread, a thread reply, or a regular
channel message.

An initial change to update the thread indicator with new replies
has been done too, but that will be improved in future as we have
more data to update on the indicators.

Still remaining is to fully move over the handleSentMessage
functionality which includes scrolling and new message indicator
things.

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2023-04-13 14:45:50 +02:00
David Battersby 967010e545
FEATURE: Add an emoji deny list site setting (#20929)
This feature will allow sites to define which emoji are not allowed. Emoji in this list should be excluded from the set we show in the core emoji picker used in the composer for posts when emoji are enabled. And they should not be allowed to be chosen to be added to messages or as reactions in chat.

This feature prevents denied emoji from appearing in the following scenarios:
- topic title and page title
- private messages (topic title and body)
- inserting emojis into a chat
- reacting to chat messages
- using the emoji picker (composer, user status etc)
- using search within emoji picker

It also takes into account the various ways that emojis can be accessed, such as:
- emoji autocomplete suggestions
- emoji favourites (auto populates when adding to emoji deny list for example)
- emoji inline translations
- emoji skintones (ie. for certain hand gestures)
2023-04-13 15:38:54 +08:00
Martin Brennan 584a17c948
FEATURE: Initial chat thread indicator and disabling echo mode in channels (#21047)
This commit introduces a new thread indicator for channels with `threading_enabled`
set to true and the `enable_exp` site setting set to true. In addition, in the main channel
stream we now hide all messages that are linked to threads except for the original message,
disabling the concept of an "echo mode" for now, we may revisit this in future. We also
remove the jigsaw puzzle "Open Thread" button for message actions, since the thread
indicator can just be used instead.

This also stops the `Chat::Publisher` from sending any messages related to chat
messages that are linked to a thread, unless that chat message is the OM of the
thread. A subsequent PR will link up all MessageBus events within the thread panel,
and for the message indicators.

Another subsequent PR will add the excerpt of the latest message in each thread,
as well as the avatars of the users messaging in the thread.

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2023-04-12 11:09:06 +10:00
Martin Brennan e34fb7e0b2
DEV: Chat thread reply counter cache (#21050)
Similar to 22a55ef0ce,
this commit adds a replies_count to the Chat::Thread
table, which is updated every 15 minutes via PeriodicalUpdates.
This is done so the new thread indicator for the UI can
show the count without intense serializer queries, but
in future we likely want this to update more frequently.
2023-04-11 15:40:25 +10:00
Martin Brennan 894586afa9
FIX: Trashing message should reset last read (#20912)
When a chat message is trashed and the message is used
for someone's UserChatChannelMembership#last_read_message_id,
the user would end up with some read state issues until
someone posted a new message in the channel, since we didn't
clear it like we did on bulk message delete.

This commit fixes the issue, and also takes the opportunity
to start a MessagesController in the API namespace, and move
the trash message functionality into the new service format.
2023-04-04 09:30:38 +10:00
Martin Brennan c00d17535f
DEV: Drop chat_uploads table and model and remove old references (#20926)
Followup to 0924f874bd,
we migrated Chat::Upload records to UploadReference records
there and have not been making new Chat::Upload records
for some time, we can now delete the model and table.
2023-04-04 09:13:39 +10:00
Jarek Radosz 29e2e3ff3b
DEV: Fix random typos (#20937) 2023-04-03 19:27:32 +02:00
Martin Brennan f8ff97b2ec
DEV: Add specs to ChannelSerializer and StructuredChannelSerializer (#20872)
This adds specs to the mentioned serializers to catch regressions
with MessageBus last_ids and to ensure the correct ones are being
returned and passed down to the ChannelSerializer.

Followup to d8ad5c3
2023-03-30 12:16:32 +10:00
Martin Brennan d8ad5c38b9
FIX: channel_message_bus_last_id was incorrect (#20858)
Followup to 3ea8df4b06,

I forgot to wrap the call to Chat::Publisher.root_message_bus_channel(object.chat_channel.id)
in MessageBus.last_id, so the channel_message_bus_last_id key was
ending up as e.g. "/chat/58" instead of the last ID value.
2023-03-28 16:01:27 +10:00
Martin Brennan 3ea8df4b06
DEV: Fix excessive MessageBus#last_id calls in chat (#20855)
We noticed via profiling that chat was doing N redis calls
per channel. Part of this was from the kick_message_bus_last_id
from 520d4f504b being incorrectly
passed down for DM channels rather that public channels, and the
other part was from the root MessageBus channel last_id
being fetched in ChannelSerializer for every single channel.

This commit fixes both issues, for me going from 134 redis calls
on page load to 20 locally.

Also deletes an old file missed in 12a18d4d55
2023-03-28 14:45:45 +10:00
Martin Brennan c912c58d6c
FIX: Make chat tracking payload backwards compatible (#20852)
Followup cab4b2cfba,
this was causing client JS errors because the old version
of the client was expecting the old keys, but the new
ruby version of the app was sending different keys via
the MessageBus payload. We can remove this in a couple
of weeks.
2023-03-28 12:56:32 +10:00
Martin Brennan cab4b2cfba
DEV: Introduce bulk channel tracking publisher (#20838)
This commit introduces a Chat::Publisher and MessageBus endpoint
that allows for updating a user's channel tracking state in bulk for
multiple channels, rather than having to do it for one channel
at a time.

This also required an improvement to ChannelUnreadsQuery -- now
multiple channel IDs can be passed to this to get the unread counts
and mention counts for those channels for a user, also increasing
efficiency rather than having to do a query for every individual
channel.

Followup to #20802
2023-03-28 09:36:28 +10:00
Martin Brennan 55ef4d9a98
FIX: Make sure marking channels read propagates to all tabs (#20802)
Instead of just marking the state read in JS for each channel
after the AJAX call, we can instead just rely on the MessageBus
user-tracking-state chat channel, and publish the state to all
the channels affected in MarkAllUserChannelsRead. This will make
it so the blue dots for the channels are cleared across all tabs.
2023-03-24 10:12:35 +01:00
Martin Brennan a0381157e9
FEATURE: Mark all chat channels read with a shortcut (#20629)
This commit adds a keyboard shortcut (Shift+ESC) for chat which marks all
of the chat channels that the user is currently a following member of as read,
updating their `last_read_message_id`. This is done via a new service.

It also includes some refactors and controller changes:

* The old mark message read route from `ChatController` is now supplanted
  by the `Chat::Api::ReadsController#update` route.
* The new controller can handle either marking a single or all messages read,
  and uses the correct service based on the route and params.
* The `UpdateUserLastRead` service is now used (it wasn't before), and has been slightly
  updated to just use the guardian user ID.
2023-03-22 13:24:07 +10:00
Martin Brennan 520d4f504b
FEATURE: Auto-remove users without permission from channel (#20344)
There are many situations that may cause users to lose permission to
send messages in a chat channel. Until now we have relied on security
checks in `Chat::ChatChannelFetcher` to remove channels which the
user may have a `UserChatChannelMembership` record for but which
they do not have access to.

This commit takes a more proactive approach. Now any of these following
`DiscourseEvent` triggers may cause `UserChatChannelMembership`
records to be deleted:

* `category_updated` - Permissions of the category changed
   (i.e. CategoryGroup records changed)
* `user_removed_from_group` - Means the user may not be able to access the
   channel based on `GroupUser` or also `chat_allowed_groups`
* `site_setting_changed` - The `chat_allowed_groups` was updated, some
   users may no longer be in groups that can access chat.
* `group_destroyed` - Means the user may not be able to access the
   channel based on `GroupUser` or also `chat_allowed_groups`

All of these are handled in a distinct service run in a background
job. Users removed are logged via `StaffActionLog` and then we
publish messages on a per-channel basis to users who had their
memberships deleted.

When the user has a channel they are kicked from open, we show
a dialog saying "You no longer have access to this channel".

When they click OK we redirect them either:

* To their first other public channel, if they have any followed
* The chat browse page if they don't

This is to save on tons of requests from kicked out users getting messages
from other channels.

When the user does not have the kicked channel open, we can just
silently yoink it out of their sidebar and turn off subscriptions.
2023-03-22 10:19:59 +10:00
Martin Brennan 0a06974a8a
DEV: Rough chat message loading and posting in thread (#20579)
This commit does a couple of things:

* Adds the ability to load messages in the chat thread panel when it is open. This just loads the most recent N messages, same as a channel, and does nothing more, no scrolling or anything like that.
* Displays the messages in an extremely simple unordered list with no additional features.
* Allows posting new messages to the thread, and echoes them into the main channel, but does not respond to any sort of MessageBus events.

I've moved messages/clearMessages/addMessages/findMessage code out of the `ChatChannel` model
and into a new `ChatMessagesManager` class, which is instantiated in both the `ChatChannel` model
and the `ChatThread` model. This allows both to manage messages in the same way via the
`TrackedArray` pattern.

This is all hidden behind experimental flags, there is no way to make this not completely broken
in a single commit. Much more work and refactoring needs to be done first.

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2023-03-21 15:29:10 +10:00
Joffrey JAFFEUX 25d06faa8b
FIX: correctly checks for chat enabled in incoming webhooks controller (#20730)
This commit also moves the spec into the correct folder
2023-03-20 09:42:36 +01:00
Joffrey JAFFEUX 12a18d4d55
DEV: properly namespace chat (#20690)
This commit main goal was to comply with Zeitwerk and properly rely on autoloading. To achieve this, most resources have been namespaced under the `Chat` module.

- Given all models are now namespaced with `Chat::` and would change the stored types in DB when using polymorphism or STI (single table inheritance), this commit uses various Rails methods to ensure proper class is loaded and the stored name in DB is unchanged, eg: `Chat::Message` model will be stored as `"ChatMessage"`, and `"ChatMessage"` will correctly load `Chat::Message` model.
- Jobs are now using constants only, eg: `Jobs::Chat::Foo` and should only be enqueued this way

Notes:
- This commit also used this opportunity to limit the number of registered css files in plugin.rb
- `discourse_dev` support has been removed within this commit and will be reintroduced later

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2023-03-17 14:24:38 +01:00
Blake Erickson 6dcb099547 FIX: Escaped mentions in chat excerpts
Mentions are now displayed as using the non-cooked message which fixes
the problem. This is not ideal. I think we might want to rework how
these excerpts are created and rendered in the near future.

Co-authored-by: Jan Cernik <jancernik12@gmail.com>
2023-03-16 15:27:09 -06:00
Andrei Prigorshnev e292c45924
DEV: better split create_notification! and send_notifications logic (#20562)
`create_notification!` - creates a notification in the database, `send_notifications` sends desktop and mobile notifications. This PR moves some code to decouple these two tasks more explicitly. It only moves code without changing any behavior, and the job is covered with tests (see chat_notify_mentioned_spec).
2023-03-09 22:17:18 +04:00
Andrei Prigorshnev fa543cda06
DEV: Always create chat mention records (#20470)
Before this commit, we created a chat mention record only in case we wanted to send a notification about that mention to the user. Notifications were the only use case for the chat_mention db table. Now we want to use that table for other features, so we have to always create a chat_mention record.
2023-03-07 19:07:11 +04:00
Joffrey JAFFEUX b5e736504a
PERF: applies optimisations on chat-live pane (#20532)
- group writes when computing separators positions
- shows skeleton only on initial load
- forces date separator to be pinned when first message to prevent a pinned - not pinned - pinned sequence when loading more in past
- relies on `message.visible` property instead of checking `isElementInViewport`
- attempts to load next/prev messages earlier
- do not scroll to on fetch more
- hides `last visit` text while pinned
2023-03-06 16:42:11 +01:00
Joffrey JAFFEUX 1c9f300896
DEV: removes dead code (#20529) 2023-03-03 22:07:40 +01:00
Joffrey JAFFEUX 6b0aeced7e
DEV: rework the chat-live-pane (#20519)
This PR is introducing glimmer usage in the chat-live-pane, for components but also for models. RestModel usage has been dropped in favor of native classes.

Other changes/additions in this PR:

sticky dates, scrolling will now keep the date separator of the current section at the top of the screen
better unread management, marking a channel as unread will correctly mark the correct message and not mark the whole channel as read. Tracking state will also now correctly return unread count and unread mentions.
adds an animation on bottom arrow
better scrolling behavior, we should now always correctly keep the scroll position while loading more
reactions are now more reactive, and will update their tooltip without needed to close/reopen it
skeleton has been improved with placeholder images and reactions
when making a reaction on the desktop message actions, the menu won't move anymore
simplify logic and stop maintaining a list of unloaded messages
2023-03-03 13:09:25 +01:00
Joffrey JAFFEUX 435761ef58
Revert "DEV: rework the chat-live-pane (#20511)" (#20514)
This reverts commit 10e1831139.
2023-03-02 20:20:39 +01:00
Joffrey JAFFEUX 10e1831139
DEV: rework the chat-live-pane (#20511)
This PR is introducing glimmer usage in the chat-live-pane, for components but also for models. RestModel usage has been dropped in favor of native classes.

Other changes/additions in this PR:

- sticky dates, scrolling will now keep the date separator of the current section at the top of the screen
- better unread management, marking a channel as unread will correctly mark the correct message and not mark the whole channel as read. Tracking state will also now correctly return unread count and unread mentions.
- adds an animation on bottom arrow
- better scrolling behavior, we should now always correctly keep the scroll position while loading more
- reactions are now more reactive, and will update their tooltip without needed to close/reopen it
- skeleton has been improved with placeholder images and reactions
- when making a reaction on the desktop message actions, the menu won't move anymore
- simplify logic and stop maintaining a list of unloaded messages
2023-03-02 19:46:03 +01:00
Joffrey JAFFEUX 31480bde92
Revert "DEV: rework the chat-live-pane" (#20510)
This reverts commit 67c0498f64.
2023-03-02 16:59:11 +01:00
Joffrey JAFFEUX 67c0498f64
DEV: rework the chat-live-pane
This PR is introducing glimmer usage in the chat-live-pane, for components but also for models. RestModel usage has been dropped in favor of native classes.

Other changes/additions in this PR:

- sticky dates, scrolling will now keep the date separator of the current section at the top of the screen
- better unread management, marking a channel as unread will correctly mark the correct message and not mark the whole channel as read. Tracking state will also now correctly return unread count and unread mentions.
- adds an animation on bottom arrow
- better scrolling behavior, we should now always correctly keep the scroll position while loading more
- reactions are now more reactive, and will update their tooltip without needed to close/reopen it
- skeleton has been improved with placeholder images and reactions
- when making a reaction on the desktop message actions, the menu won't move anymore
- simplify logic and stop maintaining a list of unloaded messages
2023-03-02 16:34:25 +01:00
Loïc Guitaut bc4e016b49 DEV: Make the `model` step compatible with collections 2023-02-27 14:24:43 +01:00
Loïc Guitaut b8762172e4 DEV: Allow `with_service` in jobs
This patch introduces a new `ServiceJob` class allowing the use of
`with_service` in jobs.

This way, it’s easier to use the chat service objects in jobs and
provides the same level of functionality than the one we have in
controllers.
2023-02-23 09:28:53 +01:00
Loïc Guitaut f7c57fbc19 DEV: Enable `unless` cops
We discussed the use of `unless` internally and decided to enforce
available rules from rubocop to restrict its most problematic uses.
2023-02-21 10:30:48 +01:00
Gerhard Schlager 7ef482a292
REFACTOR: Fix pluralized strings in chat plugin (#20357)
* FIX: Use pluralized string

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Fix misuse of pluralized string

* DEV: Remove linting of `one` key in MessageFormat string, it doesn't work

* REFACTOR: Fix misuse of pluralized string

This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff. The string is quite complicated, so the best option was to switch to MessageFormat.

* REFACTOR: Fix misuse of pluralized string

* FIX: Use pluralized string

This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff.

* REFACTOR: Correctly pluralize reaction tooltips in chat

This also ensures that maximum 5 usernames are shown and fixes the number of "others" which was off by 1 if the current user reacted on a message.

* REFACTOR: Use translatable string as comma separator

* DEV: Add comment to translation to clarify the meaning of `%{identifier}`

* REFACTOR: Use translatable comma separator and use explicit interpolation keys

* REFACTOR: Don't interpolate lowercase channel status

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Don't interpolate channel status

* REFACTOR: Use %{count} interpolation key

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Correctly pluralize DM chat channel titles
2023-02-20 10:31:02 +01:00
Joffrey JAFFEUX 39be3cb619
DEV: fixes spec failing with ruby 3.2.1 (#20356)
The error was:

```
Failures:

  1) Chat::Endpoint.call(service, &block) when using the on_failed_contract action when the service contract does not fail does not run the provided block
     Failure/Error: subject(:endpoint) { described_class.call(service, controller, &actions_block) }

     NoMethodError:
       private method `run' called for #<SuccessContractService:0x000000011e3b28a0 @initial_context={"guardian"=>nil}, @context=#<Chat::Service::Base::Context guardian=nil, __steps__=[#<Chat::Service::Base::ContractStep:0x000000011de51230 @name=:default, @method_name=:default, @class_name=SuccessContractService::Contract, @default_values_from=nil>]>>
     # ./plugins/chat/app/services/base.rb:305:in `call'
     # ./plugins/chat/app/helpers/with_service_helper.rb:20:in `run_service'
     # ./plugins/chat/lib/endpoint.rb:76:in `call'
     # ./plugins/chat/lib/endpoint.rb:70:in `call'
     # ./plugins/chat/spec/lib/endpoint_spec.rb:80:in `block (3 levels) in <main>'
     # ./plugins/chat/spec/lib/endpoint_spec.rb:198:in `block (5 levels) in <main>'
     # ./spec/rails_helper.rb:358:in `block (2 levels) in <top (required)>'
```
2023-02-18 12:13:41 +01:00
Joffrey JAFFEUX 8144730ebb
FIX: correctly add user info data to message serializer (#20348)
Previous commit 479c0a3051 was done with the assumption that this info was defined on user serializer but it was actually defined on post serializer in core. This commit extends the user serializer for messages to add this data to the user.

Also correctly adds serializer test to ensure we actually have this data.
2023-02-17 17:07:44 +01:00
Martin Brennan 584d9a9438
DEV: Use service objects for chat thread lookup (#20276)
Adds a new LookupThread class that handles finding the
thread based on thread + channel ID, checking permissions
and policy/contract checks.

Co-authored-by: Loïc Guitaut <loic@discourse.org>
2023-02-16 19:17:26 +01:00
Martin Brennan c07f1e442f
DEV: Fix failing chat spec and add unexpected failure indicator (#20299)
This commit fixes the UpdateUserLastRead spec which was checking
for a message ID that did not exist -- this could fail at times
since message ID 2 could exist. Better to create + destroy a message
since then it's guaranteed we have a unique ID.

This also attempts to clarify a step that we expect to fail which
succeeds instead by adding another emoji next to the success tick and
an explanation text.

Also removes some uses of unless in Services::Base, we generally prefer
to use alternatives, since unless can be hard to parse in a lot of
cases.

Co-authored-by: Loïc Guitaut <loic@discourse.org>
2023-02-15 19:16:13 +01:00
Martin Brennan c65cdc0779
DEV: Remove chat_channel_id from chat-live-pane details (#20302)
This is unnecessary indirection, we can just have the
chat_channel_id in the message serializer and use that.
2023-02-15 08:27:09 +01:00
Martin Brennan 07ab20131a
FEATURE: Chat side panel with threads initial skeleton (#20209)
This commit introduces the skeleton of the chat thread UI. The
structure of the components looks like this. Its done this way
so the side panel can be used for other things as well if we wish,
not just for threads:

```
.main-chat-outlet
   <ChatLivePane />
   <ChatSidePanel>
     <-- rendered with {{outlet}} -->
     <ChatThread />
   </ChatSidePanel>
```

Later on the `ChatThreadList` will be rendered here as well.
Now, when you go to a channel you can open a thread by clicking
on either the Open Thread message action button or by clicking on
the reply indicator. This will take you to a route like `chat/c/:slug/:channelId/t/:threadId`.
This works on mobile as well.

This commit includes basic serializers and routes for threads,
as well as a new `ChatThreadsManager` service in JS that caches
threads for a channel the same way the channel threads manager does.

The chat messages inside the thread are intentionally left out
until a later PR.

**NOTE: These changes are gated behind the site setting enable_experimental_chat_threaded_discussions
and the threading_enabled boolean on a ChatChannel**
2023-02-14 11:38:41 +10:00
Loïc Guitaut 5f4623ba47 DEV: Refactor `UpdateUserLastRead` a little
We’re now using `contract` as the first step and validations for
mandatory parameters have been added.

To simplify specs a bit, we only assert the service contract is run as
expected without testing each validation case. We’re now testing the
contract itself in isolation.
2023-02-13 17:03:41 +01:00
Martin Brennan 60ad836313
DEV: Chat service object initial implementation (#19814)
This is a combined work of Martin Brennan, Loïc Guitaut, and Joffrey Jaffeux.

---

This commit implements a base service object when working in chat. The documentation is available at https://discourse.github.io/discourse/chat/backend/Chat/Service.html

Generating documentation has been made as part of this commit with a bigger goal in mind of generally making it easier to dive into the chat project.

Working with services generally involves 3 parts:

- The service object itself, which is a series of steps where few of them are specialized (model, transaction, policy)

```ruby
class UpdateAge
  include Chat::Service::Base

  model :user, :fetch_user
  policy :can_see_user
  contract
  step :update_age

  class Contract
    attribute :age, :integer
  end

  def fetch_user(user_id:, **)
    User.find_by(id: user_id)
  end

  def can_see_user(guardian:, **)
    guardian.can_see_user(user)
  end

  def update_age(age:, **)
    user.update!(age: age)
  end
end
```

- The `with_service` controller helper, handling success and failure of the service within a service and making easy to return proper response to it from the controller

```ruby
def update
  with_service(UpdateAge) do
    on_success { render_serialized(result.user, BasicUserSerializer, root: "user") }
  end
end
```

- Rspec matchers and steps inspector, improving the dev experience while creating specs for a service

```ruby
RSpec.describe(UpdateAge) do
  subject(:result) do
    described_class.call(guardian: guardian, user_id: user.id, age: age)
  end

  fab!(:user) { Fabricate(:user) }
  fab!(:current_user) { Fabricate(:admin) }

  let(:guardian) { Guardian.new(current_user) }
  let(:age) { 1 }

   it { expect(user.reload.age).to eq(age) }
end
```

Note in case of unexpected failure in your spec, the output will give all the relevant information:

```
  1) UpdateAge when no channel_id is given is expected to fail to find a model named 'user'
     Failure/Error: it { is_expected.to fail_to_find_a_model(:user) }

       Expected model 'foo' (key: 'result.model.user') was not found in the result object.

       [1/4] [model] 'user' 
       [2/4] [policy] 'can_see_user'
       [3/4] [contract] 'default'
       [4/4] [step] 'update_age'

       /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/update_age.rb:32:in `fetch_user': missing keyword: :user_id (ArgumentError)
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `instance_exec'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:219:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `block in run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `each'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `run!'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:411:in `run'
       	from <internal:kernel>:90:in `tap'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:302:in `call'
       	from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/spec/services/update_age_spec.rb:15:in `block (3 levels) in <main>'
```
2023-02-13 13:09:57 +01:00
Andrei Prigorshnev 0dcfd7ddec
DEV: correct a relationship – a chat message may have several mentions (#20219)
This change only makes the model reflect correctly what's 
already happening in the database. Note that there are no calls 
to chat_message.chat_mention in Core and plugins so this 
change should be safe.

Also note, that at the moment we use the chat_mentions db 
table only to support notifications about mentions, but 
we're going to start using it for other cases. This commit is 
the first step in that direction.
2023-02-09 17:44:04 +04:00
Dan Ungureanu 25f2fb61b8
DEV: Add 'chat_message_trashed' Discourse event (#20170)
Triggers a DiscourseEvent when a message is deleted, similar to
`:chat_message_created` and `:chat_message_edited`. This is not used
in this plugin, but can be used by other plugins to act when a message
is trashed.
2023-02-06 21:52:43 +01:00
Roman Rizzi 082cd13909
FIX: Delete associated notifications when trashing chat messages. (#20144)
Deleting a message with a mention doesn't clear the associated notification, confusing the mentioned user.

There are different chat notification types, but we only care about `chat_mentioned` since `chat_quoted` is associated with a post, and `chat_message` is only for push notifications.

Unfortunately, this change doesn't fix the chat bubble getting out of sync when a message gets deleted since we track unread/mentions count with an integer, making it a bit hard to manipulate. We can follow up later if we consider it necessary.
2023-02-03 12:52:13 -03:00
Roman Rizzi 5c699e4384
DEV: Pass messageId as a dynamic segment instead of a query param (#20013)
* DEV: Rnemae channel path to just c

Also swap the channel id and channel slug params to be consistent with core.

* linting

* channel_path

* Drop slugify helper and channel route without slug

* Request slug and route models through the channel model if possible

* DEV: Pass messageId as a dynamic segment instead of a query param

* Ensure change is backwards-compatible

* drop query param from oneboxes

* Correctly extract channelId from routes

* Better route organization using siblings for regular and near-message

* Ensures sessions are unique even when using parallelism

* prevents didReceiveAttrs to clear input mid test

* we disable animations in capybara so sometimes the message was barely showing

* adds wait

* ensures finished loading

* is it causing more harm than good?

* this check is slowing things for no reason

* actually target the button

* more resilient select chat message

* apply similar fix to bookmark

* fix

---------

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2023-02-01 12:39:23 -03:00
Martin Brennan e7b39e2fc1
DEV: Add ChatThread model and DB table, and ChatMessage reference (#20106)
This new table will be used to automatically group replies
for messages into one place. In future additional functionality
will be built around the thread, like pinning messages, changing
the title, etc., the columns are just the main ones needed at first.
The columns are not prefixed with `chat_*` e.g. `chat_channel` since
this is redundant and just adds duplication everywhere, we want to
move away from this generally within chat.
2023-02-01 13:50:38 +10:00
Martin Brennan 8c4c00b211
DEV: Start threading for chat (#20060)
Adds hidden `enable_experimental_chat_threaded_discussions`
setting which will control whether threads show in the UI,
alongside the `ChatChannel.threading_enabled` boolean column,
which does the same. The former is a global switch for this
feature, while the latter can be used to allow single channels
to show this new functionality if the site setting is true.

Neither setting impacts whether `ChatThread` records (which will
be added in a future PR) will be created, they will always be
made regardless.
2023-01-31 10:01:47 +10:00
Martin Brennan db5ad34508
FEATURE: Allow editing channel slug (#19948)
This commit introduces the ability to edit the channel
slug from the About tab for the chat channel when the user
is admin. Similar to the create channel modal functionality
introduced in 641e94f, if
the slug is left empty then we autogenerate a slug based
on the channel name, and if the user just changes the slug
manually we use that instead.

We do not do any link remapping or anything else of the
sort, when the category slug is changed that does not happen
either.
2023-01-30 13:18:34 +10:00
Roman Rizzi d07b472b79
DEV: `/channel` -> `/c` chat route rename (#19782)
* DEV: Rnemae channel path to just c

Also swap the channel id and channel slug params to be consistent with core.

* linting

* channel_path

* params in wrong order

* Drop slugify helper and channel route without slug

* Request slug and route models through the channel model if possible

* Add client side redirection for backwards-compatibility

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2023-01-27 09:58:12 -03:00
Natalie Tay 5eaf080239
SECURITY: Limit chat drafts length and preloaded count (#19987)
Only allow maximum of `50_000` characters for chat drafts. A hidden `max_chat_draft_length` setting can control this limit. A migration is also provided to delete any abusive draft in the database.

The number of drafts loaded on current user has also been limited and ordered by most recent update.

Note that spec files moved are not directly related to the fix.

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
Co-authored-by: Régis Hanol <regis@hanol.fr>
2023-01-25 13:50:10 +02:00
Martin Brennan 0924f874bd
DEV: Use UploadReference instead of ChatUpload in chat (#19947)
We've had the UploadReference table for some time now in core,
but it was added after ChatUpload was and chat was just never
moved over to this new system.

This commit changes all chat code dealing with uploads to create/
update/delete/query UploadReference records instead of ChatUpload
records for consistency. At a later date we will drop the ChatUpload
table, but for now keeping it for data backup.

The migration + post migration are the same, we need both in case
any chat uploads are added/removed during deploy.
2023-01-24 13:28:21 +10:00
Joffrey JAFFEUX 34d158c4aa
FIX: generates automatic slug for trashed channels (#19908)
Prior to this fix trashed channels would still prevent a channel with the same slug to be created. This commit generates a new slug on trash and frees the slug for future usage.

The format used for the slug is: `YYYYMMDD-HHMM-OLD_SLUG-deleted` truncated to the max length of a channel name.
2023-01-23 15:05:47 +01:00
Martin Brennan 641e94fc3c
FEATURE: Allow changing slug on create channel (#19928)
This commit allows us to set the channel slug when creating new chat
channels. As well as this, it introduces a new `SlugsController` which can
generate a slug using `Slug.for` and a name string for input. We call this
after the user finishes typing the channel name (debounced) and fill in
the autogenerated slug in the background, and update the slug input
placeholder.

This autogenerated slug is used by default, but if the user writes anything
else in the input it will be used instead.
2023-01-23 14:48:33 +10:00
Martin Brennan 98d5a0e63c
DEV: Remove old TODO for chat webhooks (#19839)
I inspected the JSON payload from OpsGenie and added a
note to show what it can look like.
2023-01-12 11:45:53 +10:00
Martin Brennan 387693e889
FIX: Improve error reporting and failure modes for channel archiving (#19791)
There was an issue with channel archiving, where at times the topic
creation could fail which left the archive in a bad state, as read-only
instead of archived. This commit does several things:

* Changes the ChatChannelArchiveService to validate the topic being
  created first and if it is not valid report the topic creation errors
  in the PM we send to the user
* Changes the UI message in the channel with the archive status to reflect
  that topic creation failed
* Validate the new topic when starting the archive process from the UI,
  and show the validation errors to the user straight away instead of
  creating the archive record and starting the process

This also fixes another issue in the discourse_dev config which was
failing because YAML parsing does not enable all classes by default now,
which was making the seeding rake task for chat fail.
2023-01-12 10:04:46 +10:00
Martin Brennan 5f4911dae8
FIX: Channel archive N1 when serializing current user (#19820)
* FIX: Channel archive N1 when serializing current user

The `ChatChannelSerializer` serializes the archive for the
channel if it is present, however this was causing an N1 for
the current user serializer in the case of DM channels, which
were not doing `includes(:chat_channel_archive)` in the
`ChatChannelFetcher`.

DM channels cannot be archived, so we can just never try to serialize
the archive for DM channels in `ChatChannelSerializer`, which
removes the N1.

* DEV: Add N1 performance spec for latest.html preloading

We modify current user serializer in chat, so it's a good
idea to have some N1 performance specs to avoid regressions
here.
2023-01-11 14:39:56 +10:00
David Taylor 055310cea4
DEV: Apply syntax_tree formatting to `plugins/*` 2023-01-07 11:11:37 +00:00
Joffrey JAFFEUX d464f1fc62
FIX: render_404 is not defined (#19769)
Note this endpoint is soon going to be replaced.
2023-01-06 14:38:08 +01:00
Martin Brennan 07cc8c8634
FIX: Do not error if autogenerated channel slug is blank (#19759)
In certain cases, like when `SiteSetting.slug_generation_method`
is set to `none` with certain locales, the autogenerated chat
channel slugs will end up blank. This was causing errors in
unrelated jobs calling `update!` on the channel. Instead, we
should just copy Category behaviour, which does not error
if the autogenerated slug ends up blank. We already allow
for this with chat channel URLs, using `-` in place of the
missing slug.
2023-01-06 09:04:52 +10:00
Joffrey JAFFEUX d8a19b2c9a
FIX: correctly uses request_uri to generate load_more_url (#19680)
Before this change we were generating an incorrect absolute URL: `https://chat/api/channels...`
2023-01-03 00:19:13 +01:00
Roman Rizzi 2f61d26e3d
PERF: Make chat mention notifications async. (#19666)
This PR removes the limit added to max_users_notified_per_group_mention during #19034 and improve the performance when expanding mentions for large channel or groups by removing some N+1 queries and making the whole process async.

* Fully async chat message notifications

* Remove mention setting limit and get rid of N+1 queries
2023-01-02 11:54:52 -03:00
Joffrey JAFFEUX b97fff444e
FIX: ensures loading more doesn’t erase filter on browse (#19675) 2023-01-02 14:40:13 +01:00
Joffrey JAFFEUX 908add79de
DEV: moves channel-archive-status to channels subscriptions (#19567)
It was quite an oddball because its a global subscription created on each channel. channels manager now allows us to elegantly solve this case.
2022-12-30 15:30:36 +01:00
Alan Guo Xiang Tan 6bcf558bae
PERF: Stop serializing user in `ChatChannelSerializer#current_user_membership` (#19527)
The client already has all the information about the current user so
there is no need for us to be serializing the current `User` object each
time per channel that is preloaded.

In production, profiling shows that this unneeded serializing
adds a roughly 5% overhead to a request.
2022-12-22 05:30:06 +08:00
Joffrey JAFFEUX d2e24f9569
DEV: start glimmer-ification and optimisations of chat plugin (#19531)
Note this is a very large PR, and some of it could have been splited, but keeping it one chunk made it to merge conflicts and to revert if necessary. Actual new code logic is also not that much, as most of the changes are removing js tests, adding system specs or moving things around.

To make it possible this commit is doing the following changes:

- converting (and adding new) existing js acceptances tests into system tests. This change was necessary to ensure as little regressions as possible while changing paradigm
- moving away from store. Using glimmer and tracked properties requires to have class objects everywhere and as a result works well with models. However store/adapters are suffering from many bugs and limitations. As a workaround the `chat-api` and `chat-channels-manager` are an answer to this problem by encapsulating backend calls and frontend storage logic; while still using js models.
- dropping `appEvents` as much as possible. Using tracked properties and a better local storage of channel models, allows to be much more reactive and doesn’t require arbitrary manual updates everywhere in the app.
- while working on replacing store, the existing work of a chat api (backend) has been continued to support more cases.
- removing code from the `chat` service to separate concerns, `chat-subscriptions-manager` and `chat-channels-manager`, being the largest examples of where the code has been rewritten/moved.

Future wok:
- improve behavior when closing/deleting a channel, it's already slightly buggy on live, it's rare enough that it's not a big issue, but should be improved
- improve page objects used in chat
- move more endpoints to the API
- finish temporarily skipped tests
- extract more code from the `chat` service
- use glimmer for `chat-messages`
- separate concerns in `chat-live-pane`
- eventually add js tests for `chat-api`, `chat-channels-manager` and `chat-subscriptions-manager`, they are indirectly heavy tested through system tests but it would be nice to at least test the public API

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2022-12-21 13:21:02 +01:00
Alan Guo Xiang Tan 14d54872f0
PERF: Use `MessageBus.last_ids` instead of `MessageBus.last_id` for chat (#19523)
Prior to this change, each request executed 2 Redis calls per chat channel
that was loaded. The number of Redis calls quickly adds up once a user
is following multiple channels.
2022-12-20 08:20:45 +08:00
Alan Guo Xiang Tan 80d289eb8b
PERF: Add index for chat unread counts query (#19516)
This commit adds an index for the query which the chat plugin executes
multiple times when preloading user data in `Chat::ChatChannelFetcher.unread_counts`.

Sample query plan from a query I grabbed from one of our production
instance.

Before:

```
                                                                                        QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 GroupAggregate  (cost=10.77..696.67 rows=7 width=16) (actual time=7.735..7.736 rows=0 loops=1)
   Group Key: cc.id
   ->  Nested Loop  (cost=10.77..696.54 rows=12 width=8) (actual time=7.734..7.735 rows=0 loops=1)
         Join Filter: (cc.id = cm.chat_channel_id)
         ->  Nested Loop  (cost=0.56..76.44 rows=1 width=16) (actual time=0.011..0.037 rows=7 loops=1)
               ->  Index Only Scan using chat_channels_pkey on chat_channels cc  (cost=0.28..22.08 rows=7 width=8) (actual time=0.004..0.014 rows=7 loops=1)
                     Index Cond: (id = ANY ('{192,300,228,727,8,612,1633}'::bigint[]))
                     Heap Fetches: 0
               ->  Index Scan using user_chat_channel_unique_memberships on user_chat_channel_memberships uccm  (cost=0.28..7.73 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=7)
                     Index Cond: ((user_id = 1338) AND (chat_channel_id = cc.id))
         ->  Bitmap Heap Scan on chat_messages cm  (cost=10.21..618.98 rows=89 width=12) (actual time=1.096..1.097 rows=0 loops=7)
               Recheck Cond: (chat_channel_id = uccm.chat_channel_id)
               Filter: ((deleted_at IS NULL) AND (user_id <> 1338) AND (id > COALESCE(uccm.last_read_message_id, 0)))
               Rows Removed by Filter: 2085
               Heap Blocks: exact=7106
               ->  Bitmap Index Scan on index_chat_messages_on_chat_channel_id_and_created_at  (cost=0.00..10.19 rows=270 width=0) (actual time=0.114..0.114 rows=2085 loops=7)
                     Index Cond: (chat_channel_id = uccm.chat_channel_id)
 Planning Time: 0.408 ms
 Execution Time: 7.762 ms
(19 rows)
```

After:

```
                                                                                        QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 GroupAggregate  (cost=5.84..367.39 rows=7 width=16) (actual time=0.130..0.131 rows=0 loops=1)
   Group Key: cc.id
   ->  Nested Loop  (cost=5.84..367.26 rows=12 width=8) (actual time=0.129..0.130 rows=0 loops=1)
         Join Filter: (cc.id = cm.chat_channel_id)
         ->  Nested Loop  (cost=0.56..76.44 rows=1 width=16) (actual time=0.038..0.069 rows=7 loops=1)
               ->  Index Only Scan using chat_channels_pkey on chat_channels cc  (cost=0.28..22.08 rows=7 width=8) (actual time=0.011..0.022 rows=7 loops=1)
                     Index Cond: (id = ANY ('{192,300,228,727,8,612,1633}'::bigint[]))
                     Heap Fetches: 0
               ->  Index Scan using user_chat_channel_unique_memberships on user_chat_channel_memberships uccm  (cost=0.28..7.73 rows=1 width=8) (actual time=0.006..0.006 rows=1 loops=7)
                     Index Cond: ((user_id = 1338) AND (chat_channel_id = cc.id))
         ->  Bitmap Heap Scan on chat_messages cm  (cost=5.28..289.71 rows=89 width=12) (actual time=0.008..0.008 rows=0 loops=7)
               Recheck Cond: ((chat_channel_id = uccm.chat_channel_id) AND (id > COALESCE(uccm.last_read_message_id, 0)) AND (deleted_at IS NULL))
               Filter: (user_id <> 1338)
               ->  Bitmap Index Scan on index_chat_messages_on_chat_channel_id_and_id  (cost=0.00..5.26 rows=90 width=0) (actual time=0.008..0.008 rows=0 loops=7)
                     Index Cond: ((chat_channel_id = uccm.chat_channel_id) AND (id > COALESCE(uccm.last_read_message_id, 0)))
 Planning Time: 1.217 ms
 Execution Time: 0.188 ms
(17 rows)
```
2022-12-20 05:10:53 +08:00
Daniel Waterworth d244eb0d7c
FIX: Don't validate channel slug if it hasn't changed (#19498)
This is generating lots of log noise for operations that aren't even
touching the slug.
2022-12-19 12:11:51 -06:00
Martin Brennan baf78d3d91
FIX: Add missing user_id args for ChatMessage.cook (#19508)
In both ChatMessage#rebake! and in ChatMessageProcessor
when we were calling ChatMessage.cook we were missing the
user_id to cook with, which causes missed hashtag cooks
because of missing permissions.
2022-12-19 11:05:37 +10:00
Krzysztof Kotlarek 09d15d4c7f
FIX: access to category chat only when user can create post (#19488)
Previously, restricted category chat channel was available for all groups - even `readonly`. From now on, only user who belong to group with `create_post` or `full` permissions can access that chat channel.
2022-12-19 11:35:28 +11:00
Martin Brennan d147e92953
FIX: Do not duplicate check when editing chat message to remove uploads (#19432)
There is no need to duplicate check chat messages when they are being
edited but not having their message text changed. This was leading to
a validation error when adding/removing an upload but not changing the
message text.
2022-12-14 10:48:23 +10:00
Martin Brennan 3ee4b59c64
DEV: Use guardian user for can_chat? (#19418)
Instead of passing `user` to `guardian.can_chat?`, we
can just use the inner `@user` that is part of the guardian
instance already to determine whether that user can chat,
since this is how it works for all other usages of guardian
even within chat.
2022-12-13 09:14:17 +10:00
Joffrey JAFFEUX 179d15d90e
FIX: webhook should touch channel’s last_message_sent_at (#19342) 2022-12-06 19:33:39 +01:00
Roman Rizzi 9c8043a4d2
FEATURE: Enforce mention limits for chat messages (#19034)
* FEATURE: Enforce mention limits for chat messages

The first part of these changes adds a new setting called `max_mentions_per_chat_message`, which skips notifications when the message contains too many mentions. It also respects the `max_users_notified_per_group_mention` setting
and skips notifications if expanding a group mention would exceed it.

We also include a new component to display JIT warning for these limits to the user while composing a message.

* Simplify ignoring/muting filter in chat_notifier

* Post-send warnings for unsent warnings

* Improve pluralization

* Address review feedback

* Fix test

* Address second feedback round

* Third round of feedback

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2022-12-06 14:54:04 -03:00
Martin Brennan 22a55ef0ce
DEV: Add messages_count to ChatChannel table (#19295)
This commit adds the messages_count column for ChatChannel messages,
which is the number of not-deleted messages in the channel.

This is not updated every time a message is created or deleted in a
channel, so it should not be displayed in the UI.
It is updated eventually via Jobs::ChatPeriodicalUpdates, which
will have additional functions in future after being introduced
here.

Also update these counts for existing channels in a post migration.
2022-12-06 08:40:46 +10:00
Joffrey JAFFEUX 68c4f16a73
FEATURE: channels can allow/disallow @all/@here mentions (#19317)
The settings tab of each category channel should now present the option to allow or disallow channel wide mentions: @here and @all.

When disallowed, using these mentions in the channel should have no effect.
2022-12-05 17:03:51 +01:00
Martin Brennan 7212a2ad51
FIX: Ensure chat channel slug uniqueness at DB level (#19277)
There must have been a small loophole that allowed
setting the channel slug in the DB which has led to
conflicts in some cases.

This commit fixes the conflicting chat channel
slugs and then changes the channel slug index
to a unique one in the DB.
2022-12-02 11:15:43 +10:00
Martin Brennan 8437081d94
FIX: Add MessageBust.last_id to chat channel subscriptions (#19255)
This commit adds variousMessageBus.last_ids to serializer payloads
for chat channels and the chat view (for chat live pane) so
we can use those IDs when subscribing to MessageBus channels
from chat.

This allows us to ensure that any messages created between the
server being hit and the UI loaded and subscribing end up being
delivered to the client, rather than just silently dropped.

This commit also fixes an issue where we were subscribing to
the new-messages and new-mentions MessageBus channels multiple
times when following/unfollowing a channel multiple times.
2022-12-02 10:57:53 +10:00
Roman Rizzi 07a9163ea8
FEATURE: Deleting a user with their posts also deletes chat messages. (#19194)
This commit introduce a new API for registering callbacks, which we'll execute when a user gets destroyed, and the `delete_posts` opt is true. The chat plugin registers one callback and queues a job to destroy every message from that user in batches.
2022-11-28 13:32:57 -03:00
Roman Rizzi 6d9e950cac
FIX: Unsilence users on chat message flag disagree. (#19198)
* FIX: Unsilence users on chat message flag disagree.

We have an auto silence rule in place for chat message flags, so we need to unsilence users if the flag gets rejected.

Additionally, it also fixes the `disagree_and_restore` action, which wasn't recovering a deleted message.

* Update plugins/chat/spec/models/reviewable_chat_message_spec.rb

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2022-11-28 11:11:35 -03:00
Martin Brennan 3de765c895
SECURITY: Limit chat message char length (#19207)
Only allow maximum of 6000 characters for chat messages when they
are created or edited. A hidden setting can control this limit,
6000 is the default.

There is also a migration here to truncate any existing messages to
6000 characters if the message is already over that and if the
chat_messages table exists. We also set cooked_version to NULL
for those messages so we can identify them for rebake.
2022-11-28 10:48:30 +10:00
Martin Brennan d3f02a1270
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.

**Serverside**

We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.

Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.

The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.

When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.

**Markdown**

The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.

This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.

**Chat Channels**

This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.

------

**Known Rough Edges**

- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 08:37:06 +10:00
chapoi 7e39a21de1
UX: redesign of chat settings + add chat retention info (#19017)
* UX: redesign of chat settings
2022-11-16 11:10:42 +01:00
Joffrey JAFFEUX a578bc2f5f
FIX: adds back recurring to send_message automation (#18990)
Also uses new way to get channel title
2022-11-11 15:58:05 +01:00
Martin Brennan c6764d8c74
FIX: Automatically generate category channel slugs (#18879)
This commit automatically ensures that category channels
have slugs when they are created or updated based on the
channel name, category name, or existing slug. The behaviour
has been copied from the Category model.

We also include a backfill here with a simplified version
of Slug.for with deduplication to fill the slugs for already
created Category chat channels.

The channel slug is also now used for chat notifications,
and for the UI and navigation for chat. `slugifyChannel`
is still used, but now does the following fallback:

* Uses channel.slug if it is present
* Uses channel.escapedTitle if it is present
* Uses channel.title if it is present

In future we may want to remove this altogether
and always rely on the slug being present, but this
is currently not possible because we are not generating
slugs for DM channels at this point.
2022-11-09 10:28:31 +10:00
Loïc Guitaut 9c482a645c FIX: Allow deletion of categories when chat channel is not present
Currently it’s not possible to delete a category if an associated chat
channel is present even if there are no messages in this channel.
This can lead to annoying situations for our users.

This patch addresses the issue by checking if the channel is empty
instead of just checking if there is a channel.
2022-11-08 10:25:39 +01:00
Martin Brennan 4116094e54
FIX: Make chat editor IDs not null (#18903)
Follow up to 766bcbc684

Makes ChatMessage.last_editor_id and ChatMessageRevision.user_id
NOT NULL since they are always filled in now and the last commit
had a migration to backfill this data.
2022-11-08 09:06:13 +10:00
Martin Brennan c66743ee3d
FIX: Make ChatMessageUpdater check editing access for guardian (#18902)
Follow up to 766bcbc684

This fixes a gaffe from that commit where I passed in the
guardian to ChatMessageUpdater but then forgot to remove
the old way of setting the guardian and user instance variables
from the chat_message that was passed in.

Also, it moves the ensure_can_edit_message! check from the
controller into ChatMessageUpdater so all the access
checks are in the same place.
2022-11-08 09:04:18 +10:00
Martin Brennan 766bcbc684
FIX: Add editing user ids to ChatMessage and ChatMessageRevision (#18877)
This commit adds last_editor_id to ChatMessage for parity with Post in
core, as well as adding user_id to the ChatMessageRevision record since
we need to know who is making edits and revisions to messages, in case
in future we want to allow more than just the current user to edit chat
messages. The backfill for data here simply uses the record's creating
user ID, but in future if we allow other people to edit the messages it
will use their ID.
2022-11-07 09:04:47 +10:00
Joffrey JAFFEUX 11f3618b80
DEV: initial system tests for chat and plugins (#18881)
This is a very basic to ensure it's working and open future possible work
2022-11-04 15:06:24 +01:00
Loïc Guitaut abcaa1a961 DEV: Rename direct message related models
This is a followup of the previous refactor where we created two new
models to handle all the dedicated logic that was present in the
`ChatChannel` model.

For the sake of consistency, `DMChannel` has been renamed to
`DirectMessageChannel` and the previous `DirectMessageChannel` model is
now named `DirectMessage`. This should help reasoning about direct
messages.
2022-11-03 14:39:23 +01:00
Joffrey JAFFEUX 9ff091dc01
FIX: the notification data on the client expects an identifier (#18850) 2022-11-03 09:06:05 +01:00
Roman Rizzi 0a5f548635
DEV: Move `discourse-chat` to the core repo. (#18776)
As part of this move, we are also renaming `discourse-chat` to `chat`.
2022-11-02 10:41:30 -03:00