- Limit bulk re-invite to 1 time per day
- Move bulk invite by csv behind a site setting (hidden by default)
- Bump invite expiry from 30 -> 90 days
## Updates to rate_limiter
When limiting reinvites I found that **staff** are never limited in any way. So I updated the **rate_limiter** model to allow for a few things:
- add an optional param of `staff_limit`, which (when included and passed values, and the user passes `.staff?`) will override the default `max` & `secs` values and apply them to the user.
- in the case you **do** pass values to `staff_limit` but the user **does not** pass `staff?` the standard `max` & `secs` values will be applied to the user.
This should give us enough flexibility to
1. continue to apply a strict rate limit to a standard user
2. but also apply a secondary (less strict) limit to staff
This commit adds a hover effect for drag and drop in
the admin emoji uploader. It also changes the "Add New
Emoji" button to open the file selector; previously it
was useless because it was disabled unless a name was
entered (which is not even a requirement for the emoji)
and also it didn't actually do anything on click even
if it wasn't disabled.
Now we have a way of adding files without having to drag
and drop them, which is nice.
Also in this PR, there was no indication before that the upload was
complete apart from the button becoming enabled again.
This commit adds the highlight class to the emoji list
and removes it once the highlight fade animation is done,
like we do for new posts.
* FIX: Pass category and tag IDs to the emit webhook event job.
Like webhooks won't fire when they're scoped to specific categories or tags because we're not passing the data to the job that emits it.
* Update config/initializers/012-web_hook_events.rb
Co-authored-by: Dan Ungureanu <dan@ungureanu.me>
Co-authored-by: Dan Ungureanu <dan@ungureanu.me>
This reverts commit 2c7906999a.
The changes break some things in local development (putting JS files
into minified files, not allowing debugger, and others)
This reverts commit ea84a82f77.
This is causing problems with `/theme-qunit` on legacy, non-ember-cli production sites. Reverting while we work on a fix
This is quite complex as it means that in production we have to build
Ember CLI test files and allow them to be used by our Rails application.
There is a fair bit of glue we can remove in the future once we move to
Ember CLI completely.
The `plugin:pull_compatible_all` task is intended to take incompatible plugins and downgrade them to an earlier version. Problem is, when running the rake task in development/production environments, the plugins have already been activated. If an incompatible plugin raises an error in `plugin.rb` then the rake task will be unable to start.
This commit centralises our LOAD_PLUGINS detection, adds support for LOAD_PLUGINS=0 in dev/prod, and adds a warning to `plugin:pull_compatible_all` if it's run with plugins enabled.
An admin could search for all screened ip addresses in a block by
using wildcards. 192.168.* returned all IPs in range 192.168.0.0/16.
This feature allows admins to search for a single IP address in all
screened IP blocks. 192.168.0.1 returns all IP blocks that match it,
for example 192.168.0.0/16.
* FEATURE: Remove roll up button for screened IPs
* FIX: Match more specific screened IP address first
The new warnings cover more cases and more accurate. Most of the
warnings will be visible only to staff members because otherwise they
would leak information about user's preferences.
Also:
* Remove an unused method (#fill_email)
* Replace a method that was used just once (#generate_username) with `SecureRandom.alphanumeric`
* Remove an obsolete dev puma `tmp/restart` file logic
* File.exists? is deprecated and removed in Ruby 3.2 in favor of
File.exist?
* Dir.exists? is deprecated and removed in Ruby 3.2 in favor of
Dir.exist?
If a theme/plugin raises an error while decorating post content, the decorator will be skipped, and the error reported on the console. Additionally, administrators will be shown a red warning at the top of the screen.
This commit refactors and re-uses some of the logic from the theme-initializer-error-reporting logic. In future, new error reports can be added by doing something like:
```
document.dispatchEvent(
new CustomEvent("discourse-error", {
detail: { messageKey: "some.translation.key", error },
})
);
```
This commit adds a check that runs regularly as per
2d68e5d942 which tests the
credentials of groups with SMTP or IMAP enabled. If any issues
are found with those credentials a high priority problem is added to the
admin dashboard.
This commit also formats the admin dashboard differently if
there are high priority problems, bringing them to the top of
the list and highlighting them.
The problem will be cleared if the issue is fixed before the next
problem check, or if the group's settings are updated with a valid
credential.
The previous default aspect ratio for cropping tall images was a little
too strict and was cutting off images. This new setting should allow for
a larger range of image sizes before cropping them.
This commit removes the enable_experimental_backup_uploader site
setting and the flags in backups-index.hbs to make the uppy
backup uploader the main one from now on.
A follow-up commit will delete the old backup uploader code and
also remove resumable.js from the project.
* FEATURE: hide_email_address_taken forces use of email in forgot password form
This strengthens this site setting which is meant to be used to harden sites
that are experiencing abuse on forgot password routes.
Previously we would only deny letting people know if forgot password worked on not
New change also bans usage of username for forgot password when enabled
MessageBus::Diagnostics allows anyone with access to carry out certain
operations that may result in a denial of service. The impact of this is
greater on multisiite clusters.
We send the reminder using the GroupMessage class, which supports removing previous messages. We can't match them by raw because they could mention different moderators. Also, I had to change the subject to remove dynamically generated values, which is necessary for finding them.
This commit introduces a new site setting "google_oauth2_hd_groups". If enabled, group information will be fetched from Google during authentication, and stored in the Discourse database. These 'associated groups' can be connected to a Discourse group via the "Membership" tab of the group preferences UI.
The majority of the implementation is generic, so we will be able to add support to more authentication methods in the near future.
https://meta.discourse.org/t/managing-group-membership-via-authentication/175950
We don't need it anymore. Actually, I removed using of it on the client side a long time ago, when I was working on improving blank page syndrome on user activity pages (see https://github.com/discourse/discourse/pull/14311).
This PR also removes some old resource strings that we don't use anymore. We have new strings for blank pages.
We have a couple of site setting, `slow_down_crawler_user_agents` and `slow_down_crawler_rate`, that are meant to allow site owners to signal to specific crawlers that they're crawling the site too aggressively and that they should slow down.
When a crawler is added to the `slow_down_crawler_user_agents` setting, Discourse currently adds a `Crawl-delay` directive for that crawler in `/robots.txt`. Unfortunately, many crawlers don't support the `Crawl-delay` directive in `/robots.txt` which leaves the site owners no options if a crawler is crawling the site too aggressively.
This PR replaces the `Crawl-delay` directive with proper rate limiting for crawlers added to the `slow_down_crawler_user_agents` list. On every request made by a non-logged in user, Discourse will check the User Agent string and if it contains one of the values of the `slow_down_crawler_user_agents` list, Discourse will only allow 1 request every N seconds for that User Agent (N is the value of the `slow_down_crawler_rate` setting) and the rest of requests made within the same interval will get a 429 response.
The `slow_down_crawler_user_agents` setting becomes quite dangerous with this PR since it could rate limit lots if not all of anonymous traffic if the setting is not used appropriately. So to protect against this scenario, we've added a couple of new validations to the setting when it's changed:
1) each value added to setting must 3 characters or longer
2) each value cannot be a substring of tokens found in popular browser User Agent. The current list of prohibited values is: apple, windows, linux, ubuntu, gecko, firefox, chrome, safari, applewebkit, webkit, mozilla, macintosh, khtml, intel, osx, os x, iphone, ipad and mac.
This is a big change to change over to using the uppy
upload mixin in the composer by default. This gets rid
of the temporary composer-editor-uppy component, as well
as removing the old ComposerUpload mixin and copying over
any missing functions that were not yet implemented by
ComposerUploadUppy. This has been working well on our
hosting for some time now and has led us to several
bug fixes.
This commit also deletes the old plugin API for adding
preprocessors for the uploads. The accepted method of doing
this now is via an uppy preprocessor plugin, which we have
several examples of in the core codebase.
Leaving the `enable_experimental_composer_uploader` site setting
intact for now because some plugins still rely on it, this
will be removed at a later date.
One step closer to ending the jQuery file uploader saga...
Currently when a user creates posts that are moderated (for whatever
reason), a popup is displayed saying the post needs approval and the
total number of the user’s pending posts. But then this piece of
information is kind of lost and there is nowhere for the user to know
what are their pending posts or how many there are.
This patch solves this issue by adding a new “Pending” section to the
user’s activity page when there are some pending posts to display. When
there are none, then the “Pending” section isn’t displayed at all.
* FEATURE: Optionally send a 'noindex' header in non-canonical responses
This will be used in a SEO experiment.
Co-authored-by: David Taylor <david@taylorhq.com>
When this setting is turned on, it will check that normalized emails
are unique. Normalized emails are emails without any dots or plus
aliases.
This setting can be used to block use of aliases of the same email
address.
Use @here to mention all users that were allowed to topic directly or
through group, who liked topics or read the topic. Only first 10 users
will be notified.
When rendering the markdown code blocks we replace the
offending characters in the output string with spans highlighting a textual
representation of the character, along with a title attribute with
information about why the character was highlighted.
The list of characters stripped by this fix, which are the bidirectional
characters considered relevant, are:
U+202A
U+202B
U+202C
U+202D
U+202E
U+2066
U+2067
U+2068
U+2069
* FEATURE: Always show advanced invite options
The UI is more simple and more efficient than how it was when the
advanced options toggle was introduced. It does not make sense to keep
it anymore.
* UX: Minor copy edits
* UX: Merge expire invite controls
There were two controls in the create invite modal. One was a static
text that displayed how much time is left until the invite expires. The
other one was a datetime selector that set the time the invite expires.
This commit merges the two controls in a single one: staff users will
continue to see the datetime selector without the static text and
regular users will only see the static text because they cannot set
when the invite expires.
* UX: Remove invite link
It should only be visible after the invite was created.
This commit refactors the direct external upload routes (get presigned
put, complete external, create/abort/complete multipart) into a
helper which is then included in both BackupController and the
UploadController. This is done so UploadController doesn't need
strange backup logic added to it, and so each controller implementing
this helper can do their own validation/error handling nicely.
This is a follow up to e4350bb966
Currently, Discourse rate limits all incoming requests by the IP address they
originate from regardless of the user making the request. This can be
frustrating if there are multiple users using Discourse simultaneously while
sharing the same IP address (e.g. employees in an office).
This commit implements a new feature to make Discourse apply rate limits by
user id rather than IP address for users at or higher than the configured trust
level (1 is the default).
For example, let's say a Discourse instance is configured to allow 200 requests
per minute per IP address, and we have 10 users at trust level 4 using
Discourse simultaneously from the same IP address. Before this feature, the 10
users could only make a total of 200 requests per minute before they got rate
limited. But with the new feature, each user is allowed to make 200 requests
per minute because the rate limits are applied on user id rather than the IP
address.
The minimum trust level for applying user-id-based rate limits can be
configured by the `skip_per_ip_rate_limit_trust_level` global setting. The
default is 1, but it can be changed by either adding the
`DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL` environment variable with the
desired value to your `app.yml`, or changing the setting's value in the
`discourse.conf` file.
Requests made with API keys are still rate limited by IP address and the
relevant global settings that control API keys rate limits.
Before this commit, Discourse's auth cookie (`_t`) was simply a 32 characters
string that Discourse used to lookup the current user from the database and the
cookie contained no additional information about the user. However, we had to
change the cookie content in this commit so we could identify the user from the
cookie without making a database query before the rate limits logic and avoid
introducing a bottleneck on busy sites.
Besides the 32 characters auth token, the cookie now includes the user id,
trust level and the cookie's generation date, and we encrypt/sign the cookie to
prevent tampering.
Internal ticket number: t54739.
* FEATURE: display warning when sharing a topic in a restricted category
If a topic belongs to a category that is not readable by everyone, display a text warning of "Only visible to members of groups: [group_a], [group_b]"
* DEV: Adding a new category means we need to bump this value
* DEV: pass category to showModal
It makes much more sense for these to be GlobalSettings, since, in
multisite clusters, only the default site's settings would be respected.
Co-authored-by: David Taylor <david@taylorhq.com>
This commit adds the RailsMultisite middleware in test mode when Rails.configuration.multisite is true. This allows for much more realistic integration testing. The `multisite_spec.rb` file is rewritten to avoid needing to simulate a middleware stack.
This PR introduces a new `enable_experimental_backup_uploads` site setting (default false and hidden), which when enabled alongside `enable_direct_s3_uploads` will allow for direct S3 multipart uploads of backup .tar.gz files.
To make multipart external uploads work with both the S3BackupStore and the S3Store, I've had to move several methods out of S3Store and into S3Helper, including:
* presigned_url
* create_multipart
* abort_multipart
* complete_multipart
* presign_multipart_part
* list_multipart_parts
Then, S3Store and S3BackupStore either delegate directly to S3Helper or have their own special methods to call S3Helper for these methods. FileStore.temporary_upload_path has also removed its dependence on upload_path, and can now be used interchangeably between the stores. A similar change was made in the frontend as well, moving the multipart related JS code out of ComposerUppyUpload and into a mixin of its own, so it can also be used by UppyUploadMixin.
Some changes to ExternalUploadManager had to be made here as well. The backup direct uploads do not need an Upload record made for them in the database, so they can be moved to their final S3 resting place when completing the multipart upload.
This changeset is not perfect; it introduces some special cases in UploadController to handle backups that was previously in BackupController, because UploadController is where the multipart routes are located. A subsequent pull request will pull these routes into a module or some other sharing pattern, along with hooks, so the backup controller and the upload controller (and any future controllers that may need them) can include these routes in a nicer way.
* FIX: small copy fix for embedding.sample
* FIX: improve copy for site_description and short_site_description
As part of the setup wizard, improve the description of these two
strings to add context on where they will be used, so that it is
clearer how to write each one.
Meta discussion: https://meta.discourse.org/t/unclear-double-question-in-setup-wizard/208344/
This commit changes the emoji uploader to use the UppyUploadMixin,
and makes some minor changes to the emoji uploader (tightening the
copy for drag and drop and adding a percentage for the upload).
Since no other uppy upload mixin components have needed to upload
multiple files so far, this necessitated adding a tracker for the
in progress uploads so we know when to reset the uploader once all
uploads are complete.
At the moment, the emoji uploader cannot be used for direct S3 uploads
because the admin emoji controller creates other records and does other
magic with the emojis. At some point we need to factor this kind of thing
into the ExternalUploadManager.transform! action to complete external
uploads.
Trying to use a local test hostname other than localhost
(e.g. discourse.test )for discourse development was difficult due
the fact that localhost was hardcoded in a few places. This patch
uses existing environment variables to allow a developer to use a
different domain when developing.
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
We are no longer able to display the image returned by Instagram directly within a Discourse site (either in the composer, or within a cooked post within a topic), so:
- Display an image placeholder in the composer preview
- A cooked post should use an iframe to display the Instagram 'embed' content
* DEV: Sanitize HTML admin inputs
This PR adds on-save HTML sanitization for:
Client site settings
translation overrides
badges descriptions
user fields descriptions
I used Rails's SafeListSanitizer, which [accepts the following HTML tags and attributes](018cf54073/lib/rails/html/sanitizer.rb (L108))
* Make sure that the sanitization logic doesn't corrupt settings with special characters
This commit groups `auth_overrides_*`, `discourse_connect_*` and `discourse_connect_provider_*` settings separately, rather than interspersing them.
There will be no functional change. This only affects the order in which they're shown in the admin panel
The `generate`, `rotate` and `suspicious` auth token logs are now always logged regardless of the `verbose_auth_token_logging` setting because we rely no these to detect suspicious logins.
This is a follow-up to https://github.com/discourse/discourse/pull/14541. This adds a hidden setting for restoring the old behavior for those users who rely on it. We'll likely deprecate this setting at some point in the future.
This commit makes the following change to the Edit Bookmark
modal window for clarity:
* If the user is editing an existing bookmark without a reminder set,
hide the "none needed" option. This will draw more attention to the
delete button.
* If the user is editing an existing bookmark with a reminder set for the
future, change the "none needed" option to say "remove reminder, keep bookmark"
To do this, I needed to provide an option to override the labels
for time shortcuts in certain cases, so I could keep the NONE shortcut
but have the different wording.
* FEATURE: Cache CORS preflight requests for 2h
Browsers will cache this for 5 seconds by default. If using MessageBus
in a different domain, Discourse will issue a new long polling, by
default, every 30s or so. This means we would be issuing a new preflight
request **every time**. This can be incredibly wasteful, so let's cache
the authorization in the client for 2h, which is the maximum Chromium
allows us as of today.
* fix tests
* FIX: do not display add to calendar for past dates
There is no value in saving past dates into calendar
* FIX: remove postId and move ICS to frontend
PostId is not necessary and will make the solution more generic for dates which doesn't belong to a specific post.
Also, ICS file can be generated in JavaScript to avoid calling backend.
Sometimes administrators want to permanently delete posts and topics
from the database. To make sure that this is done for a good reasons,
administrators can do this only after one minute has passed since the
post was deleted or immediately if another administrator does it.
Reimplemented following the revert in ce0daae636
This approach uses the global `e`/`q` shortcuts, rather than shifting focus to the `quote-button` component. The current `quoteState` is used to determine whether the quote-button is currently visible. If yes, an appEvent transmits the intention to the quote-button component. If no, the old behavior is maintained.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
This reverts the new e and q shortcuts for quick-edit, and quote. The current implementation of these is causing issues with quoting on mobile devices.
We intend restore these new shortcuts soon.
* Revert "FIX: Apply quote selection workaround to all browsers (#14558)"
This reverts commit 488f716c16.
* Revert "FIX: selection going missing in Safari (#14557)"
This reverts commit 538fe2cc31.
* Revert "UX: adds shortcuts for quote (q) and fast edit (e) (#14552)"
This reverts commit 2af6052307.
We aren't translating these settings, so it makes more sense to move them into the code. I added an instance method so plugins can add mappings for custom reasons.
It allows saving local date to calendar.
Modal is giving option to pick between ics and google. User choice can be remembered as a default for the next actions.
We relied on backticks to identify and replace site setting names with links. Unfortunately, some translations don't follow this convention, breaking this feature.
Additionally, this lets us linkify `category settings` and `watched words` without using HTML in the translations.
You may notice that I split the texts we want to linkify into two groups. I did this on purpose to emphasize those that should be translated (regular_links) from those who don't (site_settings_link). If you can think of a better solution, I'm open to suggestions.
* DEV: Remove HTML setting type and sanitization logic.
We concluded that we don't want settings to contain HTML, so I'm removing the setting type and sanitization logic. Additionally, we no longer allow the global-notice text to contain HTML.
I searched for usages of this setting type in the `all-the-plugins` repo and found none, so I haven't added a migration for existing settings.
* Mark Global notices containing links as HTML Safe.
The all inboxes was introduced in
016efeadf6 but we decided to roll it back
for performance reasons. The main performance challenge here is that PG
has to basically loop through all the PMs that a user is allowed to view
before being able to order by `Topic#bumped_at`. The all inboxes was not
planned as part of the new/unread filter so we've decided not to tackle
the performance issue for the upcoming release.
Follow-up to 016efeadf6
The file size error messages for max_image_size_kb and
max_attachment_size_kb are shown to the user in the KB
format, regardless of how large the limit is. Since we
are going to support uploading much larger files soon,
this KB-based limit soon becomes unfriendly to the end
user.
For example, if the max attachment size is set to 512000
KB, this is what the user sees:
> Sorry, the file you are trying to upload is too big (maximum
size is 512000KB)
This makes the user do math. In almost all file explorers that
a regular user would be familiar width, the file size is shown
in a format based on the maximum increment (e.g. KB, MB, GB).
This commit changes the behaviour to output a humanized file size
instead of the raw KB. For the above example, it would now say:
> Sorry, the file you are trying to upload is too big (maximum
size is 512 MB)
This humanization also handles decimals, e.g. 1536KB = 1.5 MB
Allows creating a bookmark with the `for_topic` flag introduced in d1d2298a4c set to true. This happens when clicking on the Bookmark button in the topic footer when no other posts are bookmarked. In a later PR, when clicking on these topic-level bookmarks the user will be taken to the last unread post in the topic, not the OP. Only the OP can have a topic level bookmark, and users can also make a post-level bookmark on the OP of the topic.
I had to do some pretty heavy refactors because most of the bookmark code in the JS topics controller was centred around instances of Post JS models, but the topic level bookmark is not centred around a post. Some refactors were just for readability as well.
Also removes some missed reminderType code from the purge in 41e19adb0d
We don't need no stinkin' denormalization! This commit ignores
the topic_id column on bookmarks, to be deleted at a later date.
We don't really need this column and it's better to rely on the
post.topic_id as the canonical topic_id for bookmarks, then we
don't need to remember to update both columns if the bookmarked
post moves to another topic.
Administrators can use second factor to confirm granting admin access
without using email. The old method of confirmation via email is still
used as a fallback when second factor is unavailable.
The previous excerpt was a simple truncated raw message. Starting with
this commit, the raw content of the draft is cooked and an excerpt is
extracted from it. The logic for extracting the excerpt mimics the the
`ExcerptParser` class, but does not implement all functionality, being
a much simpler implementation.
The two draft controllers have been merged into one and the /draft.json
route has been changed to /drafts.json to be consistent with the other
route names.
The server will automatically serve the crawler view to game console
browsers. Neither PlayStation or Xbox can render Discourse because not
all required browser APIs are present.
In order to include the new/unread count in the browse more message
under suggested topics, a couple of technical changes have to be made.
1. `PrivateMessageTopicTrackingState` is now auto-injected which is
similar to how it is done for `TopicTrackingState`. This is done so
we don't have to attempt to pass the `PrivateMessageTopicTrackingState`
object multiple levels down into the suggested-topics component. While
the object is auto-injected, we only fetch the initial state and start
tracking when the relevant private messages routes has been hit and only
when a private message's suggested topics is loaded. This is
done as we do not want to add the extra overhead of fetching the inital
state to all page loads but instead wait till the private messages
routes are hit.
2. Previously, we would stop tracking once the `user-private-messages`
route has been deactivated. However, that is not ideal since
navigating out of the route and back means we send an API call to the
server each time. Since `PrivateMessageTopicTrackingState` is kept in
sync cheaply via messageBus, we can just continue to track the state
even if the user has navigated away from the relevant stages.
Sidekiq was failing to boot in dev due to the following error. It seems
like constantizing stuff before the autoloader has kicked in caused
stuff to go weird. Root cause has not been identified but there is no
reason for us to have to warm up the cache during the initialization of
Rails.
```
2021-09-06T04:28:43.338Z pid=112028 tid=26vc WARN: NameError: uninitialized constant #<Class:0x0000564b365040d8>::RateLimiter
2021-09-06T04:28:43.338Z pid=112028 tid=26vc WARN: /discourse/app/models/post.rb:9:in `<class:Post>'
/discourse/app/models/post.rb:6:in `<top (required)>'
/ruby/2.7.4/lib/ruby/gems/2.7.0/gems/zeitwerk-2.4.2/lib/zeitwerk/kernel.rb:26:in `require'
/ruby/2.7.4/lib/ruby/gems/2.7.0/gems/zeitwerk-2.4.2/lib/zeitwerk/kernel.rb:26:in `require'
```
Improves the create account modal for screen readers by doing the following:
* Making the `modal-alert` section into an `aria-role="alert"` region and making it show and hide using height instead of display:none so screen readers pick it up. Made a change so the field-related error messages are always shown beneath the field.
* Add `aria-invalid` and `aria-describedby` attributes to each field in the modal, so the screen reader will read out the error hint on error. This necessitated an Ember component extension to allow both the `aria-*` attributes to be bound and to render on `{{input}}`.
* Moved the social login buttons to the right in the HTML structure so they are not read out first.
* Added `aria-label` attributes to the login buttons so they can have different content for screen readers.
* In some cases for modals, the title that should be used for the `aria-labelledby` attribute is within the modal content and not the discourse-modal-title title. This introduces a new titleAriaElementId property to the d-modal component that is then used by the create-account modal to read out the title
------
This is the same as e0d2de73d8 but
fixes the Ember-input-component-extension to use the public
Ember components TextField and TextArea instead of the private
TextSupport so the extension works in both normal Ember and
Ember CLI.
Improves the create account modal for screen readers by doing the following:
* Making the `modal-alert` section into an `aria-role="alert"` region and making it show and hide using height instead of display:none so screen readers pick it up. Made a change so the field-related error messages are always shown beneath the field.
* Add `aria-invalid` and `aria-describedby` attributes to each field in the modal, so the screen reader will read out the error hint on error. This necessitated an Ember component extension to allow both the `aria-*` attributes to be bound and to render on `{{input}}`.
* Moved the social login buttons to the right in the HTML structure so they are not read out first.
* Added `aria-label` attributes to the login buttons so they can have different content for screen readers.
* In some cases for modals, the title that should be used for the `aria-labelledby` attribute is within the modal content and not the discourse-modal-title title. This introduces a new titleAriaElementId property to the d-modal component that is then used by the create-account modal to read out the
PresenceChannel aims to be a generic system for allow the server, and end-users, to track the number and identity of users performing a specific task on the site. For example, it might be used to track who is currently 'replying' to a specific topic, editing a specific wiki post, etc.
A few key pieces of information about the system:
- PresenceChannels are identified by a name of the format `/prefix/blah`, where `prefix` has been configured by some core/plugin implementation, and `blah` can be any string the implementation wants to use.
- Presence is a boolean thing - each user is either present, or not present. If a user has multiple clients 'present' in a channel, they will be deduplicated so that the user is only counted once
- Developers can configure the existence and configuration of channels 'just in time' using a callback. The result of this is cached for 2 minutes.
- Configuration of a channel can specify permissions in a similar way to MessageBus (public boolean, a list of allowed_user_ids, and a list of allowed_group_ids). A channel can also be placed in 'count_only' mode, where the identity of present users is not revealed to end-users.
- The backend implementation uses redis lua scripts, and is designed to scale well. In the future, hard limits may be introduced on the maximum number of users that can be present in a channel.
- Clients can enter/leave at will. If a client has not marked itself 'present' in the last 60 seconds, they will automatically 'leave' the channel. The JS implementation takes care of this regular check-in.
- On the client-side, PresenceChannel instances can be fetched from the `presence` ember service. Each PresenceChannel can be used entered/left/subscribed/unsubscribed, and the service will automatically deduplicate information before interacting with the server.
- When a client joins a PresenceChannel, the JS implementation will automatically make a GET request for the current channel state. To avoid this, the channel state can be serialized into one of your existing endpoints, and then passed to the `subscribe` method on the channel.
- The PresenceChannel JS object is an ember object. The `users` and `count` property can be used directly in ember templates, and in computed properties.
- It is important to make sure that you `unsubscribe()` and `leave()` any PresenceChannel objects after use
An example implementation may look something like this. On the server:
```ruby
register_presence_channel_prefix("site") do |channel|
next nil unless channel == "/site/online"
PresenceChannel::Config.new(public: true)
end
```
And on the client, a component could be implemented like this:
```javascript
import Component from "@ember/component";
import { inject as service } from "@ember/service";
export default Component.extend({
presence: service(),
init() {
this._super(...arguments);
this.set("presenceChannel", this.presence.getChannel("/site/online"));
},
didInsertElement() {
this.presenceChannel.enter();
this.presenceChannel.subscribe();
},
willDestroyElement() {
this.presenceChannel.leave();
this.presenceChannel.unsubscribe();
},
});
```
With this template:
```handlebars
Online: {{presenceChannel.count}}
<ul>
{{#each presenceChannel.users as |user|}}
<li>{{avatar user imageSize="tiny"}} {{user.username}}</li>
{{/each}}
</ul>
```
See the previous commit d66b258b0e as
well.
If enable_upload_debug_mode is true, we do not want to abort the
direct S3 upload, because that will delete the file on S3 and prevent
further inspection of any errors that have come up.
There are certain design decisions that were made in this commit.
Private messages implements its own version of topic tracking state because there are significant differences between regular and private_message topics. Regular topics have to track categories and tags while private messages do not. It is much easier to design the new topic tracking state if we maintain two different classes, instead of trying to mash this two worlds together.
One MessageBus channel per user and one MessageBus channel per group. This allows each user and each group to have their own channel backlog instead of having one global channel which requires the client to filter away unrelated messages.
This pull request introduces the endpoints required, and the JavaScript functionality in the `ComposerUppyUpload` mixin, for direct S3 multipart uploads. There are four new endpoints in the uploads controller:
* `create-multipart.json` - Creates the multipart upload in S3 along with an `ExternalUploadStub` record, storing information about the file in the same way as `generate-presigned-put.json` does for regular direct S3 uploads
* `batch-presign-multipart-parts.json` - Takes a list of part numbers and the unique identifier for an `ExternalUploadStub` record, and generates the presigned URLs for those parts if the multipart upload still exists and if the user has permission to access that upload
* `complete-multipart.json` - Completes the multipart upload in S3. Needs the full list of part numbers and their associated ETags which are returned when the part is uploaded to the presigned URL above. Only works if the user has permission to access the associated `ExternalUploadStub` record and the multipart upload still exists.
After we confirm the upload is complete in S3, we go through the regular `UploadCreator` flow, the same as `complete-external-upload.json`, and promote the temporary upload S3 into a full `Upload` record, moving it to its final destination.
* `abort-multipart.json` - Aborts the multipart upload on S3 and destroys the `ExternalUploadStub` record if the user has permission to access that upload.
Also added are a few new columns to `ExternalUploadStub`:
* multipart - Whether or not this is a multipart upload
* external_upload_identifier - The "upload ID" for an S3 multipart upload
* filesize - The size of the file when the `create-multipart.json` or `generate-presigned-put.json` is called. This is used for validation.
When the user completes a direct S3 upload, either regular or multipart, we take the `filesize` that was captured when the `ExternalUploadStub` was first created and compare it with the final `Content-Length` size of the file where it is stored in S3. Then, if the two do not match, we throw an error, delete the file on S3, and ban the user from uploading files for N (default 5) minutes. This would only happen if the user uploads a different file than what they first specified, or in the case of multipart uploads uploaded larger chunks than needed. This is done to prevent abuse of S3 storage by bad actors.
Also included in this PR is an update to vendor/uppy.js. This has been built locally from the latest uppy source at d613b849a6. This must be done so that I can get my multipart upload changes into Discourse. When the Uppy team cuts a proper release, we can bump the package.json versions instead.
Major changes included:
- better support for screen readers
- trapping focus in modals
- better tabbing order in composer
- alerts on no content found/number of items found
- better autofocus in modals
- mini-tag-chooser is now a multi-select component
- each multi-select-component will now display selection on one row
Adds uppy upload functionality behind a
enable_experimental_composer_uploader site setting (default false,
and hidden).
When enabled this site setting will make the composer-editor-uppy
component be used within composer.hbs, which in turn points to
a ComposerUploadUppy mixin which overrides the relevant
functions from ComposerUpload. This uppy uploader has parity
with all the features of jQuery file uploader in the original
composer-editor, including:
progress tracking
error handling
number of files validation
pasting files
dragging and dropping files
updating upload placeholders
upload markdown resolvers
processing actions (the only one we have so far is the media optimization
worker by falco, this works)
cancelling uploads
For now all uploads still go via the /uploads.json endpoint, direct
S3 support will be added later.
Also included in this PR are some changes to the media optimization
service, to support uppy's different file data structures, and also
to make the promise tracking and resolving more robust. Currently
it uses the file name to track promises, we can switch to something
more unique later if needed.
Does not include custom upload handlers, that will come
in a later PR, it is a tricky problem to handle.
Also, this new functionality will not be used in encrypted PMs because
encrypted PM uploads rely on custom upload handlers.
Discourse Translator Bot
janzenisaac
David Taylor
Natalie Tay
Andrei Prigorshnev
Régis Hanol
Michael Brown
Martin Brennan
Roman Rizzi
Robin Ward
Jarek Radosz
Bianca Nenciu
Vinoth Kannan
Meghna
Peter Zhu
Tobias Eigen
Blake Erickson
Sam
Alan Guo Xiang Tan
Penar Musaraj
Dan Ungureanu
Angus McLeod
Kris
Michelle Bueno Saquetim Vendrame
Krzysztof Kotlarek
Osama Sayegh
Loïc Guitaut
Rafael dos Santos Silva
Arpit Jalan
RogerBW
jbrw
Daniel Waterworth
Joffrey JAFFEUX
tshenry
Leonardo Mosquera
Ryan Lerch
Jean
Joshua Rosenfeld
Simon Cossar
Gavin
Gerhard Schlager
Arpit Jalan
andrea
Kreyren