The generate_presigned_put endpoint for direct external uploads
(such as the one for the uppy-image-uploader) records allowed
S3 metadata values on the uploaded object. We use this to store
the sha1-checksum generated by the UppyChecksum plugin, for later
comparison in ExternalUploadManager.
However, we were not doing this for the create_multipart endpoint,
so the checksum was never captured and compared correctly.
Also includes a fix to make sure UppyChecksum is the last preprocessor to run.
It is important that the UppyChecksum preprocessor is the last one to
be added; the preprocessors are run in order and since other preprocessors
may modify the file (e.g. the UppyMediaOptimization one), we need to
checksum once we are sure the file data has "settled".
The user-topic-list template is also in use in other places when we want to improve blank page syndrome, so this PR is a preparation for that changes as well.
- uses tagName=""
- removes user property which is not being used
- extract utility functions
- better wording for boolean properties
- initializes all properties
- uses @action
- uses optional chaining
- other minor changes
This rolls uppy back to the previous bundle that was used,
which will break multipart functionality (which is not yet
enabled anywhere).
No other upload functionality should be affected by this change,
it will be as if d295a16dab had
not been merged.
Users can invite people to topic and they will be automatically
redirected to the topic when logging in after signing up. This commit
ensures a "invited_to_topic" notification is created when the invite is
redeemed.
The same notification is used for the "Notify" sharing method that is
found in share topic modal.
This bug was introduced by f66007ec83.
In PostJobsEnqueuer we previously did not fire the after_post_create
event and after_topic_create event for private message topics. This was
changed in the above commit in order to publish message bus messages
for topic tracking state updates. Unfortunately this caused the
NotifyMailingListSubscribers job to be enqueued for all posts including
private messages, and admins and the users involved in the PMs got
emailed the contents of the PMs if they had mailing list mode enabled.
Luckily the impact of this was mitigated by a Guardian#can_see? check
for each mailing list mode user in the NotifyMailingListSubscribers job.
We never want to notify mailing list mode subscribers for private messages
so an early return has been added there, plus the logic in PostJobsEnqueuer
has been fixed, and tests have been added to that class where there were
none before.
Since ad3ec5809f when a user chooses
the Dismiss New... option in the New topic list, we send a request
to topics/reset-new.json with ?tracked=false as the only parameter.
This then uses Topic as the scope for topics to dismiss, with no
other limitations. When we do topic_scope.pluck(:id), it gets the
ID of every single topic in the database (that is not deleted) to
pass to TopicsBulkAction, causing a huge query with severe performance
issues.
This commit changes the default scope to use
`TopicQuery.new(current_user).new_results(limit: false)`
which should only use the topics in the user's New list, which
will be a much smaller list, depending on the user's "new_topic_duration_minutes"
setting.
Whenever we `subscribe` to something there should be an equivalent
`unsubscribe` and this implements it for `LogsNotice`.
In the future we should make this closer to what Ember expects a Service
to be, but at least it's properly cleaning up after itself now.
This can be used to change the list of topic posters. For example,
discourse-solved can use this to move the user who posted the solution
after the original poster.
See the previous commit d66b258b0e as
well.
If enable_upload_debug_mode is true, we do not want to abort the
direct S3 upload, because that will delete the file on S3 and prevent
further inspection of any errors that have come up.
There are certain design decisions that were made in this commit.
Private messages implements its own version of topic tracking state because there are significant differences between regular and private_message topics. Regular topics have to track categories and tags while private messages do not. It is much easier to design the new topic tracking state if we maintain two different classes, instead of trying to mash this two worlds together.
One MessageBus channel per user and one MessageBus channel per group. This allows each user and each group to have their own channel backlog instead of having one global channel which requires the client to filter away unrelated messages.
This pull request introduces the endpoints required, and the JavaScript functionality in the `ComposerUppyUpload` mixin, for direct S3 multipart uploads. There are four new endpoints in the uploads controller:
* `create-multipart.json` - Creates the multipart upload in S3 along with an `ExternalUploadStub` record, storing information about the file in the same way as `generate-presigned-put.json` does for regular direct S3 uploads
* `batch-presign-multipart-parts.json` - Takes a list of part numbers and the unique identifier for an `ExternalUploadStub` record, and generates the presigned URLs for those parts if the multipart upload still exists and if the user has permission to access that upload
* `complete-multipart.json` - Completes the multipart upload in S3. Needs the full list of part numbers and their associated ETags which are returned when the part is uploaded to the presigned URL above. Only works if the user has permission to access the associated `ExternalUploadStub` record and the multipart upload still exists.
After we confirm the upload is complete in S3, we go through the regular `UploadCreator` flow, the same as `complete-external-upload.json`, and promote the temporary upload S3 into a full `Upload` record, moving it to its final destination.
* `abort-multipart.json` - Aborts the multipart upload on S3 and destroys the `ExternalUploadStub` record if the user has permission to access that upload.
Also added are a few new columns to `ExternalUploadStub`:
* multipart - Whether or not this is a multipart upload
* external_upload_identifier - The "upload ID" for an S3 multipart upload
* filesize - The size of the file when the `create-multipart.json` or `generate-presigned-put.json` is called. This is used for validation.
When the user completes a direct S3 upload, either regular or multipart, we take the `filesize` that was captured when the `ExternalUploadStub` was first created and compare it with the final `Content-Length` size of the file where it is stored in S3. Then, if the two do not match, we throw an error, delete the file on S3, and ban the user from uploading files for N (default 5) minutes. This would only happen if the user uploads a different file than what they first specified, or in the case of multipart uploads uploaded larger chunks than needed. This is done to prevent abuse of S3 storage by bad actors.
Also included in this PR is an update to vendor/uppy.js. This has been built locally from the latest uppy source at d613b849a6. This must be done so that I can get my multipart upload changes into Discourse. When the Uppy team cuts a proper release, we can bump the package.json versions instead.
Steps to reproduce:
1. Go to activity/bookmarks
2. Search for something that isn’t in your bookmarks, so you get no results
3. Navigate away and then click "Bookmarked" on the sidebar or open the user menu and click the View All Bookmarks button on the bottom of the bookmarks tab, and you get the message "You haven't bookmarked anything yet".
This commit fixes the problem. We have a controller with a query parameter q that contains a search query. And we also have a property searchTerm that is bound to the search box on the page and mirrors the value in q. We were using a value from searchTerm when querying the server, but ember controllers are singletons so the searchTerm value persisted between page visits and leaded to this bug.
To make things work properly, we should be using the value from q everywhere except two places when we copy a value from q to searchTerm and vice versa.
Major changes included:
- better support for screen readers
- trapping focus in modals
- better tabbing order in composer
- alerts on no content found/number of items found
- better autofocus in modals
- mini-tag-chooser is now a multi-select component
- each multi-select-component will now display selection on one row
Plugin API is allowing to add small action codes dedicated to groups.
This will be used by assign-plugin when topic is assigned or unassigned from group.
When resetting the preprocessor status states, we weren't using
the same default state as when the preprocessor status state is
first initialized with an associated plugin. This commit brings
the two into alignment, fixing a bug where if you cancelled an
upload then tried a new one the "Processing Upload" message would
never change to "Uploading... X", so any subsequent uploads were
uncancellable.
Since the state was not being reset correctly, the properties that
were supposed to be numbers ended up as `undefined`, so when calling
prop-- or prop++, they turned into NaN.
This change only applies when uppy is calling the media-optimization-worker.
Since the old way of calling the worker via jQuery file uploader will
be removed soon, there is no point coming up with some random string
to use in place of the file name for the promise resolvers there, we
can live with this for now.
When we encountered an error with the media-optimization-worker,
we stopped the worker, which made it so further messages were not
received when optimizing images in parallel. Removed this based
on an option.
Also added more debugging lines to help track down issues.
* FIX: Revoking admin or moderator status doesn't require refresh to delete/anonymize/merge user
On the /admin/users/<id>/<username> page, there are action buttons that are either visible or hidden depending on a few fields from the AdminDetailsSerializer: `can_be_deleted`, `can_be_anonymized`, `can_be_merged`, `can_delete_all_posts`.
These fields are updated when granting/revoking admin or moderator status. However, those updates were not being reflected on the page. E.g. if a user is granted moderation privileges, the 'anonymize user' and 'merge' buttons still appear on the page, which is inconsistent with the backend state of the user. It requires refreshing the page to update the state.
This commit fixes that issue, by syncing the client model state with the server state when handling a successful response from the server. Now, when revoking privileges, the buttons automatically appear without refreshing the page. Similarly, when granting moderator privileges, the buttons automatically disappear without refreshing the page.
* Add detailed user response to spec for changed routes.
Add tests to verify that the revoke_moderation, grant_moderation, and revoke_admin routes return a response formatted according to the AdminDetailedUserSerializer.
When a theme's default color scheme is not marked as user selectable, we were outputting the numeric ID in the UI. This outputs "Theme default" instead.
This is useful in the DiscourseHub mobile app, currently the app queries
the `about.json` endpoint, which can raise a CORS issue in some cases,
for example when the site only accepts logins from an external provider.
I was storing the wrong object as the event listener
reference for the paste and mobile upload button click
events so they were not being cleaned properly on element
destruction.
Also renamed `uploadButton` to the more descriptive
`mobileUploadButton`.
When the composer reply is cancelled and the draft is trashed,
the isUploading and isProcessing statuses were not being reset,
so when the composer was opened again the Uploading... or
Processing... message still showed even when the uploads had
been cancelled correctly.
The regular composer-upload mixin suffered the same problem
as the uppy one, where the Processing/Uploading message was not
reset when a reply was cancelled and the draft destroyed.
When I added the paste event for files in the composer to
send to Uppy, I inadvertently called event.preventDefault()
if the pasted data was text. I removed that now, and I only
return early if the user cannot upload, and if there are no
files on the clipboard nothing happens.
Adds uppy upload functionality behind a
enable_experimental_composer_uploader site setting (default false,
and hidden).
When enabled this site setting will make the composer-editor-uppy
component be used within composer.hbs, which in turn points to
a ComposerUploadUppy mixin which overrides the relevant
functions from ComposerUpload. This uppy uploader has parity
with all the features of jQuery file uploader in the original
composer-editor, including:
progress tracking
error handling
number of files validation
pasting files
dragging and dropping files
updating upload placeholders
upload markdown resolvers
processing actions (the only one we have so far is the media optimization
worker by falco, this works)
cancelling uploads
For now all uploads still go via the /uploads.json endpoint, direct
S3 support will be added later.
Also included in this PR are some changes to the media optimization
service, to support uppy's different file data structures, and also
to make the promise tracking and resolving more robust. Currently
it uses the file name to track promises, we can switch to something
more unique later if needed.
Does not include custom upload handlers, that will come
in a later PR, it is a tricky problem to handle.
Also, this new functionality will not be used in encrypted PMs because
encrypted PM uploads rely on custom upload handlers.
When a staff member clicks on a user's number of flagged posts, we redirect them to the review queue, so it makes sense to count the number of items there to calculate the count.
We used to look at post action items to calculate this number, which doesn't match the number of items in the queue if old flags exist.
The invite acceptance page is an alternative signup flow, so it makes sense to include the new 'link' functionality there as well.
Followup to 7dc8f8b794
We've recently added a limit to the posts history modal so it displays the last 100 revisions only for performance reasons. However, the title of the modal now always says `History, last 100 revisions` even when the post has fewer than 100 revisions which can be a bit noisy.
This PR amends the history modal so the title of the modal says `History` when the post's revisions count is ≤100, and `History, last 100 revisions` when it has more >100 revisions.
When a user signs up via an external auth method, a new link is added to the signup modal which allows them to connect an existing Discourse account. This will only happen if:
- There is at least 1 other auth method available
and
- The current auth method permits users to disconnect/reconnect their accounts themselves
This handles a few edge cases which are extremely rare (due to the UI layout), but still technically possible:
- Ensure users are authenticated before attempting association.
- Add a message and logic for when a user already has an association for a given auth provider.
- active setting should now correctly show an arrow which was previously floating in the middle of nowhere
- uses a correct color for border separation, previously the border was present but invisible as similar to the background
- slighty tweak padding
- makes arrow computation based on a variable
Martin Brennan
David Taylor
Andrei Prigorshnev
Daniel Waterworth
Joffrey JAFFEUX
Martin Brennan
Dan Ungureanu
Blake Erickson
Robin Ward
Penar Musaraj
Bianca Nenciu
Arpit Jalan
Alan Guo Xiang Tan
Kris
Osama Sayegh
Sam
Krzysztof Kotlarek
Joe
Roman Rizzi
Grayden
Rafael dos Santos Silva
Jordan Vidrine
Vinoth Kannan
Mark VanLandingham