The group card and group members page were affecting each other and were
leaking members list and the query parameters which led to bad UX
experience and sub-optimal performance (client made more queries because
it was loading fewer members).
This commit refactors the group model to make it more consistent, remove
dead code, move error handling outside of model.
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access.
A few notes:
- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
* When viewing a tag, the search widget will now show a checkbox to scope the search by tag, which will limit search results to that tag on desktop and mobile
This brings the behavior in line with native Discourse SSO. If login is required, and a user tries to visit the forum, they will be directed straight to the external login page without requiring any clicks.
This method had grown into a monster. Its query had bugs
that I couldn't fix, and new features would be hard to add.
Also I don't understand how it all works anymore...
Replace it with common table expressions that can be queried
to generate the results we need, instead of subtracting
results using lots of "NOT IN" clauses.
Fixed are bugs with tag schemas that use combinations of
tag groups, parent tags, and one-tag-per-topic restrictions.
For example: https://meta.discourse.org/t/130991/6
* Fix user title logic when badge name customized
* Fix an issue where a user's title was not considered a badge granted title when the user used a badge for their title and the badge name was customized. this affected the effectiveness of revoke_ungranted_titles! which only operates on badge_granted_titles.
* When a user's title is set now it is considered a badge_granted_title if the badge name OR the badge custom name from TranslationOverride is the same as the title
* When a user's badge is revoked we now also revoke their title if the user's title matches the badge name OR the badge custom name from TranslationOverride
* Add a user history log when the title is revoked to remove confusion about why titles are revoked
* Add granted_title_badge_id to user_profile, now when we set badge_granted_title on a user profile when updating a user's title based on a badge, we also remember which badge matched the title
* When badge name (or custom text) changes update titles of users in a background job
* When the name of a badge changes, or in the case of system badges when their custom translation text changes, then we need to update the title of all corresponding users who have a badge_granted_title and matching granted_title_badge_id. In the case of system badges we need to first get the proper badge ID based on the translation key e.g. badges.regular.name
* Add migration to backfill all granted_title_badge_ids for both normal badge name titles and titles using custom badge text.
- Allow revoking keys without deleting them
- Auto-revoke keys after a period of no use (default 6 months)
- Allow multiple keys per user
- Allow attaching a description to each key, for easier auditing
- Log changes to keys in the staff action log
- Move all key management to one place, and improve the UI
These are the changes to tags routing that correspond to the category
routing changes of d84c34ad. The new scheme is:
/tags/c/*slug_path/:id/none/:tag_id/ENDPOINT
/tags/c/*slug_path/:id/none/:tag_id
/tags/c/*slug_path/:id/:tag_id/ENDPOINT
/tags/c/*slug_path/:id/:tag_id
In a category's settings, the Tags tab has two new fields to
specify the number of tags that must be added to a topic
from a tag group. When creating a new topic, an error will be
shown to the user if the requirement isn't met.
The routes for categories are changing. The scheme that I intend to move
us to is:
/c/*slug_path/(:id)/ENDPOINT
/c/*slug_path/(:id)
This commit adds support for the new scheme to the server side without
dropping support for existing URLs. It is necessary to support existing
URLs for two reasons:
* This commit does not change any client side routing code,
* Posts that contain category hashtags that refer to a root category
are baked into URLs that do not fit this new scheme, (/c/[id]-[slug])
This is a major change to draft internals. Previously there were quite a
few cases where the draft system would say "draft saved", when in fact
we just skipped saving.
This commit ensures the draft system deals with draft ownership handover in
a predictable way.
For example:
- Window 1 editing draft
- Window 2 editing same draft at the same time
Previously we would allow window 1 and 2 to just fight on the same draft
each window overwriting the same draft over an over.
This commit introduces an ownership concept where either window 1 or 2 win
and user is prompted on the loser window to reload screen to correct the issue
This also corrects edge cases where a user could have multiple browser windows
open and posts in 1 window, later to post in the second window. Previously
drafts would break in the second window, this corrects it.
* DEV: Add the actual "tag_groups/new" route
Allows refreshing the "new" page without an error.
* DEV: Prevent attempts to create group tags if tagging is disabled
* DEV: Refactor the tag-groups controller
Gets rid of `selectedItem`, `selected`, and `selectTagGroup` action.
* DEV: Rename tag-groups-show to tag-groups-edit
* DEV: Refactor tag-groups form
* Extracted the tag-groups-form that's used by tag-groups-new and tag-groups-edit
* The model is now a buffered property
* Serialization relies more heavily on RestAdapter now
* Data is sent as JSON
* Payload is now namespaced ("tag_group")
* Update app/assets/javascripts/discourse/controllers/tag-groups-new.js.es6
Co-Authored-By: Joffrey JAFFEUX <j.jaffeux@gmail.com>
* Update app/assets/javascripts/discourse/components/tag-groups-form.js.es6
Co-Authored-By: Joffrey JAFFEUX <j.jaffeux@gmail.com>
* Update app/assets/javascripts/discourse/controllers/tag-groups-edit.js.es6
Co-Authored-By: Joffrey JAFFEUX <j.jaffeux@gmail.com>
* Require q param in /tags/filter/search route.
* If not provided this route was causing a 500 error when
DiscourseTagging.clean_tag was called, because .downcase
was being called on the param (which was nil).
* Now return a 400 error instead.
Anonymous users are only possible if allow_anonymous_posting is true,
which means that 'user.is_anonymous' check implies that
allow_anonymous_posting is true.
* FEATURE: Site setting/ui to allow users to set their primary group
* prettier and remove logic from account template
* added 1 to 43 to make web_hook_user_serializer_spec pass
This fix ensures that searches that contain a null byte return a 400
error instead of causing a 500 error.
For some reason from rspec we will reach the raise statement inside
of the `rescue_from ArgumentError` block, but outside of rspec it will
not execute the raise statement and so a 500 is thrown instead of
reaching the `rescue_from Discourse::InvalidParameters` block inside of
the application controller.
This fix raises Discourse::InvalidParameters directly from the search
controller instead of relying on `PG::Connection.escape_string` to
raise the `ArgumentError`.
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:
pluck_first, equivalent to limit(1).pluck(*columns).first
and pluck_first! which, like other finder methods, raises an exception
when no record is found
`suspend` isn't a User attribute, but was being assigned to the frontend User model as if it was. The model has a computed property that depends on `suspended_till`, so instead of overriding this property, it's better to return relevant attributes.
Fixes a computed-property.override deprecation (https://emberjs.com/deprecations/v3.x#toc_computed-property-override)
* FIX: Do not encode the URL twice
Now that we encode slugs in the server we don't need this anymore.
Reverts fe5na33
* FIX: More places do deal with encoded slugs
* the param is a string now, not a hash
* FIX: Handle the nil slug on /categories
* DEV: Add seeded? method to identity default categories
* DEV: Use SiteSetting to keep track of seeded categories
This feature amends it so instead of using one challenge and honeypot
statically per site we have a rotating honeypot and challenge value which
changes every hour.
This means you must grab a fresh copy of honeypot and challenge value once
an hour or account registration will be rejected.
We also now cycle the value of the challenge when after successful account
registration forcing an extra call to hp.json between account registrations
Client has been made aware of these changes.
Additionally this contains a JavaScript workaround for:
https://bugs.chromium.org/p/chromium/issues/detail?id=987293
This is client side code that is specific to Chrome user agent and swaps
a PASSWORD type honeypot with a TEXT type honeypot.
When an admin changes the site setting slug_generation_method to
encoded, we weren't really encoding the slug, but just allowing non-ascii
characters in the slug (unicode).
That brings problems when a user posts a link to topic without the slug, as
our topic controller tries to redirect the user to the correct URL that contains
the slug with unicode characters. Having unicode in the Location header in a
response is a RFC violation and some browsers end up in a redirection loop.
Bug report: https://meta.discourse.org/t/-/125371?u=falco
This commit also checks if a site uses encoded slugs and clear all saved slugs
in the db so they can be regenerated using an onceoff job.
We were mixing in 20 or so methods into a controller just to use a single
one.
The helper itself is not the actual implementation anyway... MobileDetection
is responsible here.
If the setting is turned on, then the user will receive information
about the subject: if it was deleted or requires some special access to
a group (only if the group is public). Otherwise, the user will receive
a generic #404 error message. For now, this change affects only the
topics and categories controller.
This commit also tries to refactor some of the code related to error
handling. To make error pages more consistent (design-wise), the actual
error page will be rendered server-side.
Using popups is becoming increasingly rare. Full page redirects are already used on mobile, and for some providers. This commit removes all logic related to popup authentication, leaving only the full page redirect method.
For more info, see https://meta.discourse.org/t/do-we-need-popups-for-login/127988
* FEATURE: Adds an extra protection layer when decompressing files.
* Rename exporter/importer to zip importer. Update old locale
* Added a new composite class to decompress a file with multiple strategies
* Set max file size inside a site setting
* Ensure that file is deleted after compression
* Sanitize path and files before compressing/decompressing
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains.
We no longer need to use Rails "require_dependency" anywhere and instead can just use standard
Ruby patterns to require files.
This is a far reaching change and we expect some followups here.
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).
Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
The current manifest validation in Chrome requires at least one
non-maskable icon to make it installable.
This commit adds the maskable entry as another entry, following up
on changes added in 3e590b8
Because:
- Chrome 78 and Firefox Fenix have support for it
- The icons will look better by using all the availiable space
- Admins can control the icon and add appropriate padding since we have a
dedicated asset for the manifest logo. Read more about it on
https://css-tricks.com/maskable-icons-android-adaptive-icons-for-your-pwa/
This commit:
- Adds a new key under the icons array in the webmanifest, named purpose
with maskable value.
In development, we track the last requested theme id, and use that to refresh the correct stylesheet targets. The after_action hook runs on every request, but the preview_theme_id parameter is only sent on the initial HTML request. This commit ensures we only fetch the development theme_id on HTML requests
New site setting: `embed_any_origin` that will send postMessages to
wildcard origins `*` instead of the referer.
Most of the time you won't want to do this, so the setting is default to
`false`. However, there are certain situations where you want to allow
embedding to send post messages when there is no HTTP REFERER.
For example, if you created a native mobile app and you wanted to embed a list
of Discourse topics as HTML. In the code your HTML would be a
static file/string, which would not be able to send a referer. In this
case, the site setting will allow the embed to work.
From a security standpoint we currently only use `postMessage` to send
data about the size of the HTML document and scroll position, so it
should be enable if required with minimal security ramifications.
* Extract QuickAccessPanel from UserNotifications.
* FEATURE: Quick access panels in user menu.
This feature adds quick access panels for bookmarks and personal
messages. It allows uses to browse recent items directly in the user
menu, without being redirected to the full pages.
* REFACTOR: Use QuickAccessItem for messages.
Reusing `DefaultNotificationItem` feels nice but it actually requires a
lot of extra work that is not needed for a quick access item.
Also, `DefaultNotificationItem` shows an incorrect tooptip ("unread
private message"), and it is not trivial to remove / override that.
* Use a plain JS object instead.
An Ember object was required when `DefaultNotificationItem` was used.
* Prefix instead suffix `_` for private helpers.
* Set to null instead of deleting object keys.
JavaScript engines can optimize object property access based on the
object’s shape. https://mathiasbynens.be/notes/shapes-ics
* Change trivial try/catch to one-liners.
* Return the promise in case needs to be waited on.
* Refactor showAll to a link with href
* Store `emptyStatePlaceholderItemText` in state.
* Store items in Session singleton instead.
We can drop `staleItems` (and `findStaleItems`) altogether. Because
`(old) items === staleItems` when switching back to a quick access
panel.
* Add `limit` parameter to the `user_actions` API.
* Explicitly import Session instead.
* FEATURE: Add tl2 threshold for editing new posts
* Adds a new setting and for tl2 editing posts (30 days same as old value)
* Sets the tl0/tl1 editing period as 1 day
* FIX: Spec uses wrong setting
* Fix site setting on guardian spec
* FIX: post editing period specs
* Avoid shared examples
* Use update_columns to avoid callbacks on user during tests
If you click a (?) icon beside the reviewable status a pop up will
appear with expanded informatio that explains how the reviewable got its
score, and how it compares to system thresholds.
When there is an overridden text in the database that belongs to
a pluralized key which doesn't exist in English anymore, the
Customize Texts admin page was unusable. This stops those keys from
ever being returned by a search.
- Adds support for iOS Universal Links via an `apple-app-site-association` endpoint
Adds support for Google Digital Asset Links at the `.well-known/assetlinks.json` endpoint
* Revert "Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)""
This reverts commit 36425eb9f0.
* Fix: Show who read only if the attribute is enabled
* PERF: Precalculate the last post readed by a group member
* Use book-reader icon instear of far-eye
* FIX: update topic groups correctly
* DEV: Tidy up read indicator update on write
Previously, a regular user could not edit the title or category
of a topic if a hidden tag had already been applied.
This also stops hidden tag names from leaking in the error message.
* Reenable: "FEATURE: Publish read state on group messages. (#7989)"
This reverts commit 67f5cc1ce8.
* FIX: Read indicator only appears when the group setting is enabled
* Enable or disable read state based on group attribute
* When read state needs to be published, the minimum unread count is calculated in the topic query. This way, we can know if someone reads the last post
* The option can be enabled/disabled from the UI
* The read indicator will live-updated using message bus
* Show read indicator on every post
* The read indicator now shows read count and can be expanded to see user avatars
* Read count gets updated everytime someone reads a message
* Simplify topic-list read indicator logic
* Unsubscribe from message bus on willDestroyElement, removed unnecesarry values from post-menu, and added a comment to explain where does minimum_unread_count comes from
This adds support for a `<d-topics-list>` tag you can embed in your site
that will be rendered as a list of discourse topics. Any attributes on
the tag will be passed as filters. For example:
`<d-topics-list discourse-url="URL" category="1234">` will filter to category 1234.
To use this feature, enable the `embed topics list` site setting. Then
on the site you want to embed, include the following javascript:
`<script
src="http://URL/javascripts/embed-topics.js"></script>`
Where `URL` is your discourse forum's URL.
Then include the `<d-topics-list discourse-url="URL">` tag in your HTML document and it will
be replaced with the list of topics.
There are 5 visibility levels (similar to group visibility)
public (default)
logged-in users
members only
staff
owners
Admins & group owners always have visibility to group members.
This displays more useful messages for the most common issues we see:
- CSRF (when the user switches browser)
- Invalid IAT (when the server clock is wrong)
- OAuth::Unauthorized for OAuth1 providers, when the credentials are incorrect
This commit also stops earlier for disabled authenticators. Now we stop at the request phase, rather than the callback phase.
This feature (when enabled) will allow for invite_only sites to require
external authentication before they can redeem an invite.
- Created hidden site setting to toggle this
- Enables sending invites with local logins disabled
- OAuth button added to invite form
- Requires OAuth email address to match invite email address
- Prevents redeeming invite if OAuth authentication fails
This feature allows @ mentions to prioritize showing members of a group who
have explicit permission to a category.
This makes it far easier to @ mention group member when composing topics in
categories where only the group has access.
For example:
If Sam, Jane an Joan have access to bugs category.
Then `@` will auto complete to (jane,joan,sam) ordered on last seen at
This feature works on new topics and existing topics. There is an explicit
exclusion of trust level 0,1,2 groups cause they get too big.
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.
Customizations can be added and edited in Admin > Customize > Email Style.
Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.
As part of this work, RTL locales are now rendered correctly for all emails.
* FIX: Better error when SSO fails due to blank secret
* Update spec/requests/session_controller_spec.rb
Co-Authored-By: Robin Ward <robin.ward@gmail.com>
- will ensure this tag is not restricted to another category, and not only ensure this category can use it
- will clean tags param, in case client is sending an empty array, eg: [""], this could be solved client-side, but we ensure it won't happen ever this way
This commit contains 3 features:
- FEATURE: Allow downloading watched words
This introduces a button that allows admins to download watched words per action in a `.txt` file.
- FEATURE: Allow clearing watched words in bulk
This adds a "Clear All" button that clears all deleted words per action (e.g. block, flag etc.)
- FEATURE: List all blocked words contained in the post when it's blocked
When a post is rejected because it contains one or more blocked words, the error message now lists all the blocked words contained in the post.
-------
This also changes the format of the file for importing watched words from `.csv` to `.txt` so it becomes inconsistent with the extension of the file when watched words are exported.
* FEATURE: Allow choice of category when making a PM public
Previously it would default to uncategorized, which was not ideal on
some forums. This gives the staff member more choice about what they'd
like to do.
* Make the optional category more explicit
* Joffrey's feedback
* Revert "Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)""
This reverts commit f89bd55576.
* Replace .tar.zip with .zip
This fixes the problem where if a route ends with a dynamic segment and the segment contains a period e.g. `my.name`, `name` is interpreted as the format. This applies a default format constraints `/(json|html)/` on all routes. If you'd like a route to have a different format constraints, you can do something like this:
```ruby
get "your-route" => "your_controlller#method", constraints: { format: /(rss|xml)/ }
#or
get "your-route" => "your_controlller#method", constraints: { format: :xml }
```
* FEATURE: Allow customization of robots.txt
This allows admins to customize/override the content of the robots.txt
file at /admin/customize/robots. That page is not linked to anywhere in
the UI -- admins have to manually type the URL to access that page.
* use Ember.computed.not
* Jeff feedback
* Feedback
* Remove unused import
* FEATURE: admin/user exports are compressed using the zip format
* Update translations. Theme exporter now exports .zip file. Theme importer supports .zip and .gz files
* Fix controller test, updated locale and skip saving the csv export to disk
This allows you to temporarily disable components without having to remove them from a theme.
This feature is very handy when doing quick fix engineering.
The global setting disable_search_queue_threshold
(DISCOURSE_DISABLE_SEARCH_QUEUE_THRESHOLD) which default to 1 second was
added.
This protection ensures that when the application is unable to keep up with
requests it will simply turn off search till it is not backed up.
To disable this protection set this to 0.
Adds a second factor landing page that centralizes a user's second factor configuration.
This contains both TOTP and Backup, and also allows multiple TOTP tokens to be registered and organized by a name. Access to this page is authenticated via password, and cached for 30 minutes via a secure session.
This can cause unbound CPU usage in some cases, and excessive logging in other cases. This commit moves redis readonly information into the local process, but maintains the DistributedCache for postgres readonly state.
* SECURITY: Add confirmation screen when logging in via email link
* SECURITY: Add confirmation screen when logging in via user-api OTP
* FIX: Correct translation key in session controller specs
* FIX: Use .email-login class for page
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
* Expose a new plugin outlet. Pass group model to the group-member-dropdown so it can be accessed by plugins
* Added controller tests for group custom fields. update custom fields when updating a group
The impersonate any user by anonymous feature in dev should require a
deliberate opt-in. This way developers are better aware of the security
implications of this development only feature.
This also corrects FileHelper.download so it supports "follow_redirect"
correctly (it used to always follow 1 redirect) and adds a `validate_url`
param that will bypass all uri validation if set to false (default is true)
Previously external domains were allowed in the client-side redirects, but not the server-side redirects. Now the behavior is to only allow local origins.
Neil Lalonde
Dan Ungureanu
Vinoth Kannan
Martin Brennan
Penar Musaraj
Roman Rizzi
Martin Brennan
Krzysztof Kotlarek
David Taylor
Gerhard Schlager
Mark VanLandingham
Daniel Waterworth
Sam Saffron
Joffrey JAFFEUX
Jarek Radosz
Régis Hanol
Bianca Nenciu
Blake Erickson
Mark VanLandingham
Rafael dos Santos Silva
Arpit Jalan
Martin Brennan
Robin Ward
Kyle Zhao
Rimian Perkins
Osama Sayegh
Rishabh
Vinoth Kannan
Saurabh Patel
Michael Brown
Guo Xiang Tan
Ralph Rooding
Jeff Wong
Maja Komel
Bianca Nenciu