This commit prevents unallowed URLs in iframe src by adding a relative path like `https://bob.com/abc/def/../ghi`. Currently, the iframe linking to the site uses the current_user, not the post's author, so users who have no access to a certain path are not able to view anything they shouldn't.
In some cases reverse chronological can be very important.
- Oldest post by sam
- Oldest topic by sam
Prior to these new filters we had no way of searching for them.
Now the 2 new orders `order:oldest` and `order:oldest_topic` can be used
to find oldest topics and posts
* Update spec/lib/search_spec.rb
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
* Update spec/lib/search_spec.rb
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
---------
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
* FIX: Video thumbnails can have duplicates
It's possible that a duplicate video or even a very similar video could
generate the same video thumbnail. Because video thumbnails are mapped
to their corresponding video by using the video sha1 in the thumbnail
filename we need to allow for duplicate thumbnails otherwise even when a
thumbnail has been generated for a topic it will not be mapped
correctly.
This will also allow you to re-upload a video on the same topic to
regenerate the thumbnail.
* fix typo
This commit makes some fundamental changes to how hashtag cooking and
icon generation works in the new experimental hashtag autocomplete mode.
Previously we cooked the appropriate SVG icon with the cooked hashtag,
though this has proved inflexible especially for theming purposes.
Instead, we now cook a data-ID attribute with the hashtag and add a new
span as an icon placeholder. This is replaced on the client side with an
icon (or a square span in the case of categories) on the client side via
the decorateCooked API for posts and chat messages.
This client side logic uses the generated hashtag, category, and channel
CSS classes added in a previous commit.
This is missing changes to the sidebar to use the new generated CSS
classes and also colors and the split square for categories in the
hashtag autocomplete menu -- I will tackle this in a separate PR so it
is clearer.
* FIX: Fix for Default to subcategory when parent category does not allow posting
* added tests for edge case scenario
* implemented correct behaviour when parent category doesn't have subcategories
* implemented new fabricator for categories and suggested changes
https://meta.discourse.org/t/improving-mailman-email-parsing/253041
When mirroring a public mailling list which uses mailman, there were some cases where the incoming email was not associated to the proper user.
As it happens, for various (undertermined) reasons, the email from the sender is often not in the `From` header but can be in any of the following headers: `Reply-To`, `CC`, `X-Original-From`, `X-MailFrom`.
It might be in other headers as well, but those were the ones we found the most reliable.
The welcome topic user tip was for admins only, but in general, user
tips should be used for guiding new users through the features that
Discourse offers. For this reason, we decided to remove the user tip.
This commit also includes a few more copy tweaks to the welcome topic.
The old method updated only existing records, without considering that
new tags might have been created or some tags might not exist anymore.
This was usually not a problem because the stats were also updated by
other code paths.
However, the ensure consistency job should be more solid and help when
other code paths fail or after importing data.
Also, update category tag stats too should happen when updating other
category stats as well.
When uploading images via direct to S3 upload, we were
assuming that we could not pre-emptively check the file
size because the client may do preprocessing to reduce
the size, and UploadCreator could also further reduce the
size.
This, however, is not true of gifs, so we would have an
issue where you upload a gif > the max_image_size_kb
setting and had to wait until the upload completed for
this error to show.
Now, instead, when we direct upload gifs to S3, we check
the size straight away and present a file size error to
the user rather than making them wait. This will increase
meme efficiency by approximately 1000%.
Currently processing emails that are blank or have a nil value for the mail will cause several errors.
This update allows emails with blank body or missing sender to log the blank email error to the mail logs rather than throwing an error.
- Improves styleguide support
- Adds toggle color scheme to styleguide
- Adds properties mutators to styleguide
- Attempts to quit a session as soon as done with it in system specs, this should at least free resources faster
- Refactors fabricators to simplify them
- Adds more fabricators (uploads for example)
- Starts implementing components pattern in system specs
- Uses Chat::Message creator to create messages in system specs, this should help to have more real specs as the side effects should now happen
### What is this change?
The lounge category was replaced with the general category in https://github.com/discourse/discourse/pull/18097.
However, there are still a few references to the lounge category in code. In particular, `Category#seeded?` is erroring out in production looking for `SiteSetting.lounge_category_id`.
What is the problem?
In `SvgSpriteController#search` and `SvgSpriteController#icon_picker_search`, the controller actions
was using the `RailsMultisite::ConnectionManagement.with_hostname` API
but `params[:hostname]` was always `nil` because the routes does not
have a `:hostname` param component and the client does not ever pass the
`:hostname` param when making the request. When `RailsMultisite::ConnectionManagement.with_hostname` is
used with a `nil` argument, it ends up connecting to the default
multisite database. Usually this would be bad because we're allowing a
site in a multisite setup to connect to another site but thankfully no
private data is being leaked here.
What is the fix?
Since `SvgSpriteController#search` and `SvgSpriteController#icon_picker_search` are login required route,
there is no need for us to switch database connections. The fix here is
to simply remove the use of `RailsMultisite::ConnectionManagement.with_hostname`.
* FIX: Displaying the wrong number of minimum tags in the composer
When the minimum number of tags set for the category is larger than the minimum number of tags
set in the category tag-groups, the composer was displaying the wrong value.
This commit fixes the value displayed in the composer to show the max value between the required
for the category and the tag-groups set for the category.
This bug was reported on Meta in https://meta.discourse.org/t/tags-from-multiple-tag-groups-required-only-suggest-select-at-least-one-tag/263817
* FIX: Limiting tags in categories not working as expected
When a category was restricted to a tag group A, which was set to only allow
one tag from the group per topic, selecting a tag belonging only to A returned
other tags from A that also belonged to other group/s (if any).
Example:
Tag group A: alpha, beta, gamma, epsilon, delta
Tag group B: alpha, beta, gamma
Both tag groups set to only allow one tag from the group per topic.
If Category 1 was set to only allow tags from the tag group A, and the first tag
selected was epsilon, then, because they also belonged to tag group B, the tags
alpha, beta, and gamma were still returned as valid options when they should not be.
This commit ensures that once a tag from a tag group that restricts its tags to
one per topic is selected, no other tag from this group is returned.
This bug was reported on Meta in https://meta.discourse.org/t/limiting-tags-to-categories-not-working-as-expected/263143.
* FIX: Moving topics does not prompt to add required tag for new category
When a topic moved from a category to another, the tag requirements
of the new category were not being checked.
This allowed a topic to be created and moved to a category:
- that limited the tags to a tag group, with the topic containing tags
not allowed.
- that required N tags from a tag group, with the topic not containing
the required tags.
This bug was reported on Meta in https://meta.discourse.org/t/moving-tagged-topics-does-not-prompt-to-add-required-tag-for-new-category/264138.
* FIX: Editing topics with tag groups from parents allows incorrect tagging
When there was a combination between parent tags defined in a tag group
set to allow only one tag from the group per topic, and other tag groups
relying on this restriction to combine the children tag types with the
parent tag, editing a topic could allow the user to insert an invalid
combination of these tags.
Example:
Automakers tag group: landhover, toyota
- group set to limit one tag from the group per topic
Toyota models group: land-cruiser, hilux, corolla
Landhover models group: evoque, defender, discovery
If a topic was initially set up with the tags toyota, land-cruiser it was
possible to edit it by removing the tag toyota and adding the tag landhover
and other landhover model tags like evoque for example.
In this case, the topic would end up with the tags toyota, land-cruiser,
landhover, evoque because Discourse will automatically insert the
missing parent tag toyota when it detects the tag land-cruiser.
This combination of tags would violate the restriction specified in
the Automakers tag group resulting in an invalid combination of tags.
This commit enforces that the "one tag from the group per topic"
restriction is verified before updating the topic tags and also
make sure the verification checks the compatibility of parent tags that
would be automatically inserted.
After the changes, the user will receive an error similar to:
The tags land-cruiser, landhover cannot be used simultaneously.
Please include only one of them.
Watched words were converted to regular expressions containing \W, which
handled only ASCII characters. Using [^[:word]] instead ensures that
UTF-8 characters are also handled correctly.
TopicsFilter is meant to generate a query scope from a given string so
we don't really need to ensure any ordering outside of the supported
order filters.
* Color for turbo_rspec in CI (`progress` and `documentation` formats)
* Show "DONE" only when `documentation` formatter is used
* Fix formatting
* Collapse RSpec commands
* Add line wrapping to the `progress` formatter (to mitigate GH Actions issue)
- Update welcome topic copy
- Edit the welcome topic automatically when the title or description changes
- Remove “Create your Welcome Topic” banner/CTA
- Add "edit welcome topic" user tip
### What is the problem?
It is possible to pass an arbitrary value to the limit parameter in `TagsController#search`, and have it flow through `DiscourseTagging.filter_allowed_tags` where it will raise an error deep in the database driver. MiniSql ensures there's no injection happening, but that ultimately results in an invalid query.
### How does this fix it?
This change checks more strictly that the parameter can be cleanly converted to an integer by replacing the loose `#to_i` conversion semantics with the stronger `Kernel#Integer` ones.
**Example:**
```ruby
"1; SELECT 1".to_i
#=> 1
Integer("1; SELECT 1")
#=> ArgumentError
```
As part of the change, I also went ahead to disallow a limit of "0", as that doesn't seem to be a useful option. Previously only negative limits were disallowed.
### Background
When SSRF detection fails, the exception bubbles all the way up, causing a log alert. This isn't actionable, and should instead be ignored. The existing `rescue` does already ignore network errors, but fails to account for SSRF exceptions coming from `FinalDestination`.
### What is this change?
This PR does two things.
---
Firstly, it introduces a common root exception class, `FinalDestination::SSRFError` for SSRF errors. This serves two functions: 1) it makes it easier to rescue both errors at once, which is generally what one wants to do and 2) prevents having to dig deep into the class hierarchy for the constant.
This change is fully backwards compatible thanks to how inheritance and exception handling works.
---
Secondly, it rescues this new exception in `UserAvatar.import_url_for_user`, which is causing sporadic errors to be logged in production. After this SSRF errors are handled the same as network errors.
This fixes a bug in the create invite API where if you passed in an
integer for the group_ids field it would fail to add the user to the
specified group.
added site toggle functionality through site settings
added tests to implemented feature
Introduced suggested correction
renamed find_new_topic method and deleted click_new_topic_button method
After this change, in order to join a chat channel, a user needs to be in a group with at least “Reply” permission for the category. If the user only has “See” permission, they are able to preview the channel, but not join it or send messages. The auto-join function also follows this new restriction.
---------
Co-authored-by: Martin Brennan <martin@discourse.org>
We were giving topics with repeated words extra weight in search index.
This meant that it was trivial to stuff words into title to dominate in search
given we search for exact title matches first.
The following tweak means that:
`invite invited invites`
and
`invite some stuff`
Both rank the same for title searching.
Titles are short and punchy, duplicating words should not give special
weight.
Requires a full reindex to take effect.
This fixes the api documentation for the create invite api endpoint so
that it uses the `group_ids` param when specifying the group instead of
the incorrect `group_id` param.
This commit fixes an issue where the Likes Received notification
count in the user digest email was not affected by the
since/last_seen date for the user, which meant that no matter
how long it had been since the user visited the count was
always constant.
Now instead for the Likes Received count, we only count the
unread notifications of that type since the user was last
seen.
The current limit (250 characters) is too low, as we have some
translations used for our badge descriptions that result in a
description length of 264 characters.
To be on the safe side, the limit is now set to 500 characters.
A category's slug can be encoded when
`SiteSetting.slug_generation_method` has been set to "encoded". As a
result, we have to support non ASCII characters as well.
This commit adds support for excluding categories when using the
`category:` filter with the `-` prefix. For example,
`-category:category-slug` will exclude all topics that belong to the
category with slug "category-slug" and all of its sub-categories.
To only exclude a particular category and not all of its sub-categories,
the `-` prefix can be used with the `=` prefix. For example,
`-=category:category-slug` will only exclude topics that belong to the
category with slug "category-slug". Topics in the sub-categories of
"category-slug" will still be included.
This amends it so our cached counting reliant specs run in synchronize mode
When running async there are situations where data is left over in the table
after a transactional test. This means that repeat runs of the test suite
fail.
What is the problem?
We are relying on RSpec custom matchers in system tests by defining
predicates in page objects. The problem is that this can result in a
system test unnecessarily waiting up till the full duration of
Capybara's default wait time when the RSpec custom matcher is used with
`not_to`. Considering this topic page object where we have a `has_post?`
predicate defined.
```
class Topic < PageObject
def has_post?
has_css?('something')
end
end
```
The assertion `expect(Topic.new).not_to have_post` will end up waiting
the full Capybara's default wait time since the RSpec custom matcher is
calling Capybara's `has_css?` method which will wait until the selector
appear. If the selector has already disappeared by the time the
assertion is called, we end up waiting for something that will never
exists.
This commit fixes such cases by introducing new predicates that uses
the `has_no_*` versions of Capybara's node matchers.
For future reference, `to have_css` and `not_to have_css` is safe to sue
because the RSpec matcher defined by Capbyara is smart enough to call
`has_css?` or `has_no_css?` based on the expectation of the assertion.
Specifying more than two tag names when using the `tag:` filter was not
working because of a bug in the code where only the first two value in
the `tag:` filter was being selected.
What is the problem?
Consider the following timeline:
1. OP starts a topic.
2. Troll responds snarkily.
3. Flagger flags the post as “inappropriate”.
4. Admin agrees and hides the post.
5. Troll ninja-edits the post within the grace period, but still snarky.
6. Flagger flags the post as inappropriate again.
The current behaviour is that the flagger is met with an error saying the post has been reviewed and can't be flagged again for the same reason.
The desired behaviour is after someone has edited a post, it should be flaggable again.
Why is this happening?
This is related to the ninja-edit feature, where within a set grace period no new revision is created, but a new revision is required to flag the same post for the same reason.
So essentially there is a window between the naughty corner cooldown where a flagged post can't be edited, and the ninja-edit grace period, where an edit can be made without a new revision. Posts that are edited within this window can't be re-flagged by the same user.
|-----------------|-------------------------------|
^ Flag accepted | ~~~~~~~~~~~~~ 🥷🏻 ~~~~~~~~~~~~ |
| ^ Editing grace period over
^ Naughty corner cooldown over
How does this fix it?
We already create a new revision when ninja-editing a post with a pending flag. The issue above happens only in the case where the flag is already accepted.
This change extends the existing behaviour so that a new revision is created when ninja-editing any flagged post, regardless of the status of the flag. (Deleted flags excluded.)
This should also help with posterity, avoiding situations where a successfully flagged post looks innocuous in the history because it was ninja-edited, and vice versa.
* DEV: move sidebar community section to database
Before, community section was hard-coded. In the future, we are planning to allow admins to edit it. Therefore, it has to be moved to database to `custom_sections` table.
Few steps and simplifications has to be made:
- custom section was hidden behind `enable_custom_sidebar_sections` feature flag. It has to be deleted so all forums, see community section;
- migration to add `section_type` column to sidebar section to show it is a special type;
- migration to add `segment` column to sidebar links to determine if link should be displayed in primary section or in more section;
- simplify more section to have one level only (secondary section links are merged);
- ensure that links like `everything` are correctly tracking state;
- make user an anonymous links position consistence. For example, from now on `faq` link for user and anonymous is visible in more tab;
- delete old community-section template.
SearchIndexer is only automatically disabled in `before_all` and `before` blocks which means at the start
of test runs. Enabling the SearchIndexer in one `fab!` block will affect
all other `fab!` blocks which is not ideal as we may be indexing stuff
for search when we don't need to.
What is the problem?
The system tests incorrectly assumes that the discobot user which is
seeded by a core plugin will always be present. This is not true as the
discobot user will only be seeded when the test databases are migrated
with plugins enabled. If we migrate test databases without plugins being
enabled, the core system tests should still pass.
This header is used by Microsoft Exchange to indicate when certain types of
autoresponses should not be generated for an email.
It triggers our "is this mail autogenerated?" detection, but should not be used
for this purpose.
`TopicQuery#latest_results` which was being used by
`TopicQuery#list_filter` defaults to ordering by `Topic#bumped_at` in
descending order and that was taking precedent over the order scopes
being applied by `TopicsFilter`.
This allows multiple ordering to be specified by using a comma seperated string.
For example, `order:created,views` would order the topics by
`Topic#created_at` and then `Topic#views.
An older change about optimising images caused the selector that adds lightboxing not to apply on quoted images. This fixes that. The selector is now not applicable as optimisation occurs in a separate place.
This change allows quoted images to be opened in a lightbox.
This new modifier can be used by plugins to modify search ordering.
Specifically plugins such as discourse_solved can amend search ordering
so solved topics bump to the top.
Also correct edge case where low and high sort priority categories did not
order correctly when it came to closed/archived
Many blog posts use these to illustrate and images were previously omitted
Additionally strip superfluous HTML and BODY tags from embed HTML.
This was incorrectly returned from server.
This commit adds support for the following ordering filters:
1. `order:activity` which orders the topics by `Topic#bumped_at` in descending order
2. `order:activity-asc` which orders the topics by `Topic#bumped_at` in ascending order
3. `order:latest-post` which orders the topics by `Topic#last_posted_at` in descending order
4. `order:latest-post-asc` which orders the topics by `Topic#last_posted_at` in ascending order
5. `order:created` which orders the topics by `Topic#created_at` in descending order
6. `order:created-asc` which orders the topics by `Topic#created_at` in ascending order
7. `order:views` which orders the topics by `Topic#views` in descending order
8. `order:views-asc` which orders the topics by `Topic#views` in ascending order
9. `order:likes` which orders the topics by `Topic#likes` in descending order
10. `order:likes-asc` which orders the topics by `Topic#likes` in ascending order
11. `order:likes-op` which orders the topics by `Post#like_count` of the first post in the topic in descending order
12. `order:likes-op-asc` which orders the topics by `Post#like_count` of the first post in the topic in ascending order
13. `order:posters` which orders the topics by `Topic#participant_count` in descending order
14. `order:posters-asc` which orders the topics by `Topic#participant_count` in ascending order
15. `order:category` which orders the topics by `Category#name` of the topic's category in descending order
16. `order:category-asc` which orders the topics by `Category#name` of the topic's category in ascending order
Multiple order filters can be composed together and the order of ordering is applied based on the position of the filter
in the query string. For example, `order:views order:created` will order the topics by `Topic#views` in descending order
and then order the topics by `Topics#created_at` in descending order.
This commit adds support for the following date filters:
1. `activity-before:<YYYY-MM-DD>` which filters for topics that have been bumped at or before given date
2. `activity-after:<YYYY-MM-DD>` which filters for topics that have been bumped at or after given date
3. `created-before:<YYYY-MM-DD>` which filters for topics that have been created at or before given date
4. `created-after:<YYYY-MM-DD>` which filters for topics that have been created at or after given date
5. `latest-post-before:<YYYY-MM-DD>` which filters for topics with the
latest post posted at or before given date
6. `latest-post-after:<YYYY-MM-DD>` which filters for topics with the
latest post posted at or after given date
If the filter has an invalid value, i.e string that cannot be converted
into a proper date in the `YYYY-MM-DD` format, the filter will be ignored.
If either of each filter is specify multiple times, only the last
occurrence of each filter will be taken into consideration.
Followup to c03f83bbea.
The `flair_group_id` parameter is now required to show the flair, and this serializer was missing that detail.
This also fixes a typo in the `include_flair_group_name?` method.
Group user event webhooks filtered by group fail silently
because the `group_ids` job arg wasn't being passed into the job.
This change add's `group_ids` to the `EmitWebHookEvent` jobs queued for
`user_added_to_group` and `user_removed_from_group` events.
Large or broken images are removed from oneboxes, but sometimes images
were removed when they were oneboxed too. The reason is that images can
be oneboxed by the AllowlistedGenericOnebox or ImageOnebox and only
AllowlistedGenericOnebox was handled correctly.
We call the `/u/search/users` URL when autocompleting users. It returns
user's name, username and avatar template, but not user ID.
We need it to return user IDs in order to display user status in certain situations.
I could add ID to FoundUserWithStatusSerializer, so it will be added only if
user status is enabled in site settings. But I feel that it's good to always return it,
it's not a lot of data comparing to what we already return, and it should be useful
in other scenarios.
This pull request is a full overhaul of the chat-composer and contains various improvements to the thread panel. They have been grouped in the same PR as lots of improvements/fixes to the thread panel needed an improved composer. This is meant as a first step.
### New features included in this PR
- A resizable side panel
- A clear dropzone area for uploads
- A simplified design for image uploads, this is only a first step towards more redesign of this area in the future
### Notable fixes in this PR
- Correct placeholder in thread panel
- Allows to edit the last message of a thread with arrow up
- Correctly focus composer when replying to a message
- The reply indicator is added instantly in the channel when starting a thread
- Prevents a large variety of bug where the composer could bug and prevent sending message or would clear your input while it has content
### Technical notes
To achieve this PR, three important changes have been made:
- `<ChatComposer>` has been fully rewritten and is now a glimmer component
- The chat composer now takes a `ChatMessage` as input which can directly be used in other operations, it simplifies a lot of logic as we are always working a with a `ChatMessage`
- `TextareaInteractor` has been created to wrap the existing `TextareaTextManipulation` mixin, it will make future migrations easier and allow us to have a less polluted `<ChatComposer>`
Note ".chat-live-pane" has been renamed ".chat-channel"
Design for upload dropzone is from @chapoi
Vinoth Kannan
Sam
Loïc Guitaut
Alan Guo Xiang Tan
Krzysztof Kotlarek
Blake Erickson
Faizaan Gagan
Martin Brennan
Natalie Tay
Juan David Martínez Cubillos
Jarek Radosz
Régis Hanol
Bianca Nenciu
David Battersby
Keegan George
Joffrey JAFFEUX
Ted Johansson
Sérgio Saquetim
Richard
Jan Cernik
David Taylor
Penar Musaraj
Michael Brown
Gerhard Schlager
Mark VanLandingham
Selase Krakani
Isaac Janzen
Andrei Prigorshnev