If there's ever a circular reference in categories, don't go into an infinite loop when generating the category slug.
Instead, keep track of parent ids, and bail out as soon as we're encountering one more than once.
In #22851 we added a dependent strategy for deleting upload references when a draft is destroyed. This, however, didn't catch all cases, because we still have some code that issues DELETE drafts queries directly to the database. Specifically in the weekly cleanup job handled by Draft#cleanup!.
This PR fixes that by turning the raw query into an ActiveRecord #destroy_all, which will invoke the dependent strategy that ultimately deletes the upload references. It also includes a post migration to clear orphaned upload references that are already in the database.
In this PR we started redirecting to the guide page after the wizard - https://github.com/discourse/discourse/pull/26696
The guide will require rebrand and until it is ready, we should redirect to `/latest`
This commit introduces the `run_theme_migration` spec helper to allow
theme developers to write RSpec tests for theme migrations. For example,
this allows the following RSpec test to be written in themes:
```
RSpec.describe "0003-migrate-small-links-setting migration" do
let!(:theme) { upload_theme_component }
it "should set target property to `_blank` if previous target component is not valid or empty" do
theme.theme_settings.create!(
name: "small_links",
theme: theme,
data_type: ThemeSetting.types[:string],
value: "some text, #|some text 2, #, invalid target",
)
run_theme_migration(theme, "0003-migrate-small-links-setting")
expect(theme.settings[:small_links].value).to eq(
[
{ "text" => "some text", "url" => "#", "target" => "_blank" },
{ "text" => "some text 2", "url" => "#", "target" => "_blank" },
],
)
end
end
```
This change is being introduced because we realised that writting just
javascript tests for the migrations is insufficient since javascript
tests do not ensure that the migrated theme settings can actually be
successfully saved into the database. Hence, we are introduce this
helper as a way for theme developers to write "end-to-end" migrations
tests.
* Simplify config nav link generation to always inject the Settings
tab
* Auto-redirect to the first non-settings config link (if there is one)
when the user lands on /admin/plugins/:plugin_id
* Add `extras` to admin plugin serializer so plugins can add more
data on first load
* Add PikadayCalendar page object for system specs, extracted from the
CalendarDateTimePicker to make it more generic.
This commit adds a `isValidUrl` helper function to the context in
which theme migrations are ran in. This helper function is to make it
easier for theme developers to check if a string is a valid URL or path
when writing theme migrations. This can be helpful in cases when
migrating a string based setting to `type: objects` which contain `type:
string` properties with URL validations enabled.
This commit also introduces the `UrlHelper.is_valid_url?` method
which actually checks that the URL string is of the valid format instead of
only checking if the URL string is parseable which is what `UrlHelper.relaxed_parse` does
and is not sufficient for our needs.
This is a follow up of 5fcb7c262d
It was missing the case where secure uploads is enabled, which creates a copy of the upload no matter what.
So this checks for the original_sha1 of the uploads as well when checking for duplicates.
In this PR we introduced an admin sidebar for moderators - https://github.com/discourse/discourse/pull/26795
`What's new` and `all reports` links were missing as moderators have access to those pages.
At the moment, there is no way to create a group of related watched words together. If a user needed a set of words to be created together, they'll have to create them individually one at a time.
This change attempts to allow related watched words to be created as a group. The idea here is to have a list of words be tied together via a common `WatchedWordGroup` record. Given a list of words, a `WatchedWordGroup` record is created and assigned to each `WatchedWord` record. The existing WatchedWord creation behaviour remains largely unchanged.
Co-authored-by: Selase Krakani <skrakani@gmail.com>
Co-authored-by: Martin Brennan <martin@discourse.org>
We were getting this error causing it to flake when
creating users:
```
ActiveRecord::StatementInvalid:
PG::ReadOnlySqlTransaction: ERROR: cannot execute INSERT in a read-only transaction
```
This commit introduces a few changes as a result of
customer issues with finding why a topic was relisted.
In one case, if a user edited the OP of a topic that was
unlisted and hidden because of too many flags, the topic
would get relisted by directly changing topic.visible,
instead of going via TopicStatusUpdater.
To improve tracking we:
* Introduce a visibility_reason_id to topic which functions
in a similar way to hidden_reason_id on post, this column is
set from the various places we change topic visibility
* Fix Post#unhide! which was directly modifying topic.visible,
instead we use TopicStatusUpdater which sets visibility_reason_id
and also makes a small action post
* Show the reason topic visibility changed when hovering the
unlisted icon in topic status on topic titles
A change in relative picker was causing a serie of events which ultimately would cause the whole list of time options to be reset and re-rendered which would cause a new instance of the picker to be created, causing a reset.
The fix is using id in the each loop to help ember identify that it doesn’t have to re-render a specific component.
This commit adds a test to ensure that settings for a theme are retained
even if saving a theme setting after a theme setting migration fails.
Prior to the fix introduced in 35bc27a36d,
failing to save the new settings will actually cause some or all of the
theme's setting to be lost.
e05628c0 introduced an optimization to remove basic-HTML content for authenticated users. The assumption is that, if they were able to log in, they must have a JS capable browser and do not need the basic HTML.
However, there are use-cases where an API-key is used to crawl a private site, or private categories of a public site. This commit re-enables those use cases by keeping the basic-html in place for crawler/bot user agents.
Selecting the +subcategories option does not work sometimes when "lazy
load categories" is enabled because the subcategories may not be
fetched. This ensures that subcategories are loaded by requesting them
before being used.
This commit fixes a bug in theme settings migrations where values of `objects` typed theme settings aren't passed to migrations even when there are overriding values for those settings. What causes this bug is that, when creating the hash that contains all the overridden settings and will be passed to migrations, the values of `objects` typed settings are incorrectly retrieved from the `value` column (which is always nil for `objects` type) instead of `json_value`. `objects` settings are different from all other types in that they store their values in the `json_value` column and they need to be special-cased when retrieving their values.
In a large forum with millions of users and millions of user_fields
updating the list of dropdown user field options will result in a
502 now due to the large number of fields.
This commit moves the indexing into a job.
Our 'page_view_crawler' / 'page_view_anon' metrics are based purely on the User Agent sent by clients. This means that 'badly behaved' bots which are imitating real user agents are counted towards 'anon' page views.
This commit introduces a new method of tracking visitors. When an initial HTML request is made, we assume it is a 'non-browser' request (i.e. a bot). Then, once the JS application has booted, we notify the server to count it as a 'browser' request. This reliance on a JavaScript-capable browser matches up more closely to dedicated analytics systems like Google Analytics.
Existing data collection and graphs are unchanged. Data collected via the new technique is available in a new 'experimental' report.
This commit fixes a bug in the `themes:update` rake task which resulted
in the ActiveRecord transaction not being rolled back when an error was
encountered. The transaction was first introduced in
7f0682f4f2 which changed a `begin..rescue`
block to `transaction do..rescue`. The problem with that change
prevented the transaction from ever rolling back as the code block
looks something like this:
```
transaction do
begin
update_theme
rescue => e
# surpress error
end
end
```
From the transaction's point of view now, it will never rollback even if
an error was encountered when updating the remote theme because it will
never see the error.
Instead we should have done something like this if we wanted to surpress
the errors encountered while still ensuring that the transaction is
rolled back.
```
begin
transaction do
update_theme
end
rescue => e
# surpress error
end
```
Fixes two issues:
- frontend was reloading the page when clicking-to-remove avatar
- backend wasn't allowing resetting the setting by deleting all avatars
- Run the CSP-nonce-related middlewares on the generated response
- Fix the readonly mode checking to avoid empty strings being passed (the `check_readonly_mode` before_action will not execute in the case of these re-dispatched exceptions)
- Move the BlockRequestsMiddleware cookie-setting to the middleware, so that it is included even for unusual HTML responses like these exceptions
This is only required in rails_helper, otherwise it is
not loaded. Allows for better debugging by allowing
navigation of the call stack from the point of `binding.pry`
c.f. https://github.com/pry/pry-stack_explorer
This commit adds a `getCategoryIdByName` helper function to the context in
which theme migrations are ran in. The purpose of this commit is to
allow themes which may have stored category names in theme settings to
migrate to objects typed theme settings which supports properties of
categories typed which stores the category ids in the value of the
property.
LinkedIn has grandfathered its old OAuth2 provider. This can only be used by existing apps. New apps have to use the new OIDC provider.
This PR adds a linkedin_oidc provider to core. This will exist alongside the discourse-linkedin-auth plugin, which will be kept for those still using the deprecated provider.
This ensures we only ever store correct post and topic timing when the client
notifies.
Previous to this change we would blindly trust the client.
Additionally this has error correction code that will correct the last seen
post number when you visit a topic with incorrect timings.
When the user sees no results in their admin sidebar query,
we are adding two additional links:
* "Search site settings" - Navigates to the site settings page
with the filter prefilled in the search
* "Admin user list" - Navigates to the user list with the filter
prefilled in the username search
This will bridge the gap until we have a full admin-wide search.
Also make admin site setting search param refresh on filter changes
---------
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
Followup to 67a8080e33
This commit makes it so the topic footer button for bookmarks
uses the new BookmarkMenu component, and makes some tweaks to
that component to allow for a label and CSS class options.
Also introduces a TopicBookmarkManager to manage the saving/editing/
deleting of the topic level bookmarks and the reactivity that happens
in the topic UI afterward.
Next commit should rip out old bookmark associated code in the
topic controller as it will no longer be needed.
---------
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
For e-mails, secure uploads redacts all secure images, and later uses the access control post to re-attached allowed ones. We pass the ID of this post through the X-Discourse-Post-Id header. As the name suggests, this assumes there's only ever one access control post. This is not true for activity summary e-mails, as they summarize across posts.
This adds a new header, X-Discourse-Post-Ids, which is used the same way as the old header, but also works for the case where an e-mail is associated with multiple posts.
Automatically add `moderators` and `admins` auto groups to specific site settings.
In the new group-based permissions systems, we just want to check the user’s groups since it more accurately reflects reality
Affected settings:
- tag_topic_allowed_groups
- create_tag_allowed_groups
- send_email_messages_allowed_groups
- personal_message_enabled_groups
- here_mention_allowed_groups
- approve_unless_allowed_groups
- approve_new_topics_unless_allowed_groups
- skip_review_media_groups
- email_in_allowed_groups
- create_topic_allowed_groups
- edit_wiki_post_allowed_groups
- edit_post_allowed_groups
- self_wiki_allowed_groups
- flag_post_allowed_groups
- post_links_allowed_groups
- embedded_media_post_allowed_groups
- profile_background_allowed_groups
- user_card_background_allowed_groups
- invite_allowed_groups
- ignore_allowed_groups
- user_api_key_allowed_groups
Full text search does not return ideal results for category dropdown.
Usually, in category dropdowns we want to search for categories as we
type. For example, while typing "theme", the dropdown should show
intermediary results for "t", "th", "the", "them" and finally "theme".
For some of these substrings (like "the"), full text search does not
return any results, which leads to an unpleasant user experience.
To add a components link to the sidebar refactoring was required to create unique URLs for themes and components. Before the query param was used. After changes, we have two URLs `/admin/customize/themes` and `/admin/customize/components`.
This commit will now change two behaviors:
- If composer is already opened on a specific post and we click on edit again for the same post, we will do nothing and not show the discard draft modal
- if composer is shrinked and we click on edit for the same currently edited post, we will just open the composer and not show the discard draft modal
This commit addresses an issue for sites where secure_uploads
is turned on after the site has been operating without it for
some time.
When uploads are linked when they are used inside a post,
we were setting the access_control_post_id unconditionally
if it was NULL to that post ID and secure_uploads was true.
However this causes issues if an upload has been used in a
few different places, especially if a post was previously
used in a PM and marked secure, so we end up with a case of
the upload using a public post for its access control, which
causes URLs to not use the /secure-uploads/ path in the post,
breaking things like image uploads.
We should only set the access_control_post_id if the post is the first time the
upload is referenced so it cannot hijack uploads from other places.
This fixes a timing issue where, if a user (or the CI) was
on a slow network connection, clicking one of the bookmark
menu options would cause an error because we hadn't yet received
the response from the server after creating the bookmark.
It should be very smooth most of the times because (paraphrasing j.jaffeux):
a) Most likely when user clicks it’s already saved
b) If it’s not saved when user clicks, it should already be almost done so
the perceived wait when click the reminder option should be rather short
This is to enable :array type attributes for Contract
attributes in services, this is a followup to the move
of services from chat to core here:
cab178a405
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
The bulk actions menu for topics has multiple options to work
with tags on topics (append, replace, remove). Our tagging system
along with categories allows for some complicated tag restrictions
to be applied via tag groups. This was a problem for the topic bulk
actions because you couldn't append restricted tags to topics.
This commit allows restricted tags to be used in bulk tagging actions
as long as all selected topics are for a sole category. The category
information will be shown in the modal, and the category ID is used
for the tag search.
Previously, when the new site was created and after the first admin login, no one will receive notifications to review the user approval queue since only the moderators would receive the PMs about it. Also, this PR will change the "pending_users_reminder_delay_minutes" site setting to 5 minutes while the site is in bootstrap mode.
When lazy load categories is enabled, categories should be loaded with
user activity items and drafts because the categories may not be
preloaded on the client side.
This will automatically enable the glimmer header when all installed themes/plugins are ready. This replaces the old group-based site setting.
In 'auto' mode, we check for calls to deprecated APIs (e.g. decorateWidget) which affect the old header. If any are present, we stick to the old header implementation and print a message to the console alongside the normal deprecation messages.
To override this automatic behavior, a new `glimmer_header_mode` site setting can be set to 'disabled' or 'enabled'.
This change also means that our test suite is running with the glimmer header. This unveiled a couple of small issues (e.g. some incorrect `aria-*` and `alt` text) which are now fixed. A number of selectors had to be updated to ensure the tests were clicking the actual `<button>` elements rather than the surrounding `<li>` elements.
When choosing the "Custom..." option in the new bookmark
menu and then choosing a date + time in the modal for the
reminder, the bookmark icon on the post was not updating to
show the one with the clock to indicate the reminder.
This was just a data syncing issue between BookmarkFormData
and what the modal sets. Ideally all this would be refactored
because the data flow is messy...but hard to find time for
that right now.
Followup 67a8080e33
This commit changes enum typed theme objects property to be optional.
Previously, an enum typed property is always required but we have found
that this might not be ideal so we want to change it.
This method name is a bit confusing; with_secure_uploads implies
it may return a block or something with the uploads of the post,
and has_secure_uploads implies that it's checking whether the post
is linked to any secure uploads.
should_secure_uploads? communicates the true intent of this method --
which is to say whether uploads attached to this post should be
secure or not.
Display additional confirmation when:
- The public section is going to be updated;
- The public section is going to be deleted;
- The public section is going to be marked as private.
This commit adds a new option `@modalForMobile` for `<DMenu />` which allows to display a `<DModal />` when expanding a menu on mobile.
This commit also adds a `@views` options to toasts which is an array accepting `['mobile', 'desktop']` and will control if the toast is show on desktop and/or mobile.
Finally this commit allows to hide the progressBar even if the toast is set to `@autoClose=true`. This is controlled through the `@showProgressBar` option.
* DEV: Add `topic_embed_import_create_args` plugin modifier
This modifier allows a plugin to change the arguments used when creating
a new topic for an imported article.
For example: let's say you want to prepend "Imported: " to the title of
every imported topic. You could use this modifier like so:
```ruby
# In your plugin's code
plugin.register_modifier(:topic_embed_import_create_args) do |args|
args[:title] = "Imported: #{args[:title]}"
args
end
```
In this example, the modifier is prepending "Imported: " to the `title` in the `create_args` hash. This modified title would then be used when the new topic is created.
Adds the new quick menu for bookmarking. When you bookmark
a post (chat message behaviour will come later) we show this new quick
menu and bookmark the item straight away.
You can then choose a reminder quick option, or choose Custom... to open
the old modal. If you click on an existing bookmark, we show the same quick menu
but with Edit and Delete options.
A later PR will introduce a new bookmark modal, but for now we
are using the old modal for Edit and Custom... options.
Using around_action means `add_early_hint_header` is in the stack for every request, and gets included in the backtrace of any errors.
We can manage with an after_action instead, which avoids adding to the stack depth (and avoids people blaming me for unrelated application errors 😉)
This PR improves the performance of the `most_replied_to_users` method on the `UserSummary` model.
### Old Query
```ruby
post_query
.joins(
"JOIN posts replies ON posts.topic_id = replies.topic_id AND posts.reply_to_post_number = replies.post_number",
)
# We are removing replies by @user, but we can simplify this by getting the using the user_id on the posts.
.where("replies.user_id <> ?", @user.id)
.group("replies.user_id")
.order("COUNT(*) DESC")
.limit(MAX_SUMMARY_RESULTS)
.pluck("replies.user_id, COUNT(*)")
.each { |r| replied_users[r[0]] = r[1] }
```
### Old Query with corrections
```ruby
post_query
.joins(
"JOIN posts replies ON posts.topic_id = replies.topic_id AND replies.reply_to_post_number = posts.post_number",
)
# Remove replies by @user but instead look on loaded posts (we do this so we don't count self replies)
.where("replies.user_id <> posts.user_id")
.group("replies.user_id")
.order("COUNT(*) DESC")
.limit(MAX_SUMMARY_RESULTS)
.pluck("replies.user_id, COUNT(*)")
.each { |r| replied_users[r[0]] = r[1] }
```
### New Query
```ruby
post_query
.joins(
"JOIN posts replies ON posts.topic_id = replies.topic_id AND posts.reply_to_post_number = replies.post_number",
)
# Only include regular posts in our joins, this makes sure we don't have the bloat of loading private messages
.joins(
"JOIN topics ON replies.topic_id = topics.id AND topics.archetype <> 'private_message'",
)
# Only include visible post types, so exclude posts like whispers, etc
.joins(
"AND replies.post_type IN (#{Topic.visible_post_types(@user, include_moderator_actions: false).join(",")})",
)
.where("replies.user_id <> posts.user_id")
.group("replies.user_id")
.order("COUNT(*) DESC")
.limit(MAX_SUMMARY_RESULTS)
.pluck("replies.user_id, COUNT(*)")
.each { |r| replied_users[r[0]] = r[1] }
```
# Conclusion
`most_replied_to_users` was untested, so I introduced a test for the logic, and have confirmed that it passes on both the new query **AND** the old query.
Thank you @danielwaterworth for the debugging assistance.
We will be collecting the logo URL and the site's default locale values along with existing basic details to display the site on the Discourse Discover listing page. It will be included only if the site is opted-in by enabling the "`include_in_discourse_discover`" site setting.
Also, we no longer going to use `about.json` and `site/statistics.json` endpoints retrieve these data. We will be using only the `site/basic-info.json` endpoint.
- Add a "Skip tips" button to first notification tip
- Add a "Skip tips" button to the admin guide tip
- Fixes the timeline tip showing when no timeline was present
- Fixes post menu tip showing when no "..." button is present
- Adds system tests
- Marks each tip as seen as soon as it is displayed so that refreshing,
clicking outside, etc. won't show it again
- Change just above means we no longer need a MessageBus track
Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
When a user is manually deactivated, they should not be deleted by our background job that purges inactive users.
In addition, site settings keywords should accept an array of keywords.
Daniel Waterworth
Régis Hanol
Ted Johansson
Krzysztof Kotlarek
Alan Guo Xiang Tan
David Taylor
Martin Brennan
Jan Cernik
Vinoth Kannan
Joffrey JAFFEUX
Bianca Nenciu
Osama Sayegh
Natalie Tay
Penar Musaraj
Gabriel Grubba
Sam
Andrei Prigorshnev
Blake Erickson
Sérgio Saquetim
Rafael dos Santos Silva
Gerhard Schlager
Isaac Janzen