Commit Graph

3198 Commits

Author SHA1 Message Date
Sam ebd3971533
FEATURE: experiment with hot sort order (#25274)
This introduces a new experimental hot sort ordering. 

It attempts to float top conversations by first prioritizing a  topics with lots of recent activity (likes and users responding) 

The schedule that updates hot topics is disabled unless the hidden site setting: `experimental_hot_topics` is enabled. 

You can control "decay" with `hot_topic_gravity` and `recency` with `hot_topics_recent_days` 

Data is stored in the new `topic_hot_scores` table and you can check it out on the `/hot` route once 
enabled. 
---------

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2024-01-17 13:01:04 +11:00
Alan Guo Xiang Tan 22614ca85b
DEV: Compile theme migrations javascript files when running theme qunit (#25219)
Why this change?

Currently, is it hard to iteratively write a theme settings migrations
because our theme migrations system does not rollback. Therefore, we
want to allow theme developers to be able to write QUnit tests for their
theme migrations files enabling them to iteratively write their theme
migrations.

What does this change do?

1. Update `Theme#baked_js_tests_with_digest` to include all `ThemeField`
records of `ThemeField#target` equal to `migrations`. Note that we do
not include the `settings` and `themePrefix` variables for migration files.

2. Don't minify JavaScript test files becasue it makes debugging in
   development hard.
2024-01-16 09:50:44 +08:00
Penar Musaraj f2cf5434f3
Revert "DEV: Convert min_trust_level_to_tag_topics to groups (#25258)" (#25262)
This reverts commit c7e3d27624 due to
test failures. This is temporary.
2024-01-15 11:33:47 -05:00
Ted Johansson c7e3d27624
DEV: Convert min_trust_level_to_tag_topics to groups (#25258)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_tag_topics site setting to tag_topic_allowed_groups.
2024-01-15 20:59:08 +08:00
Renato Atilio f5f3742166
FIX: respect creation date when paginating group activity posts (#24993)
* FIX: respect creation date when paginating group activity posts

There are scenarios where the chronological order of posts doesn't match the order of their IDs. For instance, when moving the first post from one topic or PM to another, a new post (with a higher ID) will be created, but it will retain the original creation time.

This PR changes the group activity page and endpoint to paginate posts using created_at instead of relying on ID ordering.
2024-01-11 13:37:27 -03:00
Alan Guo Xiang Tan 59839e428f
DEV: Add `skip_migrations` param when importing remote theme (#25218)
Why this change?

Importing theme with the `bundle` params is used mainly by
`discourse_theme` CLI in the development environment. However, we do not
want migrations to automatically run in the development environment
and instead want the developer to be intentional about running theme
migrations. As such, this commit adds support for a
`skip_migrations` param when importing a theme with the `bundle` params.

This commit also adds a `migrated` attribute for migrations theme fields
to indicate whether a migrations theme field has been migrated or not.
2024-01-11 14:04:02 +08:00
Blake Erickson 6ebe61ecec
FIX: Logs api scope not working (#25215) 2024-01-10 19:30:10 -07:00
Rafael dos Santos Silva 13735f35fb
FEATURE: Cache embed contents in the database (#25133)
* FEATURE: Cache embed contents in the database

This will be useful for features that rely on the semantic content of topics, like the many AI features



Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2024-01-05 10:09:31 -03:00
Ted Johansson a5f0935307
DEV: Convert min_trust_level_to_create_tag to groups (#24899)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_create_tag  site setting to create_tag_allowed_groups.

This PR maintains backwards compatibility until we can update plugins and themes using this.
2024-01-05 10:19:43 +08:00
Jarek Radosz b9f6e6d637
DEV: Skip flaky specs (#25111)
Dynamically adding and removing deprecated or TL-related site settings in specs is leaky
2024-01-03 12:32:26 +01:00
Martin Brennan e8deed874b
FIX: Do not allow setting admin and staff for TrustLevelSetting (#25107)
This fixes an issue where any string for an enum site setting
(such as TrustLevelSetting) would be converted to an integer
if the default value for the enum was an integer. This is an
issue because things like "admin" and "staff" would get silently
converted to 0 which is "valid" because it's TrustLevel[0],
but it's unexpected behaviour. It's best to just let the site
setting validator catch this broken value.
2024-01-03 16:55:28 +10:00
Natalie Tay b4f36507e0
FIX: Allow the flags to be cleaned up (#25085)
Currently, the reviewable queue includes ReviewableFlaggedPost with posts that have already been hidden. This allows for such hidden posts to be cleared up by the auto-tool.
2024-01-02 18:32:50 +08:00
Bianca Nenciu 14269232ba
DEV: No longer preload categories (#24950)
Categories will no longer be preloaded when `lazy_load_categories` is
enabled through PreloadStore.

Instead, the list of site categories will continue to be populated
by `Site.updateCategory` as more and more categories are being loaded
from different sources (topic lists, category selectors, etc).
2023-12-28 14:36:33 +02:00
Martin Brennan 89705be722
DEV: Add auto map from TL -> group site settings in DeprecatedSettings (#24959)
When setting an old TL based site setting in the console e.g.:

SiteSetting.min_trust_level_to_allow_ignore = TrustLevel[3]

We will silently convert this to the corresponding Group::AUTO_GROUP. And vice-versa, when we read the value on the old setting, we will automatically get the lowest trust level corresponding to the lowest auto group for the new setting in the database.
2023-12-26 14:39:18 +08:00
Bianca Nenciu 680cf443f4
FIX: Better infinite scrolling on categories page (#24831)
This commit refactor CategoryList to remove usage of EmberObject,
hopefully make the code more readable and fixes various edge cases with
lazy loaded categories (third level subcategories not being visible,
subcategories not being visible on category page, requesting for more
pages even if the last one did not return any results, etc).

The problems have always been here, but were not visible because a lot
of the processing was handled by the server and then the result was
serialized. With more of these being moved to the client side for the
lazy category loading, the problems became more obvious.
2023-12-18 16:46:09 +02:00
Ted Johansson 6ab1a19e93
DEV: Convert min_trust_level_to_allow_invite to groups (#24893)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_allow_invite  site setting to invite_allowed_groups.

Nothing much of note. This is used in one place and there's no fallout.
2023-12-18 12:07:36 +08:00
Krzysztof Kotlarek 1f72152e47
DEV: Remove usage of `min_trust_to_create_topic` SiteSetting (#24887)
Using min_trust_to_create_topic and create_topic_allowed_groups together was part of #24740

Now, when plugins specs are fixed, we can safely remove that part of logic.
2023-12-18 13:39:53 +11:00
Martin Brennan 6de00f89c2
FEATURE: Initial admin sidebar navigation (#24789)
This is v0 of admin sidebar navigation, which moves
all of the top-level admin nav from the top of the page
into a sidebar. This is hidden behind a enable_admin_sidebar_navigation
site setting, and is opt-in for now.

This sidebar is dynamically shown whenever the user enters an
admin route in the UI, and is hidden and replaced with either
the:

* Main forum sidebar
* Chat sidebar

Depending on where they navigate to. For now, custom sections
are not supported in the admin sidebar.

This commit removes the experimental admin sidebar generation rake
task but keeps the experimental sidebar UI for now for further
testing; it just uses the real nav as the default now.
2023-12-18 11:48:25 +10:00
Ted Johansson 294febf3c4
DEV: Convert min_trust_to_flag_posts setting to groups (#24864)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_to_flag_posts site setting to flag_post_allowed_groups.

Note: In the original setting, "posts" is plural. I have changed this to "post" singular in the new setting to match others.
2023-12-13 17:18:42 +08:00
Krzysztof Kotlarek 702d0620d7
DEV: Convert min_trust_to_create_topic to groups (#24740)
This change converts the min_trust_to_create_topic site setting to
create_topic_allowed_groups.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change
- After a couple of months, we will remove the min_trust_to_create_topicsetting entirely.

Internal ref: /t/117248
2023-12-13 14:50:13 +11:00
Bianca Nenciu dcd81d56c0
FIX: category selectors for lazy loaded categories (#24533)
A lot of work has been put in the select kits used for selecting
categories: CategorySelector, CategoryChooser, CategoryDrop, however
they still do not work as expected when these selectors already have
values set, because the category were still looked up in the list of
categories stored on the client-side Categrories.list().

This PR fixes that by looking up the categories when the selector is
initialized. This required altering the /categories/find.json endpoint
to accept a list of IDs that need to be looked up. The API is called
using Category.asyncFindByIds on the client-side.

CategorySelector was also updated to receive a list of category IDs as
attribute, instead of the list of categories, because the list of
categories may have not been loaded.

During this development, I noticed that SiteCategorySerializer did not
serializer all fields (such as permission and notification_level)
which are not a property of category, but a property of the relationship
between users and categories. To make this more efficient, the
preload_user_fields! method was implemented that can be used to
preload these attributes for a user and a list of categories.
2023-12-08 12:01:08 +02:00
Martin Brennan d9a422cf61
FIX: Do not attempt S3 ACL call if secure status did not change (#24785)
When rebaking and in various other places for posts, we
run through the uploads and call `update_secure_status` on
each of them.

However, if the secure status didn't change, we were still
calling S3 to change the ACL, which would have been a noop
in many cases and takes ~1 second per call, slowing things
down a lot.

Also, we didn't account for the s3_acls_enabled site setting
being false here, and in the specs doing an assertion
that `Discourse.store.update_ACL` is not called doesn't
work; `Discourse.store` isn't a singleton, it re-initializes
`FileStore::S3Store.new` every single time.
2023-12-08 12:58:45 +10:00
Mark VanLandingham ee05f57e2d
FEATURE: Site setting to display user avatars in user menu (#24514) 2023-12-07 11:30:44 -06:00
Martin Brennan 7afb5fc481
DEV: Use Discourse::SYSTEM_USER_ID in fixtures/009_users (#24743)
I couldn't find where we created the system user and
this is why -- everywhere else in the app we reference
SYSTEM_USER_ID but here.
2023-12-07 09:04:45 +10:00
Jarek Radosz 694b5f108b
DEV: Fix various rubocop lints (#24749)
These (21 + 3 from previous PRs) are soon to be enabled in rubocop-discourse:

Capybara/VisibilityMatcher
Lint/DeprecatedOpenSSLConstant
Lint/DisjunctiveAssignmentInConstructor
Lint/EmptyConditionalBody
Lint/EmptyEnsure
Lint/LiteralInInterpolation
Lint/NonLocalExitFromIterator
Lint/ParenthesesAsGroupedExpression
Lint/RedundantCopDisableDirective
Lint/RedundantRequireStatement
Lint/RedundantSafeNavigation
Lint/RedundantStringCoercion
Lint/RedundantWithIndex
Lint/RedundantWithObject
Lint/SafeNavigationChain
Lint/SafeNavigationConsistency
Lint/SelfAssignment
Lint/UnreachableCode
Lint/UselessMethodDefinition
Lint/Void

Previous PRs:
Lint/ShadowedArgument
Lint/DuplicateMethods
Lint/BooleanSymbol
RSpec/SpecFilePathSuffix
2023-12-06 23:25:00 +01:00
Jarek Radosz 4280c01153
DEV: Fix Lint/ShadowedArgument (#24733) 2023-12-06 13:16:10 +01:00
Roman Rizzi 037622fc7f
DEV: reviewable_user spec should assert on delete_user_block instead of delete_user (#24692) 2023-12-04 12:44:32 -03:00
Jarek Radosz 7196613e2e
DEV: Fix various spec linting issues (#24672)
Duplicated specs, incorrect descriptions, incorrect assertions, incorrect filenames, old todo
2023-12-04 13:45:19 +01:00
Jarek Radosz 74011232e9
FIX: Request html when fetching inline onebox data (#24674)
We do expect to receive html
2023-12-04 11:36:42 +10:00
Alan Guo Xiang Tan 50bafd48cd
FIX: Flaky spec due to incorrect Rack response body (#24640)
Why was the problem?

ActiveRecord's query cache for the connection pool wasn't disabled after the
`with a fake provider runs 'other_phase' for enabled auth methods` test
in `omniauth_callbacks_controller_spec.rb` was run. This was because the
Rack response body in `FakeAuthenticator::Strategy::other_phase` did not
adhere to the expected Rack body format which is "typically an Array of
String instances". Because this expectation was broken, it cascaded the
problem down where it resulted in the ActiveRecord's query cache for the
connection pool not being disabled as it normally should when the
response body is closed.

When the query cache is left enabled, common assertions pattern in RSpec
like `expect { something }.to change { Group.count }` will fail since
the query cache is enabled and the call first call to `Group.count` will
cache the result to be reused later on.

To see the bug in action, one can run the following command:

`bundle exec rspec --seed 44747
spec/requests/omniauth_callbacks_controller_spec.rb:1150
spec/models/group_spec.rb:283`
2023-11-30 10:49:55 +08:00
Sam b09422428d
DEV: update syntax tree to latest (#24623)
update format to latest syntax tree
2023-11-29 16:38:07 +11:00
Bianca Nenciu e85a81f33c
FIX: Make category-drop work with lazy_load_categories (#24187)
The category drop was rerendered after every category async change
because it updated the categories list. This is not necessary and
categories can be referenced indirectly by ID instead.
2023-11-28 17:58:47 +02:00
Martin Brennan 1fc0ce1ac2
FIX: with_secure_uploads? could return nil in some cases (#24592)
When we check upload security, one of the checks is to
run `access_control_post.with_secure_uploads?`. The problem
here is that the `topic` for the post could be deleted,
which would make the check return `nil` sometimes instead
of false because of safe navigation. We just need to be
more explicit.
2023-11-28 13:12:28 +10:00
Martin Brennan 91232847e3
FIX: Video placeholders not auto-linking post uploads (#24559)
Followup to 2443446e62

We introduced video placeholders which prevent preloading
metadata for videos in posts. The structure looks like this
in HTML when the post is cooked:

```
<div class="video-placeholder-container" data-video-src="http://some-url.com/video.mp4" dir="ltr" style="cursor: pointer;">
  <div class="video-placeholder-wrapper">
    <div class="video-placeholder-overlay">
      <svg class="fa d-icon d-icon-play svg-icon svg-string" xmlns="http://www.w3.org/2000/svg">
        <use href="#play"></use>
      </svg>
    </div>
  </div>
</div>
```

However, we did not update the code that links post uploads
to the post via UploadReference, so any videos uploaded since
this change are essentially dangling and liable to be deleted.
This also causes some uploads to be marked secure when they
shouldn't be, because they are not picked up and analysed in the
CookedPostProcessor flow.
2023-11-27 12:38:52 +10:00
Sam 85d2b5fa48
DEV: maxmind license checking failing tests (#24534)
This improves the implementation of #18993

1. Error message displayed to user is clearer
2. open_db will also be called, even if license key is blank, as it was previously
3. This in turn means no need to keep stubbing 'maxmind_license_key'
2023-11-24 09:38:46 +11:00
MichaIng c58a41cb3e
UX: Show on IP lookup if MaxMind key is missing (#18993)
as discussed in https://meta.discourse.org/t/maxminddb-not-found-error/148512/7.
 
shows a warning to the admin if no license for maxmind is found
2023-11-24 08:02:05 +11:00
Alan Guo Xiang Tan 1510e1d1ea
DEV: Fix flaky test due to ActiveRecord query caching (#24476)
Why this change?

The test was randomly failing in
https://github.com/discourse/discourse/actions/runs/6936264158/job/18868087113
with the following failure:

```
expect do user.update_ip_address!("127.0.0.1") end.to change {
  UserIpAddressHistory.where(user_id: user.id).count
}.by(1)

expected `UserIpAddressHistory.where(user_id: user.id).count` to have changed by 1, but was changed by 0
```

This is due to the fact that ActiveRecord will actually cache the result
of `UserIpAddressHistory.where(user_id: user.id).count`. However,
`User.update_ip_address!` relies on mini_sql and does not go through
ActiveRecord. As a result, the query cache is not cleared and hence the
flakiness.

What does this change do?

This change uses the `uncached` method provided by ActiveRecord when
we are fetching the count.
2023-11-21 17:03:19 +11:00
David Taylor eda79186ee
FIX: Recompile theme translations when fallback data changes (#24371)
Previously we would only recompile a theme locale when its own data changes. However, the output also includes fallback data from other locales, so we need to invalidate all locales when fallback locale data is changed. Building a list of dependent locales is tricky, so let's just invalidate them all.
2023-11-14 19:53:27 +00:00
Kris 797da5870b
FEATURE: remove category badge style options, set bullet style as default (#24198) 2023-11-13 10:46:15 -05:00
David Battersby 4b78254065
FIX: Post moved small action links should respect subfolder installs (#24336)
This change fixes an issue with small action links (when post is moved) to add the subfolder path to the url.
2023-11-13 11:06:25 +08:00
Daniel Waterworth 6e161d3e75
DEV: Allow fab! without block (#24314)
The most common thing that we do with fab! is:

    fab!(:thing) { Fabricate(:thing) }

This commit adds a shorthand for this which is just simply:

    fab!(:thing)

i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-09 16:47:59 -06:00
Andrei Prigorshnev d91456fd53
DEV: Ability to collect stats without exposing them via API (#23933)
This adds the ability to collect stats without exposing them 
among other stats via API.

The most important thing I wanted to achieve is to provide 
an API where stats are not exposed by default, and a developer 
has to explicitly specify that they should be 
exposed (`expose_via_api: true`). Implementing an opposite 
solution would be simpler, but that's less safe in terms of 
potential security issues. 

When working on this, I had to refactor the current solution. 
I would go even further with the refactoring, but the next steps 
seem to be going too far in changing the solution we have, 
and that would also take more time. Two things that can be 
improved in the future:
1. Data structures for holding stats can be further improved
2. Core stats are hard-coded in the About template (it's hard 
to fix it without correcting data structures first, see point 1):
    63a0700d45/app/views/about/index.html.erb (L61-L101)

The most significant refactorings are:
1. Introducing the `Stat` model
2. Aligning the way the core and the plugin stats' are registered
2023-11-10 00:44:05 +04:00
Bianca Nenciu bdb81b5346
DEV: Use a single registry for preloaded category custom fields (#24272)
There was a registry for preloaded site categories and a new one has
been introduced recently for categories serialized through a
CategoryList.

Having two registries created a lot of friction for developers and this
commit merges them into a single one, providing a unified API.
2023-11-09 18:23:24 +02:00
Krzysztof Kotlarek 5f20748e40 SECURITY: SSRF vulnerability in TopicEmbed
Block redirects when making the final request in TopicEmbed to prevent Server Side Request Forgery (SSRF)
2023-11-09 13:39:08 +11:00
Andrei Prigorshnev be2eb3df44
FIX: user got notified about a mention inside a chat message quote (#24229)
When quoting a chat message in a post, if that message contains a mention, 
that mention should be ignored. But we've been detecting them and sending 
notifications to users. This PR fixes the problem. Since this fix is for 
the chat plugin, I had to introduce a new API for plugins:

    # We strip posts before detecting mentions, oneboxes, attachments etc. 
    # We strip those elements that shouldn't be detected. For example, 
    # a mention inside a quote should be ignored, so we strip it off. 
    # Using this API plugins can register their own post strippers. 
    def register_post_stripper(&block) 
    end
2023-11-08 23:13:25 +04:00
Natalie Tay 2eb3f9fd44
DEV: Add an assertion for event triggered (#24259) 2023-11-07 11:59:40 +08:00
Ted Johansson 47e58357b6
DEV: Parallel scheduled admin checks (#24190)
This PR does some preparatory refactoring of scheduled admin checks in order for us to be able to do custom retry strategies for some of them.

Instead of running all checks in sequence inside a single, scheduled job, the scheduled job spawns one new job per check.

In order to be concurrency-safe, we need to change the existing Redis data structure from a string (of serialized JSON) to a list of strings (of serialized JSON).
2023-11-03 09:05:29 +08:00
Osama Sayegh 3cadd6769e
FEATURE: Theme settings migrations (#24071)
This commit introduces a new feature that allows theme developers to manage the transformation of theme settings over time. Similar to Rails migrations, the theme settings migration system enables developers to write and execute migrations for theme settings, ensuring a smooth transition when changes are required in the format or structure of setting values.

Example use cases for the theme settings migration system:

1. Renaming a theme setting.

2. Changing the data type of a theme setting (e.g., transforming a string setting containing comma-separated values into a proper list setting).

3. Altering the format of data stored in a theme setting.

All of these use cases and more are now possible while preserving theme setting values for sites that have already modified their theme settings.

Usage:

1. Create a top-level directory called `migrations` in your theme/component, and then within the `migrations` directory create another directory called `settings`.

2. Inside the `migrations/settings` directory, create a JavaScript file using the format `XXXX-some-name.js`, where `XXXX` is a unique 4-digit number, and `some-name` is a descriptor of your choice that describes the migration.

3. Within the JavaScript file, define and export (as the default) a function called `migrate`. This function will receive a `Map` object and must also return a `Map` object (it's acceptable to return the same `Map` object that the function received).

4. The `Map` object received by the `migrate` function will include settings that have been overridden or changed by site administrators. Settings that have never been changed from the default will not be included.

5. The keys and values contained in the `Map` object that the `migrate` function returns will replace all the currently changed settings of the theme.

6. Migrations are executed in numerical order based on the XXXX segment in the migration filenames. For instance, `0001-some-migration.js` will be executed before `0002-another-migration.js`.

Here's a complete example migration script that renames a setting from `setting_with_old_name` to `setting_with_new_name`:

```js
// File name: 0001-rename-setting.js

export default function migrate(settings) {
  if (settings.has("setting_with_old_name")) {
    settings.set("setting_with_new_name", settings.get("setting_with_old_name"));
  }
  return settings;
}
```

Internal topic: t/109980
2023-11-02 08:10:15 +03:00
Martin Brennan 219b071994
FIX: Revise and reject post breaks on new topics queued (#24109)
Followup to 9762e65758. This
original commit did not take into account the fact that
new topics can end up in the approval queue as a
ReviewableQueuedPost, and so there was a 500 error raised
when accessing `self.topic` when sending a PM to the user.
2023-10-27 13:05:41 +10:00
Angus McLeod 2a75656ff2
DEV: Add category custom field preloading to CategoryList (#23969)
This commit also introduced a plugin API for preloading category custom
fields.
2023-10-26 16:34:23 +03:00
Alan Guo Xiang Tan f2a90afa4c
DEV: Introduce `Theme#get_setting` (#24032)
Why this change?

Currently, we do not have a method to easily retrieve a theme setting's
value on the server side. Such a method can be useful in the test
environment where we need to retrieve the theme's setting and use its
value in assertions.

What does this change do?

This change introduces the `Theme#get_setting` instance method.
2023-10-23 07:41:40 +08:00
Sérgio Saquetim 0cfc42e0e6
FEATURE: Add dark mode option for category backgrounds (#24003)
Adds a new upload field for a dark mode category background that will be used as an alternative when Discourse is using a dark mode theme.
2023-10-20 12:48:06 +00:00
David Battersby 75441e063a
DEV: create new_post_moved event trigger when moving posts (#24005)
This change adds a new event trigger (new_post_moved) when the first post in a topic is moved to a new topic.

Plugins that listen for the new_post_moved event now have an easy way to update old data based on the post id.
2023-10-20 17:56:50 +08:00
Martin Brennan 9ef3a18ce4
DEV: Add new experimental admin UI route and sidebar (#23952)
This commit adds a new admin UI under the route `/admin-revamp`, which is
only accessible if the user is in a group defined by the new `enable_experimental_admin_ui_groups` site setting. It
also adds a special `admin` sidebar panel that is shown instead of the `main`
forum one when the admin is in this area.

![image](https://github.com/discourse/discourse/assets/920448/fa0f25e1-e178-4d94-aa5f-472fd3efd787)

We also add an "Admin Revamp" sidebar link to the community section, which
will only appear if the user is in the setting group:

![image](https://github.com/discourse/discourse/assets/920448/ec05ca8b-5a54-442b-ba89-6af35695c104)

Within this there are subroutes defined like `/admin-revamp/config/:area`,
these areas could contain any UI imaginable, this is just laying down an
initial idea of the structure and how the sidebar will work. Sidebar links are
currently hardcoded.

Some other changes:

* Changed the `main` and `chat` panels sidebar panel keys to use exported const values for reuse
* Allowed custom sidebar sections to hide their headers with the `hideSectionHeader` option
* Add a `groupSettingArray` setting on `this.siteSettings` in JS, which accepts a group site setting name
  and splits it by `|` then converts the items in the array to integers, similar to the `_map` magic for ruby
  group site settings
* Adds a `hidden` option for sidebar panels which prevents them from showing in separated mode and prevents
  the switch button from being shown

---------

Co-authored-by: Krzysztof Kotlarek <kotlarek.krzysztof@gmail.com>
2023-10-19 14:23:41 +10:00
Martin Brennan 5dc45b5dcf
FIX: Secure upload post processing race condition (#23968)
* FIX: Secure upload post processing race condition

This commit fixes a couple of issues.

A little background -- when uploads are created in the composer
for posts, regardless of whether the upload will eventually be
marked secure or not, if secure_uploads is enabled we always mark
the upload secure at first. This is so the upload is by default
protected, regardless of post type (regular or PM) or category.

This was causing issues in some rare occasions though because
of the order of operations of our post creation and processing
pipeline. When creating a post, we enqueue a sidekiq job to
post-process the post which does various things including
converting images to lightboxes. We were also enqueuing a job
to update the secure status for all uploads in that post.

Sometimes the secure status job would run before the post process
job, marking uploads as _not secure_ in the background and changing
their ACL before the post processor ran, which meant the users
would see a broken image in their posts. This commit fixes that issue
by always running the upload security changes inline _within_ the
cooked_post_processor job.

The other issue was that the lightbox wrapper link for images in
the post would end up with a URL like this:

```
href="/secure-uploads/original/2X/4/4e1f00a40b6c952198bbdacae383ba77932fc542.jpeg"
```

Since we weren't actually using the `upload.url` to pass to
`UrlHelper.cook_url` here, we weren't converting this href to the CDN
URL if the post was not in a secure context (the UrlHelper does not
know how to convert a secure-uploads URL to a CDN one). Now we
always end up with the correct lightbox href. This was less of an issue
than the other one, since the secure-uploads URL works even when the
upload has become non-secure, but it was a good inconsistency to fix
anyway.
2023-10-18 23:48:01 +00:00
Martin Brennan e91d8feab3
Revert "FEATURE: Count only approved flagged posts in user pages (#22799)" (#23962)
This reverts commit 5f0bc4557f.

Through extensive internal discussion we have decided to revert
this change, as it significantly impacted moderation flow for
some Discourse site moderators, especially around "something else"
flags. We need to re-approach how flags are counted holistically,
so to that end this change is being reverted.
2023-10-18 11:38:17 +10:00
Bianca Nenciu bf97899029
DEV: Limit preloaded categories (#23958)
Site data is preloaded on the first page load, which includes categories
data. For sites with many categories, site data takes a long time to
serialize and to transfer.

In the future, preloaded category data will be completely removed.
2023-10-17 22:04:56 +03:00
Blake Erickson 60ae69027c
DEV: Add category style deprecation check warning (#23951)
The category style site setting is being deprecated. This commit will
show a warning on the admin dashboard if a site isn't using the default
category style (bullet).
2023-10-17 10:40:31 -06:00
Martin Brennan 61c87fb59f
FIX: Properly attach secure images to email for non-secure uploads (#23865)
There are cases where a user can copy image markdown from a public
post (such as via the discourse-templates plugin) into a PM which
is then sent via an email. Since a PM is a secure context (via the
.with_secure_uploads? check on Post), the image will get a secure
URL in the PM post even though the backing upload is not secure.

This fixes the bug in that case where the image would be stripped
from the email (since it had a /secure-uploads/ URL) but not re-attached
further down the line using the secure_uploads_allow_embed_images_in_emails
setting because the upload itself was not secure.

The flow in Email::Sender for doing this is still not ideal, but
there are chicken and egg problems around when to strip the images,
how to fit in with other attachments and email size limits, and
when to apply the images inline via Email::Styles. It's convoluted,
but at least this fixes the Template use case for now.
2023-10-17 14:08:21 +10:00
Krzysztof Kotlarek 09eca87c76
FIX: synonym tags are not considered as unused (#23950)
Currently, `Tag.unused` scope is used to delete unused tags on `/tags` and by CleanUpTags job. Synonym tags, should not be included and treated as unused. Synonyms are only deleted when main tag is deleted:

https://github.com/discourse/discourse/blob/main/app/models/tag.rb#L57
2023-10-16 23:53:02 +00:00
Alan Guo Xiang Tan 4cb7472376
SECURITY: Prevent arbitrary topic custom fields from being set
Why this change?

The `PostsController#create` action allows arbitrary topic custom fields
to be set by any user that can create a topic. Without any restrictions,
this opens us up to potential security issues where plugins may be using
topic custom fields in security sensitive areas.

What does this change do?

1. This change introduces the `register_editable_topic_custom_field` plugin
API which allows plugins to register topic custom fields that are
editable either by staff users only or all users. The registered
editable topic custom fields are stored in `DiscoursePluginRegistry` and
is called by a new method `Topic#editable_custom_fields` which is then
used in the `PostsController#create` controller action. When an unpermitted custom fields is present in the `meta_data` params,
a 400 response code is returned.

2. Removes all reference to `meta_data` on a topic as it is confusing
   since we actually mean topic custom fields instead.
2023-10-16 10:34:35 -04:00
Martin Brennan 9762e65758
FEATURE: Add Revise... option for queued post reviewable (#23454)
This commit adds a new Revise... action that can be taken
for queued post reviewables. This will open a modal where
the user can select a Reason from a preconfigured list
(or by choosing Other..., a custom reason) and provide feedback
to the user about their post.

The post will be rejected still, but a PM will also be sent to
the user so they have an opportunity to improve their post when
they resubmit it.
2023-10-13 11:28:31 +10:00
Martin Brennan 542f77181a
FIX: Update upload security on post rebake from UI (#23861)
When a user creates or edits a post, we already were updating
the security of uploads in the post based on site settings and
their access control post, which is important since these uploads
may be switched from secure/not secure based on configuration.
The `with_secure_uploads?` method on a post is used to determine
whether to use the secure-uploads URL for all uploads in the post,
regardless of their individual security, so if this is false and
some of the posts are still secure when rebaking, we end up with
broken URLs.

This commit just makes it so rebaking via the UI also re-evaluates
upload security so that when the post is loaded again after processing,
all of the uploads have the correct security.
2023-10-10 11:15:51 +10:00
Krzysztof Kotlarek c468110929
FEATURE: granular webhooks (#23070)
Before this change, webhooks could be only configured for specific groups like for example, all topic events.

We would like to have more granular control like for example topic_created or topic_destroyed.

Test are failing because plugins changed has to be merged as well:
discourse/discourse-assign#498
discourse/discourse-solved#248
discourse/discourse-topic-voting#159
2023-10-09 03:35:31 +00:00
Alan Guo Xiang Tan 832b3b9e60
FEATURE: Remove support for legacy navigation menu (#23752)
Why this change?

Back in May 17 2023 along with the release of Discourse 3.1, we announced
on meta that the legacy hamburger dropdown navigation menu is
deprecated and will be dropped in Discourse 3.2. This is the link to the announcement
on meta: https://meta.discourse.org/t/removing-the-legacy-hamburger-navigation-menu-option/265274

## What does this change do?

This change removes the `legacy` option from the `navigation_menu` site
setting and migrates existing sites on the `legacy` option to the
`header dropdown` option.

All references to the `legacy` option in code and tests have been
removed as well.
2023-10-09 07:24:10 +08:00
Arpit Jalan 3669723a86
FEATURE: allow filtering posts report by multiple categories (#23669) 2023-09-26 21:56:47 +05:30
Osama Sayegh 83621ccbe7
FIX: Parse the digest_suppress_tags setting correctly (#23623)
Meta topic: https://meta.discourse.org/t/suppress-these-tags-from-summary-emails-settings-is-not-working-in-preview-digest-email/279196?u=osama

Follow-up to 477a5dd371

The `digest_suppress_tags` setting is designed to be a list of pipe-delimited tag names, but the tag-based topic suppression logic assumes (incorrectly) that the setting contains pipe-delimited tag IDs. This mismatch in expectations led to the setting not working as expected.

This PR adds a step that converts the list of tag names in the setting to their corresponding IDs, which is then used to suppress topics tagged with those specific tags.
2023-09-18 10:45:43 +03:00
Renato Atilio d93c2cb3d2
FEATURE: site settings to revoke api keys older than a number of days (#23595)
* FEATURE: site settings to revoke api keys older than a number of days
2023-09-15 16:31:29 -03:00
Sérgio Saquetim e03dd76dc6
FEATURE: add outgoing web hooks for Chat messages 2023-09-13 17:31:42 -03:00
Penar Musaraj f6326d03f0
DEV: Bump max theme sprite size to 1MB (#23556)
There is one repo with very large sprites, and it's causing missing
icons on some instances.
2023-09-13 15:00:26 +10:00
Bianca Nenciu 6f782d8e45
SECURITY: Add limits for themes and theme assets
This commit adds limits to themes and theme components on the:

- file size of about.json and .discourse-compatibility
- file size of theme assets
- number of files in a theme
2023-09-12 15:31:31 -03:00
Daniel Waterworth 290306a932
SECURITY: Reduce maximum size of SVG sprite cache to prevent DoS
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-09-12 15:31:28 -03:00
Gerhard Schlager e3a2446874
SECURITY: Limit number of drafts per user and length of `draft_key`
The hidden site setting max_drafts_per_user defaults to 10_000 drafts per user.
The longest key should be "topic_<MAX_BIG_INT>" which is 25 characters.
2023-09-12 15:31:26 -03:00
OsamaSayegh c1b5faa5fd
SECURITY: Limit name field length of TOTP authenticators and security keys 2023-09-12 15:31:17 -03:00
Ted Johansson f08c6d2756
DEV: Switch over category settings to new table - Part 3 (#20657)
In #20135 we prevented invalid inputs from being accepted in category setting form fields on the front-end. We didn't do anything on the back-end at that time, because we were still discussing which path we wanted to take. Eventually we decided we want to move this to a new CategorySetting model.

This PR moves the require_topic_approval and require_reply_approval from custom fields to the new CategorySetting model.

This PR is nearly identical to #20580, which migrated num_auto_bump_daily, but since these are slightly more sensitive, they are moved after the previous one is verified.
2023-09-12 09:51:49 +08:00
Alan Guo Xiang Tan d2e4b32c87
DEV: Add support for uploading a theme from a directory in system tests (#23402)
Why this change?

Currently, we do not have an easy way to test themes and theme components
using Rails system tests. While we support QUnit acceptance tests for
themes and theme components, QUnit acceptance tests stubs out the server
and setting up the fixtures for server responses is difficult and can lead to a
frustrating experience. System tests on the other hand allow authors to
set up the test fixtures using our fabricator system which is much
easier to use.

What does this change do?

In order for us to allow authors to run system tests with their themes
installed, we are adding a `upload_theme` helper that is made available
when writing system tests. The `upload_theme` helper requires a single
`directory` parameter where `directory` is the directory of the theme
locally and returns a `Theme` record.
2023-09-12 07:38:47 +08:00
Ted Johansson e74560f062
FIX: Don't mix up action labels between different reviewables (#23365)
Currently, if the review queue has both a flagged post and a flagged chat message, one of the two will have some of the labels of their actions replaced by those of the other. In other words, the labels are getting mixed up. For example, a flagged chat message might show up with an action labelled "Delete post".

This is happening because when using bundles, we are sending along the actions in a separate part of the response, so they can be shared by many reviewables. The bundles then index into this bag of actions by their ID, which is something generic describing the server action, e.g. "agree_and_delete".

The problem here is the same action can have different labels depending on the type of reviewable. Now that the bag of actions contains multiple actions with the same ID, which one is chosen is arbitrary. I.e. it doesn't distinguish based on the type of the reviewable.

This change adds an additional field to the actions, server_action, which now contains what used to be the ID. Meanwhile, the ID has been turned into a concatenation of the reviewable type and the server action, e.g. post-agree_and_delete.

This still provides the upside of denormalizing the actions while allowing for different reviewable types to have different labels and descriptions.

At first I thought I would prepend the reviewable type to the ID, but this doesn't work well because the ID is used on the server-side to determine which actions are possible, and these need to be shared between different reviewables. Hence the introduction of server_action, which now serves that purpose.

I also thought about changing the way that the bundle indexes into the bag of actions, but this is happening through some EmberJS mechanism, so we don't own that code.
2023-09-06 10:57:30 +08:00
Martin Brennan c532f6eb3d
FEATURE: Secure uploads in PMs only (#23398)
This adds a new secure_uploads_pm_only site setting. When secure_uploads
is true with this setting, only uploads created in PMs will be marked
secure; no uploads in secure categories will be marked as secure, and
the login_required site setting has no bearing on upload security
either.

This is meant to be a stopgap solution to prevent secure uploads
in a single place (private messages) for sensitive admin data exports.
Ideally we would want a more comprehensive way of saying that certain
upload types get secured which is a hybrid/mixed mode secure uploads,
but for now this will do the trick.
2023-09-06 09:39:09 +10:00
Jarek Radosz c285f4133f
DEV: Enable ember-this-fallback in themes (#23384)
Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-05 11:16:12 +02:00
Arpit Jalan e5f3c26d20
FEATURE: add group filter for admin reports (#23381)
FEATURE: add group filter for admin reports

DEV: add plugin outlet for admin dashboard tabs
2023-09-05 11:17:18 +05:30
Joffrey JAFFEUX f1d8cd529e
Revert "Revert "PERF: Cache each theme field value once (#23192)" (#23354)" (#23356)
This reverts commit 9821ca9413.
2023-08-31 14:12:03 -05:00
Joffrey JAFFEUX 9821ca9413
Revert "PERF: Cache each theme field value once (#23192)" (#23354)
This reverts commit 82a56334a3.
2023-08-31 19:04:43 +02:00
Daniel Waterworth 82a56334a3
PERF: Cache each theme field value once (#23192)
Previously, theme fields from components would be cached for each of
their parent themes.
2023-08-31 11:24:02 -05:00
Renato Atilio 58b49bce41
FEATURE: support to initial values for form templates through /new-topic (#23313)
* FEATURE: adds support for initial values through /new-topic to form templates
2023-08-29 18:41:33 -03:00
Juan David Martínez Cubillos 477a5dd371
FEATURE: Digest suppression by tags (#23089)
* FEATURE: Digest suppression by tags

* fixed stree issues

* fixed code so untagged topics are not suppressed when suppressing certain tags
2023-08-18 14:28:20 -05:00
David Taylor 82b16f4f47
DEV: Do not manipulate theme module paths at build-time (#23148)
Manipulating theme module paths means that the paths you author are not the ones used at runtime. This can lead to some very unexpected behavior and potential module name clashes. It also meant that the refactor in 16c6ab8661 was unable to correctly match up theme connector js/templates.

While this could technically be a breaking change, I think it is reasonably safe because:

1. Themes are already forced to use relative paths when referencing their own modules (since they're namespaced based on the site-specific id). The only time this might be problematic is when theme tests reference modules in the theme's main `javascripts` directory

2. For things like components/services/controllers/etc. our custom Ember resolver works backwards from the end of the path, so adding `discourse/` in the middle will not affect resolution.
2023-08-18 18:15:23 +01:00
Ted Johansson 79e3d4e2bd
FIX: Don't run post validations when hiding post (#23139)
When hiding a post (essentially updating hidden, hidden_at, and hidden_reason_id) our callbacks are running the whole battery of post validations. This can cause the hiding to fail in a number of edge cases. The issue is similar to the one fixed in #11680, but applies to all post validations, none of which should apply when hiding a post.

After some code reading and discussion, none of the validations in PostValidator seem to be relevant when hiding posts, so instead of just skipping unique check, we skip all post validator checks.
2023-08-18 10:55:17 +08:00
Krzysztof Kotlarek f8cd1da92a
FIX: increase sidebar URL limit to 1000 (#23120)
Before this change, sidebar URL had a limit of 200 characters. In some cases it is not enough, therefore it was increased to 1000.
2023-08-17 14:46:24 +10:00
Andrei Prigorshnev f4e424d7d4
DEV: find_each in CSV exports (#22573)
So we have to order by calling `find_each(order: :desc)`.
Note that that will order rows by Id, not by `last_match_at`
as we tried before (though that didn't work).
2023-08-17 12:33:52 +10:00
marstall 77626c088e
UX: support links in tag descriptions (#22994)
* scrub non-a html tags from tag descriptions on create, strips all tags from tag description when displayed in tag hover

* test for tag description links

* UX: basic render-tag test

* UX: fix linting

* UX: fix linting

* fix broken tests

* Update spec/models/tag_spec.rb

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>

* UX: use has_sanitizable_fields instead of has_scrubbable_fields to ensafen tag.description

---------

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-08-16 11:43:54 -04:00
Roman Rizzi fdfb3a362a
DEV: Make sure max_username_length is within MAX_USERNAME_LENGTH_RANGE (#23104) 2023-08-15 12:12:22 -03:00
Roman Rizzi ee61fe5a2b
DEV: Rewrite username validator specs without hardcoded length limits (#23102) 2023-08-15 11:09:26 -03:00
Natalie Tay aa7917d533
DEV: Add some description to help with understanding flakey test failures (#23053) 2023-08-14 12:19:05 +08:00
Renato Atilio 840bea3c51
FEATURE: add topic voting webhook event type (#23072)
* FEATURE: add topic upvote webhook event type

* DEV: use a generic event type name for other actions in the same plugin
2023-08-11 13:42:28 -03:00
Martin Brennan 09223e5ae7
DEV: Remove enable_experimental_hashtag_autocomplete logic (#22820)
This commit removes any logic in the app and in specs around
enable_experimental_hashtag_autocomplete and deletes some
old category hashtag code that is no longer necessary.

It also adds a `slug_ref` category instance method, which
will generate a reference like `parent:child` for a category,
with an optional depth, which hashtags use. Also refactors
PostRevisor which was using CategoryHashtagDataSource directly
which is a no-no.

Deletes the old hashtag markdown rule as well.
2023-08-08 11:18:55 +10:00
David Taylor e76e0ad592
DEV: In development, refresh client when theme changes are made (#22978)
This brings the theme development experience (via the discourse_theme cli) closer to the experience of making javascript changes in Discourse core/plugins via Ember CLI. Whenever a change is made to a non-css theme field, all clients will be instructed to immediately refresh via message-bus.
2023-08-04 11:02:26 +01:00
Ted Johansson 1f7e5e8e75
DEV: Switch over category settings to new table - Part 2 (#20580)
In #20135 we prevented invalid inputs from being accepted in category setting form fields on the front-end. We didn't do anything on the back-end at that time, because we were still discussing which path we wanted to take. Eventually we decided we want to move this to a new CategorySetting model.

This PR moves the num_auto_bump_daily from custom fields to the new CategorySetting model.

In addition it sets the default value to 0, which exhibits the same behaviour as when the value is NULL.
2023-08-04 10:53:22 +08:00
Renato Atilio 701ae8764e
FIX: keep first post edit history when moving/merging (#22966) 2023-08-03 22:04:35 -03:00
Martin Brennan 6286e790b2
DEV: Remove unread_private_messages and deprecation (#22893)
This was added all the way back in 2020 in b79ea986ac,
enough time has passed, we can delete this now.
2023-08-01 14:44:39 +10:00
Jarek Radosz a83c1d8f60
DEV: Fix a flaky quote post spec (#22891) 2023-08-01 00:48:40 +02:00