Commit Graph

1500 Commits

Author SHA1 Message Date
Gabriel Grubba f3a89620a1
FEATURE: Add WebHookEventsDailyAggregate (#27542)
* FEATURE: Add WebHookEventsDailyAggregate

Add WebHookEventsDailyAggregate model to store daily aggregates of web hook events.
Add AggregateWebHooksEvents job to aggregate web hook events daily.
Add spec for WebHookEventsDailyAggregate model.

* DEV: Update annotations for web_hook_events_daily_aggregate.rb

* DEV: Update app/jobs/scheduled/aggregate_web_hooks_events.rb

Co-authored-by: Martin Brennan <martin@discourse.org>

* DEV: Address review feedback

Solves:
- https://github.com/discourse/discourse/pull/27542#discussion_r1646961101
- https://github.com/discourse/discourse/pull/27542#discussion_r1646958890
- https://github.com/discourse/discourse/pull/27542#discussion_r1646976808
- https://github.com/discourse/discourse/pull/27542#discussion_r1646979846
- https://github.com/discourse/discourse/pull/27542#discussion_r1646981036

* A11Y: Add translation to retain_web_hook_events_aggregate_days key

* FEATURE: Purge old web hook events daily aggregate

Solves: https://github.com/discourse/discourse/pull/27542#discussion_r1646961101

* DEV:  Update tests for web_hook_events_daily_aggregate

Update WebHookEventsDailyAggregate to not use save! at the end
Solves: https://github.com/discourse/discourse/pull/27542#discussion_r1646984601

* PERF: Change job query to use WebHook table instead of WebHookEvent table

* DEV: Update tests to use `fab!`

* DEV: Address code review feedback.

Add idempotency to job
Add has_many to WebHook

* DEV: add test case for job and change job query

* DEV: Change AggregateWebHooksEvents job test name

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-06-25 13:56:47 -03:00
Renato Atilio 55da8a7701
FEATURE: add Untranslated filter to admin text customization (#27555)
Adds a checkbox to filter untranslated text strings in the admin UI, behind a hidden and default `false` site setting `admin_allow_filter_untranslated_text`.
2024-06-24 06:24:06 -03:00
Kelv 60d5170587
DEV: add validation on content_security_policy_script_src site setting (#27564)
* DEV: add validation on content_security_policy_script_src site setting
2024-06-21 17:00:22 +08:00
Rafael dos Santos Silva b2a9676f0b
FEATURE: Support Spotify Onebox (#27540) 2024-06-19 13:27:27 -03:00
Martin Brennan 83361b2fc5
FEATURE: Introduce site settings which require confirmation (#27315)
Many site settings can be distructive or have huge side-effects
for a site that the admin may not be aware of when changing it.

This commit introduces a `requires_confirmation` attribute that
can be added to any site setting. When it is true, a confirmation
dialog will open if that setting is changed in the admin UI,
optionally with a custom message that is defined in client.en.yml.

If the admin does not confirm, we reset the setting to its previous
clean value and do not save the new value.
2024-06-19 16:01:24 +10:00
Ted Johansson 3ff7ce78e7
FEATURE: Add hidden site setting to list 'unsafe-none' COOP referrers (#27510)
Some tooling may rely on an unsafe-none cross origin opener policy to work. This change adds a hidden site setting that can be used to list referrers where we add this header instead of the default one configured in cross_origin_opener_policy_header.
2024-06-19 11:11:35 +08:00
Krzysztof Kotlarek cc4c199680
FEATURE: optional 2FA enforcement (#27506)
A new admin setting called `enforce_second_factor_on_external_auth`. It allows users to authenticate using external providers even when 2FA is forced with `enforce_second_factor` site setting.
2024-06-19 09:32:30 +10:00
Kelv 2393234be5
DEV: remove legacy CSP implementation to make strict-dynamic only accepted behaviour (#27486)
* DEV: remove legacy CSP implementation that allowed for non-strict-dynamic behaviour
2024-06-18 16:40:53 +08:00
Jan Cernik d7ece61252
UX: Rename FAQ to Guidelines (#27451) 2024-06-13 12:09:53 -03:00
Alan Guo Xiang Tan 8cf4ed5f88
DEV: Introduce hidden `s3_inventory_bucket` site setting (#27304)
This commit introduces a hidden `s3_inventory_bucket` site setting which
replaces the `enable_s3_inventory` and `s3_configure_inventory_policy`
site setting.

The reason `enable_s3_inventory` and `s3_configure_inventory_policy`
site settings are removed is because this feature has technically been
broken since it was introduced. When the `enable_s3_inventory` feature
is turned on, the app will because configure a daily inventory policy for the
`s3_upload_bucket` bucket and store the inventories under a prefix in
the bucket. The problem here is that once the inventories are created,
there is nothing cleaning up all these inventories so whoever that has
enabled this feature would have been paying the cost of storing a whole
bunch of inventory files which are never used. Given that we have not
received any complains about inventory files inflating S3 storage costs,
we think that it is very likely that this feature is no longer being
used and we are looking to drop support for this feature in the not too
distance future.

For now, we will still support a hidden `s3_inventory_bucket` site
setting which site administrators can configure via the
`DISCOURSE_S3_INVENTORY_BUCKET` env.
2024-06-10 13:16:00 +08:00
David Taylor 9b11e9a8d1
UX: Hide 'same site cookies' site setting (#27230)
Changing this to anything other than 'Lax' will lead to pain. 'None' is much less secure, and 'Strict' will break lots of functionality.
2024-05-28 16:13:45 +01:00
Jarek Radosz 87769a83c4
DEV: Implement glimmer topic-list (#26743)
(experimental)

The initial implementation of glimmer topic-list and related components. Does not include new APIs and isn't compatible with existing customization. That's gonna come in future PRs.

Enabled by adding groups to `experimental_glimmer_topic_list_groups` setting.
2024-05-21 14:36:15 +02:00
Ted Johansson e1953f7042
DEV: Update LinkedIn OIDC client secret validation format (#27028)
The LinkedIn OIDC client secret format has changed. This updates the regular expression used for validation to reflect this.
2024-05-15 13:32:59 +09:00
David Taylor 742a3b138d
Add GoogleOther crawler user agent (#27000)
Per https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers

> GoogleOther is the generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and development.

This commit will ensure it's served the crawler view, and included in crawler metrics
2024-05-13 13:13:04 +01:00
Neil Lalonde efc21dd98b
UX: enable_powered_by_discourse site setting default is true (#26774) 2024-05-09 13:01:08 +05:30
Krzysztof Kotlarek da329d54af
FEATURE: new admin sidebar for moderators (#26795)
Enable a new sidebar for moderators with limited links.
2024-04-29 11:32:47 +10:00
Martin Brennan c791346088
FEATURE: Rate limit common AI bots crawlers by default (#26731)
This commit adds the most common AI bot crawlers seen
on our hosting (claudebot, gptbot, anthropic-ai, brightbot)
to our `slow_down_crawler_user_agents` and `crawler_user_agents`
site settings by default.

This means these AI bots will be rate limited by default instead
of site admins having to remember to do it for themselves.
2024-04-24 12:01:42 +10:00
Ted Johansson 9e31135eca
FEATURE: Allow users to sign in using LinkedIn OpenID Connect (#26281)
LinkedIn has grandfathered its old OAuth2 provider. This can only be used by existing apps. New apps have to use the new OIDC provider.

This PR adds a linkedin_oidc provider to core. This will exist alongside the discourse-linkedin-auth plugin, which will be kept for those still using the deprecated provider.
2024-04-19 18:47:30 +08:00
Krzysztof Kotlarek 98fc614162
FEATURE: mandatory fields for group site setting (#26612)
Automatically add `moderators` and `admins` auto groups to specific site settings.

In the new group-based permissions systems, we just want to check the user’s groups since it more accurately reflects reality

Affected settings:
- tag_topic_allowed_groups
- create_tag_allowed_groups
- send_email_messages_allowed_groups
- personal_message_enabled_groups
- here_mention_allowed_groups
- approve_unless_allowed_groups
- approve_new_topics_unless_allowed_groups
- skip_review_media_groups
- email_in_allowed_groups
- create_topic_allowed_groups
- edit_wiki_post_allowed_groups
- edit_post_allowed_groups
- self_wiki_allowed_groups
- flag_post_allowed_groups
- post_links_allowed_groups
- embedded_media_post_allowed_groups
- profile_background_allowed_groups
- user_card_background_allowed_groups
- invite_allowed_groups
- ignore_allowed_groups
- user_api_key_allowed_groups
2024-04-18 08:53:52 +10:00
David Taylor 3733db866c
DEV: Introduce default 'auto' mode for glimmer header (#26467)
This will automatically enable the glimmer header when all installed themes/plugins are ready. This replaces the old group-based site setting.

In 'auto' mode, we check for calls to deprecated APIs (e.g. decorateWidget) which affect the old header. If any are present, we stick to the old header implementation and print a message to the console alongside the normal deprecation messages.

To override this automatic behavior, a new `glimmer_header_mode` site setting can be set to 'disabled' or 'enabled'.

This change also means that our test suite is running with the glimmer header. This unveiled a couple of small issues (e.g. some incorrect `aria-*` and `alt` text) which are now fixed. A number of selectors had to be updated to ensure the tests were clicking the actual `<button>` elements rather than the surrounding `<li>` elements.
2024-04-10 14:35:54 +01:00
Blake Erickson 72ac675e4e
FEATURE: Consolidate link notifications (#26567)
Just like we have for consolidating likes this adds similar
functionality for consolidating links.
2024-04-09 11:53:37 -06:00
Krzysztof Kotlarek 3ce386c22f
FEATURE: enable new admin sidebar by default (#26391)
By default, enable the new admin sidebar.
In addition, migration was created for old sites to keep the old admin panel.
2024-04-08 09:57:27 +10:00
Martin Brennan d6c30a54d7
DEV: Added docs for additional site setting types (#26530)
I left out a few setting types that are not used much
(e.g. value_list) for now, we may remove them.
2024-04-05 16:20:56 +10:00
Alan Guo Xiang Tan a440e15291
DEV: Remove `experimental_objects_type_for_theme_settings` site setting (#26507)
Why this change?

Objects type for theme settings is no longer considered experimental so
we are dropping the site setting.
2024-04-04 12:01:31 +08:00
Blake Erickson ba806eec74
DEV: Various bulk-select dropdown tweaks (#26424)
* DEV: Various bulk-select dropdown tweaks

- Setting is no longer hidden
- descriptions have been moved to the modal
- Removed ... from one of the dropdown titles
2024-03-29 12:29:24 -06:00
jbrw 74d55f14fe
DEV: Add skip_email_bulk_invites hidden site setting (#26430)
This adds a hidden site setting of `skip_email_bulk_invites`

If set to `true`, the `BulkInvite` job will pass the value to `Invite`, meaning the generated invite wont trigger an email notification being sent to the newly invited user.

(This is useful if you want to manage the sending of the invite emails outside of Discourse.)
2024-03-29 13:22:00 -04:00
Martin Brennan e04b35a184
FIX: Hide footgun max_notifications_per_user site setting (#26414)
It's not really intentional to have regular admins change
this in all but pathological cases. It deletes all notifications
over this threshold for users without warning. If admins
really want to turn this on, they can do it via the app.yml file
2024-03-28 14:42:22 +10:00
Angus McLeod 7dc552c9cc
DEV: Add `import_embed_unlisted` site setting (#26222) 2024-03-27 08:57:43 -04:00
David Taylor 1cc8c72a98
DEV: Consolidate experimental 'Link' header implementations (#26377)
This commit removes the 'experimental_preconnect_link_header' site setting, and the 'preload_link_header' site setting, and introduces two new global settings: early_hint_header_mode and early_hint_header_name.

We don't actually send 103 Early Hint responses from Discourse. However, upstream proxies can be configured to cache a response header from the app and use that to send an Early Hint response to future clients.

- `early_hint_header_mode` specifies the mode for the early hint header. Can be nil (disabled), "preconnect" (lists just CDN domains) or "preload" (lists all assets).
- `early_hint_header_name` specifies which header name to use for the early hint. Defaults to "Link", but can be changed to support different proxy mechanisms.
2024-03-27 09:06:50 +00:00
David Taylor 312a5ead1f
DEV: Add Google-InspectionTool as a crawler user agent (#26375)
This user-agent is sent when URLs are inspected via the UI of Google's search console. It makes sense for us to serve it the same content as other bots, including GoogleBot.
2024-03-26 15:56:36 +00:00
Alan Guo Xiang Tan 36cdb1444c
EXPERIMENTAL: preconnect and dns-prefetch resource hints for CDN domains (#26215)
Why this change?

In https://web.dev/articles/preconnect-and-dns-prefetch, it describes
how hinting to the browser to preconnect to domains which we will
eventually use the connection for can help improve the time it takes to
load a page.

We are putting this behind an experimental flag so that we can test and
profile this in a production environment.

What does this change introduce?

Introduce a hidden experimental `experimental_preconnect_link_header`
site setting which when enabled will add the `preconnect` and
`dns-prefetch` resource hints to the response headers for full page load
requests.
2024-03-18 13:45:41 +08:00
Alan Guo Xiang Tan 003b80e62f
SECURITY: Add rate limits for uploads 2024-03-15 14:24:00 +08:00
Kris 9376a2e755
FEATURE: optionally show "Powered by Discourse" link to discourse.org (#26162) 2024-03-14 10:30:12 -04:00
David Taylor 92d357f91a
FEATURE: Enable strict-dynamic Content-Security-Policy by default (#26051)
Ref https://meta.discourse.org/t/298172 and https://meta.discourse.org/t/295603
2024-03-07 15:20:31 +00:00
Jeff Wong 00e76efb7c
FEATURE: site setting to include post in penalty messages (#26025)
Adds a site setting to include a post's content in penalty message.
When silencing/suspending a user from a post, or a reviewable with
a post, adds an option to include a post's content in the email
message by default.
2024-03-04 18:35:21 -08:00
Martin Brennan eca10e56b8
FEATURE: Allow specific groups to view raw email (#26003)
When a post is created by an incoming email, we show
an envelope icon on it which then opens a modal with the
raw email contents. Previously this was staff (admin+mod)
only, but now this commit adds the `view_raw_email_allowed_groups`
site setting, so any group can be added to give users permission
to see this.
2024-03-04 13:48:16 +10:00
Martin Brennan fef52c2ab7
DEV: Delete old enable_bookmarks_with_reminders setting (#25982)
This setting has not been used for a long time, get rid
of it and also update the historical migration to not
rely on the SiteSetting model.
2024-03-04 13:48:04 +10:00
Martin Brennan 0b60086ff8
DEV: Hide min_trust_level_to_allow_profile_background setting (#25980)
Followup to a57280cb17,
it was an oversight
2024-03-01 13:32:19 +10:00
Isaac Janzen 21f23cc032
DEV: Convert header to glimmer (#25214)
Here is a breakdown of the changes that will be implemented in this PR.

# Widgets -> Glimmer

Obviously, the intention of the todo here is to convert the header from widgets to glimmer. This PR splits the respective widgets as so:

### widgets/site-header.js
```mermaid height=200
flowchart TB
    A[widgets/site-header.js] 
    A-->B[components/glimmer-site-header.gjs]
```

### widgets/header.js and children
```mermaid height=200
flowchart TB
    A[widgets/header.js] 
    A-->B[components/glimmer-header.gjs]
    B-->C[glimmer-header/contents.gjs]
    C-->D[./auth-buttons.gjs]
    C-->E[./icons.gjs]
    C-->F[./user-menu-wrapper.gjs]
    C-->G[./hamburger-dropdown-wrapper.gjs]
    C-->H[./user-menu-wrapper.gjs]
    C-->I[./sidebar-toggle.gjs]
    C-->J[./topic/info.gjs]
```

There are additional components rendered within the `glimmer-header/*` components, but I will leave those out for now. From this view you can see that we split apart the logic of `widgets/header.js` into 10+ components. Breaking apart these mega files has many benefits (readability, etc).

# Services

I have introduced a [header](cdb42caa04/app/assets/javascripts/discourse/app/services/header.js) service. This simplifies how we pass around data in the header, as well as fixes a bug we have with "swiping" menu panels.


# Modifiers
Added a [close-on-click-outside](cdb42caa04/app/assets/javascripts/discourse/app/modifiers/close-on-click-outside.js) modifier that is built upon the [close-on-click-outside modifier](https://github.com/discourse/discourse/blob/main/app/assets/javascripts/float-kit/addon/modifiers/close-on-click-outside.js) that @jjaffeux built for float-kit. I think we could replace float-kit's implementation with mine and have it in a centralized location as they are extremely similar.

# Tests
Rewrote the existing header tests ([1](https://github.com/discourse/discourse/blob/main/app/assets/javascripts/discourse/tests/integration/components/widgets/header-test.js), [2](https://github.com/discourse/discourse/blob/main/app/assets/javascripts/discourse/tests/integration/components/site-header-test.js)) as system tests. 

# Other
- Converted `widgets/user-status-bubble.js` to a gjs component
- Converted `widgets/sidebar-toggle.js` to a gjs component
- Converted `topicFeaturedLinkNode()` to a gjs component
- Deprecated the [docking mixin](https://github.com/discourse/discourse/blob/main/app/assets/javascripts/discourse/app/mixins/docking.js)
2024-02-23 11:08:15 -07:00
Sam 207cb2052f
FIX: muted tags breaking hot page when filtered to tags (#25824)
Also, remove experimental setting and simply use top_menu for feature detection

This means that when people eventually enable the hot top menu, there will
be topics in it


Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2024-02-23 17:11:39 +11:00
Krzysztof Kotlarek fc9648578b
DEV: Make more group-based settings client: false (#25735)
Affects the following settings:

delete_all_posts_and_topics_allowed_groups
experimental_new_new_view_groups
enable_experimental_admin_ui_groups
custom_summarization_allowed_groups
pm_tags_allowed_for_groups
chat_allowed_groups
direct_message_enabled_groups
chat_message_flag_allowed_groups

This turns off client: true for these group-based settings,
because there is no guarantee that the current user gets all
their group memberships serialized to the client. Better to check
server-side first.
2024-02-19 13:25:59 +11:00
Martin Brennan a57280cb17
DEV: Change min_trust_level_to_allow_profile_background to trust level setting (#25721)
New setting name is profile_background_allowed_groups

c.f. https://meta.discourse.org/t/changes-coming-to-settings-for-giving-access-to-features-from-trust-levels-to-groups/283408
2024-02-19 10:47:47 +10:00
David Taylor b1f74ab59e
FEATURE: Add experimental option for strict-dynamic CSP (#25664)
The strict-dynamic CSP directive is supported in all our target browsers, and makes for a much simpler configuration. Instead of allowlisting paths, we use a per-request nonce to authorize `<script>` tags, and then those scripts are allowed to load additional scripts (or add additional inline scripts) without restriction.

This becomes especially useful when admins want to add external scripts like Google Tag Manager, or advertising scripts, which then go on to load a ton of other scripts.

All script tags introduced via themes will automatically have the nonce attribute applied, so it should be zero-effort for theme developers. Plugins *may* need some changes if they are inserting their own script tags.

This commit introduces a strict-dynamic-based CSP behind an experimental `content_security_policy_strict_dynamic` site setting.
2024-02-16 11:16:54 +00:00
Martin Brennan 3cc73cfd1e
FIX: Always preload admin plugin list for admin in sidebar (#25606)
When we show the links to installed plugins in the admin
sidebar (for plugins that have custom admin routes) we were
previously only doing this if you opened /admin, not if you
navigated there from the main forum. We should just always
preload this data if the user is admin.

This commit also changes `admin_sidebar_enabled_groups` to
not be sent to the client as part of ongoing efforts to
not check groups on the client, since not all a user's groups
may be serialized.
2024-02-09 12:52:22 +10:00
Alan Guo Xiang Tan 9f884cdaab
DEV: Introduce experimental `type: objects` theme setting (#25538)
Why this change?

This commit introduces an experimental `type: objects` theme setting
which will allow theme developers to store a collection of objects as
JSON in the database. Currently, the feature is still in development and
this commit is simply setting up the ground work for us to introduce the
feature in smaller pieces.

What does this change do?

1. Adds a `json_value` column as `jsonb` data type to the `theme_settings` table.
2. Adds a `experimental_objects_type_for_theme_settings` site setting to
   determine whether `ThemeSetting` records of with the `objects` data
   type can be created.
3. Updates `ThemeSettingsManager` to support read/write access from the
   `ThemeSettings#json_value` column.
2024-02-08 10:20:59 +08:00
Martin Brennan adb4eee153
DEV: Make more group-based settings client: false (#25585)
Affects the following settings:

* whispers_allowed_groups
* anonymous_posting_allowed_groups
* personal_message_enabled_groups
* shared_drafts_allowed_groups
* here_mention_allowed_groups
* uploaded_avatars_allowed_groups
* ignore_allowed_groups

This turns off `client: true` for these group-based settings,
because there is no guarantee that the current user gets all
their group memberships serialized to the client. Better to check
server-side first.
2024-02-08 09:43:34 +10:00
Isaac Janzen 5c43fd5054
DEV: Remove widget search menu (#25545) 2024-02-06 08:52:24 -07:00
Blake Erickson 3159522546
DEV: Correct some tl to group site setting changes (#25550)
- Created a new migration for here_mention
- Updated existing migration for here_mention
- Updated site settings for here_mention, create_tag, and
  send_email_messages
2024-02-05 09:50:46 -07:00
Blake Erickson a764ab5b54
DEV: Update min trust level to tag topics migration to groups (#25527)
* DEV: Update min trust level to tag topics migration to groups

- Update the existing migration to include staff and admin
- Update default values
- Added migration to include staff and admin cases
2024-02-05 09:49:54 -07:00
Arpit Jalan badc390ebe
FEATURE: allow disabling user activity tab for non admin users (#25540)
* FEATURE: allow disabling user activity tab for non admin users

* add another test case
2024-02-05 14:30:36 +05:30