Commit Graph

1117 Commits

Author SHA1 Message Date
Krzysztof Kotlarek 40d65dddf8
Revert "DEV: move post flags into database (#26951)" (#27102)
This reverts commit 7aff9806eb.
2024-05-21 16:21:07 +10:00
Krzysztof Kotlarek 7aff9806eb
DEV: move post flags into database (#26951)
This is preparation for a feature that will allow admins to define their custom flags. Current behaviour should stay untouched.
2024-05-21 13:15:32 +10:00
Ted Johansson 32aaf2e8d3
DEV: Remove deprecated AuthProvider#enabled_setting= (#27081)
AuthProvider#enabled_setting=, used primarily by plugins, has been deprecated since version 2.9, in favour of Authenticator#enabled?. This PR confirms we are seeing no more usage and removes the method.
2024-05-20 18:10:15 +08:00
Régis Hanol b908abe35a
FIX: keep topic.word_count in sync (#27065)
Whenever one creates, updates, or deletes a post, we should keep the `topic.word_count` counter in sync.

Context - https://meta.discourse.org/t/-/308062
2024-05-17 17:05:49 +02:00
Martin Brennan d964709644
DEV: Add more _map extensions for list site settings (#27045)
Following on from eea74e0e32,
this commit adds the automatic _map splitting shorthand
for emoji_list, tag_list site settings.
2024-05-16 13:43:10 +10:00
Alan Guo Xiang Tan e31cf66f11
FIX: `FinalDestination#get` forwarding `Authorization` header on redirects (#27043)
This commits updates `FinalDestination#get` to not forward
`Authorization` header on redirects since most HTTP clients I tested like
curl and wget does not it.

This also fixes a recent problem in `DiscourseIpInfo.mmdb_download`
where we will fail to download the databases when both `GlobalSetting.maxmind_account_id` and
`GlobalSetting.maxmind_license_key` has been set. The failure is due to
the bug above where the redirected URL given by MaxMind does not accept
an `Authorization` header.
2024-05-16 08:37:34 +08:00
dsims e6e3eaf472
FIX: avoid error from missing meta tags (#26927) 2024-05-14 11:41:53 -04:00
marstall 6df2f94bbc
DEV add modifiers to message_builder so plugins can customize subject/body/html (#26867) 2024-05-13 14:59:15 -04:00
David Taylor 97847f6cd8
Revert "DEV: `@babel/plugin-proposal-decorators` -> `decorator-transforms` (#25290)" (#26971)
This reverts commit 0f4520867b.

This has led to two problems:

1. An incompatibility with Cloudflare's "auto minify" feature. They've deprecated this feature because of incompatibility with modern JS syntax. But unfortunately it will remain enabled on existing properties until 2024-08-05.

2. Discourse fails to boot in Safari 15. This is strange, because Safari does support all the required features in our production JS bundles. Even more strangely, things start working as soon as you open the developer tools. That suggests the cause could be a Safari bug rather than a simple incompatibility.

Reverting while we work out a path forward on both those issues.
2024-05-10 12:48:16 +01:00
Alan Guo Xiang Tan c8da2a33e8
FIX: Attempt to onebox even if response body exceeds `max_download_kb` (#26929)
In 95a82d608d, we lowered the default for
`Onebox.options.max_download_kb` from 10mb to 2mb for security hardening
purposes. However, this resulted in multiple bug reports where seemingly
nomral URLs stopped being oneboxed. It turns out that lowering
`Onebox.options.max_download_kb` resulted in `Onebox::Helpers::DownloadTooLarge` being raised
more often for more URLs  in `Onebox::Helpers.fetch_response` which
`Onebox::Helpers.fetch_html_doc` relies on. When
`Onebox::Helpers::DownloadTooLarge` is raised in
`Onebox::Helpers.fetch_response`, we throw away whatever response body
which we have already downloaded at that point. This is not ideal
because Nokogiri can parse incomplete HTML documents and there is a
really high chance that the incomplete HTML document still contains the
information which we need for oneboxing.

Therefore, this commit updates `Onebox::Helpers.fetch_html_doc` to not
throw away the response body when the size of the response body exceeds
`Onebox.options.max_download_size`. Instead, we just take whatever
response which we have and get Nokogiri to parse it.
2024-05-09 07:00:34 +08:00
David Taylor ece0150cb7
FIX: Ensure RequestTracker handles bubbled exceptions correctly (#26940)
This can happen for various reasons including rate limiting and middleware bugs. This should resolve the warning we're seeing in the logs

```
RequestTracker.get_data failed : NoMethodError : undefined method `[]' for nil:NilClass
```
2024-05-08 16:08:39 +01:00
David Taylor 0f4520867b
DEV: `@babel/plugin-proposal-decorators` -> `decorator-transforms` (#25290)
decorator-transforms (https://github.com/ef4/decorator-transforms) is a modern replacement for babel's plugin-proposal-decorators. It provides a decorator implementation using modern browser features, without needing to enable babel's full suite of class feature transformations. This improves the developer experience and performance.

In local testing with Google's 'tachometer' tool, this reduces Discourse's 'init-to-render' time by around 3-4% (230ms -> 222ms).

It reduces our initial gzip'd JS payloads by 3.2% (2.43MB -> 2.35MB), or 7.5% (14.5MB -> 13.4MB) uncompressed.
2024-05-08 10:40:51 +01:00
Martin Brennan ce2388e40b
FEATURE: Remove "Enable Sidebar" step from setup wizard (#26926)
This keeps coming up in user testing as something
we want to get rid of. The `navigation_menu` setting
has been set to sidebar by default for some time now,
and we are rolling out admin sidebar widely. It just
doesn't make sense to let people turn this off in
the first step of the wizard -- we _want_ people to
use the sidebar.
2024-05-08 10:00:40 +10:00
Régis Hanol 12cba2ce24 PERF: bail out of expensive post validations
Whenever a post already failed "lightweight" validations, we skip all the expensive validations (that cooks the post or run SQL queries) so that we reply as soon as possible.

Also skip validating polls when there's no "[/poll]" in the raw.

Internal ref - t/115890
2024-05-07 18:56:16 +02:00
Régis Hanol 342328b622
DEV: depend less on pngquant version (#26906)
This spec has been failing forever on my machine. I guess I have a "better" version of pngquant?

---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-05-07 16:56:05 +02:00
Jarek Radosz 79870d3a1e
DEV: Fix random typos (#26881) 2024-05-06 20:52:48 +02:00
Daniel Waterworth 9f9c7f0a23
FIX: Handle tags with underscores correctly (#26839) 2024-05-01 20:01:39 -05:00
Alan Guo Xiang Tan a6624af66e
DEV: Add `isValidUrl` helper function to theme migrations (#26817)
This commit adds a `isValidUrl` helper function to the context in
which theme migrations are ran in. This helper function is to make it
easier for theme developers to check if a string is a valid URL or path
when writing theme migrations. This can be helpful in cases when
migrating a string based setting to `type: objects` which contain `type:
string` properties with URL validations enabled.

This commit also introduces the `UrlHelper.is_valid_url?` method
which actually checks that the URL string is of the valid format instead of
only checking if the URL string is parseable which is what `UrlHelper.relaxed_parse` does
and is not sufficient for our needs.
2024-04-30 16:45:07 +08:00
Régis Hanol bfc0f3f4cd FIX: prevent duplicate attachments in incoming emails - take 2
This is a follow up of 5fcb7c262d

It was missing the case where secure uploads is enabled, which creates a copy of the upload no matter what.

So this checks for the original_sha1 of the uploads as well when checking for duplicates.
2024-04-30 08:15:07 +02:00
Régis Hanol 19b7b22627 DEV: fix the fix for flakey test 😓
We should not be clearing **all** the advanced search filters and orders, because some are required by the application.
2024-04-29 21:43:38 +02:00
Jan Cernik 9fb888923d
FIX: Do not show hidden posts in search results (#26800) 2024-04-29 12:32:02 -03:00
Régis Hanol f7a1272fa4 DEV: cleanup custom filters to prevent leaks
Ensures we clean up any custom filters added in the specs to prevent any leaks when running the specs.

Follow up to https://github.com/discourse/discourse/pull/26770#discussion_r1582464760
2024-04-29 16:11:12 +02:00
Régis Hanol 803c275bd7 DEV: add support for adding custom status filter
Those can be used in the /filter routes.
2024-04-26 14:04:03 +02:00
Daniel Waterworth e0e0e0506f
DEV: Limit the number of category sidebar links a user can have (#26756) 2024-04-25 13:21:39 -05:00
David Taylor 2f2da72747
FEATURE: Add experimental tracking of 'real browser' pageviews (#26647)
Our 'page_view_crawler' / 'page_view_anon' metrics are based purely on the User Agent sent by clients. This means that 'badly behaved' bots which are imitating real user agents are counted towards 'anon' page views.

This commit introduces a new method of tracking visitors. When an initial HTML request is made, we assume it is a 'non-browser' request (i.e. a bot). Then, once the JS application has booted, we notify the server to count it as a 'browser' request. This reliance on a JavaScript-capable browser matches up more closely to dedicated analytics systems like Google Analytics.

Existing data collection and graphs are unchanged. Data collected via the new technique is available in a new 'experimental' report.
2024-04-25 11:00:01 +01:00
Ted Johansson 9e31135eca
FEATURE: Allow users to sign in using LinkedIn OpenID Connect (#26281)
LinkedIn has grandfathered its old OAuth2 provider. This can only be used by existing apps. New apps have to use the new OIDC provider.

This PR adds a linkedin_oidc provider to core. This will exist alongside the discourse-linkedin-auth plugin, which will be kept for those still using the deprecated provider.
2024-04-19 18:47:30 +08:00
Ted Johansson f3cad5f3a2
FIX: Correctly re-attach allowed images in activity summary e-mail (#26642)
For e-mails, secure uploads redacts all secure images, and later uses the access control post to re-attached allowed ones. We pass the ID of this post through the X-Discourse-Post-Id header. As the name suggests, this assumes there's only ever one access control post. This is not true for activity summary e-mails, as they summarize across posts.

This adds a new header, X-Discourse-Post-Ids, which is used the same way as the old header, but also works for the case where an e-mail is associated with multiple posts.
2024-04-18 10:27:46 +08:00
Krzysztof Kotlarek 98fc614162
FEATURE: mandatory fields for group site setting (#26612)
Automatically add `moderators` and `admins` auto groups to specific site settings.

In the new group-based permissions systems, we just want to check the user’s groups since it more accurately reflects reality

Affected settings:
- tag_topic_allowed_groups
- create_tag_allowed_groups
- send_email_messages_allowed_groups
- personal_message_enabled_groups
- here_mention_allowed_groups
- approve_unless_allowed_groups
- approve_new_topics_unless_allowed_groups
- skip_review_media_groups
- email_in_allowed_groups
- create_topic_allowed_groups
- edit_wiki_post_allowed_groups
- edit_post_allowed_groups
- self_wiki_allowed_groups
- flag_post_allowed_groups
- post_links_allowed_groups
- embedded_media_post_allowed_groups
- profile_background_allowed_groups
- user_card_background_allowed_groups
- invite_allowed_groups
- ignore_allowed_groups
- user_api_key_allowed_groups
2024-04-18 08:53:52 +10:00
Martin Brennan 7a083daf27
Revert "FIX: Post uploads setting access_control_post_id unnecessarily (#26627)" (#26643)
This reverts commit cdc8e9de1b.

It's made things worse internally and on meta.
2024-04-16 14:10:25 +10:00
Martin Brennan cdc8e9de1b
FIX: Post uploads setting access_control_post_id unnecessarily (#26627)
This commit addresses an issue for sites where secure_uploads
is turned on after the site has been operating without it for
some time.

When uploads are linked when they are used inside a post,
we were setting the access_control_post_id unconditionally
if it was NULL to that post ID and secure_uploads was true.

However this causes issues if an upload has been used in a
few different places, especially if a post was previously
used in a PM and marked secure, so we end up with a case of
the upload using a public post for its access control, which
causes URLs to not use the /secure-uploads/ path in the post,
breaking things like image uploads.

We should only set the access_control_post_id if the post is the first time the
upload is referenced so it cannot hijack uploads from other places.
2024-04-16 10:37:57 +10:00
Martin Brennan 380e5ca6cb
DEV: Move more service code to core (#26613)
This is to enable :array type attributes for Contract
attributes in services, this is a followup to the move
of services from chat to core here:

cab178a405

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2024-04-12 13:14:19 +02:00
Alan Guo Xiang Tan e2ced85757
DEV: Allow enum typed theme objects property to be optional (#26571)
This commit changes enum typed theme objects property to be optional.
Previously, an enum typed property is always required but we have found
that this might not be ideal so we want to change it.
2024-04-09 11:26:24 +08:00
Martin Brennan 0d0dbd391a
DEV: Rename with_secure_uploads? to should_secure_uploads? on Post (#26549)
This method name is a bit confusing; with_secure_uploads implies
it may return a block or something with the uploads of the post,
and has_secure_uploads implies that it's checking whether the post
is linked to any secure uploads.

should_secure_uploads? communicates the true intent of this method --
which is to say whether uploads attached to this post should be
secure or not.
2024-04-09 13:23:11 +10:00
Rafael dos Santos Silva dd83a07550
FEATURE: Hide summarization in PMs (#26532) 2024-04-05 12:12:59 -03:00
Jan Cernik cab178a405
DEV: Move chat service objects into core (#26506) 2024-04-04 10:57:41 -03:00
Gerhard Schlager 82c62fe44f
DEV: Correctly pluralize error messages (#26469) 2024-04-04 15:02:09 +02:00
Alan Guo Xiang Tan a440e15291
DEV: Remove `experimental_objects_type_for_theme_settings` site setting (#26507)
Why this change?

Objects type for theme settings is no longer considered experimental so
we are dropping the site setting.
2024-04-04 12:01:31 +08:00
Vinoth Kannan 9dc6325821
DEV: add logo URL and locale details to the Discover stats. (#26320)
We will be collecting the logo URL and the site's default locale values along with existing basic details to display the site on the Discourse Discover listing page. It will be included only if the site is opted-in by enabling the "`include_in_discourse_discover`" site setting.

Also, we no longer going to use `about.json` and `site/statistics.json` endpoints retrieve these data. We will be using only the `site/basic-info.json` endpoint.
2024-04-04 00:22:28 +05:30
Penar Musaraj 1eb70973a2
DEV: allow themes to render their own custom homepage (#26291)
This PR adds a theme modifier and route so that custom themes can opt to show their own homepage. See PR description for example usage.
2024-04-02 11:05:08 -04:00
Alan Guo Xiang Tan a84757fd91
FIX: Error not being raised for required typed categories property (#26443)
Why this change?

For a schema like this:

```
schema = {
  name: "section",
  properties: {
    category_property: {
      type: "categories",
      required: true,
    },
  },
}
```

When the value of the property is set to an empty array, we are not
raising an error which we should because the property is marked as
required.
2024-04-01 10:11:40 +08:00
Alan Guo Xiang Tan a670d6d4af
DEV: Change group type to groups type for theme object schema (#26417)
Why this change?

This is a follow-up to 86b2e3a.

Basically, we want to allow people to select more than 1 group as well.

What does this change do?

1. Change `type: group` to `type: groups` and support `min` and `max`
   validations for `type: groups`.

2. Fix the `<SchemaThemeSetting::Types::Groups>` component to support the
   `min` and `max` validations and switch it to use the `<GroupChooser>` component
   instead of the `<ComboBoxComponent>` component which previously only supported
   selecting a single group.
2024-03-28 22:05:48 +08:00
Angus McLeod 7dc552c9cc
DEV: Add `import_embed_unlisted` site setting (#26222) 2024-03-27 08:57:43 -04:00
Osama Sayegh 72c4709a5a
FIX: Skip tags-related validations when the skip_validations option is present (#26379)
The `TopicCreator` class has a `skip_validations` option that can force-create a topic without performing permission checks or validation rules. However, at the moment it doesn't skip validations that are related to tags, so topics that are created by the system or by some scrip can still fail if they use tags. This commit makes the `TopicCreator` class skip all tags-related checks if the `skip_validations` is specified.

Internal topic: t/124280.
2024-03-27 12:56:21 +03:00
Alan Guo Xiang Tan 476d91d233
DEV: Change category type to categories type for theme object schema (#26339)
Why this change?

This is a follow-up to 86b2e3aa3e.

Basically, we want to allow people to select more than 1 category as well.

What does this change do?

1. Change `type: category` to `type: categories` and support `min` and `max`
   validations for `type: categories`.

2. Fix the `<SchemaThemeSetting::Types::Categories>` component to support the
   `min` and `max` validations and switch it to use the `<CategorySelector>` component
   instead of the `<CategoryChooser>` component which only supports selecting one category.
2024-03-27 10:54:30 +08:00
David Taylor a8d20f92fb
FEATURE: Add page number to page titles for crawlers (#26367)
At the moment, all topic `?page=` views are served with exactly identical page titles. If you search for something which is mentioned many times in the same Discourse topic, this makes for some very hard-to-understand search results! All the result titles are exactly the same, with no indication of why there are multiple results showing.

This commit adds a `- Page #` suffix to the titles in this situation. This lines up with our existing strategy for topic-list pagination.
2024-03-26 15:19:00 +00:00
Jarek Radosz 4c860995e0
DEV: Remove unnecessary rails_helper requiring (#26364) 2024-03-26 11:32:01 +01:00
Ted Johansson 5ee23fc394
DEV: Make all admins TL4 in tests (#25435)
Make admins TL4 by default in tests, foregoing the need to call refresh_auto_groups on them.
2024-03-26 11:41:12 +08:00
Jarek Radosz 0aec53ee9b
DEV: Add a watched-words/mini_racer perf regression spec (#26341)
/t/121361
2024-03-25 13:35:02 +01:00
Alan Guo Xiang Tan 86b2e3aa3e
DEV: Change `tag` type to `tags` type for theme object schema (#26315)
Why this change?

While working on the tag selector for the theme object editor, I
realised that there is an extremely high possibility that users might want to select
more than one tag. By supporting the ability to select more than one
tag, it also means that we get support for a single tag for free as
well.

What does this change do?

1. Change `type: tag` to `type: tags` and support `min` and `max`
   validations for `type: tags`.

2. Fix the `<SchemaThemeSetting::Types::Tags>` component to support the
   `min` and `max` validations
2024-03-22 15:32:00 +08:00
Alan Guo Xiang Tan dfc406fdc2
FIX: Validate tags using `Tag#name` instead of `Tag#id` in `ThemeSettingsObjectValidator` (#26314)
Why this change?

Fortunately or unfortunately in Discourse core, we mainly use `Tag#name`
to look up tags and not its id. This assumption is built into the
frontend as well so we need to use the tag's name instead of the id
here.
2024-03-22 11:05:16 +08:00