Commit Graph

10078 Commits

Author SHA1 Message Date
Martin Brennan a128ce5c4c
FIX: Missing multipart/mixed boundary on emails (#27599)
Followup 96a0781bc1

When sending emails where secure uploads is enabled
and secure_uploads_allow_embed_images_in_emails is
true, we attach the images to the email, and we
do some munging with the final email so the structure
of the MIME parts looks like this:

```
multipart/mixed
  multipart/alternative
    text/plain
    text/html
  image/jpeg
  image/jpeg
  image/png
```

However, we were not specifying the `boundary` of the
`multipart/mixed` main content-type of the email, so
sometimes the email would come through appearing to
have an empty body with the entire thing attached as
one attachment, and some mail parsers considered the
entire email as the "epilogue" and/or "preamble".

This commit fixes the issue by specifying the boundary
in the content-type header per https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
2024-06-25 13:43:10 +10:00
Renato Atilio 55da8a7701
FEATURE: add Untranslated filter to admin text customization (#27555)
Adds a checkbox to filter untranslated text strings in the admin UI, behind a hidden and default `false` site setting `admin_allow_filter_untranslated_text`.
2024-06-24 06:24:06 -03:00
Loïc Guitaut ce00f83173 DEV: Upgrade Rails to version 7.1
---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-06-24 11:16:14 +02:00
Sam cb3456e18f
FEATURE: add JavaScript mode to ace (#27586)
This allows plugins to use Ace Editor in JavaScript mode. This will be used in Discourse AI when defining custom tools.
2024-06-24 13:38:24 +10:00
Penar Musaraj f4108702c8
FIX: Regression in custom homepage modifier used in theme components (#27569) 2024-06-21 11:24:11 -04:00
Loïc Guitaut 160011793a Revert "DEV: Upgrade Rails to version 7.1 (#27539)"
This reverts commit ca4af53be8.
2024-06-21 11:20:40 +02:00
Kelv 60d5170587
DEV: add validation on content_security_policy_script_src site setting (#27564)
* DEV: add validation on content_security_policy_script_src site setting
2024-06-21 17:00:22 +08:00
Loïc Guitaut ca4af53be8 DEV: Upgrade Rails to version 7.1 (#27539)
* DEV: Upgrade Rails to 7.1

* FIX: Remove references to `Rails.logger.chained`

`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.

Some code in our initializers was still using `chained` instead of
`broadcasts`.

* DEV: Make parameters optional to all FakeLogger methods

* FIX: Set `override_level` on Logster loggers (#27519)

A followup to f595d599dd

* FIX: Don’t duplicate Rack response

---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-06-21 09:44:06 +02:00
Penar Musaraj 33de5abb6e
DEV: Extract theme resolution to a helper (#27426)
This ensures that the theme id is resolved as early as possible in the
request cycle. This is necessary for the custom homepage to skip
preloading the wrong data.
2024-06-20 11:33:46 -04:00
Loïc Guitaut 982c005979 Revert "DEV: Upgrade Rails to version 7.1 (#27539)"
This reverts commit 2301dddcff.
2024-06-20 11:43:35 +02:00
Loïc Guitaut 2301dddcff
DEV: Upgrade Rails to version 7.1 (#27539)
* DEV: Upgrade Rails to 7.1

* FIX: Remove references to `Rails.logger.chained`

`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.

Some code in our initializers was still using `chained` instead of
`broadcasts`.

* DEV: Make parameters optional to all FakeLogger methods

* FIX: Set `override_level` on Logster loggers (#27519)

A followup to f595d599dd

* FIX: Don’t duplicate Rack response

---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-06-20 10:33:01 +02:00
Rafael dos Santos Silva b2a9676f0b
FEATURE: Support Spotify Onebox (#27540) 2024-06-19 13:27:27 -03:00
Ted Johansson 96a0781bc1
FIX: Avoid duplicating e-mail body in summary e-mail (#27535)
We recently fixed a problem where secure upload images weren't re-attached when sending the activity summary e-mail.

This fix contained a bug that would lead to n copies of the e-mail body being included, n being the number of duplicates. This is because #fix_parts_after_attachments! was called once per attachment, and adding more parts to the multipart e-mail.

This PR fixes that by:

Adding a failing test case for the above.
Moving the looping over multiple posts into #fix_parts_after_attachments! itself.
2024-06-19 20:11:47 +08:00
Jarek Radosz 251b3a5c47
DEV: Remove discourse-moderator-attention (#27537)
…from the official plugins list. It's archived now.
2024-06-19 13:26:57 +02:00
Bianca Nenciu 2ce83b4d2e
FIX: Do not extract links for hotlinked images (#27538)
When a post is cooked the links are extracted and `TopicLink` instances
are created for each of them. These links are used in various places,
including the topic view, user summary page, etc.

In previous commit 48e5d1a, hotlinked images from Oneboxes have been
ignored from the texts, but hotlinked images turned into Lightboxes
were still extracted.
2024-06-19 14:25:29 +03:00
Martin Brennan 83361b2fc5
FEATURE: Introduce site settings which require confirmation (#27315)
Many site settings can be distructive or have huge side-effects
for a site that the admin may not be aware of when changing it.

This commit introduces a `requires_confirmation` attribute that
can be added to any site setting. When it is true, a confirmation
dialog will open if that setting is changed in the admin UI,
optionally with a custom message that is defined in client.en.yml.

If the admin does not confirm, we reset the setting to its previous
clean value and do not save the new value.
2024-06-19 16:01:24 +10:00
Krzysztof Kotlarek cc4c199680
FEATURE: optional 2FA enforcement (#27506)
A new admin setting called `enforce_second_factor_on_external_auth`. It allows users to authenticate using external providers even when 2FA is forced with `enforce_second_factor` site setting.
2024-06-19 09:32:30 +10:00
Jarek Radosz 5cb84f8dcf
DEV: Revert rails 7.1 upgrade (#27522)
* Revert "FIX: Set `override_level` on Logster loggers (#27519)"

This reverts commit c1b0488c54.

* Revert "DEV: Make parameters optional to all FakeLogger methods"

This reverts commit 3318dad7b4.

* Revert "FIX: Remove references to `Rails.logger.chained`"

This reverts commit f595d599dd.

* Revert "DEV: Upgrade Rails to 7.1"

This reverts commit 081b00391e.
2024-06-18 23:48:30 +02:00
Loïc Guitaut f595d599dd FIX: Remove references to `Rails.logger.chained`
`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.

Some code in our initializers was still using `chained` instead of
`broadcasts`.
2024-06-18 17:46:40 +02:00
Loïc Guitaut 081b00391e DEV: Upgrade Rails to 7.1 2024-06-18 15:58:05 +02:00
Loïc Guitaut 2a22a3b51d FIX: Treat corrupt cache as cache miss
Currently when a cache entry is corrupt, we log the event without doing
anything else. It means the cache is still corrupt, and the proper value
isn’t computed again.

Normally, it’s very rare the cache becomes corrupt, but it can happen
when upgrading Rails for example and the cache format changes. This is
normally handled automatically by Rails but since we’re using a custom
cache class, we have to do it ourselves.

This patch takes the same approach the Rails team did, when a cache
entry is corrupt, we treat it as a miss, recomputing the proper value
and caching it in the new format.
2024-06-18 14:47:33 +02:00
Régis Hanol 53b3d2f0dc FIX: BBCode tag parser
Wasn't quite handling the cases where a closing bracket `]` was used in the value of one of the attributes.

```markdown
[chat quote=user channel="[broken]"]
```

Would not be correctly parsed because we would _greedily_ use the first `]` as the end of the tag even though it might be a valid character when inside proper quotes.

c39a4de139/app/assets/javascripts/discourse-markdown-it/src/features/bbcode-block.js (L62)

Re-wrote the `parseBBCodeTag` to properly handle the following cases

- A closing tag (aka `[/name]`) which are easy since they don't have any attributes
- An old `[quote=...]` format we used that doesn't uses quotes but still has various attributes of the form `key:value`
- All three valid BBCode opening tag formats we support
  - `[name]` without any attributes
  - `[name=foo]` with a default value
  - `[name foo=bar]` with some attributes

Ended up having to fix/rewrite the few bbcode rules that were using the `parseBBCodeTag` function, namely `d-wrap` and `discourse-local-dates`.

While working on this, I think I also found a way to get rid the of shims we had in place so that plugins could use the `parseBBCodeTag` function.

Reference - https://meta.discourse.org/t/having-a-right-bracket-in-a-channel-name-breaks-all-quotes-from-that-channel/308439
2024-06-18 10:47:18 +02:00
Kelv 2393234be5
DEV: remove legacy CSP implementation to make strict-dynamic only accepted behaviour (#27486)
* DEV: remove legacy CSP implementation that allowed for non-strict-dynamic behaviour
2024-06-18 16:40:53 +08:00
Alan Guo Xiang Tan 006169f782
DEV: Remove `hostname` from `DiscourseLogstashLogger` output (#27485)
This is a duplicate of the `host` field which means we are bloating the
logs unnecessarily.

Just remove without depreciation for now but we are open to properly
depreciating it if others depend on this field.
2024-06-18 07:24:42 +08:00
Daniel Waterworth 0a881a59d3
DEV: Don't enforce per ip rate limits for admin api requests (#27500) 2024-06-17 13:21:11 -05:00
Renato Atilio 49fdccbb1d
FIX: restrict a href protocols on form template description (#27472) 2024-06-14 11:39:43 -03:00
Osama Sayegh 4aea12fdcb
DEV: Allow fetching specific site settings and introduce a service for updating site settings (#27481)
This commit adds ability to fetch a subset of site settings from the `/admin/site_settings` endpoint so that it can be used in all places where the client app needs access to a subset of the site settings.

Additionally, this commit also introduces a new service class called `UpdateSiteSetting` that encapsulates all the logic that surrounds updating a site setting so that it can be used to update site setting(s) anywhere in the backend. This service comes in handy with, for example, the controller for the flags admin config area which may need to update some site settings related to flags.

Internal topic: t/130713.
2024-06-14 13:07:27 +03:00
Gabriel Grubba 93c2ae585c
FEATURE: Add tag_group option in `/filter` (#27427)
* FEATURE: Add tag_group option in `/filter`

* DEV: Update tag_group_filter in `/filter` to use SQL

* DEV: Add guardian to `/filter` by tag_group

* DEV: remove unused code

* DEV: Update tag_group filter implementation

* DEV: Add tests to tag_group filter
2024-06-13 14:03:49 -03:00
Jan Cernik d7ece61252
UX: Rename FAQ to Guidelines (#27451) 2024-06-13 12:09:53 -03:00
Loïc Guitaut c790c57415 DEV: Show parameters on a service contract failure
Now, when calling `StepsInspector#error` on a contract failure, the
output will contain the parameters provided to the contract.
2024-06-13 12:19:26 +02:00
Brendan Bell 0a2926ae03
DEV: Introduce users:exists rake task (#27163)
Checks if a user exists for given email address
2024-06-13 08:04:02 +01:00
Alan Guo Xiang Tan ae1d8c50da
DEV: Add ability to log a warning when Sidekiq job runs for too long (#27446)
This commits introduces the `sidekiq_report_long_running_jobs_minutes`
global setting which allows a site administrator to log a warning in the
Rails log when a Sidekiq job has been running for too long.

The warning is logged with the backtrace of the thread that is
processing the Sidekiq job to make it easier to figure out what a
sidekiq job is stuck on.
2024-06-13 14:24:44 +08:00
Martin Brennan e94ab11477
DEV: Allow for setting a message with SiteSetting.set_and_log (#27447)
When we turn on settings automatically for customers,
we sometimes use `.set_and_log` which will make a staff
action log for the site setting change. This is fine, but
there is no context for customers.

This change allows setting a message with `.set_and_log`, which
will be stored in the `details` column of the staff action log
created, which will show up on `/admin/logs/staff_action_logs`

---------

Co-authored-by: Kelv <kelv@discourse.org>
2024-06-13 14:59:49 +10:00
Ted Johansson a5df029be3
FIX: Email::Sender expects type to be a string (#27463)
In #26642 we introduced a change that re-attaches securely uploaded images in the digest e-mail. However, this change assumed that the type argument to the Email::Sender constructor would be a symbol, but when it is coming from the UserEmail job it is a string. This PR fixes that.
2024-06-13 11:53:25 +08:00
Alan Guo Xiang Tan 1f4aa343d8
DEV: Move spec file to right folder (#27449)
Spec file is not being run because it was in the wrong folder.
2024-06-13 06:38:03 +08:00
Sam dc8249c08a
FEATURE: align with /filter and allow multiple category search (#27440)
This introduces the syntax of

`category:a,b,c` which will search across multiple categories.

Previously there was no way to allow search across a wide selection of
categories.
2024-06-12 16:06:04 +10:00
Alan Guo Xiang Tan be4f1e3350
PERF: Cache `hostname` in `DiscourseLogstashLogger` (#27442)
Logging events happen all the time. No need for us to keep running code
to fetch the hostname when it doesn't change between deploys.
2024-06-12 13:34:07 +08:00
Alan Guo Xiang Tan 781a5f60ea
DEV: Move RSpec test file into test folder (#27437)
Test was not being run because it was in the wrong folder
2024-06-12 06:58:44 +08:00
Arkshine 29460e1422 DEV: Provide a safe agent in check_crawler_limits() 2024-06-11 14:02:46 +02:00
Arkshine 30ae313dec DEV: Apply suggested changes 2024-06-11 14:02:46 +02:00
Arkshine 1fffb236b2 FIX: crawler requests exceptions for non UTF-8 user agents with invalid bytes 2024-06-11 14:02:46 +02:00
Alan Guo Xiang Tan 9fdcdcf58d
DEV: Log error encountered when reopening sidekiq logs (#27411)
We are seeing the following error in our logs when Sidekiq is sent a
`USR1` signal in production when logrotate happens:

```
log writing failed. stream closed in another thread
Error encountered while starting Sidekiq: can't be called from trap context\n/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/unicorn-6.1.0/lib/unicorn/util.rb:71:in `reopen'
```

I'm not quite sure where the error is triggered from so I'm improving
the way we log errors.
2024-06-11 12:29:48 +08:00
Martin Brennan 10b9a32abb
FIX: Message for bulk closing topics silently (#27400)
We were using `autoclose` as the topic status update
when silently closing topics using the bulk
actions (introduced in 0464ddcd9b).

However, this resulted in a message like this showing in
the topic as a small moderator post:

> This topic was automatically closed after X days.

This is not accurate, the topic was bulk closed by someone.
Instead, we can use `closed` as the status, and a more accurate

> Closed on DATE

message is used. `TopicStatusUpdater` needed an additional
option to keep the same "fake read" behaviour as autoclose
so we can keep the same functionality for silently closing
topics in bulk actions.
2024-06-11 09:36:54 +10:00
David Taylor 565c753dd2
DEV: `@babel/plugin-proposal-decorators` -> `decorator-transforms` (#27260)
decorator-transforms (https://github.com/ef4/decorator-transforms) is a modern replacement for babel's plugin-proposal-decorators. It provides a decorator implementation using modern browser features, without needing to enable babel's full suite of class feature transformations. This improves the developer experience and performance.

In local testing with Google's 'tachometer' tool, this reduces Discourse's 'init-to-render' time by around 3-4% (230ms -> 222ms).

It reduces our initial gzip'd JS payloads by 3.2% (2.43MB -> 2.35MB), or 7.5% (14.5MB -> 13.4MB) uncompressed.

This was previously reverted in 97847f6. This version includes a babel transformation which works around the bug in Safari <= 15.

For Cloudflare compatibility issues, check https://meta.discourse.org/t/311390
2024-06-10 15:51:48 +01:00
Gerhard Schlager 7bdf47b864 FIX: `HtmlToMarkdown` should keep HTML entities for <, > and & within HTML elements
Not all HTML elements are converted into Markdown. Some are kept as HTML.
Without this fix XML/HTML entities that are formatted as text instead of code are swallowed by Discourse.
This also fixes quotes in the `title` attribute of the `<abbr>` tag.
2024-06-10 16:03:30 +02:00
Gerhard Schlager 3c9d61d302 FIX: `HtmlToMarkdown` didn't keep text from within `<center>` tag
It should ignore the `<center>` tag, but keep the text from within the element.
2024-06-10 16:03:30 +02:00
Gerhard Schlager b01905c724 FIX: `HtmlToMarkdown` didn't support `tfoot` in tables 2024-06-10 16:03:30 +02:00
Gerhard Schlager 52e81582b4 FEATURE: Use basic HTML table if it can't be converted to Markdown
Previously `HtmlToMarkdown` always converted HTML tables into Markdown tables. That lead to some badly formatted Markdown tables, e.g. when the table contained `rowspan` or `colspan`. This solves the issue by using very basic HTML tables in those cases.
2024-06-10 16:03:30 +02:00
Gerhard Schlager b8f2cbf41c DEV: Add `additional_allowed_tags` to `HtmlToMarkdown`
Import script often use subclasses of `HtmlToMarkdown` and might need to allow additional tags that can be used within the custom class.
2024-06-10 16:03:30 +02:00
Jarek Radosz 42a529f9ae
PERF: Avoid excessive object creations in watched words (#27354)
Inline the helper functions, avoid creating and then immediately destructuring arrays, use complete strings instead of string interpolation, Map instead of a pojo.
2024-06-10 14:44:31 +02:00