Commit Graph

1188 Commits

Author SHA1 Message Date
Alan Guo Xiang Tan 28f5550886
DEV: Redo `DiscourseLogstashLogger` to not rely on `logstash-logger` (#27759)
This reverts commit 92d7d24d0f.
2024-07-08 14:03:11 +08:00
David Taylor 1bc18a5bf4
DEV: Remove spec following revert in 92d7d24d0f (#27736)
This spec was introduced as part of the original commit (8e10878e1a), but was missed from the revert.
2024-07-05 12:28:16 +01:00
Alan Guo Xiang Tan bb0daa33cd
DEV: Remove `git_version` from `DiscourseLogstashLogger` log event (#27730)
In our official Docker image, running git commands results in the
following error:

```
fatal: detected dubious ownership in repository at '/var/www/discourse'
To add an exception for this directory, call:

	git config --global --add safe.directory /var/www/discourse
```
2024-07-05 15:22:03 +08:00
Alan Guo Xiang Tan 8e10878e1a
DEV: Redo `DiscourseLogstashLogger` to not rely on `logstash-logger` (#27663)
This commit rewrites `DiscourseLogstashLogger` to not be an instance
of `LogstashLogger`. The reason we don't want it to be an instance of
`LogstashLogger` is because we want the new logger to be chained to
Logster's logger which can then pass down useful information like the
request's env and error backtraces which Logster has already gathered.

Note that this commit does not bother to maintain backwards
compatibility and drops the `LOGSTASH_URI` and `UNICORN_LOGSTASH_URI`
ENV variables which were previously used to configure the destination in
which `logstash-logger` would send the logs to. Instead, we introduce
the `ENABLE_LOGSTASH_LOGGER` ENV variable to replace both ENV and remove
the need for the log paths to be specified. Note that the previous
feature was considered experimental as stated in d888d3c54c
and the new feature should be considered experimental as well. The code
may be moved into a plugin in the future.
2024-07-05 09:41:52 +08:00
Loïc Guitaut 8d249457e8 DEV: Upgrade Rails to version 7.1
---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-07-04 10:58:21 +02:00
Jan Cernik 311b737c91
SECURITY: Fix Stored-dom XSS via Facebook Oneboxes 2024-07-03 20:49:22 +08:00
Ted Johansson 67e7808603
SECURITY: Add FastImage SSRF safe freedom patch 2024-07-03 20:49:15 +08:00
Natalie Tay 751750c7f8
FIX: Do not schedule avatar download if image is '' (#27687) 2024-07-03 19:50:31 +08:00
Martin Brennan 527f02e99f
FEATURE: Only count topic views for explicit/deferred tracked views (#27533)
Followup 2f2da72747

This commit moves topic view tracking from happening
every time a Topic is requested, which is susceptible
to inflating numbers of views from web crawlers, to
our request tracker middleware.

In this new location, topic views are only tracked when
the following headers are sent:

* HTTP_DISCOURSE_TRACK_VIEW - This is sent on every page navigation when
  clicking around the ember app. We count these as browser page views
  because we know it comes from the AJAX call in our app. The topic ID
  is extracted from HTTP_DISCOURSE_TRACK_VIEW_TOPIC_ID
* HTTP_DISCOURSE_DEFERRED_TRACK_VIEW - Sent when MessageBus initializes
  after first loading the page to count the initial page load view. The
  topic ID is extracted from HTTP_DISCOURSE_DEFERRED_TRACK_VIEW.

This will bring topic views more in line with the change we
made to page views in the referenced commit and result in
more realistic topic view counts.
2024-07-03 10:38:49 +10:00
Keegan George ea58140032
DEV: Remove summarization code (#27373) 2024-07-02 08:51:47 -07:00
Jan Cernik 6599b85a75
DEV: Block accidental serialization of entire AR models (#27668) 2024-07-01 17:08:48 -03:00
Osama Sayegh 4c1c25027d
DEV: Wiring for the admin about config page (#27492)
This commit continues work laid out by ffec8163b0 for the admin config page for the /about page. The last commit set up the user interface, and this one sets up all the wiring needed to make the input fields and save buttons actually work.

Internal topic: t/128544.
2024-07-01 05:40:37 +03:00
Régis Hanol a56321efb5 FIX: topic search order
When using the full page search and filtering down to a specific topic, the sort order was overwritten to by by "post_number".

This was confusing because we allow different type of sort order in the full search page.

This fixes it by only sorting by post_number when there's no "global" sort order defined.

Since the "new topic map" uses the search endpoint behind the scene, this also fixes the "most likes" popup.

Context - https://meta.discourse.org/t/searching-order-seems-to-be-broken-when-searching-in-topic/312303
2024-06-27 18:13:26 +02:00
Loïc Guitaut f58b844f45
Revert "DEV: Upgrade Rails to version 7.1" (#27625)
This reverts commit ce00f83173.
2024-06-26 18:55:05 +02:00
Régis Hanol 54a59be617 FEATURE: new 'should_add_email_attachments' plugin modifier
That can be used by plugins to control whether email attachments should be sent.

Internal ref - t/132149
2024-06-26 12:36:35 +02:00
Martin Brennan a128ce5c4c
FIX: Missing multipart/mixed boundary on emails (#27599)
Followup 96a0781bc1

When sending emails where secure uploads is enabled
and secure_uploads_allow_embed_images_in_emails is
true, we attach the images to the email, and we
do some munging with the final email so the structure
of the MIME parts looks like this:

```
multipart/mixed
  multipart/alternative
    text/plain
    text/html
  image/jpeg
  image/jpeg
  image/png
```

However, we were not specifying the `boundary` of the
`multipart/mixed` main content-type of the email, so
sometimes the email would come through appearing to
have an empty body with the entire thing attached as
one attachment, and some mail parsers considered the
entire email as the "epilogue" and/or "preamble".

This commit fixes the issue by specifying the boundary
in the content-type header per https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
2024-06-25 13:43:10 +10:00
Loïc Guitaut ce00f83173 DEV: Upgrade Rails to version 7.1
---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-06-24 11:16:14 +02:00
Loïc Guitaut 160011793a Revert "DEV: Upgrade Rails to version 7.1 (#27539)"
This reverts commit ca4af53be8.
2024-06-21 11:20:40 +02:00
Kelv 60d5170587
DEV: add validation on content_security_policy_script_src site setting (#27564)
* DEV: add validation on content_security_policy_script_src site setting
2024-06-21 17:00:22 +08:00
Loïc Guitaut ca4af53be8 DEV: Upgrade Rails to version 7.1 (#27539)
* DEV: Upgrade Rails to 7.1

* FIX: Remove references to `Rails.logger.chained`

`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.

Some code in our initializers was still using `chained` instead of
`broadcasts`.

* DEV: Make parameters optional to all FakeLogger methods

* FIX: Set `override_level` on Logster loggers (#27519)

A followup to f595d599dd

* FIX: Don’t duplicate Rack response

---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-06-21 09:44:06 +02:00
Penar Musaraj 33de5abb6e
DEV: Extract theme resolution to a helper (#27426)
This ensures that the theme id is resolved as early as possible in the
request cycle. This is necessary for the custom homepage to skip
preloading the wrong data.
2024-06-20 11:33:46 -04:00
Loïc Guitaut 982c005979 Revert "DEV: Upgrade Rails to version 7.1 (#27539)"
This reverts commit 2301dddcff.
2024-06-20 11:43:35 +02:00
Loïc Guitaut 2301dddcff
DEV: Upgrade Rails to version 7.1 (#27539)
* DEV: Upgrade Rails to 7.1

* FIX: Remove references to `Rails.logger.chained`

`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.

Some code in our initializers was still using `chained` instead of
`broadcasts`.

* DEV: Make parameters optional to all FakeLogger methods

* FIX: Set `override_level` on Logster loggers (#27519)

A followup to f595d599dd

* FIX: Don’t duplicate Rack response

---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-06-20 10:33:01 +02:00
Rafael dos Santos Silva b2a9676f0b
FEATURE: Support Spotify Onebox (#27540) 2024-06-19 13:27:27 -03:00
Ted Johansson 96a0781bc1
FIX: Avoid duplicating e-mail body in summary e-mail (#27535)
We recently fixed a problem where secure upload images weren't re-attached when sending the activity summary e-mail.

This fix contained a bug that would lead to n copies of the e-mail body being included, n being the number of duplicates. This is because #fix_parts_after_attachments! was called once per attachment, and adding more parts to the multipart e-mail.

This PR fixes that by:

Adding a failing test case for the above.
Moving the looping over multiple posts into #fix_parts_after_attachments! itself.
2024-06-19 20:11:47 +08:00
Bianca Nenciu 2ce83b4d2e
FIX: Do not extract links for hotlinked images (#27538)
When a post is cooked the links are extracted and `TopicLink` instances
are created for each of them. These links are used in various places,
including the topic view, user summary page, etc.

In previous commit 48e5d1a, hotlinked images from Oneboxes have been
ignored from the texts, but hotlinked images turned into Lightboxes
were still extracted.
2024-06-19 14:25:29 +03:00
Ted Johansson 9cc030fe8d
DEV: Ensure digest e-mail secure image test uses actual digest e-mail (#27532)
The test that checks that securely uploaded images are re-attached to the digest e-mail wasn't rendering the actual digest e-mail template. This change fixes that.
2024-06-19 14:33:57 +08:00
Martin Brennan 83361b2fc5
FEATURE: Introduce site settings which require confirmation (#27315)
Many site settings can be distructive or have huge side-effects
for a site that the admin may not be aware of when changing it.

This commit introduces a `requires_confirmation` attribute that
can be added to any site setting. When it is true, a confirmation
dialog will open if that setting is changed in the admin UI,
optionally with a custom message that is defined in client.en.yml.

If the admin does not confirm, we reset the setting to its previous
clean value and do not save the new value.
2024-06-19 16:01:24 +10:00
Krzysztof Kotlarek cc4c199680
FEATURE: optional 2FA enforcement (#27506)
A new admin setting called `enforce_second_factor_on_external_auth`. It allows users to authenticate using external providers even when 2FA is forced with `enforce_second_factor` site setting.
2024-06-19 09:32:30 +10:00
Jarek Radosz 5cb84f8dcf
DEV: Revert rails 7.1 upgrade (#27522)
* Revert "FIX: Set `override_level` on Logster loggers (#27519)"

This reverts commit c1b0488c54.

* Revert "DEV: Make parameters optional to all FakeLogger methods"

This reverts commit 3318dad7b4.

* Revert "FIX: Remove references to `Rails.logger.chained`"

This reverts commit f595d599dd.

* Revert "DEV: Upgrade Rails to 7.1"

This reverts commit 081b00391e.
2024-06-18 23:48:30 +02:00
Loïc Guitaut 081b00391e DEV: Upgrade Rails to 7.1 2024-06-18 15:58:05 +02:00
Loïc Guitaut 2a22a3b51d FIX: Treat corrupt cache as cache miss
Currently when a cache entry is corrupt, we log the event without doing
anything else. It means the cache is still corrupt, and the proper value
isn’t computed again.

Normally, it’s very rare the cache becomes corrupt, but it can happen
when upgrading Rails for example and the cache format changes. This is
normally handled automatically by Rails but since we’re using a custom
cache class, we have to do it ourselves.

This patch takes the same approach the Rails team did, when a cache
entry is corrupt, we treat it as a miss, recomputing the proper value
and caching it in the new format.
2024-06-18 14:47:33 +02:00
Régis Hanol 53b3d2f0dc FIX: BBCode tag parser
Wasn't quite handling the cases where a closing bracket `]` was used in the value of one of the attributes.

```markdown
[chat quote=user channel="[broken]"]
```

Would not be correctly parsed because we would _greedily_ use the first `]` as the end of the tag even though it might be a valid character when inside proper quotes.

c39a4de139/app/assets/javascripts/discourse-markdown-it/src/features/bbcode-block.js (L62)

Re-wrote the `parseBBCodeTag` to properly handle the following cases

- A closing tag (aka `[/name]`) which are easy since they don't have any attributes
- An old `[quote=...]` format we used that doesn't uses quotes but still has various attributes of the form `key:value`
- All three valid BBCode opening tag formats we support
  - `[name]` without any attributes
  - `[name=foo]` with a default value
  - `[name foo=bar]` with some attributes

Ended up having to fix/rewrite the few bbcode rules that were using the `parseBBCodeTag` function, namely `d-wrap` and `discourse-local-dates`.

While working on this, I think I also found a way to get rid the of shims we had in place so that plugins could use the `parseBBCodeTag` function.

Reference - https://meta.discourse.org/t/having-a-right-bracket-in-a-channel-name-breaks-all-quotes-from-that-channel/308439
2024-06-18 10:47:18 +02:00
Kelv 2393234be5
DEV: remove legacy CSP implementation to make strict-dynamic only accepted behaviour (#27486)
* DEV: remove legacy CSP implementation that allowed for non-strict-dynamic behaviour
2024-06-18 16:40:53 +08:00
Renato Atilio 49fdccbb1d
FIX: restrict a href protocols on form template description (#27472) 2024-06-14 11:39:43 -03:00
Gabriel Grubba 93c2ae585c
FEATURE: Add tag_group option in `/filter` (#27427)
* FEATURE: Add tag_group option in `/filter`

* DEV: Update tag_group_filter in `/filter` to use SQL

* DEV: Add guardian to `/filter` by tag_group

* DEV: remove unused code

* DEV: Update tag_group filter implementation

* DEV: Add tests to tag_group filter
2024-06-13 14:03:49 -03:00
Loïc Guitaut c790c57415 DEV: Show parameters on a service contract failure
Now, when calling `StepsInspector#error` on a contract failure, the
output will contain the parameters provided to the contract.
2024-06-13 12:19:26 +02:00
Alan Guo Xiang Tan ae1d8c50da
DEV: Add ability to log a warning when Sidekiq job runs for too long (#27446)
This commits introduces the `sidekiq_report_long_running_jobs_minutes`
global setting which allows a site administrator to log a warning in the
Rails log when a Sidekiq job has been running for too long.

The warning is logged with the backtrace of the thread that is
processing the Sidekiq job to make it easier to figure out what a
sidekiq job is stuck on.
2024-06-13 14:24:44 +08:00
Martin Brennan e94ab11477
DEV: Allow for setting a message with SiteSetting.set_and_log (#27447)
When we turn on settings automatically for customers,
we sometimes use `.set_and_log` which will make a staff
action log for the site setting change. This is fine, but
there is no context for customers.

This change allows setting a message with `.set_and_log`, which
will be stored in the `details` column of the staff action log
created, which will show up on `/admin/logs/staff_action_logs`

---------

Co-authored-by: Kelv <kelv@discourse.org>
2024-06-13 14:59:49 +10:00
Ted Johansson a5df029be3
FIX: Email::Sender expects type to be a string (#27463)
In #26642 we introduced a change that re-attaches securely uploaded images in the digest e-mail. However, this change assumed that the type argument to the Email::Sender constructor would be a symbol, but when it is coming from the UserEmail job it is a string. This PR fixes that.
2024-06-13 11:53:25 +08:00
Alan Guo Xiang Tan 1f4aa343d8
DEV: Move spec file to right folder (#27449)
Spec file is not being run because it was in the wrong folder.
2024-06-13 06:38:03 +08:00
Sam dc8249c08a
FEATURE: align with /filter and allow multiple category search (#27440)
This introduces the syntax of

`category:a,b,c` which will search across multiple categories.

Previously there was no way to allow search across a wide selection of
categories.
2024-06-12 16:06:04 +10:00
Alan Guo Xiang Tan 0b8e6e7d1d
DEV: Fix broken `discourse_ip_info_spec` (#27439)
This is a follow-up to 781a5f60ea
2024-06-12 07:51:45 +08:00
Alan Guo Xiang Tan 781a5f60ea
DEV: Move RSpec test file into test folder (#27437)
Test was not being run because it was in the wrong folder
2024-06-12 06:58:44 +08:00
Arkshine 1fffb236b2 FIX: crawler requests exceptions for non UTF-8 user agents with invalid bytes 2024-06-11 14:02:46 +02:00
David Taylor 565c753dd2
DEV: `@babel/plugin-proposal-decorators` -> `decorator-transforms` (#27260)
decorator-transforms (https://github.com/ef4/decorator-transforms) is a modern replacement for babel's plugin-proposal-decorators. It provides a decorator implementation using modern browser features, without needing to enable babel's full suite of class feature transformations. This improves the developer experience and performance.

In local testing with Google's 'tachometer' tool, this reduces Discourse's 'init-to-render' time by around 3-4% (230ms -> 222ms).

It reduces our initial gzip'd JS payloads by 3.2% (2.43MB -> 2.35MB), or 7.5% (14.5MB -> 13.4MB) uncompressed.

This was previously reverted in 97847f6. This version includes a babel transformation which works around the bug in Safari <= 15.

For Cloudflare compatibility issues, check https://meta.discourse.org/t/311390
2024-06-10 15:51:48 +01:00
Gerhard Schlager 7bdf47b864 FIX: `HtmlToMarkdown` should keep HTML entities for <, > and & within HTML elements
Not all HTML elements are converted into Markdown. Some are kept as HTML.
Without this fix XML/HTML entities that are formatted as text instead of code are swallowed by Discourse.
This also fixes quotes in the `title` attribute of the `<abbr>` tag.
2024-06-10 16:03:30 +02:00
Gerhard Schlager 3c9d61d302 FIX: `HtmlToMarkdown` didn't keep text from within `<center>` tag
It should ignore the `<center>` tag, but keep the text from within the element.
2024-06-10 16:03:30 +02:00
Gerhard Schlager b01905c724 FIX: `HtmlToMarkdown` didn't support `tfoot` in tables 2024-06-10 16:03:30 +02:00
Gerhard Schlager 52e81582b4 FEATURE: Use basic HTML table if it can't be converted to Markdown
Previously `HtmlToMarkdown` always converted HTML tables into Markdown tables. That lead to some badly formatted Markdown tables, e.g. when the table contained `rowspan` or `colspan`. This solves the issue by using very basic HTML tables in those cases.
2024-06-10 16:03:30 +02:00