Commit Graph

205 Commits

Author SHA1 Message Date
Bianca Nenciu 3048d3d07d
FEATURE: Track API and user API requests (#19186)
Adds stats for API and user API requests similar to regular page views.
This comes with a new report to visualize API requests per day like the
consolidated page views one.
2022-11-29 13:07:42 +02:00
Osama Sayegh 4cd07627d5
DEV: Add integration specs for Discord, Facebook and Twitter logins (#18941)
Internal topic: t/82084.
2022-11-11 13:50:17 +03:00
Osama Sayegh e7e24843dc
DEV: Add integration specs for Github login (#18808)
Internal topic: t/82084.
2022-11-02 16:21:51 +03:00
Martin Brennan f5194aadd3
DEV: Remove usages of enable_personal_messages (#18437)
cf. e62e93f83a

This PR also makes it so `bot` (negative ID) and `system` users are always allowed
to send PMs, since the old conditional was just based on `enable_personal_messages`
2022-10-05 10:50:20 +10:00
Loïc Guitaut 26fe047724 DEV: Use AR enums in reviewables related code
This is the first patch of many to replace our custom enums in Ruby by
the ones provided by `ActiveRecord`.
2022-09-22 14:44:27 +02:00
Loïc Guitaut 3eaac56797 DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
Phil Pirozhkov 493d437e79
Add RSpec 4 compatibility (#17652)
* Remove outdated option

04078317ba

* Use the non-globally exposed RSpec syntax

https://github.com/rspec/rspec-core/pull/2803

* Use the non-globally exposed RSpec syntax, cont

https://github.com/rspec/rspec-core/pull/2803

* Comply to strict predicate matchers

See:
 - https://github.com/rspec/rspec-expectations/pull/1195
 - https://github.com/rspec/rspec-expectations/pull/1196
 - https://github.com/rspec/rspec-expectations/pull/1277
2022-07-28 10:27:38 +08:00
Loïc Guitaut 91b6b5eee7 DEV: Don’t use `change { … }.by(0)` in specs 2022-07-26 10:34:15 +02:00
David Taylor 6695d148d6
DEV: Cleanup topic thumbnail enqueuing (#17119)
- Ensure only one job is enqueued when there are no extra sizes requested by themes
- Update and un-skip relevant tests
2022-06-17 11:07:32 +01:00
David Taylor 5238f6788c
FEATURE: Allow hotlinked media to be blocked (#16940)
This commit introduces a new site setting: `block_hotlinked_media`. When enabled, all attempts to hotlink media (images, videos, and audio) will fail, and be replaced with a linked placeholder. Exceptions to the rule can be added via `block_hotlinked_media_exceptions`.

`download_remote_image_to_local` can be used alongside this feature. In that case, hotlinked images will be blocked immediately when the post is created, but will then be replaced with the downloaded version a few seconds later.

This implementation is purely server-side, and does not impact the composer preview.

Technically, there are two stages to this feature:

1. `PrettyText.sanitize_hotlinked_media` is called during `PrettyText.cook`, and whenever new images are introduced by Onebox. It will iterate over all src/srcset attributes in the post HTML and check if they're allowed. If not, the attributes will be removed and replaced with a `data-blocked-hotlinked-src(set)` attribute

2. In the `CookedPostProcessor`, we iterate over all `data-blocked-hotlinked-src(set)` attributes and check whether we have a downloaded version of the media. If yes, we update the src to use the downloaded version. If not, the entire media element is replaced with a placeholder. The placeholder is labelled 'external media', and is a link to the offsite media.
2022-06-07 15:23:04 +01:00
Loïc Guitaut 0403a8633b DEV: Apply Rails 6.1 defaults
We never applied `config.load_defaults` since its inception (Rails 5.0)
and doing so is necessary to properly upgrade to all the Rails 7 new
defaults.
2022-05-24 17:13:44 +02:00
Martin Brennan a6be4972a8
FIX: Use our header value instead of custom header on duplicates (#16711)
When we build and send emails using MessageBuilder and Email::Sender
we add custom headers defined in SiteSetting.email_custom_headers.
However this was causing errors in cases where the custom headers
defined a header that we already specify in outbound emails (e.g.
the Precedence: list header for topic/post emails).

This commit makes it so we always use the header value defined in Discourse
core if there is a duplicate, discarding the custom header value
from the site setting.

cf. https://meta.discourse.org/t/email-notifications-fail-if-duplicate-headers-exist/222960/14
2022-05-11 13:47:12 +10:00
Isaac Janzen 692e0140e2
FEATURE: Enables support for dark mode emails (#16520)
This PR enables custom email dark mode styles by default that were added here.

There is currently poor support for dark mode queries in mail clients. The main beneficiary of these changes will be Apple Mail and Outlook.

Enjoy the darkness 🕶️
2022-04-20 13:00:04 -05:00
David Taylor 68c74e9b93
FEATURE: Allow multiple required tag groups for a category (#16381)
Previously we only supported a single 'required tag group' for a category. This commit allows admins to specify multiple required tag groups, each with their own minimum tag count.

A new category_required_tag_groups database table replaces the existing columns on the categories table. Data is automatically migrated.
2022-04-06 14:08:06 +01:00
Osama Sayegh e40c4bb7f9
FIX: Validate category tag restrictions before sending new topics to review (#16292)
Tags (and tag groups) can be configured so that they can only be used in specific categories and (optionally) restrict topics in these categories to be able to add/use only these tags. These restrictions work as expected when a topic is created without going through the review queue; however, if the topic has to be reviewed by a moderator then these restrictions currently aren't checked before the topic is sent to the review queue, but they're checked later when a moderator tries to approve the topic. This is because if a user manages to submit a topic that doesn't meet the restrictions, moderators won't be able to approve and it'll be stuck in the review queue.

This PR prevents topics that don't meet the tags requirements from being sent to the review queue and shows the poster an error message that indicates which tags that cannot be used.

Internal ticket: t60562.
2022-03-28 21:25:26 +03:00
David Taylor c9dab6fd08
DEV: Automatically require 'rails_helper' in all specs (#16077)
It's very easy to forget to add `require 'rails_helper'` at the top of every core/plugin spec file, and omissions can cause some very confusing/sporadic errors.

By setting this flag in `.rspec`, we can remove the need for `require 'rails_helper'` entirely.
2022-03-01 17:50:50 +00:00
David Taylor 64be371749
DEV: Improve handling of invalid requests (#15841)
Our discourse_public_exceptions middleware is designed to catch bubbled exceptions from lower in the stack, and then use `ApplicationController.rescue_with_handler` to render an appropriate error response.

When the request itself is invalid, we had an escape-hatch to skip re-dispatching the request to ApplicationController. However, it was possible to work around this by 'layering' the errors. For example, if you made a request which resulted in a 404, but **also** had some other invalidity, the escape hatch would not be triggered.

This commit ensures that these kind of 'layered' errors are properly handled, without logging warnings. It also adds detection for invalid JSON bodies and badly-formed multipart requests.

The user-facing behavior is unchanged. This commit simply prevents warnings being logged for invalid requests.
2022-02-07 13:16:57 +00:00
Daniel Waterworth 6cae6aadf4
FIX: Make thumbnail tests start with a clean slate (#15216)
* FIX: Make thumbnail tests start with a clean slate

Unfortunately, this exposes the fact that they don't actually work.
Marking as pending for now.
2021-12-07 13:07:45 -06:00
Daniel Waterworth 3ebce550fe
DEV: Make add_api_parameter_route parameter deprecations errors (#15198)
Since we said we would remove support in 2.7, this is overdue.
2021-12-06 09:10:14 -06:00
David Taylor cfb6199a95
FIX: Don't redirect XHR/JSON requests when login is required (#15093)
When redirecting to login, we store a destination_url cookie, which the user is then redirected to after login. We never want the user to be redirected to a JSON URL. Instead, we should return a 403 in these situations.

This should also be much less confusing for API consumers - a 403 is a better representation than a 302.
2021-12-02 15:12:25 +00:00
David Taylor f45853676f
SECURITY: Ensure _forum_session cookies cannot be reused between sites (#14950)
This only affects multisite Discourse instances (where multiple forums are served from a single application server). The vast majority of self-hosted Discourse forums do not fall into this category.

On affected instances, this vulnerability could allow encrypted session cookies to be re-used between sites served by the same application instance.
2021-11-15 15:50:12 +00:00
David Taylor 13fdc979a8
DEV: Improve multisite testing (#14884)
This commit adds the RailsMultisite middleware in test mode when Rails.configuration.multisite is true. This allows for much more realistic integration testing. The `multisite_spec.rb` file is rewritten to avoid needing to simulate a middleware stack.
2021-11-11 16:44:58 +00:00
Bianca Nenciu 1f57b29147
SECURITY: Escape watched word in error message (#14434) 2021-09-24 11:55:15 +03:00
Jarek Radosz 48b92d8897
DEV: Isolate multisite specs (#13634)
Mixing multisite and standard specs can lead to issues (e.g. when using `fab!`)
Disabled the (upcoming https://github.com/discourse/rubocop-discourse/pull/11) rubocop rule for two files that have thoroughly tangled both types of specs.
2021-07-07 18:57:42 +02:00
David Taylor 4134173bbf
FEATURE: Add global admin api key rate limiter (#12527) 2021-06-03 10:52:43 +01:00
Andrei Prigorshnev 075cd07a07
No need to disable rate limiter after running tests (#13093)
We disable rate limiter before running every test here 90ab3b1c75/spec/rails_helper.rb (L109-L109)
2021-05-19 16:04:35 +04:00
Jarek Radosz 6ff888bd2c
DEV: Retry-after header values should be strings (#12475)
Fixes `Rack::Lint::LintError: a header value must be a String, but the value of 'Retry-After' is a Integer`. (see: 14a236b4f0/lib/rack/lint.rb (L676))

I found it when I got flooded by those warning a while back in a test-related accident 😉 (ember CLI tests were hitting a local rails server at a fast rate)
2021-03-23 20:32:36 +01:00
Arpit Jalan f75e1867ff
FIX: do not send rejection emails to auto-deleted reviewable users (#12160)
FIX: add context when user is deleted via auto handle queued reviewable
FIX: do not delete email_log when a user is deleted
2021-02-22 18:37:47 +05:30
Martin Brennan fb83757edb
FIX: Auto close topic from category settings based on topic created_at (#12082)
Previously when inheriting category auto-close settings for a topic, those settings were disrupted if another topic timer was assigned or if a topic was closed then manually re-opened.

This PR makes it so that when a topic is manually re-opened the topic auto-close settings are inherited from the category. However, they will now be based on the topic created_at date. As an example, for a topic with a category auto close hours setting of 72 (3 days):

* Topic was created on 2021-02-15 08:00
* Topic was closed on 2021-02-16 10:00
* Topic was opened again on 2021-02-17 06:00

Now, the topic will inherit the auto close timer again and will close automatically at **2021-02-18 08:00**, which is based on the creation date. If the current date and time is greater than the original auto-close time (e.g. we were at 2021-02-20 13:45) then no auto-close timer is created.

Note, this will not happen if the topic category auto-close setting is "based on last post".
2021-02-17 07:51:39 +10:00
Martin Brennan 0034cbda8a
DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698)
Moves the topic timer jobs from being scheduled ahead of time with enqueue_at to a 5 minute scheduled run like bookmark reminders, in a new job called Jobs::EnqueueTopicTimers. Backwards compatibility is maintained by checking if an existing topic timer job is enqueued in sidekiq for the timer, and if it is not running it inside the new job.

The functionality to close/open a topic if it is in the opposite state still remains in the after_save block of TopicTimer, with further commentary, which is used for Open/Close Temporarily.

This also removes the ensure_consistency! functionality of topic timers as it is no longer needed; the new job will always pick up the timers because they are not stored in a fragile state of sidekiq.
2021-01-19 13:30:58 +10:00
Martin Brennan 2404fa7a23
DEV: Split toggle topic close job (#11679)
Splits the `ToggleTopicClosed` job into two distinct `OpenTopic` and `CloseTopic` jobs to make the code clearer. The old job cannot be deleted yet because of outstanding sidekiq schedules, so a todo has been added to do so later this year.

Also replaced mentions of `topic_status_update` with `topic_timer` in some files, because the `topic_status_update` model is obsolete and replaced by topic timer.

Added some shortcut methods for checking if a topic is open/whether a user can change an open topic.
2021-01-13 08:49:29 +10:00
Krzysztof Kotlarek 8253f8fc5c
PERF: we don't need to use a huge image to test thumbnails (#11025)
* PERF: we don't need to use a huge image to test thumbnails

Generating images with 5000x5000 dimensions is an expensive operation.

Using smaller images reduce the time of model spec from 11s to 3s and integration spec from 6s to 2s.
2020-10-27 12:39:52 +11:00
David Taylor b7c680853d DEV: Introduce plugin API to contribute user api key scopes 2020-10-19 10:40:55 +01:00
David Taylor 23e5c605f6 DEV: Add support for allowed parameters in user api key scopes
Initially, this feature is only intended for use in core/plugins, so there is no API for requesting a parameter-scoped key. That may change in future.
2020-10-19 10:40:55 +01:00
Gerhard Schlager ce1620f2ad
FIX: Pluralized translation overrides didn't work for en_US
"en_US" doesn't contain most of the translations, so it falls back to "en". But that behavior stopped translation overrides to work for pluralized strings in "en_US", because it relies on existing translations. This fixes it by looking up the existing translation in all fallback locales.
2020-08-29 00:11:46 +02:00
Guo Xiang Tan 83314d1534
DEV: Fix the broken build caused by 1e8f216. 2020-08-26 12:36:51 +08:00
David Taylor 629ee5494d
FEATURE: Allow plugins to register parameter-based API routes (#10505)
Example usage:

```
add_api_parameter_route(
  method: :get,
  route: "users#bookmarks",
  format: :ics
)
```
2020-08-24 10:24:52 +01:00
Daniel Waterworth 9313706649 DEV: Enable preserve_email_structure_when_styling by default
In 1bd8a075, a hidden site setting was added that causes Email::Styles
to treat its input as a complete document in all cases.

This commit enables that setting by default.

Some tests were removed that were broken by this change. They tested the
behaviour of applying email styles to empty strings. They weren't useful
because:

 * Sending empty email is not something we ever intend to do,
 * They were testing incidental behaviour - there are lots of
   valid ways to process the empty string,
 * Their intent wasn't clear from their descriptions,
2020-07-20 10:21:32 +01:00
David Taylor cb1b472a0f
PERF: Do not include thumbnail information in default topic list payload (#10163)
Now it is only included when a theme/plugin has requested it.
2020-07-06 10:59:21 +01:00
Sam Saffron 38a30a6e96
DEV: correct regression and correct tests
etag change in 31976ecf was incorrect, revert it

Also correct regression in test suite.
2020-07-06 10:56:19 +10:00
Robin Ward 4e3a84c687 FIX: If creating a flag for a watched word, include the reason 2020-06-02 11:49:02 -04:00
Robin Ward fd2d7ca992 FIX: Email Styles were evaluated out of order
`yield` puts the content in the template right away unless explicitly
`capture`'d.
2020-05-25 12:47:23 -04:00
Michael Brown d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood 20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Robin Ward d11c462104 SECURITY: ERB execution in custom Email Style 2020-05-21 14:48:30 -04:00
David Taylor e9f579bd3f
DEV: Clean up thumbnail tests correctly 2020-05-19 11:04:02 +01:00
David Taylor 725e38f9d7
DEV: Allow plugins to request topic thumbnail sizes (#9828)
In plugin.rb, you can register new sizes like

```
register_topic_thumbnail_size [512, 512]
```

For more information about thumbnails see 03818e642a
2020-05-19 10:38:58 +01:00
David Taylor 0495a748d0
FIX: Use CDN URLs for topic thumbnails 2020-05-15 13:35:20 +01:00
David Taylor 6230f5c554
FEATURE: Allow parameter authentication for UserApiKeys (#9742)
This refactors default_current_user_provider in a few ways:
- Introduce a generic `api_parameter_allowed?` method which checks for whitelisted routes/formats
- Only read the api_key parameter on allowed routes. It is now completely ignored on other routes (previously it would raise a 403)
- Start reading user_api_key parameter on allowed routes
- Refactor tests as end-end integration tests

A plugin API for PARAMETER_API_PATTERNS will be added soon
2020-05-12 13:35:36 +01:00
Jarek Radosz 781e3f5e10
DEV: Use `response.parsed_body` in specs (#9615)
Most of it was autofixed with rubocop-discourse 2.1.1.
2020-05-07 17:04:12 +02:00