Commit Graph

8478 Commits

Author SHA1 Message Date
Natalie Tay 4c46c7e334
DEV: Remove xlink hrefs (#15059) 2021-11-25 15:22:43 +11:00
Dan Ungureanu 3f97f884fe
DEV: Fail stop if theme update fails (#15074)
This applies only when a single site exists. If a theme update fails
when there are multiple sites, then it will continue updating the
remaining themes.
2021-11-24 19:12:49 +02:00
janzenisaac 8c7cc426b7
FEATURE: Notify responders of post removal (#15049)
- Notify users whose posts were cascade deleted due to a flagged post
2021-11-24 09:28:20 -06:00
Alan Guo Xiang Tan a6aff40e4b
DEV: Allow default scope to be configurable per topic. (#15018)
Not exposing this as a plugin API yet as we're testing it out with a
plugin.
2021-11-24 16:40:58 +08:00
Martin Brennan 44be79f095
FIX: Strip Auto-Submitted email header from group SMTP emails (#15057)
Remove Auto-Submitted header for group private message emails, it does
not make sense there and may hurt deliverability.

From https://www.iana.org/assignments/auto-submitted-keywords/auto-submitted-keywords.xhtml:

> Indicates that a message was generated by an automatic process, and is not a direct response to another message.
2021-11-24 10:54:01 +10:00
Jarek Radosz d40e56272f
DEV: Remove unnecessary requires (#15067)
`auth/*` one was causing warnings (the file was loaded twice)

and `sha1` isn't used anymore here
2021-11-23 23:42:24 +01:00
Jarek Radosz 1c0dcbfd47
DEV: Remove `build_test_topic` task (#15068)
I don't think anyone is using it anymore
2021-11-23 23:41:49 +01:00
Bianca Nenciu 73760c77d9
FEATURE: Mention @here to notify users in topic (#14900)
Use @here to mention all users that were allowed to topic directly or
through group, who liked topics or read the topic. Only first 10 users
will be notified.
2021-11-23 22:25:54 +02:00
Dan Ungureanu fa2fd7fff8
FIX: Do not fail-stop if theme update fails (#15063) 2021-11-23 13:55:09 +02:00
Dan Ungureanu ff7acc9828
FIX: Git should not prompt for credentials (#15062)
When cloning a public remote repository (no key), git should not prompt
for credentials.
2021-11-23 13:54:51 +02:00
Dan Ungureanu 948a1523ea
FIX: Keep existent tags when editing tag topics (#15050)
Allow current user to keep existent tags when adding or removing a tag.
For example, a user could not remove a tag from a topic if the topic
had another tag that was restricted to a different category.
2021-11-23 13:00:45 +02:00
Mark VanLandingham 4da23e811b
DEV: Create CookedProcessMixin to process generic cooked (#15029) 2021-11-22 13:32:12 -06:00
Dan Ungureanu d420a7b2c8
DEV: Reuse code for TrustLevelAndStaffSetting (#15044)
The code that checked this permission was duplicated everytime a new
settings of this type was added. This commit changes the behavior of
some functionality because some feature checks were bypassed for staff
members.
2021-11-22 20:18:53 +02:00
Blake Erickson da9cd4f692
DEV: Swap out optipng with oxipng (#15013)
* DEV: Swap out optipng with oxipng

The oxipng binary has been added to our base docker image here:

244c9cb110

oxipng is a rust replacement for optipng that provides increased
performance and multi-threading. Checkout
https://github.com/shssoichiro/oxipng for more info.

* add instructions for installing oxipng
2021-11-22 10:16:35 -07:00
Jarek Radosz 648332bc9f
DEV: Don't clear cache/trigger events if site setting hasn't changed (#15045) 2021-11-22 16:43:12 +01:00
Jarek Radosz d91d67a442
FEATURE: Allow theme settings to request refresh (#15037)
Similar to site settings, adds support for `refresh` option to theme settings.

```yaml
super_feature_enabled:
  type: bool
  default: false
  refresh: true
```
2021-11-22 13:16:56 +01:00
Dan Ungureanu f43c433d50
FIX: Allow invites if must_approve_users is true (#15043)
This was implemented in 36e0e6a, but then regressed when the guardian
for invites was refactored in 76a7b75.
2021-11-22 14:04:35 +02:00
Martin Brennan fa6b87a1bf
SECURITY: Strip unrendered unicode bidirectional chars in code blocks (#15032)
When rendering the markdown code blocks we replace the
offending characters in the output string with spans highlighting a textual
representation of the character, along with a title attribute with
information about why the character was highlighted.

The list of characters stripped by this fix, which are the bidirectional
characters considered relevant, are:

U+202A
U+202B
U+202C
U+202D
U+202E
U+2066
U+2067
U+2068
U+2069
2021-11-22 10:43:03 +10:00
Penar Musaraj 20f5474be9
FEATURE: Log only topic/post search queries in search log (#14994) 2021-11-18 09:21:12 +08:00
Martin Brennan b96c10a903
DEV: Extract shared external upload routes into controller helper (#14984)
This commit refactors the direct external upload routes (get presigned
put, complete external, create/abort/complete multipart) into a
helper which is then included in both BackupController and the
UploadController. This is done so UploadController doesn't need
strange backup logic added to it, and so each controller implementing
this helper can do their own validation/error handling nicely.

This is a follow up to e4350bb966
2021-11-18 09:17:23 +10:00
Osama Sayegh b86127ad12
FEATURE: Apply rate limits per user instead of IP for trusted users (#14706)
Currently, Discourse rate limits all incoming requests by the IP address they
originate from regardless of the user making the request. This can be
frustrating if there are multiple users using Discourse simultaneously while
sharing the same IP address (e.g. employees in an office).

This commit implements a new feature to make Discourse apply rate limits by
user id rather than IP address for users at or higher than the configured trust
level (1 is the default).

For example, let's say a Discourse instance is configured to allow 200 requests
per minute per IP address, and we have 10 users at trust level 4 using
Discourse simultaneously from the same IP address. Before this feature, the 10
users could only make a total of 200 requests per minute before they got rate
limited. But with the new feature, each user is allowed to make 200 requests
per minute because the rate limits are applied on user id rather than the IP
address.

The minimum trust level for applying user-id-based rate limits can be
configured by the `skip_per_ip_rate_limit_trust_level` global setting. The
default is 1, but it can be changed by either adding the
`DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL` environment variable with the
desired value to your `app.yml`, or changing the setting's value in the
`discourse.conf` file.

Requests made with API keys are still rate limited by IP address and the
relevant global settings that control API keys rate limits.

Before this commit, Discourse's auth cookie (`_t`) was simply a 32 characters
string that Discourse used to lookup the current user from the database and the
cookie contained no additional information about the user. However, we had to
change the cookie content in this commit so we could identify the user from the
cookie without making a database query before the rate limits logic and avoid
introducing a bottleneck on busy sites.

Besides the 32 characters auth token, the cookie now includes the user id,
trust level and the cookie's generation date, and we encrypt/sign the cookie to
prevent tampering.

Internal ticket number: t54739.
2021-11-17 23:27:30 +03:00
Jarek Radosz 05423e9dfd
DEV: `I18n` global is no longer supported (#14993)
Don't allow it when linting.
2021-11-17 20:52:22 +01:00
Vinoth Kannan a3a4f16e88
UX: add more search aliases for emojis. (#14768) 2021-11-17 11:54:43 +01:00
Osama Sayegh 8f5412cccf
DEV: Skip logging in test environment (#14971)
It adds unnecessary noise when running the test suite.
2021-11-16 18:01:48 +03:00
Vinoth Kannan c47a526371
FIX: exclude topics from muted tag in category featured list. (#14925)
Topics from muted tags were visible in the categories page's featured topics section since we didn't filter it before.
2021-11-16 12:10:50 +05:30
Neil Lalonde cbfe48b990
Version bump to v2.8.0.beta8 (#14948) 2021-11-15 11:13:34 -05:00
Jarek Radosz 4ad77f3382
DEV: Remove `.es6` extensions from core (#14912)
Still supported in plugins though.
2021-11-13 12:51:53 +01:00
Rafael dos Santos Silva 6645243a26
SECURITY: Disallow caching of MIME/Content-Type errors (#14907)
This will sign intermediary proxies and/or misconfigured CDNs to not
cache those error responses.
2021-11-12 15:52:25 -03:00
Daniel Waterworth e7c0bbb9c0
DEV: Let's always give a drop_from param to deprecate (#14901)
So that we know when deprecations can be removed in the future.
2021-11-12 08:52:59 -06:00
Daniel Waterworth 388f45c570
DEV: Remove deprecated methods (#14885) 2021-11-11 12:21:25 -06:00
Osama Sayegh 69ec6899f9
Revert "DEV: increase lock timeout for multisite migration (#14831)" (#14883)
* Revert "DEV: increase lock timeout for multisite migration (#14831)"

This partially reverts commit 337ef60303.

We need to revert the mutex around `db:status:json` because the mutex is not available unless the rails environment is loaded which the `db:status:json` doesn't load before the mutex. We can't load the environment before entering the mutex because the mutex is meant to prevent other instances of the task from loading a rails environment while the database is migrating.

Co-authored-by: David Taylor <david@taylorhq.com>

Co-authored-by: David Taylor <david@taylorhq.com>
2021-11-11 16:16:53 +03:00
Martin Brennan e4350bb966
FEATURE: Direct S3 multipart uploads for backups (#14736)
This PR introduces a new `enable_experimental_backup_uploads` site setting (default false and hidden), which when enabled alongside `enable_direct_s3_uploads` will allow for direct S3 multipart uploads of backup .tar.gz files.

To make multipart external uploads work with both the S3BackupStore and the S3Store, I've had to move several methods out of S3Store and into S3Helper, including:

* presigned_url
* create_multipart
* abort_multipart
* complete_multipart
* presign_multipart_part
* list_multipart_parts

Then, S3Store and S3BackupStore either delegate directly to S3Helper or have their own special methods to call S3Helper for these methods. FileStore.temporary_upload_path has also removed its dependence on upload_path, and can now be used interchangeably between the stores. A similar change was made in the frontend as well, moving the multipart related JS code out of ComposerUppyUpload and into a mixin of its own, so it can also be used by UppyUploadMixin.

Some changes to ExternalUploadManager had to be made here as well. The backup direct uploads do not need an Upload record made for them in the database, so they can be moved to their final S3 resting place when completing the multipart upload.

This changeset is not perfect; it introduces some special cases in UploadController to handle backups that was previously in BackupController, because UploadController is where the multipart routes are located. A subsequent pull request will pull these routes into a module or some other sharing pattern, along with hooks, so the backup controller and the upload controller (and any future controllers that may need them) can include these routes in a nicer way.
2021-11-11 08:25:31 +10:00
Rafael dos Santos Silva d4e35f50c2
PERF: Update like count in visible posts without an extra GET per like (#14869)
PERF: Update like count in visible posts without an extra GET per like

Currently when a user is reading a topic and some post in it receive a
like from another user, the Ember app will be notified via MessageBus
and issue a GET to `/posts/{id}` to get the new like count. This worked
fine for us until today, but it can easily create a self-inflicted DDoS
when a topic with a large number of visitors gets a large number of
likes, since we will issue `visitors * likes` GET requests requests.

This patch optimizes this flow, by sending the new like count down in
the MessageBus notification, removing any need for the extra request.

It shouldn't cause any drift on the count because we send down the full
count instead of the difference too.

Possible follow-ups could include handling like removal.
2021-11-10 13:22:26 -03:00
Jarek Radosz b72688340f
FIX: `LocalJumpError` in rake tasks (#14874)
`return` isn't allowed in a block context. A followup to #14865
2021-11-10 16:53:55 +01:00
Martin Brennan 6a68bd4825
DEV: Limit list multipart parts to 1 (#14853)
We are only using list_multipart_parts right now in the
uploads controller for multipart uploads to check if the
upload exists; thus we don't need up to 1000 parts.

Also adding a note for future explorers that list_multipart_parts
only gets 1000 parts max, and adding params for max parts
and starting parts.
2021-11-10 08:01:28 +10:00
Jarek Radosz 25ef395af8
FIX: Don't fail if `s3_install_cors_rule` is off (#14865)
Regressed in #14802
2021-11-10 08:00:30 +10:00
David Taylor 65a389c3ac
FIX: Allow bulk invites to be used with DiscourseConnect (#14862)
Support for invites alongside DiscourseConnect was added in 355d51af. This commit fixes the guardian method so that the bulk invite button functionality also works.
2021-11-09 17:43:23 +00:00
Dan Ungureanu ec3758b573
FIX: Make PostRevisor more consistent (#14841)
* FIX: Preserve field types when updating revision

When a post was edited quickly twice by the same user, the old post
revision was updated with the newest changes. To check if the change
was reverted (i.e. rename topic A to B and then back to A) a comparison
of the initial value and last value is performed. If the check passes
then the intermediary value is dismissed and only the initial value and
the last ones are preserved. Otherwise, the modification is dismissed
because the field returned to its initial value.

This used to work well for most fields, but failed for "tags" because
the field is an array and the values were transformed to strings to
perform the comparison.

* FIX: Reset last_editor_id if revision is reverted

If a post was revised and then the same revision was reverted,
last_editor_id was still set to the ID of the user who last edited the
post. This was a problem because the same person could then edit the
same post again and because it was the same user and same post, the
system attempted to update the last one (that did not exist anymore).
2021-11-09 16:29:37 +02:00
Dan Ungureanu 46fed1303f
FIX: Regression introduced in #14715 (#14842)
* FIX: Regression introduced in #14715

Wrong method names were used to get Redis keys.

* DEV: Remove more stubs
2021-11-09 17:20:09 +11:00
Sam 337ef60303
DEV: increase lock timeout for multisite migration (#14831)
- Increase lock timeout - given multisites may take a while to migrate
- Ensure we do not check for status while db is migrating
2021-11-09 12:06:06 +11:00
Krzysztof Kotlarek 8b93da9fe0
FIX: rename action_code_href to action_code_path (#14834)
Small actions should use path instead of absolute url. getURL function is necessary to insert a potential subfolder prefix.
2021-11-08 14:32:17 +11:00
Martin Brennan fc98d1edfa
DEV: Improve s3:ensure_cors_rules logging (#14832) 2021-11-08 11:44:12 +10:00
Osama Sayegh e2afa1b799
FIX: Skip post validations for system revisions when author deletes post (#14824) 2021-11-08 09:33:41 +08:00
Jarek Radosz e6245bd603
DEV: Remove hints of jshint (#14828) 2021-11-08 09:12:37 +08:00
Martin Brennan 9a72a0945f
FIX: Ensure CORS rules exist for S3 using rake task (#14802)
This commit introduces a new s3:ensure_cors_rules rake task
that is run as a prerequisite to s3:upload_assets. This rake
task calls out to the S3CorsRulesets class to ensure that
the 3 relevant sets of CORS rules are applied, depending on
site settings:

* assets
* direct S3 backups
* direct S3 uploads

This works for both Global S3 settings and Database S3 settings
(the latter set directly via SiteSetting).

As it is, only one rule can be applied, which is generally
the assets rule as it is called first. This commit changes
the ensure_cors! method to be able to apply new rules as
well as the existing ones.

This commit also slightly changes the existing rules to cover
direct S3 uploads via uppy, especially multipart, which requires
some more headers.
2021-11-08 09:16:38 +10:00
Krzysztof Kotlarek fe8087e523
FEATURE: small action post accepts href (#14816)
Optionally add href to small action.
It can be used by discourse-assign to link to correct post from translation
2021-11-08 08:24:44 +11:00
Roman Rizzi 53abcd825d
FIX: Canonical URLs may be relative (#14825)
FinalDestination's follow_canonical mode used for embedded topics should work when canonical URLs are relative, as specified in [RFC 6596](https://datatracker.ietf.org/doc/html/rfc6596)
2021-11-05 14:20:14 -03:00
Mark VanLandingham 67265a5045
DEV: Plugin instance method for push_notification_filters (#14787) 2021-11-03 12:21:33 -05:00
Neil Lalonde 3254d35078
FIX: rake categories:move_topics shouldn't move category description topic (#14797) 2021-11-03 11:27:51 +08:00
jbrw aec125b617
FIX: Display Instagram Oneboxes in an iframe (#14789)
We are no longer able to display the image returned by Instagram directly within a Discourse site (either in the composer, or within a cooked post within a topic), so:

- Display an image placeholder in the composer preview
- A cooked post should use an iframe to display the Instagram 'embed' content
2021-11-02 14:34:51 -04:00