Commit Graph

8417 Commits

Author SHA1 Message Date
Kane York 83a6ad32ff Revert "PERF: Cache categories in Site model take 2."
This reverts commit 06fa1efd3d.

Breakage in solved plugin
2021-06-21 12:25:04 -07:00
Bianca Nenciu e70e8d8d8f
FIX: TL4 users can see their deleted posts (#13364) 2021-06-21 19:10:02 +03:00
Joffrey JAFFEUX e50b7e9111
SECURITY: ensures timeouts are correctly used on connect (#13455) 2021-06-21 17:34:01 +02:00
Penar Musaraj 515fd8a4c3
DEV: Change method used to reference custom font assets (#13446)
The `asset-url` helper has been deprecated.
2021-06-21 09:33:12 -04:00
Dan Ungureanu c69b58e22e
FIX: Downsize image last when saving uploads (#13458)
The other processing operations, such as fixing orientation or cropping,
can in rare cases increase the size of the uploaded image. Running the
downsize step after all these operations should create the best image
possible.
2021-06-21 14:13:24 +03:00
Alan Guo Xiang Tan e64f9c87fb DEV: Fix broken stylesheet watcher.
Follow-up to 8e3691d537
2021-06-21 16:43:07 +08:00
Alan Guo Xiang Tan 7901f73597 FIX: Multiple links being generated for non-theme targets.
Follow-up to 8e3691d537
2021-06-21 16:26:49 +08:00
Alan Guo Xiang Tan be655762d8 FIX: Color scheme stylesheet not loaded when no default theme is set.
Follow-up to 8e3691d537
2021-06-21 12:50:45 +08:00
Martin Brennan 7b31d8a11b
DEV: Move chrome binary check into a shared lib (#13451)
We had checks for the chrome binary in 3 different places
for tests and only one of them checked for google-chrome-stable,
which is problematic for Arch linux users (there are dozens of us!)

This PR moves all the code to one place and references it instead
of copying and pasting.
2021-06-21 13:28:48 +10:00
Alan Guo Xiang Tan 44aa46ca05 Code review comments. 2021-06-21 11:06:58 +08:00
Alan Guo Xiang Tan 8e3691d537 PERF: Eager load Theme associations in Stylesheet Manager.
Before this change, calling `StyleSheet::Manager.stylesheet_details`
for the first time resulted in multiple queries to the database. This is
because the code was modelled in a way where each `Theme` was loaded
from the database one at a time.

This PR restructures the code such that it allows us to load all the
theme records in a single query. It also allows us to eager load the
required associations upfront. In order to achieve this, I removed the
support of loading multiple themes per request. It was initially added
to support user selectable theme components but the feature was never
completed and abandoned because it wasn't a feature that we thought was
worth building.
2021-06-21 11:06:58 +08:00
Alan Guo Xiang Tan 06fa1efd3d PERF: Cache categories in Site model take 2.
Follow-up to aa4f0aee67.

Fixed the security problem in the previous attempt.
2021-06-21 09:47:05 +08:00
Martin Brennan 22b96c9ce1
FIX: Prevent resurrecting old topics via email reply for group inboxes with SMTP enabled (#13382)
We already reject email replies to public topics via `SiteSetting.disallow_reply_by_email_after_days` and raising the `OldDestinationError`. This PR introduces similar behaviour for group inboxes, but without the rejection, and **only when SMTP is enabled for the group**.

If a reply is sent via email and the post is older than `SiteSetting.disallow_reply_by_email_after_days` days ago, then we create a new topic instead of making a reply in the old one and link back to the original topic. This is done to prevent long running group inbox discussions.
2021-06-21 11:45:00 +10:00
Bianca Nenciu 1e992d9193
FIX: Do not check for duplicate links in Onebox (#13345)
If a user posted a URL that appeared inside a Onebox, then the user
got a duplicate link notice. This was fixed by skipping those links in
Ruby.

If a user posted a URL that was Oneboxes and contained other links that
appeared in previous posts, then the user got a duplicate link notice.
This was fixed by skipping those links in JavaScript.
2021-06-18 18:55:24 +03:00
Joffrey JAFFEUX 0fd55acf84 Update lib/site_settings/type_supervisor.rb
Co-authored-by: David Taylor <david@taylorhq.com>
2021-06-18 16:36:23 +08:00
jjaffeux dc63613c96 DEV: improve error message for invalid setting’s value
Before this fix we would display this exception:

```
Discourse::InvalidParameters:
  value
```

After this fix we will display:

```
Discourse::InvalidParameters:
  Invalid `x` value for `s3_region`
```
2021-06-18 16:36:23 +08:00
Martin Brennan ff6114d83f
FIX: Do not add mailing list headers to group SMTP emails (#13431)
When we are emailing people from a group inbox, we are having
a PM conversation with them, as a support account would. In this
case mailing list headers do not make sense. It is not like a forum
topic where you may have tens or hundreds of participants -- it is a
conversation between the group and a small handful of people
directly contacting the group, often just one person.

The only header left in tact was List-Unsubsribe which is important
for letting people opt out to notifications.
2021-06-18 14:36:17 +10:00
jbrw fbfd1fd80b
FIX: Allow SVG uploads if dimensions are a fraction of a unit (#13409)
* FIX: Allow SVG uploads if dimensions are a fraction of a unit

`UploadCreator` counts the number of pixels in an file to determine if it is valid. `pixels` is calculated by multiplying the width and height of the image, as determined by FastImage.

SVG files can have their width/height expressed in a variety of different units of measurement. For example, ‘px’, ‘in’, ‘cm’, ‘mm’, ‘pt’, ‘pc’, etc are all valid within SVG files. If an image has a width of `0.5in`, FastImage may interpret this as being a width of `0`, meaning it will report the `size` as being `0`.

However, we don’t need to concern ourselves with the number of ‘pixels’ in a SVG files, as that is irrelevant for this file format, so we can skip over the check for `pixels == 0` when processing this file type.

* DEV: Speed up getting SVG dimensions

The `-ping` flag prevents the entire image from being rasterized before a result is returned. See:

https://imagemagick.org/script/command-line-options.php#ping
2021-06-17 15:56:11 -04:00
Mark VanLandingham 95b51669ad
DEV: Revert 3 commits for plugin API to add directory columns (#13423) 2021-06-17 12:37:37 -05:00
Osama Sayegh c47f55253f
DEV: Add optional theme test step to the `smoke:test` rake task (#13418)
The purpose of this is to allow us to catch regressions for a feature we've built recently that allows theme tests to run in production. We recently had a regression that we didn't notice for days, so to prevent that from happening again we'll use this in our internal CI pipelines.
2021-06-17 20:09:29 +03:00
Mark VanLandingham 0c42a29dc4
DEV: Plugin API to allow creation of directory columns with item query (#13402)
The first thing we needed here was an enum rather than a boolean to determine how a directory_column was created. Now we have `automatic`, `user_field` and `plugin` directory columns.

This plugin API is assuming that the plugin has added a migration to a column to the `directory_items` table.

This was created to be initially used by discourse-solved. PR with API usage - https://github.com/discourse/discourse-solved/pull/137/
2021-06-17 09:06:18 -05:00
Alan Guo Xiang Tan aa4f0aee67 Revert "PERF: Cache categories in Site model."
This reverts commit 7dc0f88acd.
2021-06-17 15:20:35 +08:00
Alan Guo Xiang Tan 7dc0f88acd PERF: Cache categories in Site model.
Profiling showed that we were roughly 10% of a request time creating all
the ActiveRecord objects for categories in the `Site` model on a site with 61 categories.
Instead of querying for the categories each time based on which categories the user can see,
we can just preload all of the categories upfront and filter out the
categories that the user can not see.
2021-06-17 13:17:42 +08:00
Martin Brennan 6fe78cd542
FIX: Make sure reset-new for tracked is not limited by per_page count (#13395)
When dismissing new topics for the Tracked filter, the dismiss was
limited to 30 topics which is the default per page count for TopicQuery.
This happened even if you specified which topic IDs you were
selectively dismissing. This PR fixes that bug, and also moves
the per_page_count into a DEFAULT_PER_PAGE_COUNT for the TopicQuery
so it can be stubbed in tests.

Also moves the unused stub_const method into the spec helpers
for cases like this; it is much better to handle this in one place
with an ensure. In a follow up PR I will clean up other specs that
do the same thing and make them use stub_const.
2021-06-17 08:20:09 +10:00
Brandon Gastelo 20dbcbf022
FIX: Sort filelists to ensure consistant asset precompilation hash (#13393)
Dir.glob does not guarantee file order and can change when ran on different machines.
This means that running asset precompilation on the exact same codebase will output
different content hashes.
2021-06-16 11:04:21 -04:00
David Taylor b0416cb1c1
FEATURE: Upload to s3 in parallel to speed up backup restores (#13391)
Uploading lots of small files can be made significantly faster by parallelizing the `s3.put_object` calls. In testing, an UPLOAD_CONCURRENCY of 10 made a large restore 10x faster. An UPLOAD_CONCURRENCY of 20 made the same restore 18x faster.

This commit is careful to parallelize as little as possible, to reduce the chance of concurrency issues. In the worker threads, no database transactions are performed. All modification of shared objects is controlled with a mutex.

Unfortunately we do not have any existing tests for the `ToS3Migration` class. This change has been tested with a large site backup (120k uploads totalling 45GB)
2021-06-16 10:34:39 +01:00
Martin Brennan c659e3e95b
FIX: Make sure topic_user.bookmarked is synced in more places (#13383)
When we call Bookmark.cleanup! we want to make sure that
topic_user.bookmarked is updated for topics linked to the
bookmarks that were deleted. Also when PostDestroyer calls
destroy and recover. We have a job for this already --
SyncTopicUserBookmarked -- so we just utilize that.
2021-06-16 08:30:40 +10:00
Osama Sayegh 503017474c
DEV: Skip CSS watcher when running QUnit tests and expose more Chrome logs (#13390)
There are 2 changes in this PR:

1) Add a new environment variable called `DISCOURSE_SKIP_CSS_WATCHER` to disable our stylesheet watcher, and make the `qunit:test` rake task set this variable on the Unicorn/Rails server it spins up to disable our stylesheet watcher when running the tests because it doesn't really need it.

2) Print more Chrome logs (such as network/security errors) to the console.
2021-06-15 18:27:15 +03:00
Roman Rizzi fa57316a4e
FIX: Validate upload is still valid after calling the "before_upload_creation" event (#13091)
Since we use the event to perform additional validations on the file, we should check if it added any errors to the upload before saving it. This change makes the UploadCreator more consistent since we no longer have to rely on exceptions.
2021-06-15 10:10:03 -03:00
Martin Brennan 7fca7fb7ff
DEV: Add SMTP group ID to EmailLog (#13381)
Adds a new `smtp_group_id` column to `EmailLog` which is filled in if the mail `from_address` matches a group's `email_username`. This is for easier debugging, so we know which emails have been sent via group SMTP.
2021-06-15 11:29:46 +10:00
Alan Guo Xiang Tan 5f8f139ac9 PERF: Remove extra PG query.
In `Theme.list_baked_fields`, common is always included as a target.
2021-06-15 08:51:00 +08:00
Kane York c780ae9d25
FEATURE: Add a messages view for all official warnings of a user (#12659)
Moderators are allowed to see the warnings list, with an access warning.

https://meta.discourse.org/t/why-arent-warnings-easily-accessible-like-suspensions-are/164043
2021-06-14 14:01:17 -07:00
Jarek Radosz 091beaf4a2
DEV: Work around a Docker issue (#13368)
Fixes our backend spec suite in GitHub Actions CI. For more information about the Docker issue see: https://github.com/docker/for-linux/issues/1015

(It's possible that error could also happen in dev/production, though thankfully that hasn't happened yet afaik)
2021-06-14 17:38:36 +02:00
Vinoth Kannan 6abc45e57b
DEV: move `discourse_dev` gem to the core. (#13360)
And get avatar images from `discourse_dev_assets` gem.
2021-06-14 20:34:44 +05:30
Martin Brennan e9dc88a7b6
FIX: Link up reply to post correctly when emailing group (#13339)
When replying to a user_private_message email originating from
a group PM that does _not_ have a reply key (e.g. when replying
directly to the group's SMTP address), we were mistakenly linking
the new post created from the reply to the OP and the user who
created the topic, based on the first IncomingEmail message ID in
the topic, rather than using the correct reply to user and post number
that the user actually replied to.

We now use the In-Reply-To header to look up the corresponding EmailLog
record when the user who replied was sent a user_private_message email,
and use the post from that as the reply_to_user/post.

This also removes superfluous filtering of incoming_email records. After
already filtering by message_id and then addressed_to_user (which only
returns incoming emails where the to, from, or cc address includes any
of the user's emails), we were filtering again but in the ruby code for
the exact same conditions. After removing this all existing tests still
pass.
2021-06-10 15:28:50 +10:00
Penar Musaraj 6f76479054
FEATURE: Add upgrade-insecure-requests to CSP when force_https is enabled (#13348)
If force_https is enabled all resource (including markdown preview and so on) will be accessed using HTTPS

If for any reason you attempt to link to non HTTPS reachable content content may appear broken
2021-06-10 10:53:10 +10:00
Alan Guo Xiang Tan 2a4a20ad67
PERF: Avoid running a pointless PG query when theme has no variables. (#13342)
When `Theme#all_theme_variables` returns an empty array, we were running
a pointless query in `StyleSheet::Manager#uploads_digest`.

`SELECT "sha1" FROM "theme_fields" INNER JOIN "uploads" ON
"uploads"."id" = "theme_fields"."upload_id" WHERE 1=0`
2021-06-10 10:50:17 +10:00
Penar Musaraj a96f6fbdf5
FIX: Do not block SVG sprite bundle if a file is missing (#13338) 2021-06-09 08:29:00 -04:00
Neil Lalonde c2c647b990
FIX: errors loading secure uploads when secure uploads is disabled (#13047) 2021-06-08 13:25:51 -04:00
David Taylor 9811a1c5d9
DEV: Allow `transformed` values to be used in all widget hbs statements (#13331)
Previously, the `transformed.blah` shortcut could only be used in top-level hbs statements like {{transformed.blah}}. When attempting to use it in a sub-expression like `{{concat "hello" transformed.world}}`, it would raise a "transformed is not defined" error.

This commit updates the shortcut logic to make `transformed.blah` and `attrs.blah` work consistently in all hbs expressions.

Co-authored-by: Jordan Vidrine <jordan@jordanvidrine.com>
2021-06-08 16:46:07 +01:00
Neil Lalonde 41038d6cdb
Version bump to v2.8.0.beta2 (#13329) 2021-06-08 11:27:50 -04:00
Penar Musaraj 8336e732d3
DEV: Add manifest-src to CSP (#13319)
Defaults to `manifest-src: 'self'` and allows plugins/themes to extend it.
2021-06-08 09:32:31 -04:00
Alan Guo Xiang Tan 2717615f11
PERF: Preload client settings json while booting app. (#13283)
Generating the client settings json involves santizing all string based
site settings. This is expensive as per our profile in production (~120ms) and one request after each deploy has
to pay this penalty.
2021-06-08 13:15:55 +10:00
Alan Guo Xiang Tan 19a92fbadc
PERF: Switch ActiveRecord PG connection active check to use empty query. (#13323)
See https://github.com/rails/rails/pull/42368

The impact is not quantifiable at the time of this writing but
prelimary investigation shows that `SELECT 1` accounts for 0.09 of CPU
time on a database. Note that Discourse runs thousands of databases so
the small impact may be amplified by the large number of databases that
we run.
2021-06-08 13:15:07 +10:00
Penar Musaraj f90c4bd6a1
DEV: Allow plugins to extend frame-ancestors (#13316) 2021-06-07 14:59:15 -04:00
jbrw 09bc95d46b
FIX: Quoting Oneboxed content should exclude formatting (#13296)
* FIX: Quoting Oneboxed content should exclude formatting

When a post is quoted that includes Oneboxed content, we should not include the formatting generated by the Onebox. Rather, we should attempt to collapse the link referenced by the Onebox to a single line text link.

* DEV: fix tests
2021-06-07 13:03:53 -04:00
Dan Ungureanu 36e0e6a322
FIX: Allow invites if must_approve_users is true (#13257) 2021-06-07 18:57:08 +03:00
Arpit Jalan 2e4f07678e
FIX: IMDb links were being oneboxed as posters (#13310)
IMDb movie links were being rendered as posters. This was because
IMDb was sending `og:type` as `image` randomly in some cases. To
fix this we'll now default all IMDb links as article type. This will
ensure that the IMDb onebox link includes all the information instead
of showing just a poster without any context.
2021-06-07 18:45:59 +05:30
Alan Guo Xiang Tan 064bca430c PERF: Preload `Post#image_upload` in `TopicView`.
This fixes an N+1 queires problem when generating `Post#image_url` which
requires the upload to be loaded.

Follow-up to 141f16eb6b
2021-06-07 14:43:49 +08:00
Alan Guo Xiang Tan cadf5eafe6 DEV: Move Discourse app specific concern out of unicorn conf. 2021-06-04 09:13:34 +08:00
Alan Guo Xiang Tan a8667b5454 PERF: Defer setting of distributed cache in more spots.
See follow up commit for rational.

Follow-up to 8cfe203383
2021-06-04 09:13:18 +08:00
Kris cd9941e0ca
UX: more consistent setting/edit buttons (#13276) 2021-06-03 18:33:36 -04:00
David Taylor 4134173bbf
FEATURE: Add global admin api key rate limiter (#12527) 2021-06-03 10:52:43 +01:00
Alan Guo Xiang Tan 58b30fb510 PERF: Preload settings, groups and badge icons in SvgSprite.
Identified as a hot path in production. Preload it early instead of
executing the queries in a live request.
2021-06-03 16:45:56 +08:00
Martin Brennan eb2c399445
FEATURE: Use group SMTP settings for sending user notification emails (initial) (#13220)
This PR changes the `UserNotification` class to send outbound `user_private_message` using the group's SMTP settings, but only if:

* The first allowed_group on the topic has SMTP configured and enabled
* SiteSetting.enable_smtp is true
* The group does not have IMAP enabled, if this is enabled the `GroupSMTPMailer` handles things

The email is sent using the group's `email_username` as both the `from` and `reply-to` address, so when the user replies from their email it will go through the group's SMTP inbox, which needs to have email forwarding set up to send the message on to a location (such as a hosted site email address like meta@discoursemail.com) where it can be POSTed into discourse's handle_mail route.

Also includes a fix to `EmailReceiver#group_incoming_emails_regex` to include the `group.email_username` so the group does not get a staged user created and invited to the topic (which was a problem for IMAP), as well as updating `Group.find_by_email` to find using the `email_username` as well for inbound emails with that as the TO address.

#### Note

This is safe to merge without impacting anyone seriously. If people had SMTP enabled for a group they would have IMAP enabled too currently, and that is a very small amount of users because IMAP is an alpha product, and also because the UserNotification change has a guard to make sure it is not used if IMAP is enabled for the group. The existing IMAP tests work, and I tested this functionality by manually POSTing replies to the SMTP address into my local discourse.

There will probably be more work needed on this, but it needs to be tested further in a real hosted environment to continue.
2021-06-03 14:47:32 +10:00
Bianca Nenciu d184fe59ca
FEATURE: Censor Oneboxes (#12902)
Previously onebox content was not passed by the censor regex, meaning you could sneak in censored words via onebox.
2021-06-03 11:39:12 +10:00
Alan Guo Xiang Tan 58cb120aa2
DEV: Minor code clean up in assets.rake. (#13245) 2021-06-03 11:37:06 +10:00
Alan Guo Xiang Tan 8cfe203383 PERF: Defer setting of distributed cache in performance critical paths.
Setting a key/value pair in DistributedCache involves waiting on the
write to Redis to finish. In most cases, we don't need to wait on the
setting of the cache to finish. We just need to take our return value
and move on.
2021-06-03 09:30:52 +08:00
Jarek Radosz 3bb765ac92
DEV: Remove the remaining Travis code (#13255)
The second attempt at #10041 now that all our plugins use GitHub Actions CI instead.
2021-06-02 20:29:47 +02:00
Penar Musaraj fd9ef14ec0
FIX: Show required tags to staff by default and override limit (#13242)
This improves the display of available tags in categories that are
configured to require at least (x) tags from a tag group.

There are two changes included:
- regular users will now see all the available tags in the required tag
group (previously they could see a max. of 5 tags)
- staff users will now see the tags from the required tag group when
the tag group contains more tags than the default limit (also set to 5)

Both changes only apply to the default query (i.e. no search terms).
2021-06-02 12:43:34 -04:00
Penar Musaraj 9d6780f03d
DEV: Remove emoji_one files (#13236)
And add a symlink so requests to those assets fallback to the default
emoji set (Twitter).
2021-06-02 09:18:25 -04:00
Bianca Nenciu d9484db718
FIX: Split link watched words from replace (#13196)
It was not clear that replace watched words can be used to replace text
with URLs. This introduces a new watched word type that makes it easier
to understand.
2021-06-02 15:36:49 +10:00
Alan Guo Xiang Tan 2e4182b4b3 PERF: Warm up caches in `SiteSerializer` while preloading app.
Otherwise, time will be spent warming up the caches in the initial
requests.
2021-06-02 13:33:31 +08:00
Alan Guo Xiang Tan 6a79864f14 PERF: Cache categories query in `Stylesheet::Manager.color_scheme_digest`.
The query is being executed each time we try and generate the link path
for a stylesheet within the duration of a reqeust. Categories are not
updated that often so repeating this query multiple times a request is
wasteful.

At the time of this commit, there is a `publish_discourse_stylesheet`
ActiveRecord callback on the `Category` model which clears the cache of
`Stylesheet::Manager` each time a category is saved.
2021-06-02 13:33:31 +08:00
Alan Guo Xiang Tan 4390b50b12 DEV: Fix incorrect optimization in SvgSprite take 2.
Follow-up to 0700e9382d
2021-06-02 11:17:11 +08:00
Alan Guo Xiang Tan ca9e0ee40b DEV: Fix incorrect optimization in SvgSprite.
`SvgSprite.custom_sprite_paths` does not just fetch custom SVG sprite
from themes. It also picks up any custom svg icons from a pre-defined
plugin folder.

Follow-up to 0700e9382d
2021-06-02 10:42:23 +08:00
Alan Guo Xiang Tan 0700e9382d PERF: Memoize core svgs in memory to avoid expensive XML parsing.
The XML parsing of SVGs is done whenever the cache expires or on the
first load after a reboot. In one of our production instance, parsing
ranges from 30ms to 70ms which is not ideal. Instead, we've decided to
make a small memory trade off here by memoizing the core SVGs once on
boot to avoid parsing of the SVG files during the duration of a request.
The memozied hash will take up 57440 bytes or 0.05744 megabytes in size.
2021-06-02 09:16:37 +08:00
Martin Brennan e15c86e8c5
DEV: Topic tracking state improvements (#13218)
I merged this PR in yesterday, finally thinking this was done https://github.com/discourse/discourse/pull/12958 but then a wild performance regression occurred. These are the problem methods:

1aa20bd681/app/serializers/topic_tracking_state_serializer.rb (L13-L21)

Turns out date comparison is super expensive on the backend _as well as_ the frontend.

The fix was to just move the `treat_as_new_topic_start_date` into the SQL query rather than using the slower `UserOption#treat_as_new_topic_start_date` method in ruby. After this change, 1% of the total time is spent with the `created_in_new_period` comparison instead of ~20%.

----

History:

Original PR which had to be reverted **https://github.com/discourse/discourse/pull/12555**. See the description there for what this PR is achieving, plus below.

The issue with the original PR is addressed in 92ef54f402

If you went to the `x unread` link for a tag Chrome would freeze up and possibly crash, or eventually unfreeze after nearly 10 mins. Other routes for unread/new were similarly slow. From profiling the issue was the `sync` function of `topic-tracking-state.js`, which calls down to `isNew` which in turn calls `moment`, a change I had made in the PR above. The time it takes locally with ~1400 topics in the tracking state is 2.3 seconds.

To solve this issue, I have moved these calculations for "created in new period" and "unread not too old" into the tracking state serializer.

When I was looking at the profiler I also noticed this issue which was just compounding the problem. Every time we modify topic tracking state we recalculate the sidebar tracking/everything/tag counts. However this calls `forEachTracked` and `countTags` which can be quite expensive as they go through the whole tracking state (and were also calling the removed moment functions).

I added some logs and this was being called 30 times when navigating to a new /unread route because  `sync` is being called from `build-topic-route` (one for each topic loaded due to pagination). So I just added a debounce here and it makes things even faster.

Finally, I changed topic tracking state to use a Map so our counts of the state keys is faster (Maps have .size whereas objects you have to do Object.keys(obj) which is O(n).)

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2021-06-02 09:06:29 +10:00
jbrw 461a2c334b
FIX: return an empty result if response from Amazon is missing expected attributes (#13173)
* FIX: return an empty result if response from Amazon is missing attributes

Check we have the basic attributes requires to construct a Onebox for Amazon.

This is an attempt to handle scenarios where we receive a valid 200-status response from an Amazon request that does not include the data we’re expecting.

* Update lib/onebox/engine/amazon_onebox.rb

Co-authored-by: Régis Hanol <regis@hanol.fr>

Co-authored-by: Régis Hanol <regis@hanol.fr>
2021-06-01 16:23:18 -04:00
Gerhard Schlager 41ee5b7c86
FIX: Don't store translated trust level names in anonymous cache (#13224)
Refactors `TrustLevel` and moves translations from server to client

Additional changes:
  * "staff" and "admin" wasn't translatable in site settings
  * it replaces a concatenated string with a translation
  * uses translation for trust levels in users_by_trust_level report
  * adds a DB migration to rename keys of translation overrides affected by this commit
2021-06-01 22:11:48 +02:00
Penar Musaraj 4d4c3fe1e4
FIX: Delete internal links when moderator deletes a post (#13233) 2021-06-01 14:02:53 -04:00
Joffrey JAFFEUX 1cd0424ccd
FEATURE: lets users favorite 2 badges to show on user-card (#13151) 2021-06-01 10:33:40 +02:00
Gerhard Schlager c1c090419b DEV: Fix warning about already initialized constants in `FileStore::BaseStore` 2021-06-01 09:30:37 +08:00
Dan Ungureanu ccbe3bea79
UX: Improve style of GitHub PR body in emails (#13198)
On the web, we display only an excerpt in a monospace font and the rest
of the body is hidden under ellipsis. The email displayed both of them
and it did not use the same style. This commit leaves only the excerpt
in emails and makes it use a monospace font to display it.
2021-05-31 14:03:19 +03:00
Osama Sayegh b81b24dea2
Revert "DEV: Topic tracking state improvements (#12958)" (#13209)
This reverts commit 002c676344.

Perf regression, we will redo it.
2021-05-31 17:47:42 +10:00
Jarek Radosz bbcd70e8b5 DEV: Remove dead code
There's no such thing as `@in_spoiler` anymore.
2021-05-31 10:22:50 +08:00
Martin Brennan 002c676344
DEV: Topic tracking state improvements (#12958)
Original PR which had to be reverted **https://github.com/discourse/discourse/pull/12555**. See the description there for what this PR is achieving, plus below.

The issue with the original PR is addressed in 92ef54f402

If you went to the `x unread` link for a tag Chrome would freeze up and possibly crash, or eventually unfreeze after nearly 10 mins. Other routes for unread/new were similarly slow. From profiling the issue was the `sync` function of `topic-tracking-state.js`, which calls down to `isNew` which in turn calls `moment`, a change I had made in the PR above. The time it takes locally with ~1400 topics in the tracking state is 2.3 seconds.

To solve this issue, I have moved these calculations for "created in new period" and "unread not too old" into the tracking state serializer.

When I was looking at the profiler I also noticed this issue which was just compounding the problem. Every time we modify topic tracking state we recalculate the sidebar tracking/everything/tag counts. However this calls `forEachTracked` and `countTags` which can be quite expensive as they go through the whole tracking state (and were also calling the removed moment functions).

I added some logs and this was being called 30 times when navigating to a new /unread route because  `sync` is being called from `build-topic-route` (one for each topic loaded due to pagination). So I just added a debounce here and it makes things even faster.

Finally, I changed topic tracking state to use a Map so our counts of the state keys is faster (Maps have .size whereas objects you have to do Object.keys(obj) which is O(n).)
2021-05-31 09:22:28 +10:00
Penar Musaraj 06e1af2b1d
FIX: Giphy oneboxing when the response is an image (#13199) 2021-05-28 15:10:32 -04:00
Penar Musaraj 47e09700fe
FIX: Support pausing GIFs for giphy/tenor oneboxes (#13194) 2021-05-28 08:40:30 -04:00
Martin Brennan 501de809da
FIX: Do not mark badge image uploads as secure (#13193)
* FIX: Do not mark badge image uploads as secure

We do not need badge_image upload types to be marked as secure.
Post migration is the same as
https://github.com/discourse/discourse/pull/12081.

See
https://meta.discourse.org/t/secure-media-uploads/140017/122?u=martin
2021-05-28 12:35:52 +10:00
Martin Brennan 964da21817
FEATURE: Improve group email settings UI (#13083)
This overhauls the user interface for the group email settings management, aiming to make it a lot easier to test the settings entered and confirm they are correct before proceeding. We do this by forcing the user to test the settings before they can be saved to the database. It also includes some quality of life improvements around setting up IMAP and SMTP for our first supported provider, GMail. This PR does not remove the old group email config, that will come in a subsequent PR. This is related to https://meta.discourse.org/t/imap-support-for-group-inboxes/160588 so read that if you would like more backstory.

### UI

Both site settings of `enable_imap` and `enable_smtp` must be true to test this. You must enable SMTP first to enable IMAP.

You can prefill the SMTP settings with GMail configuration. To proceed with saving these settings you must test them, which is handled by the EmailSettingsValidator.

If there is an issue with the configuration or credentials a meaningful error message should be shown.

IMAP settings must also be validated when IMAP is enabled, before saving.

When saving IMAP, we fetch the mailboxes for that account and populate them. This mailbox must be selected and saved for IMAP to work (the feature acts as though it is disabled until the mailbox is selected and saved):

### Database & Backend

This adds several columns to the Groups table. The purpose of this change is to make it much more explicit that SMTP/IMAP is enabled for a group, rather than relying on settings not being null. Also included is an UPDATE query to backfill these columns. These columns are automatically filled when updating the group.

For GMail, we now filter the mailboxes returned. This is so users cannot use a mailbox like Sent or Trash for syncing, which would generally be disastrous.

There is a new group endpoint for testing email settings. This may be useful in the future for other places in our UI, at which point it can be extracted to a more generic endpoint or module to be included.
2021-05-28 09:28:18 +10:00
Robin Ward 2082abbb75
DEV: Add support for class properties in babel (#13189)
This allows us to start using JS classes instead of Ember's classes.
2021-05-27 16:13:14 -04:00
Penar Musaraj 726500bc59
FEATURE: Enable pausing images from Giphy and Tenor (#13185) 2021-05-27 15:00:38 -04:00
Bianca Nenciu 571ee4537a
FEATURE: Silence watched word (#13160)
This is a new type of watched word to replace auto_silence_first_post_
regex site setting.
2021-05-27 19:19:58 +03:00
Gerhard Schlager 157f10db4c
FEATURE: Use path from existing URL of uploads and optimized images (#13177)
Discourse shouldn't dynamically calculate the path of uploads and optimized images after a file has been stored on disk or S3. Otherwise it might calculate the wrong path if the SHA1 or extension stored in the database doesn't match the actual file path.
2021-05-27 17:42:25 +02:00
Penar Musaraj 855e854cb8
DEV: Enable optional chaining in all contexts (#13180)
* Revert "FIX: We can't use `?.` yet (#13168)"
2021-05-27 09:56:35 -04:00
Dan Ungureanu 723d7de18c
Various GitHub Onebox improvements (#13163)
* FIX: Improve GitHub folder regexp in Onebox

It used to match any GitHub URL that was not matched by the other GitHub
Oneboxes and it did not do a good job at handling those. With this
change, the generic Onebox will handle the remaining URLs.

* FEATURE: Add Onebox for GitHub Actions

* FEATURE: Add Onebox for PR check runs

* FIX: Remove image from GitHub folder Oneboxes

It is a generic, auto-generated image which does not provide any value.

* DEV: Add tests

* FIX: Strip HTML comments from PR body
2021-05-27 12:38:42 +03:00
Penar Musaraj 1270c7ad15
UX: Twitter onebox layout adjustments (#13181) 2021-05-27 15:35:32 +10:00
Arpit Jalan 283b08d45f
DEV: Absorb onebox gem into core (#12979)
* Move onebox gem in core library

* Update template file path

* Remove warning for onebox gem caching

* Remove onebox version file

* Remove onebox gem

* Add sanitize gem

* Require onebox library in lazy-yt plugin

* Remove onebox web specific code

This code was used in standalone onebox Sinatra application

* Merge Discourse specific AllowlistedGenericOnebox engine in core

* Fix onebox engine filenames to match class name casing

* Move onebox specs from gem into core

* DEV: Rename `response` helper to `onebox_response`

Fixes a naming collision.

* Require rails_helper

* Don't use `before/after(:all)`

* Whitespace

* Remove fakeweb

* Remove poor unit tests

* DEV: Re-add fakeweb, plugins are using it

* Move onebox helpers

* Stub Instagram API

* FIX: Follow additional redirect status codes (#476)

Don’t throw errors if we encounter 303, 307 or 308 HTTP status codes in responses

* Remove an empty file

* DEV: Update the license file

Using the copy from https://choosealicense.com/licenses/gpl-2.0/#

Hopefully this will enable GitHub to show the license UI?

* DEV: Update embedded copyrights

* DEV: Add Onebox copyright notice

* DEV: Add MIT license, convert COPYRIGHT.txt to md

* DEV: Remove an incorrect copyright claim

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
Co-authored-by: jbrw <jamie@goatforce5.org>
2021-05-26 15:11:35 +05:30
Roman Rizzi a5fddd5cdb
FIX: Hide a post's pending flag count from TL4 users. (#13129)
They can't see the review queue, so there's no reason to show a button with the post's pending flag count in the post controls.
2021-05-26 12:58:00 +10:00
Martin Brennan 7a79bd7da3
FEATURE: Allow selective dismissal of new and unread topics (#12976)
This PR improves the UI of bulk select so that its context is applied to the Dismiss Unread and Dismiss New buttons. Regular users (not just staff) are now able to use topic bulk selection on the /new and /unread routes to perform these dismiss actions more selectively.

For Dismiss Unread, there is a new count in the text of the button and in the modal when one or more topic is selected with the bulk select checkboxes.

For Dismiss New, there is a count in the button text, and we have added functionality to the server side to accept an array of topic ids to dismiss new for, instead of always having to dismiss all new, the same as the bulk dismiss unread functionality. To clean things up, the `DismissTopics` service has been rolled into the `TopicsBulkAction` service.

We now also show the top Dismiss/Dismiss New button based on whether the bottom one is in the viewport, not just based on the topic count.
2021-05-26 09:38:46 +10:00
Rafael dos Santos Silva 9118bb2076
FEATURE: Normalize the service worker route (#12343)
Re-lands the change initially proposed on #8359 but without a new nginx
location block, so it has less change surface.

Co-authored-by: Jeff Wong <awole20@gmail.com>

Co-authored-by: Jeff Wong <awole20@gmail.com>
2021-05-25 19:39:31 -03:00
Penar Musaraj 6e9ee3db03
FIX: Sort user bookmarks by reminder date (#13145) 2021-05-25 15:23:46 -04:00
Jeff Atwood 50926f6143
FIX: Simplify post and topic deletion language (#13128)
Based on feedback from Matt Haughey, we don't need to use so many words when describing a deleted topic or post.

Co-authored-by: Martin Brennan <martin@discourse.org>
2021-05-25 12:04:10 +10:00
Blake Erickson fccf4d4375
DEV: Switch off of our image_optim fork (#13124)
The main image_optim gem now includes the timeout feature
that we had in our fork. So it is now safe to switch off of our fork and
back to the image_optim gem.

This is the link to the commit in the image_optim repo that adds the
timeout option:

ec3767dde0

One difference with the new timeout implementation is that image_optim
now handles the timeout exceptions instead of bubbling them up:

1ed0328587/lib/image_optim.rb (L128-L129)

```
 rescue Errors::TimeoutExceeded
   handler.result
```

So a timeout will just return `nil`, which is the same response if it
couldn't optimize an image. I don't think we were really watching for
or doing anything about these timeout warnings in our logs so I think
this is an okay change to have and we will have less warnings in our
logs now too.
2021-05-24 19:27:20 -06:00
Daniel Waterworth ca809d2c40
FEATURE: Add on_preload for TopicView (#13122)
This allows plugins to load data on fetched posts before each post is
individually serialized.
2021-05-24 11:46:57 -05:00
David Taylor 93ae1b2b1c
DEV: Default to `development` RAILS_ENV when running theme tests (#13106)
When testing theme components in development, it doesn't make sense to use the `test` environment. The `test` environment almost certainly has 0 themes installed.

This change still works fine when using the `themes:install_and_test` rake task, because that rake task explicitly specifies environment/database-config.
2021-05-24 13:35:10 +01:00
Andrei Prigorshnev c62efc0f0f
FIX: Ignore max excerpt length for div excerpts too (#13058)
We support two types of custom excerpts. It can be <div class="excerpt"> or <span class="excerpt">: b21f74060e/lib/excerpt_parser.rb (L120)

We also ignore max excerpt length for custom excerpts. But we forgot to process div when ignoring max length.
2021-05-24 13:05:24 +04:00
Gerhard Schlager 332ae97555
FEATURE: Correctly convert topic title to uppercase and lowercase for Turkish default locale (#13115) 2021-05-24 18:13:30 +10:00
Bianca Nenciu 7c29ff3d85
FEATURE: Add tasks to export and import site structure (#12584) 2021-05-24 12:38:42 +10:00
Martin Brennan 292017dd25
FIX: Do not call :post_edited webhook twice when editing OP (#13112)
When editing the first post for the topic we do two AJAX requests
to two separate controllers in this order:

PUT /t/topic-name
PUT /posts/2489523

This causes two post revisor calls, which end up triggering the
:post_edited DiscourseEvent twice. This is then picked up and sent
as a WebHook event twice. However we do not need to send a :post_edited
webhook event if the first post is being edited and topic_changed is
true from the :post_edited DiscourseEvent, because a second event will
shortly come through for just the post.

See https://meta.discourse.org/t/post-webhook-fires-two-times-on-post-edited-for-first-post-in-a-topic/162408

Continued on from https://github.com/discourse/discourse/pull/10590
2021-05-24 09:10:22 +10:00
Bianca Nenciu 38af28d58b
FIX: Allow add email to group if user can invite (#13097)
It used to allow adding email addresses to a group even if invites were
disabled for the site. This does not allow user to input email address
if they cannot invite.

The second thing this commit improves is the message that is displayed
to the user when they hit the invite rate limit.
2021-05-21 11:34:17 +03:00
Josh Soref 13d40ead97
DEV: Correct spelling mistakes in comments 2021-05-21 13:37:17 +10:00
Martin Brennan afb2b4c6b9
FIX: Update upload secure status when revising posts (#13110)
When uploads are created from the composer (editing or creating a post),
for sites with secure uploads enabled we assume security by default and
that new upload is set to secure. When the post is created, we then
check whether the post uploads _actually_ need to be secure and adjust
accordingly.

We were not doing this when revising a post, so when a new upload was
created when editing a post in a public topic, the secure status stayed
true erroneously causing issues with image previews, among other things.
2021-05-21 13:32:32 +10:00
Josh Soref 59097b207f
DEV: Correct typos and spelling mistakes (#12812)
Over the years we accrued many spelling mistakes in the code base. 

This PR attempts to fix spelling mistakes and typos in all areas of the code that are extremely safe to change 

- comments
- test descriptions
- other low risk areas
2021-05-21 11:43:47 +10:00
David Taylor bd8c2b4615
DEV: Correct theme:qunit help text (#13104)
- Task name is themes:qunit, not themes:unit
- Some shells try to expand the square brackets. The whole thing should be enclosed in quotes to avoid this
2021-05-20 22:30:12 +01:00
Penar Musaraj 29f3621f45
FIX: Disable lightboxing of animated images (#13099) 2021-05-20 15:19:44 -04:00
Dan Ungureanu e2f9da9795
DEV: Remove reference to very old plugin (#13098) 2021-05-20 16:18:22 +03:00
Alan Guo Xiang Tan 958b56dc6c
PERF: Precompile child theme stylesheets. (#13040)
Previously, we only precompiled the CSS for parent themes but not for
the child themes. As a result, the CSS for child themes were being
compiled during the first request which made the respond time high for
that request.
2021-05-20 16:01:54 +10:00
Arpit Jalan 626b8465ba
FIX: do not validate uploads when running `uploads:fix_missing_s3` task (#13096) 2021-05-20 15:29:09 +10:00
Arpit Jalan 130160537c
FEATURE: add support for "skip_validations" option in UploadCreator (#13094)
FIX: do not validate uploads when running `uploads:fix_missing_s3` task
2021-05-19 20:54:52 +05:30
Andrei Prigorshnev 3e0f72f57f
FEATURE: Increase daily edit limits proportionally to trust level (#13090) 2021-05-19 13:57:21 +04:00
Arpit Jalan f96f534f3e
FIX: do not include contact url & email in client site settings payload (#13004) 2021-05-19 16:15:24 +10:00
Neil Lalonde 05c1d3c9cd
Version bump to v2.8.0.beta1 (#13092) 2021-05-18 16:38:17 -04:00
Bianca Nenciu c1dfd76658
FIX: Make replace watched words work with wildcard (#13084)
Watched words are always regular expressions, despite watched_words_
_regular_expressions being enabled or not. Internally, wildcard
characters are replaced with a regular expression that matches any non
whitespace character.
2021-05-18 12:09:47 +03:00
jbrw a24b6daa87
FIX: An unresolved blank uri should attempt an alternate Oneboxing strategy, if available (#13070) 2021-05-14 15:23:20 -04:00
Bianca Nenciu 3a1b05f219
FIX: Make autotag watched words case insensitive (#13043)
* FIX: Hide tag watched words if tagging is disabled

These 'autotag' words were shown even if tagging was disabled.

* FIX: Make autotag watched words case insensitive

This commit also fixes the bug when no tag was applied if no other tag
was already present.
2021-05-14 16:52:10 +03:00
jbrw 19182b1386
DEV: Oneboxer wildcard subdomains (#13015)
* DEV: Allow wildcards in Oneboxer optional domain Site Settings

Allows a wildcard to be used as a subdomain on Oneboxer-related SiteSettings, e.g.:

- `force_get_hosts`
- `cache_onebox_response_body_domains`
- `force_custom_user_agent_hosts`

* DEV: fix typos

* FIX: Try doing a GET after receiving a 500 error from a HEAD

By default we try to do a `HEAD` requests. If this results in a 500 error response, we should try to do a `GET`

* DEV: `force_get_hosts` should be a hidden setting

* DEV: Oneboxer Strategies

Have an alternative oneboxing ‘strategy’ (i.e., set of options) to use when an attempt to generate a Onebox fails. Keep track of any non-default strategies that were used on a particular host, and use that strategy for that host in the future.

Initially, the alternate strategy (`force_get_and_ua`) forces the FinalDestination step of Oneboxing to do a `GET` rather than `HEAD`, and forces a custom user agent.

* DEV: change stubbed return code

The stubbed status code needs to be a value not recognized by FinalDestination
2021-05-13 15:48:35 -04:00
Alan Guo Xiang Tan f05f5fde0f
PERF: Preload `JsLocaleHelper.load_translations` for default locale. (#13041)
In production, each Unicorn child process will currently hold the
default locale in memory on first load. Instead, we should preload it in
the Unicorn master process so that the memory is immediately shared when
forking.

Also, the translations are only memoized on first load now and is
adding considerable overhead to the first few requests after a fresh
boot.
2021-05-13 16:16:01 +10:00
Martin Brennan 3d2cace94f
DEV: Add service to validate email settings (#13021)
We have a few places in the code where we need to validate various email related settings, and will have another soon with the improved group email settings UI. This PR introduces a class which can validate POP3, IMAP, and SMTP credentials and also provide a friendly error message for issues if they must be presented to an end user.

This PR does not change any existing code to use the new service. I have added a TODO to change POP3 validation and the email test rake task to use the new validator post-release.
2021-05-13 15:11:23 +10:00
Dan Ungureanu 60be1556fc
FIX: Various invite system fixes (#13003)
* FIX: Ensure the same email cannot be invited twice

When creating a new invite with a duplicated email, the old invite will
be updated and returned. When updating an invite with a duplicated email
address, an error will be returned.

* FIX: not Ember helper does not exist

* FIX: Sync can_invite_to_forum? and can_invite_to?

The two methods should perform the same basic set of checks, such as
check must_approve_users site setting.

Ideally, one of the methods would call the other one or be merged and
that will happen in the future.

* FIX: Show invite to group if user is group owner
2021-05-12 13:06:39 +03:00
Krzysztof Kotlarek 4a2912233c
FIX: flaky specs after topic view custom filters (#13019)
* FIX: flaky specs after topic view custom filters

When ensuring TopicView class variables return to the original state it should use empty Hash instead of empty Array. That

https://github.com/discourse/discourse/blob/master/lib/topic_view.rb#L60

* FIX: convert to string for topic view custom filter
2021-05-11 11:24:14 +10:00
Blake Erickson 37812b8e35
Revert "DEV: Drop old IE11 intersection-observer references" (#13017)
This reverts commit 7360a0f70f.

iOS still wants this sometimes. Probably best to revert for now and we can
always remove this again later.

See: https://meta.discourse.org/t/189799/11?u=blake
2021-05-10 17:47:09 -06:00
Neil Lalonde 02f0acc41b
Version bump to v2.7.0.beta9 (#13006) 2021-05-10 11:29:47 -04:00
Krzysztof Kotlarek a4bd1806d9
FEATURE: ability to register custom filters for posts (#12938)
Allow plugins to extend TopicView to filter posts
2021-05-10 08:57:58 +10:00
Alan Guo Xiang Tan 4bad1e7e93 PERF: Preload `LetterAvatar.image_magick_version` in master process.
It is wasteful to run the clean up and command to generate the version
digest on every unicorn child process.
2021-05-07 14:00:38 +08:00
Martin Brennan 5f7bef6d20
FEATURE: Add email_encoded parameter to accept inbound base64 encoded emails (#12947)
We have found when receiving and posting inbound emails to the handle_mail route, it is better to POST the payload as a base64 encoded string to avoid strange encoding issues. This introduces a new param of `email_encoded` and maintains the legacy param of email, showing a deprecation warning. Eventually the old param of `email` will be dropped and the new one `email_encoded` will be the only way to handle_mail.
2021-05-06 12:59:52 +10:00
Blake Erickson 1b02dce594
DEV: Drop old IE11 intersection-observer references (#12942)
It's been awhile since we have supported IE11 so it should be safe to remove
IntersectionObserver now.

From a TODO task in this repo:
> drop when we eventually drop IE11

Announcement of when we removed IE11 support:

https://meta.discourse.org/t/137984/40?u=blake
2021-05-04 17:54:54 -06:00
Roman Rizzi 5794787300
FIX: Make the docker:test task install dependencies before calling yarn ember test. (#12943) 2021-05-04 16:30:35 -03:00
Joffrey JAFFEUX 374ff4c54c
FIX: topic might be night at this stage (#12929) 2021-05-03 10:37:43 +02:00
Dan Ungureanu 3ff9dbb15c
FIX: Show warning only if new value is different (#12893)
A warning is logged when a shadowed site setting is being changed. This
should not be the case when the new value is the same as the old one.
2021-05-03 09:25:02 +02:00
Joffrey JAFFEUX d16a39dc53
FIX: prevents exception when text input is nil (#12922)
nil was converted to "" and the matching regex would return [] and then be converted to nil with max usage.

Example exception:

```
NoMethodError (undefined method `<=' for nil:NilClass)

lib/text_sentinel.rb:71:in `seems_unpretentious?'
lib/validators/quality_title_validator.rb:13:in `validate_each'
lib/topic_creator.rb:25:in `valid?'
```
2021-05-03 09:21:35 +02:00
Joffrey JAFFEUX 64dda7112d
FIX: correctly use timeouts in `FileHelper` and `FinalDestination` (#12921)
Previous refactors have lost usage of read_timeout in `FileHelper.download` and `FinalDestination` was incorrectly using `Net::HTTP.start` by setting `open_timeout` in the block instead of directly during the invocation.

Couldn't figure how to write a good test for this without slowing the spec.
2021-05-03 09:21:11 +02:00
Jeff Wong 656b0ae39e
DEV: Add an option to skip a theme update from the themes:install task. (#12905)
A theme can now specify `skip_update: true` in the yml config for
update allowing for the theme to be installed only if it does not already
exist.
2021-04-30 09:31:41 -07:00
Gerhard Schlager 4f36f604c8
FIX: Looking up translation overrides by symbol failed when `count` is used (#12896) 2021-04-30 13:54:45 +02:00
David Taylor cac7725e28
DEV: Improve `bin/unicorn` boot time in development environment (#12900)
This commit makes a few changes to improve boot time in development environments. It will have no effect on production boot times.

- Skip the SchemaCache warmup. In development mode, the SchemaCache is refreshed every time there is a code change, so warmup is of limited use.

- Skip warming up PrettyText. This adds ~2s to each web worker's boot time. The vast majority of requests do not use PrettyText, so it is more efficient to defer its warmup until it's needed

- Skip the intentional 1 second pause during Unicorn worker forking. The comment (which also exists in Unicorn's documentation) suggests this works around a Unix signal handling bug, but I haven't been able to locate any more information. Skipping it in dev will significantly speed up boot. If we start to see issues, we can revert this change.

On my machine, this improves `/bin/unicorn` boot time from >10s to ~4s
2021-04-30 11:32:13 +01:00
David Taylor 1bc1a439ee
PERF: Ensure locales are always handled as symbols internally (#12897)
Sometimes, parts of the application pass in the locale as a string, not a symbol. This was causing the translate_accelerator to cache two versions of the locale separately: one cache for the symbol version, and one cache for the string version. For example, in a running production process:

```
irb(main):001:0> I18n.instance_variable_get(:@loaded_locales)
=> [:en, "en"]
```

This commit ensures the `locale` key is always converted to a symbol, and adds a spec to ensure the same locale cannot appear twice in `@loaded_locales`
2021-04-30 10:56:27 +01:00
David Taylor ad8c7714c8
DEV: Use filesystem-based SchemaCache in development (#12901)
In development we regularly restart/reload Rails, which wipes out the schema cache. This then has to be regenerated using DDL queries on the database.

Instead, we can make use of the `rake db:schema:cache:dump` command. This will dump the schema cache to a YAML file, and then load it when needed. This is significantly faster than rebuilding the cache from DDL queries every time.
2021-04-30 10:54:49 +01:00
Roman Rizzi 7423943ffe
DEV: The docker:test task can run JS tests using the Ember CLI. (#12879)
Set the EMBER_CLI env var to 1 to include these tests.
2021-04-29 14:51:34 -03:00
Neil Lalonde a7261a1f68
Version bump to v2.7.0.beta8 (#12894) 2021-04-29 13:37:14 -04:00
Bianca Nenciu e6329d3007
FIX: Gracefully handle inline images in emails (#12855) 2021-04-29 10:17:33 +03:00
Osama Sayegh 4f88f2eb15
FEATURE: Allow theme tests to be run in production (take 2) (#12845)
This commit allows site admins to run theme tests in production via a new `/theme-qunit` route. When you visit `/theme-qunit`, you'll see a list of the themes/components installed on your site that have tests, and from there you can select a theme or component that you run its tests.

We also have a new rake task `themes:install_and_test` that can be used to install a list of themes/components on a temporary database and run the tests of the themes/components that are installed. This rake task can be useful when upgrading/deploying a Discourse instance to make sure that the installed themes/components are compatible with the new Discourse version being deployed, and if the tests fail you can abort the build/deploy process so you don't end up with a broken site.
2021-04-28 23:12:08 +03:00
David Taylor 7568c7a101
PERF: Make stylesheet max_file_mtime calculation more efficient (#12876)
Plugins always store their stylesheets under `/assets/stylesheets`, so we can make the glob pattern much more specific. In my local development environment, this increases the speed of `Stylesheet::Manager.max_file_mtime` from ~65ms to ~3ms (20x faster). This significantly improves stylesheet regeneration time, and the responsiveness of the theme admin UI.

Note that this will have negligible effect in production, because in production the value of `max_file_mtime` is aggressively cached.
2021-04-28 17:27:18 +01:00
Régis Hanol cd93d1b5f7
FEATURE: new 'trim_incoming_emails' site setting (#12874)
This setting allows admin to de/activate automatic trimming of incoming email.
There are instances where it does wonders in trimming all the garbage content and other
instances where it's so bad that it trims the most important part of the email.

FIX: don't remove hidden content using the style attribute when converting HTML to Markdown.
The regexp used was doing more harm than good. It was way too broad.

FIX: properly elide signatures from emails sent with Front App.
This is fairly safe as Front App nicely identifies signatures in the HTML part.
2021-04-28 17:08:48 +02:00
Penar Musaraj 548c044809
FIX: Improvements to animated image pausing (#12839) 2021-04-28 10:48:00 -04:00
Martin Brennan 6d53005e8b
Revert "DEV: Improving topic tracking state code (#12555)" (#12864)
This reverts commit 45df579db0.

This was causing huge browser freezes and crashes.
2021-04-28 11:29:54 +10:00
Martin Brennan 45df579db0
DEV: Improving topic tracking state code (#12555)
The aim of this PR is to improve the topic tracking state JavaScript code and test coverage so further modifications can be made in plugins and in core. This is focused on making topic tracking state changes easier to respond to with callbacks, and changing it so all state modifications go through a single method instead of modifying `this.state` all over the place. I have also tried to improve documentation, make the code clearer and easier to follow, and make it clear what are public and private methods.

The changes I have made here should not break backwards compatibility, though there is no way to tell for sure if other plugin/theme authors are using tracking state methods that are essentially private methods. Any name changes made in the tracking-state.js code have been reflected in core.

----

We now have a `_trackedTopicLimit` in the tracking state. Previously, if a topic was neither new nor unread it was removed from the tracking state; now it is only removed if we are tracking more than `_trackedTopicLimit` topics (which is set to 4000). This is so plugins/themes adding topics with `TopicTrackingState.register_refine_method` can add topics to track that aren't necessarily new or unread, e.g. for totals counts.

Anywhere where we were doing `tracker.states["t" + data.topic_id] = newObject` has now been changed to flow through central `modifyState` and `modifyStateProp` methods. This is so state objects are not modified until they need to be (e.g. sometimes properties are set based on certain conditions) and also so we can run callback functions when the state is modified.

I added `onStateChange` and `onMessageIncrement` methods to register callbacks that are called when the state is changed and when the message count is incremented, respectively. This was done so we no longer need to do things like `@observes("trackingState.states")` in other Ember classes.

I split up giant functions like `sync` and `establishChannels` into smaller functions for readability and testability, and renamed many small functions to _functionName to designate them as private functions which not be called by consumers of `topicTrackingState`. Public functions are now all documented (well...at least ones that are not immediately obvious).

----

On the backend side, I have changed the MessageBus publish events for TopicTrackingState to send back tags and tag IDs for more channels, and done some extra code cleanup and refactoring. Plugins may override `TopicTrackingState.report` so I have made its footprint as small as possible and externalised the main parts of it into other methods.
2021-04-28 09:54:45 +10:00
Robin Ward 7a3f2eb3fd
FIX: Ember CLI Live Reload was not supported by the CSP (#12859) 2021-04-27 13:55:23 -04:00
David Taylor 02f1f85306
FIX: Remove whitespace from theme git versions (#12857)
Followup to 0a6fab564f
2021-04-27 17:45:54 +01:00
Régis Hanol bea1fc1ab8
FEATURE: TL4 & category moderators can merge posts (#12843)
This gives TL4 users and category moderators the ability to merge posts in topics they have moderation privileges.
2021-04-27 18:24:27 +02:00
David Taylor 0a6fab564f
PERF: Add timeout to theme import git commands (#12856)
This ensures that large git repositories cannot cause performance issues on the server. In normal use, this 20s timeout should never be hit.
2021-04-27 17:04:15 +01:00
David Taylor 1fd8f6df5f
PERF: Improve theme stylesheet compilation performance (#12850)
When building the `scss_load_paths`, we were creating a full export of the theme (including uploads), and not cleaning it up. With many uploads, this can be extremely slow (because it downloads every upload from S3), and the lack of cleanup could cause a disk to fill up over time.

This commit updates the ZipExporter to provide a `with_export_dir` API, which takes care of cleanup. It also adds a kwarg which allows exporting only extra_scss fields. This should make things much faster for themes with many uploads.
2021-04-27 14:33:43 +01:00
Krzysztof Kotlarek e29605b79f
FEATURE: the ability to search users by custom fields (#12762)
When the admin creates a new custom field they can specify if that field should be searchable or not.

That setting is taken into consideration for quick search results.
2021-04-27 15:52:45 +10:00
Bianca Nenciu 8aeeadd8b0
FIX: Replace use of regular expression (#12838)
It used a regular expression to check if message IDs were in RFC format.
2021-04-27 08:48:51 +03:00
Osama Sayegh a169dc6832
Revert "FEATURE: Allow theme tests to be run in production (#12815)" (#12840)
This reverts commit 7217dcb67a.

https://meta.discourse.org/t/failed-to-bootstrap-due-to-out-of-memory-killer/188141/18?u=osama

Precompiling test_helper.js is so expensive that it can make bootstrap
fail on servers with limited resources (2GB RAM). We will find another
way that doesn't require much resources.
2021-04-26 23:05:58 +03:00
Bianca Nenciu 765ba1ab2d
FEATURE: Ignore anonymous page views on private sites (#12800)
For sites with login_required set to true, counting anonymous pageviews is
confusing. Requests to /login and other pages would make it look like
anonymous users have access to site's content.
2021-04-26 14:19:47 +03:00
Osama Sayegh 7217dcb67a
FEATURE: Allow theme tests to be run in production (#12815)
This commit allows site admins to run theme tests in production via a new `/theme-qunit` route. When you visit `/theme-qunit`, you'll see a list of the themes/components installed on your site that have tests, and from there you can select a theme or component that you run its tests.

We also have a new rake task `themes:install_and_test` that can be used to install a list of themes/components on a temporary database and run the tests of the themes/components that are installed. This rake task can be useful when upgrading/deploying a Discourse instance to make sure that the installed themes/components are compatible with the new Discourse version being deployed, and if the tests fail you can abort the build/deploy process so you don't end up with a broken site.
2021-04-26 12:56:45 +03:00
Robin Ward e3b1d1a718
DEV: Improve Ember CLI's bootstrap logic (#12792)
* DEV: Give a nicer error when `--proxy` argument is missing

* DEV: Improve Ember CLI's bootstrap logic

Instead of having Ember CLI know which URLs to proxy or not, have it try
the URL with a special header `HTTP_X_DISCOURSE_EMBER_CLI`. If present,
and Discourse thinks we should bootstrap the application, it will
instead stop rendering and return a HTTP HEAD with a response header
telling Ember CLI to bootstrap.

In other words, any time Rails would otherwise serve up the HTML for the
Ember app, it stops and says "no, you do it."

* DEV: Support asset filters by path using a new options object

Without this, Ember CLI's bootstrap would not get the assets it wants
because the path it was requesting was different than the browser path.
This adds an optional request header to fix it.

So far this is only used by the styleguide.
2021-04-23 10:24:42 -04:00
Bianca Nenciu 8c4a11c006
DEV: Move autotag to topic creator (#12790)
This move was necessary to automatically tag the topic with the right
tags from creation time. The process post job may be delayed for a
short time.
2021-04-23 16:55:34 +03:00
Penar Musaraj c11d75da87
FEATURE: Allow pausing animated images in posts (#12795)
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2021-04-22 11:28:35 -04:00
Joffrey JAFFEUX 0eeedf307a
DEV: replaces huge generated emoji list by a simpler regex (#11053)
Note that this commit is also fixing various mistakes in emojis.

Some of them have been fixed manually in db.json/data.js/groups.json and will need to be fixed in emoji-db gem.
2021-04-22 08:43:06 +02:00
Penar Musaraj 886cf72b0e
FIX: Fall back to hardcoded version when period check disabled (#12784)
When `SiteSetting.version_checks?` is disabled, the Discourse version is
not stored in Redis, so we need to fall back to the hardcoded value.
2021-04-21 12:40:27 -04:00
Roman Rizzi 6b613e3076
FEATURE: Review every post using the review queue. (#12734)
* FEATURE: Review every post using the review queue.

If the `review_every_post` setting is enabled, posts created and edited by regular uses are sent to the review queue so staff can review them. We'll skip PMs and posts created or edited by TL4 or staff users.

Staff can choose  to:

- Approve the post (nothing happens)
- Approve and restore the post (if deleted)
- Approve and unhide the post (if hidden)
- Reject and delete it
- Reject and keep deleted (if deleted)
- Reject and suspend the user
- Reject and silence the user

* Update config/locales/server.en.yml

Co-authored-by: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2021-04-21 08:41:36 -03:00
Osama Sayegh 45ccadeeeb
DEV: Upgrade Rails to 6.1.3.1 (#12688)
Rails 6.1.3.1 deprecates a few API and has some internal changes that break our tests suite, so this commit fixes all the deprecations and errors and now Discourse should be fully compatible with Rails 6.1.3.1. We also have a new release of the rails_failover gem that's compatible with Rails 6.1.3.1.
2021-04-21 12:36:32 +03:00
Penar Musaraj c47e6a2004
FIX: Use CDN urls for theme settings of type upload (#12773) 2021-04-20 18:42:02 -04:00
Martin Brennan e3b1f5721b
FIX: Enforce tag group count validation before sending to review queue (#12728)
There is a category setting that enforces 1 or more tags must be added to a topic from a specific tag group before creating it. This validation was not being run before the topic was being sent to a review queue for categories that have that setting enabled.

There was an existing validation in `TopicCreator` but it was not correct; it was only validating when the tags did _not_ exist and also only happened on `create`. I now run the validation in `TopicCreator.valid?`

I also improved the error message shown to the user when they have not added the tags required (showing the tag names from the tag group), and changed the composer tag selector to not show "optional" if there are N tags required from a certain group.
2021-04-19 09:43:50 +10:00
Joffrey JAFFEUX 3157d5ee1b
DEV: ensures stylesheet watcher isn't crashing with gems plugins (#12733)
* DEV: ensures stylesheet watcher isn't crashing with gems plugins

This bug has been exhibited since discourse_dev is now including an auth plugin which was loaded as a relative path instead of an absolute path, eg:

`Users/bob/.gem/ruby/2.6.6/gems/discourse_dev-0.1.0/auth/plugin.rb`

Instead of

`/Users/bob/.gem/ruby/2.6.6/gems/discourse_dev-0.1.0/auth/plugin.rb`
2021-04-16 15:25:20 +02:00
David Taylor 3436fef5e3
FIX: Update database_restorer to avoid shell use (#12731)
Follow-up to 0ec5fd5262
2021-04-16 13:39:45 +01:00
Régis Hanol c43a7bb23b
FIX: ensure plugin's gems are in the gem path (#12727)
Let's say you want to use a gem in a plugin that has a dependencie.

You would write something like this:

```ruby
gem 'dependency-gem', '1.2.3'
gem 'amazing-gem', '4.5.6'
```

However, since when we install a plugin gem we install it in the `gems`
directory created inside the plugins directory, when it comes the time
to install the `amazing-gem`, it won't be able to find the `dependency-gem`.

This fixes that issue by adding the `gems` plugins directory to the global gem path.

Also fixed a frozen string error when specifying a source.
2021-04-16 10:21:39 +02:00
Vinoth Kannan e08432ddde
DEV: Bump discourse_dev to 0.1.0 (#12722)
And add the "discourse-development-auth" plugin and client locale files.
2021-04-16 06:42:34 +05:30
David Taylor 0ec5fd5262
DEV: Raise exception when execute_command will spawn a shell (#12716) 2021-04-15 16:29:37 +01:00
Dan Ungureanu 85d4b60a45
FIX: Improve error messages for invites (#12714)
The error messages used to include an unnecessary 'Validation failed:
Email' prefix which was removed.
2021-04-15 14:46:32 +03:00
Mark VanLandingham 1d85574d52
FIX: Don't show the get a room composer message in private categories (#12702) 2021-04-14 12:34:13 -05:00
Bianca Nenciu 21d1ee1065
FIX: Use Nokogiri and Loofah consistently (#12693)
CookedPostProcessor used Loofah to parse the cooked content of a post
and Nokogiri to parse cooked Oneboxes. Even though Loofah is built on
top of Nokogiri, replacing an element from the cooked post (a Nokogiri
node) with a parsed onebox (a Loofah node) produced a strange result
which included XML namespaces. Removing the mix and using Loofah
to parse Oneboxes fixed the problem.
2021-04-14 18:09:55 +03:00
Neil Lalonde 07ca35670a
Version bump to v2.7.0.beta7 (#12696) 2021-04-14 10:54:08 -04:00
David Taylor 9430185493
SECURITY: Improve theme git import (#12694) 2021-04-14 15:32:47 +01:00
Bianca Nenciu e4e2c7c66f
FIX: Improve anchor links (#12683)
* FIX: Use theme color for anchor icon

* FIX: Do not count anchor links

* FIX: Do not count hashtags links either

* DEV: Add tests for link_count

* FIX: Disable anchors in quotes and preview

* FIX: Try building some anchor slugs for unicode

* DEV: Fix tests
2021-04-14 10:27:07 +03:00
Martin Brennan eeaecd4fd2
FEATURE: Category setting to allow unlimited first post edits by the owner of the topic (#12690)
This PR adds a new category setting which is a column in the `categories` table, `allow_unlimited_owner_edits_on_first_post`.

What this does is:

* Inside the `can_edit_post?` method of `PostGuardian`, if the current user editing a post is the owner of the post, it is the first post, and the topic's category has `allow_unlimited_owner_edits_on_first_post`, then we bypass the check for `LimitedEdit#edit_time_limit_expired?` on that post.
* Also, similar to wiki topics, in `PostActionNotifier#after_create_post_revision` we send a notification to all users watching a topic when the OP is edited in a topic with the category setting `allow_unlimited_owner_edits_on_first_post` enabled.

This is useful for forums where there is a Marketplace or similar category, where topics are created and then updated indefinitely by the OP rather than the OP making new topics or additional replies. In a way this acts similar to a wiki that only one person can edit.
2021-04-14 15:54:09 +10:00
David Taylor 65647000a1
DEV: Allow `Discourse::Utils.execute_command` timeout with `env` (#12672)
Followup to 5deda5ef3e

The first argument to `Open3.capture3` can be an environment variable hash. In this case, we need to insert the `timeout` command after the env hash.
2021-04-12 13:53:41 +01:00
Osama Sayegh cd24eff5d9
FEATURE: Introduce theme/component QUnit tests (take 2) (#12661)
This commit allows themes and theme components to have QUnit tests. To add tests to your theme/component, create a top-level directory in your theme and name it `test`, and Discourse will save all the files in that directory (and its sub-directories) as "tests files" in the database. While tests files/directories are not required to be organized in a specific way, we recommend that you follow Discourse core's tests [structure](https://github.com/discourse/discourse/tree/master/app/assets/javascripts/discourse/tests).

Writing theme tests should be identical to writing plugins or core tests; all the `import` statements and APIs that you see in core (or plugins) to define/setup tests should just work in themes.

You do need a working Discourse install to run theme tests, and you have 2 ways to run theme tests:

* In the browser at the `/qunit` route. `/qunit` will run tests of all active themes/components as well as core and plugins. The `/qunit` now accepts a `theme_name` or `theme_url` params that you can use to run tests of a specific theme/component like so: `/qunit?theme_name=<your_theme_name>`.

* In the command line using the `themes:qunit` rake task. This take is meant to run tests of a single theme/component so you need to provide it with a theme name or URL like so: `bundle exec rake themes:qunit[name=<theme_name>]` or `bundle exec rake themes:qunit[url=<theme_url>]`.

There are some refactors to how Discourse processes JavaScript that comes with themes/components, and these refactors may break your JS customizations; see https://meta.discourse.org/t/upcoming-core-changes-that-may-break-some-themes-components-april-12/186252?u=osama for details on how you can check if your themes/components are affected and what you need to do to fix them.

This commit also improves theme error handling in Discourse. We will now be able to catch errors that occur when theme initializers are run and prevent them from breaking the site and other themes/components.
2021-04-12 15:02:58 +03:00
David Taylor 35e1e009fa
FIX: Allow restoring non-subfolder backup to subfolder site (#12537)
`GlobalSetting.relative_url_root` comes from the destination site. We
can't be sure whether it was the same on the original site. It's safer
to use a wildcard here, so we can backup/restore sites with different
relative_url_root values.
2021-04-12 14:00:52 +10:00
Penar Musaraj abb0a4bae2
DEV: Add SCSS helper to replace `asset-uri` and `image-uri` (#12664) 2021-04-12 13:57:39 +10:00
Sam 5deda5ef3e
FIX: automatically timeout long running image magick commands (#12670)
Previously certain images may lead to convert / identify to run for unreasonable
amounts of time

This adds a maximum amount of time these commands can run prior to forcing
them to stop
2021-04-12 13:55:54 +10:00
Sam 5b342ae505
FIX: remove superfluous spaces from CJK blurbs (#12629)
Previously we used the raw data indexed to generate blurbs even for cases
when Chinese/Korean/Japanese text was used.

This caused superfluous spaces to show up in excerpts.
2021-04-12 12:46:42 +10:00
Penar Musaraj 43d09dbabe
DEV: Skip looking for svgo, gifsicle binaries (#12655) 2021-04-09 08:52:06 -04:00
Penar Musaraj e77b9dfd45
DEV: Use Terser for JS minification/compression if available (#12656) 2021-04-09 08:51:49 -04:00
Penar Musaraj eb7ff576e5
FEATURE: Use SVG icons for some oneboxes (#12654) 2021-04-08 09:58:12 -04:00
Neil Lalonde 3d4e2595d3
Version bump to v2.7.0.beta6 (#12636) 2021-04-07 15:10:07 -04:00
jbrw 50252d803e
DEV: stub youtube embed requests (#12637)
* DEV: stub youtube embed requests

* DEV: Ignore redirects on youtube.com when oneboxing
2021-04-07 13:32:27 -04:00
Roman Rizzi 5e4c0e2caa
FEATURE: Treat site settings as plain text and add a new HTML type. (#12618)
To add an extra layer of security, we sanitize settings before shipping them to the client. We don't sanitize those that have the "html" type.

The CookedPostProcessor already uses Loofah for sanitization, so I chose to also use it for this. I added it to our gemfile since we installed it as a transitive dependency.
2021-04-07 12:51:19 -03:00
Osama Sayegh 2b9ab3a0d9
Revert "FEATURE: Introduce theme/component QUnit tests (#12517)" (#12632)
This reverts commit a53d8d3e61 and 105634435f.

Reverted because the change broke some components. Will be added back in a few days.
2021-04-07 17:45:49 +03:00
Ahmed Gagan 2308a58113
DEV: Added support for custom site setting 'emoji_list' (#12414)
Example usage:

```
best_emojis:
    type: emoji_list
    default: laughing|open_mouth|cry|angry|hugs
    client: true
```
2021-04-07 15:32:05 +02:00
Osama Sayegh 105634435f
FIX: Prevent double slashes in Ember templates paths (#12630)
Follow-up to https://github.com/discourse/discourse/pull/12517
2021-04-07 14:08:29 +03:00
Osama Sayegh a53d8d3e61
FEATURE: Introduce theme/component QUnit tests (#12517)
This commit allows themes and theme components to have QUnit tests. To add tests to your theme/component, create a top-level directory in your theme and name it `test`, and Discourse will save all the files in that directory (and its sub-directories) as "tests files" in the database. While tests files/directories are not required to be organized in a specific way, we recommend that you follow Discourse core's tests [structure](https://github.com/discourse/discourse/tree/master/app/assets/javascripts/discourse/tests).

Writing theme tests should be identical to writing plugins or core tests; all the `import` statements and APIs that you see in core (or plugins) to define/setup tests should just work in themes.

You do need a working Discourse install to run theme tests, and you have 2 ways to run theme tests:

* In the browser at the `/qunit` route. `/qunit` will run tests of all active themes/components as well as core and plugins. The `/qunit` now accepts a `theme_name` or `theme_url` params that you can use to run tests of a specific theme/component like so: `/qunit?theme_name=<your_theme_name>`.

* In the command line using the `themes:qunit` rake task. This take is meant to run tests of a single theme/component so you need to provide it with a theme name or URL like so: `bundle exec rake themes:qunit[name=<theme_name>]` or `bundle exec rake themes:qunit[url=<theme_url>]`.

There are some refactors to internal code that's responsible for processing themes/components in Discourse, most notably:

* `<script type="text/discourse-plugin">` tags are automatically converted to modules.

* The `theme-settings` service is removed in favor of a simple `lib` file responsible for managing theme settings. This was done to allow us to register/lookup theme settings very early in our Ember app lifecycle and because there was no reason for it to be an Ember service.

These refactors should 100% backward compatible and invisible to theme developers.
2021-04-07 10:39:57 +03:00
Neil Lalonde e8a9917db1
FEATURE: Allow setting avatar flair for automatic groups (#12586) 2021-04-06 11:13:06 -04:00
Bianca Nenciu e026af11ac
FIX: Simplify send PM to email settings (#12583)
This feature used to be controlled by two site settings 
enable_personal_email_messages and min_trust_to_send_email_messages.
I removed enable_personal_email_messages and unhide 
min_trust_to_send_email_messages to simplify the process of 
enabling / disabling this feature.
2021-04-06 15:39:27 +03:00
Jeff Wong 3a22c654e2
FIX: rake themes:update should fail if a theme update fails (#12608) 2021-04-05 15:53:34 -07:00
Arpit Jalan 6234d7455b
FEATURE: add maximum limit for secondary emails (#12599) 2021-04-05 20:31:42 +05:30
Arpit Jalan c478ffc662
FIX: post merging was failing silently (#12566)
https://meta.discourse.org/t/merging-very-long-posts-removes-them/183597
2021-04-01 06:46:18 +05:30
jbrw 68d0916eb5
FEATURE: Oneboxer cache response body (#12562)
* FEATURE: Cache successful HTTP GET requests during Oneboxing

Some oneboxes may fail if when making excessive and/or odd requests against the target domains. This change provides a simple mechanism to cache the results of succesful GET requests as part of the oneboxing process, with the goal of reducing repeated requests and ultimately improving the rate of successful oneboxing.

To enable:

Set `SiteSetting.cache_onebox_response_body` to `true`

Add the domains you’re interesting in caching to `SiteSetting. cache_onebox_response_body_domains` e.g. `example.com|example.org|example.net`

Optionally set `SiteSetting.cache_onebox_user_agent` to a user agent string of your choice to use when making requests against domains in the above list.

* FIX: Swap order of duration and value in redis call

The correct order for `setex` arguments is `key`, `duration`, and `value`.

Duration and value had been flipped, however the code would not have thrown an error because we were caching the value of `1.day.to_i` for a period of 1 seconds… The intention appears to be to set a value of 1 (purely as a flag) for a period of 1 day.
2021-03-31 13:19:34 -04:00
Dan Ungureanu dce48d8aa7
FIX: Redirect to provided origin after auth (#12558)
It used to redirect to the destination_url cookie which sometimes is set
incorrectly.
2021-03-31 10:23:12 +01:00
Bianca Nenciu 52a5c7e438
FIX: Improve error messages if user cannot send PM emails (#12547) 2021-03-30 12:18:57 +03:00
Penar Musaraj 5096920500
FEATURE: Implement nonces for Google Tag Manager integration (#12531) 2021-03-26 11:19:31 -04:00
Martin Brennan 8de07181a8
FEATURE: Add new :topic_tags_changed DiscourseEvent (#12530)
This is called in DiscourseTagging.tag_topic_by_names only after
all the validations etc. have been passed, and after topic.tags = X
has been called (because this is when the associations are created/
destroyed). The event has the topic, then a second param with the
old and new tag names in arrays for easy inspection.
2021-03-26 13:53:47 +10:00
David Taylor f637bf1b58
DEV: Deprecate `message` parameter in auth provider plugin API (#12523)
This has been unused since d2bceff133
2021-03-25 13:23:48 +00:00
Gerhard Schlager acc73f2989
FIX: `uploads:fix_missing_s3` rake task used wrong SHA1 (#12495) 2021-03-25 11:35:29 +01:00
Krzysztof Kotlarek c03c85e661
FIX: delete orphan post revisions (#12502)
I was adding specs to ensure that post actions and uploads are removed for permanently deleted posts.

I noticed that post revisions were not permanently destroyed. I added a migration to fix old data.
2021-03-25 12:34:53 +11:00
Martin Brennan ea6f9af08b
FIX: Add topic_diff to PostRevisor (#12518)
The instance of the PostRevisor is passed to the post_edited
event. It is useful to know what has happened to the topic in
this event (we already pass a boolean for topic_changed? but that
is not so helpful by itself).
2021-03-25 10:24:50 +10:00
Bianca Nenciu 9dca7fe9a3
DEV: Replace Rails ends_with? with Ruby end_with? (#12507) 2021-03-24 19:51:21 +02:00
Mark VanLandingham 371afc45e0
DEV: API for plugins to add post update params and handlers (#12505) 2021-03-24 10:22:16 -05:00
Bianca Nenciu e7fb45cc29
FEATURE: Treat emoji or similar characters as one (#12482)
Long messages consisting only of emojis, dots or commas used to be
valid because character-wise they were over the limit.
2021-03-24 16:47:35 +02:00
Sam dc6b547ed8
FIX: rake db:validate_indexes was broken (#12463)
A file was moved, but zeitwerk can not find it due to custom inflection.

Renamed so it can be properly found.
2021-03-24 08:53:01 +11:00
Sam 5a1318e3c3
DEV: upgrade mini_sql (#12465)
* DEV: upgrade mini_sql

Even though we are not planning on using this quite yet, mini_sql now supports
prepared statements.

Would like this upgrade merged so we can do some benchmarking.

Note, this will not work with pg_bouncer, but sites that are not using it
may benefit from the feature.

* implement multisite friendly prepared statements
2021-03-24 08:48:04 +11:00
Jarek Radosz 6ff888bd2c
DEV: Retry-after header values should be strings (#12475)
Fixes `Rack::Lint::LintError: a header value must be a String, but the value of 'Retry-After' is a Integer`. (see: 14a236b4f0/lib/rack/lint.rb (L676))

I found it when I got flooded by those warning a while back in a test-related accident 😉 (ember CLI tests were hitting a local rails server at a fast rate)
2021-03-23 20:32:36 +01:00
Penar Musaraj 066c59d0e3
FIX: Regression in colors used by non-default theme (#12492)
eb7f0ec caused this regression, where a non-default theme set to use
the base color scheme was resolving to the default theme's color scheme.
2021-03-23 12:46:25 -04:00
Rafael dos Santos Silva fb4486d5f1
FEATURE: Add CSP frame-ancestors support (#12404) 2021-03-22 16:00:25 -03:00
Dan Ungureanu 4e46732346
FEATURE: Implement browser update in crawler view (#12448)
browser-update script does not work correctly in some very old browsers
because the contents of <noscript> is not accessible in JavaScript.
For these browsers, the server can display the crawler page and add the
browser update notice.

Simply loading the browser-update script in the crawler view is not a
solution because that means all crawlers will also see it.
2021-03-22 19:41:42 +02:00
Bianca Nenciu 129aeb8c2f
FIX: Improvements to email styles (#12451) 2021-03-22 14:09:38 +02:00
Martin Brennan 6eb0d0c38d
SECURITY: Fix is_private_ip for RateLimiter to cover all cases (#12464)
The regular expression to detect private IP addresses did not always detect them successfully.
Changed to use ruby's in-built IPAddr.new(ip_address).private? method instead
which does the same thing but covers all cases.
2021-03-22 13:56:32 +10:00
Martin Brennan 49f4c548ef
FEATURE: Bookmark pinning (#12431)
Users can now pin bookmarks from their bookmark list. This will anchor the bookmark to the top of the list, and show a pin icon next to it. This also applies in the nav bookmarks panel. If there are multiple pinned bookmarks they sort by last updated order.
2021-03-22 09:50:22 +10:00
David Taylor 56a573ab4b
DEV: Remove String#match? and Regexp.match? polyfills (#12454)
This was added in Ruby 2.4. Discourse will no longer run on ruby versions older than that, so we can drop this polyfill
2021-03-19 19:55:44 +00:00
Robin Ward af5adc440e
DEV: We hadn't enabled `object-rest-spread` in all contexts (#12452)
See: https://babeljs.io/docs/en/babel-plugin-proposal-object-rest-spread

This fixes Discourse in browsers that don't support this syntax yet.
2021-03-19 13:57:04 -04:00
Martin Brennan fb0b4e26a9
DEV: Add output_sql_to_stderr! to MethodProfiler (#12445)
This PR adds MethodProfiler.output_sql_to_stderr! for easier debugging of SQL queries and their timings from the console.

This is almost the same as ensure_discourse_instrumentation! but should not
be used in production (save for debugging in the console), and is only instrumenting
PostgresSQL queries.

This is almost the same as ensure_discourse_instrumentation! but should not
be used in production. This logs all SQL queries run and their durations
between start and stop.

It also works for super long running queries. If you interrupt the long-running
query the latest query data will still be logged after stopping the profiler.

Usage:

```
MethodProfiler.output_sql_to_stderr!(filter_transactions: true)
MethodProfiler.start

# some code that runs queries

timings = MethodProfiler.stop
```
2021-03-19 17:48:30 +10:00
Martin Brennan 355d51afde
FEATURE: Allow using invites when DiscourseConnect SSO is enabled (#12419)
This PR allows invitations to be used when the DiscourseConnect SSO is enabled for a site (`enable_discourse_connect`) and local logins are disabled. Previously invites could not be accepted with SSO enabled simply because we did not have the code paths to handle that logic.

The invitation methods that are supported include:

* Inviting people to groups via email address
* Inviting people to topics via email address
* Using invitation links generated by the Invite Users UI in the /my/invited/pending route

The flow works like this:

1. User visits an invite URL
2. The normal invitation validations (redemptions/expiry) happen at that point
3. We store the invite key in a secure session
4. The user clicks "Accept Invitation and Continue" (see below)
5. The user is redirected to /session/sso then to the SSO provider URL then back to /session/sso_login
6. We retrieve the invite based on the invite key in secure session. We revalidate the invitation. We show an error to the user if it is not valid. An additional check here for invites with an email specified is to check the SSO email matches the invite email
7. If the invite is OK we create the user via the normal SSO methods
8. We redeem the invite and activate the user. We clear the invite key in secure session.
9. If the invite had a topic we redirect the user there, otherwise we redirect to /

Note that we decided for SSO-based invites the `must_approve_users` site setting is ignored, because the invite is a form of pre-approval, and because regular non-staff users cannot send out email invites or generally invite to the forum in this case.

Also deletes some group invite checks as per https://github.com/discourse/discourse/pull/12353
2021-03-19 10:20:10 +10:00
Penar Musaraj aee7ef0dc9
DEV: Fix build due to highlight.js branch issue (#12441)
Highlight.js changed their default branch from master to main. This switches to the @highlightjs/cdn-assets package, thus sidestepping the problem. It's a slightly cleaner integration though (no need to build locally anymore).
2021-03-18 18:21:23 -04:00
Dan Ungureanu 033d6b6437
FEATURE: Obfuscate emails on invite show page (#12433)
The email should not be ever displayed in clear text, except the case
when the user authenticates using another service.
2021-03-18 19:09:23 +02:00
Penar Musaraj eb7f0ec766
FIX: In components, fall back to default theme color variables (#12423)
Component SCSS compilation should use the current theme's SCSS color
variables as a fallback before using the default core colors.

This is mostly a backwards-compatibility fix, new themes and components
should use CSS custom properties, which offer better support for on-the-fly
color scheme changes (dark mode support, etc.).
2021-03-17 13:34:15 -04:00
Osama Sayegh d56b2e85aa
FIX: Escape Font Awesome icons (#12421)
This is not a security issue because regular users are not allowed to insert FA icons anywhere in the app. Admins can insert icons via custom badges, but they do have the ability to create themes with JS.
2021-03-17 16:11:40 +03:00
Arpit Jalan 64d2f260a9
FIX: do not convert format for site setting uploads (#12410) 2021-03-16 20:14:41 +05:30
Krzysztof Kotlarek e10a74694a
FIX: deprecation warning - initialization autoloaded the constant (#12400)
Get rid of deprecation related to Zeitwerk autoloader.

Original PR was reverted because of multisite bug #12381 - thank you @davidtaylorhq for fixing it.

I added the last commit to fix that multisite problem.
2021-03-16 09:47:57 +11:00
Rafael dos Santos Silva f8d9f09370
FEATURE: Fallback to system users when creating new TopicEmbed (#12386)
Previosly, if the topic embed request had a missing username parameter,
and SiteSetting.embed_by_username was empty we would fail to create the
new topic and not show any errors.

Now we will fallback using the priority:

1. Username parameter
2. SiteSetting.embed_by_username
3. SiteSetting.site_contact_username
4. system user
2021-03-15 11:58:53 -03:00
Penar Musaraj de05c410c5
DEV: Use fewer threads when watching for SCSS changes (#12393) 2021-03-12 16:18:00 -05:00
Daniel Waterworth e7ac906f21
DEV: Move TemporaryDB class to its own file (#12383)
rake tasks get repeated reloaded during the tests and it causes
redefinition warnings.
2021-03-12 11:27:13 -06:00
Penar Musaraj 52d833472c
DEV: Refactor plugin SCSS compilation (#12359) 2021-03-12 11:17:42 -05:00
David Taylor 1533cbb38b
Revert "FIX: deprecation warning - initialization autoloaded the constant (#12347)" (#12381)
This reverts commit 9a5a38a529.

This change was preventing `rake multisite:migrate` from working
2021-03-12 13:48:46 +00:00
Rafael dos Santos Silva 9737ce8c5f
FEATURE: New DiscourseEvent trigger for likes (#12374) 2021-03-11 21:07:29 -03:00
Krzysztof Kotlarek 9a5a38a529
FIX: deprecation warning - initialization autoloaded the constant (#12347)
Get rid of deprecation related to Zeitwerk autoloader.
2021-03-12 08:42:53 +11:00
Roman Rizzi 56262349a9
FIX: When destroying a topic, destroy its associated published page. (#12358)
Destroying a topic should remove the associated published page so another topic could re-use the slug.
2021-03-11 11:34:54 -03:00
Roman Rizzi 8fcad73b36
FEATURE: Admins can flag posts so they can review them later. (#12311)
Staff can send a post to the review queue by clicking the "Flag Post" button next to "Take Action...". Clicking it flags the post using the "Notify moderators" score type and hides it. A custom message will be sent to the user.
2021-03-11 08:21:24 -03:00
Penar Musaraj 10780d2448
DEV: support json_schema in theme settings (#12294) 2021-03-10 20:15:04 -05:00
jbrw aed97c7bab
FIX: Add amazon sites to force_get_hosts (#12341)
It has been observed that doing a HEAD against an Amazon store URL may result in a 405 error being returned.

Skipping the HEAD request may result in an improved oneboxing experience when requesting these URLs.
2021-03-10 14:42:17 -05:00
Neil Lalonde 5b8941c568
Version bump to v2.7.0.beta5 (#12287) 2021-03-10 13:45:19 -05:00
Penar Musaraj 4071af1d09
DEV: Refactor font and category background importers (#12312) 2021-03-10 11:05:56 -05:00
Bianca Nenciu 9bd436c20b
FIX: Do not add same email multiple times (#12322)
The user and an admin could create multiple email change requests for
the same user. If any of the requests was validated and it became
primary, the other request could not be deleted anymore.
2021-03-10 14:49:26 +02:00
Sam 10001e4e8d
DEV: add db:validate_indexes task (#12273)
Added a new task to test if indexes are coherent with a blank database

This allows us to detect for cases where somehow indexes are out of sync

FIX_INDEXES=1 or `rake db:validate_indexes[fix]` to correct the issues it finds. 

Detects: 

- Badly named indexes that need to be renamed
- Missing indexes
- Extra indexes

Can correct all 3 with the fix option
2021-03-10 10:59:20 +11:00
jbrw ac7773a30d
FEATURE: allow category group moderators to pin/unpin topics (#12325)
* FEATURE: allow category group moderators to pin/unpin topics

Category group moderators should be able to pin/unpin any topics within a category where they have appropraite category group moderator permissions.
2021-03-09 16:05:11 -05:00
Alan Guo Xiang Tan ce04db8610 FEATURE: Allow invites redemption with Omniauth providers. 2021-03-09 09:27:18 +08:00
Alan Guo Xiang Tan ebe4896e48 FEATURE: Change very high/low search priority to rank at absolute ends.
Prior to this change, we had weights for very_high, high, low and
very_low. This means there were 4 weights to tweak and what weights to
use for `very_high/high` and `very_low/low` pair was hard to explain.
This change makes it such that `very_high` search priority will always
ensure that the posts are ranked at the top while `very_low` search
priority will ensure that the posts are ranked at the very bottom.
2021-03-09 09:20:37 +08:00
Penar Musaraj 0a656767c7
DEV: use Redis data for version check in DiscourseUpdates (#12293)
A version bump caused the specs to fail, this switches to using what is
stored in Redis, which is also what is stubbed in the specs.
2021-03-08 09:05:08 -05:00
Vinoth Kannan 8d96713aa0
DEV: add `discourse_dev` gem in development dependencies. (#12285)
For more details visit https://github.com/discourse/discourse_dev
2021-03-04 23:04:51 +05:30
Rafael dos Santos Silva e06076268a
FIX: Missing quote in emoji cdn setting caused by 83f332b (#12280) 2021-03-03 17:39:00 -03:00
Gerhard Schlager 58c218a4bf
FIX: Remap old S3 endpoints during backup restore (#12276)
It also starts outputting exceptions on the console.
2021-03-03 21:10:09 +01:00
Dan Ungureanu c047640ad4
FEATURE: Various improvements to invite system (#12023)
The user interface has been reorganized to show email and link invites
in the same screen. Staff has more control over creating and updating
invites. Bulk invite has also been improved with better explanations.

On the server side, many code paths for email and link invites have
been merged to avoid duplicated logic. The API returns better responses
with more appropriate HTTP status codes.
2021-03-03 11:45:29 +02:00
Krzysztof Kotlarek 38ab45cb93
FIX: check min tag count requirement when change category (#12252)
When a category is created, we can set `minimum_required_tags` property.

When the topic is created, we are checking that field and ensuring that the minimum amount of tags were added - unless topic created by a staff member.

Problem is that validation is skipped when we change the category from for example category with 0 tags required to the category with 1 tag required.

Changing category is kind of the unicorn as it is a complicated operation:

https://github.com/discourse/discourse/blob/master/lib/post_revisor.rb#L84

https://github.com/discourse/discourse/blob/master/app/models/topic.rb#L911

https://github.com/discourse/discourse/blob/master/app/models/topic.rb#L823

Before we start to try to change the category, we should ensure that the tags requirement is fulfilled.

https://meta.discourse.org/t/the-category-setting-for-tags-is-not-respected/181214
2021-03-03 10:59:23 +11:00
Rafael dos Santos Silva 83f332b5a5
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180) 2021-03-02 16:04:16 -03:00
Gerhard Schlager a96a5db0fb
DEV: Add option to send system message to groups (#12256) 2021-03-02 18:51:50 +01:00
David Taylor 8fd46c04ea
Drop flash video onebox (#12261)
Flash was discontinued by Adobe at the end of 2020. There is no need to continue OneBox support for it
2021-03-02 17:11:14 +00:00
Mark VanLandingham 4adce0d844
DEV: APIs for plugin to add custom reviewable confirm modal (#12246) 2021-03-02 10:28:27 -06:00
Gerhard Schlager 154bfcf750
FEATURE: Include details about S3 backup storage errors (#12257) 2021-03-02 15:29:37 +01:00
Martin Brennan 6b4d066834
FIX: Skip sending PM email for user silence (#12240)
We were sending 2 emails for user silencing if a message was provided in the UI. Also always send email for user silence and user suspend with reason regardless of whether message provided.
2021-03-02 09:18:09 +10:00
Robin Ward 79502b5b10
FIX: Use `addRawTemplates` even when compiling `.hbr` files (#12228)
Without this patch, we'd be using the `__RAW_TEMPLATES` global which is
incompatible with Ember CLI.
2021-03-01 11:46:50 -05:00
jbrw a9b6f4d829
FIX - use ImageMagick to determine size of svg images (#12230)
SVG files can have dimensions expressed in inches, centimeters, etc., which may lead to the dimensions being misinterpreted (e.g. “8in” ends up as 8 pixels).

If the file type is `svg`, ask ImageMagick to work out what size the SVG file should be rendered on screen.

NOTE: The `pencil.svg` file was obtained from https://freesvg.org/1534028868, which has placed the file in to the public domain.
2021-03-01 11:44:00 -05:00
Penar Musaraj 6809cccd88
DEV: Add experimental json_scheme site setting type (#12226) 2021-03-01 09:15:17 -05:00
Penar Musaraj cf0192018e
FIX: Do not output empty style tags for components (#12229) 2021-03-01 09:14:25 -05:00
Penar Musaraj aa1442fdc3
DEV: Use separate files for theme component stylesheets (take 2) (#12225)
This switches to outputting a separate file for each theme component CSS
asset. We have separate CSS plugin files, separate JS files
(for plugins/themes/components), it makes sense to do the same for
component CSS assets.

Benefits:
- easier debugging
- fixes a regression with theme component sourcemaps
- changes to theme components are updated individually

With HTTP/2, there is also no performance downside to having additional
files in the initial request.
2021-02-26 12:30:23 -05:00
Penar Musaraj e74bdfdf8e
Revert "DEV: Use separate files for theme component stylesheets (#12214)" (#12224)
This reverts commit f57a49c2f9.

This had some unexpected side effects, needs some more work.
2021-02-26 08:20:39 -05:00
Bianca Nenciu 4b0496b2fc
FIX: Reintroduce auto_silence_first_post_regex (#12223)
Follow-up to 533800a87b.
2021-02-26 15:07:04 +02:00
Penar Musaraj 45c5fd2172
DEV: Remove JoyPixels emoji option (#12197)
- removes the option from site settings
- deletes the site setting on existing sites that have it
- marks posts using emojis as requiring a rebake

Note that the actual image files are not removed here, the plan is to
remove them in a few weeks/months (when presumably the rebaking of old
posts has been completed).
2021-02-26 07:44:52 -05:00
Penar Musaraj f57a49c2f9
DEV: Use separate files for theme component stylesheets (#12214)
This switches to outputting a separate file for each theme component CSS
asset. We have separate CSS plugin files, separate JS files
(for plugins/themes/components), it makes sense to do the same for
component CSS assets.

Benefits:
- easier debugging
- fixes a regression with theme component sourcemaps
- changes to theme components are updated individually

With HTTP/2, there is also no performance downside to having additional
files in the initial request.
2021-02-26 07:44:15 -05:00
Alan Guo Xiang Tan 3fc72543de DEV: Avoid duplicating method attributes in `Auth::Result`.
This ensures that within the class, we have a consistent order of
attributes that is passed to `UsernameSuggester`.
2021-02-26 10:20:31 +08:00
jbrw fff8a24f2b
FIX: Don’t display error if only error is a missing image (#12216)
`Onebox.preview` can return 0-to-n errors, where the errors are missing OpenGraph attributes (e.g. title, description, image, etc.). If any of these attributes are missing, we construct an error message and attach it to the Oneboxer preview HTML. The error message is something like:

 “Sorry, we were unable to generate a preview for this web page, because the following oEmbed / OpenGraph tags could not be found: description, image”

However, if the only missing tag is `image` we don’t need to display the error, as we have enough other data (title, description, etc.) to construct a useful/complete Onebox.
2021-02-25 14:30:40 -05:00
Bianca Nenciu 533800a87b
Add watched words of type "replace" (#12020)
This commit includes other various improvements to watched words.

auto_silence_first_post_regex site setting was removed because it overlapped
with 'require approval' watched words.
2021-02-25 14:00:58 +02:00
Bianca Nenciu a9a93b15ec
FIX: Strip whitespace from URL before validating (#12174) 2021-02-25 13:31:38 +02:00
Martin Brennan 13c2a4886f
FEATURE: Add disable_onebox_media_download_controls hidden site setting (#12208)
Uses discourse/onebox@ff9ec90

Adds a hidden site setting called disable_onebox_media_download_controls which will add controlslist="nodownload" to video and audio oneboxes, and also to the local video and audio oneboxes within Discourse.
2021-02-25 12:39:15 +10:00
Bianca Nenciu 0c8d658ba8
SECURITY: Prefer Loofah for processing cooked HTML 2021-02-24 17:17:49 +02:00
Penar Musaraj c0fd4ab9d6
DEV: Remove deprecated plugins variables importer (#12168) 2021-02-23 16:20:59 -05:00
Rafael dos Santos Silva 01584c5b22
FIX: emails.rake-use domain when sending mail test (#12188)
`rake emails:test` task was always sending `localhost` as the domain name rather than using `smtp[:domain]` (aka `DISCOURSE_SMTP_DOMAIN`. `discourse-setup` has recently been modified to always set `DISCOURSE_SMTP_DOMAIN`, so it's important that this test rake task actually use the value.

I tested this change on a standard production install, and it's working as expected. Hopefully I managed not to bungle the copy/paste of the single line here in the github edit window.

Co-authored-by: Jay Pfaffman <jay@literatecomputing.com>
2021-02-23 14:16:51 -03:00
Penar Musaraj 0620f6298e
UX: Remove emoji step in wizard (#12169) 2021-02-22 12:14:18 -05:00
David Taylor a040f72f96
FIX: Make email_valid handling consistent (#11556)
Previously we were checking truthiness in some places, and `== true` in
others. That can lead to some inconsistent UX where the interface says
the email is valid, but account creation fails.

This commit ensures values are boolean when set, and raises an error for
other value types.

If this safety check is triggered, it means the specific auth provider
needs to be updated to pass booleans.
2021-02-22 12:05:36 +00:00
David Taylor 04c75d417b
UX: Skip github commit avatars for topic/post thumbnails (#12157)
GitHub oneboxes use `.onebox-avatar-inline`, not `.onebox-avatar`
2021-02-22 10:40:40 +00:00
Penar Musaraj 5604ce70d4
DEV: More refactoring of SCSS importers (#12143) 2021-02-19 11:22:24 -05:00
Penar Musaraj 7ee660a017
FIX: Sort admin dashboard new updates by latest (#12146) 2021-02-19 11:03:36 -05:00
Neil Lalonde 180249112f Version bump to v2.7.0.beta4 2021-02-18 14:21:37 -05:00
Bianca Nenciu 3246c3cc92
DEV: Update mail and use fork (#10639)
Version 2.8 brings some changes to how address fields are handled and
this commits updates that and should also include a fix which handles
encoded attachment filenames.

The fork contains a bugfix to correctly decode mail attachments.
2021-02-18 20:15:02 +02:00
jbrw 447ea3f888
FIX: NewPostManager should respect category_group_moderator settings (#12116)
NewPostManager’s `post_needs_approval_in_its_category` method should allow category group moderators to create topics/reply to topics that where they have appropraite permissions.

(ie, if a user has permission to moderate a post, any posts made by them shouldn’t be sent to moderation)
2021-02-18 13:00:06 -05:00
Joshua Rosenfeld b79c06f3b7
Include UI commits when generating release notes. (#12127)
`UX` is the officially supported prefix per https://meta.discourse.org/t/19392, but sometimes `UI` is used instead. We should still include those commits.
2021-02-18 10:53:23 -05:00
Bianca Nenciu 08acf51be0
FEATURE: Use diffhtml to update composer preview (#11237)
Displaying videos, animated GIFs or any kind of rich content in preview
used to refresh on every keystroke, which could cause performance
problems.
2021-02-18 16:07:26 +02:00
David Taylor 13d2a1f82c
SECURITY: Attach DiscourseConnect (SSO) nonce to current session (#12124) 2021-02-18 10:35:10 +00:00
Roman Rizzi 07cf0f9460
FIX: Allow plugins to correctly extend API key scopes. (#12113)
Adding a scope from a plugin was broken. This commit fixes it and adds a test.

It also documents the instance method and renames the serialized "id" attribute to "scope_id" to avoid a conflict when the scope also has a parameter with the same name.
2021-02-17 14:42:44 -03:00
Arpit Jalan a174c8b8d4
FIX: hide sso payload behind a button click and log views (#12110) 2021-02-17 21:27:51 +05:30
David Taylor 9d7dc89d79
DEV: Replace deprecated sso site settings with discourse_connect (#12099) 2021-02-17 09:31:20 +00:00
Jeff Wong f17e745fe3
DEV: log rake plugin:update_all plugin_path on error (#12100)
* DEV: log rake plugin:update_all plugin_path on error

Allows a dev to know which plugin was failing to update
2021-02-16 11:50:27 -08:00
Martin Brennan 9f0f801ae3
FIX: Do not mark group_flair images as secure on upload (#12081)
See https://meta.discourse.org/t/secure-media-uploads-breaks-group-flair-image/173671/4

Group flair image uploads definitely do not need to be secure.
2021-02-16 12:34:03 +10:00
Arpit Jalan 85c4e8fd32
FEATURE: support `mark` tag (#12088)
This commit adds support for `mark` tag for highlighting text content.
2021-02-15 21:47:30 +05:30
Gerhard Schlager 4b05fc2d2d
FIX: Restoring backup via UI was broken (#12089) 2021-02-15 16:46:44 +01:00
jbrw 70050a8ba3
FIX: should_alter_quality should respect png_to_jpg_quality (#12055)
`convert_to_jpeg!` is only called if `convert_png_to_jpeg?` and/or `should_alter_quality?` is true.

`convert_png_to_jpeg?` can be disabled by setting `SiteSetting.png_to_jpg_quality` to 100.

However, `should_alter_quality?` could be true if `SiteSetting.recompress_original_jpg_quality` was lower than the quality of the uploaded file, regardless of file type.

This commits changes `should_alter_quality?` so that uploaded png files will use the `SiteSetting.png_to_jpg_quality` value, rather than ``SiteSetting.recompress_original_jpg_quality` value.
2021-02-12 13:37:35 -05:00
Penar Musaraj 74ca5ec743
FIX: Regression in admin new features (#12068) 2021-02-12 10:49:10 -05:00
Penar Musaraj 900d4187ef
DEV: Prevents rate limits for new feature checks on multisite (#12053) 2021-02-12 08:52:59 -05:00
Daniel Waterworth df8436cd7f
FIX: Don't add a slug to constructed quote urls (#12052)
A topic with the slug 'topic' might exist and may end up being linked to
by mistake when malformed (i.e. cross-site) quotes are posted.
2021-02-11 12:21:13 -06:00
David Taylor b770c30391
FEATURE: Allow onebox images to be used as topic thumbnails (#12050)
Still excludes GitHub avatars. Those were the original reason for adding
this broad exclusion. Context at https://meta.discourse.org/t/165713/4

If we find more oneboxes which are unsuitable for thumbnails, we can add
them to this selector.
2021-02-11 17:50:42 +00:00