Commit Graph

41808 Commits

Author SHA1 Message Date
Arpit Jalan b27674597c
FIX: redirect non-staff user to homepage when deleting own topic (#13267) 2021-06-03 20:27:29 +05:30
David Taylor 4134173bbf
FEATURE: Add global admin api key rate limiter (#12527) 2021-06-03 10:52:43 +01:00
Alan Guo Xiang Tan 58b30fb510 PERF: Preload settings, groups and badge icons in SvgSprite.
Identified as a hot path in production. Preload it early instead of
executing the queries in a live request.
2021-06-03 16:45:56 +08:00
Martin Brennan eb2c399445
FEATURE: Use group SMTP settings for sending user notification emails (initial) (#13220)
This PR changes the `UserNotification` class to send outbound `user_private_message` using the group's SMTP settings, but only if:

* The first allowed_group on the topic has SMTP configured and enabled
* SiteSetting.enable_smtp is true
* The group does not have IMAP enabled, if this is enabled the `GroupSMTPMailer` handles things

The email is sent using the group's `email_username` as both the `from` and `reply-to` address, so when the user replies from their email it will go through the group's SMTP inbox, which needs to have email forwarding set up to send the message on to a location (such as a hosted site email address like meta@discoursemail.com) where it can be POSTed into discourse's handle_mail route.

Also includes a fix to `EmailReceiver#group_incoming_emails_regex` to include the `group.email_username` so the group does not get a staged user created and invited to the topic (which was a problem for IMAP), as well as updating `Group.find_by_email` to find using the `email_username` as well for inbound emails with that as the TO address.

#### Note

This is safe to merge without impacting anyone seriously. If people had SMTP enabled for a group they would have IMAP enabled too currently, and that is a very small amount of users because IMAP is an alpha product, and also because the UserNotification change has a guard to make sure it is not used if IMAP is enabled for the group. The existing IMAP tests work, and I tested this functionality by manually POSTing replies to the SMTP address into my local discourse.

There will probably be more work needed on this, but it needs to be tested further in a real hosted environment to continue.
2021-06-03 14:47:32 +10:00
Osama Sayegh 3249312c81
FIX: Escape periods in current user's username before generating `RegExp` (#13247)
If we don't escape periods, they are interpreted as wildcards and it
becomes impossible to visit profiles of other users whose usernames
match. E.g., if your username was `a.c` and attempted to visit `abc`'s
profile, you would be incorrectly redirected to your own profile.
2021-06-03 14:15:38 +10:00
Kris f3e021ad45
UX: Update search panel styles, consistency (#13262) 2021-06-03 14:14:24 +10:00
Bianca Nenciu d184fe59ca
FEATURE: Censor Oneboxes (#12902)
Previously onebox content was not passed by the censor regex, meaning you could sneak in censored words via onebox.
2021-06-03 11:39:12 +10:00
Alan Guo Xiang Tan 58cb120aa2
DEV: Minor code clean up in assets.rake. (#13245) 2021-06-03 11:37:06 +10:00
jbrw b57dca90fc
Revert "FIX: Ignore `allowlistgeneric` Onebox image sizes (#13240)" (#13261)
This reverts commit abac614492.

The CSS selector I was trying to use does not appear to be widely supported.
2021-06-02 21:36:46 -04:00
Bianca Nenciu 648d2fd793
DEV: Add test for link watched words (#13251) 2021-06-03 11:36:07 +10:00
Alan Guo Xiang Tan 8cfe203383 PERF: Defer setting of distributed cache in performance critical paths.
Setting a key/value pair in DistributedCache involves waiting on the
write to Redis to finish. In most cases, we don't need to wait on the
setting of the cache to finish. We just need to take our return value
and move on.
2021-06-03 09:30:52 +08:00
Martin Brennan 83211cff25
FIX: Change order of topic_tracking_state SELECT SQL (#13259)
This allows us to do DISTINCT on the topic_id to remove
duplicates (e.g. in extensions to the report SQL), and
also introduces an additional_join_sql string to allow
extensions to JOIN additional tables.
2021-06-03 11:21:33 +10:00
Martin Brennan 006d52f32b
FIX: Dismiss topics keyboard shortcut not working (#13260)
This issue is a result of
7a79bd7da3,
where the ID for the bottom Dismiss Topic buttons changed to
dismiss-topic-bottom.
2021-06-03 11:20:20 +10:00
jbrw abac614492
FIX: Ignore `allowlistgeneric` Onebox image sizes (#13240)
* FIX: Ignore `allowlistgeneric` Onebox image sizes

The size of an image contained within the preview pane of a Composer window may vary depending on the configuration of the browser displaying the Composer (e.g., dimension of browser window, zoom level, etc.).

Presently, the dimensions of the images from the browser creating the post containing the Onebox will be used to render the Onebox to anyone who views the post. It is safer to let the backend figure out the dimensions of the images. Therefore, exclude `.onebox.allowlistedgeneric` images from the list of `image_sizes` sent to the backend.

* DEV: Replace jQuery selector with pure JS

* DEV: remove more jQuery
2021-06-02 20:02:13 -04:00
Jordan Vidrine 188ac1c51f
FIX: Add check for if element doesnt exist on ensureDropClosed (#13256) 2021-06-02 13:44:21 -05:00
Jarek Radosz 3bb765ac92
DEV: Remove the remaining Travis code (#13255)
The second attempt at #10041 now that all our plugins use GitHub Actions CI instead.
2021-06-02 20:29:47 +02:00
jbrw 4a7c043382
DEV: bundle update fastimage (#13253)
FastImage 2.2.4 - includes a fix for BOM characters in SVG files, and HEIC support
2021-06-02 14:06:20 -04:00
Penar Musaraj d3e9a028f5
SECURITY: Do not allow unauthorized access to category edit UI (#13252) 2021-06-02 13:18:45 -04:00
Penar Musaraj fd9ef14ec0
FIX: Show required tags to staff by default and override limit (#13242)
This improves the display of available tags in categories that are
configured to require at least (x) tags from a tag group.

There are two changes included:
- regular users will now see all the available tags in the required tag
group (previously they could see a max. of 5 tags)
- staff users will now see the tags from the required tag group when
the tag group contains more tags than the default limit (also set to 5)

Both changes only apply to the default query (i.e. no search terms).
2021-06-02 12:43:34 -04:00
Bianca Nenciu e81a5182b3
FEATURE: Stage post edits immediately (#13249)
After editing a post, it is refreshed by two ways. One of them is
triggered by the client side which will route the client to the edited
post and force a reload this way. The other way is via Message Bus.

This commit ignores both of the ways and tries to update the post
immediately and then refresh the post stream.
2021-06-02 17:30:36 +03:00
Andrei Prigorshnev 2672358b72
FIX: active record annotation of topic model (#13250)
I've removed constraint from the excerpt column in 00300b1, but forgot to change annotation
2021-06-02 18:16:03 +04:00
Dan Ungureanu 447d8dfc44
UX: Use icons as bulleted list in invite modal (#13229) 2021-06-02 16:28:54 +03:00
Dan Ungureanu d2135b23c4
FIX: Do not require trust level to invite to group (#13230)
It used to require SiteSetting.min_trust_level_to_allow_invite to
invite a user to a group, even if the user existed and the inviter was
a group owner.
2021-06-02 16:28:21 +03:00
Penar Musaraj 9d6780f03d
DEV: Remove emoji_one files (#13236)
And add a symlink so requests to those assets fallback to the default
emoji set (Twitter).
2021-06-02 09:18:25 -04:00
Andrei Prigorshnev 932a2fe419
FIX: PG::StringDataRightTruncation when linking posts (#13134)
Users who use encoded slugs on their sites sometimes run into 500 error when pasting a link to another topic in a post. The problem happens when generating a backward "reflection" link that would appear in a linked topic. Link URL restricted on the database level to 500 chars in length. At first glance, it should work since we have a restriction on topic title length.

But it doesn't work when a site uses encoded slugs, like here (take a look at the URL). The link to a topic, in this case, can be much longer than 500 characters.

By the way, an error happens only when generating a "reflection" link and doesn't happen with a direct link, we truncate that link. It works because, in this case, the original long link is still present in the post body and can be used for navigation. But we can't do the same for backward "reflection" links (without rewriting their implementation), the whole link must be saved to the database.

The simplest and cleanest solution will be just to remove the restriction on the database level. Abuse is impossible here since we are already protected by the restriction on topic title length. There aren’t performance benefits in using length-constrained columns in Postgres, in fact, length-constrained columns need a few extra CPU cycles to check the length when storing data.
2021-06-02 15:27:04 +04:00
Andrei Prigorshnev b7b8f5e6f3
FIX: Moderator actions and small actions shouldn't prevent fully merged topics from closing (#13200)
When a topic is fully merged into another topic we close it and schedule its deleting. But, because of a bug, if the merged topic contains some moderator actions or small actions it won't be merged. This change fixes this problem.

An important note: in general, we don't want to close a topic after moving posts if it still contains some regular posts or whispers. But when we are moving posts to a private message we don't want the notice about it to be publicly visible. So we use whispers with action_code == 'split_topic' instead of small_actions in such cases and we should ignore this specific kind of whispers when decide if we should close the merged topic.
2021-06-02 13:42:03 +04:00
Bianca Nenciu d9484db718
FIX: Split link watched words from replace (#13196)
It was not clear that replace watched words can be used to replace text
with URLs. This introduces a new watched word type that makes it easier
to understand.
2021-06-02 15:36:49 +10:00
Alan Guo Xiang Tan eea9fead63 PERF: Remove N+1 query in `SiteSerializer#user_color_schemes`. 2021-06-02 13:33:31 +08:00
Alan Guo Xiang Tan 2e4182b4b3 PERF: Warm up caches in `SiteSerializer` while preloading app.
Otherwise, time will be spent warming up the caches in the initial
requests.
2021-06-02 13:33:31 +08:00
Alan Guo Xiang Tan 6a79864f14 PERF: Cache categories query in `Stylesheet::Manager.color_scheme_digest`.
The query is being executed each time we try and generate the link path
for a stylesheet within the duration of a reqeust. Categories are not
updated that often so repeating this query multiple times a request is
wasteful.

At the time of this commit, there is a `publish_discourse_stylesheet`
ActiveRecord callback on the `Category` model which clears the cache of
`Stylesheet::Manager` each time a category is saved.
2021-06-02 13:33:31 +08:00
Alan Guo Xiang Tan c809f722f7
DEV: Remove unused comments from tests. (#13241)
Fabrication does not guarantee the ordering of records in anyway.
2021-06-02 15:33:15 +10:00
Alan Guo Xiang Tan 4390b50b12 DEV: Fix incorrect optimization in SvgSprite take 2.
Follow-up to 0700e9382d
2021-06-02 11:17:11 +08:00
Alan Guo Xiang Tan ca9e0ee40b DEV: Fix incorrect optimization in SvgSprite.
`SvgSprite.custom_sprite_paths` does not just fetch custom SVG sprite
from themes. It also picks up any custom svg icons from a pre-defined
plugin folder.

Follow-up to 0700e9382d
2021-06-02 10:42:23 +08:00
Alan Guo Xiang Tan 0700e9382d PERF: Memoize core svgs in memory to avoid expensive XML parsing.
The XML parsing of SVGs is done whenever the cache expires or on the
first load after a reboot. In one of our production instance, parsing
ranges from 30ms to 70ms which is not ideal. Instead, we've decided to
make a small memory trade off here by memoizing the core SVGs once on
boot to avoid parsing of the SVG files during the duration of a request.
The memozied hash will take up 57440 bytes or 0.05744 megabytes in size.
2021-06-02 09:16:37 +08:00
dependabot[bot] ab8949d13a Build(deps): Bump rubocop from 1.15.0 to 1.16.0
Bumps [rubocop](https://github.com/rubocop/rubocop) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/rubocop/rubocop/releases)
- [Changelog](https://github.com/rubocop/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop/rubocop/compare/v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: rubocop
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-02 09:08:22 +08:00
Martin Brennan e15c86e8c5
DEV: Topic tracking state improvements (#13218)
I merged this PR in yesterday, finally thinking this was done https://github.com/discourse/discourse/pull/12958 but then a wild performance regression occurred. These are the problem methods:

1aa20bd681/app/serializers/topic_tracking_state_serializer.rb (L13-L21)

Turns out date comparison is super expensive on the backend _as well as_ the frontend.

The fix was to just move the `treat_as_new_topic_start_date` into the SQL query rather than using the slower `UserOption#treat_as_new_topic_start_date` method in ruby. After this change, 1% of the total time is spent with the `created_in_new_period` comparison instead of ~20%.

----

History:

Original PR which had to be reverted **https://github.com/discourse/discourse/pull/12555**. See the description there for what this PR is achieving, plus below.

The issue with the original PR is addressed in 92ef54f402

If you went to the `x unread` link for a tag Chrome would freeze up and possibly crash, or eventually unfreeze after nearly 10 mins. Other routes for unread/new were similarly slow. From profiling the issue was the `sync` function of `topic-tracking-state.js`, which calls down to `isNew` which in turn calls `moment`, a change I had made in the PR above. The time it takes locally with ~1400 topics in the tracking state is 2.3 seconds.

To solve this issue, I have moved these calculations for "created in new period" and "unread not too old" into the tracking state serializer.

When I was looking at the profiler I also noticed this issue which was just compounding the problem. Every time we modify topic tracking state we recalculate the sidebar tracking/everything/tag counts. However this calls `forEachTracked` and `countTags` which can be quite expensive as they go through the whole tracking state (and were also calling the removed moment functions).

I added some logs and this was being called 30 times when navigating to a new /unread route because  `sync` is being called from `build-topic-route` (one for each topic loaded due to pagination). So I just added a debounce here and it makes things even faster.

Finally, I changed topic tracking state to use a Map so our counts of the state keys is faster (Maps have .size whereas objects you have to do Object.keys(obj) which is O(n).)

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2021-06-02 09:06:29 +10:00
Kris 9e5b8544a1
UX: Fix "last visit" line overlap in Chrome (#13232) 2021-06-01 17:33:29 -04:00
jbrw 461a2c334b
FIX: return an empty result if response from Amazon is missing expected attributes (#13173)
* FIX: return an empty result if response from Amazon is missing attributes

Check we have the basic attributes requires to construct a Onebox for Amazon.

This is an attempt to handle scenarios where we receive a valid 200-status response from an Amazon request that does not include the data we’re expecting.

* Update lib/onebox/engine/amazon_onebox.rb

Co-authored-by: Régis Hanol <regis@hanol.fr>

Co-authored-by: Régis Hanol <regis@hanol.fr>
2021-06-01 16:23:18 -04:00
Gerhard Schlager 41ee5b7c86
FIX: Don't store translated trust level names in anonymous cache (#13224)
Refactors `TrustLevel` and moves translations from server to client

Additional changes:
  * "staff" and "admin" wasn't translatable in site settings
  * it replaces a concatenated string with a translation
  * uses translation for trust levels in users_by_trust_level report
  * adds a DB migration to rename keys of translation overrides affected by this commit
2021-06-01 22:11:48 +02:00
Robin Ward 409c8585e4
DEV: Remove `ember_jquery` in most situations (#13237)
In Ember CLI, the vendor bundler includes Ember/jQuery, so this brings
our app closer to that configuration.

We have a couple pages (Reset Password / Confirm New Email) where we need
`ember_jquery` without vendor so the file still exists for those cases.
2021-06-01 15:32:51 -04:00
Jarek Radosz 2a5be90447
DEV: Fix a flaky spec (#13234)
The error being:

```
1) Post#publish_changes_to_client! send message to all users participating in private conversation
     Failure/Error: MessageBus.publish(channel, message, opts)

     Mocha::ExpectationError:
       unexpected invocation: MessageBus.publish("/topic/93", {:id => 109, :post_number => 1, :updated_at => 2021-06-01 14:53:54.508794217 +0000 (1622559234.508794 secs), :user_id => 175, :last_editor_id => 175, :type => :created, :version => 1}, {:user_ids => [174, 172, 173]})
       unsatisfied expectations:
       - expected exactly once, invoked never: MessageBus.publish("/topic/93", {:id => 109, :post_number => 1, :updated_at => 2021-06-01 14:53:54.508794217 +0000 (1622559234.508794 secs), :user_id => 175, :last_editor_id => 175, :type => :created, :version => 1}, {:user_ids => [172, 174, 173]})
       satisfied expectations:
       - allowed any number of times, invoked never: Oneboxer.onebox(any_parameters)
       - allowed any number of times, invoked never: DateTime.now(any_parameters)
       - allowed any number of times, invoked 6 times: Time.now(any_parameters)
       - allowed any number of times, invoked never: Date.today(any_parameters)
       - allowed any number of times, invoked never: TrackTimeStub.stubbed(any_parameters)
     # ./app/models/post.rb:229:in `publish_message!'
     # ./app/models/post.rb:208:in `publish_change_to_clients!'
     # ./spec/models/post_spec.rb:1740:in `block (3 levels) in <main>'
     # ./spec/rails_helper.rb:279:in `block (2 levels) in <top (required)>'
     # ./bundle/ruby/2.7.0/gems/webmock-3.13.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
```
2021-06-01 20:30:47 +02:00
Penar Musaraj 4d4c3fe1e4
FIX: Delete internal links when moderator deletes a post (#13233) 2021-06-01 14:02:53 -04:00
Andrei Prigorshnev fd64268b0d
FIX: translation issue on the bookmark modal (#13231) 2021-06-01 19:13:58 +04:00
Jarek Radosz eb6a6446ee
FIX: Update non-Ember CLI ember-buffered-proxy (#13227)
This version matches the one we have in discourse/package.json.

Fixes an issue with saving topic name form w/o any changes.
2021-06-01 16:49:03 +02:00
Discourse Translator Bot 8a99ee88bd
Update translations (#13228) 2021-06-01 15:23:14 +02:00
Gerhard Schlager 3df928d609
DEV: Fix flaky specs (#13226)
Some specs failed when `LOAD_PLUGINS=1` was set while migrating the test DB and the narrative-bot plugin disabled the `send_welcome_message` site setting.
2021-06-01 14:38:55 +02:00
Gerhard Schlager fb4dc2f35c
DEV: Fix spec (#13225)
Follow-up to 624edac3bb
2021-06-01 11:51:32 +02:00
Meghna 10c4fe5025
UX: fix PM header title alignment (#13223) 2021-06-01 14:36:56 +05:30
Joffrey JAFFEUX 624edac3bb
DEV: calls user_added_to_group on group/TL enforcement (#13222) 2021-06-01 10:34:41 +02:00
Joffrey JAFFEUX 1cd0424ccd
FEATURE: lets users favorite 2 badges to show on user-card (#13151) 2021-06-01 10:33:40 +02:00