Commit Graph

8033 Commits

Author SHA1 Message Date
Guo Xiang Tan e64e6351bd
DEV: Assert for response status in tests. 2020-08-11 13:53:37 +08:00
Martin Brennan b950b3fb3f
DEV: Add verified to uploads and fill in S3 inventory (#10406)
When we run the S3 inventory, mark uploads that exist as verified true, those that don't as verified false, and uploads not included in the check / not yet checked as verified nil.
2020-08-11 14:43:51 +10:00
Blake Erickson 2032c11f78
FIX: Return 422 when creating topics with tags w/out permission (#10400)
The UI prevents users from trying to create tags on topics when they
don't have permission, but if you are trying to add tags to a topic via
the API and you don't have permission before this change it would
silently succeed in creating the topic, but it wouldn't have any tags.

Now a 422 error will be returned with an error message when trying to
create a topic with tags when tagging is disabled or you don't have
enough trust level to add tags to a topic.

Bug report: https://meta.discourse.org/t/-/70525/14
2020-08-10 16:14:15 -06:00
jbrw d67f7a7984
Allows mod posts to be created for category group moderators on closed/archived topics (#10399) 2020-08-10 15:21:01 -04:00
David Taylor 4476680d4b
UX: Set silence_reason using the system locale
Previously we would use the user's locale, which can be confusing for moderators.
2020-08-10 18:51:54 +01:00
David Taylor fe7a7ecf6c
FIX: Include secure media URLs when linking post uploads (#10404)
Normally, secure media urls are linked like `/secure-media-uploads/...`. In this case, uploads were already being linked correctly.

But sometimes (e.g. when pulling hotlinked onebox images) secure media is referenced with a full domain name (`//example.com/secure-media-uploads`). This commit ensures that those uploads are also linked correctly.
2020-08-10 17:59:29 +01:00
Mark VanLandingham b7a092bd28
FEATURE: Group category permissions tab (#10388) 2020-08-10 09:49:05 -05:00
Penar Musaraj b44748e503
DEV: Order links by domain in spec
Should fix spec failing in macOS pgsql 11.3
2020-08-10 10:25:13 -04:00
Blake Erickson 33982c0c9e DEV: Document notifications and tags api endpoints
Added some more specs that will be used to auto generate the api docs.
2020-08-07 14:30:54 -06:00
Rafael dos Santos Silva 32f6c5c07e
DEV: Use explicit ordering so spec is realiable 2020-08-07 15:15:39 -03:00
Penar Musaraj f179510a68
FIX: include both name and id in color scheme stylesheet filename slugs (#10397) 2020-08-07 13:43:45 -04:00
Régis Hanol bc63232d2e
FIX: sync reviewable count when opening the hamburger menu (#10368)
When a tab is open but left unattended for a while, the red, green, and blue
pills tend to go out of sync.

So whevener we open the notifications menu, we sync up the notification count
(eg. blue and green pills) with the server.

However, the reviewable count (eg. the red pill) is not a notification and
is located in the hamburger menu. This commit adds a new route on the server
side to retrieve the reviewable count for the current user and a ping
(refreshReviewableCount) from the client side to sync the reviewable count
whenever they open the hamburger menu.

REFACTOR: I also refactored the hamburger-menu widget code to prevent repetitive uses
of "this.".

PERF: I improved the performance of the 'notify_reviewable' job by doing only 1 query
to the database to retrieve all the pending reviewables and then tallying based on the
various rights.
2020-08-07 18:13:02 +02:00
jbrw 3593e582a3
FIX - limit number of embedded media items in a post (#10391)
* FIX - limit number of embedded media items in a post

* Add renamed settings to DeprecatedSettings
2020-08-07 12:08:59 -04:00
Penar Musaraj 86cb5803ba
Convert HEIC uploads to JPG by default (#10395) 2020-08-07 11:17:50 -04:00
Penar Musaraj 9c9aa21726
Add site setting to pick dark mode color scheme (#10390)
Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-08-07 08:52:47 -04:00
Guo Xiang Tan 053cbe3112
PERF: Limit characters used to generate headline for search blurb.
We determined using the following benchmark script that limiting to 2500 chars would mean a maximum of
25ms spent generating headlines.

```
require 'benchmark/ips'

string = <<~STRING
Far far away, behind the word mountains...
STRING

def sql_excerpt(string, l = 1000000)
  DB.query_single(<<~SQL)
  SELECT TS_HEADLINE('english', left('#{string}', #{l}), PLAINTO_TSQUERY('mountains'))
  SQL
end

def ruby_excerpt(string)
  output = DB.query_single("SELECT '#{string}'")[0]
  Search::GroupedSearchResults::TextHelper.excerpt(output, 'mountains', radius: 100)
end

puts "Ruby Excerpt: #{ruby_excerpt(string)}"
puts "SQL Excerpt: #{sql_excerpt(string)}"
puts

Benchmark.ips do |x|
  x.time = 10

  [1000, 2500, 5000, 10000, 20000, 50000].each do |l|
    short_string = string[0..l]

    x.report("ts_headline excerpt #{l}") do
      sql_excerpt(short_string, l)
    end

    x.report("actionview excerpt #{l}") do
      ruby_excerpt(short_string)
    end
  end

  x.compare!
end
```

```
actionview excerpt 1000:    20570.7 i/s
actionview excerpt 2500:    17863.1 i/s - 1.15x  (± 0.00) slower
actionview excerpt 5000:    14228.9 i/s - 1.45x  (± 0.00) slower
actionview excerpt 10000:    10906.2 i/s - 1.89x  (± 0.00) slower
actionview excerpt 20000:     6255.0 i/s - 3.29x  (± 0.00) slower
ts_headline excerpt 1000:     4337.5 i/s - 4.74x  (± 0.00) slower
actionview excerpt 50000:     3222.7 i/s - 6.38x  (± 0.00) slower
ts_headline excerpt 2500:     2240.4 i/s - 9.18x  (± 0.00) slower
ts_headline excerpt 5000:     1258.7 i/s - 16.34x  (± 0.00) slower
ts_headline excerpt 10000:      667.2 i/s - 30.83x  (± 0.00) slower
ts_headline excerpt 20000:      348.7 i/s - 58.98x  (± 0.00) slower
ts_headline excerpt 50000:      131.9 i/s - 155.91x  (± 0.00) slower
```
2020-08-07 14:36:52 +08:00
Guo Xiang Tan e60c74d3c1
FEATURE: Use PG `ts_headline` for highlighting topic title in search. 2020-08-07 12:43:09 +08:00
Krzysztof Kotlarek 12a00d6dc5
FEATURE: add advanced order to search (#10385)
Similar to `advanced_filter` I introduced `advanced_order`.

I needed a new option because default orders are evaluated after advanced_filter so I couldn't use it.

Also, that part is a little bit more generic
```
elsif word =~ /order:\w+/
  @order = word.gsub('order:', '').to_sym
nil
```

After those changes, I can use them in plugins in this way:
```
Search.advanced_order(:votes) do |posts|
  posts.reorder("COALESCE((SELECT dvvc.counter FROM discourse_voting_vote_counters dvvc WHERE dvvc.topic_id = subquery.topic_id), 0) DESC")
end
```
2020-08-07 12:47:00 +10:00
Neil Lalonde 1ca81fbb95
FEATURE: set notification levels when added to a group (#10378)
* FEATURE: set notification levels when added to a group

This feature allows admins and group owners to define default
category and tag tracking levels that will be applied to user
preferences automatically at the time when users are added to the
group. Users are free to change those preferences afterwards.
When removed from a group, the user's notification preferences aren't
changed.
2020-08-06 12:27:27 -04:00
Penar Musaraj 87e2c9de24
DEV: Plugins can extend color definitions (#10383) 2020-08-06 09:46:17 -04:00
Penar Musaraj 6fdc711b4a
FEATURE: Allow users to opt out of automatic dark mode (#10377) 2020-08-06 09:45:37 -04:00
David Taylor df39e372d7
DEV: Add spec for removing and re-adding hotlinked images
Before the recent refactor, this would fail. https://meta.discourse.org/t/154184
2020-08-06 10:01:53 +01:00
Guo Xiang Tan 2193d02433
PERF: Use PG headlines for blurb generation and highlighting for search. 2020-08-06 14:56:29 +08:00
Guo Xiang Tan 255b0e9f14
PERF: Replace video and audio links in search blurb while indexing.
In the near future, we will be swtiching to PG headlines to generate the
search blurb. As such, we need to replace audio and video links in the
raw data used for headline generation. This also means that we avoid
replacing links each time we need to generate the blurb.
2020-08-06 12:25:03 +08:00
David Taylor ceb858c70a
PERF: Release post_upload records when downloaded image is removed (#10379)
Previously we would unconditionally keep all images downloaded via pull_hotlinked_images, even if they are later removed from the post. This commit removes that logic, and relies on the existing link_post_uploads process to pick up the downloaded images in `cooked`. Specs are added to ensure this is working correctly for regular hotlinked images, and for oneboxes.
2020-08-06 10:06:34 +10:00
jbrw 67e8bc5342
FEATURE - allow category group moderators to split/merge topics (#10351) 2020-08-05 10:33:25 -04:00
David Taylor cb12a721c4
REFACTOR: Refactor pull_hotlinked_images job
This commit should cause no functional change
- Split into functions to avoid deep nesting
- Register custom field type, and remove manual json parse/serialize
- Recover from deleted upload records

Also adds a test to ensure pull_hotlinked_images redownloads secure images only once
2020-08-05 12:14:59 +01:00
Kane York 13feb300a8
FIX: Topic map was incorrectly counting assign actions (#10360)
The assign plugin is one of two situations where a post can be both a whisper and a small-action. Check the action_code field to filter out small-actions.
2020-08-05 11:51:28 +10:00
Robin Ward 6da7a97eee FIX: Exclude shared drafts from digests 2020-08-04 13:35:48 -04:00
Penar Musaraj 266c0c50d7
FIX: Load base color scheme when default theme is not set
Followup to c937afc
2020-08-04 12:15:07 -04:00
Mark VanLandingham b76731d722
FEATURE: Invite emails to groups from add member modal (#10308) 2020-08-04 10:02:01 -05:00
Joffrey JAFFEUX 8a0478b97d
DEV: adds plugin api to add custom recipients of a post revision (#10367)
* DEV: adds plugin api to add custom recipients of a post revision

Usage:

```
add_post_revision_notifier_recipients do |post_revision|
  [78]
end
```
2020-08-04 11:57:33 +02:00
Martin Brennan 1f7b44de67
DEV: Remove flaky secure optimized image spec 2020-08-04 16:29:41 +10:00
Martin Brennan 5a3494b1e1
FIX: IMAP archive fix and group list mailbox code unification (#10355)
* Fixed an issue I introduced in the last PR where I am just archiving everything regardless of whether it is actually archived in Discourse man_facepalming
* Refactor group list_mailboxes IMAP code to use providers, add specs, and add provider code to get the correct prodivder
2020-08-04 14:19:57 +10:00
Penar Musaraj c937afc75e
FEATURE: automatic dark mode (#10341)
A first step to adding automatic dark mode color scheme switching. Adds a new SCSS file at `color_definitions.scss` that serves to output all SCSS color variables as CSS custom properties. And replaces all SCSS color variables with the new CSS custom properties throughout the stylesheets. 

This is an alpha feature at this point, can only be enabled via console using the `default_dark_mode_color_scheme_id` site setting.
2020-08-03 22:57:10 -04:00
Bianca Nenciu 2682da81ad
FIX: Get correct selectable avatar from URL (#10339)
The URL for selectable avatars was 'cooked' which means that the find_by
method was not enough.
2020-08-03 17:15:41 +03:00
Gerhard Schlager 957e851ffe Revert "FIX: Regularly reset unknown extension of uploads"
This reverts commit cc7b24b88b as it shouldn't be needed anymore for new uploads.
2020-08-03 13:37:32 +02:00
Blake Erickson 4dae9d458b
FIX: Respect query params for latest.rss (#10350)
Apparently latest.json and latest.rss are not routed to the same
controller methods. This change allows for any passed in query
parameters to actually be applied to the rss route.

This came in as a request on meta:

https://meta.discourse.org/t/-/155812/6
2020-08-03 17:03:58 +10:00
Krzysztof Kotlarek 515699776b
FIX: set mailing_list_mode to false when unsubscribing from all (#10354)
Currently, we only reset `email_digests`, `email_level` and `email_messages_level` when the user wants to unsubscribe from all email.

`mailing_list_mode` should be reset as well
2020-08-03 16:59:54 +10:00
Sam Saffron 3f054d3350
DEV: merge 2 specs
No need to run 2 specs here when 1 will do
2020-08-03 16:53:37 +10:00
Guo Xiang Tan 105d560177
SECURITY: 413 for GET, HEAD or DELETE requests with payload. 2020-08-03 14:21:33 +08:00
Guo Xiang Tan 32af607b70
DEV: Refactor anonymouse cache spec.
Mainly to properly categorize `Middleware::AnonymousCache` vs `Middleware::AnonymousCache::Helper` specs.
2020-08-03 14:17:11 +08:00
Martin Brennan 2920988b3a
FIX: IMAP sync email update uniqueness across groups and minor improvements (#10332)
Adds a imap_group_id column to IncomingEmail to deal with an issue where we were trying to update emails in the mailbox, calling IncomingEmail.where(imap_sync: true). However UID and UIDVALIDITY could be the same across accounts. So if group A used IMAP details for Gmail account A, and group B used IMAP details for Gmail account B, and both tried to sync changes to an email with UID of 3 (e.g. changing Labels), one account could affect the other. This even applied to Archiving!

Also in this PR:

* Fix error occurring if we do a uid_fetch and no emails are returned
* Allow for creating labels within the target mailbox (previously we would not do this, only use existing labels)
* Improve consistency for log messages
* Add specs for generic IMAP provider (Gmail specs still to come)
* Add custom archiving support for Gmail
* Only use Message-ID for uniqueness of IncomingEmail if it was generated by us
* Various refactors and improvements
2020-08-03 13:10:17 +10:00
Blake Erickson e7c0c53944 DEV: Document private message api endpoints
Added some more specs that will be used to auto generate the api docs.
2020-07-31 18:27:08 -06:00
Blake Erickson 8de635fe92
DEV: Show message when cannot invite user to PM (#10336)
* DEV: Show message when cannot invite user to PM

When inviting a user to a PM return a message that says, "Sorry, this
user can't be invited." if they have been muted or are not in a users
allowed pm users list.

* Minor refactor & improved some text
2020-07-31 09:52:19 -06:00
Vinoth Kannan 691edc16c9 FIX: should allow non-ASCII slugs for category pages. 2020-07-29 19:47:57 +05:30
David Taylor 16c65a94f7
PERF: Preload S3 inventory data for multisite clusters 2020-07-29 10:31:55 +01:00
Guo Xiang Tan 162958380a
DEV: Remove stray code that has been commented out. 2020-07-29 09:58:29 +08:00
Martin Brennan 9e5b213089
FIX: Ensure topic user bookmarked synced on bookmark auto-delete (#10323)
For the following conditions, the TopicUser.bookmarked column was not updated correctly:

* When a bookmark was auto-deleted because the reminder was sent
* When a bookmark was auto-deleted because the owner of the bookmark replied to the topic

This adds another migration to fix the out-of-sync column and also some refactors to BookmarkManager to allow for more of these delete cases. BookmarkManager is used instead of directly destroying the bookmark in PostCreator and BookmarkReminderNotificationHandler.
2020-07-29 09:43:32 +10:00
jbrw 8f140b8903
TopicView/PostSerializer should be able to handle topics without categories 2020-07-28 19:06:55 -04:00