Commit Graph

8042 Commits

Author SHA1 Message Date
Blake Erickson c68563a281 DEV: Improve API usage when creating * updating categories
The category model already has a default value for `color` and
`text_color` so they don't need to be required via the API. The ember UI
already requires that colors be selected.

The name of the category also doesn't need to be required when updating
the category either because we are already passing in the id for the
category we want to change.

These changes improve the api experience because you no longer have to
lookup the category name, color, or text color before updating a single
category attribute. When creating a category the name is still required.

https://meta.discourse.org/t/-/132424/2
2020-08-12 12:28:29 -06:00
Penar Musaraj 6dd9f2eca2
FIX: color scheme selection with non-default theme
This fixes an issue where a non-default theme set to use the base color
scheme (i.e. the theme had an empty `color_scheme_id`) was loading the
default theme's color scheme instead.
2020-08-12 08:49:13 -04:00
Guo Xiang Tan 603a4d1794
DEV: Fix randomly failing spec. 2020-08-12 16:34:56 +08:00
Guo Xiang Tan 6792d7f50c
Fix the build.
Follow-up to 93f8396b4b.
2020-08-12 15:55:22 +08:00
Guo Xiang Tan 93f8396b4b
FIX: Limit PG headline based search blurb generation to 200 characters.
* Recovers omission characters '...' in blurb as well.
2020-08-12 15:34:27 +08:00
Krzysztof Kotlarek 1f7f8ee72c
FIX: try to make topic_tracking_state_spec stable (#10419)
Sometime parallel spec if failing with error:
```
NoMethodError:
       undefined method `data' for nil:NilClass
     # ./spec/models/topic_tracking_state_spec.rb:339:in `block (4 levels) in <main>'
```

I have a theory that it might be related to instance variables in before block
2020-08-12 15:56:06 +10:00
Martin Brennan 95b71b35d6
FEATURE: IMAP delete email sync for group inboxes (#10392)
Adds functionality to reflect topic delete in Discourse to IMAP inbox (Gmail only for now) and reflecting Gmail deletes in Discourse.

Adding lots of tests, various refactors and code improvements.

When Discourse topic is destroyed in PostDestroyer mark the topic incoming email as imap_sync: true, and do the opposite when post is recovered.
2020-08-12 10:16:26 +10:00
Penar Musaraj c05aced094
FIX: Invalidate cache when updating color scheme colors (#10417) 2020-08-11 16:28:59 -04:00
Blake Erickson ee366f7ac7 FIX: Ensure load more directory items has a .json url
The `/directory_items` route needs to have a .json url, but the rails
url helper `_path` doesn't return the format of the route.

I tried passing in a format options to `directory_items_path`. Which
works in the rails console

```
[8] pry(main)> directory_items_path(params.merge(:format => :json))
=> "/directory_items.json?page=1"
```

but when I added that some logic to the controller it comes out as

```
/directory_items?format=json&page=1
```

(which is actually how I expect it to work based on how you pass in the
format param). Anyways, because I couldn't figure out how to pass a
format to the `_path` helper I just used URI.parse to append `.json`
manually.
2020-08-11 12:43:55 -06:00
Guo Xiang Tan e64e6351bd
DEV: Assert for response status in tests. 2020-08-11 13:53:37 +08:00
Martin Brennan b950b3fb3f
DEV: Add verified to uploads and fill in S3 inventory (#10406)
When we run the S3 inventory, mark uploads that exist as verified true, those that don't as verified false, and uploads not included in the check / not yet checked as verified nil.
2020-08-11 14:43:51 +10:00
Blake Erickson 2032c11f78
FIX: Return 422 when creating topics with tags w/out permission (#10400)
The UI prevents users from trying to create tags on topics when they
don't have permission, but if you are trying to add tags to a topic via
the API and you don't have permission before this change it would
silently succeed in creating the topic, but it wouldn't have any tags.

Now a 422 error will be returned with an error message when trying to
create a topic with tags when tagging is disabled or you don't have
enough trust level to add tags to a topic.

Bug report: https://meta.discourse.org/t/-/70525/14
2020-08-10 16:14:15 -06:00
jbrw d67f7a7984
Allows mod posts to be created for category group moderators on closed/archived topics (#10399) 2020-08-10 15:21:01 -04:00
David Taylor 4476680d4b
UX: Set silence_reason using the system locale
Previously we would use the user's locale, which can be confusing for moderators.
2020-08-10 18:51:54 +01:00
David Taylor fe7a7ecf6c
FIX: Include secure media URLs when linking post uploads (#10404)
Normally, secure media urls are linked like `/secure-media-uploads/...`. In this case, uploads were already being linked correctly.

But sometimes (e.g. when pulling hotlinked onebox images) secure media is referenced with a full domain name (`//example.com/secure-media-uploads`). This commit ensures that those uploads are also linked correctly.
2020-08-10 17:59:29 +01:00
Mark VanLandingham b7a092bd28
FEATURE: Group category permissions tab (#10388) 2020-08-10 09:49:05 -05:00
Penar Musaraj b44748e503
DEV: Order links by domain in spec
Should fix spec failing in macOS pgsql 11.3
2020-08-10 10:25:13 -04:00
Blake Erickson 33982c0c9e DEV: Document notifications and tags api endpoints
Added some more specs that will be used to auto generate the api docs.
2020-08-07 14:30:54 -06:00
Rafael dos Santos Silva 32f6c5c07e
DEV: Use explicit ordering so spec is realiable 2020-08-07 15:15:39 -03:00
Penar Musaraj f179510a68
FIX: include both name and id in color scheme stylesheet filename slugs (#10397) 2020-08-07 13:43:45 -04:00
Régis Hanol bc63232d2e
FIX: sync reviewable count when opening the hamburger menu (#10368)
When a tab is open but left unattended for a while, the red, green, and blue
pills tend to go out of sync.

So whevener we open the notifications menu, we sync up the notification count
(eg. blue and green pills) with the server.

However, the reviewable count (eg. the red pill) is not a notification and
is located in the hamburger menu. This commit adds a new route on the server
side to retrieve the reviewable count for the current user and a ping
(refreshReviewableCount) from the client side to sync the reviewable count
whenever they open the hamburger menu.

REFACTOR: I also refactored the hamburger-menu widget code to prevent repetitive uses
of "this.".

PERF: I improved the performance of the 'notify_reviewable' job by doing only 1 query
to the database to retrieve all the pending reviewables and then tallying based on the
various rights.
2020-08-07 18:13:02 +02:00
jbrw 3593e582a3
FIX - limit number of embedded media items in a post (#10391)
* FIX - limit number of embedded media items in a post

* Add renamed settings to DeprecatedSettings
2020-08-07 12:08:59 -04:00
Penar Musaraj 86cb5803ba
Convert HEIC uploads to JPG by default (#10395) 2020-08-07 11:17:50 -04:00
Penar Musaraj 9c9aa21726
Add site setting to pick dark mode color scheme (#10390)
Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-08-07 08:52:47 -04:00
Guo Xiang Tan 053cbe3112
PERF: Limit characters used to generate headline for search blurb.
We determined using the following benchmark script that limiting to 2500 chars would mean a maximum of
25ms spent generating headlines.

```
require 'benchmark/ips'

string = <<~STRING
Far far away, behind the word mountains...
STRING

def sql_excerpt(string, l = 1000000)
  DB.query_single(<<~SQL)
  SELECT TS_HEADLINE('english', left('#{string}', #{l}), PLAINTO_TSQUERY('mountains'))
  SQL
end

def ruby_excerpt(string)
  output = DB.query_single("SELECT '#{string}'")[0]
  Search::GroupedSearchResults::TextHelper.excerpt(output, 'mountains', radius: 100)
end

puts "Ruby Excerpt: #{ruby_excerpt(string)}"
puts "SQL Excerpt: #{sql_excerpt(string)}"
puts

Benchmark.ips do |x|
  x.time = 10

  [1000, 2500, 5000, 10000, 20000, 50000].each do |l|
    short_string = string[0..l]

    x.report("ts_headline excerpt #{l}") do
      sql_excerpt(short_string, l)
    end

    x.report("actionview excerpt #{l}") do
      ruby_excerpt(short_string)
    end
  end

  x.compare!
end
```

```
actionview excerpt 1000:    20570.7 i/s
actionview excerpt 2500:    17863.1 i/s - 1.15x  (± 0.00) slower
actionview excerpt 5000:    14228.9 i/s - 1.45x  (± 0.00) slower
actionview excerpt 10000:    10906.2 i/s - 1.89x  (± 0.00) slower
actionview excerpt 20000:     6255.0 i/s - 3.29x  (± 0.00) slower
ts_headline excerpt 1000:     4337.5 i/s - 4.74x  (± 0.00) slower
actionview excerpt 50000:     3222.7 i/s - 6.38x  (± 0.00) slower
ts_headline excerpt 2500:     2240.4 i/s - 9.18x  (± 0.00) slower
ts_headline excerpt 5000:     1258.7 i/s - 16.34x  (± 0.00) slower
ts_headline excerpt 10000:      667.2 i/s - 30.83x  (± 0.00) slower
ts_headline excerpt 20000:      348.7 i/s - 58.98x  (± 0.00) slower
ts_headline excerpt 50000:      131.9 i/s - 155.91x  (± 0.00) slower
```
2020-08-07 14:36:52 +08:00
Guo Xiang Tan e60c74d3c1
FEATURE: Use PG `ts_headline` for highlighting topic title in search. 2020-08-07 12:43:09 +08:00
Krzysztof Kotlarek 12a00d6dc5
FEATURE: add advanced order to search (#10385)
Similar to `advanced_filter` I introduced `advanced_order`.

I needed a new option because default orders are evaluated after advanced_filter so I couldn't use it.

Also, that part is a little bit more generic
```
elsif word =~ /order:\w+/
  @order = word.gsub('order:', '').to_sym
nil
```

After those changes, I can use them in plugins in this way:
```
Search.advanced_order(:votes) do |posts|
  posts.reorder("COALESCE((SELECT dvvc.counter FROM discourse_voting_vote_counters dvvc WHERE dvvc.topic_id = subquery.topic_id), 0) DESC")
end
```
2020-08-07 12:47:00 +10:00
Neil Lalonde 1ca81fbb95
FEATURE: set notification levels when added to a group (#10378)
* FEATURE: set notification levels when added to a group

This feature allows admins and group owners to define default
category and tag tracking levels that will be applied to user
preferences automatically at the time when users are added to the
group. Users are free to change those preferences afterwards.
When removed from a group, the user's notification preferences aren't
changed.
2020-08-06 12:27:27 -04:00
Penar Musaraj 87e2c9de24
DEV: Plugins can extend color definitions (#10383) 2020-08-06 09:46:17 -04:00
Penar Musaraj 6fdc711b4a
FEATURE: Allow users to opt out of automatic dark mode (#10377) 2020-08-06 09:45:37 -04:00
David Taylor df39e372d7
DEV: Add spec for removing and re-adding hotlinked images
Before the recent refactor, this would fail. https://meta.discourse.org/t/154184
2020-08-06 10:01:53 +01:00
Guo Xiang Tan 2193d02433
PERF: Use PG headlines for blurb generation and highlighting for search. 2020-08-06 14:56:29 +08:00
Guo Xiang Tan 255b0e9f14
PERF: Replace video and audio links in search blurb while indexing.
In the near future, we will be swtiching to PG headlines to generate the
search blurb. As such, we need to replace audio and video links in the
raw data used for headline generation. This also means that we avoid
replacing links each time we need to generate the blurb.
2020-08-06 12:25:03 +08:00
David Taylor ceb858c70a
PERF: Release post_upload records when downloaded image is removed (#10379)
Previously we would unconditionally keep all images downloaded via pull_hotlinked_images, even if they are later removed from the post. This commit removes that logic, and relies on the existing link_post_uploads process to pick up the downloaded images in `cooked`. Specs are added to ensure this is working correctly for regular hotlinked images, and for oneboxes.
2020-08-06 10:06:34 +10:00
jbrw 67e8bc5342
FEATURE - allow category group moderators to split/merge topics (#10351) 2020-08-05 10:33:25 -04:00
David Taylor cb12a721c4
REFACTOR: Refactor pull_hotlinked_images job
This commit should cause no functional change
- Split into functions to avoid deep nesting
- Register custom field type, and remove manual json parse/serialize
- Recover from deleted upload records

Also adds a test to ensure pull_hotlinked_images redownloads secure images only once
2020-08-05 12:14:59 +01:00
Kane York 13feb300a8
FIX: Topic map was incorrectly counting assign actions (#10360)
The assign plugin is one of two situations where a post can be both a whisper and a small-action. Check the action_code field to filter out small-actions.
2020-08-05 11:51:28 +10:00
Robin Ward 6da7a97eee FIX: Exclude shared drafts from digests 2020-08-04 13:35:48 -04:00
Penar Musaraj 266c0c50d7
FIX: Load base color scheme when default theme is not set
Followup to c937afc
2020-08-04 12:15:07 -04:00
Mark VanLandingham b76731d722
FEATURE: Invite emails to groups from add member modal (#10308) 2020-08-04 10:02:01 -05:00
Joffrey JAFFEUX 8a0478b97d
DEV: adds plugin api to add custom recipients of a post revision (#10367)
* DEV: adds plugin api to add custom recipients of a post revision

Usage:

```
add_post_revision_notifier_recipients do |post_revision|
  [78]
end
```
2020-08-04 11:57:33 +02:00
Martin Brennan 1f7b44de67
DEV: Remove flaky secure optimized image spec 2020-08-04 16:29:41 +10:00
Martin Brennan 5a3494b1e1
FIX: IMAP archive fix and group list mailbox code unification (#10355)
* Fixed an issue I introduced in the last PR where I am just archiving everything regardless of whether it is actually archived in Discourse man_facepalming
* Refactor group list_mailboxes IMAP code to use providers, add specs, and add provider code to get the correct prodivder
2020-08-04 14:19:57 +10:00
Penar Musaraj c937afc75e
FEATURE: automatic dark mode (#10341)
A first step to adding automatic dark mode color scheme switching. Adds a new SCSS file at `color_definitions.scss` that serves to output all SCSS color variables as CSS custom properties. And replaces all SCSS color variables with the new CSS custom properties throughout the stylesheets. 

This is an alpha feature at this point, can only be enabled via console using the `default_dark_mode_color_scheme_id` site setting.
2020-08-03 22:57:10 -04:00
Bianca Nenciu 2682da81ad
FIX: Get correct selectable avatar from URL (#10339)
The URL for selectable avatars was 'cooked' which means that the find_by
method was not enough.
2020-08-03 17:15:41 +03:00
Gerhard Schlager 957e851ffe Revert "FIX: Regularly reset unknown extension of uploads"
This reverts commit cc7b24b88b as it shouldn't be needed anymore for new uploads.
2020-08-03 13:37:32 +02:00
Blake Erickson 4dae9d458b
FIX: Respect query params for latest.rss (#10350)
Apparently latest.json and latest.rss are not routed to the same
controller methods. This change allows for any passed in query
parameters to actually be applied to the rss route.

This came in as a request on meta:

https://meta.discourse.org/t/-/155812/6
2020-08-03 17:03:58 +10:00
Krzysztof Kotlarek 515699776b
FIX: set mailing_list_mode to false when unsubscribing from all (#10354)
Currently, we only reset `email_digests`, `email_level` and `email_messages_level` when the user wants to unsubscribe from all email.

`mailing_list_mode` should be reset as well
2020-08-03 16:59:54 +10:00
Sam Saffron 3f054d3350
DEV: merge 2 specs
No need to run 2 specs here when 1 will do
2020-08-03 16:53:37 +10:00
Guo Xiang Tan 105d560177
SECURITY: 413 for GET, HEAD or DELETE requests with payload. 2020-08-03 14:21:33 +08:00