Commit Graph

4649 Commits

Author SHA1 Message Date
Sam Saffron 7f841dc21f
PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
Guo Xiang Tan 34c39b65bb
PERF: Avoid executing the same query twice. 2020-05-13 11:10:09 +08:00
Martin Brennan 9981fa4466
FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
Daniel Waterworth 497dc6eaa7 Add a global setting for CDN origin
This is so that, on a multisite cluster, when we handle a CDN request,
the hostname that is requested corresponds to one of the sites -
specifically the default site.
2020-05-12 16:43:40 +01:00
Robin Ward 088c11a12c FIX: Don't raise an exception if a TopicLink cannot be created
This can happen under concurrency and it's fine. If a `TopicLink`
already exists we can continue on without raising an error.
2020-05-12 11:15:59 -04:00
Sam Saffron a29ae17d3a
FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
Guo Xiang Tan 8fb99f218d
FEATURE: Allow plugins to register a callback to ignore DraftSequence. 2020-05-12 14:25:20 +08:00
Guo Xiang Tan d301af39bd
PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
Bianca Nenciu 8149bfbaf1
FEATURE: Filter settings by plugin (#9692) 2020-05-10 14:07:45 +03:00
Robin Ward b2e4ca968b DEV: Remove `translate` API from plugin.
It appears to be completely unused at this point.
2020-05-08 14:14:01 -04:00
Guo Xiang Tan efd0e4d02b
FIX: Don't trigger user_updated event for bot users. 2020-05-08 09:28:16 +08:00
Martin Brennan 37be2958f7
DEV: Allow specifying if a notification is high_priority on create (#9660)
This allows for special cases where we may not want a certain notification type to ALWAYS be high priority (e.g. a topic timer).
2020-05-07 14:35:32 +10:00
Martin Brennan 6fb0f36ce1
FEATURE: Optionally delete bookmark when reminder sent (#9637)
We now show an options gear icon next to the bookmark name.

When expanded we show the "delete bookmark when reminder sent" option. The value of this checkbox is saved in local storage for the user.

If this is ticked, when a reminder is sent for the bookmark the bookmark itself is deleted. This is so people can use the reminder functionality by itself.

Also remove the blue alert reminder section from the "Edit Bookmark" modal as it just added clutter, because the user can already see they had a reminder set:

Adds a default false boolean column `delete_when_reminder_sent` to bookmarks.
2020-05-07 13:37:39 +10:00
Robin Ward f5bdbd347e Revert "FIX: Discobot has not been created with our custom avatar."
This reverts commit 1062dbc3e9.

Looks like it's causing some errors on migration.

```
/var/www/discourse/lib/image_sizer.rb:6:in `resize'
/var/www/discourse/lib/upload_creator.rb:120:in `block in create_for'
/var/www/discourse/lib/distributed_mutex.rb:33:in `block in synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:14:in `synchronize'
/var/www/discourse/lib/upload_creator.rb:37:in `create_for'
```
2020-05-06 11:28:42 -04:00
Martin Brennan fa572d3a7a
DEV: Remove all code referencing at_desktop bookmark reminders (#9650)
We have found no need for these reminder types, so we are removing the code for them.
2020-05-06 15:22:43 +10:00
Guo Xiang Tan 1062dbc3e9 FIX: Discobot has not been created with our custom avatar.
Previously the image was imported from a Discourse hosted CDN but the
URL has since become invalid. However, it was not caught since all
errors are rescued. This commit fixes the issue by shipping the user
avatar with the plugin.
2020-05-06 09:25:01 +08:00
Robin Ward 612284cef3
DEV: Remove `Discourse.RAW_TEMPLATES` (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor 03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek 9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
David Taylor a51b8d9c66
FIX: Do not use cached settings during theme compilation
We compile within a database transaction, so using a cached value from redis can cause unwanted side effects
2020-05-04 09:43:06 +01:00
David Taylor 4f885d7da2
FIX: Clear theme caches after database transaction has committed
This was causing unusual side effects on high-traffic sites, where the cache would be rebuilt before the transaction had been committed
2020-05-04 09:42:50 +01:00
Kane York 784bf2a173
FIX: Unassign user titles when a badge is deleted (#9573) 2020-05-02 18:02:28 -07:00
Krzysztof Kotlarek 37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Jeff Wong 2cb9e85d14
FEATURE: add category banner for why a user cannot post (#9576)
* FEATURE: add category banner for why a user cannot post

Adds a category banner for why a user is unable to post in a category.

Also adds an extra alert for the user when a user is unable to create a topic in a
category and they still try and click on the disabled-looking new topic
button.
2020-04-30 10:39:11 -07:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
Sam Saffron 310a7edee5
DEV: remove unused columns from posts and topics
avg_time on posts and topics have not been used in a year.

This uses a re-runnable ddl transaction diasabled migration to
drop the column, cause it touchs very high traffic table and may
deadlock
2020-04-30 11:22:33 +10:00
Neil Lalonde 9d781d4959
FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
Arpit Jalan 198856f78d DEV: annotate models 2020-04-28 15:59:39 +05:30
mentalstring 67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Mark VanLandingham 051e6be7b8
DEV: Trigger 'user_added_to_group' when accepting invite with groups (#9561) 2020-04-27 15:13:34 -05:00
Mark VanLandingham d8c884a856
DEV: Add has_many category_tag_stats in tag.rb (#9558) 2020-04-27 12:42:13 -05:00
David Taylor 8a112b7464
DEV: Automatically extend CSP when themes link to external scripts (#9531) 2020-04-24 09:47:01 +01:00
Sam Saffron cbceadf48b
FEATURE: when blocking emails prefer blocking canonical
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.

This commit improves the situation by always preferring to block (and check)
canonical emails.

This means that if:

`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`

This means that `samevil+2@domain.com` (ad infinitum) will be blocked
2020-04-24 14:09:51 +10:00
Sam Saffron 6a18c9aa0b
Revert "FEATURE: enforce_canonical_emails site setting"
This reverts commit 6f9177e2ed.

We decided on a completely different approach to the problem.

Instead we will let blocked emails be treated as canonical.
2020-04-24 13:52:06 +10:00
Martin Brennan cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Krzysztof Kotlarek 52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Robin Ward e4a0e0bead FIX: Ignore removed column 2020-04-22 14:27:01 -04:00
Vinoth Kannan df0c386f8a
UX: drop the `automatic_membership_retroactive` column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Daniel Waterworth c8fed90e4e FIX: Handle sub-sub-category paths without an id 2020-04-22 12:19:54 +01:00
Dan Ungureanu 4e5f9d4cd1
DEV: Drop 'key' column from user_api_keys (#9388) 2020-04-22 12:13:19 +03:00
Dan Ungureanu e733701887
FEATURE: Make report filters reusable (#9444)
This commit also adds 'include subcategories' report filter
2020-04-22 11:52:50 +03:00
Martin Brennan 628ba9d1e2
FEATURE: Promote bookmarks with reminders to core functionality (#9369)
The main thrust of this PR is to take all the conditional checks based on the `enable_bookmarks_with_reminders` away and only keep the code from the `true` path, making bookmarks with reminders the core bookmarks feature. There is also a migration to create `Bookmark` records out of `PostAction` bookmarks for a site.

### Summary

* Remove logic based on whether enable_bookmarks_with_reminders is true. This site setting is now obsolete, the old bookmark functionality is being removed. Retain the setting and set the value to `true` in a migration.
* Use the code from the rake task to create a database migration that creates bookmarks from post actions.
* Change the bookmark report to read from the new table.
* Get rid of old endpoints for bookmarks
* Link to the new bookmarks list from the user summary page
2020-04-22 13:44:19 +10:00
Jarek Radosz a781ef7662
FIX: Reject invalid Category slugs (#9473)
Previously it would sanitize given slug and then save the resulting empty slug.
2020-04-21 03:50:50 +02:00
Jarek Radosz 17cf300b71
DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
Mark VanLandingham 06796bec8b
FIX: Set category description to first posts cooked value (#9478) 2020-04-20 14:51:49 -05:00
Robin Ward 98729c8e6e FIX: Allow embed updates of just the title 2020-04-20 14:31:24 -04:00
Robin Ward 56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
Neil Lalonde 074509fd95
FIX: don't demote users to TL2 when default trust level is 3
Within 24 hours of signing up, new users were losing their
default trust level of 3. With this fix, demotions from
trust level 3 won't happen when the "default trust level"
setting is 3 or 4.
2020-04-16 12:28:16 -04:00
Martin Brennan 51672b9121
FIX: Minor bookmark with reminder issue cleanup (#9436)
* Count user summary bookmarks from new Bookmark table if bookmarks with reminders enabled
* Update topic user bookmarked column when new topic bookmark changed
* Make in:bookmarks search work with new bookmarks
* Fix batch inserts for bookmark rake task (and thus migration). We were only inserting one bookmark at a time, completely defeating the purpose of batching!
2020-04-16 11:32:21 +10:00
Vinoth Kannan 884eea7a83 FEATURE: add support for `upload` format in theme settings. 2020-04-15 18:34:02 +05:30