Commit Graph

7960 Commits

Author SHA1 Message Date
Martin Brennan 6fb0f36ce1
FEATURE: Optionally delete bookmark when reminder sent (#9637)
We now show an options gear icon next to the bookmark name.

When expanded we show the "delete bookmark when reminder sent" option. The value of this checkbox is saved in local storage for the user.

If this is ticked, when a reminder is sent for the bookmark the bookmark itself is deleted. This is so people can use the reminder functionality by itself.

Also remove the blue alert reminder section from the "Edit Bookmark" modal as it just added clutter, because the user can already see they had a reminder set:

Adds a default false boolean column `delete_when_reminder_sent` to bookmarks.
2020-05-07 13:37:39 +10:00
tshenry 0d394e6b76
FEATURE: Include category position when exporting categories (#9658) 2020-05-07 12:17:15 +10:00
Krzysztof Kotlarek 05799538e6
FIX: correct edit notification username for PMs (#9649)
When the tag is added to PM by admin, notification has an incorrect username (post author username instead of admin username)
2020-05-07 07:52:21 +10:00
Gerhard Schlager ec2f3169ff FIX: Using the `default_locale` in locale fallbacks caused problems
Locale files get precompiled after deployment and they contained translations from the `default_locale`. That's especially bad in multisites, because the initial `default_locale` is `en_US`. Sites where the `default_locale` isn't `en_US` could see missing translations. The same thing could happen when users are allowed to chose a different locale.

This change simplifies the logic by not using the `default_locale` in the locale chain. It always falls back to `en` in case of missing translations.
2020-05-06 22:59:07 +02:00
Roman Rizzi 70bf1669be
DEV: Use a copy of the fixture file instead of the original one (#9645) 2020-05-06 11:54:08 -03:00
Sam Saffron f8c18ac888
DEV: remove fragile spec
The failover spec is very fragile and tests specific implementation
vs actual behavior

We rely on a different script during the build process to test
failover operates correctly
2020-05-06 16:21:48 +10:00
Martin Brennan fa572d3a7a
DEV: Remove all code referencing at_desktop bookmark reminders (#9650)
We have found no need for these reminder types, so we are removing the code for them.
2020-05-06 15:22:43 +10:00
Robin Ward 612284cef3
DEV: Remove `Discourse.RAW_TEMPLATES` (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor 03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek 9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
David Taylor 5901717531
DEV: Allow DB.after_commit to be used outside of a transaction
In this case, it will execute the given block immediately
2020-05-04 09:42:41 +01:00
Sam Saffron 3877ef2cfa
FEATURE: Unconditionally notify on edits at least once a day
This ensures that at a minimum you are notified once a day of
repeat edits by the same user.

Long term we may consider winding this down to say 1 hour or
making it configurable.
2020-05-04 17:55:00 +10:00
Sam Saffron 3d0ccf8642
FIX: missing edit notifications in some rare cases
Due to a refactor in e90f9e5cc4 we stopped notifying on edits if
a user liked a post and then edited.

The like could have happened a long time ago so this gets extra
confusing.

This change makes the suppression more deliberate. We only want
to suppress quite/link/mention if the user already got a reply
notification.

We can expand this suppression if it is not enough.
2020-05-04 17:40:56 +10:00
Kane York 784bf2a173
FIX: Unassign user titles when a badge is deleted (#9573) 2020-05-02 18:02:28 -07:00
Martin Brennan bcc9ad6f57
FIX: Bookmark UI tweaks (#9604)
* When hovering over the bookmark icon for a post, show the name of the bookmark at the end of the tooltip _if_ it has been set.
* Order bookmarks by `updated_at DESC` in the user list and show that instead of created at.
2020-05-01 16:14:20 +10:00
Martin Brennan 867bc3b48e
FIX: Change base importer to create new Bookmark records (#9603)
Also add a spec and fixture with a mock importer that we can use to test the create_X methods of the base importer
2020-05-01 11:34:55 +10:00
Krzysztof Kotlarek 37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Vinoth Kannan 71241a50f7 DEV: improve code readability & add tests for user guardian.
a511bea4cc
2020-04-30 20:59:33 +05:30
Régis Hanol 501b19b6e0
FIX: server-side HtmlToMarkdown improvements (#9586)
TLDR; this commit vastly improves how whitespaces are handled when converting from HTML to Markdown.
It also adds support for converting HTML <tables> to markdown tables.

The previous 'remove_whitespaces!' method was traversing the whole HTML tree and used a heuristic to remove
leading and trailing whitespaces whenever it was appropriate (ie. mostly before and after HTML block elements)

It was a good idea, but it was very limited and leaded to bad conversion when the html had leading whitespaces on several lines for example.
One such example can be found [here](https://meta.discourse.org/t/86782).

For various reasons, most of the whitespaces in a HTML file is ignored when the page is being displayed in a browser.
The rules that the browsers follow are the [CSS' White Space Processing Rules](https://www.w3.org/TR/css-text-3/#white-space-rules).
They can be quite complicated when you take into account RTL languages and other various tidbits but they boils down to the following:

- Collapse whitespaces down to one space (0x20) inside an inline context (ie. nodes/tags that are being displaying on the same line)
- Remove any leading/trailing whitespaces inside an inline context

One quick & dirty way of getting this 90% solved would be to do 'HTML.gsub!(/[[:space:]]+/, " ")'.
We would also need to hoist <pre> elements in order to not mess with their whitespaces.
Unfortunately, this solution let some whitespaces creep around HTML tags which leads to more '.strip!' calls than I can bear.

I decided to "emulate" the browser's handling of whitespaces and came up with a solution in 4 parts

1. remove_not_allowed!

The HtmlToMarkdown library is recursively "visiting" all the nodes in the HTML in order to convert them to Markdown.
All the nodes that aren't handled by the library (eg. <script>, <style> or any non-textual HTML tags) are "swallowed".
In order to reduce the number of nodes visited, the method 'remove_not_allowed!' will automatically delete all the nodes
that have no "visitor" (eg. a 'visit_<tag>' method) defined.

2. remove_hidden!

Similar purpose as the previous method (eg. reducing number of nodes visited), there's no point trying to convert something that is hidden.
The 'remove_hidden!' method removes any nodes that was hidden using the "hidden" HTML attribute, some CSS or with a width or height equal to 0.

3. hoist_line_breaks!

The 'hoist_line_breaks!' method is there to handle <br> tags. I know those tiny <br> don't do much but they can be quite annoying.
The <br> tags are inline elements but they visually work like a block element (ie. they create a new line).
If you have the following HTML "<i>Foo<br>Bar</i>", it ends up visually similar to "<i>Foo</i><br><i>Bar</i>".
The latter being much more easy to process than the former, so that's what this method is doing.
The "hoist_line_breaks" will hoist <br> tags out of inline tags until their parent is a block element.

4. remove_whitespaces!

The "remove_whitespaces!" is where all the whitespace removal is happening. It's broken down into 4 methods as well

- remove_whitespaces!
- is_inline?
- collapse_spaces!
- remove_trailing_space!

The 'remove_whitespace!' method is recursively walking the HTML tree (skipping <pre> tags).
If a node has any children, they will be chunked into groups of inline elements vs block elements.
For each chunks of inline elements, it will call the "collapse_space!" and "remove_trailing_space!" methods.
For each chunks of block elements, it will call "remote_whitespace!" to keep walking the HTML tree recursively.

The "is_inline?" method determines whether a node is part of a inline context.
A node is inline iif it's a text node or it's an inline tag, but not <br>, and all its children are also inline.

The "collapse_spaces!" method will collapse any kind of (white) space into a single space (" ") character, even accros tags.
For example, if we have "  Foo \n<i> Bar </i>\t42", it will return "Foo <i>Bar </i>42".

Finally, the "remove_trailing_space!" method is there to remove any trailing space that might creep in at the end of the inline chunk.

This solution is not 100% bullet-proof.
It does not support RTL languages at all and has some caveats that I felt were not worth the work to get properly fixed.

FIX: better detection of hidden elements when converting HTML to Markdown
FIX: take into account the 'allowed_href_schemes' site setting when converting HTML <a> to Markdown
FIX: added support for 'mailto:' scheme when converting <a> from HTML to Markdown
FIX: added support for <img> dimensions when converting from HTML to Markdown
FIX: added support for <dl>, <dd> and <dt> when converting from HTML to Markdown
FIX: added support for multilines emphases, strongs and strikes when converting from HTML to Markdown
FIX: added support for <acronym> when converting from HTML to Markdown
DEV: remove unused 'sanitize' gem

Wow, did you just read all that?! Congratz, here's a cookie: 🍪.
2020-04-30 12:21:25 +02:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
David Taylor 367cbf5d2b
FEATURE: Allow user creation with admin api when local logins disabled (#9587) 2020-04-30 11:39:24 +10:00
Martin Brennan ca539fdccf
FIX: Rename all instances of bookmarkWithReminder to just bookmark (#9579)
* Rename all instances of bookmarkWithReminder and bookmark_with_reminder to just bookmark
* Delete old bookmark code at the same time
* Add migration to remove the bookmarkWithReminder post menu item if people have it set in site settings
2020-04-30 10:09:22 +10:00
Dan Ungureanu 0a85a7aef9
FEATURE: Add user_profile to user_archive CSV export (#9571) 2020-04-29 12:09:50 +03:00
Neil Lalonde 9d781d4959
FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
Joffrey JAFFEUX a6f986b50f
FEATURE: allows to to style published page with themes/plugins (#9570) 2020-04-28 18:24:24 +02:00
Penar Musaraj b19dcac272
FIX: Include default notification level in category serializer (#9572)
Fixes an issue where the notification level state goes missing when user edits a category in the UI.
2020-04-28 12:05:53 -04:00
David Taylor ba616ffb50
DEV: Use a tmp directory for storing uploads in tests (#9554)
This avoids development-mode upload files from polluting the test environment
2020-04-28 14:03:04 +01:00
Martin Brennan 5108cf8ddf
FIX: Topic user bookmarked column logic was not correct (#9563)
Make sure the topic_user.bookmarked column is set correctly when user bookmarks/unbookmarks any post in a topic. For example, you bookmarked a post in the topic that was not the OP, the bookmark icon in the topic list would not be shown. Same if deleting a bookmark for the last bookmarked post in a topic, the bookmark icon in the topic list would not be removed.

Previously this was only setting it to true if bookmarking the OP/topic, which was not correct -- we want to show the icon on the topic list if any post is bookmarked.
Also set to false if unbookmarking the last bookmarked post in the topic.
Also in this PR is a migration to correct any out of sync topic_user.bookmarked columns, based on the new logic.
2020-04-28 16:19:25 +10:00
mentalstring 67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Blake Erickson a93ef2926d
DEV: Add rswag to aid in api documention (#9546)
Adding in rswag will allow us to write spec files to document and test
our api.
2020-04-27 16:40:07 -06:00
Joffrey JAFFEUX 01f2819dde
DEV: enables and fixes multisite-spec (#9557) 2020-04-27 20:55:36 +02:00
Roman Rizzi 394babcae3
FIX: Only show the review page to users that can see it. Do not publish the reviewable count update message to everyone. (#9556) 2020-04-27 14:51:25 -03:00
David Taylor f95609ae23
FIX: Add additional checks for automatic theme script CSP
- Skip absolute URLs
- Skip CDN URLs
- Skip invalid URLs
- Properly format protocol-less URLs
2020-04-27 15:56:29 +01:00
Benno 6e01acb3cb
FIX: Apply category priority for empty query (#9516) 2020-04-27 10:35:27 -04:00
Sam Saffron 8c1e008c59
DEV: Skip erratic spec for now
Spec fails intermittently due to CDN state.
2020-04-25 13:20:04 +10:00
David Taylor 8a112b7464
DEV: Automatically extend CSP when themes link to external scripts (#9531) 2020-04-24 09:47:01 +01:00
Sam Saffron cbceadf48b
FEATURE: when blocking emails prefer blocking canonical
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.

This commit improves the situation by always preferring to block (and check)
canonical emails.

This means that if:

`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`

This means that `samevil+2@domain.com` (ad infinitum) will be blocked
2020-04-24 14:09:51 +10:00
Sam Saffron 6a18c9aa0b
Revert "FEATURE: enforce_canonical_emails site setting"
This reverts commit 6f9177e2ed.

We decided on a completely different approach to the problem.

Instead we will let blocked emails be treated as canonical.
2020-04-24 13:52:06 +10:00
Martin Brennan cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Penar Musaraj 585e7bcfe8 DEV: update specs followup to 67e96f6 2020-04-23 16:11:17 -04:00
Arpit Jalan 39be639c37 FIX: update GitHub screen_name on login via GitHub 2020-04-23 20:54:26 +05:30
Krzysztof Kotlarek 52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Blake Erickson 9cbbaf4237 FIX: Don't throw 500 for invalid website url input
It's possible to cause a 500 error by putting in weird characters in the
input field for updating a users website on their profile.

Normal invalid input like not including the domain extension is already
handled by the user_profile model validation. This fix ensures a server
error doesn't occur for weird input characters.
2020-04-22 13:57:56 -06:00
Jarek Radosz 8adccaf98c
DEV: Use dynamic/static fabricator attrs correctly (#9519) 2020-04-22 20:49:53 +02:00
Vinoth Kannan df0c386f8a
UX: drop the `automatic_membership_retroactive` column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Krzysztof Kotlarek 81c216e04c
FIX: composer problem on the categories page (#9508) 2020-04-22 09:24:54 -07:00
Robin Ward 13f2723dcb FIX: Don't log an error to logster if a topic could not be updated.
If for some reason an update did not go through (for example,
concurrently updating the same topic twice), we were logging something
like:

```
create_errors_json called with unrecognized type: #<Topic
```

This happened because we knew an error occurred but the active record
object had no errors attached.

This patch fixes the issue by attaching a proper error message in the
event that this happens.
2020-04-22 11:56:04 -04:00
Dan Ungureanu 4e5f9d4cd1
DEV: Drop 'key' column from user_api_keys (#9388) 2020-04-22 12:13:19 +03:00
Dan Ungureanu e733701887
FEATURE: Make report filters reusable (#9444)
This commit also adds 'include subcategories' report filter
2020-04-22 11:52:50 +03:00
Vinoth Kannan a511bea4cc
FEATURE: admin UI to merge two users. (#9509) 2020-04-22 14:07:51 +05:30
Martin Brennan 628ba9d1e2
FEATURE: Promote bookmarks with reminders to core functionality (#9369)
The main thrust of this PR is to take all the conditional checks based on the `enable_bookmarks_with_reminders` away and only keep the code from the `true` path, making bookmarks with reminders the core bookmarks feature. There is also a migration to create `Bookmark` records out of `PostAction` bookmarks for a site.

### Summary

* Remove logic based on whether enable_bookmarks_with_reminders is true. This site setting is now obsolete, the old bookmark functionality is being removed. Retain the setting and set the value to `true` in a migration.
* Use the code from the rake task to create a database migration that creates bookmarks from post actions.
* Change the bookmark report to read from the new table.
* Get rid of old endpoints for bookmarks
* Link to the new bookmarks list from the user summary page
2020-04-22 13:44:19 +10:00
Martin Brennan e18aeb799e
FIX: Make sure reminder_type is parsed on bookmark update (#9503)
Otherwise we are trying to update the reminder type with a string which often evaluates to 0 (At Desktop) which causes reminders to come through early.
2020-04-22 10:44:04 +10:00
Jarek Radosz c08753dc34
DEV: Fix topic_timer_spec (#9483)
File it under "how did it even work".
2020-04-21 14:38:35 +10:00
Jarek Radosz a781ef7662
FIX: Reject invalid Category slugs (#9473)
Previously it would sanitize given slug and then save the resulting empty slug.
2020-04-21 03:50:50 +02:00
Jarek Radosz 17cf300b71
DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
Jarek Radosz ab52bed014
DEV: Remove the return value of disable_if_low_on_disk_space (#9469)
It was used only in specs.
2020-04-21 03:48:33 +02:00
Jarek Radosz 5a81e3999c
DEV: Remove `bypass_bump` from CookedPostProcessor (#9468)
It was only passing it along to `PullHotlinkedImages` and that class have not used that arg since April 2014 (c52ee665b4)
2020-04-21 03:48:19 +02:00
Jarek Radosz 9a6e4b1fa1
FIX: Make InlineUploads handle more URL formats (#9467)
It previously failed to match URLs with characters other than `[a-zA-z0-9\.\/:-]`. This meant that `PullHotlinkedImages` would sometimes download an external image and then never use it in any posts.
2020-04-21 03:47:48 +02:00
Mark VanLandingham 06796bec8b
FIX: Set category description to first posts cooked value (#9478) 2020-04-20 14:51:49 -05:00
Robin Ward 25bed4f643 FIX: Concurrency issues with making topic embedded posts visible 2020-04-20 15:11:59 -04:00
Robin Ward 56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
Robin Ward 8f5314bf98 FIX: An `opts` hash was not, in fact, optional :) 2020-04-20 14:17:13 -04:00
Daniel Waterworth 7876ee2d67 DEV: upgrade Rails
Latest version of Rails contains compatibility fixes for Ruby 2.7 and some
minor security fixes we would like to have

It also broke some of the multisite tests.

Rails tries to use the same connection for reading from a replica as writing
to the leader during tests, because, with everything happening in a
transaction, changes to the DB wouldn't otherwise be reflected in the
replica connection.

The difference now is that Rails tries to do this for connections opened
after the test has started which affected rails multisite connections.

The upshot of this is that, as things stand, you are likely to
experience problems if you try to connect to a different multisite DB in
a test when the `current_db` is not 'default'.
2020-04-20 12:55:53 +01:00
Sam Saffron ee36382640
FEATURE: improve rendering of RSS feeds
- Eliminate superfluous "author wrote" block
- Eliminate block-quote for all posts
- Move participant count and reply count to 1 line
- Prioritize name over username if forum requests
- Use fabrication in list controller spec to speed up spec
2020-04-20 16:08:24 +10:00
Martin Brennan 344ef5226c
FEATURE: Edit bookmark reminders from post and explicit delete button (#9455)
There is now an explicit "Delete Bookmark" button in the edit modal. A confirmation is shown before deleting.

Along with this, when the bookmarked post icon is clicked the modal is now shown instead of just deleting the bookmark. Also, the "Delete Bookmark" button from the user bookmark list now confirms the action.

Add a `d d` shortcut in the modal to delete the bookmark.
2020-04-20 13:30:04 +10:00
tshenry e41f1e6ec2
FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
Mark VanLandingham 0e74dd7d7c
FIX: Set user timezone on password reset login (#9461) 2020-04-17 13:39:04 -05:00
Roman Rizzi dce46086f4
FIX: Do not raise an error if the post action type is nil (#9458) 2020-04-17 14:23:33 -03:00
David Taylor 0e4497b6be
DEV: Update tl3 spec to remove hard-coded primary keys
In some test runs, the hard-coded ids could correspond to real topics created earlier in the tests, and cause random test failures
2020-04-17 17:24:14 +01:00
Martin Brennan 8f0544137a
FEATURE: Allow editing bookmark reminders (#9437)
Users can now edit the bookmark name and reminder time from their list of bookmarks.

We use "Custom" for the date and time in the modal because if the user set a reminder for "tomorrow" then edit the reminder "tomorrow", the definition of what "tomorrow" is has changed.
2020-04-17 11:08:07 +10:00
Krzysztof Kotlarek e8e764c064
FIX: flaky groups_controller_spec (#9439)
Sometimes spec which is testing order groups by user count is failing.

My theory is that cause is the randomness of Postgres when the order value is the same for 2 rows.

In spec, we got three groups
`moderator_group` - 0 users
`group` - 1 user
`other_group` - 1 user

And we are expecting that controller will return them in ascending order [moderator, group, other_group]

Because `group` and `other_group` contain the same amount of users, we are dealing with luck

Therefore, I believe that adding one more user to other_group should make that query reliable.

It was not crashing on my local machine, so I am not 100% sure.
2020-04-17 09:04:10 +10:00
David Taylor b6c19cba20
FIX: Abort emit_web_hook_event job cleanly if web hook was deleted (#9445)
Raising an error causes the job to be retried, and causes a lot of noise in the logs
2020-04-16 21:24:09 +01:00
Neil Lalonde 074509fd95
FIX: don't demote users to TL2 when default trust level is 3
Within 24 hours of signing up, new users were losing their
default trust level of 3. With this fix, demotions from
trust level 3 won't happen when the "default trust level"
setting is 3 or 4.
2020-04-16 12:28:16 -04:00
Martin Brennan 51672b9121
FIX: Minor bookmark with reminder issue cleanup (#9436)
* Count user summary bookmarks from new Bookmark table if bookmarks with reminders enabled
* Update topic user bookmarked column when new topic bookmark changed
* Make in:bookmarks search work with new bookmarks
* Fix batch inserts for bookmark rake task (and thus migration). We were only inserting one bookmark at a time, completely defeating the purpose of batching!
2020-04-16 11:32:21 +10:00
Vinoth Kannan 884eea7a83 FEATURE: add support for `upload` format in theme settings. 2020-04-15 18:34:02 +05:30
Krzysztof Kotlarek c21df2286c
FIX: add short_path to upload_serializer (#9417)
What problem I am trying to solve?

When an encrypted message is crafted and the image is added - discourse needs a hard refresh to display that image.

What is happening?

Everything starts here - when the upload is finished we add serialized object to the cache https://github.com/discourse/discourse/blob/master/app/assets/javascripts/discourse/components/composer-editor.js#L748:L757

Then, `discourse-encrypt` is trying to get an image from the cache and use `short_path` property
https://github.com/discourse/discourse-encrypt/blob/master/assets/javascripts/discourse/initializers/hook-decrypt-post.js.es6#L142:L143

Why is it working after a hard refresh?

After refresh, we populate cache once again using that function: https://github.com/discourse/discourse/blob/master/app/assets/javascripts/pretty-text/upload-short-url.js#L11:L17

And lookup_urls method from backend is returning `short_path`
https://github.com/discourse/discourse/blob/master/app/controllers/uploads_controller.rb#L55:L64

TL;DR We should expose short path in upload serializer. I ensured that this serializer is used only when attachments are uploaded so it should not affect performance.
2020-04-15 09:19:59 +10:00
Blake Erickson fec68d3d25 DEV: Confirm email when creating users via the API
This commit is for a frequently requested task on meta so that only 1
API call is needed instead of 3!

In order to create a user via the api and not have them receive an
activation email you can pass in the `active=true` flag. This prevents
sending an email, but it is only half of the solution and puts the db in
a weird state where it has an active user with an unconfirmed email.

This commit fixes that and ensures that if the `active=true` flag is set
the user's email is also marked as confirmed.

This change only applies to admins using the API.

Related topics on meta:

 - https://meta.discourse.org/t/-/68663
 - https://meta.discourse.org/t/-/33133
 - https://meta.discourse.org/t/-/36133
2020-04-14 14:56:32 -06:00
Mark VanLandingham 40531fc85e
DEV: themes_install_task find existing themes consistently (#9424) 2020-04-14 14:20:21 -05:00
Jeff Wong e3590d4ead
FEATURE: add user_session_refreshed trigger (#9412)
Trigger an event for plugins to consume when a user session is refreshed.

This allows external auth to be notified about account activity, and be
able to take action such as use oauth refresh tokens to keep oauth
tokens valid.
2020-04-14 09:32:24 -07:00
Mark VanLandingham f07c4a781c
DEV: Add options to theme install rake task - more options (#9394) 2020-04-14 09:17:00 -05:00
Sam Saffron 6f9177e2ed
FEATURE: enforce_canonical_emails site setting
The new `enforce_canonical_emails` site setting ensures that emails in the
canonical form are unique.

This mean that if `s.a.m+1@gmail.com` is registered `sam@gmail.com` will
not be allowed.

The commit contains a blanket "tag strip" (stripping everything after +)
it also contains special handling of a "dot strip" for googlemail and gmail.

The setting only impacts new registrations after `enforce_canonical_emails`

The setting is default false so it will not impact any existing installs.
2020-04-14 14:16:30 +10:00
Robin Ward b6b92a562c
FEATURE: New site setting `embed_unlisted` (#9391)
If enabled, posts imported to discourse via embeddings will default to
unlisted until they receive a reply.
2020-04-13 15:17:02 -04:00
Justin DiRose 5471c065cd
FIX: Missing timezone guess on email session login (#9404)
Timezone is guessed by moment.js if unset upon a normal login, but was not when
logging in via an email link. This adds logic to update a guessed
timezone upon email login so timezones don't end up blank.
2020-04-10 13:19:39 -05:00
Jeff Wong 5ccbc5f3ee
FEATURE: add after-reviewable-post-user plugin outlet (#9397)
FEATURE: add after-reviewable-post-user plugin outlet

Add a plugin outlet after reviewable post user

Add a basic user serializer that includes custom fields.
Allows review queue serializer to include custom fields for its users
2020-04-09 11:32:39 -07:00
Kane York 138d4aebde FEATURE: Webhooks and Event for user being granted a badge
Adding a webhook for badge revocation is left for future work as it's relatively rare.
2020-04-08 14:38:30 -07:00
Robin Ward e1f8014acd
FEATURE: Support for publishing topics as pages (#9364)
If the feature is enabled, staff members can construct a URL and publish a
topic for others to browse without the regular Discourse chrome.

This is useful if you want to use Discourse like a CMS and publish
topics as articles, which can then be embedded into other systems.
2020-04-08 12:52:36 -04:00
Roman Rizzi 8e1bdc9458
FIX: Prevent low score flags from auto-closing a topic if the reviewable default visibility is higher than low (#9375) 2020-04-08 10:44:31 -03:00
Jarek Radosz 874999941a
DEV: Fix some more flaky tests (#9384)
Includes:
* DEV: Don't overwrite a computed property in test
2020-04-08 12:46:43 +02:00
Sam Saffron b824898f61
FIX: respect automatic group membership when sso changes email
Previously we were only updating group membership when a user record was
first created in an SSO setting.

This corrects it so we also update it if SSO changes the email
2020-04-08 16:33:50 +10:00
Sam Saffron 0375a5ac0b
DEV: reduce logging when no external id is specified
Previously we were returning an unknown sso error and logging a message
when external id was blank. This noise is not needed.
2020-04-08 12:42:28 +10:00
Neil Lalonde c45574c9e7
FIX: exclude private messages from TL3 requirements
It wasn't intended that people should be able to earn trust level
3 without participating in public topics. When counting topic
views and likes given/received, don't count private topics.
2020-04-07 17:36:07 -04:00
Krzysztof Kotlarek a811976023
FIX: reset gravatar cache by adding random param to URL (#9370)
Users noticed that sometimes, avatar from Gravatar is not correctly updated - https://meta.discourse.org/t/updated-image-on-gravatar-not-seeing-it-update-on-site/54357

A potential reason for that is that even if you update your avatar in Gravatar, URL stays the same and if the cache is involved, service is still receiving the old photo.

For example. In my case, when I click the button to refresh avatar the
new Upload record is created with `origin` URL to new avatar, and `url` to
old one

I made some tests in the rails console and adding random param to Gravatar URL is deceiving cache and correct, the newest avatar is downloaded
2020-04-08 07:35:42 +10:00
Rafael dos Santos Silva 3053c6d95e
FEATURE: Enable offline browsing and fullscreen PWA in iOS by default 2020-04-07 16:58:21 -03:00
Robin Ward ce663d67e1 FIX: CSV Exports were throwing errors with invalid dates
This fix will consider any invalid dates to be non-existant.
2020-04-07 14:05:27 -04:00
Roman Rizzi e52e7f6e75
FIX: Include pending queued users regardless of their score (#9372) 2020-04-07 11:42:12 -03:00
Dan Ungureanu 0653750fbf
FEATURE: Hash user API keys in the database (#9344)
The 'key' column will be dropped in a future commit.
2020-04-07 16:42:52 +03:00
Blake Erickson d04ba4b3b2
DEPRECATION: Remove support for api creds in query params (#9106)
* DEPRECATION: Remove support for api creds in query params

This commit removes support for api credentials in query params except
for a few whitelisted routes like rss/json feeds and the handle_mail
route.

Several tests were written to valid these changes, but the bulk of the
spec changes are just switching them over to use header based auth so
that they will pass without changing what they were actually testing.

Original commit that notified admins this change was coming was created
over 3 months ago: 2db2003187

* fix tests

* Also allow iCalendar feeds

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2020-04-06 16:55:44 -06:00
Joffrey JAFFEUX b248c553c2
DEV: ensures CustomEmoji cache is cleared after spec (#9361) 2020-04-06 19:41:59 +02:00
David Taylor 3814ca06a8
DEV: Allow using .js extension for javascript modules in themes (#9358) 2020-04-06 17:24:59 +01:00
Vinoth Kannan fd39c85c1a FIX: add category hashtags support for sub-sub categories.
Hashtags will include last two levels only (ex: "parent:child").
2020-04-06 20:43:38 +05:30
Krzysztof Kotlarek ce00da3bcd
FIX: guardian always got user but sometimes it is anonymous (#9342)
* FIX: guardian always got user but sometimes it is anonymous

```
  def initialize(user = nil, request = nil)
    @user = user.presence || AnonymousUser.new
    @request = request
  end
```

AnonymouseUser defines `blank?` method
```
  class AnonymousUser
    def blank?
      true
    end
    ...
  end
```
so if we would use @user.present? it would be correct, however, just @user is always true
2020-04-06 09:56:47 +10:00
Dan Ungureanu b9d411a4eb
FIX: Topic.time_to_first_response should include sub-sub-categories (#9349) 2020-04-04 13:31:34 +03:00
David Taylor 3cfe086a94
FIX: Limit personal message participants when converting from topic (#9343)
Previously all topic posters would be added which could lead to major performance issues. Now if there are too many posters, only the acting user will be added as a participant.
2020-04-03 16:42:01 +01:00
Vinoth Kannan 82201fa466 FIX: jobs/delete_replies: Add Time+Duration, not Time+Time #9314
Co-authored-by: Kane York <kanepyork@gmail.com>
2020-04-03 09:23:40 +05:30
Vinoth Kannan 2f9879a9a1 FIX: move total rows count & load more URL inside meta.
We're fetching "total rows count" from root attributes only if meta object not found. 2b78bd01ab/app/assets/javascripts/discourse/models/store.js (L236)
2020-04-03 07:32:50 +05:30
Daniel Waterworth 76610acb6f FIX: Default to light theme in wizard so that previews are displayed
Previously, without a theme selection, the previews wouldn't show.
2020-04-02 18:37:45 +01:00
Penar Musaraj 724d2e99de
DEV: Only include "report-sample" CSP directive when reporting is enabled (#9337) 2020-04-02 11:16:38 -04:00
Kane York cdaa60b56b FEATURE: Allow admins to disable self-service account deletion
https://meta.discourse.org/t/-/146276
2020-04-01 15:16:07 -07:00
Neil Lalonde 90fcede832
FIX: replace default welcome topic post with new value from wizard
Previously the text entered in the wizard would be prepended onto the
default first paragraph.
2020-04-01 15:42:45 -04:00
Mark VanLandingham 689c61b462
DEV: Allow plugins to add wizard steps after specific steps (#9315) 2020-04-01 08:36:50 -05:00
Roman Rizzi b2f30aa0b5
FIX: FlagSockpuppets should not flag a post if a post of that user was already rejected by staff (#9328)
* FIX: FlagSockpuppets should not flag a post if a post of that user was already rejected by staff

* Update spec/services/flag_sockpuppets_spec.rb

Co-Authored-By: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-04-01 09:20:45 -03:00
Martin Brennan c07dd0d22a
FEATURE: Add lazy loading to user bookmarks list (#9317)
This is so users with huge amount of bookmarks do not have to wait a long time to see results.

* Add a bookmark list and list serializer to server-side to be able to handle paging and load more URL
* Use load-more component to load more bookmark items, 20 at a time in user activity
* Change the way current user is loaded for bookmark ember models because it was breaking/losing resolvedTimezone when loading more items
2020-04-01 14:09:07 +10:00
Blake Erickson 04b431b6a4 FIX: Staged users getting user_linked and user_quoted emails
This fix ensures that if a staged user is linked to or quoted they won't
be emailed about it.

A staged user could email into a category, and another user could quote
them inside of a completely different category and we don't want a
staged user to receive an email for this.

Bug report:

https://meta.discourse.org/t/-/145202/9
2020-03-31 20:17:56 -06:00
Martin Brennan b79ea986ac
FEATURE: High priority bookmark reminder notifications (#9290)
Introduce the concept of "high priority notifications" which include PM and bookmark reminder notifications. Now bookmark reminder notifications act in the same way as PM notifications (float to top of recent list, show in the green bubble) and most instances of unread_private_messages in the UI have been replaced with unread_high_priority_notifications.

The user email digest is changed to just have a section about unread high priority notifications, the unread PM section has been removed.

A high_priority boolean column has been added to the Notification table and relevant indices added to account for it.

unread_private_messages has been kept on the User model purely for backwards compat, but now just returns unread_high_priority_notifications count so this may cause some inconsistencies in the UI.
2020-04-01 09:09:20 +10:00
Arpit Jalan b2a0d34bb7
FEATURE: add setting `auto_approve_email_domains` to auto approve users (#9323)
* FEATURE: add setting `auto_approve_email_domains` to auto approve users

This commit adds a new site setting `auto_approve_email_domains` to
auto approve users based on their email address domain.

Note that if a domain already exists in `email_domains_whitelist` then
`auto_approve_email_domains` needs to be duplicated there as well,
since users won’t be able to register with email address that is
not allowed in `email_domains_whitelist`.

* Update config/locales/server.en.yml

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2020-03-31 23:59:15 +05:30
Krzysztof Kotlarek a0f59a55cc
FIX: when a post is moved copy notifications level (#9311)
This is a revert of 2c011252f1

More information on meta: https://meta.discourse.org/t/when-a-reply-is-moved-to-a-new-topic-the-followers-of-the-previous-topic-are-automatically-follower-of-the-new-topic-as-well/130025
2020-03-31 16:19:47 +11:00
Neil Lalonde 29e733832d
FIX: error when changing a topic's category and creating a tag 2020-03-30 16:43:01 -04:00
Robin Ward db35baba26 FEATURE: Display "Last Updated At" on user directory 2020-03-30 14:34:48 -04:00
Joffrey JAFFEUX 0996c3b7b3
FEATURE: allows multiple custom emoji groups (#9308)
Note: DBHelper would fail with a sql syntax error on columns like "group".

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2020-03-30 20:16:10 +02:00
Joffrey JAFFEUX aeaea3c154
FIX: correctly take category/group filters into csv export (#9300) 2020-03-30 19:08:47 +02:00
Joffrey JAFFEUX 96e841a635
DEV: trigger :username_changed when a user is renamed (#9301) 2020-03-30 18:02:34 +02:00
Jarek Radosz 104b647ca4 DEV: Fix another time-sensitive flaky spec 2020-03-27 20:39:25 +01:00
Michael Brown 9026c55fe4
FIX: custom SQL with a trailing comment might break BadgeGranter SQL (#9285)
For example given a custom badge with SQL:
```
SELECT 1
-- I am a comment
```

You end up with

```
FROM (SELECT 1
-- I am a comment) q
```

This fix adds newlines so you end up with the now-valid:

```
FROM (
  SELECT 1
-- I am a comment
) q
```
2020-03-27 14:16:14 -04:00
Jarek Radosz 7ff889574d
DEV: Add rubocop-rspec (#9288)
This adds rubocop-rspec, and enables some cops that were either already passing or are passing now, after fixing them in this commit.

Some new cops are disabled for now, with annotation: "TODO" or "To be decided". Those either need to be discussed first, or require manual changes, or the number of found and fixed offenses is too large to bundle them up in a single PR.

Includes:

* DEV: Update rubocop's `TargetRubyVersion` to 2.6
* DEV: Enable RSpec/VoidExpect
* DEV: Enable RSpec/SharedContext
* DEV: Enable RSpec/EmptyExampleGroup (Removed an obsolete empty spec file)
* DEV: Enable RSpec/ItBehavesLike
* DEV: Remove RSpec/ScatteredLet (It's too strict, as it doesn't recognize fab! as a let-like)
* DEV: Remove RSpec/MultipleExpectations
2020-03-27 17:35:40 +01:00
Sam Saffron 40b6e278a0
FIX: ensure wiki editor is assigned consistently
`DISTINCT ON` was not providing stability, this ensures the backfill will
be stable on a specific post avoiding risk around double grants.
2020-03-27 12:41:06 +11:00
Jarek Radosz d21d80198c
DEV: Update rubocop-discourse (#9270)
Includes:
* DEV: Use `eq_time` matcher
2020-03-26 16:32:41 +01:00
Bianca Nenciu 7952cbb9a2
FIX: Perform crop using user-specified image sizes (#9224)
* FIX: Perform crop using user-specified image sizes

It used to resize the images to max width and height first and then
perform the crop operation. This is wrong because it ignored the user
specified image sizes from the Markdown.

* DEV: Use real images in test
2020-03-26 16:40:00 +02:00
Sam Saffron 25f1f23288
FEATURE: Stricter rules for user presence
Previously we would consider a user "present" and "last seen" if the
browser window was visible.

This has many edge cases, you could be considered present and around for
days just by having a window open and no screensaver on.

Instead we now also check that you either clicked, transitioned around app
or scrolled the page in the last minute in combination with window
visibility

This will lead to more reliable notifications via email and reduce load of
message bus for cases where a user walks away from the terminal
2020-03-26 17:36:52 +11:00
Martin Brennan efd5fb665b
DEV: Fix flaky time sensitive uploads.rake specs (#9283)
Also fix issues in spec where certain uploads were not considered secure
2020-03-26 13:31:39 +10:00
Sam Saffron ecbccab159
FEATURE: invite_code is case-insensitive
Previously we required exact casing for invite code, this can cause a lot
of confusion. Relax the requirement.
2020-03-26 13:44:02 +11:00
Martin Brennan 097851c135
FIX: Change secure media to encompass attachments as well (#9271)
If the “secure media” site setting is enabled then ALL files uploaded to Discourse (images, video, audio, pdf, txt, zip etc. etc.) will follow the secure media rules. The “prevent anons from downloading files” setting will no longer have any bearing on upload security. Basically, the feature will more appropriately be called “secure uploads” instead of “secure media”.

This is being done because there are communities out there that would like all attachments and media to be secure based on category rules but still allow anonymous users to download attachments in public places, which is not possible in the current arrangement.
2020-03-26 07:16:02 +10:00
Mark VanLandingham c14f6d4ced
FEATURE: Allow plugins to exclude wizard steps (#9275) 2020-03-25 11:36:42 -05:00
Jeff Wong 7282435e17 FEATURE: Unassign the review queue topic when a flag is handled 2020-03-24 14:57:44 -07:00
Bianca Nenciu 61c1af0124 SECURITY: Ensure user can see group and group members 2020-03-24 11:59:41 +02:00
Bianca Nenciu d8640fd042
DEV: Move requested_group_id custom field from post to topic (#9127)
Follow-up-to accbbded15
2020-03-24 11:12:52 +02:00
Sam Saffron 46a9622246
FIX: prevent scheduled publishing to deleted category
We missed a dependency which left timers firing on missing categories.

Co-authored-by: tshenry
2020-03-24 16:59:42 +11:00
Kane York 58ae0d4bd9
DEV: Add test case for /srv/status probers (#9259) 2020-03-24 16:28:07 +11:00
David Taylor 5db41cd578
SECURITY: Respect topic permissions when loading bookmark metadata
Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2020-03-23 11:30:48 +00:00
David Taylor 5ff505cea6
SECURITY: Respect topic permissions when loading draft metadata
Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2020-03-23 11:30:40 +00:00
Jeff Wong 3189dab622 FIX: correctly remove authentication_data cookie on oauth login flow
Additionally correctly handle cookie path for authentication_data

There were two bugs that exposed an interesting case where two discourse
instances hosted across two subfolder installs in the same domain
with oauth may clash and cause strange redirection on first login:

Log in to example.com/forum1. authentication_data cookie is set with path /
On the first redirection, the current authentication_data cookie is not unset.
Log in to example.com/forum2. In this case, the authentication_data cookie
is already set from forum1 - the initial page load will incorrectly redirect
the user to the redirect URL from the already-stored cookie, to /forum1.

This removes this issue by:

* Setting the cookie for the correct path, and not having it on root
* Correctly removing the cookie on first login
2020-03-21 14:34:25 -07:00
Blake Erickson c97244ca11 FIX: post edited webhook does not reflect updated topic title
This fix ensures that when a topic title is edited the new title shows
up in the post webhook instead of the old title.

Rather than passing in the old topic object to the PostRevisor the
PostRevisor initializer will load the updated topic object inside of the
initializer if you don't pass it in. This will allow the post_edited
webhook to have the correct topic values.

Original bug reported at:

https://meta.discourse.org/t/post-edited-webhook-does-not-reflect-updated-topic-title/144722
2020-03-21 07:43:11 -06:00
David Taylor dbfec4b268
DEV: Update spec for category permalink
Followup to 3215f2b6ee
2020-03-20 22:26:45 +00:00
David Taylor 3215f2b6ee
FIX: Permalinks should redirect to category URL including the ID
This is a temporary fix. Urls for third-level categories should function without the id. Once that is fixed, this change can be reverted
2020-03-20 22:06:20 +00:00
Jeff Wong 4ecc0a25ae Revert "FIX: correctly remove authentication_data cookie on oauth login flow (#9238) (#9251)"
This reverts commit beaeb0c4b2.
2020-03-20 14:37:55 -07:00
Jeff Wong beaeb0c4b2
FIX: correctly remove authentication_data cookie on oauth login flow (#9238) (#9251)
Attempt 2, with more test.

Additionally correctly handle cookie path for authentication_data

There were two bugs that exposed an interesting case where two discourse
instances hosted across two subfolder installs in the same domain
with oauth may clash and cause strange redirection on first login:

Log in to example.com/forum1. authentication_data cookie is set with path /
On the first redirection, the current authentication_data cookie is not unset.
Log in to example.com/forum2. In this case, the authentication_data cookie
is already set from forum1 - the initial page load will incorrectly redirect
the user to the redirect URL from the already-stored cookie, to /forum1.

This removes this issue by:

Setting the cookie for the correct path, and not having it on root
Correctly removing the cookie on first login
2020-03-20 14:03:38 -07:00
Robin Ward 0d3386d255 Revert "FIX: correctly remove authentication_data cookie on oauth login flow (#9238)"
This reverts commit a1f9b1a7fc.

This might have caused a problem with social logins. We are confirming
via this revert and will follow up.
2020-03-20 15:25:10 -04:00
romanrizzi cfec10a568 FIX: Moderators should be able to review flagged PMs since this has always been like this 2020-03-20 12:28:36 -03:00
Jarek Radosz 20f3be1e9c
DEV: Use the `type: :multisite` spec setting (#9245)
Fixes recent spec flakiness.

Also includes:
DEV: Prevent accidental exit from specs
2020-03-20 16:18:34 +01:00
Daniel Waterworth 1b24a7b993 FIX: Include entire slug path in permalinks
This is a temporary fix since these URLs should contain the id as well.
2020-03-20 10:43:13 +00:00
Sam Saffron 10b37e1e36
FIX: add support for sub-sub category slugs in search
Previous to this change slugs for leaves in 3 level nestings would not work

Our UX picks only the last two levels

This also makes the results consistent for slugs as it enforces order.
2020-03-20 15:36:50 +11:00
David Taylor 19814c5e81
FIX: Allow CSP to work correctly for non-default hostnames/schemes (#9180)
- Define the CSP based on the requested domain / scheme (respecting force_https)
- Update EnforceHostname middleware to allow secondary domains, add specs
- Add URL scheme to anon cache key so that CSP headers are cached correctly
2020-03-19 19:54:42 +00:00
Vinoth Kannan f3ddc36ac6 FIX: update `email_digests` user option when `default_email_digest_frequency` updated. 2020-03-20 00:55:47 +05:30
Roman Rizzi 080960a15e
FIX: Show the envelope icon when the flagged post is a PM. Flagged PM must be exclusively reviewed by admins (#9232) 2020-03-19 15:12:42 -03:00
Vinoth Kannan f6d6f1701f FIX: use the new duration attribute in `set_or_create_timer` method.
New `duration` attribute is introduced for the `set_or_create_timer` method in the commit aad12822b7 for "based on last post" and "auto delete replies" topic timers.
2020-03-19 21:45:05 +05:30
Jeff Wong a1f9b1a7fc
FIX: correctly remove authentication_data cookie on oauth login flow (#9238)
Additionally correctly handle cookie path for authentication_data

There were two bugs that exposed an interesting case where two discourse
instances hosted across two subfolder installs in the same domain
with oauth may clash and cause strange redirection on first login:

Log in to example.com/forum1. authentication_data cookie is set with path /
On the first redirection, the current authentication_data cookie is not unset.
Log in to example.com/forum2. In this case, the authentication_data cookie
is already set from forum1 - the initial page load will incorrectly redirect
the user to the redirect URL from the already-stored cookie, to /forum1.

This removes this issue by:
* Setting the cookie for the correct path, and not having it on root
* Correctly removing the cookie on first login
2020-03-19 09:00:46 -07:00
Vinoth Kannan aad12822b7
FEATURE: automatically delete replies on a topic after N days. (#9209) 2020-03-19 21:06:31 +05:30
Martin Brennan 0cd502a558
FIX: N1 issues for bookmark list (#9236)
* Preload custom fields for BookmarkQuery and add preload callback. Copy TopicQuery preload methodology to allow plugins to preload data for the BookmarkQuery. This fixes assigned plugin custom fields N1
* Include topic tags in initial query to avoid tags N1

Related: discourse/discourse-assign#63
2020-03-19 15:48:23 +10:00
Martin Brennan 8769ca08bb SECURITY: Prevent access to other user's bookmark lists 2020-03-19 10:59:32 +10:00
Dan Ungureanu 1393950dbc
FIX: Improve HTML to Markdown conversion (#9231)
This commit ensures that whitespaces are preserved in <pre>, but removed
inside text paragraphs.
2020-03-18 19:31:10 +02:00
Kane York 8e98f39d9f Add spec for CSP subfolder 2020-03-17 19:09:06 -07:00
Justin DiRose 9101227266
FEATURE: Demote muted categories on category list (#9226) 2020-03-17 15:33:15 -05:00
Blake Erickson 919e405c48
FIX: Don't display webhooks for inactive plugins (#9206)
* FIX: Don't display webhooks for inactive plugins

This commit ensures that we don't show webhooks for plugins that are not
installed or that are disabled.

Bug report:

https://meta.discourse.org/t/webhookeventtype-and-the-solved-and-assign-plugins/144180

* rename to just 'active', it's cleaner
2020-03-17 10:39:24 -06:00
Jarek Radosz e950471c0f
DEV: Replace User.unstage and User#unstage API with User#unstage! (#8906)
* DEV: Replace User.unstage and User#unstage API with User#unstage!

Quoting @SamSaffron:

> User.unstage mixes concerns of both unstaging users and updating params which is fragile/surprising.
> u.unstage destroys notifications and raises a user_unstaged event prior to the user becoming unstaged and the user object being saved.

User#unstage! no longer updates user attributes and saves the object before triggering the `user_unstaged` event.

* Update one more spec

* Assign attributes after unstaging
2020-03-17 16:48:24 +01:00
Martin Brennan 16799da580
FIX: Improve bookmark modal on mobile and bookmark sync rake task (#9221)
* Improve the bookmark mobile on modal so it doesn't go all the way to the edge and the custom datetime input is easier to use
* Improve the rake task for syncing so it does not error for topics that no longer exist and batches 2000 inserts at a time, clearing the array each time
2020-03-17 15:15:22 +10:00
Roman Rizzi 4663304775
FEATURE: Show rejected posts count in user summary (#9204) 2020-03-16 09:52:08 -03:00
Martin Brennan a6e9057609
FIX: Ensure show_short URLs handle secure uploads using multisite (#9212)
Meta report: https://meta.discourse.org/t/short-url-secure-uploads-s3/144224
* if the show_short route is hit for an upload that is
  secure, we redirect to the secure presigned URL. however
  this was not taking into account multisite so the db name
  was left off the path which broke the presigned URL
* we now use the correct url_for method if we know the
  upload (like in the show_short case) which takes into
  account multisite
2020-03-16 11:54:14 +10:00
Sam Saffron a1d660d951
FEATURE: optional global invite_code for account registration
On some sites when bootstrapping communities it is helpful to bootstrap
with a "light weight" invite code.

Use the site setting `invite_code` to set a global invite code.

In this case the administrator can share the code with
a community which is very easy to remember and then anyone who has
that code can easily register accounts.

People without the invite code are not allowed account registration.

Global invite codes are less secure than indevidual codes, in that they
tend to leak in the community however in some cases when starting a brand
new community the security guarantees of invites are not needed.
2020-03-15 21:17:28 +11:00
Roman Rizzi 27bc4f51c7
FIX: Ignore suspect users that were migrated or users who were created more than six months ago (#9205) 2020-03-14 08:47:53 -03:00
Gerhard Schlager 8022e51179 FIX: Failed to restore backups from versions without translation overrides
Rails calls I18n.translate during initialization and by default translation overrides are used. Database migrations would fail if the system tried to migrate from an old version that didn't have the `translation_overrides` table with all its columns yet.

This makes restoring really old backups work again. Running `DISABLE_TRANSLATION_OVERRIDES=1 rake db:migrate` will allow you to upgrade such an old database as well.
2020-03-14 00:00:22 +01:00
David Taylor 3723c64257
DEV: Correct references to theme flags
Followup to d1474e94
2020-03-13 16:45:55 +00:00
Neil Lalonde 7c27f9bba9
FIX: remove parent tag from tag group
Having a tag be a member of a tag group and the group's parent tag at
the same time causes some unexpected behavior. When a tag is assigned
as the parent, remove it from the group.
2020-03-13 12:25:58 -04:00
David Taylor 3d71b68195
DEV: Introduce plugin api for conditionally rendering assets (#9200) 2020-03-13 15:30:31 +00:00
Martin Brennan dc02586d99 Fix error on create bookmark for topic 2020-03-13 13:33:57 +10:00
Martin Brennan af92444948
FIX: Make sure bookmark serializer works with deleted topics + posts (#9195) 2020-03-13 10:44:39 +10:00
Martin Brennan 2237ba8c9d
FIX: Add topic deleted check to email/sender (#9166)
It already had a deleted post check and log reason, add a topic one too to avoid errors
2020-03-13 10:04:15 +10:00
David Taylor ec2d49d48a
DEV: Allow plugins to add theme modifiers via db migrations (#9192) 2020-03-12 16:35:28 +00:00
Daniel Waterworth 59578dfc5b FIX: Notification emails with attachments are incorrectly structured
Two behaviors in the mail gem collide:

 1. Attachments are added as extra parts at the top level,
 2. When there are both text and html parts, the content type is set to
    'multipart/alternative'.

Since attachments aren't alternative renderings, for emails that contain
attachments and both html and text parts, some coercing is necessary.
2020-03-12 15:42:24 +00:00
Stasiek Michalski 1b8793e7a4
FEATURE: Add support for custom gravatar-like services (#9137)
Adds 3 config values that allow to set a custom provider of Gravatar-like API accessible from gravatar_base_url. The gravatar_name is purely cosmetic, but helps with associating name with the service that actually provides the avatars. gravatar_login_url is a link relative to gravatar_base_url, which provides the user with the login to the Gravatar service
2020-03-12 11:23:55 -04:00
Martin Brennan 03c012bd97 Unscope post + topic relation for bookmark
* we want to be able to get these records for the bookmark
  even if they are trashed, for serialization for the bookmark
  list
2020-03-12 16:00:45 +10:00
Martin Brennan e1eb5fb9b3
FEATURE: MVP Bookmarks with reminders user list changes (#8999)
* This PR changes the user activity bookmarks stream to show a new list of bookmarks based on the Bookmark record.
* If a bookmark has a name or reminder it will be shown as metadata above the topic title in the list
* The categories, tags, topic status, and assigned show for each bookmarked post based on the post topic
* Bookmarks can be deleted from the [...] menu in the list
* As well as this, the list of bookmarks from the quick access panel is now drawn from the Bookmarks table for a user:
* All of this new functionality is gated behind the enable_bookmarks_with_reminders site setting
The /bookmarks/ route now redirects directly to /user/:username/activity/bookmarks-with-reminders
* The structure of the Ember for the list of bookmarks is not ideal, this is an MVP PR so we can start testing this functionality internally. There is a little repeated code from topic.js.es6. There is an ongoing effort to start standardizing these lists that will be addressed in future PRs.
* This PR also fixes issues with feature detection for at_desktop bookmark reminders
2020-03-12 15:20:56 +10:00
Martin Brennan 849631188f
FEATURE: Allow custom date + time for bookmark reminders (#9185)
A custom date and time can now be selected for a bookmark reminder

The reminder will not happen at the exact time but rather at the next 5 minute interval of the bookmark reminder schedule.

This PR also fixes issues with bulk deleting topic bookmarks.
2020-03-12 10:52:15 +10:00
Martin Brennan 793f39139a
FEATURE: Send notifications for time-based and At Desktop bookmark reminders (#9071)
* This PR implements the scheduling and notification system for bookmark reminders. Every 5 minutes a schedule runs to check any reminders that need to be sent before now, limited to **300** reminders at a time. Any leftover reminders will be sent in the next run. This is to avoid having to deal with fickle sidekiq and reminders in the far-flung future, which would necessitate having a background job anyway to clean up any missing `enqueue_at` reminders.

* If a reminder is sent its `reminder_at` time is cleared and the `reminder_last_sent_at` time is filled in. Notifications are only user-level notifications for now.

* All JavaScript and frontend code related to displaying the bookmark reminder notification is contained here. The reminder functionality is now re-enabled in the bookmark modal as well.

* This PR also implements the "Remind me next time I am at my desktop" bookmark reminder functionality. When the user is on a mobile device they are able to select this option. When they choose this option we set a key in Redis saying they have a pending at desktop reminder. The next time they change devices we check if the new device is desktop, and if it is we send reminders using a DistributedMutex. There is also a job to ensure consistency of these reminders in Redis (in case Redis drops the ball) and the at desktop reminders expire after 20 days.

* Also in this PR is a fix to delete all Bookmarks for a user via `UserDestroyer`
2020-03-12 10:16:00 +10:00
Roman Rizzi b9aaa9718d
FIX: When must_approve_users is enabled, we don't want to send suspect users to the review queue. Only non-approved users should be sent. Provide a migration to auto-approve every problematic review item (#9179) 2020-03-11 17:05:44 -03:00
Robin Ward a3f0543f99
Support for transpiling `.js` files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
David Taylor d1474e94a1
FEATURE: Allow themes to specify modifiers in their about.json file (#9097)
There are three modifiers:
- serialize_topic_excerpts (boolean)
- csp_extensions (array of strings)
- svg_icons (array of strings)

When multiple themes are active, the values will be combined. The combination method varies based on the setting. CSP/SVG arrays will be combined. serialize_topic_excerpts will use `Enumerable#any`.
2020-03-11 13:30:45 +00:00
Jarek Radosz 4e5dd4105c DEV: Fix flaky `signed_url_for_path` spec
AWS gem uses internally `Time.now` to generate the presigned URLs, so often two consecutive calls with the same params would give different results.
2020-03-10 23:22:26 +01:00
Blake Erickson 6fb4c333b0
FIX: Throw error when removing a user from group fails (#9162)
This commit ensures that an error is thrown when a user fails to be
removed from a group instead of silently failing.

This means when using the api you will receive a 400 instead of a 200 if
there is a failure. The remove group endpoint allows the removal of
multiple users, this change means that if you try to delete 10 users,
but 1 of them fails you will receive a 400 instead of 200 even though
the other 9 were removed successfully. Rather than adding a bunch more
complexity I think this is more than adequate for most use cases.
2020-03-10 15:25:00 -06:00
Jarek Radosz 29b35aa64c
DEV: Improve flaky time-sensitive specs (#9141) 2020-03-10 22:13:17 +01:00
Jarek Radosz aec26ad2f0
FIX: Preserve TopicCreator's timestamp resolution (#9158)
Continuation of #9140 (e35bc8b). It's the last piece required for #9141.
2020-03-10 15:35:40 +01:00
Penar Musaraj e69b6379ad
FEATURE: Broader support for post uploads in video markup (#9152)
Ensures URLs in the following HTML attributes are included in post uploads:
- video poster
- source src
- track src
2020-03-10 09:01:40 -04:00
Roman Rizzi 826b4793c0
FEATURE: Approve suspect users is now true by default. The suspect users list was removed (#9151) 2020-03-10 08:56:42 -03:00
Kane York 218d3f5f44 Add tests for /srv/status behavior 2020-03-09 14:06:13 -07:00
Robin Ward ec21a15fa7 FIX: Incorrect message when logging in via email
If `hide email address taken` was true, it would always display a
success message, rather than the non-comittal "maybe we emailed that
user" response.
2020-03-09 12:54:33 -04:00
Jarek Radosz e35bc8bebd
FIX: Preserve PostCreator's created_at resolution (#9140)
PostMover passes to PostCreator a `created_at` that is a `ActiveSupport::WithTimeZone` instance (and also `is_a? Time`). Previously it was always being passed through `Time.zone.parse` so it would lose sub-second information. Now, it takes `Time` input as-is, while still parsing other types.
2020-03-09 17:38:13 +01:00
Jarek Radosz 85e03a7f68
DEV: Replace `Time.new` with `Time.now` (#9142)
(or `Time.zone.now`)
2020-03-09 17:37:49 +01:00
Mark VanLandingham 174764be25
FEATURE: Add embed_set_canonical_url setting (#9134) 2020-03-09 09:31:24 -05:00
Gerhard Schlager edc8d58ac3 FEATURE: Add site setting to disable staged user cleanup
... and disabled the cleanup during imports, otherwise a running Sidekiq might delete users before posts are created
2020-03-09 10:26:41 +01:00
Joffrey JAFFEUX 60b47d622e
UX: adds support for a color setting type (#9016) 2020-03-09 10:07:03 +01:00
Martin Brennan baea65e4bc
FIX: Embarassing algoriths typo -> algorithms for security keys (#9126) 2020-03-09 09:22:08 +10:00
Bianca Nenciu 20cfa7b810
FIX: Check if auth token exists before revocation (#9095) 2020-03-07 15:04:12 +02:00
Blake Erickson 43b54c631d
DEV: Reserve webhook event types to be used in plugins (#9110)
* DEV: Reserve webhook event types to be used in plugins

Based on feedback on the following PR's:

https://github.com/discourse/discourse-solved/pull/85

https://github.com/discourse/discourse-assign/pull/61

This commit reserves ID's to be used for webhook event types to ensure
that some other webhook or plugin doesn't end up using the same ID.

* Fix broken test

I don't think this test has to test ALL event types to verify that this
feature is working. Now that we added some event types that plugins are
using this test was failing for missing fabricators that exist in the
respective plugins.

* remove loop and just test first record
2020-03-06 10:16:19 -07:00
Mark VanLandingham 475f545557
DEV: Test theme js compiler append_raw_template (#8950)
* DEV: Test for append_raw_template

* remove empty line
2020-03-06 11:35:52 -05:00
Robin Ward 91682408e0 Remove very bad tests 2020-03-06 09:35:55 -05:00
Robin Ward a5f61729e0 Revert "Revert "FIX: Don't allow people to clear the upload bucket while it's enabled""
This reverts commit d4fc76b335.
2020-03-06 09:35:55 -05:00
David Taylor ff62911a89
FEATURE: New route for loading multiple user cards simultaneously (#9078)
Introduces `/user-cards.json`

Also allows the client-side user model to be passed an existing promise when loading, so that multiple models can share the same AJAX request
2020-03-06 12:23:22 +00:00
Martin Brennan 29ccdf5d35
FIX: Show a nicer error if name/code missing for TOTP/Security Keys (#9124)
Meta: https://meta.discourse.org/t/improve-error-message-when-not-including-name-setting-up-totp/143339

* when the user creates a TOTP second factor method we want
to show them a nicer error if they forget to add a name
or the code from the app, instead of the param missing error
* also add a client-side check for this and for security key name,
no need to bother the server if we can help it
2020-03-06 14:37:40 +10:00
Sam Saffron 494379201d
DEV: attempt to stabilize flaky spec
Spec was checking implementation when it did not need to, temp file was blank
so optimized image should be blank
2020-03-06 12:57:12 +11:00
Robin Ward d4fc76b335 Revert "FIX: Don't allow people to clear the upload bucket while it's enabled"
This reverts commit 4bb8db024c.
2020-03-05 16:29:59 -05:00
Robin Ward 4bb8db024c FIX: Don't allow people to clear the upload bucket while it's enabled 2020-03-05 16:04:52 -05:00
David Taylor 5b3630dba3
FIX: Do not raise an error when in:all search is performed by anon (#9113)
Also improve in:all specs to catch to catch similar failures
2020-03-05 17:50:29 +00:00
Robin Ward e01d5e2adc SECURITY: Add more restrictions on invite emails
They could be filtered and returned in some circumstances where they
shouldn't have been.
2020-03-05 09:23:21 -05:00
Robin Ward 79ce7085c2 SECURITY: Ensure the invite JSON API matches the UX
Anonymous users could query the invite json and see counts and
summaries which is not allowed in the UX of Discourse.

This commit has those endpoints return a 403 unless the user is
allowed to invite.
2020-03-05 09:23:21 -05:00
Arpit Jalan a157f4aaaa Remove invite_admin route. 2020-03-05 06:45:08 +05:30
Gerhard Schlager 8fa8bab9ff FIX: Don't optimize icons during db:migrate when restoring backup
Uploads are extracted after the DB migration, so this could lead to a failure during the restore. Site icons get optimized after extracting uploads.
2020-03-04 16:59:49 +01:00
Martin Brennan 3e54e0191e
FIX: Use full URL for secure attachments when secure media enabled (#9037)
When secure media is enabled and an attachment is marked as secure we want to use the full url instead of the short-url so we get the same access control post protections as secure media uploads.
2020-03-04 10:11:08 +11:00
David Taylor 65cc61be7a
PERF: Allow preloading 'recent time read' for a user (#9076)
This will be used when serializing multiple user cards
2020-03-03 13:57:46 +00:00
Martin Brennan 0388653a4d
DEV: Upload and secure media retroactive rake task improvements (#9027)
* Add uploads:sync_s3_acls rake task to ensure the ACLs in S3 are the correct (public-read or private) setting based on upload security

* Improved uploads:disable_secure_media to be more efficient and provide better messages to the user.

* Rename uploads:ensure_correct_acl task to uploads:secure_upload_analyse_and_update as it does more than check the ACL

* Many improvements to uploads:secure_upload_analyse_and_update

* Make sure that upload.access_control_post is unscoped so deleted posts are still fetched, because they still affect the security of the upload.

* Add escape hatch for capture_stdout in the form of RAILS_ENABLE_TEST_STDOUT. If provided the capture_stdout code will be ignored, so you can see the output if you need.
2020-03-03 10:03:58 +11:00
Roman Rizzi 537f87562e
FIX: We need to skip users with associated reviewables when auto-approving (#9080)
* FIX: We need to skip users with associated reviewables when auto-approving them

* Update spec/initializers/track_setting_changes_spec.rb

* Update spec/initializers/track_setting_changes_spec.rb

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-03-02 14:33:52 -05:00
tshenry a09e5d12c2
FIX: Topics should honor auto-close when published to category (#8963)
* FIX: Topics should honor auto-close when published to category

* Add test
2020-03-02 14:21:35 -05:00
Martin Brennan 8123538c94
DEV: Minor review fixes and fix bookmark spec logging (#9045)
As per:

https://review.discourse.org/t/fix-never-allow-custom-emoji-to-be-marked-secure-8965/9072
https://review.discourse.org/t/feature-improving-bookmarks-part-2-topic-bookmarking-8954/9038
2020-03-02 15:40:29 +10:00
Sam Saffron e23e247dff
DEV: spec suite fails on leap years
Something about year math fails here on leap years, just
set up the clock so it is consistent to avoid it.
2020-02-29 18:30:08 +11:00
David Taylor 0903aa44bb
FEATURE: Always disable customizations on the `/safe-mode` route (#9052)
This makes it easier to enter safe mode when a customization has made the UI unusable
2020-02-28 10:53:11 +00:00
Dan Ungureanu 8cbb6e35cb
DEV: Fix build
Follow up to 60184a290c.
2020-02-28 11:31:04 +02:00
romanrizzi 4673f31c75 FIX: Bulk badge awards should work even if the CSV has nil values 2020-02-27 11:08:24 -03:00
Dan Ungureanu 60908a94ec
FIX: Skip 'invited' small action if user is in an invited group (#9056)
Inviting a user that is already invited through a group used to generate
a small action and a notification. This commit skips that small action.
2020-02-27 14:45:20 +02:00
Vinoth Kannan acf337d583
FEATURE: auto archive group message if topic is closed. (#9046)
Co-Authored-By: Régis Hanol <regis@hanol.fr>
2020-02-27 11:09:37 +05:30
Vinoth Kannan 5774107a2d
FIX: downloaded image URLs incorrectly replaced in post raw. (#9014)
Previously, while replacing the downloaded image URL `http://wiki.mozilla.org/images/2/2e/Longcat1.png` similar non-image URL `http://wiki.mozilla.org/images/2` was replaced wrongly.
2020-02-27 10:22:55 +05:30
Joffrey JAFFEUX baba1cc02e
FIX: ensures destroying a user with security keys doesn't fail (#9042) 2020-02-25 14:07:57 -05:00
Dan Ungureanu 514c22e64b
FIX: Make reviewable claiming work with deleted topics (#9040)
This fixes the case when the reviewed topic is deleted (for example, in
discourse-akismet).
2020-02-25 15:49:23 +02:00
Joffrey JAFFEUX 0ea11a9d49
FIX: ensures we don't attempt to create a new PM on an existing topic (#9029)
This fix attempts to both fix it at UI level and server side. A previous attempt related to this behavior has been made in commit: 49c750ca78
2020-02-24 08:55:12 -06:00
Sam Saffron 372f6f4f22
FEATURE: limit number of notifications per user to 10,000
Introduces a new site setting `max_notifications_per_user`.

Out-of-the-box this is set to 10,000. If a user exceeds this number of
notifications, we will delete the oldest notifications keeping only 10,000.

To disable this safeguard set the setting to 0.

Enforcement happens weekly.

This is in place to protect the system from pathological states where a
single user has enormous amounts of notifications causing various queries
to time out. In practice nobody looks back more than a few hundred notifications.
2020-02-24 11:42:50 +11:00
Neil Lalonde f73ed45de9 FIX: blank popular posts in summary emails due to lightbox images
When looking for the first paragraph with content in a post,
it was matching the lightboxed image paragraph as "<p></p>".
Fix that and other potential empty paragraphs with the
p:not(:empty) selector.
Add a new selector to find the image links in lightboxed
images as valid content for emails.
2020-02-21 16:18:38 -05:00
Dan Ungureanu 533495169e
FEATURE: Publish a message when reviewable claimer changes (#9019)
This commit ensures that all users are kept in sync and no user can claim
a topic that has been claimed already.
2020-02-21 19:11:50 +02:00
Dan Ungureanu cf0c6d5761
FIX: Ensure web hooks are retried at most 5 times 2020-02-21 17:02:40 +02:00
Martin Brennan 3af2670bd5
FIX: Consider webp a supported image format for upload (#9015)
* Also fixes an issue where if webp was a downloaded hotlinked
  image and then secure + sent in an email, it was not being
  redacted because webp was not a supported media format in
  FileHelper
* Webp originally removed as an image format in
  https://github.com/discourse/discourse/pull/6377
  and there was a spec to make sure a .bin webp
  file did not get renamed from its type to webp.

  However we want to support webp images now to make
  sure they are properly redacted if secure media is
  on, so change the example in the spec to use tiff,
  another banned format, instead
2020-02-21 13:08:01 +10:00
Martin Brennan 6a2bde4d48 Fix broken secure media specs 2020-02-21 10:01:32 +10:00
Martin Brennan 04df3bd46d
FIX: Only mark attachments as secure media if SiteSetting.secure_media? (#9009)
* Attachments (non media files) were being marked as secure if just
SiteSetting.prevent_anons_from_downloading_files was enabled. this
was not correct as nothing should be marked as actually "secure" in
the DB without that site setting enabled
* Also add a proper standalone spec file for the upload security class
2020-02-21 09:35:16 +10:00
Robin Ward a47e0a3fda FIX: TOTP could not be used on sites with colons in their names
This is because the TOTP gem identifies as a colon as an addressable
protocol. The solution for now is to remove the colon in the issuer
name.

Changing the issuer changes the token values, but now it was completely
broken for colons so this should not be breaking anyone new.
2020-02-20 16:35:30 -05:00
Roman Rizzi c7787464cd
FEATURE: Admins can configure the reflag cooldown window and if posts flagged as spam by TL3+ users get automatically hidden (#9010) 2020-02-20 14:43:33 -03:00
Penar Musaraj 7a09e2cce2 DEV: Improve video onebox stripping spec
Followup to 70819080
2020-02-20 11:45:12 -05:00
Martin Brennan b0f4149d6e Suppres task spec output using capture_stdout 2020-02-20 14:47:47 +10:00
Martin Brennan 254b57c812
FIX: When admin changes staff email still enforce old email confirm (#9007)
A follow-up correction to this change https://github.com/discourse/discourse/pull/9001.

When admin changes staff email still enforce old email confirm. Only allow auto-confirm of a new email by admin IF the target user is not also an admin. If an admin gets locked out of their email the site admin can use the rails console to solve the issue in a pinch.
2020-02-20 13:42:57 +10:00
Martin Brennan 97d8f19387
FIX: When admin changes another user's email auto-confirm the change (#9001)
When admin changes a user's email from the preferences page of that user:

* The user will not be sent an email to confirm that their
  email is changing. They will be sent a reset password email
  so they can set the password for their account at the new
  email address.
* The user will still be sent an email to their old email to inform
  them that it was changed.
* Admin and staff users still need to follow the same old + new
  confirm process, as do users changing their own email.
2020-02-20 09:52:21 +10:00
Robin Ward 345764565f FIX: Respect muted tags for mailing list mode
If a user has a tag muted, don't send them emails about that tag.
We've done this forever for categories so it makes sense to do it
for tags too.
2020-02-19 15:14:42 -05:00
Robin Ward fe6fe324c9 FIX: Minor linting issue for future rubocops 2020-02-19 14:04:56 -05:00
Robin Ward c954d083df Link website when reviewing users 2020-02-19 10:18:05 -05:00
Robin Ward 53054b41d2 FIX: Mentions updater should work regardless of `.notify` 2020-02-18 16:02:26 -05:00
Robin Ward 041168c9b6 FIX: Group mentions were not being cooked the same was as previewed
If a group mention could be notified on preview it was given an `<a>`
tag with the `.notify` class. When cooked it would display differently.
This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
Roman Rizzi 9441362c72
FEATURE: Support uploading a csv with either user emails or usernames (#8971) 2020-02-18 10:53:12 -03:00
Martin Brennan 5122826bde Rubocop lint 2020-02-18 15:13:34 +10:00
Martin Brennan 500185dc11 Try fix upload_spec flakys and remove logging from tasks/uploads_spec 2020-02-18 15:08:58 +10:00
romanrizzi 17aed8d84b DEV: Add a test to make sure that flagged replies are ignored when calling the delete_and_agree_replies method 2020-02-17 13:31:01 -03:00
romanrizzi 330d1ae9bc DEV: Correct test names 2020-02-17 11:47:18 -03:00
Martin Brennan e8efdd60d4
FIX: Tweak upload security emoji check (#8981)
Further on from my earlier PR #8973 also reject upload as secure if its origin URL contains images/emoji. We still check Emoji.all first to try and be canonical.

This may be a little heavy handed (e.g. if an external URL followed this same path it would be a false positive), but there are a lot of emoji aliases where the actual Emoji url is something, but you can have another image that should not be secure that that thing is an alias for. For example slight_smile.png does not show up in Emoji.all BUT slightly_smiling_face does, and it aliases slight_smile e.g. /images/emoji/twitter/slight_smile.png?v=9 and /images/emoji/twitter/slightly_smiling_face.png?v=9 are equivalent.
2020-02-17 15:11:15 +10:00
Martin Brennan 9dcc454a07
FIX: Improvements and fixes for update_upload_acl rake task (#8980)
The rake task was broken, because the addition of the
UploadSecurity check returned true/false instead of the
upload ID to determine which uploads to set secure.
Also it was rebaking the posts in the wrong place and
pretty inefficiently at that. Also it was rebaking before
the upload was being changed to secure in the DB.
This also updates the task to set the access_control_post_id
for all uploads. the first post the upload is linked to is used
for the access control. if the upload doesn't get changed to
secure this doesn't affect anything.
Added a spec for the rake task to cover common cases.
2020-02-17 14:21:43 +10:00
Martin Brennan dac923379a
FIX: Never mark uploads based on regular emoji secure (#8973)
Sometimes PullHotlinkedImages pulls down a site emoji and creates a new upload record for it. In the cases where these happen the upload is not created via the normal path that custom emoji follows, so we need to check in UploadSecurity whether the origin of the upload is based on a regular site emoji. If it is we never want to mark it as secure (we don't want emoji not accessible from other posts because of secure media).

This only became apparent because the uploads:ensure_correct_acl rake task uses UploadSecurity to check whether an upload should be secure, which would have marked a whole bunch of regular-old-emojis as secure.
2020-02-17 12:30:47 +10:00
Robin Ward d51107e2c9 FIX: If a group is unmentionable, don't render it as mentionable
Now if a group is visible but unmentionable, users can search for it
when composing by typing with `@`, but it will be rendered without the
grey background color.

It will also no longer pop up a JIT warning saying "You are about to
mention X people" because the group will not be mentioned.
2020-02-14 12:29:56 -05:00
Neil Lalonde c31039d51f FIX: tag topic counts wrong after adding synonyms
After adding a tag as a synonym of another tag,
both tags will have the wrong topic counts. It's
corrected within 12 hours by the EnsureDbConsistency
job. This fix ensures the topic counts are updated
much sooner.
2020-02-14 12:15:29 -05:00
Roman Rizzi fadb2b7157
FEATURE: Users cannot reflag recently handled items using the same reason unless the post was edited, or it was reviewed more than 24 hours ago. (#8969) 2020-02-14 13:43:48 -03:00
Robin Ward dafa354d3d FIX: Confirm `draft_key` is present on GET
Also adds a test for the `show` action which did not exist
2020-02-14 11:06:12 -05:00
Robin Ward bfdd42c53a FIX: Return a 404 when the `draft_key` is missing
Previously if `draft_key` was missing you'd get a 500 error in the logs.
2020-02-14 10:48:56 -05:00
Régis Hanol aad46a1aba DEV: remove unused 'composer/parse_html' route 2020-02-14 16:15:34 +01:00
Dan Ungureanu 9aeb9f0ca1
DEV: Add test (#8960)
Follow-up to 3ad07aacfa.
2020-02-14 10:19:40 +02:00
Krzysztof Kotlarek e90f9e5cc4
FIX: when unread reply notification exists don't create new (#8921)
* FIX: when unread reply notification exists don't create new

From time to time, the user is creating a reply post and then they want to add additional details. They edit an existing post and for example, add a quote from a previous one.

In that situation, if the user to whom reply was directed to already have the unread notification, we should not create the new one.

That behaviour was mentioned here: https://meta.discourse.org/t/reply-then-edit-to-add-quote-notification-redundancy/138358

* FIX: dont create new notification if already exists
2020-02-14 16:41:42 +11:00
Martin Brennan 56b16bc68e
FIX: Never allow custom emoji to be marked secure (#8965)
* Because custom emoji count as post "uploads" we were
marking them as secure when updating the secure status for post uploads.
* We were also giving them an access control post id, which meant
broken image previews from 403 errors in the admin custom emoji list.
* We now check if an upload is used as a custom emoji and do not
assign the access control post + never mark as secure.
2020-02-14 11:17:09 +10:00
Robin Ward c2e58b6b85 FIX: Don't remove the topic image if posts don't have them 2020-02-13 14:00:30 -05:00
Martin Brennan e1e74abd4f
FEATURE: Improving bookmarks part 2 -- Topic Bookmarking (#8954)
### UI Changes

If `SiteSetting.enable_bookmarks_with_reminders` is enabled:

* Clicking "Bookmark" on a topic will create a new Bookmark record instead of a post + user action
* Clicking "Clear Bookmarks" on a topic will delete all the new Bookmark records on a topic
* The topic bookmark buttons control the post bookmark flags correctly and vice-versa
Disabled selecting the "reminder type" for bookmarks in the UI because the backend functionality is not done yet (of sending users notifications etc.)

### Other Changes

* Added delete bookmark route (but no UI yet)
* Added a rake task to sync the old PostAction bookmarks to the new Bookmark table, which can be run as many times as we want for a site (it will not create duplicates).
2020-02-13 16:26:02 +10:00
Blake Erickson 965ac3567b FIX: Handle SSO Provider Parse exception
Prevent unnecessary 500 errors from appearing in the logs and return a
422 response instead.
2020-02-12 16:08:04 -07:00
Robin Ward 726d97b29d FIX: Linking to a category via hashtag had a broken URL. 2020-02-12 14:23:09 -05:00
Gerhard Schlager 453bec9394 FEATURE: Add small action post to indicate forwarded email
This happens only when the sender of the email didn't write anything in their email.
2020-02-12 16:23:17 +01:00
Dan Ungureanu ec40242b5c
FIX: Make inline oneboxes work with secured topics in secured contexts (#8895) 2020-02-12 12:11:28 +02:00
Dan Ungureanu 3f50481188
Improvements to group mentions (#8927)
* FIX: Avoid highlight mention to groups that are not public
* UX: Composer autocomplete will suggest all visible group names
2020-02-12 10:11:10 +02:00
Kris e69ffa86b3 Fix test another way 2020-02-11 17:07:18 -05:00
Kris 5e6f0d9f1c Fix test 2020-02-11 16:09:41 -05:00
Mark VanLandingham 3e89774908
DEV: Use .hbr for raw template file extension (#8883) 2020-02-11 13:38:12 -06:00
Roman Rizzi 3413ec0a5c
FEATURE: Pending queued posts are included even if they don't pass the minimum priority threshold (#8925) 2020-02-11 15:29:22 -03:00
David Taylor e32833cf1a
DEV: Make badge test resilient to disabled badges
This can happen when plugins seed disabled badges in the database
2020-02-11 18:01:33 +00:00
Joffrey JAFFEUX 6a7e82c44c
DEV: Apply rubocop (#8926) 2020-02-11 16:21:03 +00:00
Jay Pfaffman d294e13225
add postmark webhook handling (#8919) 2020-02-11 10:09:07 -05:00
Dan Ungureanu ecaf2c2f4e
FIX: Make category slug validation less strict (#8915)
This was changed recently and caused issues saving old categories which
already had digits at the beginning of the slug (for example, '30-days').
2020-02-11 17:01:12 +02:00
Jeff Wong 1a1bb7a2c9
FEATURE: Add logging when claiming and unclaiming reviewable flagged posts (#8920) 2020-02-10 15:40:01 -08:00
Jarek Radosz 6cfd16656f
FIX: Ignore group mentions inside quotes (#8905)
Also includes:
* DEV: Reuse found elements
2020-02-10 18:31:42 +01:00
Dan Ungureanu 478c095e5c DEV: Set commit.gpgsign=false for test Git repos (#8891) 2020-02-10 17:34:00 +11:00
Blake Erickson bd49d4af1a FIX: Flair icon being removed when updating other profile info
The commit: 75069ff179

allows users to remove their primary group, but this introduced a bug
where if you were to edit any other profile info like location or
website which is a form on a separate page then the flair dropdown,
would cause the selected flair to be removed.

This fix ensures that if the `primary_group_id` parameter is missing
from the update payload it does not remove the existing
`primary_group_id`. It will only remove the `primary_group_id` if it is
present in the payload and empty.
2020-02-07 16:26:33 -07:00
David Taylor 5919618a87
DEV: Drop legacy OpenID 2.0 support (#8894)
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.

Context at https://meta.discourse.org/t/-/113249
2020-02-07 17:32:35 +00:00
Penar Musaraj 99fd65328c FIX: Skip absolutizing URLs when source URI is invalid 2020-02-07 10:54:24 -05:00
Vinoth Kannan a0e0b1ef4b FIX: reset `edit_reason` in posts when creating a new version. 2020-02-07 09:40:16 +05:30
Penar Musaraj 49843f327e
FIX: Strip audio/video content from excerpt (#8881) 2020-02-06 15:08:13 -05:00
Joffrey JAFFEUX 20944e69e4
FEATURE: adds trust_level_growth report (#8878) 2020-02-06 19:44:30 +01:00
Penar Musaraj 4b6a47be48 DEV: do not persist force_custom_user_agent_hosts setting
Followup to f029e2
2020-02-06 11:56:54 -05:00
Penar Musaraj 0fd39cc511 FIX: Remove post/topic image_url on post edits
- resets image_url when image is removed from first post on edit
- excludes onebox icons from being featured as topic/post images
2020-02-06 11:23:08 -05:00
Penar Musaraj f029e2eaf6 FEATURE: Add site setting for specific hosts using custom user agent when oneboxing
Followup to #00c406
2020-02-06 10:32:42 -05:00
Daniel Waterworth 1183f8df52 DEV: Made test deterministic and re-enabled
Follow-up to 6b0cc9e2
2020-02-06 14:58:37 +00:00
Blake Erickson 9207c370d9 FIX: Spoiler logic should live inside of spoiler plugin
This commit removes logic about spoilers because it should live inside
of the discourse-spoiler-alert plugin.

This PR:

https://github.com/discourse/discourse-spoiler-alert/pull/38

also completely removes spoilers from excerpts in order to keep them
from leaking in topic previews and notifications.
2020-02-06 07:46:46 -07:00
Dan Ungureanu c5e3faac00
FIX: Recovered posts with no user will be taken over by system user (#8834) 2020-02-06 10:19:04 +02:00
Martin Brennan 697d4720df
DEV: Add DEBUG_PRELOADED_APP_DATA to log pre-loaded JSON in development (#8873)
Extracted from #8772

This will allow developers (in rails development mode only) to log pre-loaded JSON app data to the browser console for inspection.
2020-02-06 13:14:33 +10:00
Neil Lalonde 542e62ccf0 FIX: tag info misleading message saying it's not restricted
When a tag is restricted to a secured category that the user can't see,
the message was saying that it wasn't restricted to any categories.
Now it will say it's restricted to categories you can't access.
2020-02-05 15:23:39 -05:00
David Taylor 88779d849f
UX: Ignore name parameter from IDP when it is equal to email (#8869)
Some auth providers (e.g. Auth0 with default configuration) send the email address in the name field. In Discourse, the name field is made public, so this commit adds a safeguard to prevent emails being made public.
2020-02-05 16:03:18 +00:00
Jarek Radosz 53529a3427
DEV: Upgrade Ember to version 3.12.2 (#8753)
* DEV: Use Ember 3.12.2
* Add Ember version to ThemeField's DEPENDENT_CONSTANTS
* DEV: Use `id` instead of `elementId` (See: https://github.com/emberjs/ember.js/issues/18147)
* FIX: Don't leak event listeners (bug introduced in 999e2ff)
2020-02-05 14:51:00 +01:00
Régis Hanol 13d4b05963 FIX: allow both meta_data and custom_fields when creating a topic
For some reasons, we have two ways of associating "custom fields" to a new topic:
using 'meta_data' and 'custom_fields'.

However, if we were to provide both arguments, the 'meta_data' would be overwritten
by any 'custom_fields' provided.

This commit ensures we can use both and merges the 'custom_fields' with the 'meta_data'.
2020-02-05 10:40:22 +01:00
Joffrey JAFFEUX f0fe2ba9ac
UX: introduces icon-picker component for badges (#8844) 2020-02-05 00:41:10 +01:00
Blake Erickson 241d8f6452 FIX: Edit title respects min trust to edit post
This fix ensures that the site setting `post_edit_time_limit` does not
bypass the limit of the site setting `min_trust_to_edit_post`. This
prevents a bug where users that did not meet the minimum trust level to
edit could edit the title of topics.
2020-02-04 16:31:16 -07:00
Rafael dos Santos Silva f88fa99b0b
DEV: Fix web manifest short_title tests 2020-02-04 14:46:33 -03:00
Gerhard Schlager e84d88ddea FIX: Email attachments with a size of 0 bytes caused error 2020-02-04 12:30:43 +01:00
Robin Ward 14ae574bc5 DEV: Improve spec to use the constant length 2020-02-03 15:15:02 -05:00
Robin Ward ee17138c0f FIX: Better error messages when name is too long
Previously you'd get a server side generic error due to a password check
failing. Now the input element has a maxlength attribute and the server
side will respond with a nicer error message if the value is too long.
2020-02-03 14:14:32 -05:00
David Taylor 31775c996c DEV: Remove redundant category list spec
The behavior has been deliberately changed in 7640914552
2020-02-03 19:01:24 +00:00
Robin Ward 37888d9818 FIX: Never return the same reply more than once via reply_ids
If our reply tree somehow ends up with cycles or other odd
structures, we only want to consider a reply once, at the first
level in the tree that it appears.
2020-02-03 13:41:18 -05:00
David Taylor 7640914552
UX: Include muted categories on the category page by default (#8842)
* DEV: Add data-notification-level attribute to category UI

* Show muted categories on the category page by default

This reverts commit ed9c21e42c.

* Remove redundant spec - muted categories are now visible by default
2020-02-03 10:40:02 -08:00
Robin Ward f83362b05b FIX: Don't return post replies from other topics
It seems in some situations replies have been moved to other topics but
the `PostReply` table has not been updated. I will try and fix this in a
follow up PR, but for now this fix ensures that every time we ask a post
for its replies that we restrict it to the same topic.
2020-02-03 13:12:27 -05:00
Matt Marjanović ad2aa7b52c
FEATURE: Add logout functionality to SSO Provider protocol (#8816)
This commit adds support for an optional "logout" parameter in the
payload of the /session/sso_provider endpoint.  If an SSO Consumer
adds a "logout=true" parameter to the encoded/signed "sso" payload,
then Discourse will treat the request as a logout request instead
of an authentication request.  The logout flow works something like
this:

 * User requests logout at SSO-Consumer site (e.g., clicks "Log me out!"
   on web browser).
 * SSO-Consumer site does whatever it does to destroy User's session on
   the SSO-Consumer site.
 * SSO-Consumer then redirects browser to the Discourse sso_provider
   endpoint, with a signed request bearing "logout=true" in addition
   to the usual nonce and the "return_sso_url".
 * Discourse destroys User's discourse session and redirects browser back
   to the "return_sso_url".
 * SSO-Consumer site does whatever it does --- notably, it cannot request
   SSO credentials from Discourse without the User being prompted to login
   again.
2020-02-03 12:53:14 -05:00
Martin Brennan baca59c1a6 Merge branch 'master' of github.com:discourse/discourse 2020-02-03 15:13:33 +10:00
Martin Brennan 89cb8f769a Remove redundant spec in TopicConverter
* the spec to check if changing the topic between PM and
  public to see if the upload security status is changed
  is already covered extensively in topic_upload_security_manager_spec.rb
2020-02-03 15:08:11 +10:00
Martin Brennan dd3a7f4825
FIX: Admin user list not showing 2FA icon for only security keys enabled (#8839)
If someone only had security keys enabled, the icon to say they had 2FA enabled would not show in the admin staff user list. It would only show if they had TOTP enabled.
2020-02-03 14:37:46 +10:00
Krzysztof Kotlarek 5b03f35614
FIX: suppress notification flood when post is edited (#8838) 2020-02-03 11:27:18 +11:00
Jarek Radosz 73b04976e5
FIX: Use updated_at in the S3 inventory job (#8823)
When we change upload's sha1 (e.g. when resizing images) it won't match the data in the most recent S3 inventory index. With this change the uploads that have been updated since the inventory has been generated are ignored.
2020-01-31 11:02:44 +01:00
Martin Brennan 8d77e99827
FIX: Stop encoding presigned URLs with UrlHelper (#8818)
When FinalDestination is given a URL it encodes it before doing anything else. however S3 presigned URLs should not be messed with in any way otherwise we can end up with 400 errors when downloading the URL e.g.

<Error><Code>InvalidToken</Code><Message>The provided token is malformed or otherwise invalid.</Message>

The signature of presigned URLs is very important and is automatically generated and should be preserved.
2020-01-31 09:09:34 +10:00
Dan Ungureanu 62b9a432bd
FIX: Import sub-sub-categories (#8810)
This should make the importer more resilient to incomplete or damaged
backups. It will disable some validations and attempt to automatically
repair category permissions before importing.
2020-01-30 18:46:33 +02:00
Mark VanLandingham 089d38e9d3
FIX: Disallow featuring hidden topics (#8814) 2020-01-30 10:00:49 -06:00
Dan Ungureanu bf50087d72
Revert "FIX: Create post notices only for public posts (#8708)"
This reverts commit c2d051315d.
2020-01-30 14:26:08 +02:00
Arpit Jalan 62c21ba649 Remove `bounce_score_threshold_deactivate` setting.
Removed `bounce_score_threshold_deactivate` setting as the deactivate
threshold is not getting reached.
2020-01-30 16:17:31 +05:30
Martin Brennan 1150cd4621
FIX: Stop secure media URLs being censored too liberally in emails (#8817)
For example /t/ URLs were being replaced if they contained secure-media-uploads so if you made a topic called "Secure Media Uploads Are Cool" the View Topic link in the user notifications would be stripped out.

Refactored code so this secure URL detection happens in one place.
2020-01-30 16:19:14 +10:00
Martin Brennan bd04b9cbbe Fix extremely slow topic_converter_spec 2020-01-30 10:43:05 +10:00
Neil Lalonde 34f564acd6 FIX: customized email subjects was ignored for some notifications
Email templates for notifications about new posts and topics weren't
using customized values.
2020-01-29 14:49:08 -05:00
Roman Rizzi 2ee6a615b7
FEATURE: Send suspect users to the review queue (#8811) 2020-01-29 15:38:27 -03:00
Dan Ungureanu 09e8be3209
UX: Introduce automatic 'categories topics' setting (#8804)
When 'categories topics' setting is set to 0, the system will
automatically try to find a value to keep the two columns (categories
and topics) symmetrical.

The value is computed as 1.5x the number of top level categories and at
least 5 topics will always be returned.
2020-01-29 20:30:48 +02:00
Mark VanLandingham e29fef9e99
FEATURE: Users can feature any public topic on his/her profile (#8809) 2020-01-29 10:10:23 -06:00
Sam Saffron 7f3a30d79f FIX: blank cooked markdown could raise an exception in logs
Previously if somehow a user created a blank markdown document using tag
tricks (eg `<p></p><p></p><p></p><p></p><p></p><p></p>`) and so on, we would
completely strip the document down to blank on post process due to onebox
hack.

Needs a followup cause I am still unclear about the reason for empty p stripping
and it can cause some unclear cases when we re-cook posts.
2020-01-29 11:37:25 +11:00
Martin Brennan ab3bda6cd0
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802)
Basically, say you had already downloaded a certain image from a certain URL
using pull_hotlinked_images and the onebox. The upload would be stored
by its sha as an upload record. Whenever you linked to the same URL again
in a post (e.g. in our case an og:image on review.discourse) we would
would reuse the original upload record because of the sha1.

However when you turned on secure media this could cause problems as
the first post that uses that upload after secure media is enabled
will set the access control post for the upload to the new post.
Then if the post is deleted every single onebox/link to that same image
URL will fail forever with 403 as the secure-media-uploads URL fails
if the access control post has been deleted.

To fix this when cooking posts and pulling hotlinked images, we only
allow using an original upload by URL if its access control post
matches the current post, and if the original_sha1 is filled in,
meaning it was uploaded AFTER secure media was enabled. otherwise
we just redownload the media again to be safe, as the URL will always
be new then.
2020-01-29 10:11:38 +10:00
Krzysztof Kotlarek 20e7fb1c95
FIX: correct notification when tag or category is added (#8801)
Regression was created here:
https://github.com/discourse/discourse/pull/8750

When tag or category is added and the user is watching that category/tag
we changed notification type to `edited` instead of `new post`.

However, the logic here should be a little bit more sophisticated.

If the user has already seen the post, notification should be `edited`.

However, when user hasn't yet seen post, notification should be "new
reply". The case for that is when for example topic is under private
category and set for publishing later. In that case, we modify an
existing topic, however, for a user, it is like a new post.

Discussion on meta:
https://meta.discourse.org/t/publication-of-timed-topics-dont-trigger-new-topic-notifications/139335/13
2020-01-29 11:03:47 +11:00
Gerhard Schlager 9621af2214 FIX: Award "User of the month" badge at the beginning of month
Previously the badge was granted one month after the last time the badge was granted. The exact date shifted by one day each month. The new logic tries to grant the badge always at the beginning of a new month by looking at new users of the previous month. The "granted at" date is set to the end of the previous month.
2020-01-28 22:59:13 +01:00
David Taylor a9d0d55817 FEATURE: Add message to log when admins are automatically deactivated 2020-01-28 12:16:24 +00:00
David Taylor 25fd2b544a
PERF: Use a separate route for user cards, and split user serializer (#8789)
Adds a new route `/u/{username}/card.json`, which has a reduced number of fields. This change is behind a hidden site setting, so we can test compatibility before rolling out.
2020-01-28 11:55:46 +00:00
David Taylor c344f43211 UX: Admins should only see their own PMs when searching in:all
Admins are technically allowed to access all PMs, but it can be confusing to include them all in search. Follow-up to e0605029dc
2020-01-28 11:26:42 +00:00
Dan Ungureanu 5060811464
FIX: Export all category names in user archives (#8790) 2020-01-28 12:52:07 +02:00
Dan Ungureanu 6e8a1ac09a
DEV: Add test for categories_breadcrumb (#8791) 2020-01-28 12:50:27 +02:00
adam j hartz e0605029dc FEATURE: allow searching public topics and personal messages simultaneously (#8784)
The new search modifier `in:all` can be used to include both public and personal messages in the same search.

Co-authored-by: adam j hartz <hz@mit.edu>
2020-01-28 10:11:33 +00:00
Mark VanLandingham 20e3c0f386
FIX: Post reviser picking up edits for hidden posts (#8792) 2020-01-27 10:05:48 -06:00
Vinoth Kannan b6765aac4b FIX: add 'noindex' header to rss feed responses. 2020-01-24 09:30:27 +05:30
Martin Brennan 45b37a8bd1
FIX: Resolve pull hotlinked image and broken link issues for secure media URLs (#8777)
When pull_hotlinked_images tried to run on posts with secure media (which had already been downloaded from external sources) we were getting a 404 when trying to download the image because the secure endpoint doesn't allow anon downloads.

Also, we were getting into an infinite loop of pull_hotlinked_images because the job didn't consider the secure media URLs as "downloaded" already so it kept trying to download them over and over.

In this PR I have also refactored secure-media-upload URL checks and mutations into single source of truth in Upload, adding a SECURE_MEDIA_ROUTE constant to check URLs against too.
2020-01-24 11:59:30 +10:00
Krzysztof Kotlarek 8cc09fc668
Revert "FIX: Redis fallback handler refactoring (#8771)" (#8776)
This reverts commit 4f677854d3.
2020-01-24 09:20:17 +11:00
Roman Rizzi 9eb622985a
FEATURE: Replace existing badge owners when using the bulk award feature (#8770)
* FEATURE: Replace existing badge owners when using the bulk award feature

* Use ActiveRecord to sanitize title update query, Change replace checkbox text

Co-Authored-By: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-01-23 14:14:58 -03:00
Régis Hanol 821e920711 DEV: fix deprecation warnings in specs
Also fixed a typo in a string key "max_pm_recepients" -> "max_pm_recipients"
2020-01-23 16:37:48 +01:00
Krzysztof Kotlarek 4f677854d3
FIX: Redis fallback handler refactoring (#8771)
* DEV: Add a fake Mutex that for concurrency testing with Fibers

* DEV: Support running in sleep order in concurrency tests

* FIX: A separate FallbackHandler should be used for each redis pair

This commit refactors the FallbackHandler and Connector:

 * There were two different ways to determine whether the redis master
   was up. There is now one way and it is the responsibility of the
   new RedisStatus class.

 * A background thread would be created whenever `verify_master` was
   called unless the thread already existed. The thread would
   periodically check the status of the redis master. However, checking
   that a thread is `alive?` is an ineffective way of determining
   whether it will continue to check the redis master in the future
   since the thread may be in the process of winding down.

   Now, this thread is created when the recorded master status goes from
   up to down. Since this thread runs the only part of the code that is
   able to bring the recorded status up again, we ensure that only one
   thread is probing the redis master at a time and that there is always
   a thread probing redis master when it is recorded as being down.

 * Each time the status of the redis master was checked periodically, it
   would spawn a new thread and immediately join on it. I assume this
   happened to isolate the check from the current execution, but since
   the join rethrows exceptions in the parent thread, this was not
   effective.

 * The logic for falling back was spread over the FallbackHandler and
   the Connector. The connector is now a dumb object that delegates
   responsibility for determining the status of redis to the
   FallbackHandler.

 * Previously, failing to connect to a master redis instance when it was
   not recorded as down would raise an exception. Now, this exception is
   passed to `Discourse.warn_exception` and the connection is made to
   the slave.

This commit introduces the FallbackHandlers singleton:

 * It is responsible for holding the set of FallbackHandlers.

 * It adds callbacks to the fallback handlers for when a redis master
   comes up or goes down. Main redis and message bus redis may exist on
   different or the same redis hosts and so these callbacks may all
   exist on the same FallbackHandler or on separate ones.

These objects are tested using fake concurrency provided by the
Concurrency module:

 * An `around(:each)` hook is used to cause each test to run inside a
   Scenario so that the test body, mocking cleanup and `after(:each)`
   callbacks are run in a different Fiber.

 * Therefore, holting the execution of the Execution abruptly (so that
   the fibers aren't run to completion), prevents the mocking cleaning
   and `after(:each)` callbacks from running. I have tried to prevent
   this by recovering from all exceptions during an Execution.

* FIX: Create frozen copies of passed in config where possible

* FIX: extract start_reset method and remove method used by tests

Co-authored-by: Daniel Waterworth <me@danielwaterworth.com>
2020-01-23 13:39:29 +11:00
Martin Brennan 1b3b0708c0
FEATURE: Update upload security status on post move, topic conversion, category change (#8731)
Add TopicUploadSecurityManager to handle post moves. When a post moves around or a topic changes between categories and public/private message status the uploads connected to posts in the topic need to have their secure status updated, depending on the security context the topic now lives in.
2020-01-23 12:01:10 +10:00
Blake Erickson e85f4f6cc8 DEV: Add a test for ensure only edited badge titles updates a users
title

Adding a test to make sure we don't regress here in the future.

Follow up to: 8a89b7e108
2020-01-22 16:42:45 -07:00
Martin Brennan 4646a38ae6
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764)
When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 09:31:46 +10:00
Mark VanLandingham c5eec19368
FIX: Featuring topic on other users profile shows their topics (#8769) 2020-01-22 14:16:17 -06:00
mintsaxon 4b54791bcc FIX: Notify staged users about private categories (#8765)
group membership and `CategoryUser` notification level should be
respected to determine whether to notify staged users about activity in
private categories, instead of only ever generating notifications for staged
users' own topics (which has been the behaviour since
0c4ac2a7bc)
2020-01-22 16:33:25 -03:00
Bianca Nenciu 7b7e1717f2
FIX: Quoting a quote preserves the original post information (#8746)
Let's say post #2 quotes post number #1. If a user decides to quote the
quote in post #2, it should keep the information of post #1
("user_1, post: 1, topic: X"), instead of replacing with current post
info ("user_2, post: 2, topic: X").
2020-01-22 16:10:23 +02:00