Commit Graph

7310 Commits

Author SHA1 Message Date
Krzysztof Kotlarek a3e1e9ef16
FIX: remove deprecated URI.escape (#9697)
During Nokogumbo changes I introduced back URI.escape which is deprecated.
2020-05-08 11:14:59 +10:00
Martin Brennan 22dffe6f69 FIX: Do not allow null options for bookmark manager 2020-05-08 15:24:59 +00:00
Robin Ward f9608c0af5 DEV: Remove INLINE_ONEBOX_* constants
There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and
`INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they
were DRYing up: `inline-onebox-loading` and `inline-onebox`

I normally appreciate constants, but in this case it meant that we had
a lot of JS imports resulting in many more lines of code (and CPU cycles
spent figuring them out.)

It also meant we had an `.erb` file and had to invoke Ruby to create the
JS file, which meant the app was harder to port to Ember CLI.

I removed the constants. It's less DRY but faster and simpler, and
arguably the loss of DRYness is not significant as you can still search
for the `inline-onebox-loading` and `inline-onebox` strings easily if
you are refactoring.
2020-05-07 16:14:38 -04:00
Vinoth Kannan 744bbf6904 FEATURE: exclude muted categories from the "top" topics list. 2020-05-08 00:34:53 +05:30
Joffrey JAFFEUX 23d585f255
REFACTOR: removes unreachable statement (#9680) 2020-05-07 16:37:02 +02:00
Martin Brennan 6fb0f36ce1
FEATURE: Optionally delete bookmark when reminder sent (#9637)
We now show an options gear icon next to the bookmark name.

When expanded we show the "delete bookmark when reminder sent" option. The value of this checkbox is saved in local storage for the user.

If this is ticked, when a reminder is sent for the bookmark the bookmark itself is deleted. This is so people can use the reminder functionality by itself.

Also remove the blue alert reminder section from the "Edit Bookmark" modal as it just added clutter, because the user can already see they had a reminder set:

Adds a default false boolean column `delete_when_reminder_sent` to bookmarks.
2020-05-07 13:37:39 +10:00
tshenry 0d394e6b76
FEATURE: Include category position when exporting categories (#9658) 2020-05-07 12:17:15 +10:00
Gerhard Schlager ec2f3169ff FIX: Using the `default_locale` in locale fallbacks caused problems
Locale files get precompiled after deployment and they contained translations from the `default_locale`. That's especially bad in multisites, because the initial `default_locale` is `en_US`. Sites where the `default_locale` isn't `en_US` could see missing translations. The same thing could happen when users are allowed to chose a different locale.

This change simplifies the logic by not using the `default_locale` in the locale chain. It always falls back to `en` in case of missing translations.
2020-05-06 22:59:07 +02:00
Robin Ward 01929e3505 DEV: Move `preload-store` to `discourse/lib/preload-store`
It's only used inside Discourse so it needn't be its own module
2020-05-06 15:28:06 -04:00
David Taylor 72ad701df0
DEV: Stub #flush in StdOutDemux for multisite:migrate
https://meta.discourse.org/t/multisite-migrate-error/150579/2
2020-05-06 11:58:35 +01:00
Martin Brennan fa572d3a7a
DEV: Remove all code referencing at_desktop bookmark reminders (#9650)
We have found no need for these reminder types, so we are removing the code for them.
2020-05-06 15:22:43 +10:00
Robin Ward 612284cef3
DEV: Remove `Discourse.RAW_TEMPLATES` (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor 03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek 9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
Rafael dos Santos Silva d59d170452
FIX: PostgreSQL fallback was broken due to Rails masking exception (#9633)
The PR https://github.com/rails/rails/pull/36612 changes the raised exception
if the error message includes the target database name.

Since the error message contains the hostname, this could be triggered when
the hostname contains the database name.
2020-05-05 10:34:25 +10:00
Penar Musaraj 5706cab897
FIX: Improve digest email styling on Outlook 2016 (#9626) 2020-05-04 14:07:03 -04:00
Neil Lalonde c6b31464db
Version bump to v2.5.0.beta4 2020-05-04 11:44:42 -04:00
David Taylor 5901717531
DEV: Allow DB.after_commit to be used outside of a transaction
In this case, it will execute the given block immediately
2020-05-04 09:42:41 +01:00
Robin Ward e57fd283db DEV: Rename `deprecated` to the more appropriate app-boot 2020-05-01 15:19:19 -04:00
Martin Brennan bcc9ad6f57
FIX: Bookmark UI tweaks (#9604)
* When hovering over the bookmark icon for a post, show the name of the bookmark at the end of the tooltip _if_ it has been set.
* Order bookmarks by `updated_at DESC` in the user list and show that instead of created at.
2020-05-01 16:14:20 +10:00
Vinoth Kannan 5ff24b6891 FIX: do not raise error if 'class' attribute is not found. 2020-05-01 10:03:40 +05:30
Krzysztof Kotlarek 37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Robin Ward d615de9139
DEV: Support for `import Handlebars from 'handlebars'`; (#9600)
* Remove Handlebars.SafeString usage

* DEV: Support for `import Handlebars from 'handlebars'`;

* FIX: Sprockets was broken when `node_modules` was present

By default the old version of sprockets looks for application.js
anywhere, including in a node_modules folder if this exists
(which it will when we move to Ember CLI.)
2020-04-30 16:41:02 -04:00
Rafael dos Santos Silva 08e4af6636 FEATURE: Add setting to controle the Expect header on S3 calls
Some providers don't implement the Expect: 100-continue support,
which results in a mismatch in the object signature.

With this settings, users can disable the header and use such providers.
2020-04-30 12:12:00 -03:00
Régis Hanol 501b19b6e0
FIX: server-side HtmlToMarkdown improvements (#9586)
TLDR; this commit vastly improves how whitespaces are handled when converting from HTML to Markdown.
It also adds support for converting HTML <tables> to markdown tables.

The previous 'remove_whitespaces!' method was traversing the whole HTML tree and used a heuristic to remove
leading and trailing whitespaces whenever it was appropriate (ie. mostly before and after HTML block elements)

It was a good idea, but it was very limited and leaded to bad conversion when the html had leading whitespaces on several lines for example.
One such example can be found [here](https://meta.discourse.org/t/86782).

For various reasons, most of the whitespaces in a HTML file is ignored when the page is being displayed in a browser.
The rules that the browsers follow are the [CSS' White Space Processing Rules](https://www.w3.org/TR/css-text-3/#white-space-rules).
They can be quite complicated when you take into account RTL languages and other various tidbits but they boils down to the following:

- Collapse whitespaces down to one space (0x20) inside an inline context (ie. nodes/tags that are being displaying on the same line)
- Remove any leading/trailing whitespaces inside an inline context

One quick & dirty way of getting this 90% solved would be to do 'HTML.gsub!(/[[:space:]]+/, " ")'.
We would also need to hoist <pre> elements in order to not mess with their whitespaces.
Unfortunately, this solution let some whitespaces creep around HTML tags which leads to more '.strip!' calls than I can bear.

I decided to "emulate" the browser's handling of whitespaces and came up with a solution in 4 parts

1. remove_not_allowed!

The HtmlToMarkdown library is recursively "visiting" all the nodes in the HTML in order to convert them to Markdown.
All the nodes that aren't handled by the library (eg. <script>, <style> or any non-textual HTML tags) are "swallowed".
In order to reduce the number of nodes visited, the method 'remove_not_allowed!' will automatically delete all the nodes
that have no "visitor" (eg. a 'visit_<tag>' method) defined.

2. remove_hidden!

Similar purpose as the previous method (eg. reducing number of nodes visited), there's no point trying to convert something that is hidden.
The 'remove_hidden!' method removes any nodes that was hidden using the "hidden" HTML attribute, some CSS or with a width or height equal to 0.

3. hoist_line_breaks!

The 'hoist_line_breaks!' method is there to handle <br> tags. I know those tiny <br> don't do much but they can be quite annoying.
The <br> tags are inline elements but they visually work like a block element (ie. they create a new line).
If you have the following HTML "<i>Foo<br>Bar</i>", it ends up visually similar to "<i>Foo</i><br><i>Bar</i>".
The latter being much more easy to process than the former, so that's what this method is doing.
The "hoist_line_breaks" will hoist <br> tags out of inline tags until their parent is a block element.

4. remove_whitespaces!

The "remove_whitespaces!" is where all the whitespace removal is happening. It's broken down into 4 methods as well

- remove_whitespaces!
- is_inline?
- collapse_spaces!
- remove_trailing_space!

The 'remove_whitespace!' method is recursively walking the HTML tree (skipping <pre> tags).
If a node has any children, they will be chunked into groups of inline elements vs block elements.
For each chunks of inline elements, it will call the "collapse_space!" and "remove_trailing_space!" methods.
For each chunks of block elements, it will call "remote_whitespace!" to keep walking the HTML tree recursively.

The "is_inline?" method determines whether a node is part of a inline context.
A node is inline iif it's a text node or it's an inline tag, but not <br>, and all its children are also inline.

The "collapse_spaces!" method will collapse any kind of (white) space into a single space (" ") character, even accros tags.
For example, if we have "  Foo \n<i> Bar </i>\t42", it will return "Foo <i>Bar </i>42".

Finally, the "remove_trailing_space!" method is there to remove any trailing space that might creep in at the end of the inline chunk.

This solution is not 100% bullet-proof.
It does not support RTL languages at all and has some caveats that I felt were not worth the work to get properly fixed.

FIX: better detection of hidden elements when converting HTML to Markdown
FIX: take into account the 'allowed_href_schemes' site setting when converting HTML <a> to Markdown
FIX: added support for 'mailto:' scheme when converting <a> from HTML to Markdown
FIX: added support for <img> dimensions when converting from HTML to Markdown
FIX: added support for <dl>, <dd> and <dt> when converting from HTML to Markdown
FIX: added support for multilines emphases, strongs and strikes when converting from HTML to Markdown
FIX: added support for <acronym> when converting from HTML to Markdown
DEV: remove unused 'sanitize' gem

Wow, did you just read all that?! Congratz, here's a cookie: 🍪.
2020-04-30 12:21:25 +02:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
Dan Ungureanu c85018cdfd
Improve support for old browsers (#9515)
* FEATURE: Improve crawler view

* FIX: Make lazyYT crawler-friendly

* DEV: Rename discourse-internet-explorer to discourse-unsupported-browser

* DEV: Detect more unsupported browsers

Follow-up to 4eebbd2212.

* FIX: Hide browser update notice in print view
2020-04-29 21:40:21 +03:00
Robin Ward 08fbf199ad FIX: S3 rake task can ignore yarn.lock 2020-04-29 13:14:04 -04:00
Robin Ward 9ec908950d DEV: Better error handling for s3 task 2020-04-29 12:54:39 -04:00
Robin Ward cbb27241c4
DEV: Make `discourse-common` an Ember addon. (#9578)
This is to help with the migration to Ember CLI. In the current running
version of Discourse everything should be the same as before, just with
a few extra files that are not used. However, using Ember CLI this can
be installed as an Ember addon.

Co-Authored-By: Jarek Radosz <jradosz@gmail.com>
2020-04-29 12:18:21 -04:00
Robin Ward 1e603d7003 DEV: This file does not seem to be used anymore 2020-04-28 14:03:39 -04:00
Jarek Radosz c1c211365a
FIX: Improve clearing store cache (#9568)
1. Shorter
2. Simpler
3. Doesn't depend on external binaries
4. Doesn't fail on large amounts of files
5. Hopefully eliminates flaky spec errors
2020-04-28 17:24:04 +02:00
Penar Musaraj ec2943c5bc
DEV: Update jquery.fileupload and dependencies (#9466) 2020-04-28 10:39:29 -04:00
David Taylor ba616ffb50
DEV: Use a tmp directory for storing uploads in tests (#9554)
This avoids development-mode upload files from polluting the test environment
2020-04-28 14:03:04 +01:00
Sam Saffron 6b62d75c50
FIX: remove unsuppored rake task
rebuilding user_actions is not something that should be done.

Plugins such as solved and assigned extend it, there are tons of
little rules that were not captured in `user_actions:rebuild`
2020-04-28 16:27:42 +10:00
Martin Brennan 5108cf8ddf
FIX: Topic user bookmarked column logic was not correct (#9563)
Make sure the topic_user.bookmarked column is set correctly when user bookmarks/unbookmarks any post in a topic. For example, you bookmarked a post in the topic that was not the OP, the bookmark icon in the topic list would not be shown. Same if deleting a bookmark for the last bookmarked post in a topic, the bookmark icon in the topic list would not be removed.

Previously this was only setting it to true if bookmarking the OP/topic, which was not correct -- we want to show the icon on the topic list if any post is bookmarked.
Also set to false if unbookmarking the last bookmarked post in the topic.
Also in this PR is a migration to correct any out of sync topic_user.bookmarked columns, based on the new logic.
2020-04-28 16:19:25 +10:00
mentalstring 67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Roman Rizzi 394babcae3
FIX: Only show the review page to users that can see it. Do not publish the reviewable count update message to everyone. (#9556) 2020-04-27 14:51:25 -03:00
Robin Ward 056327c0c9
DEV: Move `discourse.js` to `app/app.js` (#9545)
This is another thing to get our application in line with what Ember CLI
expects.
2020-04-27 13:28:10 -04:00
David Taylor f95609ae23
FIX: Add additional checks for automatic theme script CSP
- Skip absolute URLs
- Skip CDN URLs
- Skip invalid URLs
- Properly format protocol-less URLs
2020-04-27 15:56:29 +01:00
Benno 6e01acb3cb
FIX: Apply category priority for empty query (#9516) 2020-04-27 10:35:27 -04:00
Guo Xiang Tan 76fc48b318
DEV: Avoid an additional query in `SpamHandler`.
* Also run query against human users only
2020-04-27 13:03:57 +08:00
Sam Saffron 1f6eaf26a1
Revert "UX: replace closed topic icon with discourse-no-entry"
This reverts commit 040b8c00a4.

We decided to keep status quo for now
2020-04-25 13:12:56 +10:00
Kris 040b8c00a4 UX: replace closed topic icon with discourse-no-entry 2020-04-24 17:31:34 -04:00
David Taylor 8a112b7464
DEV: Automatically extend CSP when themes link to external scripts (#9531) 2020-04-24 09:47:01 +01:00
Martin Brennan cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Kris 9f52997be1 UX: Let's try X for the closed topic icon instead of a \ 2020-04-23 17:23:25 -04:00
Gerhard Schlager 67e96f6c54 DEV: discourse-backup-uploads-to-s3 plugin isn't supported anymore 2020-04-23 21:37:38 +02:00
Arpit Jalan 39be639c37 FIX: update GitHub screen_name on login via GitHub 2020-04-23 20:54:26 +05:30
Roman Rizzi 7a2e8d3ead
DEV: Add the missing `app` subdirectory (#9499)
* DEV: Add missing  directory to the Discourse ember app

* DEV: Resolve imports correctly
2020-04-23 10:07:54 -03:00
Krzysztof Kotlarek 52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Robin Ward 80a572d3b7 FIX: Multisite spec was failing in parallel environment
We were not adding the test number to the path in all places.
2020-04-22 14:05:39 -04:00
Rafael dos Santos Silva b810fc282d FEATURE: Digital Ocean Spaces compatibility for Object Storage
DO does not implement tagging support for S3 objects. Removing our default
empty tag fixes compatibility.

The expire_missing_assets rake task can't be used with that service still,
but this patch allows normal operation.
2020-04-22 13:57:14 -03:00
Kris bf0c055a9a UX: add comment-plus icon for reopening topics 2020-04-22 12:29:27 -04:00
Neil Lalonde 58bdc04aac
Version bump to v2.5.0.beta3 2020-04-22 10:51:19 -04:00
Vinoth Kannan e5c44f6dcf UX: display 'merge' button in all non-staff user profiles. 2020-04-22 17:42:09 +05:30
Vinoth Kannan a511bea4cc
FEATURE: admin UI to merge two users. (#9509) 2020-04-22 14:07:51 +05:30
Martin Brennan 628ba9d1e2
FEATURE: Promote bookmarks with reminders to core functionality (#9369)
The main thrust of this PR is to take all the conditional checks based on the `enable_bookmarks_with_reminders` away and only keep the code from the `true` path, making bookmarks with reminders the core bookmarks feature. There is also a migration to create `Bookmark` records out of `PostAction` bookmarks for a site.

### Summary

* Remove logic based on whether enable_bookmarks_with_reminders is true. This site setting is now obsolete, the old bookmark functionality is being removed. Retain the setting and set the value to `true` in a migration.
* Use the code from the rake task to create a database migration that creates bookmarks from post actions.
* Change the bookmark report to read from the new table.
* Get rid of old endpoints for bookmarks
* Link to the new bookmarks list from the user summary page
2020-04-22 13:44:19 +10:00
Martin Brennan e18aeb799e
FIX: Make sure reminder_type is parsed on bookmark update (#9503)
Otherwise we are trying to update the reminder type with a string which often evaluates to 0 (At Desktop) which causes reminders to come through early.
2020-04-22 10:44:04 +10:00
Kris b6489d2690 UX: Replaced locked topic "lock" icon with "comment-slash" icon 2020-04-21 17:34:59 -04:00
Jarek Radosz 17cf300b71
DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
Jarek Radosz ab52bed014
DEV: Remove the return value of disable_if_low_on_disk_space (#9469)
It was used only in specs.
2020-04-21 03:48:33 +02:00
Jarek Radosz 5a81e3999c
DEV: Remove `bypass_bump` from CookedPostProcessor (#9468)
It was only passing it along to `PullHotlinkedImages` and that class have not used that arg since April 2014 (c52ee665b4)
2020-04-21 03:48:19 +02:00
Robin Ward 25bed4f643 FIX: Concurrency issues with making topic embedded posts visible 2020-04-20 15:11:59 -04:00
Robin Ward e997a1f315 FIX: Don't make topics visible unless the posts are regular 2020-04-20 14:56:21 -04:00
Robin Ward 8f5314bf98 FIX: An `opts` hash was not, in fact, optional :) 2020-04-20 14:17:13 -04:00
Mark VanLandingham 0a3cc22cec
FIX: Wizard was creating duplicate Light theme if Light was selected (#9464) 2020-04-20 08:31:43 -05:00
Martin Brennan 344ef5226c
FEATURE: Edit bookmark reminders from post and explicit delete button (#9455)
There is now an explicit "Delete Bookmark" button in the edit modal. A confirmation is shown before deleting.

Along with this, when the bookmarked post icon is clicked the modal is now shown instead of just deleting the bookmark. Also, the "Delete Bookmark" button from the user bookmark list now confirms the action.

Add a `d d` shortcut in the modal to delete the bookmark.
2020-04-20 13:30:04 +10:00
Gerhard Schlager c6b411f6c1 FIX: Restore to S3 didn't work without env variables
The `uplaods:migrate_to_s3` rake task should always use the environment variables, because you usually don't want to break your site's uploads during the migration. But restoring a backup should work with site settings as well as environment variables, otherwise you can't restore uploads to S3 from the web interface.
2020-04-19 20:24:40 +02:00
Gerhard Schlager baae0e7446 FIX: Infinite loop in migrate_to_s3 rake task 2020-04-19 20:24:40 +02:00
tshenry e41f1e6ec2
FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
Martin Brennan 8f0544137a
FEATURE: Allow editing bookmark reminders (#9437)
Users can now edit the bookmark name and reminder time from their list of bookmarks.

We use "Custom" for the date and time in the modal because if the user set a reminder for "tomorrow" then edit the reminder "tomorrow", the definition of what "tomorrow" is has changed.
2020-04-17 11:08:07 +10:00
David Taylor ed6b3b82bd
FIX: Reopen sidekiq log files after rotation (#9429)
Unicorn uses the USR1 to indicate that log files should be reopened. This commit implements the same functionality for our forked sidekiq workers:

- USR1 is intercepted in the unicorn master, and re-issued to all child processes
- USR1 is trapped in the sidekiq processes, and `Unicorn::Util.reopen_logs` is used to re-open log files
2020-04-16 12:13:13 +01:00
Martin Brennan 51672b9121
FIX: Minor bookmark with reminder issue cleanup (#9436)
* Count user summary bookmarks from new Bookmark table if bookmarks with reminders enabled
* Update topic user bookmarked column when new topic bookmark changed
* Make in:bookmarks search work with new bookmarks
* Fix batch inserts for bookmark rake task (and thus migration). We were only inserting one bookmark at a time, completely defeating the purpose of batching!
2020-04-16 11:32:21 +10:00
Martin Brennan d7f744490a
FEATURE: Decorate topic-level bookmark button with reminder time (#9426)
* Show the correct bookmark with clock icon when topic-level bookmark reminder time is set and show the time of the reminder in the title on hover.
* Add a new bookmark lib and reminder time formatting function to show time with today/tomorrow shorthand for readability. E.g. tomorrow at 8:00am instead of Apr 16 2020 at 8:00am. This only applies to today + tomorrow, future dates are still treated the same.
2020-04-16 09:20:44 +10:00
Vinoth Kannan 884eea7a83 FEATURE: add support for `upload` format in theme settings. 2020-04-15 18:34:02 +05:30
Jeff Wong e3590d4ead
FEATURE: add user_session_refreshed trigger (#9412)
Trigger an event for plugins to consume when a user session is refreshed.

This allows external auth to be notified about account activity, and be
able to take action such as use oauth refresh tokens to keep oauth
tokens valid.
2020-04-14 09:32:24 -07:00
Mark VanLandingham f07c4a781c
DEV: Add options to theme install rake task - more options (#9394) 2020-04-14 09:17:00 -05:00
Sam Saffron e2284cf739
Revert "We have had errors reported due to migrations breaking and are reverting"
This reverts commit 8b46f14744.

It corrects the reason for the revert:

We rely on SafeMigrate existing cause we call it from migrations,
Zeitwerk will autoload it.

Instead of previous pattern we explicitly bypass all the hacks in
production mode.

We need to disable SafeMigrate cause it is not thread safe.

A thread safe implementation is possible but not worth the effort,
we catch the issues in dev and test.
2020-04-14 11:31:07 +10:00
Robin Ward b6b92a562c
FEATURE: New site setting `embed_unlisted` (#9391)
If enabled, posts imported to discourse via embeddings will default to
unlisted until they receive a reply.
2020-04-13 15:17:02 -04:00
Robin Ward 7b4fdebbce
FEATURE: Plugin support for transpiling regular `.js` files (#9398)
This adds support for a new piece of metadata to your plugin.rb
files. If you add:

```
transpile_js: true
```

Then Discourse will support transpilation of assets in your
`assets/javascripts` directory. Previously they had to be named
`.js.es6` but now regular `.js` will work.

Note this is opt-in because some plugins currently have `.js` files in
app/assets that are not meant to be transpiled.

Going forward all plugins should migrate to this setting as they are
comfortable able to do so.
2020-04-13 15:05:46 -04:00
Vinoth Kannan b57d4586d7 FIX: use correct command line attribute for `gifsicle` while scale down the gif. 2020-04-10 18:16:47 +05:30
Robin Ward 8b46f14744 We have had errors reported due to migrations breaking and are reverting
these series of commits.

See:
https://meta.discourse.org/t/new-installation-fails-on-migration/147425

This reverts commit 80e832662b.
This reverts commit de5f2d3308.
This reverts commit 6b192d29fa.
This reverts commit a5b582f686.
This reverts commit 708dd97dfd.
2020-04-09 14:18:56 -04:00
Dan Ungureanu 3d9c320aab
PERF: Cache Category.subcategory_ids (#9350)
Also reset category cache after backup restore.
2020-04-09 15:42:24 +03:00
Sam Saffron 80e832662b
DEV: fake stdout must respond to close
If it does not respond to close Logger will refuse to use it
2020-04-09 14:36:01 +10:00
Sam Saffron de5f2d3308
DEV: avoid loading module in production 2020-04-09 13:54:18 +10:00
Sam Saffron 6b192d29fa
DEV: only enable SafeMigrate in development and test
SafeMigrate outputs text when we detect attempts to unsafely drop tables
and columns

It is unfortunately not thread safe

This is not needed in production as we would have already caught it by then
in our test suite.
2020-04-09 13:41:03 +10:00
Sam Saffron a5b582f686
DEV: demux stdout when running multisite migrate
This avoids mixing up output in such a way that we can not tell which
site ran which migrations

Avoids threads all fighting for output
2020-04-09 12:50:14 +10:00
Sam Saffron 708dd97dfd
PERF: speed up migrations on multisite
Previously we were migrating multisites serially, this is extremely slow
especially when 200 dbs are involved.

The new implementation defaults to running 20 migrations concurrently, leading
to a 20x speedup.

We also amended it so errors are printed out last, something that makes
debugging failures easier.

This is code specific to Discourse cause we integrate SeedFu with our
migrations and can not include this in the multisite gem.
2020-04-09 11:58:20 +10:00
Sam Saffron f062ebf274
DEV: reduce noise in logs regarding migration
If a migration performs no changes it should not output stuff.

Previously we would output information about seeds which was very noisy.

On multisite this was particularly bad
2020-04-09 10:05:22 +10:00
Robin Ward e1f8014acd
FEATURE: Support for publishing topics as pages (#9364)
If the feature is enabled, staff members can construct a URL and publish a
topic for others to browse without the regular Discourse chrome.

This is useful if you want to use Discourse like a CMS and publish
topics as articles, which can then be embedded into other systems.
2020-04-08 12:52:36 -04:00
Dan Ungureanu 0653750fbf
FEATURE: Hash user API keys in the database (#9344)
The 'key' column will be dropped in a future commit.
2020-04-07 16:42:52 +03:00
Blake Erickson d04ba4b3b2
DEPRECATION: Remove support for api creds in query params (#9106)
* DEPRECATION: Remove support for api creds in query params

This commit removes support for api credentials in query params except
for a few whitelisted routes like rss/json feeds and the handle_mail
route.

Several tests were written to valid these changes, but the bulk of the
spec changes are just switching them over to use header based auth so
that they will pass without changing what they were actually testing.

Original commit that notified admins this change was coming was created
over 3 months ago: 2db2003187

* fix tests

* Also allow iCalendar feeds

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2020-04-06 16:55:44 -06:00
Krzysztof Kotlarek ce00da3bcd
FIX: guardian always got user but sometimes it is anonymous (#9342)
* FIX: guardian always got user but sometimes it is anonymous

```
  def initialize(user = nil, request = nil)
    @user = user.presence || AnonymousUser.new
    @request = request
  end
```

AnonymouseUser defines `blank?` method
```
  class AnonymousUser
    def blank?
      true
    end
    ...
  end
```
so if we would use @user.present? it would be correct, however, just @user is always true
2020-04-06 09:56:47 +10:00
Dan Ungureanu b9d411a4eb
FIX: Topic.time_to_first_response should include sub-sub-categories (#9349) 2020-04-04 13:31:34 +03:00
Gerhard Schlager ad6709772a PERF: Backup with lots of uploads stored on S3 was slow
Creating the backup needs a lot more disk space after this change, but it is a lot faster.
2020-04-03 18:13:34 +02:00
Daniel Waterworth 76610acb6f FIX: Default to light theme in wizard so that previews are displayed
Previously, without a theme selection, the previews wouldn't show.
2020-04-02 18:37:45 +01:00
Daniel Waterworth bca126f3f5 REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.

Follow-up-to: ca1208a636
2020-04-02 16:44:44 +01:00
Penar Musaraj 724d2e99de
DEV: Only include "report-sample" CSP directive when reporting is enabled (#9337) 2020-04-02 11:16:38 -04:00
Daniel Waterworth ca1208a636 Revert "REFACTOR: Move the multisite middleware to the front"
Looks like this is causing problems.

Follow-up-to: a91843f0dc
2020-04-02 15:20:28 +01:00
Daniel Waterworth a91843f0dc REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.
2020-04-02 10:15:38 +01:00
Sam Saffron 8a50ab45aa
DEV: use REPORT_REQUESTS=1 to find all requests
We can use this to profile our pretender and ensure nothing is superfluous
and nothing is missing
2020-04-02 16:01:49 +11:00
Kane York cdaa60b56b FEATURE: Allow admins to disable self-service account deletion
https://meta.discourse.org/t/-/146276
2020-04-01 15:16:07 -07:00
Neil Lalonde 90fcede832
FIX: replace default welcome topic post with new value from wizard
Previously the text entered in the wizard would be prepended onto the
default first paragraph.
2020-04-01 15:42:45 -04:00
Gerhard Schlager 13b4eb9cce FIX: Restore failed if schema contained objects not owned by the current DB user 2020-04-01 18:04:43 +02:00
Mark VanLandingham 689c61b462
DEV: Allow plugins to add wizard steps after specific steps (#9315) 2020-04-01 08:36:50 -05:00
Martin Brennan c07dd0d22a
FEATURE: Add lazy loading to user bookmarks list (#9317)
This is so users with huge amount of bookmarks do not have to wait a long time to see results.

* Add a bookmark list and list serializer to server-side to be able to handle paging and load more URL
* Use load-more component to load more bookmark items, 20 at a time in user activity
* Change the way current user is loaded for bookmark ember models because it was breaking/losing resolvedTimezone when loading more items
2020-04-01 14:09:07 +10:00
Arpit Jalan b2a0d34bb7
FEATURE: add setting `auto_approve_email_domains` to auto approve users (#9323)
* FEATURE: add setting `auto_approve_email_domains` to auto approve users

This commit adds a new site setting `auto_approve_email_domains` to
auto approve users based on their email address domain.

Note that if a domain already exists in `email_domains_whitelist` then
`auto_approve_email_domains` needs to be duplicated there as well,
since users won’t be able to register with email address that is
not allowed in `email_domains_whitelist`.

* Update config/locales/server.en.yml

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2020-03-31 23:59:15 +05:30
Sam Saffron 6d3d9ecf6c
DEV: adjust rake autospec to work with renamed es6 files
We no longer use es6 files, update autospec
2020-03-31 14:40:58 +11:00
Joffrey JAFFEUX 5b6cdd6fb5
DEV: adds a loading property to d-button (#9072)
Usage:

```
{{d-button icon="times" label="foo.bar" isLoading=true}}
```

Note that  a button loading without an icon will shrink text size to prevent button to jump in size.

A button while loading is disabled.
2020-03-30 23:17:00 +02:00
Joffrey JAFFEUX 0996c3b7b3
FEATURE: allows multiple custom emoji groups (#9308)
Note: DBHelper would fail with a sql syntax error on columns like "group".

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2020-03-30 20:16:10 +02:00
Bianca Nenciu 3914e9cb5c
FIX: get_size_from_image_sizes should return [width, height] or nil (#9298) 2020-03-28 20:20:51 +02:00
Robin Ward b2b7afd310 Rename the server side widget hbs compiler 2020-03-27 12:06:14 -04:00
Roman Rizzi 57321b90f0
Completely remove read only mode during backups (#9279) 2020-03-27 07:38:55 -03:00
Sam Saffron 40b6e278a0
FIX: ensure wiki editor is assigned consistently
`DISTINCT ON` was not providing stability, this ensures the backfill will
be stable on a specific post avoiding risk around double grants.
2020-03-27 12:41:06 +11:00
Robin Ward 7d2c71dd5f FIX: Auto redirect had invalid extension 2020-03-26 13:21:04 -04:00
Robin Ward 388e14b9dc FIX: Broken transpilation 2020-03-26 13:12:17 -04:00
Robin Ward 358ddb9b95 FIX: `nil` != `false` 2020-03-26 12:47:10 -04:00
Robin Ward 60df2ade8d Retry: Rename all test files from JS -> ES6 2020-03-26 12:25:46 -04:00
Gerhard Schlager 5bffb033df FIX: The migrate_to_s3 rake task couldn't find the AWS SDK 2020-03-26 16:41:10 +01:00
Jarek Radosz d21d80198c
DEV: Update rubocop-discourse (#9270)
Includes:
* DEV: Use `eq_time` matcher
2020-03-26 16:32:41 +01:00
Bianca Nenciu 7952cbb9a2
FIX: Perform crop using user-specified image sizes (#9224)
* FIX: Perform crop using user-specified image sizes

It used to resize the images to max width and height first and then
perform the crop operation. This is wrong because it ignored the user
specified image sizes from the Markdown.

* DEV: Use real images in test
2020-03-26 16:40:00 +02:00
Bianca Nenciu ba1a08510e
FIX: Use correct spacing in emails with code (#9274)
Follow-up-to 778454e26b
2020-03-26 14:24:07 +02:00
Sam Saffron 25f1f23288
FEATURE: Stricter rules for user presence
Previously we would consider a user "present" and "last seen" if the
browser window was visible.

This has many edge cases, you could be considered present and around for
days just by having a window open and no screensaver on.

Instead we now also check that you either clicked, transitioned around app
or scrolled the page in the last minute in combination with window
visibility

This will lead to more reliable notifications via email and reduce load of
message bus for cases where a user walks away from the terminal
2020-03-26 17:36:52 +11:00
Martin Brennan 6f978bc95c
FIX: First pass to improve efficiency of secure uploads rake task (#9284)
Get rid of harmful each loop over uploads to update. Instead we put all the unique access control posts for the uploads into a map for fast access (vs using the slow .find through array) and look up the post when it is needed when looping through the uploads in batches.

On a Discourse instance with ~93k uploads, a simplified version of the old method takes > 1 minute, and a simplified version of the new method takes ~18s and uses a lot less memory.
2020-03-26 15:59:57 +10:00
Martin Brennan efd5fb665b
DEV: Fix flaky time sensitive uploads.rake specs (#9283)
Also fix issues in spec where certain uploads were not considered secure
2020-03-26 13:31:39 +10:00
Martin Brennan 097851c135
FIX: Change secure media to encompass attachments as well (#9271)
If the “secure media” site setting is enabled then ALL files uploaded to Discourse (images, video, audio, pdf, txt, zip etc. etc.) will follow the secure media rules. The “prevent anons from downloading files” setting will no longer have any bearing on upload security. Basically, the feature will more appropriately be called “secure uploads” instead of “secure media”.

This is being done because there are communities out there that would like all attachments and media to be secure based on category rules but still allow anonymous users to download attachments in public places, which is not possible in the current arrangement.
2020-03-26 07:16:02 +10:00
Robin Ward 4fa580fbd1 Revert "Rename all test files from JS -> ES6"
This reverts commit 2abe85b834.
2020-03-25 16:13:43 -04:00
Robin Ward eaa324ecbd Revert "Move the widget-hbs compiler to js from es6"
This reverts commit 5d66a2c16e.
2020-03-25 16:13:26 -04:00
Robin Ward 5d66a2c16e Move the widget-hbs compiler to js from es6 2020-03-25 15:03:21 -04:00
Robin Ward 2abe85b834 Rename all test files from JS -> ES6 2020-03-25 15:03:21 -04:00
Mark VanLandingham c14f6d4ced
FEATURE: Allow plugins to exclude wizard steps (#9275) 2020-03-25 11:36:42 -05:00
Robin Ward f2f8ede22c Last ES6 files in our app/assets/javascripts 2020-03-25 12:30:20 -04:00
Dan Ungureanu 49395ec577
FIX: Fix image optimization pipeline (#9257)
* FIX: Do not use original filename to extract the original filename

Prefer extracting filename from the destination path, which is build
using extracted image information.

* UX: Show better error images
2020-03-25 12:59:16 +02:00
Robin Ward a11938d58d Rename more es6 files to js 2020-03-24 16:32:56 -04:00
Robin Ward 4d190c93b6 Migrate ember-addons from ES6 -> JS 2020-03-24 16:11:56 -04:00
Bianca Nenciu d8640fd042
DEV: Move requested_group_id custom field from post to topic (#9127)
Follow-up-to accbbded15
2020-03-24 11:12:52 +02:00
Robin Ward 27641f21e4 Migrate `discourse-common` from es6 -> js 2020-03-23 15:05:58 -04:00
Robin Ward 1ac0242201 Convert wizard es6 files to js 2020-03-23 14:15:16 -04:00
David Taylor 5db41cd578
SECURITY: Respect topic permissions when loading bookmark metadata
Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2020-03-23 11:30:48 +00:00
Roman Rizzi c4bc734b11
FIX: Backfill topic timer duration (#9249) 2020-03-23 09:38:18 +05:30
Robin Ward 07813c4a91
Convert select-kit from es6 to js (#9246)
* Convert select-kit from es6 to js

* Hide more git blames
2020-03-20 12:40:32 -04:00
Robin Ward c150566506
Migrate pretty-text to `.js` extensions (#9243) 2020-03-20 09:55:42 -04:00
Sam Saffron 145c90419d
DEV: increase timeout for initial context eval
Due to JS refactors our initial payload transpile is taking a bit longer
instead of failing the test suite attempt to give this a bit more time
2020-03-20 15:48:25 +11:00
Sam Saffron 10b37e1e36
FIX: add support for sub-sub category slugs in search
Previous to this change slugs for leaves in 3 level nestings would not work

Our UX picks only the last two levels

This also makes the results consistent for slugs as it enforces order.
2020-03-20 15:36:50 +11:00
David Taylor 19814c5e81
FIX: Allow CSP to work correctly for non-default hostnames/schemes (#9180)
- Define the CSP based on the requested domain / scheme (respecting force_https)
- Update EnforceHostname middleware to allow secondary domains, add specs
- Add URL scheme to anon cache key so that CSP headers are cached correctly
2020-03-19 19:54:42 +00:00
Joffrey JAFFEUX 86eb03af85
DEV: allows to run up/down migrations of a plugin (#9241)
This is mostly useful while developing a plugin, to avoid manual actions of deleting tables and schema_migrations rows.

Usage:
bundle exec rake plugin:migrate:down[discourse-calendar]
2020-03-19 19:30:08 +01:00
Vinoth Kannan f6d6f1701f FIX: use the new duration attribute in `set_or_create_timer` method.
New `duration` attribute is introduced for the `set_or_create_timer` method in the commit aad12822b7 for "based on last post" and "auto delete replies" topic timers.
2020-03-19 21:45:05 +05:30
Martin Brennan 0cd502a558
FIX: N1 issues for bookmark list (#9236)
* Preload custom fields for BookmarkQuery and add preload callback. Copy TopicQuery preload methodology to allow plugins to preload data for the BookmarkQuery. This fixes assigned plugin custom fields N1
* Include topic tags in initial query to avoid tags N1

Related: discourse/discourse-assign#63
2020-03-19 15:48:23 +10:00
Dan Ungureanu 1393950dbc
FIX: Improve HTML to Markdown conversion (#9231)
This commit ensures that whitespaces are preserved in <pre>, but removed
inside text paragraphs.
2020-03-18 19:31:10 +02:00
Bianca Nenciu 778454e26b
FIX: Condense line codes in emails (#9225) 2020-03-18 16:21:24 +02:00
Kane York ccc9b64a99 FIX: theme-javascripts using incorrect subfolder setting 2020-03-17 19:09:06 -07:00
Jarek Radosz e950471c0f
DEV: Replace User.unstage and User#unstage API with User#unstage! (#8906)
* DEV: Replace User.unstage and User#unstage API with User#unstage!

Quoting @SamSaffron:

> User.unstage mixes concerns of both unstaging users and updating params which is fragile/surprising.
> u.unstage destroys notifications and raises a user_unstaged event prior to the user becoming unstaged and the user object being saved.

User#unstage! no longer updates user attributes and saves the object before triggering the `user_unstaged` event.

* Update one more spec

* Assign attributes after unstaging
2020-03-17 16:48:24 +01:00
Martin Brennan 16799da580
FIX: Improve bookmark modal on mobile and bookmark sync rake task (#9221)
* Improve the bookmark mobile on modal so it doesn't go all the way to the edge and the custom datetime input is easier to use
* Improve the rake task for syncing so it does not error for topics that no longer exist and batches 2000 inserts at a time, clearing the array each time
2020-03-17 15:15:22 +10:00
Rafael dos Santos Silva e50abe1317
FIX: sync-alt is used on composer draft indicator 2020-03-16 15:32:38 -03:00
Blake Erickson 269ec70ba8 DEV: Only show deprecation message for non-whitelisted requests
As another step toward fully dreprecating query parameter authentication
in API requests this change prevents an admin dashboard message from
showing up if using a whitelisted route like rss feeds or the
mail-receiver route.
2020-03-16 12:05:24 -06:00
Daniel Waterworth 15ae3145ec FIX: Plugins may have relative symlinks 2020-03-15 11:26:25 +00:00
Sam Saffron a14313e9d0
FIX: RANDOM_PASSWORD not working rake admin:create
We regressed behavior of this env var which enable admin account
creation with a random password
2020-03-15 17:24:11 +11:00
Gerhard Schlager 8022e51179 FIX: Failed to restore backups from versions without translation overrides
Rails calls I18n.translate during initialization and by default translation overrides are used. Database migrations would fail if the system tried to migrate from an old version that didn't have the `translation_overrides` table with all its columns yet.

This makes restoring really old backups work again. Running `DISABLE_TRANSLATION_OVERRIDES=1 rake db:migrate` will allow you to upgrade such an old database as well.
2020-03-14 00:00:22 +01:00
Robin Ward e40e06d78c Rename `.js.es6` to `.js` in the admin application 2020-03-13 17:03:08 -04:00
David Taylor 3d71b68195
DEV: Introduce plugin api for conditionally rendering assets (#9200) 2020-03-13 15:30:31 +00:00
Martin Brennan af92444948
FIX: Make sure bookmark serializer works with deleted topics + posts (#9195) 2020-03-13 10:44:39 +10:00
Martin Brennan 2237ba8c9d
FIX: Add topic deleted check to email/sender (#9166)
It already had a deleted post check and log reason, add a topic one too to avoid errors
2020-03-13 10:04:15 +10:00
David Taylor ec2d49d48a
DEV: Allow plugins to add theme modifiers via db migrations (#9192) 2020-03-12 16:35:28 +00:00
Daniel Waterworth 59578dfc5b FIX: Notification emails with attachments are incorrectly structured
Two behaviors in the mail gem collide:

 1. Attachments are added as extra parts at the top level,
 2. When there are both text and html parts, the content type is set to
    'multipart/alternative'.

Since attachments aren't alternative renderings, for emails that contain
attachments and both html and text parts, some coercing is necessary.
2020-03-12 15:42:24 +00:00
Martin Brennan 89a1b3a791 Make sure reminder not sent for deleted post bookmark 2020-03-12 16:10:56 +10:00
Martin Brennan e1eb5fb9b3
FEATURE: MVP Bookmarks with reminders user list changes (#8999)
* This PR changes the user activity bookmarks stream to show a new list of bookmarks based on the Bookmark record.
* If a bookmark has a name or reminder it will be shown as metadata above the topic title in the list
* The categories, tags, topic status, and assigned show for each bookmarked post based on the post topic
* Bookmarks can be deleted from the [...] menu in the list
* As well as this, the list of bookmarks from the quick access panel is now drawn from the Bookmarks table for a user:
* All of this new functionality is gated behind the enable_bookmarks_with_reminders site setting
The /bookmarks/ route now redirects directly to /user/:username/activity/bookmarks-with-reminders
* The structure of the Ember for the list of bookmarks is not ideal, this is an MVP PR so we can start testing this functionality internally. There is a little repeated code from topic.js.es6. There is an ongoing effort to start standardizing these lists that will be addressed in future PRs.
* This PR also fixes issues with feature detection for at_desktop bookmark reminders
2020-03-12 15:20:56 +10:00
Martin Brennan 849631188f
FEATURE: Allow custom date + time for bookmark reminders (#9185)
A custom date and time can now be selected for a bookmark reminder

The reminder will not happen at the exact time but rather at the next 5 minute interval of the bookmark reminder schedule.

This PR also fixes issues with bulk deleting topic bookmarks.
2020-03-12 10:52:15 +10:00
Martin Brennan 793f39139a
FEATURE: Send notifications for time-based and At Desktop bookmark reminders (#9071)
* This PR implements the scheduling and notification system for bookmark reminders. Every 5 minutes a schedule runs to check any reminders that need to be sent before now, limited to **300** reminders at a time. Any leftover reminders will be sent in the next run. This is to avoid having to deal with fickle sidekiq and reminders in the far-flung future, which would necessitate having a background job anyway to clean up any missing `enqueue_at` reminders.

* If a reminder is sent its `reminder_at` time is cleared and the `reminder_last_sent_at` time is filled in. Notifications are only user-level notifications for now.

* All JavaScript and frontend code related to displaying the bookmark reminder notification is contained here. The reminder functionality is now re-enabled in the bookmark modal as well.

* This PR also implements the "Remind me next time I am at my desktop" bookmark reminder functionality. When the user is on a mobile device they are able to select this option. When they choose this option we set a key in Redis saying they have a pending at desktop reminder. The next time they change devices we check if the new device is desktop, and if it is we send reminders using a DistributedMutex. There is also a job to ensure consistency of these reminders in Redis (in case Redis drops the ball) and the at desktop reminders expire after 20 days.

* Also in this PR is a fix to delete all Bookmarks for a user via `UserDestroyer`
2020-03-12 10:16:00 +10:00
Robin Ward c9d1577087 Let's not log the username/password
This could easily be seen by someone who shouldn't.
2020-03-11 12:48:10 -04:00
Robin Ward a3f0543f99
Support for transpiling `.js` files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
Joffrey JAFFEUX fd4ce6ab8f
DEV: hbs extensions are misleading in this case (#9170)
This would also prevent any linting tool to attempt to lint this incorrectly.
2020-03-11 14:42:14 +01:00
David Taylor d1474e94a1
FEATURE: Allow themes to specify modifiers in their about.json file (#9097)
There are three modifiers:
- serialize_topic_excerpts (boolean)
- csp_extensions (array of strings)
- svg_icons (array of strings)

When multiple themes are active, the values will be combined. The combination method varies based on the setting. CSP/SVG arrays will be combined. serialize_topic_excerpts will use `Enumerable#any`.
2020-03-11 13:30:45 +00:00
Dan Ungureanu 0754c7c404
FIX: Various fixes to support posts with no user (#8877)
* Do not grant badges for posts with no user
* Ensure instructions are correct in Change Owner modal
* Hide user-dependent actions from posts with no user
* Make PostRevisor work with posts with no user
* Ensure posts with no user can be deleted
* discourse-narrative-bot should ignore posts with no user
* Skip TopicLink creation for posts with no user
2020-03-11 14:03:20 +02:00
Sam Saffron 1f34f653bf
FIX: last ip address could point at wrong ip
Due to unicorn env object recycling request.ip could point at the wrong
ip address by the time defer block is called. This usually would happen
under load.

This also avoids keeping the entire request object as referenced by the
closure.
2020-03-11 17:42:56 +11:00
romanrizzi f795c1b8e8 Revert "DEV: enforces ember-template-lint: no-triple-curlies (#9150)"
This reverts commit d436b600fb.

Triple curlies are still necessary for some raw templates.
2020-03-10 15:00:12 -03:00
Michael Brown 74acbad505
FIX: don't break the private key when writing it out during theme import
* when importing a private theme using the themes:install rake task the SSH key is written out to a file for use by the git-clone command
* if the private key is written out without a newline at end-of-file (i.e. after it's been stripped) it's not recognized as a valid key by SSH
* so: don't strip it when writing it out, we should be fine
2020-03-10 13:20:11 -04:00
Joffrey JAFFEUX d436b600fb
DEV: enforces ember-template-lint: no-triple-curlies (#9150)
This pr replaces `{{{ }}}` usage by a {{html-safe}} helper. While it doesn't solve the underlying issue, it gives us a path forward without risking breaking too much existing behavior.

Also introduces an htmlSafe computed macro:

```
import { htmlSafe } from "discourse/lib/computed";

htmlDescription: htmlSafe("description")
```

Overtime {{html-safe}} usage should be removed and moved to components properties or specialized components/helpers.
2020-03-10 16:46:57 +01:00
Jarek Radosz aec26ad2f0
FIX: Preserve TopicCreator's timestamp resolution (#9158)
Continuation of #9140 (e35bc8b). It's the last piece required for #9141.
2020-03-10 15:35:40 +01:00
Roman Rizzi 826b4793c0
FEATURE: Approve suspect users is now true by default. The suspect users list was removed (#9151) 2020-03-10 08:56:42 -03:00
Mark VanLandingham 3ad5cb0cbc
FIX: Error message for 403 when featuring topic on profile (#9149) 2020-03-09 11:41:07 -05:00
Jarek Radosz e35bc8bebd
FIX: Preserve PostCreator's created_at resolution (#9140)
PostMover passes to PostCreator a `created_at` that is a `ActiveSupport::WithTimeZone` instance (and also `is_a? Time`). Previously it was always being passed through `Time.zone.parse` so it would lose sub-second information. Now, it takes `Time` input as-is, while still parsing other types.
2020-03-09 17:38:13 +01:00
Jarek Radosz 85e03a7f68
DEV: Replace `Time.new` with `Time.now` (#9142)
(or `Time.zone.now`)
2020-03-09 17:37:49 +01:00
Mark VanLandingham 174764be25
FEATURE: Add embed_set_canonical_url setting (#9134) 2020-03-09 09:31:24 -05:00
Joffrey JAFFEUX 60b47d622e
UX: adds support for a color setting type (#9016) 2020-03-09 10:07:03 +01:00
Roman Rizzi 87687c0819
Drop unnecessary readonly_during_backup setting (#9112) 2020-03-06 14:29:00 -03:00
Mark VanLandingham 475f545557
DEV: Test theme js compiler append_raw_template (#8950)
* DEV: Test for append_raw_template

* remove empty line
2020-03-06 11:35:52 -05:00
Robin Ward a5f61729e0 Revert "Revert "FIX: Don't allow people to clear the upload bucket while it's enabled""
This reverts commit d4fc76b335.
2020-03-06 09:35:55 -05:00
Robin Ward d4fc76b335 Revert "FIX: Don't allow people to clear the upload bucket while it's enabled"
This reverts commit 4bb8db024c.
2020-03-05 16:29:59 -05:00
Neil Lalonde 3e21d40bc4
Version bump to v2.5.0.beta2 2020-03-05 16:10:28 -05:00
Robin Ward 4bb8db024c FIX: Don't allow people to clear the upload bucket while it's enabled 2020-03-05 16:04:52 -05:00
Kane York 10ddb8a9c4 FIX: Use destroy_all instead of delete_all for shared drafts
Rails has an odd behavior for calling .delete_all on a has_many relation - the
default behavior is to nullify the foreign key fields instead of actually
'DELETE'ing the records.

Additionally, publishing a shared draft topic creates a PostRevision that the
NotifyPostRevision job picks up which is then promptly deleted.

Use destroy_all when cleaning up the revisions and have the NotifyPostRevision
job tolerate deleted PostRevision records.

This takes a small performance hit (several SQL DELETEs instead of just one)
but shouldn't be too much of an issue (high cardinalities range from 30-100).
2020-03-05 11:13:43 -08:00
David Taylor 5b3630dba3
FIX: Do not raise an error when in:all search is performed by anon (#9113)
Also improve in:all specs to catch to catch similar failures
2020-03-05 17:50:29 +00:00
Robin Ward e01d5e2adc SECURITY: Add more restrictions on invite emails
They could be filtered and returned in some circumstances where they
shouldn't have been.
2020-03-05 09:23:21 -05:00
Daniel Waterworth 83e649d08e FIX: Clobbering plugin files on startup is problematic
On startup, (including when starting a rails console) we manipule a
collection of plugin files. Writing these files is done in multiple
observable steps, which presents opportunities for race conditions and
causes temporary corruption.

This commit uses the write, fsync and rename trick to atomically
overwrite these files instead, but reads them first to avoid unnecessary
writes.

c457d3bf was a previous attempt to fix the same problem.
2020-03-05 11:51:56 +00:00
tshenry 1c60c731ba
DEV: Mark `discourse-categories-suppressed` as official 2020-03-04 09:35:04 -08:00
Gerhard Schlager 8fa8bab9ff FIX: Don't optimize icons during db:migrate when restoring backup
Uploads are extracted after the DB migration, so this could lead to a failure during the restore. Site icons get optimized after extracting uploads.
2020-03-04 16:59:49 +01:00
Gerhard Schlager 93b8b04b06 FIX: Migrating uploads to S3 could miss files
The rake task aborted the migration with "Already migrated" when all upload URLs linked to the correct S3 bucket even though the files didn't exist on S3. By removing the first check we force the rake task to check for the existance of uploads on S3.
2020-03-04 12:50:48 +01:00
Sam Saffron 9cbed7ee46
Revert "FIX: under concurrent usage booting rails could cause plugin corruption"
This reverts commit c457d3bf28.

Something is not right about this fix, reverting.
2020-03-04 17:10:36 +11:00
Sam Saffron 0054ccc522
Revert "DEV: correct typo in param name"
This reverts commit dae5b3d673.

Something is not right about my fix reverting
2020-03-04 17:10:14 +11:00