Commit Graph

6947 Commits

Author SHA1 Message Date
Martin Brennan 5ad6dc01c1
FEATURE: Support RS256 algorithm for webauthn (#8385)
* Add support for RS256 webauthn algorithm (-257)
2019-11-21 15:45:36 +10:00
Sam Saffron 423ad5f0a4 FIX: do not log if an invalid mime type is passed to app
Previously our custom exception handler was unable to handle situations
where an invalid mime type was sent, resulting in a warning log

This ensures we pretend a request is HTML for the purpose of rendering
the error page if an invalid mime type from a scanner is shipped to the app
2019-11-21 15:51:34 +11:00
Jeff Wong c6d8dbd4a9 Revert "FEATURE: Normalize the service worker route (#8359)"
This reverts commit 9799a651b6.
2019-11-20 14:10:17 -08:00
Jeff Wong 9799a651b6
FEATURE: Normalize the service worker route (#8359)
* FEATURE: Normalize the service worker route

Update cache headers so they are not immutable outside of the rails app

Add the ability to purge the service worker cache from localhost

Rails -> nginx will pass immutable flags so the file is cached until reloaded.
In most cases, nginx will have its cache flushed on rebuild (new image)

For those needing dynamic re-caching (such as upgrading via the UI),
a rake task for flushing the service worker script is provided
through `assets:flush_sw`
2019-11-20 11:33:41 -08:00
David Taylor b8e5208162 DEV: Add convenience method to check info about a site setting
This is only defined in a console environment. For example:

```
[1] pry(main)> SiteSetting.info(:title)
=> {:resolved_value=>"Globally Overridden Title",
 :default_value=>"Discourse",
 :global_override=>"Globally Overridden Title",
 :database_value=>"Test Discourse",
 :refresh?=>false,
 :client?=>true,
 :secret?=>false}
```
2019-11-20 16:39:47 +00:00
Martin Brennan d45bd7f131
FIX: Abort CensoredWordsValidator early if censored_words_regexp nil (#8375)
* Abort CensoredWordsValidator early if censored_words_regexp nil. Sometimes censored_words_regex can end up nil, erroring the validator. This handles the nil condition and also adds a spec for the validator
2019-11-20 20:05:06 +10:00
Martin Brennan 02cb01406e
FIX: Allow secure uploads if global s3 setting active and enable_s3_uploads validations (#8373)
The secure media functionality relied on `SiteSetting.enable_s3_uploads?` which, as we found in dev, did not take into account global S3 settings via `GlobalSetting.use_s3?`. We now use `SiteSetting.Upload.enable_s3_uploads` instead to be more consistent.

Also, we now validate `enable_s3_uploads` changes, because if `GlobalSetting.use_s3?` is true users should NOT be enabling S3 uploads manually.
2019-11-20 07:46:44 +10:00
Vinoth Kannan d3e7768ea8 Revert "FIX: parallel spec system needs needs a dedicated upload folder for each worker. (#8372)"
This reverts commit 42e5176bc3.
2019-11-19 15:02:18 +05:30
Vinoth Kannan 42e5176bc3
FIX: parallel spec system needs needs a dedicated upload folder for each worker. (#8372) 2019-11-19 13:16:20 +05:30
Vinoth Kannan f83125f0c2 DEV: minor refactoring to reduce the code duplication. 2019-11-19 08:04:24 +05:30
Vinoth Kannan 57bbcf4c5d FIX: 'default_categories_muted' site setting not working for anonymous users. 2019-11-19 07:48:16 +05:30
Martin Brennan 93d7abe372
FIX: Automatically recover from bad sprockets cache in development (#8364)
We were having issues in development mode where the JS code had errors due to a bad cache. When starting a server in development mode in bin/unicorn we now get the git sha of the discourse HEAD and get a git sha of all plugins, and store them in a file. If the sha has changed then we delete tmp/cache to refresh the assets cache.
2019-11-19 09:15:09 +10:00
Neil Lalonde 228c4814be FIX: errors when using tags with colons in their name 2019-11-18 13:20:37 -05:00
David Taylor eaf6096890 DEV: Use rubocop-discourse gem to add custom chdir cop
Followup to b27e009655
2019-11-18 15:39:41 +00:00
Vinoth Kannan 3bb7ad4be1
FEATURE: remove support for 'suppress_from_latest' category setting. (#8308) 2019-11-18 12:28:35 +05:30
Martin Brennan af091c49e9
FIX: Ensure revisions are made to store edit reasons and no reasons get wiped (#8363)
* Fix an issue where if an edit was made to a post with a reason provided, and then another edit was made with no reason, the original edit reason got wiped out
* We now always make a post revision (even with ninja edits) if an edit reason has been provided and it is different from the current edit reason

Co-Authored-By: Sam <sam.saffron@gmail.com>
2019-11-18 13:08:54 +10:00
Penar Musaraj 102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Robin Ward 397852a9a6 FIX: Support for `rake db:rollback` with plugins
For this to work we need to overwrite `db:rollback` in our Rakefile like
we do for migrate, so that it removes the load_config dependency. This
allows our custom migration paths to work.
2019-11-15 16:38:46 -05:00
Martin Brennan 5c59247c3a Block enabling force 2FA if local logins disabled & vice-versa (#8355) 2019-11-15 17:05:10 +11:00
Penar Musaraj 067696df8f DEV: Apply Rubocop redundant return style 2019-11-14 15:10:51 -05:00
David Taylor f2da630c19 FIX: SVG Sprite version hash should be based on bundle result
This version hash is used for the filename, and so browsers/CDNs cache based on it. Previously the version hash was based only on the list of requested icons. This can cause issues in a couple of situations, most commonly when developing themes with custom icons:

- A requested icon does not exist, and then later is added to the theme. The bundle output changes, but the hash did not
- The SVG content of an icon changes, but the name of the icon does not. The bundle output changes, but the hash did not
2019-11-14 13:20:16 +00:00
Martin Brennan e7226a8c84
FEATURE: Allow scoping search to tag (#8345)
* When viewing a tag, the search widget will now show a checkbox to scope the search by tag, which will limit search results to that tag on desktop and mobile
2019-11-14 10:40:26 +10:00
Krzysztof Kotlarek 6e1fe22a9d
FEATURE: Dismiss new per category (#8330)
Ability to dismiss new topics per category.
2019-11-14 11:16:13 +11:00
David Taylor d095c2cee7 FIX: Allow importing themes with subdirectories in extra_js
The folder/file detection was broken by 9fea43e46a. This commit fixes and adds relevant specs
2019-11-13 23:45:09 +00:00
romanrizzi 0ba0aa00e7 DEV: Use a bigger suite for warming up the tmp folder 2019-11-13 18:34:10 -03:00
Roman Rizzi c63e84dc62
FEATURE: Send a message to new TL2 users and point them to the advanced tutorial (#8335)
* FEATURE: Send a message to new TL2 users and point them to the advanced tutorial

* DEV: Use a method to find the discobot user
2019-11-13 18:31:49 -03:00
David Taylor 481efebe76
DEV: Update backup/restore pipeline to avoid `cd` (#8347) 2019-11-13 15:52:28 +00:00
David Taylor 9fea43e46a
DEV: Remove use of `cd` in the app (#8337)
`FileUtils.cd` and `Dir.chdir` cause the working directory to change for the entire process. We run sidekiq jobs, hijacked requests and deferred jobs in threads, which can make working directory changes have unintended side-effects.

- Add a rubocop rule to warn about usage of Dir.chdir and FileUtils.cd
- Added rubocop:disable for scripts used outside the app
- Refactored code using cd to use alternative methods
- Temporarily skipped the rubocop check for lib/backup_restore. This will require more complex refactoring, so I will create a separate PR for review
2019-11-13 09:57:39 +00:00
Neil Lalonde 7711df40e6
REFACTOR: redo DiscourseTagging.filter_allowed_tags (#8328)
This method had grown into a monster. Its query had bugs
that I couldn't fix, and new features would be hard to add.
Also I don't understand how it all works anymore...
Replace it with common table expressions that can be queried
to generate the results we need, instead of subtracting
results using lots of "NOT IN" clauses.

Fixed are bugs with tag schemas that use combinations of
tag groups, parent tags, and one-tag-per-topic restrictions.
For example: https://meta.discourse.org/t/130991/6
2019-11-12 14:28:44 -05:00
Robin Ward 7f651b9666 REFACTOR: Deprecate `Discourse.Site` and `Discourse.User` 2019-11-12 10:53:22 -05:00
Mark VanLandingham 3dd2f2f701
DEV: Remove RSS feed polling in favor of plugin (#8233) 2019-11-12 09:49:02 -06:00
Robin Ward 6e481851e7 REFACTOR: Don't generate `Discourse` constants for mixins 2019-11-12 10:21:43 -05:00
Daniel Waterworth e79cec9fc7 DEV: Remove handling of category top menu items
Support for these kinds of navigation items was dropped in 88f52514, but
the code for handling these menu items was never removed.
2019-11-11 17:36:29 +00:00
Simon Cossar 544a1379b6
Reference posts table for deleted_at and user_id (#8332) 2019-11-11 11:03:57 -05:00
Sam Saffron 91daafc674 DEV: correct implementation of expiry api
Previously we were always hard-coding expiry, this allows the secure session
to correctly handle custom expiry times

Also adds a ttl method for looking up time to live
2019-11-11 11:18:12 +11:00
Sam Saffron b68eab2b46 DEV: remove deprecated syntax
Ruby 2.7 is going to deprecate "Magic Proc Init". This feature is enormously
magical anyway and the new code is far easier to comprehend.
2019-11-11 09:36:40 +11:00
Blake Erickson 15f6f57cdc DEV: Add update message for an outdated mail-receiver
Previous versions of the mail-receiver used query based api credentials,
if we detect this we will show a message in the admin panel to update
the mail receiver.
2019-11-08 17:32:22 -07:00
Jay Pfaffman 63bd07492e FIX: Update api_key rake task for recent changes
New usage: 

```
rake api_key:get_or_create_master["Onboarding Key"]
```
2019-11-08 23:34:15 +00:00
Robin Ward 920f8c6d75 REFACTOR: Remove `Discourse.Route` in lieu of importing 2019-11-08 15:18:13 -05:00
Robin Ward f03edb36be REFACTOR: Remove `Discourse.Model` 2019-11-08 14:13:35 -05:00
Robin Ward fa71919693 REFACTOR: Don't transpile `Discourse.Session` 2019-11-08 13:48:18 -05:00
Robin Ward 7ac2a55588 REFACTOR: Remove `Discourse.Category` constants 2019-11-08 13:31:00 -05:00
Robin Ward 7e44065bcb REFACTOR: Migrate `Discourse.Post` to imports 2019-11-08 13:13:13 -05:00
Robin Ward 932c169d46 REFACTOR: Remove Discourse.NavItem constants 2019-11-08 12:56:13 -05:00
Robin Ward 779ee3558d REFACTOR: Remove `Discourse.UserAction` from code 2019-11-08 12:52:39 -05:00
Robin Ward 1990baa457 REFACTOR: Remove constant definition for discourse/routes/grouped 2019-11-08 12:47:39 -05:00
David Taylor 1998be3b27
DEV: Raise errors when cleaning the download cache, and fix for macOS (#8319)
POSIX's `head` specification states: "The application shall ensure that the number option-argument is a positive decimal integer"

Negative values are supported on GNU `head`, so this works in the discourse docker image. However, in some environments (e.g. macOS), the system `head` version fails with a negative `n` parameter.

This commit does two things:

Checks the status at each stage of the pipe, so it cannot fail silently
Flip the `ls` command to list in descending time order, and use `tail -n +501` instead of `head -n -500`.

The visible result is that macOS users no longer see head: illegal line count -- -500 printed throughout the test suite.
2019-11-08 15:34:03 +00:00
David Taylor cd57c3bf5a
DEV: Abort autospec on [ENTER], even if no specs have failed (#8320)
When starting autospec, it says

> Press [ENTER] to stop the current run

However, [ENTER] does nothing unless a spec has failed. Sometimes I want to abort anyway, so that the run is restarted.
2019-11-08 14:23:12 +00:00
David Taylor da50cd554a
DEV: Optionally allow autospec without auto-running the whole suite (#8321)
I want to use autospec while working on a single spec file. At the moment, it will start running all specs once it completes the file I'm working on. With parallel mode enabled, this causes CPU usage to spike dramatically, affecting IDE performance, battery life, and fan noise. I would prefer that it only runs all specs when I explicitly press [ENTER]

This commit adds a new ENV variable `AUTO_RUN_ALL`. To prevent auto-running all specs, set it to 0. The default behavior remains unchanged.
2019-11-08 14:22:57 +00:00
Sam Saffron d4c9ed68a9 DEV: properly pass keyword args
Ruby 2.7 is going to deprecate our old and incorrect usage, use the correct
form for passing kwargs
2019-11-08 17:30:42 +11:00
Sam Saffron 26c0199c01 DEV: update Rails to version 6.0.1
This version of Rails eliminates a monkey patch that is no longer needed!

Additionally it preps us for Ruby 2.7 support.
2019-11-08 16:56:30 +11:00
Vinoth Kannan ba5b78a348
FEATURE: support to mute all categories by default. (#8295)
Instead of enabling `suppress_from_latest` setting on many categories now we can enable `mute_all_categories_by_default` site setting. Then users should opt-in to categories for them to appear in the latest and categories pages.
2019-11-08 08:28:11 +05:30
Blake Erickson 2db2003187 DEV: Add deprecation warning of non-header based API auth
This change adds a message to the admin panel if it detects an api
requests that doesn't use the new header based authentication method.

The message is to warn people to switch to header based auth and links
to the api documention topic on meta for more info.
2019-11-07 17:06:13 -07:00
Robin Ward d74546f50e REFACTOR: Remove `Discourse.Topic` constant 2019-11-07 15:46:58 -05:00
Robin Ward 9d457fa51f REFACTOR: Remove `Discourse.TrustLevel` constant 2019-11-07 15:26:26 -05:00
David Taylor 60a235d128
DEV: Allow execute_command to receive a block (#8303)
This makes it easy to run multiple commands with the same keyword arguments. The main use is for using `chdir` across multiple commands. The `Dir.chdir` method is not concurrency safe because it switches the working directory of the entire process.
2019-11-07 15:47:16 +00:00
Arpit Jalan 6a417c308f FIX: include onebox default options in development environment 2019-11-07 15:42:53 +05:30
Arpit Jalan 00c406520e FEATURE: allow FinalDestination to use custom user agent for specific hosts 2019-11-07 14:47:51 +05:30
Krzysztof Kotlarek 17366d3bcc
FEATURE: notify tag watchers when tag was added to post (#8299)
Issue was mentioned in this [meta topic](https://meta.discourse.org/t/send-a-notification-to-watching-users-when-adding-tag/125314)

It is working well when category is changed because NotifyCategoryChange job already got that code:
```
if post&.topic&.visible?
  post_alerter = PostAlerter.new
  post_alerter.notify_post_users(post, User.where(id: args[:notified_user_ids]))
  post_alerter.notify_first_post_watchers(post, post_alerter.category_watchers(post.topic))
end
```

For NotifyTagChange job notify post users were missing so it worked only when your notification was set to `watching first post`
2019-11-07 08:20:15 +11:00
Penar Musaraj 61e35ce5dc DEV: Ignore renames when diffing theme changes (take 2)
Better fix than ca6adfbd
2019-11-06 14:45:18 -05:00
Neil Lalonde 1aa96a5da4 Version bump to v2.4.0.beta7 2019-11-06 12:31:03 -05:00
Penar Musaraj 0dfc594784 FIX: skip invalid URLs when checking for audio/video in search blurbs
Fixes 500 errors on search queries introduced in 580a4a8
2019-11-06 10:32:15 -05:00
Gerhard Schlager 61b1f9c36b FEATURE: Load translation overrides without JS `eval` 2019-11-05 19:16:38 +01:00
Penar Musaraj ca6adfbdd6 DEV: Ignore renames when diffing theme changes
Renamed assets are quite noisy when checking whether a remote theme has local changes.
2019-11-05 11:52:04 -05:00
David Taylor 52c5cf33f8
FEATURE: Overhaul of admin API key system (#8284)
- Allow revoking keys without deleting them
- Auto-revoke keys after a period of no use (default 6 months)
- Allow multiple keys per user
- Allow attaching a description to each key, for easier auditing
- Log changes to keys in the staff action log
- Move all key management to one place, and improve the UI
2019-11-05 14:10:23 +00:00
David Taylor ee5799805c
FIX: Allow theme translations to be accessed in initializers (#8285)
Previously theme translations were loaded along with other plugin API scripts. These run after pre-initializers and initializers when the app boots. This commit moves theme translation loading into pre-initializers, so their behaviour matches core translations more closely.
2019-11-05 11:54:12 +00:00
Neil Lalonde b6d140e4bd UX: tag input suggests required tags if none have been selected
This is a follow-up to the new feature that allows a category to
require a certain number of tags from a tag group. The tag input will
shows results from the required group if none have been chosen yet.
Once a require tag is selected, the tag input will include other
results as usual. Staff users can ignore this restriction, so the input
behaviour is unchanged for them.
2019-11-04 16:51:54 -05:00
Penar Musaraj 74869b8a7f FIX: Do not consider mobile app traffic as crawler visits
Followup to a4eb523a
2019-11-04 09:16:50 -05:00
Arpit Jalan b7327d2c34 UX: show user email address on "grant admin access" email and UI 2019-11-04 14:47:00 +05:30
Arpit Jalan 72bc0f82b9 FIX: no need to pass `cache` option in onebox 2019-11-04 10:59:28 +05:30
Joe ce0bac7a3d FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286)
* use image alt as a fallback when there's no title

* update spec

we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored.

The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 10:15:14 +11:00
Joffrey JAFFEUX 6f6a02645c
remove debug statement (#8289) 2019-11-03 09:46:14 +01:00
Vinoth Kannan 2cb805a683 DEV: Add option to keep onebox body content in post excerpt. 2019-11-02 17:14:04 +05:30
Daniel Waterworth 200cef90ea FIX: TopicQuery doesn't react well to subcategories without definitions
Also:

Move includes call higher which makes it possible to run all of the
intermediate queries for easier debugging.

Add tests for TagsController with categories in the path.
2019-11-02 08:34:43 +00:00
Robin Ward 785ebb674d REFACTOR: Split off raw handlebars helpers from compiler
This allows us to compile without Ember being present
2019-11-01 13:50:15 -04:00
David Taylor 4312bbe1e7
FIX: Do not load plugin CSS/JS assets when disabled (#8275)
Follow-up to 839916aa49 and 5bd6b70d98
2019-11-01 09:50:31 +00:00
Vinoth Kannan 72aa26c8c5
FEATURE: New site settings for default tags in user preferences. (#8283) 2019-11-01 12:40:13 +05:30
Martin Brennan f753643cb1
FIX: Unread topics not clearing when whisper is last post (#8271)
Meta thread: https://meta.discourse.org/t/cant-dismiss-unread-if-last-post-is-an-assign-or-whisper/131823/7

* when sending a whisper, the highest_staff_post_number is set
in the next_post_number method for a Topic, but the
highest_post_number is left alone. this leaves a situation
where highest_staff_post_number is > highest_post_number
* when TopicsBulkAction#dismiss_posts was run, it was only setting the topic_user
highest_seen_post_number using the highest_post_number from the topic, so if
the user was staff and the last post in a topic was a whisper
their highest seen number was not set, and the topic stayed unread

Found through testing that the bug wasn't to do with Assign/Unassign as they do not affect the post numbers, only whispering does.
2019-11-01 09:19:43 +10:00
Neil Lalonde d777844ed6 FEATURE: categories can require topics have a tag from a tag group
In a category's settings, the Tags tab has two new fields to
specify the number of tags that must be added to a topic
from a tag group. When creating a new topic, an error will be
shown to the user if the requirement isn't met.
2019-10-31 16:10:19 -04:00
David Taylor 1f88ecf6d8
FIX: Cache `Discourse.system_user` separately for each multisite tenant (#8276)
This was not causing any known issue, because the system user ID is always the same across all sites. However, we should cache this on a per-site basis to be safe.
2019-10-31 15:16:26 +00:00
Penar Musaraj 15b25547bb DEV: Cleanup misspelled TextHelper param 2019-10-31 09:32:42 -04:00
Penar Musaraj f8b72d9835 DEV: Refactor excluding audio/video URLs from search result blurbs
Followup to 580a4a82
2019-10-31 09:13:24 -04:00
Sam Saffron d8f7f363cd FEATURE: stop updating last_posted_at on users for messages and whispers
This ensures we only update last_posted_at which is user facing for non messages
and non whispers.

We still update this date for secure categories, we do not revert it for
deleted posts.
2019-10-31 09:01:26 +11:00
Penar Musaraj 580a4a827b Exclude audio/video URLs from search result blurbs
Displays translatable "[audio]" or "[video]" placeholders instead of ugly (and often long) URLs.
2019-10-30 13:07:16 -04:00
Justin DiRose c3f06943c7
FIX: Account for empty uploads directory upon backup restore (#8262)
This commit fixes a case where backup restores would fail if the uploads/default directory is empty.
2019-10-30 09:33:07 -05:00
Robin Ward 8d34f4bbd9 Revert "Revert Ember.run refactors"
This reverts commit fcb1ca52f9.
2019-10-30 09:48:24 -04:00
Krzysztof Kotlarek c32bd8ae48 FEATURE: Remove attachments and truncate raw field for incoming emails (#8253)
Adds the settings: 

raw_email_max_length, raw_rejected_email_max_length, delete_rejected_email_after_days. 

These settings control retention of the "raw" emails logs.

raw_email_max_length ensures that if we get incoming email that is huge we will truncate it removing uploads from the raw log.

raw_rejected_email_max_length introduces an even more aggressive truncation for rejected incoming mail. 

delete_rejected_email_after_days controls how many days we will keep rejected emails for (default 90)
2019-10-30 16:54:35 +11:00
Robin Ward fcb1ca52f9 Revert Ember.run refactors
This reverts commit 5ca60fcb6b.
2019-10-29 17:10:47 -04:00
Robin Ward cfa5d123a3 Temporary fix: Don't use ember imports in pretty text 2019-10-29 16:45:05 -04:00
romanrizzi 197ad8cd6e DEV: Run a faster test when warming up 2019-10-28 17:19:55 -03:00
Mark VanLandingham 4eb54f08b2
FEATURE: Site setting/UI to allow users to set their primary group (#8244)
* FEATURE: Site setting/ui to allow users to set their primary group

* prettier and remove logic from account template

* added 1 to 43 to make web_hook_user_serializer_spec pass
2019-10-28 12:46:27 -05:00
Leo McArdle e7ff6809a3 FEATURE: add SES spam header to recognised spam headers (#8254) 2019-10-28 12:46:53 -04:00
Daniel Waterworth 790e1b7191 FIX: TopicQuery category lookup by slug
If we are searching for categories by their slugs, it doesn't make sense
to include subcategories since a slug, by itself, does not necessarily
uniquely identify a subcategory.

Similarly, the empty string as a slug is not a good category identifier.
2019-10-28 15:29:26 +00:00
David Taylor eec464d8d1 SECURITY: Check permissions when autocompleting mentions 2019-10-28 11:01:47 +00:00
Dan Ungureanu 8ca5aad1e2
FIX: Reload plugin translations in development (#8243)
The default locale is :en_US, which is just a thin layer over :en. In
other words, :en_US has the :en locale as a fallback. When "en.yml" is
edited, only the :en locale is refreshed and :en_US becomes stale.

This commit ensures that there is a dependency on the fallback locales
too.
2019-10-26 12:55:54 +03:00
romanrizzi d76d0e75ec DEV: Move warmup inside docker rake task 2019-10-25 16:31:05 -03:00
Roman Rizzi 070a3dcf9b
FIX: When running the wizard and using a custom theme, fallback to the color_scheme name if the base_scheme_id is nil (#8236) 2019-10-25 09:29:51 -03:00
Gerhard Schlager 452d541b74 DEV: Add env variable to allow easier debugging of smoke tests
Set `DEBUG_NODE=1` when running `rake smoke:test` and use your favorite tool to debug the smoke tests. See https://nodejs.org/en/docs/guides/debugging-getting-started/ for more information.

The debugger will break at the beginning of the smoke tests when the env variable is set.
2019-10-24 17:48:15 +02:00
Krzysztof Kotlarek f530378df3 FIX: Restore for non-multisite is not raising an error on reconnect step (#8237)
That commit introduced a bug to the system: f69dacf979

Restore works fine for multisite, however, stopped working for non-multisite.

Reason for that was that `establish_connection` method got a check if the multisite instance is available:
```
    def self.instance
      @instance
    end

    def self.establish_connection(opts)
      @instance.establish_connection(opts) if @instance
    end
```
However, the reload method don't have that check
```
    def self.reload
      @instance = new(instance.config_filename)
    end
```

To solve it, let's ensure we are in a multisite environment before call reload
2019-10-24 11:46:22 +11:00
Neil Lalonde f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Vinoth Kannan 726bf37a12 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:50:10 +05:30
Vinoth Kannan 31577b2131 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:35:38 +05:30
Robin Ward 6287eccb35 REFACTOR: Remove `Ember.Controller` in favor of import 2019-10-23 13:06:54 -04:00
Krzysztof Kotlarek f34a0141c7 FIX: Correct path to ImportExport module (#8227)
During the move from Classic autoloader to Zeitwerk import_export module was moved to correct file name convention.
427d54b2b0 (diff-d896ec33b95afb7fae9f8bfe73d0580b)

Problem is that export/import is still using old path to require that module

Meta: https://meta.discourse.org/t/topic-and-category-export-import/38930/40
2019-10-23 17:27:14 +11:00
Krzysztof Kotlarek f69dacf979 FIX: Reconnect in restore process connects to correct DB (#8218)
Simplified flow of restore is like that
```
migrate_database
reconnect
extract_uploads
```

Problem with incorrect current database started with this fix https://github.com/discourse/discourse/commit/025d4ee91f4

Dump task is reconnecting to default database https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L429

And then, we are trying to reconnect to the original database with that code:
```
def reconnect_database
  log "Reconnecting to the database..."
  RailsMultisite::ConnectionManagement::establish_connection(db: @current_db)
end
```

This reconnect is not switching us back to correct database because of that check
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L181
Basically, it finds existing handler and it thinks that we are connected to correct DB and this step can be skipped.

To solve it, we can reload RailsMultisite::ConnectionManagement which creates a new instance of that class
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L38
2019-10-23 17:23:50 +11:00
Arpit Jalan 1e9d9d9346
FIX: respect `tl3 links no follow` setting (#8232) 2019-10-22 22:41:04 +05:30
Mark VanLandingham e5311ab645 DEV: Import DiscourseRoute rather than Discourse.Route (#8225)
* DEV: Discourse.Route -> DiscourseRoute with import

* ran prettier on all routes

* prettiered one more file
2019-10-22 09:46:10 -04:00
Nacho Caballero d5121e5ddb FIX: Add common HTML5 media extensions to onebox audio and video tags (#8216) 2019-10-21 12:10:40 -04:00
Vinoth Kannan 5e55e75aed FIX: 'only_hidden_tags_changed?' method returned 'true' even when tags are not changed.
While editing the first post it does't bumped the topic when the new post revision created. Because we wrongly assumed that the hidden tags are changed even when no tags are updated.
2019-10-21 17:57:31 +05:30
Daniel Waterworth 55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Joffrey JAFFEUX 8f51445e9f
FIX: fixes plugin generator with zeitwerk (#8220) 2019-10-21 09:24:14 +02:00
Krzysztof Kotlarek 858cf5836c
FIX: update Redis gem to version 4.1.3
I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

Redis gem is manipulating Redis config https://github.com/redis/redis-rb/blob/master/lib/redis/client.rb#L95
therefore we cannot pass the frozen config object.

Pass of the copy of the object is protecting original config
2019-10-21 09:59:24 +11:00
tshenry 95114f87ae
Improve spam_hosts copy (#8203)
- Increase size of the reviewable's conversation excerpt to prevent truncation of the new copy
- Remove the `domain` parameter from the `flag_linked_posts_as_spam` method in the user model since it is no longer needed
- Remove the `domain` interpolation variable from all translation files
- Add "All posts from this user that include links should be reviewed." to server.en.yml for added clarity on why the posts entered the queue
2019-10-18 09:31:15 -07:00
Rafael dos Santos Silva 2304dcf993 FIX: Do not truncate encoded slugs
Trying to truncate encoded slugs will mean that we have to keep the URL
valid, which can be tricky as you have to be aware of multibyte
characters.

Since we already have upper bounds for the title, the slug won't grow
for more than title*6 in the worst case. The slug column in the topic
table can store that just fine.

Added a test to ensure that a generated slug is a valid URL too, so we
don't introduce regressions in the future.
2019-10-17 13:38:31 -03:00
David Taylor 815dcb25ba UX: Improve new GitHub onebox rendering in emails
Unfortunately SVGs are not supported in most email clients, so remove them
2019-10-17 10:54:04 +01:00
Sam Saffron f5d1aff8dd FEATURE: experimental hidden setting for draft backups
Under exceptional situations the automatic draft feature can fail.

This new **hidden, default off** site setting
`backup_drafts_to_pm_length` will automatically backup any draft that is
saved by the system to a dedicated PM (originating from self)

The body of that PM will contain the text of the reply.

We can enable this feature strategically on sites exhibiting issues to
diagnose issues with the draft system and offer a recourse to users who
appear to lose drafts. We automatically checkpoint these drafts every 5
minutes forcing a new revision each 5 minutes so you can revert to old
content.

Longer term we are considering automatically enabling this kind of feature
for extremely long drafts where the risk is really high one could lose
days of writing.
2019-10-17 16:58:21 +11:00
Sam Saffron ae2a56999e Revert "FIX: update Redis gem to version 4.1.3 (#8197)"
This reverts commit ab74a50d85.

We really want to upgrade redis, but discovered some edge cases
around failover we need to test.

Holding off on the upgrade till a bit more testing happens
2019-10-17 11:41:46 +11:00
Krzysztof Kotlarek ab74a50d85 FIX: update Redis gem to version 4.1.3 (#8197)
* FIX: update Redis gem to version 4.1.3

I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

* FIX: get rid of redis freedom patch
2019-10-17 08:49:23 +11:00
Neil Lalonde 5ef49692e0 FIX: tag cannot be used if it belongs to two tag groups with parent tag
If two tag groups exist with a mandatory parent tag, and one tag is
added to both tag groups, then the tag couldn't be used on any topics.
2019-10-16 14:28:04 -04:00
Sam Saffron d5d8db7fa8 FEATURE: improve honeypot and challenge logic
This feature amends it so instead of using one challenge and honeypot
statically per site we have a rotating honeypot and challenge value which
changes every hour.

This means you must grab a fresh copy of honeypot and challenge value once
an hour or account registration will be rejected.

We also now cycle the value of the challenge when after successful account
registration forcing an extra call to hp.json between account registrations

Client has been made aware of these changes.

Additionally this contains a JavaScript workaround for:
https://bugs.chromium.org/p/chromium/issues/detail?id=987293

This is client side code that is specific to Chrome user agent and swaps
a PASSWORD type honeypot with a TEXT type honeypot.
2019-10-16 16:53:44 +11:00
David Taylor ac2cd07775 DEV: Add discourse-restricted-replies as official plugin 2019-10-15 22:49:46 +01:00
Vinoth Kannan f92a6f7ac5 FIX: 'local_cdn_url' method should work for local relative urls too. 2019-10-14 11:39:16 +05:30
Vinoth Kannan e4fe864c0b FIX: use upload's cdn url in composer preview if available. 2019-10-14 08:47:15 +05:30
Roman Rizzi 01bc465db8
DEV: Split max decompressed setting for themes and backups (#8179) 2019-10-11 14:38:10 -03:00
Rafael dos Santos Silva 76ab0350f1
FIX: Properly encoded slugs when configured to (#8158)
When an admin changes the site setting slug_generation_method to
encoded, we weren't really encoding the slug, but just allowing non-ascii
characters in the slug (unicode).

That brings problems when a user posts a link to topic without the slug, as
our topic controller tries to redirect the user to the correct URL that contains
the slug with unicode characters. Having unicode in the Location header in a
response is a RFC violation and some browsers end up in a redirection loop.

Bug report: https://meta.discourse.org/t/-/125371?u=falco

This commit also checks if a site uses encoded slugs and clear all saved slugs
in the db so they can be regenerated using an onceoff job.
2019-10-11 12:38:16 -03:00
Régis Hanol 9a81cb9e55 FIX: ensure we remove tempfiles from disk when creating an upload
Follow-up to 46d12c5ad3
2019-10-11 11:13:10 +02:00
David Taylor 67787799bb DEV: Allow plugins to mark user custom fields as editable only by staff
This adds a staff_only parameter to the register_editable_user_custom_field API. The default is false, to maintain backwards compatibility.
2019-10-11 09:57:55 +01:00
Nacho Caballero 7d2f5240d9 FIX: Show a correct diff when editing consecutive paragraphs (#8177) 2019-10-11 03:50:37 -04:00
Neil Lalonde 09021c35a3 Version bump to v2.4.0.beta6 2019-10-10 11:46:28 -04:00
romanrizzi fa66290b61 FIX: Don't fail when there's no directory to strip 2019-10-10 11:21:36 -03:00
Joffrey JAFFEUX c5326682d6 DEV: improves speed of live reload css in core and plugins (#8161)
This corrects live refresh of CSS when working on plugins (and core) in many cases live refresh was not working.
2019-10-10 16:10:23 +11:00
Sam Saffron 5aaf7e3316 FIX: during concurrent emails generation renderer should not be reused
Our instance used for template rendering needs a lock to ensure there is
no race condition where rendering happens on 2 threads at the same time.

This can lead to local poisoning which can cause unexpected results in
emails
2019-10-10 08:50:48 +11:00
Roman Rizzi 5357ab3324
SECURITY: Safely decompress backups when restoring. (#8166)
* SECURITY: Safely decompress backups when restoring.

* Fix tests and update theme_controller_spec to work with zip files instead of .tar.gz
2019-10-09 11:41:16 -03:00
Mark VanLandingham 9b4aba0d39
DEV: support --fail-fast in bin/turbo_rspec (#8170)
* [WIP] - default turbo spec env to test

* FEATURE: support for --fast-fail in bin/turbo_rspec

* fast-fail -> fail_fast to match rspec

* Moved thread killing outside of fail-fast check

* Removed failure_count incrementation from fast_fail_met
2019-10-09 09:40:06 -05:00
Roman Rizzi 10478cb395
DEV: Bump uglifyjs (#7834)
* Rewrite uglifyjs command to work with 3.x

* Use ES5 syntax in plain JS files

* Use the older command if uglifyJS V2.x is installed
2019-10-09 10:02:49 -03:00
Régis Hanol 349c1cd085 FIX: remove site setting 'shadowed-by-global' option (#8061) 2019-10-08 12:43:26 -04:00
Dan Ungureanu fdb1d3404c
FEATURE: Add site setting to show more detailed 404 errors. (#8014)
If the setting is turned on, then the user will receive information
about the subject: if it was deleted or requires some special access to
a group (only if the group is public). Otherwise, the user will receive
a generic #404 error message. For now, this change affects only the
topics and categories controller.

This commit also tries to refactor some of the code related to error
handling. To make error pages more consistent (design-wise), the actual
error page will be rendered server-side.
2019-10-08 14:15:08 +03:00
David Taylor d2bceff133
FEATURE: Use full page redirection for all external auth methods (#8092)
Using popups is becoming increasingly rare. Full page redirects are already used on mobile, and for some providers. This commit removes all logic related to popup authentication, leaving only the full page redirect method.

For more info, see https://meta.discourse.org/t/do-we-need-popups-for-login/127988
2019-10-08 12:10:43 +01:00
Krzysztof Kotlarek 3f9673f23c FIX: solution for pending migrations for bin/turbo_rspec
Currently, if you try to run `./bin/turbo_rspec` you will got that error `There are pending migrations, run rake parallel:migrate`

Reason for that is that command is running in `development` mode which includes plugins migration files in ActiveRecord::Migrator.migrations_paths:
```
["db/migrate",
 "/home/lis2/projects/discourse/plugins/discourse-details/db/migrate",
 "/home/lis2/projects/discourse/plugins/discourse-details/db/post_migrate",
 "/home/lis2/projects/discourse/plugins/discourse-local-dates/db/migrate",
 "/home/lis2/projects/discourse/plugins/discourse-local-dates/db/post_migrate",
...
]
```

A workaround solution would be to run the command with the TEST environment like `RAILS_ENV=test ./bin/turbo_rspec`

I want to propose in this PR to override migration_paths to check only Discourse core migrations.
2019-10-08 10:32:40 +00:00
Sam Saffron 586dfcc795 PERF: avoid regex for hot path
Camelize is called quite a lot in zeitwerk, avoid using a regex here which
is far slower than using ends_with?
2019-10-08 13:43:54 +11:00
Sam Saffron 8d5f47dded PREF: optimise preloading application
We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)

This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes

1. We attempt to localize a string on each site, ensuring we warmup
the i18n

2. We preload all our templates (compiling .erb to class)

3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
2019-10-07 00:33:37 -04:00
Sam Saffron 71ea4ad7fc PERF: reuse renderer when rendering email templates
Previous to this fix we were leaking methods on the internal action view
template class per render.

This caused email generation to be very low and a steady memory leak in the
application in sidekiq when sending out emails

The behavior change is new to Rails 6 so this fix does not need to be
backported into stable.
2019-10-06 23:57:03 -04:00
Jarek Radosz bfe0bc0cbd
FIX: Check for category conflicts in SiteSetting validations (#8137)
It was possible to add a category to more than one default group, e.g. "default categories muted" and "default categories watching first post".

The bug was caused by category validations inadvertently comparing strings and numbers.
2019-10-06 20:50:07 +02:00
Jarek Radosz 87be6fe98c
DEV: Add a plugin incompatibility message (#8151)
* DEV: Add a plugin incompatibility message

* Extract the plugin_initialization_guard
2019-10-06 20:47:33 +02:00
Joffrey JAFFEUX e23b9051ff
DEV: restart watcher after css change notifier crashed (#8157) 2019-10-04 13:20:51 -04:00
Régis Hanol 46d12c5ad3 FIX: properly downsize image on upload
Overwriting the same file with 'convert' is not always working as expected.
Adding a temporary file as the destination of the downsize makes this operation much more reliable.

Also switched to using (the more aggressive) 50% resize instead of halving the number of pixels.
2019-10-04 16:22:57 +02:00
Sam Saffron 6471fbd453 FIX: maxminddb:get task no longer working
Following our zeitwerk upgrade we needed to be more explicit about the
require
2019-10-04 09:47:33 +10:00
Robin Ward a404cab509 The generated plugin should also freeze strings 2019-10-03 11:43:36 -04:00
Robin Ward 32a11a93ac Update the template for plugin generator to be rubocop compliant 2019-10-03 11:41:53 -04:00
Roman Rizzi 10565e4623
SECURITY: Safely decompress files. (#8124)
* FEATURE: Adds an extra protection layer when decompressing files.

* Rename exporter/importer to zip importer. Update old locale

* Added a new composite class to decompress a file with multiple strategies

* Set max file size inside a site setting

* Ensure that file is deleted after compression

* Sanitize path and files before compressing/decompressing
2019-10-03 10:19:35 -03:00
Justin DiRose 63fabdb6f2 FIX: first post true if user creates topic first (#8139)
Reported here: https://meta.discourse.org/t/user-was-banned-for-posting-a-reply-within-3-seconds/128823/12
The problem here is the user could create a topic, then reply and get
silenced on the second time (though technically their first post) for being below the
min_first_post_typing_time threshold.
2019-10-02 15:51:40 -04:00
Daniel Waterworth 1fdba2c5b2 FIX: Harden DistributedMutex
Threadsafety

  Since we use the same redis connection in multiple threads, a rogue
  transaction in another thread can trample the connection state
  (watched keys) that we need to acquire and release the lock properly.

  This is fixed by preventing other threads from using the connection
  when we are performing these actions.

Off-by-one error

  A distributed mutex is now consistently determined to be expired if
  the current time is strictly greater than the expire time.

Unwatch before transaction

  Since the redis connection is used by so much of the code, it is
  difficult to ensure that any watched keys have been cleared. In order
  to defend against this rogue connection state, an unwatch has been
  added before locking and unlocking.

Logging

  Hopefully this log message is more clear.
2019-10-02 13:00:41 +00:00
Michael Brown dbe0111822 FEATURE: allow UploadRecovery to be run on a single post (#8094) 2019-10-02 14:57:36 +10:00
Krzysztof Kotlarek 35b1185a08 FIX: Revert Demon::DemonBase back to Demon::Base (#8132)
I introduced DemonBase because I had got some conflict between `demon/base.rb` and `jobs/base.rb`, however, to not rename base class, it is possible to use regex on absolute path in Zeitwerk custom inflector.
2019-10-02 14:54:08 +10:00
Sam Saffron 55ee9abecb DEV: clean up dependencies in spec
Follow up on zeitwork we needed to be a bit more explicit about a few
dependencies internally.

On certain orders the test suite could fail.
2019-10-02 14:50:54 +10:00
Sam Saffron 3f6af54fe1 DEV: minor fixes related to zeitwerk
A couple of specs stopped working post zeitwerk. Adding missing require
corrected inheritance to avoid a circular ref.
2019-10-02 14:28:18 +10:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Martin Brennan 68d35b14f4 FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099)
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).

Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
2019-10-01 19:08:41 -07:00
Joffrey JAFFEUX e5af03be01 DEV: prevents attemps to redefine THEME_TARGETS constant (#8121) 2019-10-02 10:54:31 +10:00
Krzysztof Kotlarek f331b5eab2 FEATURE: topic title is validated for blocked words (#8127)
Currently, the topic is only validated for censored words and should be validated for blocked words as well.

Blocked word validation is now used by both Post and Topic. To avoid code duplication, I extracted blocked words validation code into separate Validator, and use it in both places.

The only downside is that even if the topic contains blocked words validation message is saying "Your post contains a word that's not allowed: tomato" but I think this is descriptive enough.
2019-10-02 10:38:34 +10:00
Neil Lalonde 279a94da90 Version bump to v2.4.0.beta5 2019-10-01 16:51:36 -04:00
Gerhard Schlager 9e4fb262cf FIX: Respect unicode whitelist when suggesting username 2019-10-01 20:33:09 +02:00
Joffrey JAFFEUX cb8fa46970
DEV: prevents populate task to crash (#8111)
Generated emails/names/sentences were crashing with the following error:

```
Can not transliterate strings with ASCII-8BIT encoding
```
2019-09-22 05:18:48 -07:00
Daniel Waterworth 563253e9ed FIX: Fix options given to per-minute rate limiter
Previously the options for the per-minute and per-10-second rate
limiters were the same.
2019-09-20 10:48:59 +01:00
Robin Ward 3c6a5836c2 FIX: Sensitivity did not work by default
Forums without previously calculated scores would return the same values
for low/medium/high sensitivity. Now those are scaled based on the
default value.

The default value has also been changed from 10.0 to 12.5 based on
observing data from live discourse forums.
2019-09-19 13:26:17 -04:00
Penar Musaraj c6cfbebf1f
FIX: ignore min_trust_to_send_messages when messaging groups (#8104)
This means that TL0 users can message groups with "Who can message this
group?" set to "Everyone".

It also means that members of a group with "Who can message this
group?" set to "members, moderators and admins" can also message the
group, even when their trust level is below min_trust_to_send_messages.
2019-09-18 15:23:13 -04:00
Robin Ward 7ae071282a FIX: Only apply post hide logic to flag actions 2019-09-18 09:39:09 -04:00
David Taylor 479fdaaea1
DEV: Allow specifying button class in reviewable action definitions (#8093)
This avoids the need for using `@extend` in SCSS, which can be problematic in plugins

For context, see https://review.discourse.org/t/fix-make-compatible-with-debundled-plugin-css-assets-feature/5297/7
2019-09-18 11:28:59 +01:00
Sam Saffron 1ca257be79 DEV: db:migrate no longer works after db:schema:load
In Rails 6 due to internal changes, the following sequence no longer works:

```
RAILS_ENV=test bin/rake db:migrate
RAILS_ENV=test bin/rake db:schema:dump
dropdb discourse_test
createdb discourse_test
RAILS_ENV=test bin/rake db:schema:load
RAILS_ENV=test bin/rake db:migrate
```

What appears to be happening is that our tracking of plugin migrations is
being missed on schema:dump or load.

A more comprehensive fix restoring schema:dump / load support will be
investigated.
2019-09-18 13:17:49 +10:00
Penar Musaraj 3debdc8131 SECURITY: XSS when oneboxing user profile location field
The XSS here is only possible if CSP is disabled. Low impact since CSP is enabled by default in SiteSettings.
2019-09-17 16:12:50 -04:00
Robin Ward c3bbf643b1 FIX: Put back the TL3 -> TL0 spam thing
We talked about it and decided it's still relevant in the score world.
2019-09-17 16:04:46 -04:00
Robin Ward 4cd620e36e Remove special cases for flagging
Prior to the new review queue there were a couple special cases where
posts would be auto hidden:

* If a TL3 or above flagged a TL0 post as spam
* If a TL4 or above flagged a non-staff, non-TL4 post as spam, inappropriate or off
topic.

These cases are now removed in favour of the scoring system.
2019-09-17 13:44:15 -04:00
David Taylor e1e8cac58f FIX: Correct theme SCSS error handling 2019-09-17 10:20:32 +01:00
David Taylor e74f851728 FIX: Live reload plugin stylesheets when editing in development 2019-09-17 09:54:59 +01:00
David Taylor 3da9b99dbf FIX: Live reload plugin stylesheets when the color scheme changes 2019-09-17 09:54:55 +01:00
David Taylor 081c36a459 FIX: Do not include theme variables in plugin SCSS, and fix register_css 2019-09-17 09:54:52 +01:00
Sam Saffron 445d305154 DEV: initial migration can fail
db:migrate can issue translations due to module loading localizing
end user messages

This allows db:migrate to work even when db is blank
2019-09-17 13:38:01 +10:00
Sam Saffron 025d4ee91f FIX: Rails 6 multisite migrations and plugin migrations
Prior to this change plugin migrations were not working and multisite
migrations not working.

Rails internals changed so we need to account for it.

Specifically semantics of `db:migrate` in rails changed so it is sort of
a "multisite:migrate".
2019-09-13 09:17:54 +10:00
Roman Rizzi 568232052e
DEV: Remove FlagQuery class and old code (#8064) 2019-09-12 13:21:33 -03:00
David Taylor 67a98946b8 FIX: Do not log 'pull_hotlinked_images' edits in the staff action log 2019-09-12 15:55:45 +01:00
Sam Saffron 082f59842d DEV: bind for thread local vars should yield block
followup on #015051ec without this improvement monkey patch does not work.

bind should yield the block it is passed for it to work.
2019-09-12 17:57:57 +10:00
Sam Saffron 015051ecaf PERF: avoid spinning a thread each time we close a connection
This is a temporary workaround for the issue in https://github.com/rails/rails/pull/36949

Discussing a proper fix in Rails with the Rails team.

Prior to this fix we were spinning up a thread every time we closed a connection
to the db.
2019-09-12 17:34:04 +10:00
Krzysztof Kotlarek 32b8a2ccff DEV: Upgrade Discourse to Rails 6 (#8083)
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
2019-09-12 10:41:50 +10:00
Gerhard Schlager 56c6e8e961 FIX: POP3 doesn't work with TLS 1.3 2019-09-11 18:43:19 +02:00
Michael Brown 503a11cc88 FIX: inline_uploads and subfolder (#8076)
* FIX: inline_uploads and subfolder

* if subfolder, also look for images with a path containing
  cdn_url + relative_url_root

* FIX: migrate_to_s3 task and subfolder
2019-09-11 11:50:48 +10:00
Bianca Nenciu 0d22beb81d
FIX: Improve Onebox detection (#8019)
Follow-up to 7c83d2eeb2.
2019-09-10 13:59:48 +03:00
Kyle Zhao 9b10a78d82 FEATURE: Quick access panels in user menu (#8073)
* Extract QuickAccessPanel from UserNotifications.

* FEATURE: Quick access panels in user menu.

This feature adds quick access panels for bookmarks and personal
messages. It allows uses to browse recent items directly in the user
menu, without being redirected to the full pages.

* REFACTOR: Use QuickAccessItem for messages.

Reusing `DefaultNotificationItem` feels nice but it actually requires a
lot of extra work that is not needed for a quick access item.

Also, `DefaultNotificationItem` shows an incorrect tooptip ("unread
private message"), and it is not trivial to remove / override that.

* Use a plain JS object instead.

An Ember object was required when `DefaultNotificationItem` was used.

* Prefix instead suffix `_` for private helpers.

* Set to null instead of deleting object keys.

JavaScript engines can optimize object property access based on the
object’s shape. https://mathiasbynens.be/notes/shapes-ics

* Change trivial try/catch to one-liners.

* Return the promise in case needs to be waited on.

* Refactor showAll to a link with href

* Store `emptyStatePlaceholderItemText` in state.

* Store items in Session singleton instead.

We can drop `staleItems` (and `findStaleItems`) altogether. Because
`(old) items === staleItems` when switching back to a quick access
panel.

* Add `limit` parameter to the `user_actions` API.

* Explicitly import Session instead.
2019-09-09 11:03:57 -04:00
Robin Ward 800e49f16e Version bump to 2.4.0.beta4 2019-09-06 16:08:03 -04:00
Gerhard Schlager 631315624d FIX: Topics with muted tag didn't show up when filtering by category and tag
It also removes the redundant `filter` parameter. Previously URLs looked like this:

```
http://example.com/tags/c/some-category/muted-tag/l/latest.json?filter=tags/c/some-category/muted-tag/l/latest
```

But it looks like the `filter` parameter was only used to find out if topics with a muted tag should be removed or not. But the same thing can be accomplished by using the first tag ID. The following URL looks a lot cleaner.

```
http://example.com/tags/c/some-category/muted-tag/l/latest.json
```
2019-09-06 20:38:03 +02:00
Rimian Perkins 6bbd83067d FEATURE: New post editing period for >= tl2 users (#8070)
* FEATURE: Add tl2 threshold for editing new posts

* Adds a new setting and for tl2 editing posts (30 days same as old value)
* Sets the tl0/tl1 editing period as 1 day

* FIX: Spec uses wrong setting

* Fix site setting on guardian spec

* FIX: post editing period specs

* Avoid shared examples
* Use update_columns to avoid callbacks on user during tests
2019-09-06 07:44:12 -04:00
Neil Lalonde e90636eadc Version bump to v2.4.0.beta3 2019-09-04 11:16:01 -04:00
Gerhard Schlager 874985db71 DEV: Ignore duplicate prefixes when generating release notes 2019-09-04 15:45:48 +02:00
Sam Saffron ed00f35306 FEATURE: improve performance of anonymous cache
This commit introduces 2 features:

1. DISCOURSE_COMPRESS_ANON_CACHE (true|false, default false): this allows
you to optionally compress the anon cache body entries in Redis, can be
useful for high load sites with Redis that lives on a separate server to
to webs

2. DISCOURSE_ANON_CACHE_STORE_THRESHOLD (default 2), only pop entries into
redis if we observe them more than N times. This avoids situations where
a crawler can walk a big pile of topics and store them all in Redis never
to be used. Our default anon cache time for topics is only 60 seconds. Anon
cache is in place to avoid the "slashdot" effect where a single topic is
hit by 100s of people in one minute.
2019-09-04 17:18:32 +10:00
Sam Saffron dc9110cc43 FEATURE: track date api key was last used
Start tracking the date an api key was last used. This has already been
the case for user_api_keys.

This information can provide us with the ability to automatically expire
unused api keys after N days.
2019-09-03 18:11:30 +10:00
Sam Saffron b9954b53bb FIX: report cached controller and action to loggers
Previously we would treat all cached hits in anon cache as "other"

This hinders analysis of cache performance and makes logging inaccurate
2019-09-03 10:55:16 +10:00
Sam Saffron 08743e8ac0 FEATURE: anon cache reports data to loggers
This allows custom plugins such as prometheus exporter to log how many
requests are stored in the anon cache vs used by the anon cache.

This metric allows us to fine tune cache behaviors
2019-09-02 18:45:35 +10:00
Daniel Waterworth afeb7e4b55 DEV: Use recorded runtimes in turbo_rspec when running the whole test suite 2019-09-02 07:25:41 +01:00
Régis Hanol 75eebc904e FEATURE: new 'Discourse-Render' HTTP header 2019-08-30 20:45:18 +02:00
Roman Rizzi e515324afa
Feature/Fix: Flagged posts user notifications (#8041)
* FIX: User should get notified when a post is deleted

* FEATURE: Notify posters when restoring flagged posts

* Fix typo

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Improve tests
2019-08-30 10:27:52 -03:00
David Taylor e2449f9f23 Revert "Revert "Revert "FIX: Heartbeat check per sidekiq process (#7873)"""
This reverts commit c3497559be.
2019-08-30 11:26:16 +01:00
David Taylor be96c4478e
FEATURE: Login with Discord (#8053)
This migrates the functionality of discourse-plugin-discord-auth into core. 

The plugin will automatically disable itself when core is updated: fd0867844d?w=1

For setup instructions, visit https://meta.discourse.org/t/configuring-discord-login-for-discourse/127129
2019-08-30 10:54:19 +01:00
Sam Saffron e9cf0de588 FEATURE: do not replace &harr; with an emoji
&harr; technically can be an emoji, but usually will not be used in that way
2019-08-30 15:06:23 +10:00
Sam Saffron c3497559be Revert "Revert "FIX: Heartbeat check per sidekiq process (#7873)""
This reverts commit e805d44965.
We now have mechanisms in place to ensure heartbeat will always
be scheduled even if the scheduler is overloaded per: 098f938b
2019-08-30 10:12:10 +10:00
Roman Rizzi ebb389ef8a
UX: Read indicator improvements. (#8049)
* The read indicator now shows up when no member has read the last post of the topic (written by a non-member)
* The read indicator works on mobile and receives live updates from message bus
* The icon we display in the topic list was changed
* Added a title to the indicator to indicate its purpose when hovering over it
2019-08-29 12:03:43 -03:00
romanrizzi 1e89939383 DEV: Added wrapper to signal that NewPostManager#add_plugin_payload_attribute is part of the public API 2019-08-29 11:58:19 -03:00
Daniel Waterworth 6b9784cf8a FIX: Made turbo_rspec display errors in shared groups correctly 2019-08-29 12:41:14 +01:00
Daniel Waterworth 15c02c03c7 DEV: Split out multisite tests in bin/turbo_rspec
* A new process is started that just runs the multisite tests
 * The other processes are instructed to exclude the multisite tests
2019-08-29 11:47:58 +01:00
Sam Saffron 0d5d478146 PERF: avoid filtering shared drafts when not used
In some very specific cases (large sites) shared drafts can introduce a
performance hit due to the mechanism used to filter out topics

This avoids the entire process when shared drafts are not enabled
2019-08-29 11:37:20 +10:00
Régis Hanol 00dbc260d3 FIX: ensure avatar sizes are integers 2019-08-27 17:03:20 +02:00
Roman Rizzi 7c741fa0d6
FEATURE: Publish read state on group messages. (Originally introduced in #7989) (#8025)
* Revert "Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)""

This reverts commit 36425eb9f0.

* Fix: Show who read only if the attribute is enabled

* PERF: Precalculate the last post  readed by a group member

* Use book-reader icon instear of far-eye

* FIX: update topic groups correctly

* DEV: Tidy up read indicator update on write
2019-08-27 09:09:00 -03:00
OsamaSayegh e805d44965 Revert "FIX: Heartbeat check per sidekiq process (#7873)"
This reverts commit 340855da55.
2019-08-27 11:56:23 +00:00
Gerhard Schlager aecadcb267 FIX: Overriding _MF texts didn't work for en_US 2019-08-27 13:17:07 +02:00
Sam Saffron bd5fa1737d FIX: add_to_serializer not correctly accounting for inheritance chains
This is a very long standing bug we had, if a plugin attempted to amend a
serializer core was not "correcting" the situation for all descendant classes
this often only showed up in production cause production eager loads serializers
prior to plugins amending them.

This is a critical fix for various plugins
2019-08-27 18:21:53 +10:00
Osama Sayegh 340855da55
FIX: Heartbeat check per sidekiq process (#7873)
* FIX: Heartbeat check per sidekiq process

* Rename method

* Remove heartbeat queues of previous bootups

* Regis feedback

* Refactor before_start

* Update lib/demon/sidekiq.rb

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Update lib/demon/sidekiq.rb

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Expire redis keys after 3600 seconds

* Don't use redis to store the list of queues
2019-08-26 09:33:49 +03:00
Sam Saffron 8cea78c833 Revert "FEATURE: Protect against replay attacks when using TLS 1.3 0-RTT (#8020)"
This reverts commit 39c31a3d76.

Sorry about this, we have decided againse supporting 0-RTT directly in
core, this can be supported with similar hacks to this commit in a
plugin.

That said, we recommend against using a 0-RTT proxy for the Discourse
app due to inherit risk of replay attacks.
2019-08-26 08:56:49 +10:00
Rafael dos Santos Silva 39c31a3d76
FEATURE: Protect against replay attacks when using TLS 1.3 0-RTT (#8020) 2019-08-23 11:52:47 -03:00
Osama Sayegh 68708db721 DEV: `S3Inventory#unsorted_files` should always return an array (#8034) 2019-08-23 17:59:31 +10:00
Gerhard Schlager 1f118b1309 FEATURE: Allow plugins to manipulate site settings during backup restore 2019-08-22 22:41:26 +02:00
Vinoth Kannan 41f22946c3 FIX: properly load desktop and mobile only plugin css assets.
5bd6b70d98
2019-08-22 08:39:10 +05:30
Daniel Waterworth 6924f1ab15 FIX: Race-condition in fallback handlers (#8005)
Calling `verify_master` in multiple threads simultaneously would cause
multiple threads to be spawned.
2019-08-21 15:47:44 +02:00
Arpit Jalan 75f37ac16a Rename `lazyYT` plugin directory name to `lazy-yt` 2019-08-21 14:35:14 +05:30
Sam Saffron 719a93c312 FEATURE: treat theme_uploads as settings in JavaScript
This change allows themes and components access to theme assets.

This means that inside theme js you can now get the URL for an asset with:

```
settings.theme_uploads.name
```
2019-08-21 16:51:10 +10:00
Vinoth Kannan 5bd6b70d98
DEV: debundle plugin css assets and don't load if disabled (#7646) 2019-08-20 22:09:52 +05:30
romanrizzi 36425eb9f0 Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)"
This reverts commit 5dda5c2f7c.
2019-08-20 13:29:22 -03:00
Roman Rizzi 5dda5c2f7c
FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)
* Reenable: "FEATURE: Publish read state on group messages. (#7989)"

This reverts commit 67f5cc1ce8.

* FIX: Read indicator only appears when the group setting is enabled
2019-08-20 11:57:25 -03:00
romanrizzi 67f5cc1ce8 Revert "FEATURE: Publish read state on group messages. (#7989)"
This reverts commit 1630dae2db.
2019-08-20 10:24:34 -03:00
Roman Rizzi 1630dae2db
FEATURE: Publish read state on group messages. (#7989)
* Enable or disable read state based on group attribute

* When read state needs to be published, the minimum unread count is calculated in the topic query. This way, we can know if someone reads the last post

* The option can be enabled/disabled from the UI

* The read indicator will live-updated using message bus

* Show read indicator on every post

* The read indicator now shows read count and can be expanded to see user avatars

* Read count gets updated everytime someone reads a message

* Simplify topic-list read indicator logic

* Unsubscribe from message bus on willDestroyElement, removed unnecesarry values from post-menu, and added a comment to explain where does minimum_unread_count comes from
2019-08-20 09:46:57 -03:00
Blake Erickson 893d30fb92 REVERT: External auth when redeeming invites
Reverting this commit:

87a0a6664e

because I'm extracting all of this logic into a plugin instead.
2019-08-16 12:44:14 -06:00
Sam Saffron 9a9e31f927 FIX: properly set notification levels on group invite
Previously we relied on side effects to set tracking state correctly
when inviting groups to messages

Also has a minor optimisation in that we use pluck instead of pulling in
full record
2019-08-16 18:23:51 +10:00
Robin Ward 23367e79ea
FEATURE: Embed topics list on remote sites via Javascript API. (#8008)
This adds support for a `<d-topics-list>` tag you can embed in your site
that will be rendered as a list of discourse topics. Any attributes on
the tag will be passed as filters. For example:

`<d-topics-list discourse-url="URL" category="1234">` will filter to category 1234.

To use this feature, enable the `embed topics list` site setting. Then
on the site you want to embed, include the following javascript:

`<script
src="http://URL/javascripts/embed-topics.js"></script>`

Where `URL` is your discourse forum's URL.

Then include the `<d-topics-list discourse-url="URL">` tag in your HTML document and it will
be replaced with the list of topics.
2019-08-15 13:41:06 -04:00
David Taylor fafc6bcde0 FIX: Clear banner topic cache after remapping 2019-08-15 11:24:20 +01:00
Sam Saffron d05b724c4e FIX: correct race condition loading library
In some very rare cases CssParser could be loaded but CssParser::Parser not
this ensures we check for the actual constant we plan to call for concurrent
digest generations
2019-08-15 17:16:41 +10:00
Sam Saffron e53a171916 FIX: hold s3 related distributed locks longer
These operations are pretty expensive and can take multiple minutes due to
networking.

Hold distributed mutex for much longer.
2019-08-15 11:48:44 +10:00
Roman Rizzi 79957706b1
FIX: Store custom attributes that are needed by plugins in queuedpost payload (#8009) 2019-08-14 15:02:59 -03:00
Vinoth Kannan 88359b0f16
FEATURE: add support for group members visibility level (#8004)
There are 5 visibility levels (similar to group visibility)

public (default)
logged-in users
members only
staff
owners

Admins & group owners always have visibility to group members.
2019-08-14 19:00:04 +05:30
David Taylor f4aa6096ab FIX: Convert omniauth authenticator names to symbols before comparing
This is necessary because some auth plugins define their name as a string
2019-08-14 12:57:11 +01:00
David Taylor 5755d3886f FIX: Do not raise exception if the authenticator email is missing
Followup to 296cdc53ee
2019-08-14 12:08:59 +01:00
Blake Erickson 296cdc53ee FIX: Downcase email coming back from auth-provider 2019-08-13 10:53:50 -06:00
Vinoth Kannan 9919ee1900 FIX: remove the tmp inventory files after the s3 uploads check. 2019-08-13 11:52:57 +05:30
Sam Saffron 7632fe0b58 PERF: rake posts:rebake_uncooked_posts runs inline
Running this inline makes more sense otherwise there is extreme risk in
saturating sidekiq queue.

This also reworks ordering and selection so we double check if a post needs
rebaking prior to rebaking, this unlocks the ability to run this rake task
from multiple consoles.
2019-08-13 10:28:42 +10:00
David Taylor 923c46f5df DEV: Provide method for auth plugins to generate a CSRF token 2019-08-13 01:13:08 +01:00
Gerhard Schlager d686318133 FIX: Prevent failed remaps during restores
Additional changes:
* Verbose logging of remaps during restores
* Exclude the backup_metadata table from restores
2019-08-12 17:15:01 +02:00
Gerhard Schlager c05739c3c8 FIX: Truncate topic_links.url to 500 chars during remap
This column often breaks remaps because of some weird, long URLs. The data isn't that important, so truncating a couple of URLs doesn't hurt that much.
2019-08-12 17:15:00 +02:00
Gerhard Schlager 8b6341669a REFACTOR: Less duplicate code in DbHelper.remap 2019-08-12 17:15:00 +02:00
Dan Ungureanu 2a98becfde
FIX: Do not create a double like notification. (#7999)
When a user liked, unliked and liked again the same post, the poster
would receive a notification such as "X and X liked ...". This happened
because PostActionNotifier.post_action_created was called twice.
2019-08-12 16:22:46 +03:00
David Taylor 1a8fee11a0 DEV: If only one auth provider is enabled allow GET request
In this case, the auth provider is acting as a SSO provider, and can be trusted to maintain its own CSRF protections.
2019-08-12 11:03:05 +01:00
David Taylor 750802bf56
UX: Improve error handling for common OmniAuth exceptions (#7991)
This displays more useful messages for the most common issues we see:
- CSRF (when the user switches browser)
- Invalid IAT (when the server clock is wrong)
- OAuth::Unauthorized for OAuth1 providers, when the credentials are incorrect

This commit also stops earlier for disabled authenticators. Now we stop at the request phase, rather than the callback phase.
2019-08-12 10:55:02 +01:00
Blake Erickson 87a0a6664e FEATURE: External auth when redeeming invites
This feature (when enabled) will allow for invite_only sites to require
external authentication before they can redeem an invite.

- Created hidden site setting to toggle this
- Enables sending invites with local logins disabled
- OAuth button added to invite form
- Requires OAuth email address to match invite email address
- Prevents redeeming invite if OAuth authentication fails
2019-08-11 12:20:02 -06:00
Gerhard Schlager e4f14ca3d7 FIX: Disallow user self-delete when user posted in PMs
All posts created by the user are counted unless they are deleted,
belong to a PM sent between a non-human user and the user or belong
to a PM created by the user which doesn't have any other recipients.

It also makes the guardian prevent self-deletes when SSO is enabled.
2019-08-10 12:30:16 +02:00
Gerhard Schlager 74d78e3636 FIX: Drop readonly function when dropping table 2019-08-09 11:39:46 +02:00