Commit Graph

6949 Commits

Author SHA1 Message Date
Roman Rizzi 0c4ac2a7bc
Merge pull request from GHSA-569c-22ff-pj3x 2020-01-16 15:17:16 -03:00
Gerhard Schlager f216c6d60b FEATURE: Drop "backup" schema 7 days after restore
The "backup" schema is used to rollback a failed restore. It isn't useful after a longer period of time and turns into a waste of disk space.
2020-01-16 17:48:47 +01:00
Martin Brennan 7c32411881
FEATURE: Secure media allowing duplicated uploads with category-level privacy and post-based access rules (#8664)
### General Changes and Duplication

* We now consider a post `with_secure_media?` if it is in a read-restricted category.
* When uploading we now set an upload's secure status straight away.
* When uploading if `SiteSetting.secure_media` is enabled, we do not check to see if the upload already exists using the `sha1` digest of the upload. The `sha1` column of the upload is filled with a `SecureRandom.hex(20)` value which is the same length as `Upload::SHA1_LENGTH`. The `original_sha1` column is filled with the _real_ sha1 digest of the file. 
* Whether an upload `should_be_secure?` is now determined by whether the `access_control_post` is `with_secure_media?` (if there is no access control post then we leave the secure status as is).
* When serializing the upload, we now cook the URL if the upload is secure. This is so it shows up correctly in the composer preview, because we set secure status on upload.

### Viewing Secure Media

* The secure-media-upload URL will take the post that the upload is attached to into account via `Guardian.can_see?` for access permissions
* If there is no `access_control_post` then we just deliver the media. This should be a rare occurrance and shouldn't cause issues as the `access_control_post` is set when `link_post_uploads` is called via `CookedPostProcessor`

### Removed

We no longer do any of these because we do not reuse uploads by sha1 if secure media is enabled.

* We no longer have a way to prevent cross-posting of a secure upload from a private context to a public context.
* We no longer have to set `secure: false` for uploads when uploading for a theme component.
2020-01-16 13:50:27 +10:00
Gerhard Schlager 5e3fc31f2c DEV: Less hacky way of rolling back DB changes
Some specs use psql to test database restores and dropping the table after the test needs to happen outside of rspec because of transactions. The previous attempt lead to some changes to be stored in the test database.
2020-01-15 23:37:42 +01:00
Gerhard Schlager 68a7ae3091 REFACTOR: Simplify backup version check
Adds specs for inalid version number in metadata file.
Follow-up to c3cd2389fe
2020-01-15 23:37:40 +01:00
Dan Ungureanu c2d051315d FIX: Create post notices only for public posts (#8708)
This also ensures only public posts are considered when creating post
notices for new and returning users.
2020-01-15 11:40:19 +01:00
Martin Brennan 66f2db4ea4 SECURITY: 2FA with U2F / TOTP 2020-01-15 11:27:12 +01:00
Martin Brennan c3cd2389fe SECURITY: use strict JSON parsing when parsing backup metadata 2020-01-15 11:24:41 +01:00
Régis Hanol 5d75f90b27 FIX: group membership leak
FIX: raised a proper NotFound exception when filtering groups by username with invalid username.
FIX: properly filter the groups based on current user visibility when viewing another user's groups.
DEV: Guardian.can_see_group?(group) is now using Guardian.can_see_groups(groups) instead of duplicating the same code.
FIX: spec for groups_controller#index when group directory is disabled for logged in user.
FIX: groups_controller.sortable specs to actually test all sorting combinations.
DEV: s/response_body/body/g for slightly shorter spec code.
FIX: rewrote the "view another user's groups" specs to test all group_visibility and members_group_visibility combinations.
DEV: Various refactoring for cleaner and more consistent code.
2020-01-15 11:21:58 +01:00
Gerhard Schlager e474cda321 REFACTOR: Restoring of backups and migration of uploads to S3 2020-01-14 11:41:35 +01:00
Robin Ward 556d66a97c FIX: Use CDN for the `discourse-internet-explorer`
Previously the CDN was skipped because the assets were not in the proper
place to be uploaded.
2020-01-10 15:06:55 -05:00
Martin Brennan cb660ef952 SECURITY: Improve second factor auth logic 2020-01-10 10:45:56 +10:00
Gerhard Schlager 2a06f67099 FIX: Decompressing lots of small files triggered error
An archive containing lots of small files could trigger an error even though the amount of decompressed data was way below the maximum allowed size. This happened because the decompression algorithm used the chunk size for calculating the remaining size instead of the actual size of the decompressed chunk.
2020-01-09 15:11:31 +01:00
Penar Musaraj 2d84bea653 DEV: watch for changes to plugin mobile/desktop SCSS
Previously, the watcher was only refreshing changes to non-targeted plugin stylesheets.
2020-01-08 13:25:36 -05:00
Gerhard Schlager 2683b7793b REFACTOR: Simplify user title comparison
Follow-up to 6d6ffe51
2020-01-08 18:39:10 +01:00
Gerhard Schlager 6d6ffe5131 FIX: Allow users to change title in locales other than English 2020-01-08 18:08:04 +01:00
David Taylor c8d438cc63
DEV: Allow CSP to be enabled during QUnit tests (#8668)
The QUnit rake task starts a server in test mode. We need a tweak to allow dynamic CSP hostnames in test mode. This tweak is already present in development mode.

To allow CSP to work, the browser host/port must match what the server sees. Therefore we need to disable the enforce_hostname middleware in test mode. To keep rspec and production as similar as possible, we skip enforce_hostname using an environment variable.

Also move the qunit rake task to use unicorn, for consistency with development and production.
2020-01-07 12:22:58 +00:00
Martin Brennan abca91cc4d
FEATURE: Add rake task to disable secure media (#8669)
* Add a rake task to disable secure media. This sets all uploads to `secure: false`, changes the upload ACL to public, and rebakes all the posts using the uploads to make sure they point to the correct URLs. This is in a transaction for each upload with the upload being updated the last step, so if the task fails it can be resumed.
* Also allow viewing media via the secure url if secure media is disabled, redirecting to the normal CDN url, because otherwise media links will be broken while we go and rebake all the posts + update ACLs
2020-01-07 12:27:24 +10:00
Blake Erickson b7b85f9ade
FEATURE: Turn csp on by default (#8665)
* turn csp on by default

* remove csp migration for new sites now that is is on by default

* Ensure CSP is off before starting qunit
2020-01-06 13:42:21 -07:00
David Taylor d1779346e8 FIX: topic_tracking_state when mute_all_categories_by_default is enabled 2020-01-06 18:22:42 +00:00
Sam Saffron d0630ea6ee FIX: MaxMind DB file not downloading correctly
Previously we had the ability to download a simple .gz file
new changes mean we have a a tar.gz file that needs some levels
of fiddling to get extracted correctly
2020-01-05 22:08:13 +11:00
Vinoth Kannan 9a6606dd30 DEV: Add option to keep quoted content in post excerpt. 2020-01-04 18:56:52 +05:30
Blake Erickson fc6b093dce FIX: Ensure CSP is off for qunit
If CSP is turned on qunit won't run at all, not even via the CLI, this
is causing the js tests to fail.

Follow up to: 3193b0f6e6
2020-01-03 18:43:19 -07:00
Sam Saffron a8ffb6949c FEATURE: support MaxMind DB downloads using a license key
MaxMind now requires an account with a license key to download files.

Discourse admins can register for such an account at:

https://www.maxmind.com/en/geolite2/signup

License key generation is available in the profile section.

Once registered you can set the license key using `DISCOURSE_MAXMIND_LICENSE_KEY`

This amends it so we unconditionally skip MaxMind DB downloads if no license key exists.
2020-01-03 16:32:48 +11:00
David Taylor 45c5f56ffc
PERF: Reduce DB queries when serializing ignore/mute information (#8629)
* PERF: Cache ignored and muted user ids in the current_user object
* PERF: Avoid DB queries when checking ignore/mute permission in guardian
2020-01-02 13:04:08 +00:00
Sam Saffron a9f90cdec3 FIX: avoid superflous logging when mime type is bad
Many security scanners ship invalid mime types, this ensures we return
a very cheap response to the clients and do not log anything.

Previous attempt still re-dispatched the request to get proper error page
but in this specific case we want no error page.
2020-01-02 12:34:38 +11:00
Martin Brennan c031434b86
FIX: Catch error when unknown COSE algorithm is supplied for Security Key (#8649)
Added a fix to gracefully error with a Webauthn::SecurityKeyError if somehow a user provides an unkown COSE algorithm when logging in with a security key.

If `COSE::Algorithm.find` returns nil we now fail gracefully and log the algorithm used along with the user ID and the security key params for debugging, as this will help us find other common algorithms to implement for webauthn
2020-01-02 10:14:22 +10:00
Sam Saffron 412e1ebbe2 DEV: correct parallel specs rake tasks
This used to work due to side effects.

`rake parallel:migrate` used to work very inconsistently and would only migrate
some of the databases.

This introduces the recommended change to db.yml so the correct database is
found based off TEST_ENV_NUMBER if for some reason we did not set it using
RAILS_DB

Also avoids a bunch of schema dumping which is not needed when migrating
parallel specs



DB number 1 is very odd cause for whatever reason parallel spec is not
setting it.
2019-12-31 14:07:55 +11:00
David Taylor bc4c40abd4
DEV: Remove unsafe-eval from development CSP (#8569)
- Refactor source_url to avoid using eval in development
- Precompile handlebars in development
- Include template compilers when running qunit
- Remove unsafe-eval in development CSP
- Include unsafe-eval only for qunit routes in development
2019-12-30 12:17:12 +00:00
Roman Rizzi 16d97573f9
DEV: Improve support for extending flags. (#8619)
- Ensure that the 'notify_moderators' flag is always the last flag when using custom flags.
- Support passign a custom FlagSettings object when replacing flags to reuse existing ones.
2019-12-27 08:41:50 -03:00
Mark VanLandingham 6ec3d42b16
FEATURE: Allow complex post params from plugin (#8598) 2019-12-20 10:37:12 -06:00
David Taylor 61919ad39f
UX: Do not use avatars as fallback opengraph images for replies (#8605)
People rarely want to have their avatars show up as the preview image on social media platforms. Instead, we should fall back to the site opengraph image.
2019-12-20 13:17:14 +00:00
Bianca Nenciu 1bccd8eca9
FIX: Remove full nested quotes on direct reply (#8581)
It used to check how many quotes were inside a post, without taking
considering that some quotes can contain other quotes. This commit
selects only top level quotes.

I had to use XPath because I could not find an equivalent CSS
selector.
2019-12-20 10:24:34 +02:00
Kris 2ad40d5f71 UX: New bell icons for notification/tracking statuses 2019-12-19 14:39:29 -05:00
Neil Lalonde ef5d9a6cb5 Version bump to v2.4.0.beta9 2019-12-19 14:07:24 -05:00
Robin Ward 9fc1a754fe DEV: Error message if multisite DB can't be created 2019-12-19 11:52:27 -05:00
Vinoth Kannan 3b7f5db5ba
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547) 2019-12-18 11:21:57 +05:30
Joffrey JAFFEUX 81b4de39ee
FIX: prevents crash in discourse_tagging with empty term (#8548) 2019-12-17 10:55:06 +01:00
Martin Brennan 6ab12ed96b
FIX: If a prettified slug is a number, return defaultt (#8554)
Meta thread: https://meta.discourse.org/t/sending-a-pm-with-the-following-title-causes-an-error/135654/3

We had an issue where if someone sent a PM with crazy
characters that are stripped and we end up with only
a number, the topic redirect errored because the slug was
a number. so instead we return the default as well if
the slug is a number after prettification
2019-12-17 10:34:20 +10:00
Robin Ward 480e2ec523 FIX: If we run `db:migrate` on its own, it should load the environment 2019-12-16 14:46:33 -05:00
Robin Ward ce78eff888 FIX: Migration paths were being forgotten
According to the [Rails
Source](https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L20)
the `ActiveRecord::Migrator.migrations_paths` are overwritten with the
value of `ActiveRecord::Tasks::DatabaseTasks.migrations_paths` every
time the config is loaded.

This caused a bug for Discourse development where if you ran:

`rake db:drop db:create db:migrate` in one line, you would not get our
post migrations, as those had a custom value for `migrations_paths`.

The fix is to use `ActiveRecord::Tasks::DatabaseTasks.migrations_paths`
to set up all our custom paths. Everything seems to work as expected.
2019-12-16 14:13:47 -05:00
Mark VanLandingham 8c4ffaea1b
FEATURE: Modal for profile featured topic & admin wrench refactor (#8545) 2019-12-16 08:41:34 -08:00
Martin Brennan b6b05d3b48
Add bookmark-clock icon (#8553)
Adds a custom bookmark-clock icon to discourse-additional.svg for use with the new bookmarks with reminder functionality.

Also add some code to correctly refresh the post-stream icon for bookmark to show the clock after save.
2019-12-16 12:37:34 +10:00
Gerhard Schlager e7402b3feb DEV: Allow `unsafe-eval` in development mode
It's needed because of this: 5f1e346e16/lib/source_url.rb (L18-L24)
2019-12-14 22:36:50 +01:00
Gerhard Schlager f62215046f FEATURE: Completely remove `unsafe-eval` from CSP
Plugins can add it via API if they need to use `eval`:
```
extend_content_security_policy(script_src: [:unsafe_eval])
```

See https://meta.discourse.org/t/104243
2019-12-13 12:38:32 +01:00
Hector Bustillos 42758379b9 FEATURE: Adds a message when the passwords doesn't match in rake admin:create (#8543) 2019-12-13 11:40:33 +11:00
David Taylor 4c9ca24ccf
FEATURE: Hash API keys in the database (#8438)
API keys are now only visible when first created. After that, only the first four characters are stored in the database for identification, along with an sha256 hash of the full key. This makes key usage easier to audit, and ensures attackers would not have access to the live site in the event of a database leak.

This makes the merge lower risk, because we have some time to revert if needed. Once the change is confirmed to be working, we will add a second commit to drop the `key` column.
2019-12-12 11:45:00 +00:00
Martin Brennan ce03092e61 Fix DiscourseCops/NoURIEscapeEncode errors and re-enable 2019-12-12 14:54:26 +10:00
Martin Brennan edbc356593
FIX: Replace deprecated URI.encode, URI.escape, URI.unescape and URI.unencode (#8528)
The following methods have long been deprecated in ruby due to flaws in their implementation per http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/29293?29179-31097:

URI.escape
URI.unescape
URI.encode
URI.unencode
escape/encode are just aliases for one another. This PR uses the Addressable gem to replace these methods with its own encode, unencode, and encode_component methods where appropriate.

I have put all references to Addressable::URI here into the UrlHelper to keep them corralled in one place to make changes to this implementation easier.

Addressable is now also an explicit gem dependency.
2019-12-12 12:49:21 +10:00
Dan Ungureanu 4e130f1e03
SECURITY: Remove event handlers from SVG files 2019-12-11 17:04:50 +02:00
Robin Ward 4abe4454dd
FEATURE: Create IE Support Plugin (#8520)
This core plugin, which could be split off in the future, allows us to load IE specific code on demand.

Co-authored-by: jjaffeux <j.jaffeux@gmail.com>
2019-12-11 09:07:22 -05:00
Martin Brennan 6261339da9
Improving bookmarks part 1 (#8466)
Note: All of this functionality is hidden behind a hidden, default false, site setting called `enable_bookmarks_with_reminders`. Also, any feedback on Ember code would be greatly appreciated!

This is part 1 of the bookmark improvements. The next PR will address the backend logic to send reminder notifications for bookmarked posts to users. This PR adds the following functionality:

* We are adding a new `bookmarks` table and `Bookmark` model to make the bookmarks a first-class citizen and to allow attaching reminders to them.
* Posts now have a new button in their actions menu that has the icon of an actual book
* Clicking the button opens the new bookmark modal.
* Both name and the reminder type are optional.
* If you close the modal without doing anything, the bookmark is saved with no reminder.
* If you click the Cancel button, no bookmark is saved at all.
* All of the reminder type tiles are dynamic and the times they show will be based on your user timezone set in your profile (this should already be set for you).
* If for some reason a user does not have their timezone set they will not be able to set a reminder, but they will still be able to create a bookmark.
* A bookmark can be deleted by clicking on the book icon again which will be red if the post is bookmarked.

This PR does NOT do anything to migrate or change existing bookmarks in the form of `PostActions`, the two features live side-by-side here. Also this does nothing to the topic bookmarking.
2019-12-11 14:04:02 +10:00
David Taylor 67605b55d5 DEV: Prevent plugin loading when running core db migrations in tests
This is required because bin/rake automatically loads plugins when migrating. In our continuous integration, we don't want plugins to break the core build. They should only be loaded for the plugin build.
2019-12-10 16:11:49 +00:00
Neil Lalonde 20464116ae FIX: tag input doesn't show all top 5 permitted tags 2019-12-10 10:30:01 -05:00
Sam Saffron f5396e2700 DEV: Sidekiq::Logging is gone use Sidekiq.logger instead
This 6.0 upgrade of sidekiq moved this around.
2019-12-10 15:09:51 +11:00
Sam Saffron 0c52537f10 DEV: update rubocop to version 0.77
We like to stay as close as possible to latest with rubocop cause the cops
get better.

This update required some code changes, specifically the default is to avoid
explicit returns where implicit is done

Also this renames a few rules
2019-12-10 11:48:39 +11:00
Sam Saffron 8e489ec8f8 DEV: mark discourse-rss-polling as official 2019-12-10 11:48:39 +11:00
Mark VanLandingham 14cb386f1e
FEATURE: Featured topic for user profile & card (#8461) 2019-12-09 11:15:47 -08:00
Dan Ungureanu ebe6fa95be
FIX: Optimize images in Onebox (#8471)
This commit ensures that images in Onebox are being optimized, but not
converted to lightbox too.
2019-12-09 15:39:25 +02:00
Sam Saffron 494fe335d3 DEV: allow handling crawler reqs with no user agent
Followup to e440ec25 we treat no user agent as crawler reqs.
2019-12-09 18:40:10 +11:00
Sam Saffron e440ec2519 FIX: crawler requests not tracked for non UTF-8 user agents
Non UTF-8 user_agent requests were bypassing logging due to PG always
wanting UTF-8 strings.

This adds some conversion to ensure we are always dealing with UTF-8
2019-12-09 17:43:51 +11:00
Jarek Radosz 02ca6fa6c8 DEV: See if the store is external before checking disk space (#8480)
`available_disk_space` calls `df` which exits with an error if the `uploads` path doesn't exist. That's often the case when the `Discourse.store.external?` is true.

By doing the `external?` check first the `disable_if_low_on_disk_space` does less work and doesn't output any errors to the console.
2019-12-09 12:48:45 +11:00
Jeff Wong e7bd288d7b FIX: Skip validation on enforcing second factor change if the value is "no"
Without this, this site setting breaks plugin tests if local logins are
disabled in the plugin.
2019-12-06 16:42:15 -08:00
Vinoth Kannan e51091f199 REFACTOR: do `X-Frame-Options` header removal in application controller.
Co-authored-by: Sam <sam.saffron@gmail.com>
Previous commit: f7084a4339
2019-12-06 18:25:32 +05:30
Gerhard Schlager c88797bf0e FIX: Ignore DMARC for emails sent to mailing list mirror 2019-12-06 13:29:39 +01:00
Vinoth Kannan f7084a4339 FEATURE: add site setting to remove `X-Frame-Options` header. 2019-12-06 03:15:09 +05:30
Robin Ward 895d5cb592 FIX: Anonymous cache regression 2019-12-05 15:07:48 -05:00
Robin Ward 532fea1460
DEV: Provide API for anonymous cache segments (#8455)
This can be used from a plugin that needs to establish something new in
the anonymous cache. For example `is_ie` for an internet explorer
plugin.
2019-12-05 14:57:18 -05:00
Neil Lalonde 2987a46f48 Version bump to v2.4.0.beta8 2019-12-05 13:50:33 -05:00
Neil Lalonde 891b71bdf5 Fix release notes rake task
It breaks when a commit message is "FIX:"
2019-12-05 11:53:10 -05:00
Jarek Radosz d07f039468 FIX: Secure Upload URLs in lightbox (#8451)
This fixes the following issues:

* The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image
* Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 09:13:09 +10:00
Mark VanLandingham a54aaeeb79
REFACTOR: post_destroyer destroy method (#8462) 2019-12-04 14:13:31 -06:00
romanrizzi b13f427450 DEV: Drop uglifyJS V2 support 2019-12-04 15:58:38 -03:00
Neil Lalonde 875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
Robin Ward 888d56774a
DEV: HTML Builders should respect if a plugin is enabled or not (#8454)
Previously they would return the HTML regardless of whether the plugin
was enabled or not.
2019-12-04 12:26:23 -05:00
David Taylor 46d8fd3831 FIX: Allow for nil upload record when migrating to S3 2019-12-04 15:13:39 +00:00
David Taylor be1510b09d FIX: Use filtered posts when determining the next page
This bug was causing some unusual behavior when the last post is filtered (e.g. from an ignored user). In some situations this would cause suggested topics to be omitted from the payload.

The next_page specs have been updated to remove most of the stubs
2019-12-04 12:52:24 +00:00
Daniel Waterworth 226fc07a17 FIX: Don't allow category slugs to be entirely digits 2019-12-04 01:22:29 +00:00
Gerhard Schlager 2cca14d510 FEATURE: Add hidden setting to allow `unsafe-eval` in CSP
This new setting defaults to `true` for now, until we make sure that all official plugins and theme components work without `unsafe-eval` in the CSP.
2019-12-03 21:09:08 +01:00
Gerhard Schlager 9ebb69e8eb FIX: Respect `enable_inline_emoji_translation` setting in titles 2019-12-03 20:39:11 +01:00
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Arpit Jalan 016732cced FEATURE: improve suspect user discovery 2019-12-02 16:39:28 +05:30
Robin Ward 7fee3c61de
Support for custom messages and redirects when creating posts (#8434)
* Support for custom messages and redirects when creating posts

When a post/topic is created Discourse serializes a `NewPostResult`
object. Normally this contains a status like `created_post` or
errors describing why the post could not be created.

There are times when a plugin might want to take the inputted post
and do something in the background. In this case, the plugin
can return a custom `message` and `route_to` attribute in the
`NewPostResult`.

If present, the message will be displayed in an alert, and when "Ok" is
clicked the user will be routed to the new URL.

* Destroy the draft in parallel
2019-11-29 09:30:54 -05:00
Dan Ungureanu 1e0c2235a3
FIX: Optimize quoted images (#8427)
Only images that were part of a lightbox used to be optimized. This
patch ensures that quoted images are also optimized.
2019-11-29 15:18:42 +02:00
Sam Saffron 7371b427cd DEV: correct a few Ruby 2.7 deprecations
Note:

```
def foo(bar: 1)
end

foo({bar: 2})
# raises a deprecation, instead use:

foo(**{bar: 2})
```

Additionally when matching regexes always use strings. It does not make
sense to match a non string to a regex.
2019-11-28 13:13:29 +11:00
Martin Brennan 901054fd75
FIX: Cache failed onebox URL request server-side (#8421)
We already cache failed onebox URL requests client-side, we now want to cache this on the server-side for extra protection. failed onebox previews will be cached for 1 hour, and any more requests for that URL will fail with a 404 status. Forcing a rebake via the Rebake HTML action will delete the failed URL cache (like how the oneboxer preview cache is deleted).
2019-11-28 07:48:29 +10:00
Roman Rizzi 1c9d18f094
FIX: Correctly ignore/approve replies when acting on a flagged post (#8425) 2019-11-27 17:19:44 -03:00
David Taylor a227083c1c
FEATURE: Limit the number of active sessions for a user (#8411)
If a user has more than 60 active sessions, the oldest sessions will be terminated automatically. This protects performance when logging in and when loading the list of recently used devices.
2019-11-27 12:39:31 +00:00
Arpit Jalan 520a83aa62 FIX: correct hostname in vimeo.com 2019-11-27 14:52:28 +05:30
Arpit Jalan 52c8cab7f2 FIX: bypass finaldestination check for Vimeo links. 2019-11-27 14:00:46 +05:30
Sam Saffron 88ecb650a9 DEV: Implement a faster Discourse.cache
This is a bottom up rewrite of Discourse cache to support faster performance
and a limited surface area.

ActiveSupport::Cache::Store accepts many options we do not use, this partial
implementation only picks the bits out that we do use and want to support.

Additionally params are named which avoids typos such as "expires_at" vs "expires_in"

This also moves a few spots in Discourse to use Discourse.cache over setex
Performance of setex and Discourse.cache.write is similar.
2019-11-27 16:11:49 +11:00
Sam Saffron 0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
Simon Cossar 935d4cfa60
Check site default locale if Rtl class is initialized without a user (#8417) 2019-11-26 15:01:37 -05:00
Leo McArdle 2714149fd2 FEATURE: hide posts from incoming email based on dmarc verdict (#8333) 2019-11-26 15:55:22 +01:00
Sam Saffron 941162e90e FIX: draft not clearing when replying to new topic
This amends our API so we provide it with the draft key when saving a post
this means post creator can clean up the draft consistently even if we are
doing fancy stuff like replying to a new topic or new pm or whatever.

There will be some followup work to clean it up so client never calls destroy
on draft during normal operation and the #create/#update endpoints takes care of it
every time
2019-11-26 18:23:20 +11:00
Penar Musaraj 5c5d8a307a FIX: Only trigger upload ACL update when needed
In `post_creator`, the ACL update is only necessary when uploads need to be secured.

This should fix a regression with S3 clones that do not support updating ACLs.
2019-11-25 11:20:33 -05:00
Dan Ungureanu a992caf741
DEV: Replace magic values (#8398)
Follow-up to 35942f7c7c.
2019-11-25 14:32:19 +02:00
Joffrey JAFFEUX 22eb1828f6
DEV: supports unicorn emoji 13.0beta (#8402)
This update will most likely reduce images weight and also adds support for 🏳️‍⚧️
2019-11-25 10:23:18 +01:00
Martin Brennan afb5533581
FEATURE: Add timezone to core user_options (#8380)
* Add timezone to user_options table

* Also migrate existing timezone values from UserCustomField,
  which is where the discourse-calendar plugin is storing them

* Allow user to change their core timezone from Profile

* Auto guess & set timezone on login & invite accept & signup

* Serialize user_options.timezone for group members. this is so discourse-group-timezones can access the core user timezone, as it is being removed in discourse-calendar.

* Annotate user_option with timezone

* Validate timezone values
2019-11-25 10:49:27 +10:00
Roman Rizzi c92f0b8775
FEATURE: New API to apply custom filters to the review queue (#8392) 2019-11-22 16:33:10 -03:00
Penar Musaraj 0a6b29a942 DEV: Allow skipping --fail-fast rspec option for plugin specs 2019-11-22 13:27:47 -05:00
Martin Brennan 23714e77c4
FIX: Always return secure_proxy_without_cdn url for secure media (#8394)
There was an issue on dev where when uploading secure media, the href of the media was correctly being replaced in the CookedPostProcessor, but the srcset urls were not being replaced correctly. This is because UrlHelper.cook_url was returning the asset host URL for the media for secure media instead of returning early with the proxied secure proxy url.
2019-11-22 15:29:31 +10:00
Martin Brennan 5ad6dc01c1
FEATURE: Support RS256 algorithm for webauthn (#8385)
* Add support for RS256 webauthn algorithm (-257)
2019-11-21 15:45:36 +10:00
Sam Saffron 423ad5f0a4 FIX: do not log if an invalid mime type is passed to app
Previously our custom exception handler was unable to handle situations
where an invalid mime type was sent, resulting in a warning log

This ensures we pretend a request is HTML for the purpose of rendering
the error page if an invalid mime type from a scanner is shipped to the app
2019-11-21 15:51:34 +11:00
Jeff Wong c6d8dbd4a9 Revert "FEATURE: Normalize the service worker route (#8359)"
This reverts commit 9799a651b6.
2019-11-20 14:10:17 -08:00
Jeff Wong 9799a651b6
FEATURE: Normalize the service worker route (#8359)
* FEATURE: Normalize the service worker route

Update cache headers so they are not immutable outside of the rails app

Add the ability to purge the service worker cache from localhost

Rails -> nginx will pass immutable flags so the file is cached until reloaded.
In most cases, nginx will have its cache flushed on rebuild (new image)

For those needing dynamic re-caching (such as upgrading via the UI),
a rake task for flushing the service worker script is provided
through `assets:flush_sw`
2019-11-20 11:33:41 -08:00
David Taylor b8e5208162 DEV: Add convenience method to check info about a site setting
This is only defined in a console environment. For example:

```
[1] pry(main)> SiteSetting.info(:title)
=> {:resolved_value=>"Globally Overridden Title",
 :default_value=>"Discourse",
 :global_override=>"Globally Overridden Title",
 :database_value=>"Test Discourse",
 :refresh?=>false,
 :client?=>true,
 :secret?=>false}
```
2019-11-20 16:39:47 +00:00
Martin Brennan d45bd7f131
FIX: Abort CensoredWordsValidator early if censored_words_regexp nil (#8375)
* Abort CensoredWordsValidator early if censored_words_regexp nil. Sometimes censored_words_regex can end up nil, erroring the validator. This handles the nil condition and also adds a spec for the validator
2019-11-20 20:05:06 +10:00
Martin Brennan 02cb01406e
FIX: Allow secure uploads if global s3 setting active and enable_s3_uploads validations (#8373)
The secure media functionality relied on `SiteSetting.enable_s3_uploads?` which, as we found in dev, did not take into account global S3 settings via `GlobalSetting.use_s3?`. We now use `SiteSetting.Upload.enable_s3_uploads` instead to be more consistent.

Also, we now validate `enable_s3_uploads` changes, because if `GlobalSetting.use_s3?` is true users should NOT be enabling S3 uploads manually.
2019-11-20 07:46:44 +10:00
Vinoth Kannan d3e7768ea8 Revert "FIX: parallel spec system needs needs a dedicated upload folder for each worker. (#8372)"
This reverts commit 42e5176bc3.
2019-11-19 15:02:18 +05:30
Vinoth Kannan 42e5176bc3
FIX: parallel spec system needs needs a dedicated upload folder for each worker. (#8372) 2019-11-19 13:16:20 +05:30
Vinoth Kannan f83125f0c2 DEV: minor refactoring to reduce the code duplication. 2019-11-19 08:04:24 +05:30
Vinoth Kannan 57bbcf4c5d FIX: 'default_categories_muted' site setting not working for anonymous users. 2019-11-19 07:48:16 +05:30
Martin Brennan 93d7abe372
FIX: Automatically recover from bad sprockets cache in development (#8364)
We were having issues in development mode where the JS code had errors due to a bad cache. When starting a server in development mode in bin/unicorn we now get the git sha of the discourse HEAD and get a git sha of all plugins, and store them in a file. If the sha has changed then we delete tmp/cache to refresh the assets cache.
2019-11-19 09:15:09 +10:00
Neil Lalonde 228c4814be FIX: errors when using tags with colons in their name 2019-11-18 13:20:37 -05:00
David Taylor eaf6096890 DEV: Use rubocop-discourse gem to add custom chdir cop
Followup to b27e009655
2019-11-18 15:39:41 +00:00
Vinoth Kannan 3bb7ad4be1
FEATURE: remove support for 'suppress_from_latest' category setting. (#8308) 2019-11-18 12:28:35 +05:30
Martin Brennan af091c49e9
FIX: Ensure revisions are made to store edit reasons and no reasons get wiped (#8363)
* Fix an issue where if an edit was made to a post with a reason provided, and then another edit was made with no reason, the original edit reason got wiped out
* We now always make a post revision (even with ninja edits) if an edit reason has been provided and it is different from the current edit reason

Co-Authored-By: Sam <sam.saffron@gmail.com>
2019-11-18 13:08:54 +10:00
Penar Musaraj 102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Robin Ward 397852a9a6 FIX: Support for `rake db:rollback` with plugins
For this to work we need to overwrite `db:rollback` in our Rakefile like
we do for migrate, so that it removes the load_config dependency. This
allows our custom migration paths to work.
2019-11-15 16:38:46 -05:00
Martin Brennan 5c59247c3a Block enabling force 2FA if local logins disabled & vice-versa (#8355) 2019-11-15 17:05:10 +11:00
Penar Musaraj 067696df8f DEV: Apply Rubocop redundant return style 2019-11-14 15:10:51 -05:00
David Taylor f2da630c19 FIX: SVG Sprite version hash should be based on bundle result
This version hash is used for the filename, and so browsers/CDNs cache based on it. Previously the version hash was based only on the list of requested icons. This can cause issues in a couple of situations, most commonly when developing themes with custom icons:

- A requested icon does not exist, and then later is added to the theme. The bundle output changes, but the hash did not
- The SVG content of an icon changes, but the name of the icon does not. The bundle output changes, but the hash did not
2019-11-14 13:20:16 +00:00
Martin Brennan e7226a8c84
FEATURE: Allow scoping search to tag (#8345)
* When viewing a tag, the search widget will now show a checkbox to scope the search by tag, which will limit search results to that tag on desktop and mobile
2019-11-14 10:40:26 +10:00
Krzysztof Kotlarek 6e1fe22a9d
FEATURE: Dismiss new per category (#8330)
Ability to dismiss new topics per category.
2019-11-14 11:16:13 +11:00
David Taylor d095c2cee7 FIX: Allow importing themes with subdirectories in extra_js
The folder/file detection was broken by 9fea43e46a. This commit fixes and adds relevant specs
2019-11-13 23:45:09 +00:00
romanrizzi 0ba0aa00e7 DEV: Use a bigger suite for warming up the tmp folder 2019-11-13 18:34:10 -03:00
Roman Rizzi c63e84dc62
FEATURE: Send a message to new TL2 users and point them to the advanced tutorial (#8335)
* FEATURE: Send a message to new TL2 users and point them to the advanced tutorial

* DEV: Use a method to find the discobot user
2019-11-13 18:31:49 -03:00
David Taylor 481efebe76
DEV: Update backup/restore pipeline to avoid `cd` (#8347) 2019-11-13 15:52:28 +00:00
David Taylor 9fea43e46a
DEV: Remove use of `cd` in the app (#8337)
`FileUtils.cd` and `Dir.chdir` cause the working directory to change for the entire process. We run sidekiq jobs, hijacked requests and deferred jobs in threads, which can make working directory changes have unintended side-effects.

- Add a rubocop rule to warn about usage of Dir.chdir and FileUtils.cd
- Added rubocop:disable for scripts used outside the app
- Refactored code using cd to use alternative methods
- Temporarily skipped the rubocop check for lib/backup_restore. This will require more complex refactoring, so I will create a separate PR for review
2019-11-13 09:57:39 +00:00
Neil Lalonde 7711df40e6
REFACTOR: redo DiscourseTagging.filter_allowed_tags (#8328)
This method had grown into a monster. Its query had bugs
that I couldn't fix, and new features would be hard to add.
Also I don't understand how it all works anymore...
Replace it with common table expressions that can be queried
to generate the results we need, instead of subtracting
results using lots of "NOT IN" clauses.

Fixed are bugs with tag schemas that use combinations of
tag groups, parent tags, and one-tag-per-topic restrictions.
For example: https://meta.discourse.org/t/130991/6
2019-11-12 14:28:44 -05:00
Robin Ward 7f651b9666 REFACTOR: Deprecate `Discourse.Site` and `Discourse.User` 2019-11-12 10:53:22 -05:00
Mark VanLandingham 3dd2f2f701
DEV: Remove RSS feed polling in favor of plugin (#8233) 2019-11-12 09:49:02 -06:00
Robin Ward 6e481851e7 REFACTOR: Don't generate `Discourse` constants for mixins 2019-11-12 10:21:43 -05:00
Daniel Waterworth e79cec9fc7 DEV: Remove handling of category top menu items
Support for these kinds of navigation items was dropped in 88f52514, but
the code for handling these menu items was never removed.
2019-11-11 17:36:29 +00:00
Simon Cossar 544a1379b6
Reference posts table for deleted_at and user_id (#8332) 2019-11-11 11:03:57 -05:00
Sam Saffron 91daafc674 DEV: correct implementation of expiry api
Previously we were always hard-coding expiry, this allows the secure session
to correctly handle custom expiry times

Also adds a ttl method for looking up time to live
2019-11-11 11:18:12 +11:00
Sam Saffron b68eab2b46 DEV: remove deprecated syntax
Ruby 2.7 is going to deprecate "Magic Proc Init". This feature is enormously
magical anyway and the new code is far easier to comprehend.
2019-11-11 09:36:40 +11:00
Blake Erickson 15f6f57cdc DEV: Add update message for an outdated mail-receiver
Previous versions of the mail-receiver used query based api credentials,
if we detect this we will show a message in the admin panel to update
the mail receiver.
2019-11-08 17:32:22 -07:00
Jay Pfaffman 63bd07492e FIX: Update api_key rake task for recent changes
New usage: 

```
rake api_key:get_or_create_master["Onboarding Key"]
```
2019-11-08 23:34:15 +00:00
Robin Ward 920f8c6d75 REFACTOR: Remove `Discourse.Route` in lieu of importing 2019-11-08 15:18:13 -05:00
Robin Ward f03edb36be REFACTOR: Remove `Discourse.Model` 2019-11-08 14:13:35 -05:00
Robin Ward fa71919693 REFACTOR: Don't transpile `Discourse.Session` 2019-11-08 13:48:18 -05:00
Robin Ward 7ac2a55588 REFACTOR: Remove `Discourse.Category` constants 2019-11-08 13:31:00 -05:00
Robin Ward 7e44065bcb REFACTOR: Migrate `Discourse.Post` to imports 2019-11-08 13:13:13 -05:00
Robin Ward 932c169d46 REFACTOR: Remove Discourse.NavItem constants 2019-11-08 12:56:13 -05:00
Robin Ward 779ee3558d REFACTOR: Remove `Discourse.UserAction` from code 2019-11-08 12:52:39 -05:00
Robin Ward 1990baa457 REFACTOR: Remove constant definition for discourse/routes/grouped 2019-11-08 12:47:39 -05:00
David Taylor 1998be3b27
DEV: Raise errors when cleaning the download cache, and fix for macOS (#8319)
POSIX's `head` specification states: "The application shall ensure that the number option-argument is a positive decimal integer"

Negative values are supported on GNU `head`, so this works in the discourse docker image. However, in some environments (e.g. macOS), the system `head` version fails with a negative `n` parameter.

This commit does two things:

Checks the status at each stage of the pipe, so it cannot fail silently
Flip the `ls` command to list in descending time order, and use `tail -n +501` instead of `head -n -500`.

The visible result is that macOS users no longer see head: illegal line count -- -500 printed throughout the test suite.
2019-11-08 15:34:03 +00:00
David Taylor cd57c3bf5a
DEV: Abort autospec on [ENTER], even if no specs have failed (#8320)
When starting autospec, it says

> Press [ENTER] to stop the current run

However, [ENTER] does nothing unless a spec has failed. Sometimes I want to abort anyway, so that the run is restarted.
2019-11-08 14:23:12 +00:00