Commit Graph

6601 Commits

Author SHA1 Message Date
Sam Saffron 6471fbd453 FIX: maxminddb:get task no longer working
Following our zeitwerk upgrade we needed to be more explicit about the
require
2019-10-04 09:47:33 +10:00
Robin Ward a404cab509 The generated plugin should also freeze strings 2019-10-03 11:43:36 -04:00
Robin Ward 32a11a93ac Update the template for plugin generator to be rubocop compliant 2019-10-03 11:41:53 -04:00
Roman Rizzi 10565e4623
SECURITY: Safely decompress files. (#8124)
* FEATURE: Adds an extra protection layer when decompressing files.

* Rename exporter/importer to zip importer. Update old locale

* Added a new composite class to decompress a file with multiple strategies

* Set max file size inside a site setting

* Ensure that file is deleted after compression

* Sanitize path and files before compressing/decompressing
2019-10-03 10:19:35 -03:00
Justin DiRose 63fabdb6f2 FIX: first post true if user creates topic first (#8139)
Reported here: https://meta.discourse.org/t/user-was-banned-for-posting-a-reply-within-3-seconds/128823/12
The problem here is the user could create a topic, then reply and get
silenced on the second time (though technically their first post) for being below the
min_first_post_typing_time threshold.
2019-10-02 15:51:40 -04:00
Daniel Waterworth 1fdba2c5b2 FIX: Harden DistributedMutex
Threadsafety

  Since we use the same redis connection in multiple threads, a rogue
  transaction in another thread can trample the connection state
  (watched keys) that we need to acquire and release the lock properly.

  This is fixed by preventing other threads from using the connection
  when we are performing these actions.

Off-by-one error

  A distributed mutex is now consistently determined to be expired if
  the current time is strictly greater than the expire time.

Unwatch before transaction

  Since the redis connection is used by so much of the code, it is
  difficult to ensure that any watched keys have been cleared. In order
  to defend against this rogue connection state, an unwatch has been
  added before locking and unlocking.

Logging

  Hopefully this log message is more clear.
2019-10-02 13:00:41 +00:00
Michael Brown dbe0111822 FEATURE: allow UploadRecovery to be run on a single post (#8094) 2019-10-02 14:57:36 +10:00
Krzysztof Kotlarek 35b1185a08 FIX: Revert Demon::DemonBase back to Demon::Base (#8132)
I introduced DemonBase because I had got some conflict between `demon/base.rb` and `jobs/base.rb`, however, to not rename base class, it is possible to use regex on absolute path in Zeitwerk custom inflector.
2019-10-02 14:54:08 +10:00
Sam Saffron 55ee9abecb DEV: clean up dependencies in spec
Follow up on zeitwork we needed to be a bit more explicit about a few
dependencies internally.

On certain orders the test suite could fail.
2019-10-02 14:50:54 +10:00
Sam Saffron 3f6af54fe1 DEV: minor fixes related to zeitwerk
A couple of specs stopped working post zeitwerk. Adding missing require
corrected inheritance to avoid a circular ref.
2019-10-02 14:28:18 +10:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Martin Brennan 68d35b14f4 FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099)
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).

Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
2019-10-01 19:08:41 -07:00
Joffrey JAFFEUX e5af03be01 DEV: prevents attemps to redefine THEME_TARGETS constant (#8121) 2019-10-02 10:54:31 +10:00
Krzysztof Kotlarek f331b5eab2 FEATURE: topic title is validated for blocked words (#8127)
Currently, the topic is only validated for censored words and should be validated for blocked words as well.

Blocked word validation is now used by both Post and Topic. To avoid code duplication, I extracted blocked words validation code into separate Validator, and use it in both places.

The only downside is that even if the topic contains blocked words validation message is saying "Your post contains a word that's not allowed: tomato" but I think this is descriptive enough.
2019-10-02 10:38:34 +10:00
Neil Lalonde 279a94da90 Version bump to v2.4.0.beta5 2019-10-01 16:51:36 -04:00
Gerhard Schlager 9e4fb262cf FIX: Respect unicode whitelist when suggesting username 2019-10-01 20:33:09 +02:00
Joffrey JAFFEUX cb8fa46970
DEV: prevents populate task to crash (#8111)
Generated emails/names/sentences were crashing with the following error:

```
Can not transliterate strings with ASCII-8BIT encoding
```
2019-09-22 05:18:48 -07:00
Daniel Waterworth 563253e9ed FIX: Fix options given to per-minute rate limiter
Previously the options for the per-minute and per-10-second rate
limiters were the same.
2019-09-20 10:48:59 +01:00
Robin Ward 3c6a5836c2 FIX: Sensitivity did not work by default
Forums without previously calculated scores would return the same values
for low/medium/high sensitivity. Now those are scaled based on the
default value.

The default value has also been changed from 10.0 to 12.5 based on
observing data from live discourse forums.
2019-09-19 13:26:17 -04:00
Penar Musaraj c6cfbebf1f
FIX: ignore min_trust_to_send_messages when messaging groups (#8104)
This means that TL0 users can message groups with "Who can message this
group?" set to "Everyone".

It also means that members of a group with "Who can message this
group?" set to "members, moderators and admins" can also message the
group, even when their trust level is below min_trust_to_send_messages.
2019-09-18 15:23:13 -04:00
Robin Ward 7ae071282a FIX: Only apply post hide logic to flag actions 2019-09-18 09:39:09 -04:00
David Taylor 479fdaaea1
DEV: Allow specifying button class in reviewable action definitions (#8093)
This avoids the need for using `@extend` in SCSS, which can be problematic in plugins

For context, see https://review.discourse.org/t/fix-make-compatible-with-debundled-plugin-css-assets-feature/5297/7
2019-09-18 11:28:59 +01:00
Sam Saffron 1ca257be79 DEV: db:migrate no longer works after db:schema:load
In Rails 6 due to internal changes, the following sequence no longer works:

```
RAILS_ENV=test bin/rake db:migrate
RAILS_ENV=test bin/rake db:schema:dump
dropdb discourse_test
createdb discourse_test
RAILS_ENV=test bin/rake db:schema:load
RAILS_ENV=test bin/rake db:migrate
```

What appears to be happening is that our tracking of plugin migrations is
being missed on schema:dump or load.

A more comprehensive fix restoring schema:dump / load support will be
investigated.
2019-09-18 13:17:49 +10:00
Penar Musaraj 3debdc8131 SECURITY: XSS when oneboxing user profile location field
The XSS here is only possible if CSP is disabled. Low impact since CSP is enabled by default in SiteSettings.
2019-09-17 16:12:50 -04:00
Robin Ward c3bbf643b1 FIX: Put back the TL3 -> TL0 spam thing
We talked about it and decided it's still relevant in the score world.
2019-09-17 16:04:46 -04:00
Robin Ward 4cd620e36e Remove special cases for flagging
Prior to the new review queue there were a couple special cases where
posts would be auto hidden:

* If a TL3 or above flagged a TL0 post as spam
* If a TL4 or above flagged a non-staff, non-TL4 post as spam, inappropriate or off
topic.

These cases are now removed in favour of the scoring system.
2019-09-17 13:44:15 -04:00
David Taylor e1e8cac58f FIX: Correct theme SCSS error handling 2019-09-17 10:20:32 +01:00
David Taylor e74f851728 FIX: Live reload plugin stylesheets when editing in development 2019-09-17 09:54:59 +01:00
David Taylor 3da9b99dbf FIX: Live reload plugin stylesheets when the color scheme changes 2019-09-17 09:54:55 +01:00
David Taylor 081c36a459 FIX: Do not include theme variables in plugin SCSS, and fix register_css 2019-09-17 09:54:52 +01:00
Sam Saffron 445d305154 DEV: initial migration can fail
db:migrate can issue translations due to module loading localizing
end user messages

This allows db:migrate to work even when db is blank
2019-09-17 13:38:01 +10:00
Sam Saffron 025d4ee91f FIX: Rails 6 multisite migrations and plugin migrations
Prior to this change plugin migrations were not working and multisite
migrations not working.

Rails internals changed so we need to account for it.

Specifically semantics of `db:migrate` in rails changed so it is sort of
a "multisite:migrate".
2019-09-13 09:17:54 +10:00
Roman Rizzi 568232052e
DEV: Remove FlagQuery class and old code (#8064) 2019-09-12 13:21:33 -03:00
David Taylor 67a98946b8 FIX: Do not log 'pull_hotlinked_images' edits in the staff action log 2019-09-12 15:55:45 +01:00
Sam Saffron 082f59842d DEV: bind for thread local vars should yield block
followup on #015051ec without this improvement monkey patch does not work.

bind should yield the block it is passed for it to work.
2019-09-12 17:57:57 +10:00
Sam Saffron 015051ecaf PERF: avoid spinning a thread each time we close a connection
This is a temporary workaround for the issue in https://github.com/rails/rails/pull/36949

Discussing a proper fix in Rails with the Rails team.

Prior to this fix we were spinning up a thread every time we closed a connection
to the db.
2019-09-12 17:34:04 +10:00
Krzysztof Kotlarek 32b8a2ccff DEV: Upgrade Discourse to Rails 6 (#8083)
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
2019-09-12 10:41:50 +10:00
Gerhard Schlager 56c6e8e961 FIX: POP3 doesn't work with TLS 1.3 2019-09-11 18:43:19 +02:00
Michael Brown 503a11cc88 FIX: inline_uploads and subfolder (#8076)
* FIX: inline_uploads and subfolder

* if subfolder, also look for images with a path containing
  cdn_url + relative_url_root

* FIX: migrate_to_s3 task and subfolder
2019-09-11 11:50:48 +10:00
Bianca Nenciu 0d22beb81d
FIX: Improve Onebox detection (#8019)
Follow-up to 7c83d2eeb2.
2019-09-10 13:59:48 +03:00
Kyle Zhao 9b10a78d82 FEATURE: Quick access panels in user menu (#8073)
* Extract QuickAccessPanel from UserNotifications.

* FEATURE: Quick access panels in user menu.

This feature adds quick access panels for bookmarks and personal
messages. It allows uses to browse recent items directly in the user
menu, without being redirected to the full pages.

* REFACTOR: Use QuickAccessItem for messages.

Reusing `DefaultNotificationItem` feels nice but it actually requires a
lot of extra work that is not needed for a quick access item.

Also, `DefaultNotificationItem` shows an incorrect tooptip ("unread
private message"), and it is not trivial to remove / override that.

* Use a plain JS object instead.

An Ember object was required when `DefaultNotificationItem` was used.

* Prefix instead suffix `_` for private helpers.

* Set to null instead of deleting object keys.

JavaScript engines can optimize object property access based on the
object’s shape. https://mathiasbynens.be/notes/shapes-ics

* Change trivial try/catch to one-liners.

* Return the promise in case needs to be waited on.

* Refactor showAll to a link with href

* Store `emptyStatePlaceholderItemText` in state.

* Store items in Session singleton instead.

We can drop `staleItems` (and `findStaleItems`) altogether. Because
`(old) items === staleItems` when switching back to a quick access
panel.

* Add `limit` parameter to the `user_actions` API.

* Explicitly import Session instead.
2019-09-09 11:03:57 -04:00
Robin Ward 800e49f16e Version bump to 2.4.0.beta4 2019-09-06 16:08:03 -04:00
Gerhard Schlager 631315624d FIX: Topics with muted tag didn't show up when filtering by category and tag
It also removes the redundant `filter` parameter. Previously URLs looked like this:

```
http://example.com/tags/c/some-category/muted-tag/l/latest.json?filter=tags/c/some-category/muted-tag/l/latest
```

But it looks like the `filter` parameter was only used to find out if topics with a muted tag should be removed or not. But the same thing can be accomplished by using the first tag ID. The following URL looks a lot cleaner.

```
http://example.com/tags/c/some-category/muted-tag/l/latest.json
```
2019-09-06 20:38:03 +02:00
Rimian Perkins 6bbd83067d FEATURE: New post editing period for >= tl2 users (#8070)
* FEATURE: Add tl2 threshold for editing new posts

* Adds a new setting and for tl2 editing posts (30 days same as old value)
* Sets the tl0/tl1 editing period as 1 day

* FIX: Spec uses wrong setting

* Fix site setting on guardian spec

* FIX: post editing period specs

* Avoid shared examples
* Use update_columns to avoid callbacks on user during tests
2019-09-06 07:44:12 -04:00
Neil Lalonde e90636eadc Version bump to v2.4.0.beta3 2019-09-04 11:16:01 -04:00
Gerhard Schlager 874985db71 DEV: Ignore duplicate prefixes when generating release notes 2019-09-04 15:45:48 +02:00
Sam Saffron ed00f35306 FEATURE: improve performance of anonymous cache
This commit introduces 2 features:

1. DISCOURSE_COMPRESS_ANON_CACHE (true|false, default false): this allows
you to optionally compress the anon cache body entries in Redis, can be
useful for high load sites with Redis that lives on a separate server to
to webs

2. DISCOURSE_ANON_CACHE_STORE_THRESHOLD (default 2), only pop entries into
redis if we observe them more than N times. This avoids situations where
a crawler can walk a big pile of topics and store them all in Redis never
to be used. Our default anon cache time for topics is only 60 seconds. Anon
cache is in place to avoid the "slashdot" effect where a single topic is
hit by 100s of people in one minute.
2019-09-04 17:18:32 +10:00
Sam Saffron dc9110cc43 FEATURE: track date api key was last used
Start tracking the date an api key was last used. This has already been
the case for user_api_keys.

This information can provide us with the ability to automatically expire
unused api keys after N days.
2019-09-03 18:11:30 +10:00
Sam Saffron b9954b53bb FIX: report cached controller and action to loggers
Previously we would treat all cached hits in anon cache as "other"

This hinders analysis of cache performance and makes logging inaccurate
2019-09-03 10:55:16 +10:00
Sam Saffron 08743e8ac0 FEATURE: anon cache reports data to loggers
This allows custom plugins such as prometheus exporter to log how many
requests are stored in the anon cache vs used by the anon cache.

This metric allows us to fine tune cache behaviors
2019-09-02 18:45:35 +10:00