Commit Graph

7460 Commits

Author SHA1 Message Date
Roman Rizzi c751291769
FIX: The 'reviewed' status filter should include deleted elements (#8630) 2019-12-30 14:56:17 -03:00
Robin Ward 5214f82841 API endpoints should return a response, not nil 2019-12-30 10:14:29 -05:00
David Taylor bc4c40abd4
DEV: Remove unsafe-eval from development CSP (#8569)
- Refactor source_url to avoid using eval in development
- Precompile handlebars in development
- Include template compilers when running qunit
- Remove unsafe-eval in development CSP
- Include unsafe-eval only for qunit routes in development
2019-12-30 12:17:12 +00:00
David Taylor df8444e813
FIX: Update topic/post counter correctly when category has zero topics (#8600)
Previously, categories without any topics were being excluded from the UPDATE query. This means the counter gets stuck, and the category cannot be deleted. This change ensures that the counters get correctly set to zero.
2019-12-30 11:20:44 +00:00
David Taylor 9348d2cfdf
PERF: Cache user badge count in user_stats table (#8610)
This means that we no longer need to run a `count()` on the user_badges table when serializing a user
2019-12-30 11:19:59 +00:00
Arpit Jalan 1820347d58 fix the build (take 2). 2019-12-24 19:27:35 +05:30
Krzysztof Kotlarek ac626a0dc3
FIX: bug when revoking badge as title (#8616)
When user revokes badge as title, UserHistory item is created. However,
that object is not accepting revoke_reson parameter.
2019-12-24 08:03:51 +11:00
Jeff Wong eca1c643d5 FEATURE: allow TL3 promotions for overturned penalties
Restore tl3 reachability by calculating penalty counts vs unsuspend/unsilence.

Only count unsuspend or unsilences that have been made by a user other than
the Discourse system user.
2019-12-20 15:25:21 -08:00
Mark VanLandingham 6ec3d42b16
FEATURE: Allow complex post params from plugin (#8598) 2019-12-20 10:37:12 -06:00
David Taylor 61919ad39f
UX: Do not use avatars as fallback opengraph images for replies (#8605)
People rarely want to have their avatars show up as the preview image on social media platforms. Instead, we should fall back to the site opengraph image.
2019-12-20 13:17:14 +00:00
Bianca Nenciu 1bccd8eca9
FIX: Remove full nested quotes on direct reply (#8581)
It used to check how many quotes were inside a post, without taking
considering that some quotes can contain other quotes. This commit
selects only top level quotes.

I had to use XPath because I could not find an equivalent CSS
selector.
2019-12-20 10:24:34 +02:00
Jeff Wong 036a24cf30
FIX: correct user serializer user method for extended serializer (#8590)
A small fix for Basic User Serializers where some downstream serializers do not correctly set user objects. This caused some issues in certain plugins that depend on the user method to return a user.
2019-12-19 09:48:01 -08:00
Vinoth Kannan 3b7f5db5ba
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547) 2019-12-18 11:21:57 +05:30
Penar Musaraj ea05a68df7
FIX: Use updated_at date to denote expired invites (#8521) 2019-12-17 10:13:49 -05:00
David Taylor 8106d94c05 SECURITY: Correct permission check when revoking user API keys 2019-12-17 10:56:16 +00:00
Joffrey JAFFEUX 81b4de39ee
FIX: prevents crash in discourse_tagging with empty term (#8548) 2019-12-17 10:55:06 +01:00
Martin Brennan 6ab12ed96b
FIX: If a prettified slug is a number, return defaultt (#8554)
Meta thread: https://meta.discourse.org/t/sending-a-pm-with-the-following-title-causes-an-error/135654/3

We had an issue where if someone sent a PM with crazy
characters that are stripped and we end up with only
a number, the topic redirect errored because the slug was
a number. so instead we return the default as well if
the slug is a number after prettification
2019-12-17 10:34:20 +10:00
Martin Brennan beb91e7eff
FIX: require: false for rotp gem (#8540)
The ROTP gem is only used in a very small amount of places in the app, we don't need to globally require it.

Also set the Addressable gem to not have a specific version range, as it has not been a problem yet.

Some slight refactoring of UserSecondFactor here too to use SecondFactorManager to avoid code repetition
2019-12-17 10:33:51 +10:00
Mark VanLandingham 8c4ffaea1b
FEATURE: Modal for profile featured topic & admin wrench refactor (#8545) 2019-12-16 08:41:34 -08:00
Krzysztof Kotlarek d8360b4c82 SECURITY: vulnerability in WildcardUrlChecker 2019-12-13 09:29:09 -05:00
David Taylor f47909a2b6 DEV: Check for pending migrations before running rspec suite
This should catch a few scenarios which can waste a lot of time in development
 - Forgetting to run migrations
 - Missing plugin migrations
 - Missing post_deploy migrations
2019-12-13 12:50:18 +00:00
Gerhard Schlager c068804a0a Fixing the specs. Take 2. 2019-12-13 13:05:17 +01:00
Gerhard Schlager 2e8c23f0d0 Fix spec 2019-12-13 12:40:52 +01:00
Roman Rizzi b6a2875749
FIX: Granting staff status should auto-approve users waiting approval (#8533)
* FIX: Granting staff status should auto-approve users waiting approval

* Update app/models/concerns/roleable.rb

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2019-12-12 16:26:38 -03:00
Robin Ward 76f676879c FIX: Constraint error when inserting the same topic group twice
If that happens, update the value instead of raising an error.
2019-12-12 13:10:46 -05:00
David Taylor 4c9ca24ccf
FEATURE: Hash API keys in the database (#8438)
API keys are now only visible when first created. After that, only the first four characters are stored in the database for identification, along with an sha256 hash of the full key. This makes key usage easier to audit, and ensures attackers would not have access to the live site in the event of a database leak.

This makes the merge lower risk, because we have some time to revert if needed. Once the change is confirmed to be working, we will add a second commit to drop the `key` column.
2019-12-12 11:45:00 +00:00
Bianca Nenciu 3ec2081059
UX: Include public groups in mentionable groups set (#8516) 2019-12-12 13:13:40 +02:00
Martin Brennan 9f4c9bafa1
FIX: Hide old bookmark button on post-menu if SiteSetting.enable_bookmarks_with_reminders
Hide old bookmark post-menu item if the site setting for the new bookmark reminders is enabled and change icon for the new bookmark functionality to the same as the old bookmark button
Fix null @topic_view error in post serializer for post_bookmark, as new posts do not have a @topic_view
2019-12-12 16:28:01 +10:00
Sam Saffron 9d6a9bba96 DEV: correct flaky test
This is a slight workaround which helps somewhat now but is pending a larger
fix.

When this spec ran in parallel mode uploads could start cross talking and
an upload you expect to be there may vanish.

This works around the issue by making the upload unique every time it is
created

It also folds up an expensive test into the main one.
2019-12-12 16:54:22 +11:00
Martin Brennan edbc356593
FIX: Replace deprecated URI.encode, URI.escape, URI.unescape and URI.unencode (#8528)
The following methods have long been deprecated in ruby due to flaws in their implementation per http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/29293?29179-31097:

URI.escape
URI.unescape
URI.encode
URI.unencode
escape/encode are just aliases for one another. This PR uses the Addressable gem to replace these methods with its own encode, unencode, and encode_component methods where appropriate.

I have put all references to Addressable::URI here into the UrlHelper to keep them corralled in one place to make changes to this implementation easier.

Addressable is now also an explicit gem dependency.
2019-12-12 12:49:21 +10:00
Mark VanLandingham 09d9baa6d7
FIX: Update S3 stubs for more aws-sdk API changes (#8534) 2019-12-11 11:26:52 -08:00
dependabot-preview[bot] 863d3f89de Build(deps): Bump aws-sdk-s3 from 1.36.1 to 1.59.0 (#8502)
* Build(deps): Bump aws-sdk-s3 from 1.36.1 to 1.59.0

Bumps [aws-sdk-s3](https://github.com/aws/aws-sdk-ruby) from 1.36.1 to 1.59.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-s3/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/v1.36.1...v1.59.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Dependabot/bundler/aws sdk s3 1.59.0 (#8532)
2019-12-11 10:18:11 -08:00
Dan Ungureanu 4e130f1e03
SECURITY: Remove event handlers from SVG files 2019-12-11 17:04:50 +02:00
Dan Ungureanu adfa793731
SECURITY: Ensure only image uploads can be inlined
This prevents malicious files (for example special crafted XMLs) to be
used in XSS attacks.
2019-12-11 17:04:49 +02:00
dependabot-preview[bot] b90a592146 DEV: Bump aws-sdk-sns from 1.13.0 to 1.21.0 (#8490)
Bumps [aws-sdk-sns](https://github.com/aws/aws-sdk-ruby) from 1.13.0 to 1.21.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-sns/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/1.13.0...1.21.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 06:13:17 -08:00
Martin Brennan 6261339da9
Improving bookmarks part 1 (#8466)
Note: All of this functionality is hidden behind a hidden, default false, site setting called `enable_bookmarks_with_reminders`. Also, any feedback on Ember code would be greatly appreciated!

This is part 1 of the bookmark improvements. The next PR will address the backend logic to send reminder notifications for bookmarked posts to users. This PR adds the following functionality:

* We are adding a new `bookmarks` table and `Bookmark` model to make the bookmarks a first-class citizen and to allow attaching reminders to them.
* Posts now have a new button in their actions menu that has the icon of an actual book
* Clicking the button opens the new bookmark modal.
* Both name and the reminder type are optional.
* If you close the modal without doing anything, the bookmark is saved with no reminder.
* If you click the Cancel button, no bookmark is saved at all.
* All of the reminder type tiles are dynamic and the times they show will be based on your user timezone set in your profile (this should already be set for you).
* If for some reason a user does not have their timezone set they will not be able to set a reminder, but they will still be able to create a bookmark.
* A bookmark can be deleted by clicking on the book icon again which will be red if the post is bookmarked.

This PR does NOT do anything to migrate or change existing bookmarks in the form of `PostActions`, the two features live side-by-side here. Also this does nothing to the topic bookmarking.
2019-12-11 14:04:02 +10:00
Neil Lalonde 20464116ae FIX: tag input doesn't show all top 5 permitted tags 2019-12-10 10:30:01 -05:00
Sam Saffron 0c52537f10 DEV: update rubocop to version 0.77
We like to stay as close as possible to latest with rubocop cause the cops
get better.

This update required some code changes, specifically the default is to avoid
explicit returns where implicit is done

Also this renames a few rules
2019-12-10 11:48:39 +11:00
Krzysztof Kotlarek 81c7d6a462
FIX: Muted tags are respected by TopicTrackingState (#8467)
When the tag is muted and topic contains that tag, we should not mark that message as NEW.

There are 3 possible settings which site admin can set.
remove_muted_tags_from_latest - always
It means that if the topic got at least one muted tag, we should not mark that topic as NEW

remove_muted_tags_from_latest - only muted
Similar to above, however, if at least one tag is not muted, the topic is marked as NEW

remove_muted_tags_from_latest - never
Basically, mute tag setting is ignored and all topics are set as NEW
2019-12-10 09:50:05 +11:00
Mark VanLandingham 14cb386f1e
FEATURE: Featured topic for user profile & card (#8461) 2019-12-09 11:15:47 -08:00
Dan Ungureanu aa24be1a9a
DEV: Extend plugin API for uploads (#8440)
* DEV: Add API to alter uploads Markdown

* DEV: Extract data attributes from image / download Markdown

For example '[test|attachment|hello=world]' will generate an 'a' element
with a data attribute: 'data-hello=world'.

This commit also makes MarkdownIt to transform '|attachment' into
'class="attachment"'. This transformation used to be a part of the
process which resolves short URLs (i.e. upload://).

* DEV: Export imageNameFromFileName
2019-12-09 16:20:03 +02:00
Dan Ungureanu ebe6fa95be
FIX: Optimize images in Onebox (#8471)
This commit ensures that images in Onebox are being optimized, but not
converted to lightbox too.
2019-12-09 15:39:25 +02:00
Sam Saffron 494fe335d3 DEV: allow handling crawler reqs with no user agent
Followup to e440ec25 we treat no user agent as crawler reqs.
2019-12-09 18:40:10 +11:00
Sam Saffron e440ec2519 FIX: crawler requests not tracked for non UTF-8 user agents
Non UTF-8 user_agent requests were bypassing logging due to PG always
wanting UTF-8 strings.

This adds some conversion to ensure we are always dealing with UTF-8
2019-12-09 17:43:51 +11:00
Jarek Radosz 067c4a7c10 DEV: Fix CookedPostProcessor specs
Broken in #8480
2019-12-09 03:16:25 +01:00
Gerhard Schlager c88797bf0e FIX: Ignore DMARC for emails sent to mailing list mirror 2019-12-06 13:29:39 +01:00
Sam Saffron a06fccae1b DEV: update dependencies and add notes about exceptions
Previously it was unclear why certain gems are being held back cause Gemfile
had no comment explaining it.

I tried to add some explanation from memory and remove some exceptions that
seemed to be superfluous.

This upgrades shoulda to latest, it appears to work once a couple of assertions
are removed

Also update http accept language used to auto detect language from http header
this is tested

Zeitwerk small update seems fine
2019-12-06 13:00:28 +11:00
Vinoth Kannan f7084a4339 FEATURE: add site setting to remove `X-Frame-Options` header. 2019-12-06 03:15:09 +05:30
Vinoth Kannan 11c4d5eda7 Fix the build - take 3.
754e1238b0
2019-12-05 20:35:39 +05:30
Vinoth Kannan e6dfcda0bc DEV: handle all notification consolidations in new 'NotificationConsolidator' class.
481c8314f0
2019-12-05 14:36:06 +05:30
Jarek Radosz d07f039468 FIX: Secure Upload URLs in lightbox (#8451)
This fixes the following issues:

* The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image
* Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 09:13:09 +10:00
Neil Lalonde 875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
Vinoth Kannan 15c2755b7b Fix the `WebHookPostSerializer` spec.
400f79cffc
2019-12-04 23:30:04 +05:30
Robin Ward 888d56774a
DEV: HTML Builders should respect if a plugin is enabled or not (#8454)
Previously they would return the HTML regardless of whether the plugin
was enabled or not.
2019-12-04 12:26:23 -05:00
David Taylor be1510b09d FIX: Use filtered posts when determining the next page
This bug was causing some unusual behavior when the last post is filtered (e.g. from an ignored user). In some situations this would cause suggested topics to be omitted from the payload.

The next_page specs have been updated to remove most of the stubs
2019-12-04 12:52:24 +00:00
Krzysztof Kotlarek 46fc45de99
FIX: inbound link when the only slug available (#8457)
Problem mentioned in [meta](https://meta.discourse.org/t/inbound-links-dont-show-up-when-topic-id-is-not-included/100551)

When there is an internal link without ID, only slug, we should still try to create reflection link.
2019-12-04 17:13:20 +11:00
Gerhard Schlager 9ebb69e8eb FIX: Respect `enable_inline_emoji_translation` setting in titles 2019-12-03 20:39:11 +01:00
romanrizzi c86ca3609e FIX: Filter readers avatars correctly when the post is a whisper 2019-12-03 10:50:02 -03:00
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Arpit Jalan 016732cced FEATURE: improve suspect user discovery 2019-12-02 16:39:28 +05:30
Bianca Nenciu 57f108899a
FEATURE: Make site texts controller handle pages and locales (#8408)
Some endpoints are returning i18n keys instead of translated messages
and with these changes, the site_texts endpoint can help translating
those.

Pagination part is needed for better wildcard support. For example,
looking for 'js.notifications' would set 'has_more' to true, but return
only the first 50 messages with no way of fetching the remaining.
2019-11-30 15:16:36 +02:00
David Taylor 06c2e28bbb DEV: Remove two more references to instagram_user_infos
Follow-up to ad6f33e5b1
2019-11-29 17:12:06 +00:00
David Taylor a6aada16bd DEV: Refactor API key specs to avoid hard-coding keys
By hard-coding keys, we are not testing the API key system end to end. This change also makes the specs more resilient to upcoming API key changes
2019-11-29 15:16:22 +00:00
Robin Ward 7fee3c61de
Support for custom messages and redirects when creating posts (#8434)
* Support for custom messages and redirects when creating posts

When a post/topic is created Discourse serializes a `NewPostResult`
object. Normally this contains a status like `created_post` or
errors describing why the post could not be created.

There are times when a plugin might want to take the inputted post
and do something in the background. In this case, the plugin
can return a custom `message` and `route_to` attribute in the
`NewPostResult`.

If present, the message will be displayed in an alert, and when "Ok" is
clicked the user will be routed to the new URL.

* Destroy the draft in parallel
2019-11-29 09:30:54 -05:00
Dan Ungureanu 1e0c2235a3
FIX: Optimize quoted images (#8427)
Only images that were part of a lightbox used to be optimized. This
patch ensures that quoted images are also optimized.
2019-11-29 15:18:42 +02:00
Krzysztof Kotlarek b120728999
FEATURE: Ability to add components to all themes (#8404)
* FEATURE: Ability to add components to all themes

This is the first and functional step from that topic https://dev.discourse.org/t/adding-a-theme-component-is-too-much-work/15398/16

The idea here is that when a new component is added, the user can easily assign it to all themes (parents).

To achieve that, I needed to change a site-setting component to accept `setDefaultValues` action and `setDefaultValuesLabel` translated label.
Also, I needed to add `allowAny` option to disable that for theme selector.

I also refactored backend to accept both parent and child ids with one method to avoid duplication (Renamed `add_child_theme!` to more general `add_relative_theme!`)

* FIX: Improvement after code review

* FIX: Improvement after code review2

* FIX: use mapBy and filterBy directly
2019-11-28 16:19:01 +11:00
Vinoth Kannan 481c8314f0 FEATURE: consolidate group membership request notifications. 2019-11-28 04:02:35 +05:30
Martin Brennan 901054fd75
FIX: Cache failed onebox URL request server-side (#8421)
We already cache failed onebox URL requests client-side, we now want to cache this on the server-side for extra protection. failed onebox previews will be cached for 1 hour, and any more requests for that URL will fail with a 404 status. Forcing a rebake via the Rebake HTML action will delete the failed URL cache (like how the oneboxer preview cache is deleted).
2019-11-28 07:48:29 +10:00
Martin Brennan e7c7a05097
FIX: Mark secure media upload insecure automatically if used for theme component (#8413)
When uploading a file to a theme component, and that file is existing and has already been marked as secure, we now automatically mark the file as secure: false, change the ACL, and log the action as the user (also rebake the posts for the upload)
2019-11-28 07:32:17 +10:00
Gerhard Schlager d12f2580de FIX: Serve crawler view to Google PageSpeed 2019-11-27 22:15:34 +01:00
Roman Rizzi 1c9d18f094
FIX: Correctly ignore/approve replies when acting on a flagged post (#8425) 2019-11-27 17:19:44 -03:00
David Taylor a227083c1c
FEATURE: Limit the number of active sessions for a user (#8411)
If a user has more than 60 active sessions, the oldest sessions will be terminated automatically. This protects performance when logging in and when loading the list of recently used devices.
2019-11-27 12:39:31 +00:00
Sam Saffron 88ecb650a9 DEV: Implement a faster Discourse.cache
This is a bottom up rewrite of Discourse cache to support faster performance
and a limited surface area.

ActiveSupport::Cache::Store accepts many options we do not use, this partial
implementation only picks the bits out that we do use and want to support.

Additionally params are named which avoids typos such as "expires_at" vs "expires_in"

This also moves a few spots in Discourse to use Discourse.cache over setex
Performance of setex and Discourse.cache.write is similar.
2019-11-27 16:11:49 +11:00
Sam Saffron 0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
Leo McArdle 2714149fd2 FEATURE: hide posts from incoming email based on dmarc verdict (#8333) 2019-11-26 15:55:22 +01:00
Sam Saffron 941162e90e FIX: draft not clearing when replying to new topic
This amends our API so we provide it with the draft key when saving a post
this means post creator can clean up the draft consistently even if we are
doing fancy stuff like replying to a new topic or new pm or whatever.

There will be some followup work to clean it up so client never calls destroy
on draft during normal operation and the #create/#update endpoints takes care of it
every time
2019-11-26 18:23:20 +11:00
Sam Saffron ebc82eb63e DEV: correct spec failures in PG 12
PG 12 changes internals in a subtle way, time jitter is noticed in a few new
spots (which is normal) and default ordering is a bit different which is meant
to be random anyway.
2019-11-26 16:39:14 +11:00
Krzysztof Kotlarek 6e403f20ee
FIX: Don't error when the empty current value in dif (#8406)
If current value is nil we should use `&.` combined with `dig` to protect diff from erroring

It is happening when for example theme is delete (new value is empty)
2019-11-26 09:17:14 +11:00
Dan Ungureanu a992caf741
DEV: Replace magic values (#8398)
Follow-up to 35942f7c7c.
2019-11-25 14:32:19 +02:00
Dan Ungureanu ae9e881333
DEV: Add test to ensure :after_auth event is triggered (#8400)
Follow-up to ee8669d778.
2019-11-25 14:31:57 +02:00
Martin Brennan afb5533581
FEATURE: Add timezone to core user_options (#8380)
* Add timezone to user_options table

* Also migrate existing timezone values from UserCustomField,
  which is where the discourse-calendar plugin is storing them

* Allow user to change their core timezone from Profile

* Auto guess & set timezone on login & invite accept & signup

* Serialize user_options.timezone for group members. this is so discourse-group-timezones can access the core user timezone, as it is being removed in discourse-calendar.

* Annotate user_option with timezone

* Validate timezone values
2019-11-25 10:49:27 +10:00
Krzysztof Kotlarek 43ddf60cdf
FIX: When dismissing category inform via MessageBus (#8371)
When category is dismissed, `dismiss_new` message is sent to fronted to clean state.

In addition, I noticed that when old dismiss new button is clicked, no message is sent so I decided to kill two birds with one stone.
2019-11-25 06:17:31 +11:00
Roman Rizzi c92f0b8775
FEATURE: New API to apply custom filters to the review queue (#8392) 2019-11-22 16:33:10 -03:00
Martin Brennan 23714e77c4
FIX: Always return secure_proxy_without_cdn url for secure media (#8394)
There was an issue on dev where when uploading secure media, the href of the media was correctly being replaced in the CookedPostProcessor, but the srcset urls were not being replaced correctly. This is because UrlHelper.cook_url was returning the asset host URL for the media for secure media instead of returning early with the proxied secure proxy url.
2019-11-22 15:29:31 +10:00
Penar Musaraj 11d22293fb FIX: Allow private media uploads to be reused in login_required sites
In non-login-required sites, we prevent secure uploads already used in PMs from being used in public topics.

In login_required sites, secure uploads should be reusable in any topic, PM or not.
2019-11-21 09:14:06 -05:00
Martin Brennan 5ad6dc01c1
FEATURE: Support RS256 algorithm for webauthn (#8385)
* Add support for RS256 webauthn algorithm (-257)
2019-11-21 15:45:36 +10:00
Sam Saffron b57e108e84 FEATURE: improve email change workflow
- Show old and new email address during the process
- Ensure correct user is logged on when attempting to make email changes
- Support reloading a page during the email reset process without resubmit
of form
- Improve tests
- Fixed issue where redirect back to site was not linking correctly in
subfolder setups

Internal refactor of single action into 4 distinct actions that are simpler
to reason about.

This also removes the step that logs on an account after you confirm an
email change, since it is no longer needed which leaves us with safer
internals.

This left me no choice but to amend translations cause the old route was
removed.
2019-11-21 16:28:35 +11:00
Sam Saffron 423ad5f0a4 FIX: do not log if an invalid mime type is passed to app
Previously our custom exception handler was unable to handle situations
where an invalid mime type was sent, resulting in a warning log

This ensures we pretend a request is HTML for the purpose of rendering
the error page if an invalid mime type from a scanner is shipped to the app
2019-11-21 15:51:34 +11:00
Martin Brennan d45bd7f131
FIX: Abort CensoredWordsValidator early if censored_words_regexp nil (#8375)
* Abort CensoredWordsValidator early if censored_words_regexp nil. Sometimes censored_words_regex can end up nil, erroring the validator. This handles the nil condition and also adds a spec for the validator
2019-11-20 20:05:06 +10:00
Sam Saffron dd89a13e22 DEV: disable flaky spec
This is erratically failing in our production CI
2019-11-20 08:56:07 +11:00
Martin Brennan 02cb01406e
FIX: Allow secure uploads if global s3 setting active and enable_s3_uploads validations (#8373)
The secure media functionality relied on `SiteSetting.enable_s3_uploads?` which, as we found in dev, did not take into account global S3 settings via `GlobalSetting.use_s3?`. We now use `SiteSetting.Upload.enable_s3_uploads` instead to be more consistent.

Also, we now validate `enable_s3_uploads` changes, because if `GlobalSetting.use_s3?` is true users should NOT be enabling S3 uploads manually.
2019-11-20 07:46:44 +10:00
David Taylor 46841888b7 FIX: For a single authenticator, do not interrupt registration flow
Followup to 0a14b9b42a
2019-11-19 19:15:11 +00:00
Neil Lalonde 565a967192 FIX: email excerpts for posts starting with a quote were displaying a username
If a post starts with a post quote and has no other text content,
then the email excerpt was the name of the person quoted and
nothing else. The intention was to show the contents of the
first paragraph or div after the quote.

With this change, a quote followed by an image will use the
image as the excerpt. A quote followed by a onebox will use the
onebox.
2019-11-19 12:35:23 -05:00
Blake Erickson 266e486037 FIX: Turn off auto bumping for topics with scheduled bumps
If a topic has a timer scheduled to bump a topic it should be excluded
from being auto bumped.
2019-11-19 07:27:49 -07:00
Neil Lalonde a4dbec5bf9 FIX: unable to remove required tag group from a category 2019-11-18 15:54:00 -05:00
Neil Lalonde 228c4814be FIX: errors when using tags with colons in their name 2019-11-18 13:20:37 -05:00
Penar Musaraj 0e759add0f DEV: Better topic participants summary spec 2019-11-18 09:43:14 -05:00
Vinoth Kannan 3bb7ad4be1
FEATURE: remove support for 'suppress_from_latest' category setting. (#8308) 2019-11-18 12:28:35 +05:30
Sam Saffron 7d389df5e7 DEV: correct spec to allow for new default
b4bfc27b changes the default so the spec should be changed as well.
2019-11-18 16:05:58 +11:00
Martin Brennan af091c49e9
FIX: Ensure revisions are made to store edit reasons and no reasons get wiped (#8363)
* Fix an issue where if an edit was made to a post with a reason provided, and then another edit was made with no reason, the original edit reason got wiped out
* We now always make a post revision (even with ninja edits) if an edit reason has been provided and it is different from the current edit reason

Co-Authored-By: Sam <sam.saffron@gmail.com>
2019-11-18 13:08:54 +10:00
Penar Musaraj 102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Vinoth Kannan 56b19ba740 UX: instead of total user count display only the count of users going to be affected.
edec922803
2019-11-18 00:09:38 +05:30
Penar Musaraj 99b4f79a90 FIX: Include 5 participants in topic summary 2019-11-15 15:11:09 -05:00
Roman Rizzi 4cf3c9cccb
FEATURE: Filter reviewables by date range (#8354) 2019-11-15 15:29:59 -03:00
Martin Brennan 5c59247c3a Block enabling force 2FA if local logins disabled & vice-versa (#8355) 2019-11-15 17:05:10 +11:00
Sam Saffron e7cf4579a8 DEV: improve usability of subfolder specs
Previously people were not consistent about mocking which left internals in
a fragile state when running subfolder specs.

This introduces a simple helper `set_subfolder` which you can use to set
the subfolder for the spec. It takes care of proper configuration of subfolder
and teardown.

```
# usage
set_subfolder "/my_amazing_subfolder"
```

You should no longer stub base_uri or global_settings
2019-11-15 16:48:24 +11:00
Jeff Wong 40d14ba166 Fix the actual failing test 2019-11-14 16:20:45 -08:00
Jeff Wong 25dd045ba5 Fix failing test 2019-11-14 16:11:42 -08:00
Jeff Wong 786fcceb62 Return full URLs in review serializer for both post and topic
Return full URLs, this fixes reviewable links to support subfolder
2019-11-14 15:58:21 -08:00
Blake Erickson 3b16eb7abb FIX: Confirm new email with backup codes enabled
This is a fix for this bug:

https://meta.discourse.org/t/-/133185?u=blake

where rails would throw a missing template error when trying to confirm
a new email address when you had two factor backup codes enabled.

Apparently this feature broke during this commit:

68d35b14f4

when a partial that contained a lot of javascript was removed most
likely because it didn't comply with our Content Security Policy, so as
a fix I rewrote the previous js functionality without using any
javascript and then added a spec to verify that the correct backup code
form is displayed when that page is loaded.
2019-11-14 16:27:42 -07:00
romanrizzi ff355ad204 FIX: Don't show bots as post readers 2019-11-14 17:53:52 -03:00
Penar Musaraj 067696df8f DEV: Apply Rubocop redundant return style 2019-11-14 15:10:51 -05:00
David Taylor f2da630c19 FIX: SVG Sprite version hash should be based on bundle result
This version hash is used for the filename, and so browsers/CDNs cache based on it. Previously the version hash was based only on the list of requested icons. This can cause issues in a couple of situations, most commonly when developing themes with custom icons:

- A requested icon does not exist, and then later is added to the theme. The bundle output changes, but the hash did not
- The SVG content of an icon changes, but the name of the icon does not. The bundle output changes, but the hash did not
2019-11-14 13:20:16 +00:00
Krzysztof Kotlarek f434de2536
FIX: Tracking Topic State know about category_seen_at (#8351)
If category got last_seen_at is set TrackingTopicState should know about it and exclude those topics from marking them as new
2019-11-14 16:11:34 +11:00
Martin Brennan e7226a8c84
FEATURE: Allow scoping search to tag (#8345)
* When viewing a tag, the search widget will now show a checkbox to scope the search by tag, which will limit search results to that tag on desktop and mobile
2019-11-14 10:40:26 +10:00
Krzysztof Kotlarek 6e1fe22a9d
FEATURE: Dismiss new per category (#8330)
Ability to dismiss new topics per category.
2019-11-14 11:16:13 +11:00
David Taylor d095c2cee7 FIX: Allow importing themes with subdirectories in extra_js
The folder/file detection was broken by 9fea43e46a. This commit fixes and adds relevant specs
2019-11-13 23:45:09 +00:00
David Taylor 0a14b9b42a FEATURE: Automatically redirect to authenticator when there is only one
This brings the behavior in line with native Discourse SSO. If login is required, and a user tries to visit the forum, they will be directed straight to the external login page without requiring any clicks.
2019-11-13 17:28:12 +00:00
Neil Lalonde 7711df40e6
REFACTOR: redo DiscourseTagging.filter_allowed_tags (#8328)
This method had grown into a monster. Its query had bugs
that I couldn't fix, and new features would be hard to add.
Also I don't understand how it all works anymore...
Replace it with common table expressions that can be queried
to generate the results we need, instead of subtracting
results using lots of "NOT IN" clauses.

Fixed are bugs with tag schemas that use combinations of
tag groups, parent tags, and one-tag-per-topic restrictions.
For example: https://meta.discourse.org/t/130991/6
2019-11-12 14:28:44 -05:00
David Taylor 836b3f4d82
FIX: Do not deactivate admin accounts with recent posts or api keys (#8342)
This prevents 'bot' users being deactivated
2019-11-12 16:56:01 +00:00
Mark VanLandingham 3dd2f2f701
DEV: Remove RSS feed polling in favor of plugin (#8233) 2019-11-12 09:49:02 -06:00
David Taylor 5f927ceeb3
DEV: Display a warning when themes hard-code optimized image links (#8304) 2019-11-12 14:30:19 +00:00
Dan Ungureanu bbcce08712
FIX: Update quotes after moving posts (#8326) 2019-11-12 15:16:39 +02:00
Krzysztof Kotlarek 69266f60ed FIX: tag and category watchers regression (#8336)
I made a regression here 17366d3bcc (diff-ddeebb36d131f89ca91be9d04c2baefaR10)

When the tag is added, people watching specific tag are notified but also people watching specific category.

Therefore, `notify_post_users` should accept options who should be notified.

So when `category` is added to the topic, users watching topic and users watching category are notified.

When `tag` is added to the topic, users watching topic and users watching tag are notified

Finally, when a new post is created, everybody is notified, topic watchers, category watchers, tag watchers.
2019-11-12 16:44:46 +11:00
Gerhard Schlager 6ebffaaf6e FIX: Better error handling for invalid locale bundle versions 2019-11-11 22:30:32 +01:00
Dan Ungureanu 6672dcc985
FIX: Keep emoji images in group bio excerpt (#8329) 2019-11-11 18:42:08 +02:00
Sam Saffron 0c0192e7da DEV: disable all upload recovery specs
These specs are inherently fragile when they are run in a concurrent mode
in CI.

One process will create an upload, another will destroy the upload on disk
at the same time. We need a new safe mechanism here.
2019-11-11 16:04:18 +11:00
Sam Saffron 91daafc674 DEV: correct implementation of expiry api
Previously we were always hard-coding expiry, this allows the secure session
to correctly handle custom expiry times

Also adds a ttl method for looking up time to live
2019-11-11 11:18:12 +11:00
Dan Ungureanu dfc002d331
FIX: Count current penalty if it started more than 6 months ago (#8313) 2019-11-08 12:23:57 +02:00
Martin Brennan 56d3e29a69
FIX: Badge and user title interaction fixes (#8282)
* Fix user title logic when badge name customized
* Fix an issue where a user's title was not considered a badge granted title when the user used a badge for their title and the badge name was customized. this affected the effectiveness of revoke_ungranted_titles! which only operates on badge_granted_titles.
* When a user's title is set now it is considered a badge_granted_title if the badge name OR the badge custom name from TranslationOverride is the same as the title
* When a user's badge is revoked we now also revoke their title if the user's title matches the badge name OR the badge custom name from TranslationOverride
* Add a user history log when the title is revoked to remove confusion about why titles are revoked
* Add granted_title_badge_id to user_profile, now when we set badge_granted_title on a user profile when updating a user's title based on a badge, we also remember which badge matched the title
* When badge name (or custom text) changes update titles of users in a background job
* When the name of a badge changes, or in the case of system badges when their custom translation text changes, then we need to update the title of all corresponding users who have a badge_granted_title and matching granted_title_badge_id. In the case of system badges we need to first get the proper badge ID based on the translation key e.g. badges.regular.name
* Add migration to backfill all granted_title_badge_ids for both normal badge name titles and titles using custom badge text.
2019-11-08 15:34:24 +10:00
Martin Brennan 64b4a7ba45
FIX: Ensure enforce 2FA for staff satisfied by security keys (#8316)
* If a staff user created only a security key as their single 2FA option. they continued to be prompted to create a 2FA option because we only considered this condition satisfied if a TOTP was added.
* The condition is now satisfied if TOTP OR security keys are enabled.
2019-11-08 15:11:53 +10:00
Vinoth Kannan ba5b78a348
FEATURE: support to mute all categories by default. (#8295)
Instead of enabling `suppress_from_latest` setting on many categories now we can enable `mute_all_categories_by_default` site setting. Then users should opt-in to categories for them to appear in the latest and categories pages.
2019-11-08 08:28:11 +05:30
David Taylor 60a235d128
DEV: Allow execute_command to receive a block (#8303)
This makes it easy to run multiple commands with the same keyword arguments. The main use is for using `chdir` across multiple commands. The `Dir.chdir` method is not concurrency safe because it switches the working directory of the entire process.
2019-11-07 15:47:16 +00:00
Krzysztof Kotlarek 17366d3bcc
FEATURE: notify tag watchers when tag was added to post (#8299)
Issue was mentioned in this [meta topic](https://meta.discourse.org/t/send-a-notification-to-watching-users-when-adding-tag/125314)

It is working well when category is changed because NotifyCategoryChange job already got that code:
```
if post&.topic&.visible?
  post_alerter = PostAlerter.new
  post_alerter.notify_post_users(post, User.where(id: args[:notified_user_ids]))
  post_alerter.notify_first_post_watchers(post, post_alerter.category_watchers(post.topic))
end
```

For NotifyTagChange job notify post users were missing so it worked only when your notification was set to `watching first post`
2019-11-07 08:20:15 +11:00
Mark VanLandingham bf778d66b5
FIX: unicode group names encoded for url (#8302) 2019-11-06 14:25:45 -06:00
Penar Musaraj 0dfc594784 FIX: skip invalid URLs when checking for audio/video in search blurbs
Fixes 500 errors on search queries introduced in 580a4a8
2019-11-06 10:32:15 -05:00
Krzysztof Kotlarek 5ad0dd0c23 FIX: tweak restorer spec to make it stableish (#8300) 2019-11-06 17:05:33 +11:00
Gerhard Schlager 61b1f9c36b FEATURE: Load translation overrides without JS `eval` 2019-11-05 19:16:38 +01:00
Penar Musaraj ca6adfbdd6 DEV: Ignore renames when diffing theme changes
Renamed assets are quite noisy when checking whether a remote theme has local changes.
2019-11-05 11:52:04 -05:00
David Taylor 52c5cf33f8
FEATURE: Overhaul of admin API key system (#8284)
- Allow revoking keys without deleting them
- Auto-revoke keys after a period of no use (default 6 months)
- Allow multiple keys per user
- Allow attaching a description to each key, for easier auditing
- Log changes to keys in the staff action log
- Move all key management to one place, and improve the UI
2019-11-05 14:10:23 +00:00
Vinoth Kannan 671f303b53
FEATURE: Add welcome message for admins. (#8293) 2019-11-05 18:15:55 +05:30
Neil Lalonde b6d140e4bd UX: tag input suggests required tags if none have been selected
This is a follow-up to the new feature that allows a category to
require a certain number of tags from a tag group. The tag input will
shows results from the required group if none have been chosen yet.
Once a require tag is selected, the tag input will include other
results as usual. Staff users can ignore this restriction, so the input
behaviour is unchanged for them.
2019-11-04 16:51:54 -05:00
Penar Musaraj 74869b8a7f FIX: Do not consider mobile app traffic as crawler visits
Followup to a4eb523a
2019-11-04 09:16:50 -05:00
Joe ce0bac7a3d FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286)
* use image alt as a fallback when there's no title

* update spec

we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored.

The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 10:15:14 +11:00
Joffrey JAFFEUX 24d4241e35 FIX: prevents whitelisted_generic_onebox_spec to fail with zeitwerk (#8288) 2019-11-04 09:15:09 +11:00
Vinoth Kannan 2cb805a683 DEV: Add option to keep onebox body content in post excerpt. 2019-11-02 17:14:04 +05:30
Daniel Waterworth 200cef90ea FIX: TopicQuery doesn't react well to subcategories without definitions
Also:

Move includes call higher which makes it possible to run all of the
intermediate queries for easier debugging.

Add tests for TagsController with categories in the path.
2019-11-02 08:34:43 +00:00
Daniel Waterworth 2b8a013e32 DEV: Reduce duplication 2019-11-01 13:07:51 +00:00
Daniel Waterworth f9a05e4ae7 DEV: Switch flaky blame to restorer_spec.rb 2019-11-01 12:27:38 +00:00
Daniel Waterworth eae7c0e395 DEV: Mark flaky test 2019-11-01 11:35:57 +00:00
Daniel Waterworth b6d2b78a6b DEV: More prefabrication 2019-11-01 10:40:58 +00:00
David Taylor 4312bbe1e7
FIX: Do not load plugin CSS/JS assets when disabled (#8275)
Follow-up to 839916aa49 and 5bd6b70d98
2019-11-01 09:50:31 +00:00
Vinoth Kannan 72aa26c8c5
FEATURE: New site settings for default tags in user preferences. (#8283) 2019-11-01 12:40:13 +05:30
Sam Saffron 25a3bd333e DEV: correct fragile test
This test would fail sometimes if for some reason topic id 1 was in the db

No reason to pretend here, this only makes for a fragile test suite.
2019-11-01 15:40:00 +11:00
Martin Brennan f753643cb1
FIX: Unread topics not clearing when whisper is last post (#8271)
Meta thread: https://meta.discourse.org/t/cant-dismiss-unread-if-last-post-is-an-assign-or-whisper/131823/7

* when sending a whisper, the highest_staff_post_number is set
in the next_post_number method for a Topic, but the
highest_post_number is left alone. this leaves a situation
where highest_staff_post_number is > highest_post_number
* when TopicsBulkAction#dismiss_posts was run, it was only setting the topic_user
highest_seen_post_number using the highest_post_number from the topic, so if
the user was staff and the last post in a topic was a whisper
their highest seen number was not set, and the topic stayed unread

Found through testing that the bug wasn't to do with Assign/Unassign as they do not affect the post numbers, only whispering does.
2019-11-01 09:19:43 +10:00
Neil Lalonde d777844ed6 FEATURE: categories can require topics have a tag from a tag group
In a category's settings, the Tags tab has two new fields to
specify the number of tags that must be added to a topic
from a tag group. When creating a new topic, an error will be
shown to the user if the requirement isn't met.
2019-10-31 16:10:19 -04:00
Daniel Waterworth d84c34ad75 DEV: Server-side category routing changes
The routes for categories are changing. The scheme that I intend to move
us to is:

/c/*slug_path/(:id)/ENDPOINT
/c/*slug_path/(:id)

This commit adds support for the new scheme to the server side without
dropping support for existing URLs. It is necessary to support existing
URLs for two reasons:

 * This commit does not change any client side routing code,
 * Posts that contain category hashtags that refer to a root category
   are baked into URLs that do not fit this new scheme, (/c/[id]-[slug])
2019-10-31 18:06:22 +00:00
David Taylor 1f88ecf6d8
FIX: Cache `Discourse.system_user` separately for each multisite tenant (#8276)
This was not causing any known issue, because the system user ID is always the same across all sites. However, we should cache this on a per-site basis to be safe.
2019-10-31 15:16:26 +00:00
Penar Musaraj f8b72d9835 DEV: Refactor excluding audio/video URLs from search result blurbs
Followup to 580a4a82
2019-10-31 09:13:24 -04:00
Sam Saffron c5e67726fd FIX: under some conditions draft would say it was saving when not
This is a major change to draft internals. Previously there were quite a
few cases where the draft system would say "draft saved", when in fact
we just skipped saving.

This commit ensures the draft system deals with draft ownership handover in
a predictable way.

For example:

- Window 1 editing draft
- Window 2 editing same draft at the same time

Previously we would allow window 1 and 2 to just fight on the same draft
each window overwriting the same draft over an over.

This commit introduces an ownership concept where either window 1 or 2 win
and user is prompted on the loser window to reload screen to correct the issue

This also corrects edge cases where a user could have multiple browser windows
open and posts in 1 window, later to post in the second window. Previously
drafts would break in the second window, this corrects it.
2019-10-31 17:15:58 +11:00
Vinoth Kannan d355506123 FEATURE: Option to update user preferences of all users when default site setting changed. 2019-10-31 06:48:03 +05:30
Joffrey JAFFEUX 0ba28e344b FIX: tags can be filtered on categoryId without a q param (#8264) 2019-10-31 10:16:23 +10:00
Sam Saffron d8f7f363cd FEATURE: stop updating last_posted_at on users for messages and whispers
This ensures we only update last_posted_at which is user facing for non messages
and non whispers.

We still update this date for secure categories, we do not revert it for
deleted posts.
2019-10-31 09:01:26 +11:00
Daniel Waterworth 32107a9a72 FIX: Correct slug validation
We were allowing new categories to use slugs like "2342-category".
2019-10-30 17:13:28 +00:00
Penar Musaraj 580a4a827b Exclude audio/video URLs from search result blurbs
Displays translatable "[audio]" or "[video]" placeholders instead of ugly (and often long) URLs.
2019-10-30 13:07:16 -04:00
Régis Hanol 92213813d4 Revert "FIX: Require q param in /tags/filter/search route (#8263)"
This reverts commit 968a1419df.
2019-10-30 10:37:18 +01:00
Martin Brennan 968a1419df
FIX: Require q param in /tags/filter/search route (#8263)
* Require q param in /tags/filter/search route.
* If not provided this route was causing a 500 error when
  DiscourseTagging.clean_tag was called, because .downcase
  was being called on the param (which was nil).
* Now return a 400 error instead.
2019-10-30 16:32:48 +10:00
Arpit Jalan 2ae71b7a87 FEATURE: allow sending bulk invites to staged users 2019-10-30 11:40:03 +05:30
Krzysztof Kotlarek c32bd8ae48 FEATURE: Remove attachments and truncate raw field for incoming emails (#8253)
Adds the settings: 

raw_email_max_length, raw_rejected_email_max_length, delete_rejected_email_after_days. 

These settings control retention of the "raw" emails logs.

raw_email_max_length ensures that if we get incoming email that is huge we will truncate it removing uploads from the raw log.

raw_rejected_email_max_length introduces an even more aggressive truncation for rejected incoming mail. 

delete_rejected_email_after_days controls how many days we will keep rejected emails for (default 90)
2019-10-30 16:54:35 +11:00
Jeff Wong 74dc37c07c FIX: upserting custom fields using keywords converts the array key to a string 2019-10-29 11:35:38 -07:00
Daniel Waterworth 29e41dc511 Revert "DEV: Mark flaky tests as pending"
This reverts commit 0c47529713.
2019-10-29 13:32:42 +00:00
Krzysztof Kotlarek c5bcad10a6
FIX: more flaky jobs_base_spec.rb (#8259)
This commit is follow up of 417be323a2

We need more steps to properly switch from multi-site to single-site
2019-10-29 21:32:43 +11:00
Daniel Waterworth 0c47529713 DEV: Mark flaky tests as pending 2019-10-29 10:00:00 +00:00
Krzysztof Kotlarek 417be323a2 FIX: flaky jobs_base_spec.rb (#8258)
I was searching for a reason for randomly failing jobs_base_spec.rb. The reason was that after restorer_spec, the database is not restored to default.
After restorer spec RailsMultisite::ConnectionManagement.all_dbs is returning array of ['default', 'second']

Then base job execution is evaluated twice
```
dbs = RailsMultisite::ConnectionManagement.all_dbs
dbs.each do |db|
     execute(opts)
end
```
2019-10-29 18:25:00 +11:00
Mark VanLandingham 4eb54f08b2
FEATURE: Site setting/UI to allow users to set their primary group (#8244)
* FEATURE: Site setting/ui to allow users to set their primary group

* prettier and remove logic from account template

* added 1 to 43 to make web_hook_user_serializer_spec pass
2019-10-28 12:46:27 -05:00
Leo McArdle e7ff6809a3 FEATURE: add SES spam header to recognised spam headers (#8254) 2019-10-28 12:46:53 -04:00
David Taylor 6de49b88df FIX: Respond to user search correctly when category_id is blank
Previously it would search for category_id=0, which does not exist. With the new permission checks, this returns a 404
2019-10-28 13:18:47 +00:00
David Taylor f0617f96ed DEV: Update users controller spec following user_search update 2019-10-28 12:32:56 +00:00
David Taylor eec464d8d1 SECURITY: Check permissions when autocompleting mentions 2019-10-28 11:01:47 +00:00
Sam Saffron 3d85cc1e69 PERF: run expensive clean up uploads less frequently
Previously every hour we would run a full scan of the entire DB searching
for expired uploads that need to be moved to the tombstone folder.

This commit amends it so we only run the job 2 times per clean_orpha_uploads_grace_period_hours

There is a upper bound of 7 days so even if the grace period is set really
high it will still run at least once a week.

By default we have a 48 grace period so this amends it to run this cleanup
daily instead of hourly. This eliminates 23 times we run this ultra expensive
query.
2019-10-28 11:14:52 +11:00
Neil Lalonde 4c2d6e19ba PERF: cache new users counts in summary emails
The query to count how many new users there are since a given date
is expensive. It's the least personalized stat and the one we fallback
to last when no better number can be found for the target user.
Give up accuracy so we can aggressively cache the user counts
that appear in this email.
2019-10-25 16:33:36 -04:00
Roman Rizzi 070a3dcf9b
FIX: When running the wizard and using a custom theme, fallback to the color_scheme name if the base_scheme_id is nil (#8236) 2019-10-25 09:29:51 -03:00
David Taylor f5d6236ad2 DEV: Use `before_all` to run code before prefabrication
Followup to e9897d295f
2019-10-24 12:27:40 +01:00
David Taylor e9897d295f DEV: Refactor user search spec to use prefabrication
Break up single large example into multiple examples, using fab! to maintain performance. On my machine, this speeds up the test slightly, and also makes it more readable.
2019-10-24 11:58:33 +01:00
Krzysztof Kotlarek b8688c4af7
FIX: Rubocop rule on restorer spec (#8238) 2019-10-24 12:03:27 +11:00
Krzysztof Kotlarek f530378df3 FIX: Restore for non-multisite is not raising an error on reconnect step (#8237)
That commit introduced a bug to the system: f69dacf979

Restore works fine for multisite, however, stopped working for non-multisite.

Reason for that was that `establish_connection` method got a check if the multisite instance is available:
```
    def self.instance
      @instance
    end

    def self.establish_connection(opts)
      @instance.establish_connection(opts) if @instance
    end
```
However, the reload method don't have that check
```
    def self.reload
      @instance = new(instance.config_filename)
    end
```

To solve it, let's ensure we are in a multisite environment before call reload
2019-10-24 11:46:22 +11:00
Neil Lalonde cfe26eb301 FIX: backwards compatibility for uncompiled email style css 2019-10-23 19:22:33 -04:00
Neil Lalonde f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Vinoth Kannan 726bf37a12 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:50:10 +05:30
Vinoth Kannan 31577b2131 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:35:38 +05:30
Daniel Waterworth 1a72a61822 FIX: Fixed testsuite
The backup restorer tests weren't cleaning up after themselves and
playing poorly with prefabrication.
2019-10-23 09:43:05 +01:00
Krzysztof Kotlarek f69dacf979 FIX: Reconnect in restore process connects to correct DB (#8218)
Simplified flow of restore is like that
```
migrate_database
reconnect
extract_uploads
```

Problem with incorrect current database started with this fix https://github.com/discourse/discourse/commit/025d4ee91f4

Dump task is reconnecting to default database https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L429

And then, we are trying to reconnect to the original database with that code:
```
def reconnect_database
  log "Reconnecting to the database..."
  RailsMultisite::ConnectionManagement::establish_connection(db: @current_db)
end
```

This reconnect is not switching us back to correct database because of that check
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L181
Basically, it finds existing handler and it thinks that we are connected to correct DB and this step can be skipped.

To solve it, we can reload RailsMultisite::ConnectionManagement which creates a new instance of that class
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L38
2019-10-23 17:23:50 +11:00
Sam Saffron 950da34826 DEV: waiting for 10ms is hardly enough
tests that test thread behavior and rely on scheduling need to allow for
a wider amount of error margin
2019-10-23 16:18:41 +11:00
Arpit Jalan 1e9d9d9346
FIX: respect `tl3 links no follow` setting (#8232) 2019-10-22 22:41:04 +05:30
Blake Erickson 7d09af7eda
FIX: Prevent null-byte searches causing 500 error (#8226)
This fix ensures that searches that contain a null byte return a 400
error instead of causing a 500 error.

For some reason from rspec we will reach the raise statement inside
of the `rescue_from ArgumentError` block, but outside of rspec it will
not execute the raise statement and so a 500 is thrown instead of
reaching the `rescue_from Discourse::InvalidParameters` block inside of
the application controller.

This fix raises Discourse::InvalidParameters directly from the search
controller instead of relying on `PG::Connection.escape_string` to
raise the `ArgumentError`.
2019-10-22 08:44:52 -06:00
Blake Erickson 3201613f13 FEATURE: Include image url in topic serializer
The `image_url` is already included when fetching a list of topics, and
this commit adds it to the individual topic serializer so that it is
available via the api.

See

https://meta.discourse.org/t/single-topic-api-endpoint-should-contain-image-url/131020

for more details.
2019-10-21 22:02:49 -06:00
Blake Erickson ef0fe51e05
FIX: Include user id in notification webhook (#8195)
The payload when receiving a notification webhook is pointless without
knowing which user the notification is for. This fix adds the user_id to
the notification serializer so that when you receive a notification
webhook you can properly identify which user the notification is for.

See

https://meta.discourse.org/t/getting-the-target-user-for-notification-webhook-events/129052?u=blake

for more details.
2019-10-21 16:24:41 -06:00
Dan Ungureanu ceb74bef8f
DEV: Add test.
Follow-up to 2f54dd5e3d.
2019-10-21 21:33:58 +03:00
Michael Brown 40b0b549ff DEV: update the comments re the encoded slug changes
* followup to 2e73985d
2019-10-21 13:39:45 -04:00
Rafael dos Santos Silva 2e73985d1b FIX: Make category updates slug validation idempotent
Co-authored-by: Michael Brown <supermathie@gmail.com>
2019-10-21 14:33:19 -03:00
Dan Ungureanu 3ad07aacfa
FIX: Reload only notifications when refreshing notification count (#8221)
Previously, we used to reload the whole User instance which discarded
any changes made (for example setting 'unstage' to false).
2019-10-21 20:25:46 +03:00
Vinoth Kannan 5e55e75aed FIX: 'only_hidden_tags_changed?' method returned 'true' even when tags are not changed.
While editing the first post it does't bumped the topic when the new post revision created. Because we wrongly assumed that the hidden tags are changed even when no tags are updated.
2019-10-21 17:57:31 +05:30
Daniel Waterworth 55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Rafael dos Santos Silva 2304dcf993 FIX: Do not truncate encoded slugs
Trying to truncate encoded slugs will mean that we have to keep the URL
valid, which can be tricky as you have to be aware of multibyte
characters.

Since we already have upper bounds for the title, the slug won't grow
for more than title*6 in the worst case. The slug column in the topic
table can store that just fine.

Added a test to ensure that a generated slug is a valid URL too, so we
don't introduce regressions in the future.
2019-10-17 13:38:31 -03:00
Mark VanLandingham bd969332e0 FIX: Display site text overrides for non '_MF' keys (#8189)
FIX: Transform pluralized keys to `.other`, to check valid interpolation
2019-10-17 18:34:07 +02:00
David Taylor 11a9e0fc70 DEV: Add test for creating user with custom fields and user fields 2019-10-17 15:46:23 +01:00
Sam Saffron cb0443792b DEV: correct revision debouncer for backup draft
We were debouncing on update_at which we kept on pushing forward, this
was leading to not enough revisions.
2019-10-17 17:41:28 +11:00
Sam Saffron f5d1aff8dd FEATURE: experimental hidden setting for draft backups
Under exceptional situations the automatic draft feature can fail.

This new **hidden, default off** site setting
`backup_drafts_to_pm_length` will automatically backup any draft that is
saved by the system to a dedicated PM (originating from self)

The body of that PM will contain the text of the reply.

We can enable this feature strategically on sites exhibiting issues to
diagnose issues with the draft system and offer a recourse to users who
appear to lose drafts. We automatically checkpoint these drafts every 5
minutes forcing a new revision each 5 minutes so you can revert to old
content.

Longer term we are considering automatically enabling this kind of feature
for extremely long drafts where the risk is really high one could lose
days of writing.
2019-10-17 16:58:21 +11:00
Daniel Waterworth 7a0c06691c FIX: Account for nil when looking up subcategories 2019-10-16 20:04:54 +01:00
Neil Lalonde 5ef49692e0 FIX: tag cannot be used if it belongs to two tag groups with parent tag
If two tag groups exist with a mandatory parent tag, and one tag is
added to both tag groups, then the tag couldn't be used on any topics.
2019-10-16 14:28:04 -04:00
Neil Lalonde 61f6a6e836 Remove focus from specs 2019-10-16 14:28:04 -04:00
Sam Saffron d5d8db7fa8 FEATURE: improve honeypot and challenge logic
This feature amends it so instead of using one challenge and honeypot
statically per site we have a rotating honeypot and challenge value which
changes every hour.

This means you must grab a fresh copy of honeypot and challenge value once
an hour or account registration will be rejected.

We also now cycle the value of the challenge when after successful account
registration forcing an extra call to hp.json between account registrations

Client has been made aware of these changes.

Additionally this contains a JavaScript workaround for:
https://bugs.chromium.org/p/chromium/issues/detail?id=987293

This is client side code that is specific to Chrome user agent and swaps
a PASSWORD type honeypot with a TEXT type honeypot.
2019-10-16 16:53:44 +11:00
Krzysztof Kotlarek 0bf55fe2ff
Revert "FIX: public_file_server.enabled is false in test (#8192)" (#8196)
This reverts commit 5a8fdd02fe.
2019-10-16 10:39:31 +11:00
Krzysztof Kotlarek 5a8fdd02fe FIX: public_file_server.enabled is false in test (#8192)
After a small conversation, we decided that we can set `public_file_server.enabled` to false in the `test` environment to have the same value as `production`.
2019-10-16 09:18:48 +11:00
Daniel Waterworth 5f5b232cde FIX: Category.find_by_slug
find_by_slug should ensure that the parent actually exists when its
looking for a parent.
2019-10-15 16:44:24 +01:00
Vinoth Kannan b2f682f35e
FEATURE: Option to update category preferences of all users when site setting changed (#8180) 2019-10-15 18:41:27 +05:30
Daniel Waterworth 0de7e4339c FIX: Subcategory permissions validation
When a category has a subcategory, we ensure that no one who can see the
subcategory cannot see the parent. However, we don't take into account
the fact that, when no CategoryGroups exist, the default is that
everyone has full permissions.
2019-10-14 21:13:22 +01:00
Daniel Waterworth c49b20a1a2 FIX: Validation of category tree depth
This prevents the creation of sub-sub-categories in multiple tabs
2019-10-14 21:13:22 +01:00
Gerhard Schlager 2c011252f1 FIX: Move notification level only when user posted
Moving posts also moves the read state (`topic_users` table) to the destination topic. This changes that behavior so that only users who posted in the destination topic will have the original notification level (probably "watching") of the original topic. The notification level for all other users will be set to "regular".
2019-10-14 15:06:09 +02:00
Dan Ungureanu 96b8710b39
DEV: Fix heisentest (ensure that user ID really does not exist). 2019-10-14 12:25:43 +03:00
Vinoth Kannan 519fe290e3 SPEC: 'lookup_upload_urls' method should use cdn url if available.
e4fe864c0b
2019-10-14 12:57:33 +05:30
Vinoth Kannan 129e308dac fix the incorrect specs
Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 12:49:11 +05:30
Vinoth Kannan f92a6f7ac5 FIX: 'local_cdn_url' method should work for local relative urls too. 2019-10-14 11:39:16 +05:30
Krzysztof Kotlarek 99086edf85
FIX: Allow themes to upload and serve js files (#8188)
If you set `config.public_file_server.enabled = false` when you try to get uploaded js file you will get an error:
`Security warning: an embedded <script> tag on another site requested protected JavaScript. If you know what you're doing, go ahead and disable forgery protection on this action to permit cross-origin JavaScript embedding.`

The reason is that content type is `application/javascript` and in Rails 5 guard looked like that:
https://github.com/rails/rails/blob/5-2-stable/actionpack/lib/action_controller/metal/request_forgery_protection.rb#L278-L280
However, in Rails 6 `application` was added to regex:
https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/request_forgery_protection.rb#L282-L284

This pull request is related to https://meta.discourse.org/t/uploaded-js-file-for-theme-causes-a-rejection/129753/8
2019-10-14 15:40:33 +11:00
Roman Rizzi 01bc465db8
DEV: Split max decompressed setting for themes and backups (#8179) 2019-10-11 14:38:10 -03:00
Mark VanLandingham f63db1c4c8
FIX: Accurate sub_total calculation for reviewable_scores (#8184) 2019-10-11 11:07:19 -05:00
Rafael dos Santos Silva 76ab0350f1
FIX: Properly encoded slugs when configured to (#8158)
When an admin changes the site setting slug_generation_method to
encoded, we weren't really encoding the slug, but just allowing non-ascii
characters in the slug (unicode).

That brings problems when a user posts a link to topic without the slug, as
our topic controller tries to redirect the user to the correct URL that contains
the slug with unicode characters. Having unicode in the Location header in a
response is a RFC violation and some browsers end up in a redirection loop.

Bug report: https://meta.discourse.org/t/-/125371?u=falco

This commit also checks if a site uses encoded slugs and clear all saved slugs
in the db so they can be regenerated using an onceoff job.
2019-10-11 12:38:16 -03:00
David Taylor 67787799bb DEV: Allow plugins to mark user custom fields as editable only by staff
This adds a staff_only parameter to the register_editable_user_custom_field API. The default is false, to maintain backwards compatibility.
2019-10-11 09:57:55 +01:00
Nacho Caballero 7d2f5240d9 FIX: Show a correct diff when editing consecutive paragraphs (#8177) 2019-10-11 03:50:37 -04:00
Krzysztof Kotlarek 694a5bf229 FIX: Prevent from creation of duplicated TopicAllowedUsers (#8169)
Ensure that we don't try to create duplicated TopicAllowedUsers

Related to https://meta.discourse.org/t/error-message-topic-allowed-users-is-invalid/130382/5

Spec amended to cover a case creating an overlap.
2019-10-11 17:44:29 +11:00
Roman Rizzi 5357ab3324
SECURITY: Safely decompress backups when restoring. (#8166)
* SECURITY: Safely decompress backups when restoring.

* Fix tests and update theme_controller_spec to work with zip files instead of .tar.gz
2019-10-09 11:41:16 -03:00
Gerhard Schlager bee000bcec FIX: Existing post timings could prevent moving posts
Post timings are created by `topic_id` and `post_number` and it's possible that the destination topic already contains post timings for non-existent posts. For example, this can happen if the destination topic was previously split and Discourse recorded post timings for moved posts in the destination topic.

This commit ensures that all timings which reference non-existent posts are deleted from the destination topic before the posts are moved.
2019-10-08 21:07:29 +02:00
Roman Rizzi b805037825
FIX: Decrement posts read count when destroying post timings (#8172) 2019-10-08 15:39:23 -03:00
Régis Hanol 349c1cd085 FIX: remove site setting 'shadowed-by-global' option (#8061) 2019-10-08 12:43:26 -04:00
Mark VanLandingham 057f698e37 FIX: Store user_accuracy_bonus to clarify explanations (#8165)
* FIX: Store user_accuracy_bonus to clarify explanations

* Fixed specs from rounding change

* migration cleanup

* user_accuracy_bonus column not nullable
2019-10-08 09:49:07 -04:00
Dan Ungureanu fdb1d3404c
FEATURE: Add site setting to show more detailed 404 errors. (#8014)
If the setting is turned on, then the user will receive information
about the subject: if it was deleted or requires some special access to
a group (only if the group is public). Otherwise, the user will receive
a generic #404 error message. For now, this change affects only the
topics and categories controller.

This commit also tries to refactor some of the code related to error
handling. To make error pages more consistent (design-wise), the actual
error page will be rendered server-side.
2019-10-08 14:15:08 +03:00
David Taylor d2bceff133
FEATURE: Use full page redirection for all external auth methods (#8092)
Using popups is becoming increasingly rare. Full page redirects are already used on mobile, and for some providers. This commit removes all logic related to popup authentication, leaving only the full page redirect method.

For more info, see https://meta.discourse.org/t/do-we-need-popups-for-login/127988
2019-10-08 12:10:43 +01:00
Jarek Radosz 20514f2e44
DEV: Update markdown-it from 8.4.1 to 10.0.0 (#8164) 2019-10-08 13:00:22 +02:00
Joffrey JAFFEUX f6f0fcc8e3 DEV: attemps to male top-topic-spec more resilient (#8168) 2019-10-08 11:38:13 +11:00
Sam Saffron 78c5adc093 DEV: stop hardcoding ids in specs
hardcoded ids are unreliable for running specs and lead to erratic failures
2019-10-08 09:37:07 +11:00
Jarek Radosz bfe0bc0cbd
FIX: Check for category conflicts in SiteSetting validations (#8137)
It was possible to add a category to more than one default group, e.g. "default categories muted" and "default categories watching first post".

The bug was caused by category validations inadvertently comparing strings and numbers.
2019-10-06 20:50:07 +02:00
Mark VanLandingham 868303e5d9 FEATURE: Weighted reviewable user accuracy (#8156)
* FEATURE: Inaccurate users have negative review accuracy

* FIX: disallow negative reviewable score even if the accuracy would make it negative
2019-10-04 15:10:56 -04:00
Sam Saffron 926bd712bf DEV: improve flaky spec
This handles a lot of date calculations, keep the baseline time consistent
so it is not prone to issues when running out UTC threshold
2019-10-04 11:11:03 +10:00
Sam Saffron a49e484519 DEV: correct flaky test testing message content
This test was expecting user id ordering which is not really required.
Order of users does not matter
2019-10-04 09:56:28 +10:00
tshenry df02930c67
Correct filename typo 2019-10-03 15:08:56 -07:00
Osama Sayegh e27f332318
PERF: speed up about page render time and limit category mods (#8112)
* PERF: speed up about page render time and limit category mods

* Remove return

* Remove widgets

* Convert admins and mods lists

* Rename component

* Apply Joffrey's patch

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>

* Make limit 100
2019-10-03 21:48:56 +03:00
Roman Rizzi 10565e4623
SECURITY: Safely decompress files. (#8124)
* FEATURE: Adds an extra protection layer when decompressing files.

* Rename exporter/importer to zip importer. Update old locale

* Added a new composite class to decompress a file with multiple strategies

* Set max file size inside a site setting

* Ensure that file is deleted after compression

* Sanitize path and files before compressing/decompressing
2019-10-03 10:19:35 -03:00
Justin DiRose 63fabdb6f2 FIX: first post true if user creates topic first (#8139)
Reported here: https://meta.discourse.org/t/user-was-banned-for-posting-a-reply-within-3-seconds/128823/12
The problem here is the user could create a topic, then reply and get
silenced on the second time (though technically their first post) for being below the
min_first_post_typing_time threshold.
2019-10-02 15:51:40 -04:00
Sam Saffron 55ee9abecb DEV: clean up dependencies in spec
Follow up on zeitwork we needed to be a bit more explicit about a few
dependencies internally.

On certain orders the test suite could fail.
2019-10-02 14:50:54 +10:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00