Commit Graph

7235 Commits

Author SHA1 Message Date
David Taylor 4c9ca24ccf
FEATURE: Hash API keys in the database (#8438)
API keys are now only visible when first created. After that, only the first four characters are stored in the database for identification, along with an sha256 hash of the full key. This makes key usage easier to audit, and ensures attackers would not have access to the live site in the event of a database leak.

This makes the merge lower risk, because we have some time to revert if needed. Once the change is confirmed to be working, we will add a second commit to drop the `key` column.
2019-12-12 11:45:00 +00:00
Bianca Nenciu 3ec2081059
UX: Include public groups in mentionable groups set (#8516) 2019-12-12 13:13:40 +02:00
Martin Brennan 9f4c9bafa1
FIX: Hide old bookmark button on post-menu if SiteSetting.enable_bookmarks_with_reminders
Hide old bookmark post-menu item if the site setting for the new bookmark reminders is enabled and change icon for the new bookmark functionality to the same as the old bookmark button
Fix null @topic_view error in post serializer for post_bookmark, as new posts do not have a @topic_view
2019-12-12 16:28:01 +10:00
Sam Saffron 9d6a9bba96 DEV: correct flaky test
This is a slight workaround which helps somewhat now but is pending a larger
fix.

When this spec ran in parallel mode uploads could start cross talking and
an upload you expect to be there may vanish.

This works around the issue by making the upload unique every time it is
created

It also folds up an expensive test into the main one.
2019-12-12 16:54:22 +11:00
Martin Brennan edbc356593
FIX: Replace deprecated URI.encode, URI.escape, URI.unescape and URI.unencode (#8528)
The following methods have long been deprecated in ruby due to flaws in their implementation per http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/29293?29179-31097:

URI.escape
URI.unescape
URI.encode
URI.unencode
escape/encode are just aliases for one another. This PR uses the Addressable gem to replace these methods with its own encode, unencode, and encode_component methods where appropriate.

I have put all references to Addressable::URI here into the UrlHelper to keep them corralled in one place to make changes to this implementation easier.

Addressable is now also an explicit gem dependency.
2019-12-12 12:49:21 +10:00
Mark VanLandingham 09d9baa6d7
FIX: Update S3 stubs for more aws-sdk API changes (#8534) 2019-12-11 11:26:52 -08:00
dependabot-preview[bot] 863d3f89de Build(deps): Bump aws-sdk-s3 from 1.36.1 to 1.59.0 (#8502)
* Build(deps): Bump aws-sdk-s3 from 1.36.1 to 1.59.0

Bumps [aws-sdk-s3](https://github.com/aws/aws-sdk-ruby) from 1.36.1 to 1.59.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-s3/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/v1.36.1...v1.59.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Dependabot/bundler/aws sdk s3 1.59.0 (#8532)
2019-12-11 10:18:11 -08:00
Dan Ungureanu 4e130f1e03
SECURITY: Remove event handlers from SVG files 2019-12-11 17:04:50 +02:00
Dan Ungureanu adfa793731
SECURITY: Ensure only image uploads can be inlined
This prevents malicious files (for example special crafted XMLs) to be
used in XSS attacks.
2019-12-11 17:04:49 +02:00
dependabot-preview[bot] b90a592146 DEV: Bump aws-sdk-sns from 1.13.0 to 1.21.0 (#8490)
Bumps [aws-sdk-sns](https://github.com/aws/aws-sdk-ruby) from 1.13.0 to 1.21.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-sns/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/1.13.0...1.21.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 06:13:17 -08:00
Martin Brennan 6261339da9
Improving bookmarks part 1 (#8466)
Note: All of this functionality is hidden behind a hidden, default false, site setting called `enable_bookmarks_with_reminders`. Also, any feedback on Ember code would be greatly appreciated!

This is part 1 of the bookmark improvements. The next PR will address the backend logic to send reminder notifications for bookmarked posts to users. This PR adds the following functionality:

* We are adding a new `bookmarks` table and `Bookmark` model to make the bookmarks a first-class citizen and to allow attaching reminders to them.
* Posts now have a new button in their actions menu that has the icon of an actual book
* Clicking the button opens the new bookmark modal.
* Both name and the reminder type are optional.
* If you close the modal without doing anything, the bookmark is saved with no reminder.
* If you click the Cancel button, no bookmark is saved at all.
* All of the reminder type tiles are dynamic and the times they show will be based on your user timezone set in your profile (this should already be set for you).
* If for some reason a user does not have their timezone set they will not be able to set a reminder, but they will still be able to create a bookmark.
* A bookmark can be deleted by clicking on the book icon again which will be red if the post is bookmarked.

This PR does NOT do anything to migrate or change existing bookmarks in the form of `PostActions`, the two features live side-by-side here. Also this does nothing to the topic bookmarking.
2019-12-11 14:04:02 +10:00
Neil Lalonde 20464116ae FIX: tag input doesn't show all top 5 permitted tags 2019-12-10 10:30:01 -05:00
Sam Saffron 0c52537f10 DEV: update rubocop to version 0.77
We like to stay as close as possible to latest with rubocop cause the cops
get better.

This update required some code changes, specifically the default is to avoid
explicit returns where implicit is done

Also this renames a few rules
2019-12-10 11:48:39 +11:00
Krzysztof Kotlarek 81c7d6a462
FIX: Muted tags are respected by TopicTrackingState (#8467)
When the tag is muted and topic contains that tag, we should not mark that message as NEW.

There are 3 possible settings which site admin can set.
remove_muted_tags_from_latest - always
It means that if the topic got at least one muted tag, we should not mark that topic as NEW

remove_muted_tags_from_latest - only muted
Similar to above, however, if at least one tag is not muted, the topic is marked as NEW

remove_muted_tags_from_latest - never
Basically, mute tag setting is ignored and all topics are set as NEW
2019-12-10 09:50:05 +11:00
Mark VanLandingham 14cb386f1e
FEATURE: Featured topic for user profile & card (#8461) 2019-12-09 11:15:47 -08:00
Dan Ungureanu aa24be1a9a
DEV: Extend plugin API for uploads (#8440)
* DEV: Add API to alter uploads Markdown

* DEV: Extract data attributes from image / download Markdown

For example '[test|attachment|hello=world]' will generate an 'a' element
with a data attribute: 'data-hello=world'.

This commit also makes MarkdownIt to transform '|attachment' into
'class="attachment"'. This transformation used to be a part of the
process which resolves short URLs (i.e. upload://).

* DEV: Export imageNameFromFileName
2019-12-09 16:20:03 +02:00
Dan Ungureanu ebe6fa95be
FIX: Optimize images in Onebox (#8471)
This commit ensures that images in Onebox are being optimized, but not
converted to lightbox too.
2019-12-09 15:39:25 +02:00
Sam Saffron 494fe335d3 DEV: allow handling crawler reqs with no user agent
Followup to e440ec25 we treat no user agent as crawler reqs.
2019-12-09 18:40:10 +11:00
Sam Saffron e440ec2519 FIX: crawler requests not tracked for non UTF-8 user agents
Non UTF-8 user_agent requests were bypassing logging due to PG always
wanting UTF-8 strings.

This adds some conversion to ensure we are always dealing with UTF-8
2019-12-09 17:43:51 +11:00
Jarek Radosz 067c4a7c10 DEV: Fix CookedPostProcessor specs
Broken in #8480
2019-12-09 03:16:25 +01:00
Gerhard Schlager c88797bf0e FIX: Ignore DMARC for emails sent to mailing list mirror 2019-12-06 13:29:39 +01:00
Sam Saffron a06fccae1b DEV: update dependencies and add notes about exceptions
Previously it was unclear why certain gems are being held back cause Gemfile
had no comment explaining it.

I tried to add some explanation from memory and remove some exceptions that
seemed to be superfluous.

This upgrades shoulda to latest, it appears to work once a couple of assertions
are removed

Also update http accept language used to auto detect language from http header
this is tested

Zeitwerk small update seems fine
2019-12-06 13:00:28 +11:00
Vinoth Kannan f7084a4339 FEATURE: add site setting to remove `X-Frame-Options` header. 2019-12-06 03:15:09 +05:30
Vinoth Kannan 11c4d5eda7 Fix the build - take 3.
754e1238b0
2019-12-05 20:35:39 +05:30
Vinoth Kannan e6dfcda0bc DEV: handle all notification consolidations in new 'NotificationConsolidator' class.
481c8314f0
2019-12-05 14:36:06 +05:30
Jarek Radosz d07f039468 FIX: Secure Upload URLs in lightbox (#8451)
This fixes the following issues:

* The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image
* Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 09:13:09 +10:00
Neil Lalonde 875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
Vinoth Kannan 15c2755b7b Fix the `WebHookPostSerializer` spec.
400f79cffc
2019-12-04 23:30:04 +05:30
Robin Ward 888d56774a
DEV: HTML Builders should respect if a plugin is enabled or not (#8454)
Previously they would return the HTML regardless of whether the plugin
was enabled or not.
2019-12-04 12:26:23 -05:00
David Taylor be1510b09d FIX: Use filtered posts when determining the next page
This bug was causing some unusual behavior when the last post is filtered (e.g. from an ignored user). In some situations this would cause suggested topics to be omitted from the payload.

The next_page specs have been updated to remove most of the stubs
2019-12-04 12:52:24 +00:00
Krzysztof Kotlarek 46fc45de99
FIX: inbound link when the only slug available (#8457)
Problem mentioned in [meta](https://meta.discourse.org/t/inbound-links-dont-show-up-when-topic-id-is-not-included/100551)

When there is an internal link without ID, only slug, we should still try to create reflection link.
2019-12-04 17:13:20 +11:00
Gerhard Schlager 9ebb69e8eb FIX: Respect `enable_inline_emoji_translation` setting in titles 2019-12-03 20:39:11 +01:00
romanrizzi c86ca3609e FIX: Filter readers avatars correctly when the post is a whisper 2019-12-03 10:50:02 -03:00
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Arpit Jalan 016732cced FEATURE: improve suspect user discovery 2019-12-02 16:39:28 +05:30
Bianca Nenciu 57f108899a
FEATURE: Make site texts controller handle pages and locales (#8408)
Some endpoints are returning i18n keys instead of translated messages
and with these changes, the site_texts endpoint can help translating
those.

Pagination part is needed for better wildcard support. For example,
looking for 'js.notifications' would set 'has_more' to true, but return
only the first 50 messages with no way of fetching the remaining.
2019-11-30 15:16:36 +02:00
David Taylor 06c2e28bbb DEV: Remove two more references to instagram_user_infos
Follow-up to ad6f33e5b1
2019-11-29 17:12:06 +00:00
David Taylor a6aada16bd DEV: Refactor API key specs to avoid hard-coding keys
By hard-coding keys, we are not testing the API key system end to end. This change also makes the specs more resilient to upcoming API key changes
2019-11-29 15:16:22 +00:00
Robin Ward 7fee3c61de
Support for custom messages and redirects when creating posts (#8434)
* Support for custom messages and redirects when creating posts

When a post/topic is created Discourse serializes a `NewPostResult`
object. Normally this contains a status like `created_post` or
errors describing why the post could not be created.

There are times when a plugin might want to take the inputted post
and do something in the background. In this case, the plugin
can return a custom `message` and `route_to` attribute in the
`NewPostResult`.

If present, the message will be displayed in an alert, and when "Ok" is
clicked the user will be routed to the new URL.

* Destroy the draft in parallel
2019-11-29 09:30:54 -05:00
Dan Ungureanu 1e0c2235a3
FIX: Optimize quoted images (#8427)
Only images that were part of a lightbox used to be optimized. This
patch ensures that quoted images are also optimized.
2019-11-29 15:18:42 +02:00
Krzysztof Kotlarek b120728999
FEATURE: Ability to add components to all themes (#8404)
* FEATURE: Ability to add components to all themes

This is the first and functional step from that topic https://dev.discourse.org/t/adding-a-theme-component-is-too-much-work/15398/16

The idea here is that when a new component is added, the user can easily assign it to all themes (parents).

To achieve that, I needed to change a site-setting component to accept `setDefaultValues` action and `setDefaultValuesLabel` translated label.
Also, I needed to add `allowAny` option to disable that for theme selector.

I also refactored backend to accept both parent and child ids with one method to avoid duplication (Renamed `add_child_theme!` to more general `add_relative_theme!`)

* FIX: Improvement after code review

* FIX: Improvement after code review2

* FIX: use mapBy and filterBy directly
2019-11-28 16:19:01 +11:00
Vinoth Kannan 481c8314f0 FEATURE: consolidate group membership request notifications. 2019-11-28 04:02:35 +05:30
Martin Brennan 901054fd75
FIX: Cache failed onebox URL request server-side (#8421)
We already cache failed onebox URL requests client-side, we now want to cache this on the server-side for extra protection. failed onebox previews will be cached for 1 hour, and any more requests for that URL will fail with a 404 status. Forcing a rebake via the Rebake HTML action will delete the failed URL cache (like how the oneboxer preview cache is deleted).
2019-11-28 07:48:29 +10:00
Martin Brennan e7c7a05097
FIX: Mark secure media upload insecure automatically if used for theme component (#8413)
When uploading a file to a theme component, and that file is existing and has already been marked as secure, we now automatically mark the file as secure: false, change the ACL, and log the action as the user (also rebake the posts for the upload)
2019-11-28 07:32:17 +10:00
Gerhard Schlager d12f2580de FIX: Serve crawler view to Google PageSpeed 2019-11-27 22:15:34 +01:00
Roman Rizzi 1c9d18f094
FIX: Correctly ignore/approve replies when acting on a flagged post (#8425) 2019-11-27 17:19:44 -03:00
David Taylor a227083c1c
FEATURE: Limit the number of active sessions for a user (#8411)
If a user has more than 60 active sessions, the oldest sessions will be terminated automatically. This protects performance when logging in and when loading the list of recently used devices.
2019-11-27 12:39:31 +00:00
Sam Saffron 88ecb650a9 DEV: Implement a faster Discourse.cache
This is a bottom up rewrite of Discourse cache to support faster performance
and a limited surface area.

ActiveSupport::Cache::Store accepts many options we do not use, this partial
implementation only picks the bits out that we do use and want to support.

Additionally params are named which avoids typos such as "expires_at" vs "expires_in"

This also moves a few spots in Discourse to use Discourse.cache over setex
Performance of setex and Discourse.cache.write is similar.
2019-11-27 16:11:49 +11:00
Sam Saffron 0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
Leo McArdle 2714149fd2 FEATURE: hide posts from incoming email based on dmarc verdict (#8333) 2019-11-26 15:55:22 +01:00