Commit Graph

2122 Commits

Author SHA1 Message Date
Robin Ward 65ac80b014 FEATURE: Log Staff edits in Staff Action Logs
Why? Some edits by staff are not tracked. For example, during the grace
period, or via the flags/silence dialog.

If a staff member is editing someone else's post, it now goes into the
Staff Action Logs so it can be audited by other staff members.
2018-03-12 13:51:40 -04:00
Gerhard Schlager d243b82fb3 FIX: Calculation of text length for <details> in excerpt was wrong 2018-03-12 16:55:23 +01:00
Sam 758b9a7dda FEATURE: prototype of local theme directory watcher
(note this will be documented a bit late)
2018-03-12 18:36:06 +11:00
Arpit Jalan f862122978 FIX: do not log personal message view if there exists a similar log in previous hour 2018-03-11 09:23:32 +05:30
Sam 5b6e49ae1d FEATURE: split out max diff to 2 settings
We trust staff + tl2 and up to perform edits in grace period.
Allow them significantly more edit room in grace period prior to storing
a revision.

editing_grace_period_max_diff_high_trust applies to users with tl2 and up.

So

tl0 / 1 : we store an extra revision if more than 100 chars change
tl2 and up : we store an extra revision if more than 400 chars change

We may tweak these numbers as we go.
2018-03-09 11:58:50 +11:00
Arpit Jalan a8149f8969 FIX: user should not be able to invite to PM if trust level requirment not met
FIX: when personal messages are disabled let user invite to a public topic
2018-03-08 14:59:04 +05:30
Sam e162cd16b6 FEATURE: editing_grace_period_max_diff to force revisions in grace period
If a user performs a substantive edit of 20 chars or more during grace period
we will store a revision to track the change

This allows for better auditing of changes that happen during the grace period
2018-03-07 18:34:34 +11:00
Sam f0d5f83424 FEATURE: limit assets less that non asset paths
By default assets can be requested up to 200 times per 10 seconds
from the app, this includes CSS and avatars
2018-03-06 15:20:39 +11:00
Robin Ward 17a615165c FIX: Don't lock wiki posts when they're edited 2018-03-05 14:50:06 -05:00
Arpit Jalan 003b03d939 allow staff to delete user if posts are 5 or less irrespective of delete_user_max_post_age 2018-03-05 23:31:29 +05:30
Joffrey JAFFEUX ce1994beea FIX: do not treat :: as a valid emoji 2018-03-05 15:35:24 +01:00
OsamaSayegh 282f53f0cd FEATURE: Theme settings (2) (#5611)
Allows theme authors to specify custom theme settings for the theme. 

Centralizes the theme/site settings into a single construct
2018-03-04 19:04:23 -05:00
Robin Ward cd6c5fc5fb FIX: Disable "Make Personal Message" if they are disabled 2018-03-02 20:28:39 -05:00
Robin Ward 730201d423 New interface to upsert custom fields 2018-03-02 12:45:52 -05:00
Régis Hanol 6a78669ca3 FIX: 'reply by email addresses' site settings should allow email addresses without a 'reply_key' when 'find related post with key' is disabled 2018-03-02 17:53:18 +01:00
Sam d39d2b9352 FEATURE: whitelist data for themes 2018-03-02 14:52:09 +11:00
Guo Xiang Tan 939180efa8 FIX: Missing 2FA guards when sso is enabled or when local login is disabled. 2018-03-02 10:39:10 +08:00
Guo Xiang Tan 4f301905b6 Make rubocop happy. 2018-03-02 10:15:53 +08:00
Robin Ward b3883f5c32 FIX: Don't lock a post on edit unless the raw changes 2018-03-01 20:40:19 -05:00
Sam 75172024ca SECURITY: ensure users have permission when moving categories 2018-03-02 12:13:27 +11:00
Guo Xiang Tan 81ca3677f7 Add guard for `nil` in our `RateLimiter`. 2018-03-01 13:20:42 +08:00
Guo Xiang Tan 5d9f9c2614 FIX: `RateLimiter` max of zero or less should raise rate limit exceeded. 2018-03-01 13:14:46 +08:00
Guo Xiang Tan e7a7356986 Remove ancient votes code that is no longer used. 2018-02-28 14:37:22 +08:00
Guo Xiang Tan 902c5d11cf FIX: Don't allow other flag actions after `notify_moderator` has happened.
https://meta.discourse.org/t/receiving-sorry-an-error-has-occurred-during-flagging-step-of-discobot-tutorial/77233/5
2018-02-28 11:27:56 +08:00
Sam f295a18e94 FIX: stop double counting net calls in logs 2018-02-28 10:45:11 +11:00
Régis Hanol fd33090646 FEATURE: automatically elides gmail quotes 2018-02-26 23:54:02 +01:00
Régis Hanol 26d5ae61dd FIX: handle <pre> inside <blockquote> in html_to_markdown 2018-02-26 23:28:02 +01:00
Neil Lalonde 3313072957 Remove censored_pattern site setting, which is replaced by watched words 2018-02-26 16:29:27 -05:00
Régis Hanol 3be0294465 FIX: local post onebox was always pointing to 1st post 2018-02-26 16:05:35 +01:00
Régis Hanol 7d7f6faf40 FIX: properly render emojis in local oneboxes 2018-02-26 11:16:53 +01:00
Arpit Jalan b9a669ba32 FIX: do not log personal message view if user can't see the message 2018-02-25 22:39:25 +05:30
Régis Hanol 0559a4736a FIX: don't double request when downloading a file 2018-02-24 12:35:57 +01:00
Robin Ward 69af881f7f New site setting `trusted_users_can_edit_others`
The default is true to keep with previous discourse behavior. If
disabled, high trust level users cannot edit the topics or posts of
other users.
2018-02-22 20:39:24 -05:00
Guo Xiang Tan dd26bbe868
Merge pull request #5610 from discourse/pm-tags
FEATURE: Allow staffs to tag PMs
2018-02-23 07:07:41 +08:00
Vinoth Kannan 7cbda949f1 REFACTOR: New spec tests and code improvement 2018-02-22 20:27:02 +05:30
Régis Hanol 7a13e50aa6 fix build 2018-02-22 11:17:49 +01:00
Gerhard Schlager 97e19a7d02 Fix the build 2018-02-21 11:26:41 +01:00
Guo Xiang Tan 8964e75ad6
Merge pull request #5612 from discourse/featheredtoast-two-factor-login
Featheredtoast two factor login
2018-02-21 15:00:10 +08:00
Guo Xiang Tan 14f3594f9f Review Changes for f4f8a293e7. 2018-02-21 14:55:49 +08:00
Arpit Jalan 94fb8094c6 further optimize spec
thanks @tgxworld for the review.
2018-02-21 11:32:40 +05:30
Régis Hanol 0799831dbe FIX: use the avatar of the post rather than the topic in local oneboxes 2018-02-20 19:49:39 +01:00
Arpit Jalan a4bc54a686 FIX: strip zero width spaces from topic title 2018-02-21 00:12:39 +05:30
Arpit Jalan ed422285f0 optimize spec 2018-02-20 22:03:13 +05:30
Sam 86d12bd44b FEATURE: search within title using in:title
Also

- Significantly improved search ranking, title is treated most strongly
- Adds tag names to the index
- Run search re-indexer more aggressively
- Re-index topic and all posts on category change
2018-02-20 14:41:21 +11:00
Régis Hanol 60ec483caa FIX: include title in local onebox when linking to a different topic 2018-02-19 22:40:14 +01:00
Arpit Jalan c419c26f56 FEATURE: new site setting 'max_emojis_in_title' 2018-02-19 18:15:26 +05:30
Gerhard Schlager b6277e208b FIX: Cookies header didn't have the right format 2018-02-19 12:46:57 +01:00
Régis Hanol 61930e092a FIX: support incoming emails with just an attachment 2018-02-16 18:14:56 +01:00
Sam 94b2c70c0d PERF: remove oga gem
oga gem is automatically required by the aws gem
the oga gem retains about 1mb of memory, aws now uses nokogiri

This also removes the html normalize from the pretty text specs that was
a fair bit buggy as the polls test shows.
2018-02-15 14:36:40 +11:00
Robin Ward b4aa0b096e FIX: Couldn't like staff when `allow_flagging_staff` was set 2018-02-14 15:46:04 -05:00
Sam f028ffaf29 SECURITY: correct local onebox category checks
Also removes ugly "source_topic_id" from cooked posts

Patch was authored by @zogstrip

Signed-off-by: Sam <sam.saffron@gmail.com>
2018-02-14 10:40:46 +11:00
Erick Guan 03b3e57a44 FEATURE: login by a link from email
Co-authored-by: tgxworld <tgx@discourse.org>
2018-02-13 16:14:39 +08:00
Robin Ward 4dfe659189 Rename `allow staff flags` to `allow flagging staff` 2018-02-12 15:27:26 -05:00
Robin Ward 6287631745 FEATURE: New site setting, `allow staff flags`, false by default
For some large communities, it makes sense to disable flagging of
staff posts.
2018-02-12 14:56:21 -05:00
Robin Ward dedeb2deb8 FIX: Don't show the link button in the composer if linking is disabled 2018-02-08 12:56:10 -05:00
Robin Ward 1bab15c757 FEATURE: A site setting for a minimum TL to post links 2018-02-06 18:07:58 -05:00
Robin Ward b2b6dc68a6 FEATURE: a setting to customize the minimum TL to flag a post 2018-02-06 17:12:27 -05:00
Sam Saffron df8e43abdd use lazy & instead of try
unregister ip skipper in test
raise if called when a skipper is in play
2018-02-06 10:38:15 +11:00
Robin Ward eefd226611 Add extensibility point to `request_tracker` to skip IP addresses
This is useful if you want to run a per IP rate limiter but want to be
able to skip some IPs with custom logic.
2018-02-05 17:49:40 -05:00
Arpit Jalan 7e48c47d37 rename 'enable_private_email_messages' to 'enable_personal_email_messages' 2018-02-01 13:25:29 +05:30
Arpit Jalan ff0376a80b rename 'enable_private_messages' to 'enable_personal_messages' 2018-02-01 13:25:29 +05:30
Arpit Jalan 25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
Sam ee0d3f15c1 FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify
New site settings:

enable_markdown_linkify: which is default on, auto links https:// and http:// and mail://

markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-02-01 13:22:38 +11:00
Régis Hanol 7d2283167a UX: only crops images taller than 18:9 instead of 16:9 2018-01-31 22:31:16 +01:00
Régis Hanol b2f18fc98f FIX: system user edits should not generate notifications 2018-01-30 22:21:07 +01:00
Maja Komel 018cb7f36b add a custom user onebox (#5542)
* add custom user onebox

* add specs
2018-01-30 11:03:08 +01:00
Maja Komel 330912e1e5 FIX: allowed href scheme link can start with a + (#5537)
* allowed href scheme link can start with a +

* allow tel:// links only to start with +

* add missing semicolon

* add test
2018-01-30 11:02:23 +11:00
Sam f3502853fa correct spec regression 2018-01-30 08:54:37 +11:00
Sam f946db4afe FIX: inline oneboxer min title length of 2
also: cache mini onebox misses as well to cut down traffic
2018-01-30 08:40:04 +11:00
Arpit Jalan 8ab585e25f add more tests for 'log private message views' feature 2018-01-29 13:11:20 +05:30
Sam fa5880e04f PERF: ability to crawl for titles without extra HEAD req
Also, introduces a much more aggressive timeout for title crawling
and introduces gzip to body that is crawled
2018-01-29 15:40:12 +11:00
Arpit Jalan 1f6adbea5c FEATURE: log private message views 2018-01-29 08:08:08 +05:30
Robin Ward 44e2038b53 Setting to automatically lock posts when edited by staff 2018-01-26 14:01:30 -05:00
Robin Ward 6b04967e2f FEATURE: Staff members can lock posts
Locking a post prevents it from being edited. This is useful if the user
has posted something which has been edited out, and the staff members don't
want them to be able to edit it back in again.
2018-01-26 14:01:30 -05:00
Arpit Jalan 7b4e6d508b improve reviving_old_topic education message 2018-01-26 00:06:53 +05:30
Gerhard Schlager eb52c5469e FEATURE: Allow plugins to register a new locale 2018-01-25 14:57:41 +01:00
Gerhard Schlager ce060e2b86 FIX: Server didn't use default_locale as fallback locale 2018-01-25 14:57:41 +01:00
Sam adae963751 ensure we do not override charset for content type 2018-01-25 18:43:42 +11:00
Sam 3492a91056 FEATURE: allow site operators to disable emoji shortcuts 2018-01-24 12:21:44 +11:00
Robin Ward 782d75069e FIX: UX improvements for system messages when PMs are disabled 2018-01-23 13:12:11 -05:00
Robin Ward 17ebfd1715 FIX: Don't show suggested messages if private messages are disabled 2018-01-23 12:05:44 -05:00
Régis Hanol cbb321658f FIX: support for generating excerpt when nesting <details> blocks 2018-01-22 19:17:35 +01:00
Sam f26ff290c3 FEATURE: Shorten setting name to max_reqs
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
Sam fc36f095a7 FIX: ensure proper header transfer (except for cache control)
allows discourse special headers to be visible on hijacked reqs
2018-01-21 14:26:42 +11:00
Sam 12872d03be PERF: run post timings in background
This means that if a very large amount of registered users hit
a single topic we will handle it gracefully, even if db gets slow.
2018-01-19 08:27:29 +11:00
Gerhard Schlager 2a22b90538 SECURITY: email domain whitelist could be bypassed 2018-01-17 21:45:32 +01:00
Robin Ward 34ed6088b9 FEATURE: New modal to show flags received for a user 2018-01-17 15:08:08 -05:00
Arpit Jalan e04fb9a877 fix the build 2018-01-17 12:57:33 +05:30
Arpit Jalan 79eb9d7086 FEATURE: show header search results on search log term details page 2018-01-17 12:47:16 +05:30
Arpit Jalan 1208254961 FIX: validate presence of 'top menu' setting 2018-01-17 01:43:53 +05:30
Sam d7657d8e47 correct specs, ensure crawler layout only applies to html 2018-01-16 16:28:11 +11:00
Sam 7b562d2f46 FEATURE: much improved and simplified crawler detection
- phase one does it match 'trident|webkit|gecko|chrome|safari|msie|opera'
    yes- well it is possibly a browser

- phase two does it match 'rss|bot|spider|crawler|facebook|archive|wayback|ping|monitor'
    probably a crawler then

Based off: https://gist.github.com/SamSaffron/6cfad7ea3e6df321ffb7a84f93720a53
2018-01-16 15:41:45 +11:00
Sam 215c0d5569 FEATURE: allow system api to target users via external id or user id
usage ?api_key=XYZ&api_user_external_id=ABC
usage ?api_key=XYZ&api_user_id=123
2018-01-12 17:40:18 +11:00
Vinoth Kannan 988b13ac77 FIX: GitHub auth always asking to verify email for new users (#5487) 2018-01-12 15:17:29 +11:00
Gerhard Schlager 9f7ae908d8 Add specs to check email domain whitelist/blacklist for To and Cc 2018-01-10 16:57:26 +01:00
Sam cecd7d0d07 FEATURE: global rate limiter can bypass local IPs 2018-01-08 08:39:17 +11:00
Gerhard Schlager f086d28b30 FIX: Do not validate messages sent to mailing list mirror 2018-01-05 11:21:53 +01:00
Gerhard Schlager e0d73a957d FEATURE: Allow posting via email to read-only mailing list mirror category 2018-01-05 11:21:53 +01:00
Gerhard Schlager d7cd7e4dc7 FIX: Never mark emails sent to mailing list mirror as auto-generated 2018-01-05 11:21:53 +01:00
Gerhard Schlager ceb7590bcb FIX: bounced email can contain multiple status codes 2018-01-03 17:59:20 +01:00
Guo Xiang Tan 805d1c25d3
Merge pull request #5451 from tgxworld/treat_non_ascii_urls_as_valid
Treat non-ascii URLs in `UrlValidator`.
2017-12-27 14:14:20 +08:00
Sam a9e2fc59c4 FIX: [constructor] bbcode would cause markdown crash 2017-12-27 16:11:30 +11:00
Arpit Jalan ef4c6c67ba fix the build 2017-12-23 14:42:40 +05:30
Arpit Jalan 0514ac4ee2 FIX: verify presence of 'sso url' before enabling 'enable sso' 2017-12-23 13:30:49 +05:30
Régis Hanol d6b22e6cc1 FIX: whitelist oneboxed iframes 2017-12-23 01:56:33 +01:00
Guo Xiang Tan 4b51871f6a Treat non-ascii URLs in `UrlValidator`. 2017-12-21 14:22:55 +08:00
Guo Xiang Tan 6ecf37c482 Improve URL validation to check for a valid host.
Parsing a URL with `URI` is not sufficient as the following cases
are considered valid:

URI.parse("http://https://google.com")
=> #<URI::HTTP http://https//google.com>
2017-12-21 13:50:15 +08:00
Robin Ward 21e1b05c7e FIX: Don't disable details when below truncate limit 2017-12-20 15:45:00 -05:00
Robin Ward a0aca83c12 FIX: Broken spec 2017-12-19 17:55:41 -05:00
Robin Ward b3fda0ea86 FIX: details tags broke excerpts 2017-12-19 17:28:55 -05:00
Sam 57a1190b07 FIX: correct issue with search omitting words with multiple dots
Previously we used to break up words with dots incorrectly leading to
missing search terms
2017-12-19 16:04:24 +11:00
Sam 81b3a4a3da improve spec 2017-12-15 11:42:51 +11:00
Guo Xiang Tan f2565f6c7e SECURITY: Any group can be invited into a PM. 2017-12-14 14:57:48 +08:00
Sam 67aecff59c FEATURE: store twitter supplied email for auditing 2017-12-14 15:54:32 +11:00
Gerhard Schlager e30851e45a Move escape_uri method to a more suitable place 2017-12-12 20:17:46 +01:00
Guo Xiang Tan 6ade508f39 FIX: Prevent 'rack.input' missing error. 2017-12-12 16:40:35 +08:00
Arpit Jalan ff6dda85b7 FIX: replace curly quotes to regular quotes in search terms 2017-12-12 11:17:28 +05:30
Sam 4986ebcf24 FEATURE: optional default off global per ip rate limiter 2017-12-11 17:52:57 +11:00
Sam 68d3c2c74f FEATURE: add global rate limiter for admin api 60 per minute
Also move configuration of admin and user api rate limiting into global
settings. This is not intended to be configurable per site
2017-12-11 11:07:22 +11:00
Sam 90a55d6f7c FIX: handle CORS in hijacked requests 2017-12-07 10:31:04 +11:00
Gerhard Schlager 16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
Kyle Zhao 5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Sam 995bf3c84e correct spec on Ruby 2.3 2017-12-05 07:04:41 +11:00
Sam 5a9622163d FIX: regression around rate limiter 2017-12-04 21:44:16 +11:00
Sam dd70ef3abf Revert "Revert "PERF: improve speed of rate limiter""
This reverts commit 2373d85239.
2017-12-04 21:23:11 +11:00
Sam 2373d85239 Revert "PERF: improve speed of rate limiter"
This reverts commit a9bcdd7f27.
2017-12-04 21:19:28 +11:00
Sam d041377ccf correct test that does not work with discobot 2017-12-04 18:20:05 +11:00
Sam a9bcdd7f27 PERF: improve speed of rate limiter
Also

- adds a global rate limiter option
- cleans up usage in tests
- fixes freeze_time so it handles clock_gettime
2017-12-04 18:17:30 +11:00
Guo Xiang Tan b18cc81609 Make rubocop happy. 2017-12-04 10:55:31 +08:00
Guo Xiang Tan 22140efa70 Tests are still leaking connection after skipping.
* Could be in the setup.
2017-12-04 10:46:30 +08:00
Guo Xiang Tan 4c8402c50f Skip test that is leaking connections. 2017-12-04 09:26:51 +08:00
Vinoth Kannan 7f2eeaf767 FIX: Password required flag should be cleared whenever clearing the raw password (#5384) 2017-12-01 15:19:24 +11:00
Guo Xiang Tan b1375ef44e Ensure that we disconnect connection in test. 2017-11-29 20:57:13 +08:00
Gerhard Schlager 44ee388070 FEATURE: omit images from og and twitter description tags 2017-11-28 21:34:02 +01:00
Jeff Wong b094894c94 Feature: Add service worker registration method to plugin API 2017-11-28 14:01:41 +08:00
Sam df84e1c358 Correctly track hijacked requests 2017-11-28 16:47:20 +11:00
Sam 0caa335ef0 FIX: Handle more cases where HTTP status is not correct
HTTP status was not correct with send_file which uses streaming
2017-11-28 11:00:13 +11:00
Sam ca7af7b88f FIX: displaying wrong avatar and letter avatar
correct regression where params and env is reused in production
2017-11-28 09:28:40 +11:00
Sam 608207b2e5 FEATURE: avatar proxy happens in background
This ensures that even if it is slow to download avatars site will
continue to work

Also simplifies hijack pattern
2017-11-27 17:43:24 +11:00
Guo Xiang Tan 2e04ef97d9 Fix the build. 2017-11-27 10:53:05 +08:00
Guo Xiang Tan a7030e80bf Skip randomly failing test. 2017-11-27 10:51:18 +08:00
Guo Xiang Tan 71942e4f62
Merge pull request #5352 from tgxworld/method_for_replica_postgres_connection
Expose `replica_postgresql_connection` to `ActiveRecord::Base`.
2017-11-27 08:38:10 +08:00
Sam e0e99d4bbd PERF: hijack onebox requests so they do not use up a unicorn worker 2017-11-24 15:31:40 +11:00
Guo Xiang Tan a509f466a0 Expose `replica_postgresql_connection` to `ActiveRecord::Base`. 2017-11-24 09:35:45 +08:00
Gerhard Schlager 613f4d737a FIX: updating topic stats failed silently for invalid topics 2017-11-23 18:47:45 +01:00
Sam 2b8d4508e5 PERF: stop running background work between requests
Use a dedicated thread to run Scheduler::Defer

This avoids blocking of a worker during operations that require waiting.

In particular uploads risked blocking a unicorn.

This also add a queue "length" that discourse prometheus consumes.
2017-11-23 15:48:47 +11:00
Guo Xiang Tan f7642e076d REFACTOR: Avoid duplicated logic on server and client. 2017-11-21 17:10:04 +08:00
Guo Xiang Tan 385372e384 Revert "PERF: Reduce number of topics to filter while querying for unread."
This reverts commit c06b782cab.
2017-11-20 11:49:09 +08:00
Gerhard Schlager 546b206da0 replace indentation with non-breaking spaces when cooking emails 2017-11-18 17:16:44 +01:00
Neil Lalonde 87ec11e298 FIX: more accurate counting of posts read. Skipping to the end of a topic does not count all posts as read in user stats. 2017-11-17 16:08:46 -05:00