Commit Graph

2055 Commits

Author SHA1 Message Date
Sam f3502853fa correct spec regression 2018-01-30 08:54:37 +11:00
Sam f946db4afe FIX: inline oneboxer min title length of 2
also: cache mini onebox misses as well to cut down traffic
2018-01-30 08:40:04 +11:00
Arpit Jalan 8ab585e25f add more tests for 'log private message views' feature 2018-01-29 13:11:20 +05:30
Sam fa5880e04f PERF: ability to crawl for titles without extra HEAD req
Also, introduces a much more aggressive timeout for title crawling
and introduces gzip to body that is crawled
2018-01-29 15:40:12 +11:00
Arpit Jalan 1f6adbea5c FEATURE: log private message views 2018-01-29 08:08:08 +05:30
Robin Ward 44e2038b53 Setting to automatically lock posts when edited by staff 2018-01-26 14:01:30 -05:00
Robin Ward 6b04967e2f FEATURE: Staff members can lock posts
Locking a post prevents it from being edited. This is useful if the user
has posted something which has been edited out, and the staff members don't
want them to be able to edit it back in again.
2018-01-26 14:01:30 -05:00
Arpit Jalan 7b4e6d508b improve reviving_old_topic education message 2018-01-26 00:06:53 +05:30
Gerhard Schlager eb52c5469e FEATURE: Allow plugins to register a new locale 2018-01-25 14:57:41 +01:00
Gerhard Schlager ce060e2b86 FIX: Server didn't use default_locale as fallback locale 2018-01-25 14:57:41 +01:00
Sam adae963751 ensure we do not override charset for content type 2018-01-25 18:43:42 +11:00
Sam 3492a91056 FEATURE: allow site operators to disable emoji shortcuts 2018-01-24 12:21:44 +11:00
Robin Ward 782d75069e FIX: UX improvements for system messages when PMs are disabled 2018-01-23 13:12:11 -05:00
Robin Ward 17ebfd1715 FIX: Don't show suggested messages if private messages are disabled 2018-01-23 12:05:44 -05:00
Régis Hanol cbb321658f FIX: support for generating excerpt when nesting <details> blocks 2018-01-22 19:17:35 +01:00
Sam f26ff290c3 FEATURE: Shorten setting name to max_reqs
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
Sam fc36f095a7 FIX: ensure proper header transfer (except for cache control)
allows discourse special headers to be visible on hijacked reqs
2018-01-21 14:26:42 +11:00
Sam 12872d03be PERF: run post timings in background
This means that if a very large amount of registered users hit
a single topic we will handle it gracefully, even if db gets slow.
2018-01-19 08:27:29 +11:00
Gerhard Schlager 2a22b90538 SECURITY: email domain whitelist could be bypassed 2018-01-17 21:45:32 +01:00
Robin Ward 34ed6088b9 FEATURE: New modal to show flags received for a user 2018-01-17 15:08:08 -05:00
Arpit Jalan e04fb9a877 fix the build 2018-01-17 12:57:33 +05:30
Arpit Jalan 79eb9d7086 FEATURE: show header search results on search log term details page 2018-01-17 12:47:16 +05:30
Arpit Jalan 1208254961 FIX: validate presence of 'top menu' setting 2018-01-17 01:43:53 +05:30
Sam d7657d8e47 correct specs, ensure crawler layout only applies to html 2018-01-16 16:28:11 +11:00
Sam 7b562d2f46 FEATURE: much improved and simplified crawler detection
- phase one does it match 'trident|webkit|gecko|chrome|safari|msie|opera'
    yes- well it is possibly a browser

- phase two does it match 'rss|bot|spider|crawler|facebook|archive|wayback|ping|monitor'
    probably a crawler then

Based off: https://gist.github.com/SamSaffron/6cfad7ea3e6df321ffb7a84f93720a53
2018-01-16 15:41:45 +11:00
Sam 215c0d5569 FEATURE: allow system api to target users via external id or user id
usage ?api_key=XYZ&api_user_external_id=ABC
usage ?api_key=XYZ&api_user_id=123
2018-01-12 17:40:18 +11:00
Vinoth Kannan 988b13ac77 FIX: GitHub auth always asking to verify email for new users (#5487) 2018-01-12 15:17:29 +11:00
Gerhard Schlager 9f7ae908d8 Add specs to check email domain whitelist/blacklist for To and Cc 2018-01-10 16:57:26 +01:00
Sam cecd7d0d07 FEATURE: global rate limiter can bypass local IPs 2018-01-08 08:39:17 +11:00
Gerhard Schlager f086d28b30 FIX: Do not validate messages sent to mailing list mirror 2018-01-05 11:21:53 +01:00
Gerhard Schlager e0d73a957d FEATURE: Allow posting via email to read-only mailing list mirror category 2018-01-05 11:21:53 +01:00
Gerhard Schlager d7cd7e4dc7 FIX: Never mark emails sent to mailing list mirror as auto-generated 2018-01-05 11:21:53 +01:00
Gerhard Schlager ceb7590bcb FIX: bounced email can contain multiple status codes 2018-01-03 17:59:20 +01:00
Guo Xiang Tan 805d1c25d3
Merge pull request #5451 from tgxworld/treat_non_ascii_urls_as_valid
Treat non-ascii URLs in `UrlValidator`.
2017-12-27 14:14:20 +08:00
Sam a9e2fc59c4 FIX: [constructor] bbcode would cause markdown crash 2017-12-27 16:11:30 +11:00
Arpit Jalan ef4c6c67ba fix the build 2017-12-23 14:42:40 +05:30
Arpit Jalan 0514ac4ee2 FIX: verify presence of 'sso url' before enabling 'enable sso' 2017-12-23 13:30:49 +05:30
Régis Hanol d6b22e6cc1 FIX: whitelist oneboxed iframes 2017-12-23 01:56:33 +01:00
Guo Xiang Tan 4b51871f6a Treat non-ascii URLs in `UrlValidator`. 2017-12-21 14:22:55 +08:00
Guo Xiang Tan 6ecf37c482 Improve URL validation to check for a valid host.
Parsing a URL with `URI` is not sufficient as the following cases
are considered valid:

URI.parse("http://https://google.com")
=> #<URI::HTTP http://https//google.com>
2017-12-21 13:50:15 +08:00
Robin Ward 21e1b05c7e FIX: Don't disable details when below truncate limit 2017-12-20 15:45:00 -05:00
Robin Ward a0aca83c12 FIX: Broken spec 2017-12-19 17:55:41 -05:00
Robin Ward b3fda0ea86 FIX: details tags broke excerpts 2017-12-19 17:28:55 -05:00
Sam 57a1190b07 FIX: correct issue with search omitting words with multiple dots
Previously we used to break up words with dots incorrectly leading to
missing search terms
2017-12-19 16:04:24 +11:00
Sam 81b3a4a3da improve spec 2017-12-15 11:42:51 +11:00
Guo Xiang Tan f2565f6c7e SECURITY: Any group can be invited into a PM. 2017-12-14 14:57:48 +08:00
Sam 67aecff59c FEATURE: store twitter supplied email for auditing 2017-12-14 15:54:32 +11:00
Gerhard Schlager e30851e45a Move escape_uri method to a more suitable place 2017-12-12 20:17:46 +01:00
Guo Xiang Tan 6ade508f39 FIX: Prevent 'rack.input' missing error. 2017-12-12 16:40:35 +08:00
Arpit Jalan ff6dda85b7 FIX: replace curly quotes to regular quotes in search terms 2017-12-12 11:17:28 +05:30
Sam 4986ebcf24 FEATURE: optional default off global per ip rate limiter 2017-12-11 17:52:57 +11:00
Sam 68d3c2c74f FEATURE: add global rate limiter for admin api 60 per minute
Also move configuration of admin and user api rate limiting into global
settings. This is not intended to be configurable per site
2017-12-11 11:07:22 +11:00
Sam 90a55d6f7c FIX: handle CORS in hijacked requests 2017-12-07 10:31:04 +11:00
Gerhard Schlager 16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
Kyle Zhao 5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Sam 995bf3c84e correct spec on Ruby 2.3 2017-12-05 07:04:41 +11:00
Sam 5a9622163d FIX: regression around rate limiter 2017-12-04 21:44:16 +11:00
Sam dd70ef3abf Revert "Revert "PERF: improve speed of rate limiter""
This reverts commit 2373d85239.
2017-12-04 21:23:11 +11:00
Sam 2373d85239 Revert "PERF: improve speed of rate limiter"
This reverts commit a9bcdd7f27.
2017-12-04 21:19:28 +11:00
Sam d041377ccf correct test that does not work with discobot 2017-12-04 18:20:05 +11:00
Sam a9bcdd7f27 PERF: improve speed of rate limiter
Also

- adds a global rate limiter option
- cleans up usage in tests
- fixes freeze_time so it handles clock_gettime
2017-12-04 18:17:30 +11:00
Guo Xiang Tan b18cc81609 Make rubocop happy. 2017-12-04 10:55:31 +08:00
Guo Xiang Tan 22140efa70 Tests are still leaking connection after skipping.
* Could be in the setup.
2017-12-04 10:46:30 +08:00
Guo Xiang Tan 4c8402c50f Skip test that is leaking connections. 2017-12-04 09:26:51 +08:00
Vinoth Kannan 7f2eeaf767 FIX: Password required flag should be cleared whenever clearing the raw password (#5384) 2017-12-01 15:19:24 +11:00
Guo Xiang Tan b1375ef44e Ensure that we disconnect connection in test. 2017-11-29 20:57:13 +08:00
Gerhard Schlager 44ee388070 FEATURE: omit images from og and twitter description tags 2017-11-28 21:34:02 +01:00
Jeff Wong b094894c94 Feature: Add service worker registration method to plugin API 2017-11-28 14:01:41 +08:00
Sam df84e1c358 Correctly track hijacked requests 2017-11-28 16:47:20 +11:00
Sam 0caa335ef0 FIX: Handle more cases where HTTP status is not correct
HTTP status was not correct with send_file which uses streaming
2017-11-28 11:00:13 +11:00
Sam ca7af7b88f FIX: displaying wrong avatar and letter avatar
correct regression where params and env is reused in production
2017-11-28 09:28:40 +11:00
Sam 608207b2e5 FEATURE: avatar proxy happens in background
This ensures that even if it is slow to download avatars site will
continue to work

Also simplifies hijack pattern
2017-11-27 17:43:24 +11:00
Guo Xiang Tan 2e04ef97d9 Fix the build. 2017-11-27 10:53:05 +08:00
Guo Xiang Tan a7030e80bf Skip randomly failing test. 2017-11-27 10:51:18 +08:00
Guo Xiang Tan 71942e4f62
Merge pull request #5352 from tgxworld/method_for_replica_postgres_connection
Expose `replica_postgresql_connection` to `ActiveRecord::Base`.
2017-11-27 08:38:10 +08:00
Sam e0e99d4bbd PERF: hijack onebox requests so they do not use up a unicorn worker 2017-11-24 15:31:40 +11:00
Guo Xiang Tan a509f466a0 Expose `replica_postgresql_connection` to `ActiveRecord::Base`. 2017-11-24 09:35:45 +08:00
Gerhard Schlager 613f4d737a FIX: updating topic stats failed silently for invalid topics 2017-11-23 18:47:45 +01:00
Sam 2b8d4508e5 PERF: stop running background work between requests
Use a dedicated thread to run Scheduler::Defer

This avoids blocking of a worker during operations that require waiting.

In particular uploads risked blocking a unicorn.

This also add a queue "length" that discourse prometheus consumes.
2017-11-23 15:48:47 +11:00
Guo Xiang Tan f7642e076d REFACTOR: Avoid duplicated logic on server and client. 2017-11-21 17:10:04 +08:00
Guo Xiang Tan 385372e384 Revert "PERF: Reduce number of topics to filter while querying for unread."
This reverts commit c06b782cab.
2017-11-20 11:49:09 +08:00
Gerhard Schlager 546b206da0 replace indentation with non-breaking spaces when cooking emails 2017-11-18 17:16:44 +01:00
Neil Lalonde 87ec11e298 FIX: more accurate counting of posts read. Skipping to the end of a topic does not count all posts as read in user stats. 2017-11-17 16:08:46 -05:00
Gerhard Schlager 1a3ab7c02e ignore some site settings for emails sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager aea161fabd suppress rejection email when email was sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Régis Hanol 678e28794a FIX: properly handle too large & broken images in posts 2017-11-16 15:45:07 +01:00
Guo Xiang Tan 3ab6318e5f Disable postgresql adapter tests until we figure out what is leaking
connections.
2017-11-16 07:17:22 +08:00
Sam f52111f787 FEATURE: allow plugins to easily detect if running in Rack
Usage: Discourse.running_in_rack? to tell if rack was booted
2017-11-16 08:39:29 +11:00
Gerhard Schlager 9207dee69a FEATURE: escape HTML when cooking plaintext emails 2017-11-15 20:22:11 +01:00
Régis Hanol 648832a643 fix the build 2017-11-15 11:53:14 +01:00
Vinoth Kannan 7b494a65c9 NEW: large image placeholder added in cooked html (#5291) 2017-11-15 11:30:47 +01:00
Guo Xiang Tan 89d9ffa884 EXPERIMENTAL: Allow logs to be shipped via different methods.
See https://github.com/dwbutler/logstash-logger#uri-configuration for
a list of available methods.
2017-11-15 09:11:33 +08:00
Sam 813e21d0e8 FIX: current user serializer consistently returns {} for custom_fields
Resolves: #5210
2017-11-15 11:55:37 +11:00
Régis Hanol 23baaa5dcc FIX: don't crop iPhone X screenshots 2017-11-14 20:37:27 +01:00
Robin Ward 971e302ff2 FEATURE: Support an end date for user silencing 2017-11-14 13:20:19 -05:00
Gerhard Schlager d3baae5365 removes whitespaces and uses scope 2017-11-13 15:23:24 +01:00
Sam 8a66446849 FEATURE: add overflow-y auto to Markdown tables 2017-11-13 17:52:15 +11:00
Sam 4f28c71b50 FIX: error setting tombstone bucket when set to old version 2017-11-13 15:36:45 +11:00
Sam 3ac7d041ae UX: generic onebox treats all square images as avatars and renders them smaller 2017-11-13 11:21:19 +11:00
Gerhard Schlager 5210e3e744 FEATURE: accept incoming email with reply_key mismatch when original email was forwarded 2017-11-12 23:44:22 +01:00
Gerhard Schlager fc6de6863b WIP 2017-11-12 21:12:34 +01:00
Gerhard Schlager 4dc4bc70c8 FIX: ignore_by_title should match case-insensitive 2017-11-12 01:43:18 +01:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Neil Lalonde 9dc9ca4ac0 FIX: be consistent with how first posts in topics are counted. do like DirectoryItem.refresh_period :all 2017-11-10 12:18:25 -05:00
Sam 06365023c4 FEATURE: new API to search for a user by email 2017-11-09 17:04:21 +11:00
Sam 2d5bf0705a PERF: exact email match bypass
instead of scanning full table when there is an exact email match ONLY
return the actual user.
2017-11-09 16:53:14 +11:00
Guo Xiang Tan 4bb454d889 FIX: JSON custom fields incorrectly being converted to an array.
https://meta.discourse.org/t/custom-fields-simultaneous-save-with-json-becomes-an-array/73647
2017-11-08 11:17:37 +08:00
Régis Hanol be0c7609f1 FIX: validates attachments against current authorized extensions 2017-11-07 19:17:33 +01:00
Arpit Jalan 8f560f0bf4
Merge pull request #5282 from vinothkannans/local_onebox_slash
FIX: Topic links onebox differently if end in /
2017-11-07 17:39:38 +05:30
Guo Xiang Tan d9602fe729 Fix incorrect severity field. 2017-11-07 11:24:54 +08:00
Guo Xiang Tan c9df21e131 FEATURE: Allow Unicorn logs to be JSON formatted. 2017-11-06 12:46:14 +08:00
Vinoth Kannan 230fec68ca FIX: Topic links onebox differently if end in / 2017-11-06 01:41:36 +05:30
Robin Ward 2f0c9793f1 FEATURE: Allow multiple html builders to be registered via plugins 2017-11-03 11:32:32 -04:00
ckeboss 93633865d9 Adds primary user group as a class to quote (#5285)
* Adds primary user group as a class to quote

This feature addition will add the class `group-PRIMARY_USER_GROUP` to
the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group
of the user being quoted. This is similar to the class that is added to
a `topic-post`.

* Remove trailing whitespace

* Fix avatar in test

* Address PR comments

* Fix trailing whitespace
2017-11-03 09:51:40 -04:00
Guo Xiang Tan af01860a3d Clear all active connections after PostgreSQL failover tests. 2017-11-03 09:21:10 +08:00
Neil Lalonde 21dd2ccd43 FIX: only count regular posts in user stats when deleting 2017-11-02 18:05:23 -04:00
Neil Lalonde 30689783db don't decrement post_count for a post in a deleted topic that has already been uncounted 2017-11-02 18:05:23 -04:00
Neil Lalonde f29290ad11 FIX: don't count whispers in user stats post_count 2017-11-02 18:05:23 -04:00
Neil Lalonde 24af9b7d97 FIX: when a topic is deleted, update the post count stats of all user who replied 2017-11-02 18:05:23 -04:00
Robin Ward 64cb8a3ce3 FIX: Normalizer wasn't working with attributes without values 2017-11-02 14:11:20 -04:00
Gerhard Schlager 880d154381 FIX: deleting staged user of rejected email shouldn't delete incoming email 2017-10-31 15:13:23 +01:00
Guo Xiang Tan 53cadbdfc5 Skip failing tests on Travis first. 2017-10-28 00:22:41 +08:00
Guo Xiang Tan b636e858bb Fix randomly failing spec on Travis. 2017-10-27 23:53:58 +08:00
Sam 70aed105a6 FIX: bypass all site setting work for shadowed method 2017-10-27 11:12:44 +11:00
Erick Guan 7c3123a2dd Downcase encoded slug by default and more specs 2017-10-26 16:50:29 +08:00
Guo Xiang Tan 1c7b4381cc Improve postgresql failover specs. 2017-10-26 15:54:50 +08:00
Guo Xiang Tan 94782152c6 Ensure we clean up state in PostgreSQL failover specs. 2017-10-26 09:22:51 +08:00
Sam 877b7be579 FIX: in readonly mode don't double count pages 2017-10-25 13:19:43 +11:00
Neil Lalonde fb5b9b6d06 FIX: don't offer the "Something Else" flag reason to TL0 users since they don't have permission to send private messages 2017-10-24 11:47:42 -04:00
Neil Lalonde 4452d67a23 Revert "FIX: TL0 users' messages to moderators were not being posted when flagging private messages" 2017-10-23 18:17:53 -04:00
Panayotis Matsinopoulos 821ed23799 Use the CategoryHashtag::SEPARATOR (#5258)
Use the CategoryHashtag::SEPARATOR like in the previous examples.
2017-10-23 14:59:08 +02:00
Guo Xiang Tan 54455fa40b Use MessageBus to get other processes to failover faster. 2017-10-23 17:58:24 +08:00
Guo Xiang Tan 509ecbe9fb Fix test not starting with the right state. 2017-10-23 12:20:39 +08:00
Arpit Jalan 804b4f32f8 better error message when API authentication fails 2017-10-20 20:05:34 +05:30
Guo Xiang Tan fe1e78ddf4 Make PostgreSQL failover work with distributed cache. 2017-10-20 17:15:29 +08:00
Gerhard Schlager 1481462cbf PERF: Move oneboxing from cook method "email" to postprocessing 2017-10-20 10:26:45 +02:00
Guo Xiang Tan 057571d173 Raise errors in PostgreSQLFallbackHandler threads. 2017-10-20 12:06:03 +08:00
Robin Ward 838568cbc3 Refactor flag types for more customization 2017-10-19 13:55:23 -04:00
Guo Xiang Tan 25c25ae423 FEATURE: Allow user to leave a PM. 2017-10-19 12:32:55 +08:00
Guo Xiang Tan 79de10b212 FEATURE: Allow users to disable new PMs.
https://meta.discourse.org/t/is-it-possible-to-disable-private-messaging-for-a-specific-user/46391
2017-10-19 12:32:55 +08:00
Guo Xiang Tan 814c7ab503 Skip randomly failing tests first. 2017-10-19 12:25:50 +08:00
Régis Hanol cbdfc85466 FIX: images aren't lightboxed anymore (partially reverts 646c6eb7cd) 2017-10-18 23:54:36 +02:00
Sam a4c539bade FEATURE: Allow registration of detailed request logger
Detailed request loggers can be used to gather rich timing info
from all requests (which in turn can be forwarded to monitoring solution)

Middleware::RequestTracker.detailed_request_logger(->|env, data| do
   # do stuff with env and data
end
2017-10-18 12:10:30 +11:00
Sam 1dd2b51059 remove redundent stubs 2017-10-18 12:10:30 +11:00
Guo Xiang Tan 1b5ee0ae72 FIX: Couldn't boot Discourse app with a readonly postgres. 2017-10-17 13:44:30 +08:00
Sam Saffron 8185b8cb06 FEATURE: cache https redirects per hostname
If a hostname does an https redirect we cache that so next
lookup does not incur it.

Also, only rate limit per ip once per final destination

Raise final destination protection to 1000 ip lookups an hour
2017-10-17 16:22:54 +11:00
Sam Saffron 646c6eb7cd FEATURE: add :before_post_process_cooked hook
Also reduce amount of image downloading
2017-10-17 16:22:54 +11:00
Neil Lalonde 2db66072d7 SECURITY: signup without verified email using Google auth 2017-10-16 13:51:41 -04:00
Guo Xiang Tan 5f76e5062d Pause Sidekiq when postgres failovers. 2017-10-16 19:48:31 +08:00
Neil Lalonde b124e5f19f FIX: TL0 users' messages to moderators were not being posted when flagging private messages 2017-10-13 11:55:49 -04:00