Commit Graph

64 Commits

Author SHA1 Message Date
David Taylor cd6b7459a7
DEV: Improve background-request information in request_tracker (#16037)
This will allow consumers (e.g. the discourse-prometheus plugin) to separate topic-timings and message-bus requests. It also fixes the is_background boolean for subfolder sites.
2022-02-23 12:45:42 +00:00
Osama Sayegh b86127ad12
FEATURE: Apply rate limits per user instead of IP for trusted users (#14706)
Currently, Discourse rate limits all incoming requests by the IP address they
originate from regardless of the user making the request. This can be
frustrating if there are multiple users using Discourse simultaneously while
sharing the same IP address (e.g. employees in an office).

This commit implements a new feature to make Discourse apply rate limits by
user id rather than IP address for users at or higher than the configured trust
level (1 is the default).

For example, let's say a Discourse instance is configured to allow 200 requests
per minute per IP address, and we have 10 users at trust level 4 using
Discourse simultaneously from the same IP address. Before this feature, the 10
users could only make a total of 200 requests per minute before they got rate
limited. But with the new feature, each user is allowed to make 200 requests
per minute because the rate limits are applied on user id rather than the IP
address.

The minimum trust level for applying user-id-based rate limits can be
configured by the `skip_per_ip_rate_limit_trust_level` global setting. The
default is 1, but it can be changed by either adding the
`DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL` environment variable with the
desired value to your `app.yml`, or changing the setting's value in the
`discourse.conf` file.

Requests made with API keys are still rate limited by IP address and the
relevant global settings that control API keys rate limits.

Before this commit, Discourse's auth cookie (`_t`) was simply a 32 characters
string that Discourse used to lookup the current user from the database and the
cookie contained no additional information about the user. However, we had to
change the cookie content in this commit so we could identify the user from the
cookie without making a database query before the rate limits logic and avoid
introducing a bottleneck on busy sites.

Besides the 32 characters auth token, the cookie now includes the user id,
trust level and the cookie's generation date, and we encrypt/sign the cookie to
prevent tampering.

Internal ticket number: t54739.
2021-11-17 23:27:30 +03:00
Rafael dos Santos Silva b136375582
FEATURE: Rate limit exceptions via ENV (#14033)
Allow admins to configure exceptions to our Rails rate limiter.

Configuration happens in the environment variables, and work with both
IPs and CIDR blocks.

Example:

```
env:
  DISCOURSE_MAX_REQS_PER_IP_EXCEPTIONS: >-
    14.15.16.32/27
    216.148.1.2
```
2021-08-13 12:00:23 -03:00
Bianca Nenciu 765ba1ab2d
FEATURE: Ignore anonymous page views on private sites (#12800)
For sites with login_required set to true, counting anonymous pageviews is
confusing. Requests to /login and other pages would make it look like
anonymous users have access to site's content.
2021-04-26 14:19:47 +03:00
Jarek Radosz 6ff888bd2c
DEV: Retry-after header values should be strings (#12475)
Fixes `Rack::Lint::LintError: a header value must be a String, but the value of 'Retry-After' is a Integer`. (see: 14a236b4f0/lib/rack/lint.rb (L676))

I found it when I got flooded by those warning a while back in a test-related accident 😉 (ember CLI tests were hitting a local rails server at a fast rate)
2021-03-23 20:32:36 +01:00
Martin Brennan 6eb0d0c38d
SECURITY: Fix is_private_ip for RateLimiter to cover all cases (#12464)
The regular expression to detect private IP addresses did not always detect them successfully.
Changed to use ruby's in-built IPAddr.new(ip_address).private? method instead
which does the same thing but covers all cases.
2021-03-22 13:56:32 +10:00
Dan Ungureanu 1f2f84a6df
FIX: Add Retry-Header to rate limited responses (#11736)
It returned a 429 error code with a 'Retry-After' header if a
RateLimiter::LimitExceeded was raised and unhandled, but the header was
missing if the request was limited in the 'RequestTracker' middleware.
2021-01-19 11:35:46 +02:00
Tobias Eigen 0a0fd6eace
DEV: fixed capitalization in rate limit message (#11193) 2020-11-11 12:35:03 +11:00
Sam 2686d14b9a
PERF: introduce aggressive rate limiting for anonymous (#11129)
Previous to this change our anonymous rate limits acted as a throttle.
New implementation means we now also consider rate limited requests towards
the limit.

This means that if an anonymous user is hammering the server it will not be
able to get any requests through until it subsides with traffic.
2020-11-05 16:36:17 +11:00
Aman Gupta Karmani 8a86705e51
FIX: handle heroku style HTTP_X_REQUEST_START (#10087) 2020-06-19 10:17:24 -04:00
Daniel Waterworth bca126f3f5 REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.

Follow-up-to: ca1208a636
2020-04-02 16:44:44 +01:00
Daniel Waterworth ca1208a636 Revert "REFACTOR: Move the multisite middleware to the front"
Looks like this is causing problems.

Follow-up-to: a91843f0dc
2020-04-02 15:20:28 +01:00
Daniel Waterworth a91843f0dc REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.
2020-04-02 10:15:38 +01:00
Sam Saffron 494fe335d3 DEV: allow handling crawler reqs with no user agent
Followup to e440ec25 we treat no user agent as crawler reqs.
2019-12-09 18:40:10 +11:00
Sam Saffron e440ec2519 FIX: crawler requests not tracked for non UTF-8 user agents
Non UTF-8 user_agent requests were bypassing logging due to PG always
wanting UTF-8 strings.

This adds some conversion to ensure we are always dealing with UTF-8
2019-12-09 17:43:51 +11:00
Daniel Waterworth 563253e9ed FIX: Fix options given to per-minute rate limiter
Previously the options for the per-minute and per-10-second rate
limiters were the same.
2019-09-20 10:48:59 +01:00
Sam Saffron 08743e8ac0 FEATURE: anon cache reports data to loggers
This allows custom plugins such as prometheus exporter to log how many
requests are stored in the anon cache vs used by the anon cache.

This metric allows us to fine tune cache behaviors
2019-09-02 18:45:35 +10:00
Sam Saffron 1f47ed1ea3 PERF: message_bus will be deferred by server when flooded
The message_bus performs a fair amount of work prior to hijacking requests
this change ensures that if there is a situation where the server is flooded
message_bus will inform client to back off for 30 seconds + random(120 secs)

This back-off is ultra cheap and happens very early in the middleware.

It corrects a situation where a flood to message bus could cause the app
to become unresponsive

MessageBus update is here to ensure message_bus gem properly respects
Retry-After header and status 429.

Under normal state this code should never trigger, to disable raise the
value of DISCOURSE_REJECT_MESSAGE_BUS_QUEUE_SECONDS, default is to tell
message bus to go away if we are queueing for 100ms or longer
2019-08-09 17:48:01 +10:00
Sam Saffron 62141b6316 FEATURE: enable_performance_http_headers for performance diagnostics
This adds support for DISCOURSE_ENABLE_PERFORMANCE_HTTP_HEADERS
when set to `true` this will turn on performance related headers

```text
X-Redis-Calls: 10     # number of redis calls
X-Redis-Time: 1.02    # redis time in seconds
X-Sql-Commands: 102   # number of SQL commands
X-Sql-Time: 1.02      # duration in SQL in seconds
X-Queue-Time: 1.01    # time the request sat in queue (depends on NGINX)
```

To get queue time NGINX must provide: HTTP_X_REQUEST_START

We do not recommend you enable this without thinking, it exposes information
about what your page is doing, usually you would only enable this if you
intend to strip off the headers further down the stream in a proxy
2019-06-05 16:08:11 +10:00
David Taylor 8963f1af30
FEATURE: Optional detailed performance logging for Sidekiq jobs (#7091)
By default, this does nothing. Two environment variables are available:

- `DISCOURSE_LOG_SIDEKIQ`

  Set to `"1"` to enable logging. This will log all completed jobs to `log/rails/sidekiq.log`, along with various db/redis/network statistics. This is useful to track down poorly performing jobs.

- `DISCOURSE_LOG_SIDEKIQ_INTERVAL`

  (seconds) Check running jobs periodically, and log their current duration. They will appear in the logs with `status:pending`. This is useful to track down jobs which take a long time, then crash sidekiq before completing.
2019-03-05 11:19:11 +00:00
Guo Xiang Tan c732ae9ca9 FIX: Don't update `User#last_seen_at` when PG is in readonly. 2019-01-21 13:29:29 +08:00
Guo Xiang Tan e2a20d90fe FIX: Don't log request when Discourse is in readonly due to PG. 2019-01-21 11:04:32 +08:00
Sam a19170a4c2 DEV: avoid require_dependency for some libs
This avoids require dependency on method_profiler and anon cache.

It means that if there is any change to these files the reloader will not pick it up.

Previously the reloader was picking up the anon cache twice causing it to double load on boot.

This caused warnings.

Long term my plan is to give up on require dependency and instead use:

https://github.com/Shopify/autoload_reloader
2018-12-31 10:53:30 +11:00
Sam 955cdad649 FIX: exec_params needs instrumentation
the method no longer routes to "exec" or "async_exec" in latest PG so we
need to explicitly intercept
2018-12-10 14:28:10 +11:00
Sam 168ffd8384 FEATURE: group warnings about IP level rate limiting 2018-08-13 14:38:20 +10:00
Sam 7f98ed69cd FIX: move crawler blocking to app controller
We need access to site settings in multisite, we do not have access
yet if we attempt to get them in request tracker middleware
2018-07-04 10:30:50 +10:00
Neil Lalonde e8a6323bea remove crawler blocking until multisite support 2018-07-03 17:54:45 -04:00
Sam 4810ce3607 correct regression 2018-04-18 21:04:08 +10:00
Neil Lalonde b87fa6d749 FIX: blacklisted crawlers could get through by omitting the accept header 2018-04-17 12:39:30 -04:00
Sam 9980f18d86 FEATURE: track request queueing as early as possible 2018-04-17 18:06:17 +10:00
Neil Lalonde 4d12ff2e8a when writing cache, remove elements from the user agents list. also return a message and content type when blocking a crawler. 2018-03-27 13:44:14 -04:00
Neil Lalonde a84bb81ab5 only applies to get html requests 2018-03-22 17:57:44 -04:00
Neil Lalonde ced7e9a691 FEATURE: control which web crawlers can access using a whitelist or blacklist 2018-03-22 15:41:02 -04:00
Sam 0134e41286 FEATURE: detect when client thinks user is logged on but is not
This cleans up an error condition where UI thinks a user is logged on
but the user is not. If this happens user will be prompted to refresh.
2018-03-06 16:49:31 +11:00
Sam f0d5f83424 FEATURE: limit assets less that non asset paths
By default assets can be requested up to 200 times per 10 seconds
from the app, this includes CSS and avatars
2018-03-06 15:20:39 +11:00
Sam f295a18e94 FIX: stop double counting net calls in logs 2018-02-28 10:45:11 +11:00
Sam ca1a3f37e3 FEATURE: add instrumentation for all external net calls 2018-02-21 15:20:29 +11:00
Guo Xiang Tan 3e835047da Remove "already initialized" constant warning. 2018-02-13 08:55:15 +08:00
Sam Saffron df8e43abdd use lazy & instead of try
unregister ip skipper in test
raise if called when a skipper is in play
2018-02-06 10:38:15 +11:00
Robin Ward eefd226611 Add extensibility point to `request_tracker` to skip IP addresses
This is useful if you want to run a per IP rate limiter but want to be
able to skip some IPs with custom logic.
2018-02-05 17:49:40 -05:00
Sam f26ff290c3 FEATURE: Shorten setting name to max_reqs
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
Sam 8bf91b8dca correct tracking of x runtime 2018-01-19 17:51:19 +11:00
Sam 8ad43f01c2 FIX: correctly log topic timings as background 2018-01-19 10:37:43 +11:00
Sam 12872d03be PERF: run post timings in background
This means that if a very large amount of registered users hit
a single topic we will handle it gracefully, even if db gets slow.
2018-01-19 08:27:29 +11:00
Sam cecd7d0d07 FEATURE: global rate limiter can bypass local IPs 2018-01-08 08:39:17 +11:00
Sam 715cb98e95 add better diagnostics for rate limits 2018-01-05 12:14:28 +11:00
Sam bbc606988f improve message 2017-12-20 10:12:33 +11:00
Sam 4986ebcf24 FEATURE: optional default off global per ip rate limiter 2017-12-11 17:52:57 +11:00
Sam df84e1c358 Correctly track hijacked requests 2017-11-28 16:47:20 +11:00
Sam a4c539bade FEATURE: Allow registration of detailed request logger
Detailed request loggers can be used to gather rich timing info
from all requests (which in turn can be forwarded to monitoring solution)

Middleware::RequestTracker.detailed_request_logger(->|env, data| do
   # do stuff with env and data
end
2017-10-18 12:10:30 +11:00