Commit Graph

102 Commits

Author SHA1 Message Date
Sam 70269c7c97 FEATURE: tighter limits on per cluster post rebakes
We have the periodical job that regularly will rebake old posts. This is
used to trickle in update to cooked markdown. The problem is that each rebake
can issue multiple background jobs (post process and pull hotlinked images)

Previously we had no per-cluster limit so cluster running 100s of sites could
flood the sidekiq queue with rebake related jobs.

New system introduces a hard limit of 300 rebakes per 15 minutes across a
cluster to ensure the sidekiq job is not dominated by this.

We also reduced `rebake_old_posts_count` to 80, which is a safer default.
2019-01-04 09:24:46 +11:00
Rishabh a6c589d882 FEATURE: Add custom S3 Endpoint and DigitalOcean Spaces/Minio support for Backups (#6045)
- Add custom S3 Endpoints and DigitalOcean Spaces support
- Add Minio support using 'force_path_style' option and fix uploads to custom endpoint
2018-07-16 14:44:55 +10:00
Sam 87fabdc2f3 FIX: correct pool reaper
This removes a freedom patch and replaces with a custom reaper thread
it also captures an issue where reaper would fail when connections where
empty
2018-06-14 18:22:02 +10:00
Sam ded84a4b58 PERF: improve performance once logged in rate limiter hits
If "logged in" is being forced anonymous on certain routes, trigger
the protection for any requests that spend 50ms queueing

This means that ...

1. You need to trip it by having 3 requests take longer than 1 second in 10 second interval
2. Once tripped, if your route is still spending 50m queueuing it will continue to be protected

This means that site will continue to function with almost no delays while it is scaling up to handle the new load
2018-04-23 11:55:25 +10:00
Sam 59cd7894d9 FEATURE: if site is under extreme load show anon view
If a particular path is being hit extremely hard by logged on users,
revert to anonymous cached view.

This will only come into effect if 3 requests queue for longer than 2 seconds
on a *single* path.

This can happen if a URL is shared with the entire forum base and everyone
is logged on
2018-04-18 16:58:57 +10:00
Blake Erickson 720dd2432e remove change from descourse_defaults.conf 2018-04-10 14:27:03 -06:00
Blake Erickson 0337a8f6d5 ensure correct '/'s for relative_url_root in route file 2018-04-10 14:24:29 -06:00
Guo Xiang Tan a89f3160a5 Add new config to ensure backup/restore connects to PG directly.
* In `pg_dump` 10.3+ and 9.5.12+, in
  it does a `SELECT pg_catalog.set_config('search_path', '', false)`
  which changes the state of the current connection. This is known
  to be problematic with Pgbouncer which reuses connections. As such,
  we'll always try to connect directly to PG directly during
  the backup/restore process.
2018-03-09 10:28:03 +08:00
Sam f0d5f83424 FEATURE: limit assets less that non asset paths
By default assets can be requested up to 200 times per 10 seconds
from the app, this includes CSS and avatars
2018-03-06 15:20:39 +11:00
Sam f26ff290c3 FEATURE: Shorten setting name to max_reqs
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
Sam cecd7d0d07 FEATURE: global rate limiter can bypass local IPs 2018-01-08 08:39:17 +11:00
Sam 4986ebcf24 FEATURE: optional default off global per ip rate limiter 2017-12-11 17:52:57 +11:00
Sam 68d3c2c74f FEATURE: add global rate limiter for admin api 60 per minute
Also move configuration of admin and user api rate limiting into global
settings. This is not intended to be configurable per site
2017-12-11 11:07:22 +11:00
Guo Xiang Tan 6c04eb911d Fix typo. 2017-10-17 12:34:49 +08:00
Guo Xiang Tan b54eb8f53c FIX: Set PG `connect_timeout` to 5 seconds.
* 30 seconds is alittle too long.
2017-10-17 12:32:41 +08:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Sam c106ca6778 FEATURE: fallback asset path for multi host setups 2017-03-20 15:59:17 -04:00
Sam ff49f72ad9 FEATURE: per client user tokens
Revamped system for managing authentication tokens.

- Every user has 1 token per client (web browser)
- Tokens are rotated every 10 minutes

New system migrates the old tokens to "legacy" tokens,
so users still remain logged on.

Also introduces weekly job to expire old auth tokens.
2017-02-07 09:22:16 -05:00
Guo Xiang Tan 89b1998174 Add default port for redis_slave. 2016-03-11 15:07:07 +08:00
Guo Xiang Tan c07c474575 FEATURE: Master-Slave Redis configuration with fallback and switch over. 2016-03-11 12:18:58 +08:00
Guo Xiang Tan 46589a1a0c FEATURE: AR adapter to failover to a replica DB server. 2016-02-05 08:51:10 +08:00
Sam Saffron 209b022385 PERF: cut down on memory usage allowed to redis
This limits the amount of backlog message bus channels can have.
2016-02-04 13:58:38 +11:00
Sam dfe3ecb914 PERF: disable prepared statements
see: https://github.com/rails/rails/issues/21992
2015-10-19 14:02:22 +11:00
Sam 805120fc95 FEATURE: add connnection reaping based on maximum age
This feature ensures connections to the db are always attempted to be closed
after 600 seconds of idle time.
2015-10-17 11:29:16 +11:00
Sam 786cf6e61f format does not allow numbers 2015-07-23 15:33:38 +10:00
Sam c056c3ba7e FEATURE: allow users to specify a second hostname if needed
(very rarely needed feature, mostly for multisite and origin pull cdns)
2015-07-23 15:22:54 +10:00
Sam 8252f4e110 FEATURE: allow use of redis sentinel via redis_sentinels
Use: DISCOURSE_REDIS_SENTINELS and DISCOURSE_REDIS_HOST to configure redis
sentinel
2015-06-25 16:51:48 +10:00
Sam f3188ab8f2 remove unused setting 2015-04-30 12:06:32 +10:00
Sam f5af4768eb FEATURE: add clean support for running Discourse in a subfolder
To setup set DISCOURSE_RELATIVE_URL_ROOT to the folder you wish
2015-03-09 13:14:29 +11:00
Sam fff55e0ee3 FEATURE: allow users to specify if prepared statements are allowed
(they need to be disabled for pgpool based setups)
2015-02-18 11:16:53 +11:00
Sam d56b71851b FEATURE: configurable connection reaping settings 2015-02-17 09:58:43 +11:00
Neil Lalonde f88075cbba FEATURE: CORS settings per-site in a multisite env 2014-10-15 15:20:39 -04:00
Sam 2be03371ae FEATURE: allow a central redis cache for assets 2014-09-23 16:50:17 +10:00
Sam f493eb5d12 FEATURE: allow version emails to be disabled globally 2014-08-23 10:02:32 +10:00
Zohar Arad 557af8b8bd Add RTL support to Discourse CSS pre-processing using the RTLit gem and configurable via discourse.conf 2014-08-08 09:31:31 +03:00
Sam 9468ebeb2e CHANGE: Mini Profiler only enabled for developers in prd 2014-07-17 08:34:41 +10:00
Adam Loving 3f1c5bef00 spelling in comment 2014-06-05 09:52:35 -07:00
Sam 30c86d834f * Increase pool size to allow for a few extra bg threads in sidekiq 2014-05-22 09:00:24 +10:00
Sam 810eda448f Merge branch 'environment-smtp-verify' of https://github.com/cfstras/discourse 2014-05-14 11:25:05 +10:00
Sam dc06401479 PERF: reduce sidekiq worker count to 5 2014-05-14 10:21:11 +10:00
Claus Strasburger b53248178b Added `openssl_verify_mode` parameter for action_mailer.
- parameter in `environments/production.rb`
- documentation & default value (nil) in `discourse_defaults.conf`
2014-05-08 22:21:36 +02:00
Sam 884346cbea Merge pull request #1951 from thoughtbot/bb-erb
Use ERB to pull conf files into app
2014-02-18 16:23:55 +11:00
Sam 73aca9063d FEATURE: allow overriding server static asset if needed 2014-02-17 10:44:37 +11:00
Britt Ballard 60e1a5aa69 Use ERB when importing conf files into app 2014-02-14 16:14:55 -08:00
slainer68 a89018db87 Use GlobalSetting to enable CORS at application level 2014-02-09 23:11:52 -08:00
Vikhyat Korrapati 6d50504a83 Make SMTP authentication mechanism configurable. 2014-01-06 18:19:42 +05:30
Sam c7d99a288b try to default to sockets, so less people have issues configuring in prd 2014-01-06 12:25:29 +11:00
Sam 887ac10013 get rid of cache db, not needed 2014-01-06 12:24:11 +11:00
David Celis 0ed42707cc Allow configuration of Redis DB and cache DB
Hardcoding the Redis DB and Redis Caching DB to 0 and 2 in
`config/database.yml` makes an unsafe assumption that Discourse is the
only application using that install of redis-server. Instead of forcing
users to undergo yet another form of configuration, allow Discourse
admins a nicer way to configure the Redis databases used.

Signed-off-by: David Celis <me@davidcel.is>
2013-12-30 13:39:43 -08:00
Sam 8b33b45c2d should be commented out 2013-12-20 18:01:41 +11:00
Sam 4b04ac4c2b correct defaults 2013-12-20 16:23:01 +11:00
Sam 7b8d2547d0 globals now implemented and documented 2013-12-20 16:17:21 +11:00