Commit Graph

57 Commits

Author SHA1 Message Date
Krzysztof Kotlarek 858cf5836c
FIX: update Redis gem to version 4.1.3
I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

Redis gem is manipulating Redis config https://github.com/redis/redis-rb/blob/master/lib/redis/client.rb#L95
therefore we cannot pass the frozen config object.

Pass of the copy of the object is protecting original config
2019-10-21 09:59:24 +11:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Daniel Waterworth 6924f1ab15 FIX: Race-condition in fallback handlers (#8005)
Calling `verify_master` in multiple threads simultaneously would cause
multiple threads to be spawned.
2019-08-21 15:47:44 +02:00
David Taylor afb5ec811d FIX: Don't use DistributedCache to store redis readonly state
This can cause unbound CPU usage in some cases, and excessive logging in other cases. This commit moves redis readonly information into the local process, but maintains the DistributedCache for postgres readonly state.
2019-06-25 11:20:34 +08:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan 152238b4cf DEV: Prefer `public_send` over `send`. 2019-05-07 09:33:21 +08:00
Guo Xiang Tan 4dbebb4915 Follow up to f6f2c38183.
Master connection might fail.
2019-01-22 12:51:59 +08:00
Guo Xiang Tan f6f2c38183 FIX: Fallback Redis by checking status on master instead of slave. 2019-01-22 12:38:12 +08:00
Guo Xiang Tan 2cbb513c98 FIX: Don't use `Redis#keys` in production.
As per the documentation for KEYS

```
Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout.
```

Instead SCAN

```
Since these commands allow for incremental iteration, returning only a small number of elements per call, they can be used in production without the downside of commands like KEYS or SMEMBERS that may block the server for a long time (even several seconds) when called against big collections of keys or elements.
```
2018-12-25 15:12:59 +02:00
Guo Xiang Tan 2adae2a0b6 FIX: Be more aggressive in checking master when trying to fallback. 2018-06-12 00:09:15 +08:00
Guo Xiang Tan 45fe5dc793 `$redis.client` -> `$redis._client`.
See c239abb43c
2018-04-20 13:01:17 +08:00
Sam 877b7be579 FIX: in readonly mode don't double count pages 2017-10-25 13:19:43 +11:00
Sam 61f32ed431 FIX: we can not namespace eval automatically, it is impossible 2017-10-11 16:41:26 +11:00
Guo Xiang Tan b2127600fb Remove use of concurrent timer for Redis failover.
* Uses the same logic for Postgres failover.
2017-10-05 15:57:08 +08:00
Sam aeedecd27c correct erratically failing spec 2017-08-18 15:10:37 -04:00
Guo Xiang Tan 9bc3038728 Fix Redis command errors when trying to start app with a readonly Redis. 2017-08-02 21:33:56 +09:00
Guo Xiang Tan 7d5b6e2b3c Disable MessageBus's keepalive when Redis is readonly. 2017-08-01 23:07:52 +09:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Régis Hanol d792e81372 don't pollute test suite with warning message from readonly Redis 2017-07-24 10:22:32 +02:00
Guo Xiang Tan dd4cab3be2 FIX: Set master to true before forcing slave connections to reconnect. 2016-11-23 14:04:43 +08:00
Guo Xiang Tan 02025207d5 FIX: Make sure Redis fallback don't fall into a permanent readonly state. 2016-11-23 11:31:20 +08:00
Guo Xiang Tan fbbcde1230 FIX: Don't treat master as up if it is still loading data. 2016-11-07 15:28:10 +08:00
Guo Xiang Tan 9375dcb6fe PERF: Spawn a seperate timer task to check if Redis master is up. 2016-11-07 15:04:28 +08:00
Matt Palmer aa56100660 Failover messages really aren't warnings
"I'm going to do something entirely sane and reasonable" doesn't warrant a
warning-level log message.  It's perfectly fine and reasonable to just log
that sort of thing at info level.
2016-08-08 16:02:23 +10:00
Guo Xiang Tan 36e3f1f5e4
FIX: Reset state of `DiscourseRedis::FallbackHandler` in tests. 2016-04-18 10:41:40 +08:00
Guo Xiang Tan 862b4fc9da
FIX: `DiscourseRedis::FallbackHandler` not restricting number of threads spawned. 2016-04-15 17:05:03 +08:00
Guo Xiang Tan 60c21a3393 Extract valid connection types into a constant. 2016-04-07 15:45:42 +08:00
Guo Xiang Tan 7f6fe8fe13 FIX: Pubsub connections need to be killed too. 2016-04-07 15:30:05 +08:00
Guo Xiang Tan 1481b3420e Add more logging so that we know what is happening with Redis fallback. 2016-04-01 14:24:09 +08:00
Sam 643d7f07c3 correct very odd error on connect to redis 2016-03-22 10:02:19 +11:00
Guo Xiang Tan bf209d8344 FIX: Redis hostname may resolve to nothing. 2016-03-11 19:06:37 +08:00
Guo Xiang Tan c07c474575 FEATURE: Master-Slave Redis configuration with fallback and switch over. 2016-03-11 12:18:58 +08:00
Sam da41b11309 FIX: mget not implemented correctly 2015-09-28 16:50:24 +10:00
Sam 8252f4e110 FEATURE: allow use of redis sentinel via redis_sentinels
Use: DISCOURSE_REDIS_SENTINELS and DISCOURSE_REDIS_HOST to configure redis
sentinel
2015-06-25 16:51:48 +10:00
Sam b7e7da766d avoid static method so we can easily inherit off this class 2015-05-06 09:53:29 +10:00
Robin Ward 16408cee06 Allow Postgres to trigger readonly mode for the site. 2015-04-29 11:49:58 -04:00
Robin Ward 3a6efa25f0 Allow ReadOnly to propogate up to the Ember app via Response Header 2015-04-24 14:37:16 -04:00
Robin Ward 5b3f99aa50 Don't blow up if Redis switches to READONLY 2015-04-24 14:37:16 -04:00
Robin Ward f15b0d205f FIX: The "too similar" check happened when trying to make a post a wiki 2015-02-02 12:44:56 -05:00
Robin Ward 6e862e0e59 FIX: Should flush rate limit keys before testing it 2015-01-29 11:45:12 -05:00
Sam b703d8c77a BUGFIX: redis-rails has always been a problem child
implemented an ActiveSupport::Cache::Store for our internal use.
* allows for expire by family
* works correctly in multisite
* namespaced correctly

Removed redis-rails from the project, no longer needed
2014-01-06 16:50:04 +11:00
Sam 7b7d6f3fa7 Merge pull request #1464 from GarantiaData/single_redis_db
Support single redis DB.
2014-01-05 16:30:41 -08:00
Neil Lalonde df220ae973 Get detailed info about new versions from the Discourse Hub. Include version notes from the latest version in notification email. 2013-12-31 15:52:35 -05:00
Neil Lalonde ab12695d63 Block passwords that are in the top 5000 most common passwords. Site setting block_common_passwords can disable this feature. 2013-12-27 11:00:21 -05:00
Discourse application 0c8f6ba9a4 Support single redis DB. 2013-09-24 15:47:35 +03:00
Neil Lalonde 2df46a9d0a Show a more useful message when redis config can't be found during init. 2013-05-15 10:08:08 -04:00
Sam caa3cae535 remove pointless to_sym 2013-05-06 09:51:09 +10:00
Michael Witrant 6018c2f98b use symbols in redis proxy methods 2013-05-05 15:58:37 +02:00
Michael Witrant 2031b69e6e use method name instead of __method__ (which is wrong in this situation on Debian Squeeze Ruby 1.9) 2013-05-05 13:34:54 +02:00
Sam 850b042cab introduce rack:cache as a default, so users don't need to configure apache or nginx
under rack cache we are able to serve 620reqs a second per thin (on my machine) before it 12 (on my machine)

reorganised so mini profilers can be cleanly disabled from config file

added caching for categories index

move production.rb to production.sample.rb
2013-04-11 16:24:21 +10:00