Discource-C
/
discourse
mirror of https://github.com/discourse/discourse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PERF: Add exponential backoff for DistributedMutex (#17886) 

Polling every 0.001s can cause extreme load on the redis instance, especially in scenarios where multiple app instances are waiting on the same lock. This commit introduces an exponential backoff starting from 0.001s and reaching a maximum interval of 1s.

Previously `CHECK_READONLY_ATTEMPTS` was 10, and resulted in a block for 0.01s. Under the new logic, 10 attempts take more than 1s. Therefore CHECK_READONLY_ATTEMPTS is reduced to 5, bringing its total time to around 0.031s
This commit is contained in:
David Taylor 2022-08-12 18:39:01 +01:00 committed by GitHub
parent 636be8cac5
commit 2422ca0e67
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
lib/distributed_mutex.rb
View File

@ -4,7 +4,7 @@
# Expiration happens when the current time is greater than the expire time # Expiration happens when the current time is greater than the expire time
class DistributedMutex class DistributedMutex
DEFAULT_VALIDITY = 60 DEFAULT_VALIDITY = 60
CHECK_READONLY_ATTEMPTS = 10 CHECK_READONLY_ATTEMPTS = 5
LOCK_SCRIPT = DiscourseRedis::EvalHelper.new <<~LUA LOCK_SCRIPT = DiscourseRedis::EvalHelper.new <<~LUA
local now = redis.call("time")[1] local now = redis.call("time")[1]
@ -85,7 +85,9 @@ class DistributedMutex
return expire_time if expire_time return expire_time if expire_time
sleep 0.001 # Exponential backoff, max duration 1s
interval = attempts < 10 ? (0.001 * 2**attempts) : 1
sleep interval
# in readonly we will never be able to get a lock # in readonly we will never be able to get a lock
if @using_global_redis && Discourse.recently_readonly? if @using_global_redis && Discourse.recently_readonly?