Commit Graph

33 Commits

Author SHA1 Message Date
David Taylor e2449f9f23 Revert "Revert "Revert "FIX: Heartbeat check per sidekiq process (#7873)"""
This reverts commit c3497559be.
2019-08-30 11:26:16 +01:00
Sam Saffron c3497559be Revert "Revert "FIX: Heartbeat check per sidekiq process (#7873)""
This reverts commit e805d44965.
We now have mechanisms in place to ensure heartbeat will always
be scheduled even if the scheduler is overloaded per: 098f938b
2019-08-30 10:12:10 +10:00
OsamaSayegh e805d44965 Revert "FIX: Heartbeat check per sidekiq process (#7873)"
This reverts commit 340855da55.
2019-08-27 11:56:23 +00:00
Osama Sayegh 340855da55
FIX: Heartbeat check per sidekiq process (#7873)
* FIX: Heartbeat check per sidekiq process

* Rename method

* Remove heartbeat queues of previous bootups

* Regis feedback

* Refactor before_start

* Update lib/demon/sidekiq.rb

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Update lib/demon/sidekiq.rb

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Expire redis keys after 3600 seconds

* Don't use redis to store the list of queues
2019-08-26 09:33:49 +03:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam 384135845b FEATURE: introduce ultra_low priority queue
This commit introduces an ultra low priority queue for post rebakes. This
way rebakes can never interfere with regular sidekiq processing for cases
where we perform a large scale rebake.

Additionally it allows Post.rebake_old to be run with rate_limiter: false
to avoid triggering the limiter when rebaking. This is handy for cases
where you want to just force the full rebake and not wait for it to trickle
2019-01-17 14:53:19 +11:00
Sam df460b4abd PERF: run sidekiq with nice 5
This ensures that unicorn master forks of sidekiq run with a lower priority
than the webs. It means that a busy sidekiq is less likely to impact web
performance
2019-01-09 09:29:14 +11:00
Sam cc3fc87dd7 DEV: handle termination cleanly in autospec 2018-06-19 16:13:36 +10:00
Sam fc05164667 demo script for demonizing using fork exec
minor refinements to demon
2018-01-11 13:51:52 +11:00
Guo Xiang Tan 470b1a5bc1 Don't print Sidekiq starting message to STDERR. 2017-11-03 21:02:31 +08:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan 7ea288140d Allow multiple host when restricting Sidekiq queues. 2017-06-19 14:45:51 +09:00
Guo Xiang Tan 84490c4558 Allow a sidekiq queue to be configured to only run on a certain hostname. 2017-04-27 15:32:16 +08:00
Sam 0b3aec9c94 FEATURE: set UNICORN_STATS_SOCKET_DIR for status socket
eg:

sam@ubuntu stats_sockets % socat - UNIX-CONNECT:9622.sock
gc_stat
{"count":46,"heap_allocated_pages":2459,"heap_sorted_length":2460,"heap_allocatable_pages":0,"heap_available_slots":1002267,"heap_live_slots":647293,"heap_free_slots":354974,"heap_final_slots":0,"heap_marked_slots":503494,"heap_swept_slots":498773,"heap_eden_pages":2459,"heap_tomb_pages":0,"total_allocated_pages":2459,"total_freed_pages":0,"total_allocated_objects":4337014,"total_freed_objects":3689721,"malloc_increase_bytes":6448248,"malloc_increase_bytes_limit":29188387,"minor_gc_count":36,"major_gc_count":10,"remembered_wb_unprotected_objects":19958,"remembered_wb_unprotected_objects_limit":39842,"old_objects":462019,"old_objects_limit":895782,"oldmalloc_increase_bytes":6448696,"oldmalloc_increase_bytes_limit":19350882}
2017-04-21 11:37:03 -04:00
Sam 8ec7fd84fd FEATURE: prioritize sidekiq jobs
This commit introduces 3 queues for sidekiq

"critical" for urgent jobs (weighted at 4x weight)
"default" for standard jobs(weighted at 2x weight)
"low" for less important jobs


"critical jobs"

Reset Password emails has been seperated to its own job
Heartbeat which is required to keep sidekiq running
Test email which needs to return real quick


"low priority jobs"

Notify mailing list
Pull hotlinked images
Update gravatar

"default"

All the rest

Note: for people running sidekiq from command line use

bin/sidekiq -q critical,4 -q default,2 -q low
2016-04-07 12:56:43 +10:00
Sam 65edbb609c Revert "Revert message bus upgrade"
This reverts commit 47e718f5b2.
2015-12-09 11:48:41 +11:00
Sam 47e718f5b2 Revert message bus upgrade 2015-12-09 11:45:11 +11:00
Sam 2cc95af69b Revert "REVERT: message bus changes"
This reverts commit 4820d5c7b0.
2015-12-09 07:36:36 +11:00
Robin Ward 4820d5c7b0 REVERT: message bus changes 2015-12-08 15:32:31 -05:00
Sam a3ba564b03 missing spot where initializer was renamed 2015-12-08 07:13:29 +11:00
Sam 0e4883a8ae correct regression where monitoring thread crashed out
add logging
2015-06-16 11:16:33 +10:00
Sam 861cd5d9b0 FIX: ensure child demon is correctly terminated from parent on stop 2015-06-15 12:36:47 +10:00
Sam 1721872084 cleanup out-of-memory detection and correction code 2015-03-27 15:44:52 +11:00
Sam 58f3fcbc1a BUGFIX: not terminating self correctly on hangups from parent 2014-06-13 11:15:40 +10:00
Sam 95159fb82a BUGFIX: Sidekiq could be initialized incorrectly in some cases
Symptom, no jobs run
2014-06-03 17:17:10 +10:00
Sam dc06401479 PERF: reduce sidekiq worker count to 5 2014-05-14 10:21:11 +10:00
Sam 7c57d74e85 FEATURE: unicorn sidekiq will restart sidekiq on complete failure.
(checks every 30 minutes for complete failure)
2014-04-23 13:13:18 +10:00
Sam ead7c52a06 Refactor demonizer in prep for unicorn forking
Upgrade sidekiq
2014-04-17 15:58:00 +10:00
Sam f3cc7360e0 BUGFIX: Correct after_fork semantics
After fork SiteSettings was not getting a new process id,
causing site settings not to refresh properly in unicorn

This code also centralizes the logic
2014-03-31 12:34:13 +11:00
Régis Hanol b56b11d96a add qunit to autospec 2013-11-01 23:57:50 +01:00
Sam 89f801ac04 fix no sidetiq when using demon 2013-10-24 15:58:28 +11:00
Sam 28a0cb494a rails 4 upgrade
rack lock is trouble, nuke it out of orbit
more aggressive suicide for forked sidekiq
2013-10-10 14:23:24 +11:00
Sam c4bab8915c fix initialization issues with unicorn
amend unicorn script to demonize sidekiq
create a sidekiq demon that unicorn consumes
correct bug in exec_sql with empty params
2013-10-10 14:23:24 +11:00