Unicorn uses the USR1 to indicate that log files should be reopened. This commit implements the same functionality for our forked sidekiq workers:
- USR1 is intercepted in the unicorn master, and re-issued to all child processes
- USR1 is trapped in the sidekiq processes, and `Unicorn::Util.reopen_logs` is used to re-open log files
We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)
This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes
1. We attempt to localize a string on each site, ensuring we warmup
the i18n
2. We preload all our templates (compiling .erb to class)
3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
This makes sure that all processes that fork off the master have a fully
operation schema cache.
In Rails 6, schema cache is now bolted to the connection pool. This change
ensures the cache on all pools is fully populated prior to forking.
The bolting of cache to connection pool does lead to some strange cases
where a connection can "steal" the cache from another connection, which
can cause stuff to possibly hang or deadlock. This change minimizes the risk
of this happening cause it is already primed.
We make a STRONG assumption that the schema is always the same on all sites
when we spin up a multisite cluster.
This reverts commit e805d44965.
We now have mechanisms in place to ensure heartbeat will always
be scheduled even if the scheduler is overloaded per: 098f938b
* FIX: Heartbeat check per sidekiq process
* Rename method
* Remove heartbeat queues of previous bootups
* Regis feedback
* Refactor before_start
* Update lib/demon/sidekiq.rb
Co-Authored-By: Régis Hanol <regis@hanol.fr>
* Update lib/demon/sidekiq.rb
Co-Authored-By: Régis Hanol <regis@hanol.fr>
* Expire redis keys after 3600 seconds
* Don't use redis to store the list of queues
v8 forking is not supported and can lead to memory leaks.
This commit handles the most common case which is the unicorn master forking
There are still some cases related to backup where we fork, however those
forks are usually short lived so the memory leak is not severe, burning
the contexts in the master process could break sidekiq or web process that
do the actual forking
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.
Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
I get this error if I stop a dev server, ``rm -rf tmp`` and start it again:
```
`mkdir': No such file or directory @ dir_s_mkdir - /Users/angusmcleod/discourse/discourse/tmp/pids (Errno::ENOENT)
```
This fixes it.
See: f3549291a3 (diff-26ac62db6c6a4582de3bbf2615790c23R22)
This commit also cleans up a bunch of pointless noise each time we boot app
- narrative was loading i18n cause redefinition of consts
- discourse.rb was loaded twice as was auth
- bin/unicorn now does all the smart things and boots unicron in dev
- bin/rails s will boot unicorn with no params
- remove bin/puma which only causes confusion
Out of the box this signal "suspends" the process, but
we already use usr1 and usr2 and this is for an edge case
where the end user suspends it by typing "stop"
Dan Ungureanu
Guo Xiang Tan
David Taylor
Jarek Radosz
Joffrey JAFFEUX
Jeff Wong
Sam Saffron
OsamaSayegh
Guo Xiang Tan
Angus McLeod
Robin Ward
Régis Hanol