Commit Graph

41 Commits

Author SHA1 Message Date
Sam Saffron 76173dea87 DEV: ensure we never fork v8 contexts from unicorn
v8 forking is not supported and can lead to memory leaks.

This commit handles the most common case which is the unicorn master forking

There are still some cases related to backup where we fork, however those
forks are usually short lived so the memory leak is not severe, burning
the contexts in the master process could break sidekiq or web process that
do the actual forking
2019-05-16 09:50:34 +10:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan 4774633dac DEV: Remove `StatsSocket`.
Removed in favor of https://github.com/discourse/discourse-prometheus.
2019-03-26 18:16:58 +08:00
Guo Xiang Tan edbcc992d4 Allow unicorn timeout to be configurable via ENV. 2018-09-04 13:21:41 +08:00
Sam 1172e141cd adjust timeouts in dev 2018-08-15 11:13:43 +10:00
Angus McLeod 6c41b54b2e FIX: create tmp if it doesn't exist when creating tmp/pids
I get this error if I stop a dev server, ``rm -rf tmp`` and start it again:
```
`mkdir': No such file or directory @ dir_s_mkdir - /Users/angusmcleod/discourse/discourse/tmp/pids (Errno::ENOENT)
```
This fixes it.

See: f3549291a3 (diff-26ac62db6c6a4582de3bbf2615790c23R22)
2018-08-08 14:49:09 +10:00
Sam f3549291a3 DEV: use unicorn in development
This commit also cleans up a bunch of pointless noise each time we boot app

- narrative was loading i18n cause redefinition of consts
- discourse.rb was loaded twice as was auth
- bin/unicorn now does all the smart things and boots unicron in dev
- bin/rails s will boot unicorn with no params
- remove bin/puma which only causes confusion
2018-08-07 17:13:47 +10:00
Guo Xiang Tan 805fd17b23 ActiveRecord in Rails 5.2 discards connection pools after fork. 2018-06-12 09:30:52 +08:00
Guo Xiang Tan 01f126e38f Simplify unicorn config. 2018-06-07 09:03:16 +08:00
Guo Xiang Tan a4e6662833 FIX: Disconnects all connections in the pool before forking.
* We were leaking connections as a result. Connections opened
  before the fork were never closed.
2018-06-06 14:45:05 +08:00
Guo Xiang Tan 45fe5dc793 `$redis.client` -> `$redis._client`.
See c239abb43c
2018-04-20 13:01:17 +08:00
Guo Xiang Tan db68434db1 More control over Unicorn logstash logger. 2017-11-15 12:28:36 +08:00
Guo Xiang Tan 89d9ffa884 EXPERIMENTAL: Allow logs to be shipped via different methods.
See https://github.com/dwbutler/logstash-logger#uri-configuration for
a list of available methods.
2017-11-15 09:11:33 +08:00
Guo Xiang Tan c9df21e131 FEATURE: Allow Unicorn logs to be JSON formatted. 2017-11-06 12:46:14 +08:00
Sam 220a961898 correct unicorn sidekiq messaging 2017-10-23 17:12:25 +11:00
Sam 15932f391e FEATURE: Add events when web and sidekiq forks start 2017-10-23 15:40:45 +11:00
Guo Xiang Tan 91dfe23d6f Exclude versions table when loading models in Unicorn. 2017-10-12 17:14:36 +08:00
Sam a4d4db4f0c PERF: code not correctly caching git commands
Every check for Discourse version could result in shelling out.
2017-10-04 14:22:38 +11:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan 87e77267fa Add ENV variable to specify path to write unicorn's pid. 2017-06-06 11:46:01 +09:00
Sam 20778fbf58 fix handling of stats socket 2017-04-21 12:37:28 -04:00
Sam 0b3aec9c94 FEATURE: set UNICORN_STATS_SOCKET_DIR for status socket
eg:

sam@ubuntu stats_sockets % socat - UNIX-CONNECT:9622.sock
gc_stat
{"count":46,"heap_allocated_pages":2459,"heap_sorted_length":2460,"heap_allocatable_pages":0,"heap_available_slots":1002267,"heap_live_slots":647293,"heap_free_slots":354974,"heap_final_slots":0,"heap_marked_slots":503494,"heap_swept_slots":498773,"heap_eden_pages":2459,"heap_tomb_pages":0,"total_allocated_pages":2459,"total_freed_pages":0,"total_allocated_objects":4337014,"total_freed_objects":3689721,"malloc_increase_bytes":6448248,"malloc_increase_bytes_limit":29188387,"minor_gc_count":36,"major_gc_count":10,"remembered_wb_unprotected_objects":19958,"remembered_wb_unprotected_objects_limit":39842,"old_objects":462019,"old_objects_limit":895782,"oldmalloc_increase_bytes":6448696,"oldmalloc_increase_bytes_limit":19350882}
2017-04-21 11:37:03 -04:00
Sam f9f38873a2 FEATURE: add support for SIGTSTP which stops sidekiqs
Out of the box this signal "suspends" the process, but
we already use usr1 and usr2 and this is for an edge case
where the end user suspends it by typing "stop"
2017-03-10 11:36:10 -05:00
Guo Xiang Tan f69f225f65 FIX: Don't prevent unicorn worker from starting if warmup fails. 2016-09-06 14:02:08 +08:00
Sam Saffron 6777bd2629 warm up v8 after fork 2016-07-16 15:11:34 +10:00
Robin Ward 2d1d9a78b8 Revert "Try warming up pretty text before we fork"
This reverts commit 2e71e6fc6f.
2016-07-15 18:10:22 -04:00
Robin Ward 2e71e6fc6f Try warming up pretty text before we fork 2016-07-13 17:07:39 -04:00
Sam a3885a18f8 extra logic to force kill orphan sidekiqs 2015-06-18 15:32:04 +10:00
Régis Hanol 0a41ec59e8 fix substitution - :fired: 2015-05-12 19:43:25 +02:00
Régis Hanol 8c57aef3f8 log the hostname when restarting sidekiq 2015-05-12 19:39:17 +02:00
Sam 1721872084 cleanup out-of-memory detection and correction code 2015-03-27 15:44:52 +11:00
Sam 251ebb0d65 disable memory check till we confirm all is good 2015-03-27 14:51:22 +11:00
Sam 43eb030402 PERF: restart sidekiq if it consumes more than 500MB
configurable with UNICORN_SIDEKIQ_MAX_RSS
2015-03-27 14:27:01 +11:00
Sam 7c57d74e85 FEATURE: unicorn sidekiq will restart sidekiq on complete failure.
(checks every 30 minutes for complete failure)
2014-04-23 13:13:18 +10:00
Sam 4aaedb82d0 FEATURE: alllow unicorn master to fork sidekiqs, keep alive coming 2014-04-23 11:03:36 +10:00
Sam f3cc7360e0 BUGFIX: Correct after_fork semantics
After fork SiteSettings was not getting a new process id,
causing site settings not to refresh properly in unicorn

This code also centralizes the logic
2014-03-31 12:34:13 +11:00
Sam c0d947aa98 allow bench to run with unicorn optionally
memstats can output yaml now
2014-02-16 16:44:51 +11:00
Sam bcc9d749ad load up routes in master 2013-11-20 12:36:19 +11:00
Sam 461972844e unicorn out of band GC 2013-11-15 12:15:49 +11:00
Sam 0e8914cee6 add unicorn launcher to support live reloads
add oobgc rudimentary support
2013-11-13 15:29:36 +11:00
Sam ab6efe8323 I tire of sample files, make number of workers configurable 2013-10-30 16:34:25 +11:00