595ec8c948
Today Elasticsearch limits the number of processors used in computing thread counts to 32. This was from a time when Elasticsearch created more threads than it does now and users would run into out of memory errors. It appears the real cause of these out of memory errors was not well understood (it's often due to ulimit settings) and so users were left hitting these out of memory errors on boxes with high core counts. Today Elasticsearch creates less threads (but still a lot) and we have a bootstrap check in place to ensure that the relevant ulimit is not too low. There are some caveats still to having too many concurrent indexing threads as it can lead to too many little segments, and it's not a magical go faster knob if indexing is already bottlenecked by disk, but this limitation is artificial and surprising to users and so it should be removed. This commit also increases the lower bound of the max processes ulimit, to prepare for a world where Elasticsearch instances might be running with more the previous cap of 32 processors. With the current settings, Elasticsearch wants to create roughly 576 + 25 * p / 2 threads, where p is the number of processors. Add in roughly 7 * p / 8 threads for the GC threads and a fudge factor, and 4096 should cover us pretty well up to 256 cores. Relates #20874 |
||
|---|---|---|
| .. | ||
| licenses | ||
| src | ||
| build.gradle | ||