Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

SOLR-13771: Add -v and -m to ulimit section of reference guide and bin/solr checks

This commit is contained in:
Erick Erickson 2019-09-30 17:58:35 -04:00
parent 0ec7986fc0
commit a1f3d2c29a
2 changed files with 32 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
solr/bin/solr
View File

@ -1521,6 +1521,8 @@ if [ -z "$SOLR_ULIMIT_CHECKS" ] || [ "$SOLR_ULIMIT_CHECKS" != "false" ]; then
if hash ulimit 2>/dev/null; then
openFiles=$(ulimit -n)
maxProcs=$(ulimit -u)
virtualMemory=$(ulimit -v)
maxMemory=$(ulimit -m)
if [ $openFiles != "unlimited" ] && [ $openFiles -lt "$SOLR_RECOMMENDED_OPEN_FILES" ]; then
echo "*** [WARN] *** Your open file limit is currently $openFiles. "
echo " It should be set to $SOLR_RECOMMENDED_OPEN_FILES to avoid operational disruption. "
@ -1532,10 +1534,23 @@ if [ -z "$SOLR_ULIMIT_CHECKS" ] || [ "$SOLR_ULIMIT_CHECKS" != "false" ]; then
echo " It should be set to $SOLR_RECOMMENDED_MAX_PROCESSES to avoid operational disruption. "
echo " If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh"
fi
if [ $virtualMemory != "unlimited" ]; then
echo "*** [WARN] *** Your Virtual Memory limit is $virtualMemory. "
echo " It should be set to 'unlimited' to avoid operational disruption. "
echo " If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh"
fi
if [ $maxMemory != "unlimited" ]; then
echo "*** [WARN] *** Your Max Memory Size limit is $maxMemory. "
echo " It should be set to 'unlimited' to avoid operational disruption. "
echo " If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh"
fi
else
echo "Could not check ulimits for processes and open files, recommended values are"
echo " max processes: $SOLR_RECOMMENDED_MAX_PROCESSES "
echo " open files: $SOLR_RECOMMENDED_OPEN_FILES"
echo " max processes: $SOLR_RECOMMENDED_MAX_PROCESSES "
echo " open files: $SOLR_RECOMMENDED_OPEN_FILES"
echo " virtual memory: unlimited"
echo " max memorh size: unlimited"
fi
fi
fi

23
solr/solr-ref-guide/src/taking-solr-to-production.adoc
View File

@ -273,26 +273,33 @@ The `bin/solr` script simply passes options starting with `-D` on to the JVM dur
SOLR_OPTS="$SOLR_OPTS -Dsolr.autoSoftCommit.maxTime=10000"
----
=== File Handles and Processes (ulimit settings)
=== Ulimit settings (*nix operating systems)
Two common settings that result in errors on *nix systems are file handles and user processes.
It is common for the default limits for number of processes and file handles to default to values that are too low for a large Solr installation. The required number of each of these will increase based on a combination of the number of replicas hosted per node and the number of segments in the index for each replica.
The usual recommendation is to make processes and file handles at least 65,000 each, unlimited if possible. On most *nix systems, this command will show the currently-defined limits:
There are several settings that should be monitored and set as high as possible, "unlimited" by preference. On most "*nix" operating systems, you can see the current values by typing the following at a command prompt.
[source,bash]
----
ulimit -a
----
It is strongly recommended that file handle and process limits be permanently raised as above. The exact form of the command will vary per operating system, and some systems require editing configuration files and restarting your server. Consult your system administrators for guidance in your particular environment.
These four settings in particular are important to have set very high, unlimited by preference.
* max processes (ulimit -u): 65,000 is the recommended _minimum_
* file handles (ulimit -n): 65,000 is the recommended _minimum_. All the files used by all replicas have their file handles open at once so this can grow quite large.
* virtual memory (ulimit -v): Set to unlimited. This is used to by MMapping the indexes.
* max memory size (ulimit -m): Also used by MMap, set to unlimited.
* If your system supports it, `sysctl vm.max_map_count`, should be set to unlimited as well.
We strongly recommend that these settings be permanently raised. The exact process to permanently raise them will vary per operating system. Some systems require editing configuration files and restarting your server. Consult your system administrators for guidance in your particular environment.
[WARNING]
====
Check these limits every time you upgrade your kernel or operating system. These operations can reset these to their defaults.
====
[WARNING]
====
If these limits are exceeded, the problems reported by Solr vary depending on the specific operation responsible for exceeding the limit. Errors such as "too many open files", "connection error", and "max processes exceeded" have been reported, as well as SolrCloud recovery failures.
Since exceeding these limits can result in such varied symptoms it is _strongly_ recommended that these limits be permanently raised as recommended above.
====
== Running Multiple Solr Nodes per Host