mirror of https://github.com/apache/lucene.git
SOLR-10509, SOLR-10649: add docs for new metric features; add <metrics> to solr.xml docs
This commit is contained in:
Cassandra Targett
parent
a477ebb985
commit
9e99a23f31
|
|
@ -22,7 +22,6 @@ The `solr.xml` file defines some global configuration options that apply to all
|
|||
|
||||
This section will describe the default `solr.xml` file included with Solr and how to modify it for your needs. For details on how to configure `core.properties`, see the section <<defining-core-properties.adoc#defining-core-properties,Defining core.properties>>.
|
||||
|
||||
[[Formatofsolr.xml-Definingsolr.xml]]
|
||||
== Defining solr.xml
|
||||
|
||||
You can find `solr.xml` in your `$SOLR_HOME` directory (usually `server/solr`) in standalone mode or in Zookeeper when using SolrCloud. The default `solr.xml` file looks like this:
|
||||
|
|
@ -50,10 +49,9 @@ You can find `solr.xml` in your `$SOLR_HOME` directory (usually `server/solr`) i
|
|||
|
||||
As you can see, the discovery Solr configuration is "SolrCloud friendly". However, the presence of the `<solrcloud>` element does _not_ mean that the Solr instance is running in SolrCloud mode. Unless the `-DzkHost` or `-DzkRun` are specified at startup time, this section is ignored.
|
||||
|
||||
[[Formatofsolr.xml-Solr.xmlParameters]]
|
||||
=== Solr.xml Parameters
|
||||
== Solr.xml Parameters
|
||||
|
||||
==== The `<solr>` Element
|
||||
=== The <solr> Element
|
||||
|
||||
There are no attributes that you can specify in the `<solr>` tag, which is the root element of `solr.xml`. The tables below list the child nodes of each XML element in `solr.xml`.
|
||||
|
||||
|
|
@ -74,7 +72,7 @@ There are no attributes that you can specify in the `<solr>` tag, which is the r
|
|||
|`configSetBaseDir` |The directory under which configsets for Solr cores can be found. Defaults to `$SOLR_HOME/configsets`.
|
||||
|===
|
||||
|
||||
==== The `<solrcloud>` Element
|
||||
=== The <solrcloud> Element
|
||||
|
||||
This element defines several parameters that relate so SolrCloud. This section is ignored unless theSolr instance is started with either `-DzkRun` or `-DzkHost`
|
||||
|
||||
|
|
@ -96,7 +94,7 @@ This element defines several parameters that relate so SolrCloud. This section i
|
|||
|`zkCredentialsProvider` & ` zkACLProvider` |Optional parameters that can be specified if you are using <<zookeeper-access-control.adoc#zookeeper-access-control,ZooKeeper Access Control>>.
|
||||
|===
|
||||
|
||||
==== The `<logging>` Element
|
||||
=== The <logging> Element
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
|
|
@ -107,7 +105,7 @@ This element defines several parameters that relate so SolrCloud. This section i
|
|||
|`enabled` |true/false - whether to enable logging or not.
|
||||
|===
|
||||
|
||||
===== The `<logging><watcher>` Element
|
||||
==== The <logging><watcher> Element
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
|
|
@ -118,7 +116,7 @@ This element defines several parameters that relate so SolrCloud. This section i
|
|||
|`threshold` |The logging level above which your particular logging implementation will record. For example when using log4j one might specify DEBUG, WARN, INFO, etc.
|
||||
|===
|
||||
|
||||
==== The `<shardHandlerFactory>` Element
|
||||
=== The <shardHandlerFactory> Element
|
||||
|
||||
Custom shard handlers can be defined in `solr.xml` if you wish to create a custom shard handler.
|
||||
|
||||
|
|
@ -146,14 +144,19 @@ Since this is a custom shard handler, sub-elements are specific to the implement
|
|||
|`fairnessPolicy` |A boolean to configure if the threadpool favours fairness over throughput. Default is false to favor throughput.
|
||||
|===
|
||||
|
||||
[[Formatofsolr.xml-SubstitutingJVMSystemPropertiesinsolr.xml]]
|
||||
=== The <metrics> Element
|
||||
|
||||
The `<metrics>` element in `solr.xml` allows you to customize the metrics reported by Solr. You can define system properties that should not be returned, or define custom suppliers and reporters.
|
||||
|
||||
In a default `solr.xml` you will not see any `<metrics>` configuration. If you would like to customize the metrics for your installation, see the section <<metrics-reporting.adoc#metrics-configuration,Metrics Configuration>>.
|
||||
|
||||
== Substituting JVM System Properties in solr.xml
|
||||
|
||||
Solr supports variable substitution of JVM system property values in `solr.xml`, which allows runtime specification of various configuration options. The syntax is `${propertyname[:option default value]`}. This allows defining a default that can be overridden when Solr is launched. If a default value is not specified, then the property must be specified at runtime or the `solr.xml` file will generate an error when parsed.
|
||||
Solr supports variable substitution of JVM system property values in `solr.xml`, which allows runtime specification of various configuration options. The syntax is `${propertyname[:option default value]}`. This allows defining a default that can be overridden when Solr is launched. If a default value is not specified, then the property must be specified at runtime or the `solr.xml` file will generate an error when parsed.
|
||||
|
||||
Any JVM system properties usually specified using the -D flag when starting the JVM, can be used as variables in the `solr.xml` file.
|
||||
Any JVM system properties usually specified using the `-D` flag when starting the JVM, can be used as variables in the `solr.xml` file.
|
||||
|
||||
For example, in the `solr.xml` file shown below, the `socketTimeout` and `connTimeout` values are each set to "0". However, if you start Solr using '`bin/solr -DsocketTimeout=1000`', the `socketTimeout` option of the `HttpShardHandlerFactory` to be overridden using a value of 1000ms, while the `connTimeout` option will continue to use the default property value of "0".
|
||||
For example, in the `solr.xml` file shown below, the `socketTimeout` and `connTimeout` values are each set to "0". However, if you start Solr using `bin/solr -DsocketTimeout=1000`, the `socketTimeout` option of the `HttpShardHandlerFactory` to be overridden using a value of 1000ms, while the `connTimeout` option will continue to use the default property value of "0".
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@ Internally this feature uses the http://metrics.dropwizard.io[Dropwizard Metrics
|
|||
* *timers* - measure the number and duration of events. They provide a count and histogram of timings.
|
||||
* *gauges* - offer instantaneous reading of a current value, e.g., current queue depth, current number of active connections, free heap size.
|
||||
|
||||
Each group of related metrics with unique names is managed in a *metric registry*. Solr maintains several such registries, each corresponding to a high-level group such as: `jvm`, `jetty`, `http`, `node`, and `core` (see <<Metric Registries>> below).
|
||||
Each group of related metrics with unique names is managed in a *metric registry*. Solr maintains several such registries, each corresponding to a high-level group such as: `jvm`, `jetty`, `node`, and `core` (see <<Metric Registries>> below).
|
||||
|
||||
For each group (and/or for each registry) there can be several *reporters*, which are components responsible for communication of metrics from selected registries to external systems. Currently implemented reporters support emitting metrics via JMX, Ganglia, Graphite and SLF4J.
|
||||
|
||||
|
|
@ -42,7 +42,9 @@ Metrics are maintained and accumulated through all lifecycles of components from
|
|||
|
||||
These are the major groups of metrics that are collected:
|
||||
|
||||
=== JVM Registry (`solr.jvm`):
|
||||
=== JVM Registry
|
||||
|
||||
This registry is returned at `solr.jvm` and includes the following information. When making requests with the <<Metrics API>>, you can specify `&group=jvm` to limit to only these metrics.
|
||||
|
||||
* direct and mapped buffer pools
|
||||
* class loading / unloading
|
||||
|
|
@ -50,26 +52,28 @@ These are the major groups of metrics that are collected:
|
|||
* GC count and time
|
||||
* heap, non-heap memory and GC pools
|
||||
* number of threads, their states and deadlocks
|
||||
* System properties such as Java information, various installation directory paths, ports, and similar information. You can control what appears here by modifying `solr.xml`.
|
||||
// TODO for 7.0 fix this
|
||||
|
||||
=== Node / CoreContainer Registry (`solr.node`):
|
||||
=== Node / CoreContainer Registry
|
||||
|
||||
This registry is returned at `solr.node` and includes the following information. When making requests with the <<Metrics API>>, you can specify `&group=node` to limit to only these metrics.
|
||||
|
||||
* handler requests (count, timing): collections, info, admin, configSets, etc.
|
||||
* number of cores (loaded, lazy, unloaded)
|
||||
|
||||
=== Core (SolrCore) Registry:
|
||||
=== Core (SolrCore) Registry
|
||||
|
||||
The <<Core Level Metrics,Core (SolrCore) Registry>> includes `solr.core.<collection>`, one for each core.
|
||||
The <<Core Level Metrics,Core (SolrCore) Registry>> includes `solr.core.<collection>`, one for each core. When making requests with the <<Metrics API>>, you can specify `&group=core` to limit to only these metrics.
|
||||
|
||||
* all common RequestHandler-s report: request timers / counters, timeouts, errors.
|
||||
* <<Index Merge Metrics,index-level events>>: meters for minor / major merges, number of merged docs, number of deleted docs, gauges for currently running merges and their size.
|
||||
* shard replication and transaction log replay on replicas (TBD, SOLR-9856)
|
||||
* TBD: caches, update handler details, and other relevant SolrInfoMBean-s
|
||||
|
||||
=== HTTP Registry (`solr.http`):
|
||||
|
||||
* open / available / pending connections for shard handler and update handler
|
||||
|
||||
=== Jetty Registry (`solr.jetty`):
|
||||
=== Jetty Registry
|
||||
|
||||
This registry is returned at `solr.jetty` and includes the following information. When making requests with the <<Metrics API>>, you can specify `&group=jetty` to limit to only these metrics.
|
||||
|
||||
* threads and pools,
|
||||
* connection and request timers,
|
||||
|
|
@ -77,6 +81,69 @@ The <<Core Level Metrics,Core (SolrCore) Registry>> includes `solr.core.<collect
|
|||
|
||||
In the future, metrics will be added for shard leaders and cluster nodes, including aggregations from per-core metrics.
|
||||
|
||||
== Metrics Configuration
|
||||
|
||||
The metrics available in your system can be customized by modifying the `<metrics>` element in `solr.xml`.
|
||||
|
||||
TIP: See also the section <<format-of-solr-xml.adoc#format-of-solr-xml,Format of Solr.xml>> for more information about the `solr.xml` file, where to find it, and how to edit it.
|
||||
|
||||
=== The <metrics><hiddenSysProps> Element
|
||||
|
||||
This section of `solr.xml` allows you to define the system properties which are considered system-sensitive and should not be exposed via the Metrics API.
|
||||
|
||||
If this section is not defined, the following default configuration is used which hides password and authentication information:
|
||||
|
||||
[source,xml]
|
||||
<metrics>
|
||||
<hiddenSysProps>
|
||||
<str>javax.net.ssl.keyStorePassword</str>
|
||||
<str>javax.net.ssl.trustStorePassword</str>
|
||||
<str>basicauth</str>
|
||||
<str>zkDigestPassword</str>
|
||||
<str>zkDigestReadonlyPassword</str>
|
||||
</hiddenSysProps>
|
||||
</metrics>
|
||||
|
||||
=== The <metrics><reporters> Element
|
||||
|
||||
Reporters consume the metrics data generated by Solr. See the section <<Reporters>> below for more details on how to configure custom reporters.
|
||||
|
||||
=== The <metrics><suppliers> Element
|
||||
|
||||
Suppliers help Solr generate metrics data. The `<metrics><suppliers>` section of `solr.xml` allows you to define your own implementations of metrics and configure parameters for them.
|
||||
|
||||
Implementation of a custom metrics supplier is beyond the scope of this guide, but there are other customizations possible with the default implementation, via the elements described below.
|
||||
|
||||
<counter>:: This element defines the implementation and configuration of a `Counter` supplier. The default implementation does not support any configuration.
|
||||
|
||||
<meter>:: This element defines the implementation of a `Meter` supplier. The default implementation supports an additional parameter:
|
||||
`<str name="clock">`::: The type of clock to use for calculating EWMA rates. The supported values are:
|
||||
* `user`, the default, which uses `System.nanoTime()`
|
||||
* `cpu`, which uses the current thread's CPU time
|
||||
|
||||
<histogram>:: This element defines the implementation of a `Histogram` supplier. This element also supports the `clock` parameter shown above with the `meter` element, and also:
|
||||
`<str name="reservoir">`::: The fully-qualified class name of the `Reservoir` implementation to use. The default is `com.codahale.metrics.ExponentiallyDecayingReservoir` but there are other options available with the http://metrics.dropwizard.io/3.1.0/manual/core/#histograms[Codahale Metrics library] that Solr uses. The following parameters are supported, within the mentioned limitations:
|
||||
* `size`, the reservoir size. The default is 1028.
|
||||
* `alpha`, the decay parameter. The default is 0.015. This is only valid for the `ExponentiallyDecayingReservoir`.
|
||||
* `window`, the window size, in seconds, and only valid for the `SlidingTimeWindowReservoir`. The default is 300 (5 minutes).
|
||||
|
||||
<timer>:: This element defines an implementation of a `Timer` supplier. The default implementation supports the `clock` and `reservoir` parameters described above.
|
||||
|
||||
As an example of a section of `solr.xml` that defines some of these custom parameters, the following defines the default `Meter` supplier with a non-default `clock` and the default `Timer` is used with a non-default reservoir:
|
||||
|
||||
[source,xml]
|
||||
<metrics>
|
||||
<suppliers>
|
||||
<meter>
|
||||
<str name="clock">cpu</str>
|
||||
</meter>
|
||||
<timer>
|
||||
<str name="reservoir">com.codahale.metrics.SlidingTimeWindowReservoir</str>
|
||||
<long name="window">600</long>
|
||||
</timer>
|
||||
</suppliers>
|
||||
</metrics>
|
||||
|
||||
== Reporters
|
||||
|
||||
Reporter configurations are specified in `solr.xml` file in `<metrics><reporter>` sections, for example:
|
||||
|
|
@ -101,6 +168,8 @@ Reporter configurations are specified in `solr.xml` file in `<metrics><reporter>
|
|||
</solr>
|
||||
----
|
||||
|
||||
This example configures two reporters: <<Graphite Reporter,Graphite>> and <<SLF4J Reporter,SLF4J>>. See below for more details on how to configure reporters.
|
||||
|
||||
=== Reporter Arguments
|
||||
|
||||
Reporter plugins use the following arguments:
|
||||
|
|
@ -260,15 +329,17 @@ If the boolean flag `mergeDetails` is true then the following additional metrics
|
|||
|
||||
The `admin/metrics` endpoint provides access to all the metrics for all metric groups.
|
||||
|
||||
A few query parameters are available to limit the request:
|
||||
A few query parameters are available to limit your request to only certain metrics:
|
||||
|
||||
group:: The metric group to retrieve. The default is `all` to retrieve all metrics for all groups. Other possible values are: `jvm`, `jetty`, `node`, and `core`. More than one group can be specified in a request; multiple group names should be separated by a comma.
|
||||
|
||||
type:: The type of metric to retrieve. The default is `all` to retrieve all metric types. Other possible values are `counter`, `gauge`, `histogram`, `meter`, and `timer`. More than one type can be specified in a request; multiple types should be separated by a comma.
|
||||
|
||||
prefix:: The first characters of metric name that will filter the metrics returned to those starting with the provided string. It can be combined with group and/or type parameters. More than one prefix can be specified in a request; multiple prefixes should be separated by a comma. Prefix matching is also case-sensitive.
|
||||
prefix:: The first characters of metric name that will filter the metrics returned to those starting with the provided string. It can be combined with `group` and/or `type` parameters. More than one prefix can be specified in a request; multiple prefixes should be separated by a comma. Prefix matching is also case-sensitive.
|
||||
|
||||
compact:: When true, a more compact format of the response will be returned. Instead of a response like this:
|
||||
property:: Allows requesting only this metric from any compound metric. Multiple `property` parameters can be combined to act as an OR request. For example, to only get the 99th and 999th percentile values from all metric types and groups, you can add `&property=p99_ms&property=p999_ms` to your request. This can be combined with `group`, `type`, and `prefix` as necessary.
|
||||
|
||||
compact:: When false, a more verbose format of the response will be returned. Instead of a response like this:
|
||||
+
|
||||
[source,json]
|
||||
"metrics": [
|
||||
|
|
|
|||
Loading…
Reference in New Issue