mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-05-23 00:55:06 +00:00
32e23b9100
Most aggregations (terms, histogram, stats, percentiles, geohash-grid) now support a new `missing` option which defines the value to consider when a field does not have a value. This can be handy if you eg. want a terms aggregation to handle the same way documents that have "N/A" or no value for a `tag` field. This works in a very similar way to the `missing` option on the `sort` element. One known issue is that this option sometimes cannot make the right decision in the unmapped case: it needs to replace all values with the `missing` value but might not know what kind of values source should be produced (numerics, strings, geo points?). For this reason, we might want to add an `unmapped_type` option in the future like we did for sorting. Related to #5324
111 lines
3.5 KiB
Plaintext
111 lines
3.5 KiB
Plaintext
[[search-aggregations-metrics-percentile-rank-aggregation]]
|
|
=== Percentile Ranks Aggregation
|
|
|
|
A `multi-value` metrics aggregation that calculates one or more percentile ranks
|
|
over numeric values extracted from the aggregated documents. These values
|
|
can be extracted either from specific numeric fields in the documents, or
|
|
be generated by a provided script.
|
|
|
|
[NOTE]
|
|
==================================================
|
|
Please see <<search-aggregations-metrics-percentile-aggregation-approximation>>
|
|
and <<search-aggregations-metrics-percentile-aggregation-compression>> for advice
|
|
regarding approximation and memory use of the percentile ranks aggregation
|
|
==================================================
|
|
|
|
Percentile rank show the percentage of observed values which are below certain
|
|
value. For example, if a value is greater than or equal to 95% of the observed values
|
|
it is said to be at the 95th percentile rank.
|
|
|
|
Assume your data consists of website load times. You may have a service agreement that
|
|
95% of page loads completely within 15ms and 99% of page loads complete within 30ms.
|
|
|
|
Let's look at a range of percentiles representing load time:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"load_time_outlier" : {
|
|
"percentile_ranks" : {
|
|
"field" : "load_time", <1>
|
|
"values" : [15, 30]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
<1> The field `load_time` must be a numeric field
|
|
|
|
The response will look like this:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
...
|
|
|
|
"aggregations": {
|
|
"load_time_outlier": {
|
|
"values" : {
|
|
"15": 92,
|
|
"30": 100
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
From this information you can determine you are hitting the 99% load time target but not quite
|
|
hitting the 95% load time target
|
|
|
|
|
|
==== Script
|
|
|
|
The percentile rank metric supports scripting. For example, if our load times
|
|
are in milliseconds but we want to specify values in seconds, we could use
|
|
a script to convert them on-the-fly:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"load_time_outlier" : {
|
|
"percentile_ranks" : {
|
|
"values" : [3, 5],
|
|
"script" : "doc['load_time'].value / timeUnit", <1>
|
|
"params" : {
|
|
"timeUnit" : 1000 <2>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
<1> The `field` parameter is replaced with a `script` parameter, which uses the
|
|
script to generate values which percentile ranks are calculated on
|
|
<2> Scripting supports parameterized input just like any other script
|
|
|
|
TIP: The `script` parameter expects an inline script. Use `script_id` for indexed scripts and `script_file` for scripts in the `config/scripts/` directory.
|
|
|
|
==== Missing value
|
|
|
|
The `missing` parameter defines how documents that are missing a value should be treated.
|
|
By default they will be ignored but it is also possible to treat them as if they
|
|
had a value.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"grade_ranks" : {
|
|
"percentile_ranks" : {
|
|
"field" : "grade",
|
|
"missing": 10 <1>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> Documents without a value in the `grade` field will fall into the same bucket as documents that have the value `10`.
|