374 lines
12 KiB
Plaintext
374 lines
12 KiB
Plaintext
[[search-aggregations-bucket-histogram-aggregation]]
|
|
=== Histogram
|
|
|
|
A multi-bucket values source based aggregation that can be applied on numeric values extracted from the documents.
|
|
It dynamically builds fixed size (a.k.a. interval) buckets over the values. For example, if the documents have a field
|
|
that holds a price (numeric), we can configure this aggregation to dynamically build buckets with interval `5`
|
|
(in case of price it may represent $5). When the aggregation executes, the price field of every document will be
|
|
evaluated and will be rounded down to its closest bucket - for example, if the price is `32` and the bucket size is `5`
|
|
then the rounding will yield `30` and thus the document will "fall" into the bucket that is associated withe the key `30`.
|
|
To make this more formal, here is the rounding function that is used:
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
rem = value % interval
|
|
if (rem < 0) {
|
|
rem += interval
|
|
}
|
|
bucket_key = value - rem
|
|
--------------------------------------------------
|
|
|
|
The following snippet "buckets" the products based on their `price` by interval of `50`:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
And the following may be the response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggregations": {
|
|
"prices" : {
|
|
"buckets": [
|
|
{
|
|
"key": 0,
|
|
"doc_count": 2
|
|
},
|
|
{
|
|
"key": 50,
|
|
"doc_count": 4
|
|
},
|
|
{
|
|
"key": 150,
|
|
"doc_count": 3
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
The response above shows that none of the aggregated products has a price that falls within the range of `[100 - 150)`.
|
|
By default, the response will only contain those buckets with a `doc_count` greater than 0. It is possible change that
|
|
and request buckets with either a higher minimum count or even 0 (in which case elasticsearch will "fill in the gaps"
|
|
and create buckets with zero documents). This can be configured using the `min_doc_count` setting:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"min_doc_count" : 0
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
Response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggregations": {
|
|
"prices" : {
|
|
"buckets": [
|
|
{
|
|
"key": 0,
|
|
"doc_count": 2
|
|
},
|
|
{
|
|
"key": 50,
|
|
"doc_count": 4
|
|
},
|
|
{
|
|
"key" : 100,
|
|
"doc_count" : 0 <1>
|
|
},
|
|
{
|
|
"key": 150,
|
|
"doc_count": 3
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> No documents were found that belong in this bucket, yet it is still returned with zero `doc_count`.
|
|
|
|
[[search-aggregations-bucket-histogram-aggregation-extended-bounds]]
|
|
By default the date_/histogram returns all the buckets within the range of the data itself, that is, the documents with
|
|
the smallest values (on which with histogram) will determine the min bucket (the bucket with the smallest key) and the
|
|
documents with the highest values will determine the max bucket (the bucket with the highest key). Often, when when
|
|
requesting empty buckets (`"min_doc_count" : 0`), this causes a confusion, specifically, when the data is also filtered.
|
|
|
|
To understand why, let's look at an example:
|
|
|
|
Lets say the you're filtering your request to get all docs with values between `0` and `500`, in addition you'd like
|
|
to slice the data per price using a histogram with an interval of `50`. You also specify `"min_doc_count" : 0` as you'd
|
|
like to get all buckets even the empty ones. If it happens that all products (documents) have prices higher than `100`,
|
|
the first bucket you'll get will be the one with `100` as its key. This is confusing, as many times, you'd also like
|
|
to get those buckets between `0 - 100`.
|
|
|
|
With `extended_bounds` setting, you now can "force" the histogram aggregation to start building buckets on a specific
|
|
`min` values and also keep on building buckets up to a `max` value (even if there are no documents anymore). Using
|
|
`extended_bounds` only makes sense when `min_doc_count` is 0 (the empty buckets will never be returned if `min_doc_count`
|
|
is greater than 0).
|
|
|
|
Note that (as the name suggest) `extended_bounds` is **not** filtering buckets. Meaning, if the `extended_bounds.min` is higher
|
|
than the values extracted from the documents, the documents will still dictate what the first bucket will be (and the
|
|
same goes for the `extended_bounds.max` and the last bucket). For filtering buckets, one should nest the histogram aggregation
|
|
under a range `filter` aggregation with the appropriate `from`/`to` settings.
|
|
|
|
Example:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"query" : {
|
|
"filtered" : { "range" : { "price" : { "to" : "500" } } }
|
|
},
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"min_doc_count" : 0,
|
|
"extended_bounds" : {
|
|
"min" : 0,
|
|
"max" : 500
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
==== Order
|
|
|
|
By default the returned buckets are sorted by their `key` ascending, though the order behaviour can be controled
|
|
using the `order` setting.
|
|
|
|
Ordering the buckets by their key - descending:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"order" : { "_key" : "desc" }
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
Ordering the buckets by their `doc_count` - ascending:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"order" : { "_count" : "asc" }
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
If the histogram aggregation has a direct metrics sub-aggregation, the latter can determine the order of the buckets:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"order" : { "price_stats.min" : "asc" } <1>
|
|
},
|
|
"aggs" : {
|
|
"price_stats" : { "stats" : {} } <2>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> The `{ "price_stats.min" : asc" }` will sort the buckets based on `min` value of their their `price_stats` sub-aggregation.
|
|
|
|
<2> There is no need to configure the `price` field for the `price_stats` aggregation as it will inherit it by default from its parent histogram aggregation.
|
|
|
|
It is also possible to order the buckets based on a "deeper" aggregation in the hierarchy. This is supported as long
|
|
as the aggregations path are of a single-bucket type, where the last aggregation in the path may either by a single-bucket
|
|
one or a metrics one. If it's a single-bucket type, the order will be defined by the number of docs in the bucket (i.e. `doc_count`),
|
|
in case it's a metrics one, the same rules as above apply (where the path must indicate the metric name to sort by in case of
|
|
a multi-value metrics aggregation, and in case of a single-value metrics aggregation the sort will be applied on that value).
|
|
|
|
The path must be defined in the following form:
|
|
|
|
--------------------------------------------------
|
|
AGG_SEPARATOR := '>'
|
|
METRIC_SEPARATOR := '.'
|
|
AGG_NAME := <the name of the aggregation>
|
|
METRIC := <the name of the metric (in case of multi-value metrics aggregation)>
|
|
PATH := <AGG_NAME>[<AGG_SEPARATOR><AGG_NAME>]*[<METRIC_SEPARATOR><METRIC>]
|
|
--------------------------------------------------
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"order" : { "promoted_products>rating_stats.avg" : "desc" } <1>
|
|
},
|
|
"aggs" : {
|
|
"promoted_products" : {
|
|
"filter" : { "term" : { "promoted" : true }},
|
|
"aggs" : {
|
|
"rating_stats" : { "stats" : { "field" : "rating" }}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
The above will sort the buckets based on the avg rating among the promoted products
|
|
|
|
|
|
==== Minimum document count
|
|
|
|
It is possible to only return buckets that have a document count that is greater than or equal to a configured
|
|
limit through the `min_doc_count` option.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"min_doc_count": 10
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
The above aggregation would only return buckets that contain 10 documents or more. Default value is `1`.
|
|
|
|
NOTE: The special value `0` can be used to add empty buckets to the response between the minimum and the maximum buckets.
|
|
Here is an example of what the response could look like:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggregations": {
|
|
"prices": {
|
|
"buckets": {
|
|
"0": {
|
|
"key": 0,
|
|
"doc_count": 2
|
|
},
|
|
"50": {
|
|
"key": 50,
|
|
"doc_count": 0
|
|
},
|
|
"150": {
|
|
"key": 150,
|
|
"doc_count": 3
|
|
},
|
|
"200": {
|
|
"key": 150,
|
|
"doc_count": 0
|
|
},
|
|
"250": {
|
|
"key": 150,
|
|
"doc_count": 0
|
|
},
|
|
"300": {
|
|
"key": 150,
|
|
"doc_count": 1
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
==== Response Format
|
|
|
|
By default, the buckets are returned as an ordered array. It is also possible to request the response as a hash
|
|
instead keyed by the buckets keys:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"prices" : {
|
|
"histogram" : {
|
|
"field" : "price",
|
|
"interval" : 50,
|
|
"keyed" : true
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
Response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggregations": {
|
|
"prices": {
|
|
"buckets": {
|
|
"0": {
|
|
"key": 0,
|
|
"doc_count": 2
|
|
},
|
|
"50": {
|
|
"key": 50,
|
|
"doc_count": 4
|
|
},
|
|
"150": {
|
|
"key": 150,
|
|
"doc_count": 3
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|