285 lines
8.2 KiB
Plaintext
285 lines
8.2 KiB
Plaintext
[[search-aggregations-pipeline-derivative-aggregation]]
|
|
=== Derivative Aggregation
|
|
|
|
experimental[]
|
|
|
|
A parent pipeline aggregation which calculates the derivative of a specified metric in a parent histogram (or date_histogram)
|
|
aggregation. The specified metric must be numeric and the enclosing histogram must have `min_doc_count` set to `0` (default
|
|
for `histogram` aggregations).
|
|
|
|
==== Syntax
|
|
|
|
A `derivative` aggregation looks like this in isolation:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"derivative": {
|
|
"buckets_path": "the_sum"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
.`derivative` Parameters
|
|
|===
|
|
|Parameter Name |Description |Required |Default Value
|
|
|`buckets_path` |The path to the buckets we wish to find the derivative for (see <<buckets-path-syntax>> for more
|
|
details) |Required |
|
|
|`gap_policy` |The policy to apply when gaps are found in the data (see <<gap-policy>> for more
|
|
details)|Optional, defaults to `skip` |
|
|
|`format` |format to apply to the output value of this aggregation |Optional, defaults to `null` |
|
|
|===
|
|
|
|
|
|
==== First Order Derivative
|
|
|
|
The following snippet calculates the derivative of the total monthly `sales`:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"sales_per_month" : {
|
|
"date_histogram" : {
|
|
"field" : "date",
|
|
"interval" : "month"
|
|
},
|
|
"aggs": {
|
|
"sales": {
|
|
"sum": {
|
|
"field": "price"
|
|
}
|
|
},
|
|
"sales_deriv": {
|
|
"derivative": {
|
|
"buckets_path": "sales" <1>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> `buckets_path` instructs this derivative aggregation to use the output of the `sales` aggregation for the derivative
|
|
|
|
And the following may be the response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggregations": {
|
|
"sales_per_month": {
|
|
"buckets": [
|
|
{
|
|
"key_as_string": "2015/01/01 00:00:00",
|
|
"key": 1420070400000,
|
|
"doc_count": 3,
|
|
"sales": {
|
|
"value": 550
|
|
} <1>
|
|
},
|
|
{
|
|
"key_as_string": "2015/02/01 00:00:00",
|
|
"key": 1422748800000,
|
|
"doc_count": 2,
|
|
"sales": {
|
|
"value": 60
|
|
},
|
|
"sales_deriv": {
|
|
"value": -490 <2>
|
|
}
|
|
},
|
|
{
|
|
"key_as_string": "2015/03/01 00:00:00",
|
|
"key": 1425168000000,
|
|
"doc_count": 2, <3>
|
|
"sales": {
|
|
"value": 375
|
|
},
|
|
"sales_deriv": {
|
|
"value": 315
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> No derivative for the first bucket since we need at least 2 data points to calculate the derivative
|
|
<2> Derivative value units are implicitly defined by the `sales` aggregation and the parent histogram so in this case the units
|
|
would be $/month assuming the `price` field has units of $.
|
|
<3> The number of documents in the bucket are represented by the `doc_count` f
|
|
|
|
==== Second Order Derivative
|
|
|
|
A second order derivative can be calculated by chaining the derivative pipeline aggregation onto the result of another derivative
|
|
pipeline aggregation as in the following example which will calculate both the first and the second order derivative of the total
|
|
monthly sales:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"sales_per_month" : {
|
|
"date_histogram" : {
|
|
"field" : "date",
|
|
"interval" : "month"
|
|
},
|
|
"aggs": {
|
|
"sales": {
|
|
"sum": {
|
|
"field": "price"
|
|
}
|
|
},
|
|
"sales_deriv": {
|
|
"derivative": {
|
|
"buckets_path": "sales"
|
|
}
|
|
},
|
|
"sales_2nd_deriv": {
|
|
"derivative": {
|
|
"buckets_path": "sales_deriv" <1>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> `buckets_path` for the second derivative points to the name of the first derivative
|
|
|
|
And the following may be the response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggregations": {
|
|
"sales_per_month": {
|
|
"buckets": [
|
|
{
|
|
"key_as_string": "2015/01/01 00:00:00",
|
|
"key": 1420070400000,
|
|
"doc_count": 3,
|
|
"sales": {
|
|
"value": 550
|
|
} <1>
|
|
},
|
|
{
|
|
"key_as_string": "2015/02/01 00:00:00",
|
|
"key": 1422748800000,
|
|
"doc_count": 2,
|
|
"sales": {
|
|
"value": 60
|
|
},
|
|
"sales_deriv": {
|
|
"value": -490
|
|
} <1>
|
|
},
|
|
{
|
|
"key_as_string": "2015/03/01 00:00:00",
|
|
"key": 1425168000000,
|
|
"doc_count": 2,
|
|
"sales": {
|
|
"value": 375
|
|
},
|
|
"sales_deriv": {
|
|
"value": 315
|
|
},
|
|
"sales_2nd_deriv": {
|
|
"value": 805
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
<1> No second derivative for the first two buckets since we need at least 2 data points from the first derivative to calculate the
|
|
second derivative
|
|
|
|
==== Units
|
|
|
|
The derivative aggregation allows the units of the derivative values to be specified. This returns an extra field in the response
|
|
`normalized_value` which reports the derivative value in the desired x-axis units. In the below example we calculate the derivative
|
|
of the total sales per month but ask for the derivative of the sales as in the units of sales per day:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggs" : {
|
|
"sales_per_month" : {
|
|
"date_histogram" : {
|
|
"field" : "date",
|
|
"interval" : "month"
|
|
},
|
|
"aggs": {
|
|
"sales": {
|
|
"sum": {
|
|
"field": "price"
|
|
}
|
|
},
|
|
"sales_deriv": {
|
|
"derivative": {
|
|
"buckets_path": "sales",
|
|
"unit": "day" <1>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> `unit` specifies what unit to use for the x-axis of the derivative calculation
|
|
|
|
And the following may be the response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"aggregations": {
|
|
"sales_per_month": {
|
|
"buckets": [
|
|
{
|
|
"key_as_string": "2015/01/01 00:00:00",
|
|
"key": 1420070400000,
|
|
"doc_count": 3,
|
|
"sales": {
|
|
"value": 550
|
|
} <1>
|
|
},
|
|
{
|
|
"key_as_string": "2015/02/01 00:00:00",
|
|
"key": 1422748800000,
|
|
"doc_count": 2,
|
|
"sales": {
|
|
"value": 60
|
|
},
|
|
"sales_deriv": {
|
|
"value": -490, <1>
|
|
"normalized_value": -17.5 <2>
|
|
}
|
|
},
|
|
{
|
|
"key_as_string": "2015/03/01 00:00:00",
|
|
"key": 1425168000000,
|
|
"doc_count": 2,
|
|
"sales": {
|
|
"value": 375
|
|
},
|
|
"sales_deriv": {
|
|
"value": 315,
|
|
"normalized_value": 10.16129032258065
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
<1> `value` is reported in the original units of 'per month'
|
|
<2> `normalized_value` is reported in the desired units of 'per day'
|