OpenSearch/docs/reference/search/aggregations/reducer/derivative-aggregation.asciidoc

[[search-aggregations-reducer-derivative-aggregation]]
=== Derivative Aggregation

A parent reducer aggregation which calculates the derivative of a specified metric in a parent histogram (or date_histogram) 
aggregation. The specified metric must be numeric and the enclosing histogram must have `min_doc_count` set to `0` (default
for `histogram` aggregations).

The following snippet calculates the derivative of the total monthly `sales`:

[source,js]
--------------------------------------------------
{
    "aggs" : {
        "sales_per_month" : {
            "date_histogram" : {
                "field" : "date",
                "interval" : "month"
            },
            "aggs": {
                "sales": {
                    "sum": {
                        "field": "price"
                    }
                },
                "sales_deriv": {
                    "derivative": {
                        "buckets_paths": "sales" <1>
                    }
                }
            }
        }
    }
}
--------------------------------------------------

<1> `bucket_paths` instructs this derivative aggregation to use the output of the `sales` aggregation for the derivative

And the following may be the response:

[source,js]
--------------------------------------------------
{
   "aggregations": {
      "sales_per_month": {
         "buckets": [
            {
               "key_as_string": "2015/01/01 00:00:00",
               "key": 1420070400000,
               "doc_count": 3,
               "sales": {
                  "value": 550
               } <1>
            },
            {
               "key_as_string": "2015/02/01 00:00:00",
               "key": 1422748800000,
               "doc_count": 2,
               "sales": {
                  "value": 60
               },
               "sales_deriv": {
                  "value": -490 <2>
               }
            },
            {
               "key_as_string": "2015/03/01 00:00:00",
               "key": 1425168000000,
               "doc_count": 2, <3>
               "sales": {
                  "value": 375
               },
               "sales_deriv": {
                  "value": 315
               }
            }
         ]
      }
   }
}
--------------------------------------------------

<1> No derivative for the first bucket since we need at least 2 data points to calculate the derivative
<2> Derivative value units are implicitly defined by the `sales` aggregation and the parent histogram so in this case the units 
would be $/month assuming the `price` field has units of $.
<3> The number of documents in the bucket are represented by the `doc_count` value

==== Second Order Derivative

A second order derivative can be calculated by chaining the derivative reducer aggregation onto the result of another derivative 
reducer aggregation as in the following example which will calculate both the first and the second order derivative of the total 
monthly sales:

[source,js]
--------------------------------------------------
{
    "aggs" : {
        "sales_per_month" : {
            "date_histogram" : {
                "field" : "date",
                "interval" : "month"
            },
            "aggs": {
                "sales": {
                    "sum": {
                        "field": "price"
                    }
                },
                "sales_deriv": {
                    "derivative": {
                        "buckets_paths": "sales"
                    }
                },
                "sales_2nd_deriv": {
                    "derivative": {
                        "buckets_paths": "sales_deriv" <1>
                    }
                }
            }
        }
    }
}
--------------------------------------------------

<1> `bucket_paths` for the second derivative points to the name of the first derivative

And the following may be the response:

[source,js]
--------------------------------------------------
{
   "aggregations": {
      "sales_per_month": {
         "buckets": [
            {
               "key_as_string": "2015/01/01 00:00:00",
               "key": 1420070400000,
               "doc_count": 3,
               "sales": {
                  "value": 550
               } <1>
            },
            {
               "key_as_string": "2015/02/01 00:00:00",
               "key": 1422748800000,
               "doc_count": 2,
               "sales": {
                  "value": 60
               },
               "sales_deriv": {
                  "value": -490
               } <1>
            },
            {
               "key_as_string": "2015/03/01 00:00:00",
               "key": 1425168000000,
               "doc_count": 2,
               "sales": {
                  "value": 375
               },
               "sales_deriv": {
                  "value": 315
               },
               "sales_2nd_deriv": {
                  "value": 805
               }
            }
         ]
      }
   }
}
--------------------------------------------------
<1> No second derivative for the first two buckets since we need at least 2 data points from the first derivative to calculate the 
second derivative

==== Dealing with gaps in the data

There are a couple of reasons why the data output by the enclosing histogram may have gaps:

* There are no documents matching the query for some buckets
* The data for a metric is missing in all of the documents falling into a bucket (this is most likely with either a small interval 
on the enclosing histogram or with a query matching only a small number of documents)

Where there is no data available in a bucket for a given metric it presents a problem for calculating the derivative value for both 
the current bucket and the next bucket. In the derivative reducer aggregation has a `gap_policy` parameter to define what the behavior
should be when a gap in the data is found. There are currently two options for controlling the gap policy:

_ignore_::
				This option will not produce a derivative value for any buckets where the value in the current or previous bucket is 
				missing

_insert_zeros_::
				This option will assume the missing value is `0` and calculate the derivative with the value `0`.