honeymoose/OpenSearch
1
0
Fork 0
mirror of https://github.com/honeymoose/OpenSearch.git synced 2025-03-24 17:09:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add // CONSOLE to much of pipeline agg docs

Browse Source 
Most of the examples in the pipeline aggregation docs use a small
"sales" test data set and I converted all of the examples that use
it to `// CONSOLE`. There are still a bunch of snippets in the pipeline
aggregation docs that aren't `// CONSOLE` so they aren't tested. Most
of them are "this is the most basic form of this aggregation" so they
are more immune to errors and bit rot then the examples that I converted.
I'd like to do something with them as well but I'm not sure what.

Also, the moving average docs and serial diff docs didn't get a lot of
love from this pass because they don't use the test data set or follow
the same general layout.

Relates to #18160
This commit is contained in:
Nik Everett 2016-08-12 18:42:19 -04:00
parent 7da9d826ff
commit c66db9a81e
15 changed files with 369 additions and 179 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
docs/build.gradle
View File

@ -105,3 +105,38 @@ buildRestTests.setups['host'] = '''
- is_true: nodes.$master.http.publish_address
- set: {nodes.$master.http.publish_address: host}
'''
// Used by pipeline aggregation docs
buildRestTests.setups['sales'] = '''
- do:
indices.create:
index: sales
body:
settings:
number_of_shards: 2
number_of_replicas: 1
mappings:
sale:
properties:
type:
type: keyword
- do:
bulk:
index: sales
type: sale
refresh: true
body: |
{"index":{}}
{"date": "2015/01/01 00:00:00", "price": 200, "type": "hat"}
{"index":{}}
{"date": "2015/01/01 00:00:00", "price": 200, "type": "t-shirt"}
{"index":{}}
{"date": "2015/01/01 00:00:00", "price": 150, "type": "bag"}
{"index":{}}
{"date": "2015/02/01 00:00:00", "price": 50, "type": "hat"}
{"index":{}}
{"date": "2015/02/01 00:00:00", "price": 10, "type": "t-shirt"}
{"index":{}}
{"date": "2015/03/01 00:00:00", "price": 200, "type": "hat"}
{"index":{}}
{"date": "2015/03/01 00:00:00", "price": 175, "type": "t-shirt"}'''

50
docs/reference/aggregations/pipeline.asciidoc
View File

@ -51,23 +51,27 @@ metric `"the_sum"`:
[source,js]
--------------------------------------------------
POST /_search
{
"my_date_histo":{
"date_histogram":{
"field":"timestamp",
"interval":"day"
},
"aggs":{
"the_sum":{
"sum":{ "field": "lemmings" } <1>
"aggs": {
"my_date_histo":{
"date_histogram":{
"field":"timestamp",
"interval":"day"
},
"the_movavg":{
"moving_avg":{ "buckets_path": "the_sum" } <2>
"aggs":{
"the_sum":{
"sum":{ "field": "lemmings" } <1>
},
"the_movavg":{
"moving_avg":{ "buckets_path": "the_sum" } <2>
}
}
}
}
}
--------------------------------------------------
// CONSOLE
<1> The metric is called `"the_sum"`
<2> The `buckets_path` refers to the metric via a relative path `"the_sum"`
@ -77,6 +81,7 @@ a metric embedded inside a sibling aggregation:
[source,js]
--------------------------------------------------
POST /_search
{
"aggs" : {
"sales_per_month" : {
@ -100,6 +105,8 @@ a metric embedded inside a sibling aggregation:
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this max_bucket aggregation that we want the maximum value of the `sales` aggregation in the
`sales_per_month` date histogram.
@ -111,20 +118,24 @@ the pipeline aggregation to use the document count as it's input. For example,
[source,js]
--------------------------------------------------
POST /_search
{
"my_date_histo":{
"date_histogram":{
"field":"timestamp",
"interval":"day"
},
"aggs":{
"the_movavg":{
"moving_avg":{ "buckets_path": "_count" } <1>
"aggs": {
"my_date_histo": {
"date_histogram": {
"field":"timestamp",
"interval":"day"
},
"aggs": {
"the_movavg": {
"moving_avg": { "buckets_path": "_count" } <1>
}
}
}
}
}
--------------------------------------------------
// CONSOLE
<1> By using `_count` instead of a metric name, we can calculate the moving average of document counts in the histogram
The `buckets_path` can also use `"_bucket_count"` and path to a multi-bucket aggregation to use the number of buckets
@ -133,6 +144,7 @@ used here to filter out buckets which contain no buckets for an inner terms aggr
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs": {
@ -162,6 +174,8 @@ used here to filter out buckets which contain no buckets for an inner terms aggr
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> By using `_bucket_count` instead of a metric name, we can filter out `histo` buckets where they contain no buckets
for the `categories` aggregation

59
docs/reference/aggregations/pipeline/avg-bucket-aggregation.asciidoc
View File

@ -3,7 +3,7 @@
experimental[]
A sibling pipeline aggregation which calculates the (mean) average value of a specified metric in a sibling aggregation.
A sibling pipeline aggregation which calculates the (mean) average value of a specified metric in a sibling aggregation.
The specified metric must be numeric and the sibling aggregation must be a multi-bucket aggregation.
==== Syntax
@ -33,30 +33,35 @@ The following snippet calculates the average of the total monthly `sales`:
[source,js]
--------------------------------------------------
POST /_search
{
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
"field" : "date",
"interval" : "month"
},
"aggs": {
"sales": {
"sum": {
"field": "price"
}
}
}
},
"avg_monthly_sales": {
"avg_bucket": {
"buckets_path": "sales_per_month>sales" <1>
}
"size": 0,
"aggs": {
"sales_per_month": {
"date_histogram": {
"field": "date",
"interval": "month"
},
"aggs": {
"sales": {
"sum": {
"field": "price"
}
}
}
},
"avg_monthly_sales": {
"avg_bucket": {
"buckets_path": "sales_per_month>sales" <1>
}
}
}
}
--------------------------------------------------
<1> `buckets_path` instructs this avg_bucket aggregation that we want the (mean) average value of the `sales` aggregation in the
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this avg_bucket aggregation that we want the (mean) average value of the `sales` aggregation in the
`sales_per_month` date histogram.
And the following may be the response:
@ -64,6 +69,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -72,7 +81,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
}
},
{
@ -80,7 +89,7 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
}
},
{
@ -88,7 +97,7 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
}
}
]
@ -99,4 +108,6 @@ And the following may be the response:
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]

44
docs/reference/aggregations/pipeline/bucket-script-aggregation.asciidoc
View File

@ -3,7 +3,7 @@
experimental[]
A parent pipeline aggregation which executes a script which can perform per bucket computations on specified metrics
A parent pipeline aggregation which executes a script which can perform per bucket computations on specified metrics
in the parent multi-bucket aggregation. The specified metric must be numeric and the script must return a numeric value.
==== Syntax
@ -22,16 +22,16 @@ A `bucket_script` aggregation looks like this in isolation:
}
}
--------------------------------------------------
<1> Here, `my_var1` is the name of the variable for this buckets path to use in the script, `the_sum` is the path to
<1> Here, `my_var1` is the name of the variable for this buckets path to use in the script, `the_sum` is the path to
the metrics to use for that variable.
.`bucket_script` Parameters
|===
|Parameter Name |Description |Required |Default Value
|`script` |The script to run for this aggregation. The script can be inline, file or indexed. (see <<modules-scripting>>
|`script` |The script to run for this aggregation. The script can be inline, file or indexed. (see <<modules-scripting>>
for more details) |Required |
|`buckets_path` |A map of script variables and their associated path to the buckets we wish to use for the variable
|`buckets_path` |A map of script variables and their associated path to the buckets we wish to use for the variable
(see <<buckets-path-syntax>> for more details) |Required |
|`gap_policy` |The policy to apply when gaps are found in the data (see <<gap-policy>> for more
details)|Optional, defaults to `skip` |
@ -42,7 +42,9 @@ The following snippet calculates the ratio percentage of t-shirt sales compared
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -71,7 +73,7 @@ The following snippet calculates the ratio percentage of t-shirt sales compared
},
"t-shirt-percentage": {
"bucket_script": {
"buckets_path": {
"buckets_path": {
"tShirtSales": "t-shirts>sales",
"totalSales": "total_sales"
},
@ -83,12 +85,18 @@ The following snippet calculates the ratio percentage of t-shirt sales compared
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -97,33 +105,33 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"total_sales": {
"value": 50
"value": 550.0
},
"t-shirts": {
"doc_count": 2,
"doc_count": 1,
"sales": {
"value": 10
"value": 200.0
}
},
"t-shirt-percentage": {
"value": 20
"value": 36.36363636363637
}
},
{
"key_as_string": "2015/02/01 00:00:00",
"key": 1422748800000,
"doc_count": 2
"doc_count": 2,
"total_sales": {
"value": 60
"value": 60.0
},
"t-shirts": {
"doc_count": 1,
"sales": {
"value": 15
"value": 10.0
}
},
"t-shirt-percentage": {
"value": 25
"value": 16.666666666666664
}
},
{
@ -131,16 +139,16 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"total_sales": {
"value": 40
"value": 375.0
},
"t-shirts": {
"doc_count": 1,
"sales": {
"value": 20
"value": 175.0
}
},
"t-shirt-percentage": {
"value": 50
"value": 46.666666666666664
}
}
]
@ -148,4 +156,6 @@ And the following may be the response:
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]

35
docs/reference/aggregations/pipeline/bucket-selector-aggregation.asciidoc
View File

@ -3,12 +3,12 @@
experimental[]
A parent pipeline aggregation which executes a script which determines whether the current bucket will be retained
in the parent multi-bucket aggregation. The specified metric must be numeric and the script must return a boolean value.
If the script language is `expression` then a numeric return value is permitted. In this case 0.0 will be evaluated as `false`
A parent pipeline aggregation which executes a script which determines whether the current bucket will be retained
in the parent multi-bucket aggregation. The specified metric must be numeric and the script must return a boolean value.
If the script language is `expression` then a numeric return value is permitted. In this case 0.0 will be evaluated as `false`
and all other values will evaluate to true.
Note: The bucket_selector aggregation, like all pipeline aggregations, executions after all other sibling aggregations. This means that
Note: The bucket_selector aggregation, like all pipeline aggregations, executions after all other sibling aggregations. This means that
using the bucket_selector aggregation to filter the returned buckets in the response does not save on execution time running the aggregations.
==== Syntax
@ -27,26 +27,28 @@ A `bucket_selector` aggregation looks like this in isolation:
}
}
--------------------------------------------------
<1> Here, `my_var1` is the name of the variable for this buckets path to use in the script, `the_sum` is the path to
<1> Here, `my_var1` is the name of the variable for this buckets path to use in the script, `the_sum` is the path to
the metrics to use for that variable.
.`bucket_selector` Parameters
|===
|Parameter Name |Description |Required |Default Value
|`script` |The script to run for this aggregation. The script can be inline, file or indexed. (see <<modules-scripting>>
|`script` |The script to run for this aggregation. The script can be inline, file or indexed. (see <<modules-scripting>>
for more details) |Required |
|`buckets_path` |A map of script variables and their associated path to the buckets we wish to use for the variable
|`buckets_path` |A map of script variables and their associated path to the buckets we wish to use for the variable
(see <<buckets-path-syntax>> for more details) |Required |
|`gap_policy` |The policy to apply when gaps are found in the data (see <<gap-policy>> for more
details)|Optional, defaults to `skip` |
|===
The following snippet only retains buckets where the total sales for the month is less than or equal to 50:
The following snippet only retains buckets where the total sales for the month is more than 400:
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -64,7 +66,7 @@ The following snippet only retains buckets where the total sales for the month i
"buckets_path": {
"totalSales": "total_sales"
},
"script": "totalSales <= 50"
"script": "totalSales > 200"
}
}
}
@ -72,12 +74,18 @@ The following snippet only retains buckets where the total sales for the month i
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -86,7 +94,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"total_sales": {
"value": 50
"value": 550.0
}
},<1>
{
@ -94,7 +102,7 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"total_sales": {
"value": 40
"value": 375.0
},
}
]
@ -102,4 +110,7 @@ And the following may be the response:
}
}
--------------------------------------------------
<1> Bucket for `2015/02/01 00:00:00` has been removed as its total sales exceeded 50
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
<1> Bucket for `2015/02/01 00:00:00` has been removed as its total sales was less than 200

25
docs/reference/aggregations/pipeline/cumulative-sum-aggregation.asciidoc
View File

@ -3,7 +3,7 @@
experimental[]
A parent pipeline aggregation which calculates the cumulative sum of a specified metric in a parent histogram (or date_histogram)
A parent pipeline aggregation which calculates the cumulative sum of a specified metric in a parent histogram (or date_histogram)
aggregation. The specified metric must be numeric and the enclosing histogram must have `min_doc_count` set to `0` (default
for `histogram` aggregations).
@ -32,7 +32,9 @@ The following snippet calculates the cumulative sum of the total monthly `sales`
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -55,6 +57,8 @@ The following snippet calculates the cumulative sum of the total monthly `sales`
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this cumulative sum aggregation to use the output of the `sales` aggregation for the cumulative sum
@ -63,6 +67,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -71,10 +79,10 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
},
"cumulative_sales": {
"value": 550
"value": 550.0
}
},
{
@ -82,10 +90,10 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
},
"cumulative_sales": {
"value": 610
"value": 610.0
}
},
{
@ -93,10 +101,10 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
},
"cumulative_sales": {
"value": 985
"value": 985.0
}
}
]
@ -104,3 +112,6 @@ And the following may be the response:
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]

91
docs/reference/aggregations/pipeline/derivative-aggregation.asciidoc
View File

@ -3,7 +3,7 @@
experimental[]
A parent pipeline aggregation which calculates the derivative of a specified metric in a parent histogram (or date_histogram)
A parent pipeline aggregation which calculates the derivative of a specified metric in a parent histogram (or date_histogram)
aggregation. The specified metric must be numeric and the enclosing histogram must have `min_doc_count` set to `0` (default
for `histogram` aggregations).
@ -13,10 +13,8 @@ A `derivative` aggregation looks like this in isolation:
[source,js]
--------------------------------------------------
{
"derivative": {
"buckets_path": "the_sum"
}
"derivative": {
"buckets_path": "the_sum"
}
--------------------------------------------------
@ -37,7 +35,9 @@ The following snippet calculates the derivative of the total monthly `sales`:
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -60,6 +60,8 @@ The following snippet calculates the derivative of the total monthly `sales`:
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this derivative aggregation to use the output of the `sales` aggregation for the derivative
@ -68,6 +70,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -76,7 +82,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
} <1>
},
{
@ -84,10 +90,10 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
},
"sales_deriv": {
"value": -490 <2>
"value": -490.0 <2>
}
},
{
@ -95,10 +101,10 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2, <3>
"sales": {
"value": 375
"value": 375.0
},
"sales_deriv": {
"value": 315
"value": 315.0
}
}
]
@ -106,21 +112,26 @@ And the following may be the response:
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
<1> No derivative for the first bucket since we need at least 2 data points to calculate the derivative
<2> Derivative value units are implicitly defined by the `sales` aggregation and the parent histogram so in this case the units
<2> Derivative value units are implicitly defined by the `sales` aggregation and the parent histogram so in this case the units
would be $/month assuming the `price` field has units of $.
<3> The number of documents in the bucket are represented by the `doc_count` f
==== Second Order Derivative
A second order derivative can be calculated by chaining the derivative pipeline aggregation onto the result of another derivative
pipeline aggregation as in the following example which will calculate both the first and the second order derivative of the total
A second order derivative can be calculated by chaining the derivative pipeline aggregation onto the result of another derivative
pipeline aggregation as in the following example which will calculate both the first and the second order derivative of the total
monthly sales:
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -148,6 +159,8 @@ monthly sales:
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` for the second derivative points to the name of the first derivative
@ -156,6 +169,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 50,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -164,7 +181,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
} <1>
},
{
@ -172,10 +189,10 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
},
"sales_deriv": {
"value": -490
"value": -490.0
} <1>
},
{
@ -183,13 +200,13 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
},
"sales_deriv": {
"value": 315
"value": 315.0
},
"sales_2nd_deriv": {
"value": 805
"value": 805.0
}
}
]
@ -197,18 +214,24 @@ And the following may be the response:
}
}
--------------------------------------------------
<1> No second derivative for the first two buckets since we need at least 2 data points from the first derivative to calculate the
// TESTRESPONSE[s/"took": 50/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
<1> No second derivative for the first two buckets since we need at least 2 data points from the first derivative to calculate the
second derivative
==== Units
The derivative aggregation allows the units of the derivative values to be specified. This returns an extra field in the response
`normalized_value` which reports the derivative value in the desired x-axis units. In the below example we calculate the derivative
The derivative aggregation allows the units of the derivative values to be specified. This returns an extra field in the response
`normalized_value` which reports the derivative value in the desired x-axis units. In the below example we calculate the derivative
of the total sales per month but ask for the derivative of the sales as in the units of sales per day:
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -232,7 +255,8 @@ of the total sales per month but ask for the derivative of the sales as in the u
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `unit` specifies what unit to use for the x-axis of the derivative calculation
And the following may be the response:
@ -240,6 +264,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 50,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -248,7 +276,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
} <1>
},
{
@ -256,11 +284,11 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
},
"sales_deriv": {
"value": -490, <1>
"normalized_value": -17.5 <2>
"value": -490.0, <1>
"normalized_value": -15.806451612903226 <2>
}
},
{
@ -268,11 +296,11 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
},
"sales_deriv": {
"value": 315,
"normalized_value": 10.16129032258065
"value": 315.0,
"normalized_value": 11.25
}
}
]
@ -280,5 +308,8 @@ And the following may be the response:
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 50/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
<1> `value` is reported in the original units of 'per month'
<2> `normalized_value` is reported in the desired units of 'per day'

39
docs/reference/aggregations/pipeline/extended-stats-bucket-aggregation.asciidoc
View File

@ -36,7 +36,9 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -53,12 +55,15 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
},
"stats_monthly_sales": {
"extended_stats_bucket": {
"buckets_paths": "sales_per_month>sales" <1>
"buckets_path": "sales_per_month>sales" <1>
}
}
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `bucket_paths` instructs this `extended_stats_bucket` aggregation that we want the calculate stats for the `sales` aggregation in the
`sales_per_month` date histogram.
@ -67,6 +72,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -75,7 +84,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
}
},
{
@ -83,7 +92,7 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
}
},
{
@ -91,26 +100,28 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
}
}
]
},
"stats_monthly_sales": {
"count": 3,
"min": 60,
"max": 550,
"avg": 328.333333333,
"sum": 985,
"sum_of_squares": 446725,
"variance": 41105.5555556,
"std_deviation": 117.054909559,
"min": 60.0,
"max": 550.0,
"avg": 328.3333333333333,
"sum": 985.0,
"sum_of_squares": 446725.0,
"variance": 41105.55555555556,
"std_deviation": 202.74505063146563,
"std_deviation_bounds": {
"upper": 562.443152451,
"lower": 94.2235142151
"upper": 733.8234345962646,
"lower": -77.15676792959795
}
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]

25
docs/reference/aggregations/pipeline/max-bucket-aggregation.asciidoc
View File

@ -4,7 +4,7 @@
experimental[]
A sibling pipeline aggregation which identifies the bucket(s) with the maximum value of a specified metric in a sibling aggregation
and outputs both the value and the key(s) of the bucket(s). The specified metric must be numeric and the sibling aggregation must
and outputs both the value and the key(s) of the bucket(s). The specified metric must be numeric and the sibling aggregation must
be a multi-bucket aggregation.
==== Syntax
@ -34,7 +34,9 @@ The following snippet calculates the maximum of the total monthly `sales`:
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -57,7 +59,10 @@ The following snippet calculates the maximum of the total monthly `sales`:
}
}
--------------------------------------------------
<1> `buckets_path` instructs this max_bucket aggregation that we want the maximum value of the `sales` aggregation in the
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this max_bucket aggregation that we want the maximum value of the `sales` aggregation in the
`sales_per_month` date histogram.
And the following may be the response:
@ -65,6 +70,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -73,7 +82,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
}
},
{
@ -81,7 +90,7 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
}
},
{
@ -89,18 +98,20 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
}
}
]
},
"max_monthly_sales": {
"keys": ["2015/01/01 00:00:00"], <1>
"value": 550
"value": 550.0
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
<1> `keys` is an array of strings since the maximum value may be present in multiple buckets

26
docs/reference/aggregations/pipeline/min-bucket-aggregation.asciidoc
View File

@ -3,8 +3,8 @@
experimental[]
A sibling pipeline aggregation which identifies the bucket(s) with the minimum value of a specified metric in a sibling aggregation
and outputs both the value and the key(s) of the bucket(s). The specified metric must be numeric and the sibling aggregation must
A sibling pipeline aggregation which identifies the bucket(s) with the minimum value of a specified metric in a sibling aggregation
and outputs both the value and the key(s) of the bucket(s). The specified metric must be numeric and the sibling aggregation must
be a multi-bucket aggregation.
==== Syntax
@ -35,7 +35,9 @@ The following snippet calculates the minimum of the total monthly `sales`:
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -58,8 +60,10 @@ The following snippet calculates the minimum of the total monthly `sales`:
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this max_bucket aggregation that we want the minimum value of the `sales` aggregation in the
<1> `buckets_path` instructs this max_bucket aggregation that we want the minimum value of the `sales` aggregation in the
`sales_per_month` date histogram.
And the following may be the response:
@ -67,6 +71,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -75,7 +83,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
}
},
{
@ -83,7 +91,7 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
}
},
{
@ -91,18 +99,20 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
}
}
]
},
"min_monthly_sales": {
"keys": ["2015/02/01 00:00:00"], <1>
"value": 60
"value": 60.0
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
<1> `keys` is an array of strings since the minimum value may be present in multiple buckets

26
docs/reference/aggregations/pipeline/movavg-aggregation.asciidoc
View File

@ -52,23 +52,29 @@ embedded like any other metric aggregation:
[source,js]
--------------------------------------------------
POST /_search
{
"my_date_histo":{ <1>
"date_histogram":{
"field":"timestamp",
"interval":"day"
},
"aggs":{
"the_sum":{
"sum":{ "field": "lemmings" } <2>
"size": 0,
"aggs": {
"my_date_histo":{ <1>
"date_histogram":{
"field":"timestamp",
"interval":"day"
},
"the_movavg":{
"moving_avg":{ "buckets_path": "the_sum" } <3>
"aggs":{
"the_sum":{
"sum":{ "field": "lemmings" } <2>
},
"the_movavg":{
"moving_avg":{ "buckets_path": "the_sum" } <3>
}
}
}
}
}
--------------------------------------------------
// CONSOLE
<1> A `date_histogram` named "my_date_histo" is constructed on the "timestamp" field, with one-day intervals
<2> A `sum` metric is used to calculate the sum of a field. This could be any metric (sum, min, max, etc)
<3> Finally, we specify a `moving_avg` aggregation which uses "the_sum" metric as its input.

29
docs/reference/aggregations/pipeline/percentiles-bucket-aggregation.asciidoc
View File

@ -34,7 +34,9 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -49,7 +51,7 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
}
}
},
"sum_monthly_sales": {
"percentiles_monthly_sales": {
"percentiles_bucket": {
"buckets_path": "sales_per_month>sales", <1>
"percents": [ 25.0, 50.0, 75.0 ] <2>
@ -58,6 +60,9 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this percentiles_bucket aggregation that we want to calculate percentiles for
the `sales` aggregation in the `sales_per_month` date histogram.
<2> `percents` specifies which percentiles we wish to calculate, in this case, the 25th, 50th and 75th percentil
@ -67,6 +72,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -75,7 +84,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
}
},
{
@ -83,7 +92,7 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
}
},
{
@ -91,22 +100,24 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
}
}
]
},
"percentiles_monthly_sales": {
"values" : {
"25.0": 60,
"50.0": 375",
"75.0": 550
"25.0": 60.0,
"50.0": 375.0,
"75.0": 550.0
}
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
==== Percentiles_bucket implementation
@ -116,4 +127,4 @@ interpolate between data points.
The percentiles are calculated exactly and is not an approximation (unlike the Percentiles Metric). This means
the implementation maintains an in-memory, sorted list of your data to compute the percentiles, before discarding the
data. You may run into memory pressure issues if you attempt to calculate percentiles over many millions of
data-points in a single `percentiles_bucket`.
data-points in a single `percentiles_bucket`.

12
docs/reference/aggregations/pipeline/serial-diff-aggregation.asciidoc
View File

@ -61,7 +61,9 @@ A `serial_diff` aggregation looks like this in isolation:
[source,js]
--------------------------------------------------
POST /_search
{
"size": 0,
"aggs": {
"my_date_histo": { <1>
"date_histogram": {
@ -85,6 +87,8 @@ A `serial_diff` aggregation looks like this in isolation:
}
}
--------------------------------------------------
// CONSOLE
<1> A `date_histogram` named "my_date_histo" is constructed on the "timestamp" field, with one-day intervals
<2> A `sum` metric is used to calculate the sum of a field. This could be any metric (sum, min, max, etc)
<3> Finally, we specify a `serial_diff` aggregation which uses "the_sum" metric as its input.
@ -93,11 +97,3 @@ Serial differences are built by first specifying a `histogram` or `date_histogra
add normal metrics, such as a `sum`, inside of that histogram. Finally, the `serial_diff` is embedded inside the histogram.
The `buckets_path` parameter is then used to "point" at one of the sibling metrics inside of the histogram (see
<<buckets-path-syntax>> for a description of the syntax for `buckets_path`.

27
docs/reference/aggregations/pipeline/stats-bucket-aggregation.asciidoc
View File

@ -33,7 +33,9 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -56,6 +58,9 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
}
}
--------------------------------------------------
// CONSOLE
// TEST[setup:sales]
<1> `bucket_paths` instructs this `stats_bucket` aggregation that we want the calculate stats for the `sales` aggregation in the
`sales_per_month` date histogram.
@ -64,6 +69,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -72,7 +81,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
}
},
{
@ -80,7 +89,7 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
}
},
{
@ -88,19 +97,21 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
}
}
]
},
"stats_monthly_sales": {
"count": 3,
"min": 60,
"max": 550,
"avg": 328.333333333,
"sum": 985
"min": 60.0,
"max": 550.0,
"avg": 328.3333333333333,
"sum": 985.0
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]

25
docs/reference/aggregations/pipeline/sum-bucket-aggregation.asciidoc
View File

@ -3,7 +3,7 @@
experimental[]
A sibling pipeline aggregation which calculates the sum across all bucket of a specified metric in a sibling aggregation.
A sibling pipeline aggregation which calculates the sum across all bucket of a specified metric in a sibling aggregation.
The specified metric must be numeric and the sibling aggregation must be a multi-bucket aggregation.
==== Syntax
@ -33,7 +33,9 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
[source,js]
--------------------------------------------------
POST /sales/_search
{
"size": 0,
"aggs" : {
"sales_per_month" : {
"date_histogram" : {
@ -56,7 +58,10 @@ The following snippet calculates the sum of all the total monthly `sales` bucket
}
}
--------------------------------------------------
<1> `buckets_path` instructs this sum_bucket aggregation that we want the sum of the `sales` aggregation in the
// CONSOLE
// TEST[setup:sales]
<1> `buckets_path` instructs this sum_bucket aggregation that we want the sum of the `sales` aggregation in the
`sales_per_month` date histogram.
And the following may be the response:
@ -64,6 +69,10 @@ And the following may be the response:
[source,js]
--------------------------------------------------
{
"took": 11,
"timed_out": false,
"_shards": ...,
"hits": ...,
"aggregations": {
"sales_per_month": {
"buckets": [
@ -72,7 +81,7 @@ And the following may be the response:
"key": 1420070400000,
"doc_count": 3,
"sales": {
"value": 550
"value": 550.0
}
},
{
@ -80,7 +89,7 @@ And the following may be the response:
"key": 1422748800000,
"doc_count": 2,
"sales": {
"value": 60
"value": 60.0
}
},
{
@ -88,15 +97,17 @@ And the following may be the response:
"key": 1425168000000,
"doc_count": 2,
"sales": {
"value": 375
"value": 375.0
}
}
]
},
"sum_monthly_sales": {
"value": 985
"value": 985.0
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/"took": 11/"took": $body.took/]
// TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]