Commit Graph

216 Commits

Author SHA1 Message Date
Ryan Biesemeyer 9f1525255a Update link to mapper-murmur3 plugin in card docs (#19788) 2016-08-04 15:56:59 +02:00
Adrien Grand a0818d3b87 Split regular histograms from date histograms. #19551
Currently both aggregations really share the same implementation. This commit
splits the implementations so that regular histograms can support decimal
intervals/offsets and compute correct buckets for negative decimal values.

However the response API is still the same. So for intance both regular
histograms and date histograms will produce an
`org.elasticsearch.search.aggregations.bucket.histogram.Histogram`
aggregation.

The optimization to compute an identifier of the rounded value and the
rounded value itself has been removed since it was only used by regular
histograms, which now do the rounding themselves instead of relying on the
Rounding abstraction.

Closes #8082
Closes #4847
2016-08-03 08:39:48 +02:00
Adrien Grand dcc598c414 Make the heuristic to compute the default shard size less aggressive.
The current heuristic to compute a default shard size is pretty aggressive,
it returns `max(10, number_of_shards * size)` as a value for the shard size.
I think making it less aggressive has the benefit that it would reduce the
likelyness of running into OOME when there are many shards (yearly
aggregations with time-based indices can make numbers of shards in the
thousands) and make the use of breadth-first more likely/efficient.

This commit replaces the heuristic with `size * 1.5 + 10`, which is enough
to have good accuracy on zipfian distributions.
2016-07-29 09:59:29 +02:00
Jared McQueen d97b3fd817 [docs] missing a comma in the terms aggregation example 2016-07-27 12:59:38 -04:00
Colin Goodheart-Smithe 3f344d3154 [DOCS] fix documentation for selecting algorithm for percentiles agg 2016-07-27 08:48:51 +01:00
Colin Goodheart-Smithe 7ed64af639 [DOCS] fix callout in buckets path docs 2016-07-26 11:33:54 +01:00
Colin Goodheart-Smithe 2c12c3e628 Add _bucket_count option to buckets_path
This change adds a new special path to the buckets_path syntax
`_bucket_count`. This new option will return the number of buckets for a
multi-bucket aggregation, which can then be used in pipeline
aggregations.

Closes #19553
2016-07-26 09:28:21 +01:00
Adrien Grand 1ed6c5d110 Docs: Add more points to the chart that gives accuracy for the cardinality aggregation.
This also adds instructions how to regenerate the chart.
2016-07-20 10:37:12 +02:00
Adrien Grand bde99bad2e Use a static default precision for the cardinality aggregation. #19215
Today the default precision for the cardinality aggregation depends on how many
parent bucket aggregations it had. The reasoning was that the more parent bucket
aggregations, the more buckets the cardinality had to be computed on. And this
number could be huge depending on what the parent aggregations actually are.

However now that we run terms aggregations in breadth-first mode by default when
there are sub aggregations, it is less likely that we have to run the cardinality
aggregation on kagilions of buckets. So we could use a static default, which will
be less confusing to users.
2016-07-18 11:30:41 +02:00
Jim Ferenczi afe99fcdcd Restore reverted change now that alpha4 is out:
Rename `fields` to `stored_fields` and add `docvalue_fields`

`stored_fields` parameter will no longer try to retrieve fields from the _source but will only return stored fields.
`fields` will throw an exception if the user uses it.
Add `docvalue_fields` as an adjunct to `fielddata_fields` which is deprecated. `docvalue_fields` will try to load the value from the docvalue and fallback to fielddata cache if docvalues are not enabled on that field.

Closes #18943
2016-07-04 10:39:49 +02:00
Leon Weidauer 1297a707da non-binary gender option in term aggr. example (#19188)
* non-binary gender option in term aggr. example

* replace gender with music genre for term aggregation docs
2016-07-01 14:59:03 +02:00
Jason Tedor 00356edd33 Clarify time units usage in docs
This commit clarifies the distinction between supported time units for
durations and supported time units for durations in the docs.

Relates #19159
2016-06-29 17:02:15 -04:00
Robert Muir 6d52cec2a0 Merge pull request #19092 from rmuir/more_painless_docs
cutover some docs to painless
2016-06-28 13:40:25 -04:00
Jim Ferenczi eb1e231a63 Revert "Rename `fields` to `stored_fields` and add `docvalue_fields`"
This reverts commit 2f46f53dc8.
2016-06-27 17:20:32 +02:00
Robert Muir 6fc1a22977 cutover some docs to painless 2016-06-27 09:55:16 -04:00
Jerry Liu 1863ab95f8 fixed typo 'if' -> 'is' (#19051) 2016-06-27 14:20:23 +02:00
Nik Everett ee2a77143b Docs: Convert aggs/misc to CONSOLE
They should be more readable and tested during the build.
2016-06-22 14:52:06 -04:00
Jim Ferenczi 2f46f53dc8 Rename `fields` to `stored_fields` and add `docvalue_fields`
`stored_fields` parameter will no longer try to retrieve fields from the _source but will only return stored fields.
`fields` will throw an exception if the user uses it.
Add `docvalue_fields` as an adjunct to `fielddata_fields` which is deprecated. `docvalue_fields` will try to load the value from the docvalue and fallback to fielddata cache if docvalues are not enabled on that field.

Closes #18943
2016-06-22 17:38:30 +02:00
Jim Ferenczi fb2a48d0f0 Revert "Remove support for sorting terms aggregation by ascending count"
This is delayed after alpha4 since Kibana relies on it.
2016-06-17 17:14:01 +02:00
Jim Ferenczi 755721953b Remove support for sorting terms aggregation by ascending count
closes #17614
2016-06-17 15:06:49 +02:00
Glen Smith 5284c5094d grammar 2016-06-17 10:09:21 +02:00
Jim Ferenczi ad232aebbe Set collection mode to breadth_first in the terms aggregation when the cardinality of the field is unknown or smaller than the requested size.
closes #9825
2016-06-16 11:33:40 +02:00
Colin Goodheart-Smithe cfd3356ee3 Remove size 0 options in aggregations
This removes the ability to set `size: 0` in the `terms`, `significant_terms` and `geohash_grid` aggregations for the reasons described in https://github.com/elastic/elasticsearch/issues/18838

Closes #18838
2016-06-14 13:07:02 +01:00
Nicholas Knize 371c73e140 refactor matrix agg documentation from modules to main agg section 2016-06-06 07:39:00 -05:00
Adrien Grand 638da06c1d Add back support for `ip` range aggregations. #17859
This commit adds support for range aggregations on `ip` fields. However it will
only work on 5.x indices.

Closes #17700
2016-05-13 17:22:01 +02:00
Robert Muir c5532d3df0 add a rest test for this that seems to work, fix the documentation. thanks @s1monw 2016-05-11 16:07:08 -04:00
Jim Ferenczi 052191f2a2 Add the ability to use the breadth_first mode with nested aggregations (such as `top_hits`) which require access to score information.
The score is recomputed lazily for each document belonging to a top bucket.
Relates to #9825
2016-05-04 15:35:45 +02:00
Sergii Golubev 2f6405ee27 serial-diff-aggregation.asciidoc: fix a mistake (#17950) 2016-04-25 07:45:54 -04:00
ericamick 069eb72604 Update bucket.asciidoc 2016-04-22 10:54:25 -06:00
Martijn van Groningen 8e63ce00f0 docs: removed confusing statement. 2016-04-19 11:49:51 +02:00
Sergii Golubev 5ce3eb96b0 tophits-aggregation.asciidoc: fix a typo 2016-04-18 09:23:39 +02:00
Sergii Golubev 434a563fe0 terms-aggregation.asciidoc tiny edit 2016-04-13 16:51:47 -06:00
Sergii Golubev 39b914bd77 histogram-aggregation.asciidoc: tiny edit (#17706) 2016-04-13 14:19:05 +02:00
Adrien Grand 1d0239c125 Add a warning about the impact of sorting terms aggregations on the accuracy of doc counts. 2016-04-07 16:57:44 +02:00
Dmitrii Izgurskii 272f3eb140 Add missing comma
Added missing comma
2016-04-06 15:03:37 -06:00
Adrien Grand b42f66c8ac Document 5.0 mapping changes. 2016-03-22 16:22:58 +01:00
Clinton Gormley 0ed0fea558 Updated link to Joda time zones 2016-03-14 12:24:58 +01:00
Christoph Büscher ff46303f15 Simplify mock scripts 2016-03-07 15:39:35 +01:00
Christoph Büscher 6b0f63e1a6 Adding `time_zone` parameter to daterange-aggregation docs 2016-03-07 15:38:24 +01:00
Clinton Gormley 9674cbbe62 Documented [] syntax for buckets_path
Closes #15707
2016-03-01 09:55:01 +01:00
Clinton Gormley 300554841e Merge pull request #16738 from robertlyson/patch-1
Update to serial differencing aggregation doc
2016-02-28 20:09:14 +01:00
evanfreed 7ed30a9c00 Spelling
Corrected spelling.
2016-02-26 13:39:25 -05:00
Robert 7844804874 Update to serial differencing aggregation doc
Hi,

`thirtieth_difference` should use `the_sum` metric as the `buckets_path`.
2016-02-20 12:13:02 +01:00
Colin Goodheart-Smithe e546db0753 [DOCS] fix to sampler agg documentation 2016-02-15 13:17:19 +00:00
Colin Goodheart-Smithe 5f489b99bf fixed docs link error 2016-02-15 12:12:16 +00:00
Colin Goodheart-Smithe 1f760bd1bd Merge branch 'master' into feature/aggs-refactoring 2016-02-10 12:16:26 +00:00
Dongjoon Hyun 21ea552070 Fix typos in docs. 2016-02-09 02:07:32 -08:00
Colin Goodheart-Smithe 5d9d91b761 Merge branch 'master' into feature/aggs-refactoring 2016-02-03 14:45:16 +00:00
Clinton Gormley 53662b0be9 Merge pull request #16345 from lbrito1/patch-1
Changes "that is" to "for example".
2016-02-02 15:13:29 +01:00
Colin Goodheart-Smithe 3b35754f59 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/percolator/PercolateDocumentParserTests.java
2016-01-26 13:17:53 +00:00
Clinton Gormley 7cde0d47bc Merge pull request #16215 from eemp/patch-1
Update filters-aggregation.asciidoc
2016-01-26 12:56:43 +01:00
Colin Goodheart-Smithe cd8320b171 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filter/FilterAggregator.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filters/FiltersAggregator.java
#	core/src/main/java/org/elasticsearch/search/SearchModule.java
2016-01-25 10:42:20 +00:00
Kevin Adams 768d171f77 Timezone: use forward slash
Using a backslash causes errors when querying elasticsearch, but changing the back slash to forward slash on the timezone fixes it.

Closes #16148
2016-01-22 14:26:49 +01:00
Colin Goodheart-Smithe 2c33f78192 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/children/ChildrenParser.java
#	core/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParser.java
#	test/framework/src/main/java/org/elasticsearch/test/TestSearchContext.java
2016-01-06 09:35:53 +00:00
Eugene Pirogov d48af9a155 Fix indent in example
Previously it would look like if `warnings` key is nested under `errors`.
2016-01-05 14:41:09 +01:00
omiend 0c878f3bf6 add double quotation 2016-01-04 11:55:24 +09:00
Colin Goodheart-Smithe 1aea0faa86 Aggregations Refactor: Refactor Sampler Aggregation 2015-12-21 09:35:46 +00:00
KangYongKyun b5d49641fb colon is added
"predict"  10 => "predict" : 10
2015-11-05 11:32:20 +09:00
Nicholas Knize b31d3ddd3e Adds geo_centroid metric aggregator
This commit adds a new metric aggregator for computing the geo_centroid over a set of geo_point fields. This can be combined with other aggregators (e.g., geohash_grid, significant_terms) for computing the geospatial centroid based on the document sets from other aggregation results.
2015-10-14 16:19:09 -05:00
Clinton Gormley 3e7201ef63 Merge pull request #14096 from speedplane/patch-2
Fixed a typo ("when when")
2015-10-13 21:17:09 +02:00
Clinton Gormley dc018cf622 Updated docs for 3.0.0-beta 2015-10-07 13:27:46 +02:00
Alex 4077a322c5 Docs: Fix typo - datehistogram
date_histogram in place of datehistogram

Closes #13886
2015-10-06 19:22:21 +02:00
Taehee Kim 45e0ccd274 Fix typo 2015-09-25 06:42:21 +09:00
Adrien Grand 86f1b07df0 Docs: Remove docs for the `filtered`, `and`, `or` and `(f)query` queries. 2015-09-11 11:00:54 +02:00
Clinton Gormley 8aba6ce93a Docs: Improved the date histogram docs for time_zone and offset 2015-09-07 19:54:00 +02:00
Zachary Tong 397d5beae1 Aggregations: Add stats_bucket / extended_stats_bucket pipeline aggregations
These are the complements to the stats/extended_stats metric aggregations, and can be used
to calculate a variety of statistics over buckets
2015-09-04 15:23:48 -04:00
Zachary Tong c5b39ce85e [DOCS] Fix broken inter-page link 2015-09-03 23:17:01 -04:00
Zachary Tong 1016734b4c Aggregations: Add percentiles_bucket pipeline aggregations
This pipeline will calculate percentiles over a set of sibling buckets.  This is an exact
implementation, meaning it needs to cache a copy of the series in memory and sort it to determine
the percentiles.

This comes with a few limitations: to prevent serializing data around, only the requested percentiles
are calculated (unlike the TDigest version, which allows the java API to ask for any percentile).
It also needs to store the data in-memory, resulting in some overhead if the requested series is
very large.
2015-09-03 22:24:14 -04:00
Lee Hinman 118eab5462 Merge pull request #13257 from elastic/docsfix
Fixed non-valid JSON (though ES would accept it)
2015-09-02 07:51:13 -06:00
Colin Goodheart-Smithe 1d9905a798 [DOCS] Added note about valid return types for scripts in the scripted_metric aggregation 2015-09-02 12:13:15 +01:00
Shane Connelly 5e385d5bf2 Fixed non-valid JSON (though ES would accept it) 2015-09-01 13:17:07 -07:00
Clinton Gormley aa52c4f712 Docs: Fixed variations of spelling of buckets_path
Closes #13201
2015-08-31 13:47:40 +02:00
Colin Goodheart-Smithe 9112217869 Merge pull request #13024 from iantruslove/patch-1
[DOCS] Couple of typos - various misspellings of `buckets-path`
2015-08-24 15:37:05 +02:00
Murilo Pereira a960b3cac4 Here too. 2015-08-20 18:07:51 -03:00
Murilo Pereira 13f961a3d3 s/bucket_paths/buckets_path/
Using "bucket_paths" makes the server return a 400 with "Unknown key for a VALUE_STRING in [aggregation-name]: [buckets_paths]."
2015-08-20 18:05:02 -03:00
Ian Truslove ae0a74eb1c Couple of typos - various misspellings of `buckets-path` 2015-08-20 14:57:09 -06:00
Adrien Grand a91b3fcbb9 Move the `murmur3` field to a plugin and fix defaults.
This move the `murmur3` field to the `mapper-murmur3` plugin and fixes its
defaults so that values will not be indexed by default, as the only purpose
of this field is to speed up `cardinality` aggregations on high-cardinality
string fields, which only requires doc values.

I also removed the `rehash` option from the `cardinality` aggregation as it
doesn't bring much value (rehashing is cheap) and allowed to remove the
coupling between the `cardinality` aggregation and the `murmur3` field.

Close #12874
2015-08-18 11:41:52 +02:00
Clinton Gormley c6c3a40cb6 Docs: Updated annotations for 2.0.0-beta1 2015-08-14 10:51:09 +02:00
Asimov4 60f3ea0131 Fixing typo 2015-08-08 14:14:59 -07:00
Sylvain Zimmer c2f774ac57 Warning in the docs for negative histogram values
As requested in https://github.com/elastic/elasticsearch/issues/8082#issuecomment-127962374
2015-08-07 13:10:03 +02:00
Clinton Gormley ac2b8951c6 Docs: Mapping docs completely rewritten for 2.0 2015-08-06 17:24:51 +02:00
Sylvain Zimmer 12a2db5417 Fix typo in docs 2015-07-31 19:11:04 -04:00
Colin Goodheart-Smithe 3e0532a0c5 Aggregations: Add HDRHistogram as an option in percentiles and percentile_ranks aggregations
HDRHistogram has been added as an option in the percentiles and percentile_ranks aggregation. It has one option `number_significant_digits` which controls the accuracy and memory size for the algorithm

Closes #8324
2015-07-24 17:55:36 +01:00
Ryan Ernst dba42a83e2 Docs: Update time_zone specification
closes #12317
2015-07-21 00:22:53 -07:00
Zachary Tong 8790989a47 [DOCS] Fix link to serial_diff docs 2015-07-10 19:01:18 -04:00
Zachary Tong bb9c160855 Merge pull request #11196 from polyfractal/feature/aggs_2_0_diff
Aggregations: add serial differencing pipeline aggregation
2015-07-10 18:26:19 -04:00
Zachary Tong e3f9d561e4 Aggregations: add serial differencing pipeline aggregation 2015-07-10 18:22:01 -04:00
Zachary Tong 0f76e656dd Aggregations: add cost minimizer to moving_avg aggregation 2015-07-08 16:20:34 -04:00
Zachary Tong c898dd252b [DOCS] Update section about gap_policy 2015-07-07 15:40:15 -04:00
Colin Goodheart-Smithe 1d7fc6b4f2 Aggregations: Pipeline Aggregation to filter buckets based on a script
This pipeline aggregation runs a script on each bucket in the parent aggregation to determine whether the bucket is kept in the final aggregation tree. If the script returns true the bucket is retained, if it returns false the bucket is dropped
2015-07-07 09:51:16 +01:00
Colin Goodheart-Smithe e366d0380d Aggregations: Adds other bucket to filters aggregation
The filters aggregation now has an option to add an 'other' bucket which will, when turned on, contain all documents which do not match any of the defined filters. There is also an option to change the name of the 'other' bucket from the default of '_other_'

Closes #11289
2015-07-01 10:44:04 +01:00
William Li 2be3fe31a4 Docs: Update filter-aggregation.asciidoc
Closes #11782
2015-07-01 10:17:45 +02:00
Colin Goodheart-Smithe 62cbeecadf [DOCS] marked pipeline aggregator documentation as Experimental 2015-06-30 10:30:50 +01:00
Adrien Grand 38f5cc236a Rename caches.
In order to be more consistent with what they do, the query cache has been
renamed to request cache and the filter cache has been renamed to query
cache.

A known issue is that package/logger names do no longer match settings names,
please speak up if you think this is an issue.

Here are the settings for which I kept backward compatibility. Note that they
are a bit different from what was discussed on #11569 but putting `cache` before
the name of what is cached has the benefit of making these settings consistent
with the fielddata cache whose size is configured by
`indices.fielddata.cache.size`:
 * index.cache.query.enable -> index.requests.cache.enable
 * indices.cache.query.size -> indices.requests.cache.size
 * indices.cache.filter.size -> indices.queries.cache.size

Close #11569
2015-06-29 10:15:27 +02:00
Christoph Büscher f5f73259e4 Docs: Update Joda URLs in documentation. 2015-06-26 10:23:02 +02:00
Colin Goodheart-Smithe f21924ae0d Aggregations: Adds cumulative sum aggregation
This adds a new pipeline aggregation, the cumulative sum aggregation. This is a parent aggregation which must be specified as a sub-aggregation to a histogram or date_histogram aggregation. It will add a new aggregation to each bucket containing the sum of a specified metrics over this and all previous buckets.
2015-06-25 14:27:57 +01:00
Clinton Gormley 37eae789a0 Merge pull request #11801 from golubev/patch-6
fix json syntax in filters-aggregation.asciidoc
2015-06-23 20:02:04 +02:00
Colin Goodheart-Smithe f26311e88b Aggregations: Rename `series_arithmetic` agg to `bucket_script` 2015-06-23 14:00:17 +01:00
Clinton Gormley f123a53d72 Docs: Refactored modules and index modules sections 2015-06-22 23:49:45 +02:00
caldwecr 1ac728d22b Docs: Update filter-aggregation.asciidoc
Replace the previous example which leveraged a range filter, which causes unnecessary confusion about when to use a range filter to create a single bucket or a range aggregation with exactly one member in ranges.

Closes #11704
2015-06-19 12:24:42 +02:00
Clinton Gormley 64ec18afa0 Merge pull request #11661 from pjcard/patch-1
Make explicit the requirement for intervals to be integers
Conflicts:
	docs/reference/search/aggregations/bucket/histogram-aggregation.asciidoc
2015-06-15 11:42:12 +02:00
Colin Goodheart-Smithe a216062d88 Aggregations: allow users to perform simple arithmetic operations on histogram aggregations
Closes #11029
2015-06-12 09:25:52 +01:00
Colin Goodheart-Smithe 35a58d874e Scripting: Unify script and template requests across codebase
This change unifies the way scripts and templates are specified for all instances in the codebase. It builds on the Script class added previously and adds request building and parsing support as well as the ability to transfer script objects between nodes. It also adds a Template class which aims to provide the same functionality for template APIs

Closes #11091
2015-05-29 16:52:04 +01:00
Zachary Tong d32a80f37b Docs: Fix misplaced images in moving_avg docs 2015-05-27 16:13:36 -04:00
Zachary Tong 491afbe01c Aggregations: Add Holt-Winters model to `moving_avg` pipeline aggregation
Closes #11043
2015-05-27 14:45:45 -04:00
Colin Goodheart-Smithe 35deb7efea Aggregations: Renaming reducers to Pipeline Aggregators 2015-05-21 14:57:23 +01:00
Adrien Grand 32e23b9100 Aggs: Make it possible to configure missing values.
Most aggregations (terms, histogram, stats, percentiles, geohash-grid) now
support a new `missing` option which defines the value to consider when a
field does not have a value. This can be handy if you eg. want a terms
aggregation to handle the same way documents that have "N/A" or no value
for a `tag` field.

This works in a very similar way to the `missing` option on the `sort`
element.

One known issue is that this option sometimes cannot make the right decision
in the unmapped case: it needs to replace all values with the `missing` value
but might not know what kind of values source should be produced (numerics,
strings, geo points?). For this reason, we might want to add an `unmapped_type`
option in the future like we did for sorting.

Related to #5324
2015-05-15 16:26:58 +02:00
Adrien Grand a0af88e996 Query DSL: Remove filter parsers.
This commit makes queries and filters parsed the same way using the
QueryParser abstraction. This allowed to remove duplicate code that we had
for similar queries/filters such as `range`, `prefix` or `term`.
2015-05-07 20:14:34 +02:00
Colin Goodheart-Smithe cf1251796f Aggregations: Adding Sum Bucket Aggregation
Closes #11007
2015-05-06 14:44:56 +01:00
Zachary Tong e70a8d4ee9 Merge pull request #10964 from polyfractal/feature/aggs_movavg_rename
Rename Moving Average models to their "common" names
2015-05-06 09:07:23 -04:00
Zachary Tong 3eb9cb913d Rename Moving Average models to their "common" names
Previously, we were using the "statistical", technically accurate name.  Instead, we
should probably use the name that people are familiar with, e.g. "Holt Winters" instead
of "triple exponential".  To that end:

- `single_exp` becomes `ewma` (exponentially weighted moving average)
- `double_exp` becomes `holt`

When the `triple_exp` is added, it will be called `holt_winters`.
2015-05-06 09:04:44 -04:00
Colin Goodheart-Smithe 72d99773dc Aggregations: Adding Average Bucket Aggregation
Also includes changes to the other bucket metric aggregations to share code

Closes #11006
2015-05-06 13:53:57 +01:00
Colin Goodheart-Smithe 644fd00714 Aggregations: x-axis units normalisation for derivative aggregation 2015-05-06 10:31:16 +01:00
Pascal Borreli af6d890ad5 Docs: Fixed typos
Closes #10973
2015-05-05 10:38:05 +02:00
Zachary Tong 967e05ea76 [DOCS] Fix section levels for Sampler agg 2015-05-04 09:18:24 -04:00
Zachary Tong e3ae1df6f0 [DOCS] Restructure Aggs documentation 2015-05-01 16:04:55 -04:00