OpenSearch/server
Nik Everett a451dd87aa
Reduce merge map memory overhead in the Variable Width Histogram Aggregation (#59366) (#60171)
When a document which is distant from existing buckets gets collected, the
`variable_width_histogram` will create a new bucket and then insert it
into the ordered list of buckets.

Currently, a new merge map array is created to move this bucket. This is
very expensive as there might be thousands of buckets.

This PR creates `mergeBuckets(UnaryOperator<Long> mergeMap)` methods in
`BucketsAggregator` and `MergingBucketsDefferingCollector`, and updates
the `variable_width_histogram`  to use them. This eliminates the need to
create an entire merge map array for each new bucket and reduces the
memory overhead of the algorithm.

Co-authored-by: James Dorfman <jamesdorfman@users.noreply.github.com>
2020-07-27 09:23:06 -04:00
..
licenses upgrade to lucene-8.6.0 release (#59596) (#59599) 2020-07-15 12:40:57 +02:00
src Reduce merge map memory overhead in the Variable Width Histogram Aggregation (#59366) (#60171) 2020-07-27 09:23:06 -04:00
build.gradle Remove data stream feature flag (#59572) 2020-07-14 23:50:41 +02:00