Commit Graph

10 Commits

Author SHA1 Message Date
Zachary Tong aa877161ff [Rollup] Register FeatureSetUsage with xpack, add tests (elastic/x-pack-elasticsearch#4040)
We had a Usage class before, but weren't registering it with XPack.
Would be nice to add more usage info in the future (like the running
jobs on each node), but unclear the best way to do it since we'd need
to filter through the list of allocated tasks.

Original commit: elastic/x-pack-elasticsearch@5207d2758b
2018-03-08 08:06:42 -08:00
Jason Tedor ead1c6c315 Fix Javadocs for o.e.x.r.j.RollupIndexer
This commit fixes the Javadocs for the class o.e.x.r.j.RollupIndexer as
these Javadocs were referring to instance methods on the class
incorrectly (using a this prefix).

Original commit: elastic/x-pack-elasticsearch@fdcc7338f9
2018-03-06 14:12:42 -08:00
Lee Hinman 2147d217df Wrap stream passed to createParser in try-with-resources (elastic/x-pack-elasticsearch#4055)
This wraps the stream (`.streamInput()`) that is passed to many of the
`createParser` instances in the enclosing (or a new) try-with-resources block.
This ensures the `BytesReference.streamInput()` is closed.

Relates to elastic/x-pack-elasticsearch#28504

Original commit: elastic/x-pack-elasticsearch@7546e3b4d4
2018-03-04 16:48:15 -07:00
polyfractal 933738c264 [Rollup] Don't use lucene's newSearcher() method in tests
Use AggregatorTestCase's `newIndexSearcher()` instead.  Lucene's
version can randomly wrap with IndexReader with things we can't handle
like ParallelCompositeReader

Original commit: elastic/x-pack-elasticsearch@b4c0e9a601
2018-03-02 17:07:57 -08:00
polyfractal faac0d2a52 [Rollup] Don't randomize index name in test
The test job helper randomizes the index name with 1-10 characters,
which can lead to randomized index names to overlap and show fewer
caps than the test expects.

The solution is to just use index names "0"-"24" to ensure none
of the names overlap, and thus the caps don't overlap.

Original commit: elastic/x-pack-elasticsearch@74a6d13213
2018-03-02 16:16:11 -08:00
polyfractal 7fbe289d30 [Rollup] Fix bad await in tests
The arrangement of the final latch meant the latch could countdown,
then the test ends before the await() triggers which caused the
thread to be interrupted and fail.  The whole arrangement was incorrect
anyhow.

We need to await the latch before sending the search response as before,
but move the final atomicBoolean to the second time the persistent
task status is updated which is a signal that we are done
and can end the test

If these tests continues to be flaky, we should probably just remove them.
The headers are tested elsewhere and not required to be tested in this
context.

Original commit: elastic/x-pack-elasticsearch@0cf5603972
2018-03-02 16:05:36 -08:00
Zachary Tong 3b474d8868 [Test] Fix slow rollup job task test
Incorrect latch caused this test to run slowly (until the await
finished), and could probably cause failure due to incorrect ordering

Original commit: elastic/x-pack-elasticsearch@ebeb8655da
2018-02-24 20:04:33 +00:00
Zachary Tong eb82e3cf61 [Test] Fix bad latches in rollup state tests
The latches were not placed correctly, allowing the aborts
to be set before we checked the state for Indexing the first time.

This was due to using the DelayingIndexer's built in latch, which
isn't placed quite where we needed it.

Original commit: elastic/x-pack-elasticsearch@590cfa07b0
2018-02-24 18:44:51 +00:00
Zachary Tong 390e64aabd Add empty policy file to Rollups
Packaging tests seem to require a policy file for the time being

Original commit: elastic/x-pack-elasticsearch@ce34b023db
2018-02-24 03:28:28 +00:00
Zachary Tong bf1550a0b2 Rollups for Elasticsearch (elastic/x-pack-elasticsearch#4002)
This adds a new Rollup module to XPack, which allows users to configure periodic "rollup jobs" to pre-aggregate data.  That data is then available later for search through a special RollupSearch API, which mimics the DSL and functionality of regular search.

Rollups are used to drastically reduce the on-disk footprint of metric-based data (e.g. timestamped document with numeric and keyword fields).  It can also be used to speed up aggregations over large datasets, since the rolled data will be considerably smaller and fewer documents to search.

The PR adds seven new endpoints to interact with Rollups; create/get/delete job, start/stop job, a capabilities API similar to field-caps, and a Rollup-enabled search.

Original commit: elastic/x-pack-elasticsearch@dcde91aacf
2018-02-23 17:10:37 -05:00