10781 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe
5a2c4ab5ae Added test for second_derivative 2015-02-17 11:37:28 +00:00
Colin Goodheart-Smithe
7f844660a8 Cleaning up NOCOMMITs 2015-02-17 11:31:24 +00:00
Colin Goodheart-Smithe
f03fe5b8b6 Cleaning up NOCOMMITs which are resolved 2015-02-17 11:31:02 +00:00
Colin Goodheart-Smithe
6c12cfd465 updated derivative tests to test _count 2015-02-17 11:15:04 +00:00
Colin Goodheart-Smithe
3375c02b42 Added support for _count and _key as bucketsPaths 2015-02-17 11:14:47 +00:00
Colin Goodheart-Smithe
19cdfe256e DerivativeReducer now copies histogram options from old histogram instance 2015-02-17 10:08:25 +00:00
Colin Goodheart-Smithe
269d4bc30e InternalHistogram.Factory.create() can now work from prototype
Another InternalHistogram instance can be passed into the method with the buckets and the name and will be used to set all the options such as minDocCount, formatter, Order etc.
2015-02-17 10:07:56 +00:00
Colin Goodheart-Smithe
f68bce51f1 Tests for derivative reducer
Most tests have been marked with @AwaitsFix since they require functionality to be implemented before they will pass
2015-02-17 09:05:24 +00:00
Colin Goodheart-Smithe
511e275825 More update to support Reducer Builders 2015-02-16 16:32:42 +00:00
Colin Goodheart-Smithe
c97dd84bad Added Builder for Derivatives Reducer 2015-02-16 11:57:01 +00:00
Colin Goodheart-Smithe
e994044d28 Added Builder classes for Reducers 2015-02-16 11:56:41 +00:00
Colin Goodheart-Smithe
247b6a7e13 removed obselete NOCOMMIT and left over sysout call 2015-02-16 10:54:37 +00:00
Colin Goodheart-Smithe
58f2ceca12 Derivative Reducer now supported nth order derivatives 2015-02-16 10:52:00 +00:00
Colin Goodheart-Smithe
f20dae85a9 getProperty method in the aggregations framework now throws a specific exception 2015-02-16 10:51:31 +00:00
Colin Goodheart-Smithe
3ab3ffa989 First (rough) pass at dependancy resolution for reducers
uses the depth-first algorithm from http://en.wikipedia.org/wiki/Topological_sorting#Algorithms

Needs some cleaning up
2015-02-16 10:42:08 +00:00
Colin Goodheart-Smithe
63f3281f12 Merge branch 'master' into feature/aggs_2_0 2015-02-16 09:27:27 +00:00
Simon Willnauer
0698af93c2 [BUILD] Add marvel and license to the release script 2015-02-16 09:46:12 +01:00
Alexander Reelsen
5092904885 Packaging: Add antlr and asm dependencies 2015-02-13 23:55:35 +01:00
Lee Hinman
305ba33710 [CORE] Move as much as possible into abstract Engine
This paves the way for more shared code between the `InternalEngine` and
`ShadowEngine` by way of the abstract `Engine` class. No actual
functionality has been changed.
2015-02-13 14:43:58 -07:00
Blake Niemyjski
8cba6c3abb Fixed an invalid query
Closes #9682
2015-02-13 21:11:42 +01:00
Colin Goodheart-Smithe
9357fc4f95 bucketsPath is now in the Reducer class since every Reducer implementation will need it 2015-02-13 15:43:39 +00:00
Christoph Büscher
c597d8d56b Aggregations: Prevent negative intervals in date_histogram
Negative settings for interval in date_histogram could lead to OOM errors in conjunction
with min_doc_count=0. This fix raises exceptions in the histogram builder and the
 TimeZoneRounding classes so that the query fails before this can happen.

Closes #9634
Closes #9690
2015-02-13 16:05:46 +01:00
Colin Goodheart-Smithe
18c2cb64b7 Validation of the reducer factories is now called from within the AggregatorFactories 2015-02-13 14:33:44 +00:00
Simon Willnauer
616d0c044f remove dead code 2015-02-13 13:43:00 +01:00
Simon Willnauer
287fab46a1 [STORE] use Directory#fileLength() less during calculating checksums 2015-02-13 13:42:11 +01:00
Simon Willnauer
09eb8d1383 [RECOVERY] Handle corruptions during recovery finalization
Today we sometimes have to transfer files without verifying the checksum
ie. if the file had an old alder32 checksum but was using random access
while writing such that we can only verify they files length. We will likely
not detect corruptions there and with the new checks during recovery finalization
we might run into corrupt index exceptions in that stage. This causes
the primary to be failed as well since we don't handle the exception today. This commit
adds better handling and a test for this scenario.
2015-02-13 12:10:37 +01:00
Boaz Leskes
c9480783b3 Rest: expose master_timeout flag on GET _template & HEAD _template
These are master level operation but we currently don't expose this parameter

Closes #9688
2015-02-13 11:25:26 +01:00
Boaz Leskes
51ba120b54 Internal: only ack index store deletion on data nodes
When an index is deleted we wait on all nodes to ack the delete. Data nodes are expected to both ack the remove of the index from their IndicesService and also the deletion of the store from disk. At the moment all nodes sends this ack which causes wrong counting on the master side. On top of this, we currently have an unneeded WARN message in the logs when client nodes try to acquire locks but do not have a data folder.

Relates to #9605

Closes #9672
2015-02-13 10:38:41 +01:00
Simon Willnauer
8dad914a1c Remove obsolet deleteFile call in Store 2015-02-13 10:18:52 +01:00
javanna
8c0ea251f7 [TEST] declare support for groovy-scripting feature in the Java REST tests runner
Groovy was disabled by default, but we turn it on in our test infra. We can then declare support for it so we go and execute script related tests as part of the REST tests suite.
2015-02-13 16:18:28 +11:00
javanna
c8b9228dd6 [TEST] allow to disable REST spec validation
With #9629 we introduced REST spec validation, which barfs whenever the REST spec don't follow the defined conventions. That said, we sometimes execute tests against previous branches and tags which have spec that needs fixing but we can't go back and fix them. We now support the `-Dtests.rest.validate_spec` system property that allows to turn off REST spec validation (enabled by default) so that we can still run tests against old branches/tags.
2015-02-13 15:50:34 +11:00
Ryan Ernst
1e67b9edbd Fix test failure resulting from #8872 change. 2015-02-12 13:59:59 -08:00
Clinton Gormley
75e7ce9c51 REST Test: Lucene segment format can have two or three numbers: 4.10.2 or 4.9 2015-02-12 19:59:43 +01:00
Ryan Ernst
533fdbdf75 Mappings: Remove support for field access by short name
When multiple fields under object fields share the same name, accessing
by short name is ambiguous.  This removes support for short names,
always requiring the full name when used in queries.

closes #8872
2015-02-12 09:58:37 -08:00
Colin Goodheart-Smithe
0f22d7e65e Can now specify a format for the returned derivative values 2015-02-12 17:13:59 +00:00
Clinton Gormley
93fe4495ae REST tests - add feature groovy_scripting to skip tests if groovy not enabled 2015-02-12 18:08:35 +01:00
Clinton Gormley
36758ce81b Fixed cat.segments and indices.segments tests - bad YAML indenting 2015-02-12 17:35:19 +01:00
Andreas Kohn
01b8479179 Allow configuration of the GC log file via an environment variable
Enabling GC logging works now by setting the environment variable ES_GC_LOG_FILE
to the full path to the GC log file. Missing directories will be created as needed.

The ES_USE_GC_LOGGING environment variable is no longer used.

Closes #8471
Closes #8479
2015-02-12 17:07:57 +01:00
Colin Goodheart-Smithe
9805b8359b can now reference single value metrics directly instead of having to add '.value' to the path 2015-02-12 15:47:06 +00:00
Colin Goodheart-Smithe
3a777545de derivative reducer now works with both date_histogram and histogram 2015-02-12 15:36:38 +00:00
Boaz Leskes
35f6496694 Internal: Pending Task listing should account for task being nulled
This caused by #9671 and #9621 working together and cause an NPE
2015-02-12 15:50:54 +01:00
Adrien Grand
f00a9b8557 Minor indentation/validation fix in AggregatorParsers. 2015-02-12 15:50:11 +01:00
Alexander Reelsen
ce24e10783 Testing: Ensure RepeatOnExceptionRule is available in test-jar
This moves the rule, so it is made available in the test.jar. In
addition, you can now specify the exception, which triggers a rerun
of the test in order to make it reusable for others.

Also ensured that the NettyTransportTest frees all resources inside
of its testing method instead of pre/post running methods, as those
are still called only once, even though a failed test might be repeated.
2015-02-12 15:12:54 +01:00
Victor Buldakov
f500293ffb Dependencies: Update joda-time to v2.7
Closes #9338
Closes #9610
2015-02-12 15:06:18 +01:00
Boaz Leskes
d6e9101f42 Internal: Introduce TimedPrioritizedRunnable base class to all commands that go into InternalClusterService.updateTasksExecutor
At the moment we sometime submit generic runnables, which make life slightly harder when generated pending task list which have to account for them. This commit adds an abstract TimedPrioritizedRunnable class which should always be used. This class also automatically measures time in queue, which is needed for the pending task reporting.

Relates to #8077

Closes #9354
Closes #9671
2015-02-12 15:05:34 +01:00
Colin Goodheart-Smithe
ef4a910078 fix to the name of the injected aggregation for derivatives 2015-02-12 14:01:34 +00:00
Colin Goodheart-Smithe
d65e9a4a90 Fixing compile issues after rebase with master
Mostly due to @jpountz's leaf collector changes
2015-02-12 13:52:56 +00:00
Christoph Büscher
41befaf6b5 Change behaviour of indices segments api to allow no indices
Using '_cat/segments' or the indices segments api without matching any index
now returns empty result instead of throwing IndexMissingException.

Closes #9219
2015-02-12 14:42:15 +01:00
Colin Goodheart-Smithe
9cfa6c6af7 Basic derivative reducer 2015-02-12 13:37:16 +00:00
Colin Goodheart-Smithe
55b82db346 Reducers are now wired end-to-end into the agg framework 2015-02-12 13:37:16 +00:00