1725 Commits

Author SHA1 Message Date
Nicholas Knize
bc75fe4a45 add a weighted centroid to the geohash_grid aggregator and cut over to lucene GeoHashUtils 2015-09-11 09:57:08 -05:00
Britta Weber
73b4391a46 [test] fix urls for windows 2015-09-11 16:25:19 +02:00
Britta Weber
53bb17c83c [test] use tmp dir otherwise the security manager will complain 2015-09-11 14:48:19 +02:00
javanna
79c58a122e adapt to upstream lucene changes 2015-09-11 14:37:27 +02:00
javanna
e522458735 remove guava usage from IdsQueryBuilder 2015-09-11 14:20:21 +02:00
javanna
73f7df510e Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/FunctionScoreQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/FunctionScoreQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/factor/FactorParser.java
	core/src/test/java/org/elasticsearch/ExceptionSerializationTests.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchBenchmark.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchShortCircuitBenchmark.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
	core/src/test/java/org/elasticsearch/percolator/PercolatorIT.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchIT.java
	docs/reference/query-dsl/has-parent-query.asciidoc
2015-09-11 14:15:12 +02:00
Masaru Hasegawa
6a9ef99d0f Take relocating shard into consideration during awareness allocation
Previous fix  counted twice for relocating shard (source and target).
Fix it to consider only target node.
2015-09-11 21:09:31 +09:00
Simon Willnauer
6b4699fbf7 Only add unicast provider if all settings are present
Closes 
2015-09-11 13:21:03 +02:00
Britta Weber
2c618a11de plugins: fix print of url when it contains spaces 2015-09-11 13:13:48 +02:00
javanna
380fe52b30 Java api: remove filterBuilder element from FunctionScoreQueryBuilder
The filter element has been deprecated in the function_score query parser. Whenever a filter is found it gets wrapped into a query automatically. The filter in the java api builder is always null, there is no way to set its value, just a leftover.
2015-09-11 13:08:33 +02:00
javanna
a641c7ae52 Function score query: remove deprecated support for boost_factor
boost_factor was deprecated in 1.4.0.beta1. We can remove the support for it in 3.0, its replacement is `weight`.

Closes 
2015-09-11 12:58:25 +02:00
Colin Goodheart-Smithe
d49a744b7e [TEST] Removed @Repeat Annotation 2015-09-11 09:36:25 +01:00
Colin Goodheart-Smithe
f8c47345b0 Fixed Typo in Test Name 2015-09-11 09:30:55 +01:00
Colin Goodheart-Smithe
2118936deb Query Refactoring: Refactor of GeoShapeQuery
Moving the query building functionality from the parser to the builders
new toQuery() method analogous to other recent query refactorings.

Relates to 

PR goes against the query-refactoring branch
2015-09-11 09:26:09 +01:00
Simon Willnauer
853b7fdb7c Fold MoreLikeThisFetchService into MoreLikeThisQueryParser
now that we have a Client on the Shard context we can fold
the doc fetching into the parser / builder.

Relates to 
2015-09-11 10:21:40 +02:00
Boaz Leskes
1b8047e51c merge master 2015-09-11 09:56:13 +02:00
Adrien Grand
e988a9edc6 Merge pull request from rmuir/1702090
upgrade lucene to r1702265
2015-09-11 09:48:43 +02:00
Boaz Leskes
80b59e0d66 Discovery: Add a dedicate queue for incoming ClusterStates
The initial implementation of two phase commit based cluster state publishing () relied on a single in memory "pending" cluster state that is only processed by ZenDiscovery once committed by the master. While this is fine on it's own, it resulted in an issue with acknowledged APIs, such as the open index API, in the extreme case where a node falls behind and receives a commit message after a new cluster state has been published. Specifically:

1) Master receives and acked-API call and publishes cluster state CS1
2) Master waits for a min-master nodes to receives CS1 and commits it.
3) All nodes that have responded to CS1 are sent a commit message, however, node N didn't respond yet
4) Master waits for publish timeout (defaults to 30s) for all nodes to process the commit. Node N fails to do so.
5) Master publishes a cluster state CS2. Node N responds to cluster state CS1's publishing but receives cluster state CS2 before the commit for CS1 arrives.
6) The commit message for cluster CS1 is processed on node N, but fails because CS2 is pending. This caused the acked API in step 1 to return (but CS2 , is not yet processed).

In this case, the action indicated by CS1 is not yet executed on node N and therefore the acked API calls return pre-maturely. Note that once CS2 is processed but the change in CS1 takes effect (cluster state operations are safe to batch and we do so all the time).

An example failure can be found on: http://build-us-00.elastic.co/job/es_feature_two_phase_pub/314/

This commit extracts the already existing pending cluster state queue (processNewClusterStates) from ZenDiscovery into it's own class, which serves as a temporary container for in-flight cluster states. Once committed the cluster states are transferred to ZenDiscovery as they used to before. This allows "lagging" cluster states to still be successfully committed and processed (and likely to be ignored as a newer cluster state has already been processed).

As a side effect, all batching logic is now extracted from ZenDiscovery and is unit tested.
2015-09-11 09:23:41 +02:00
Simon Willnauer
94a37d486f Remove ShapeFetchService and TermsLookupFetchService and use a Client instead
This commit removes all the optional injects etc. for the FetchServices and
provides a Client via IndexQueryParserService / Context. This allows direct
injection instead of optional injection. It also allows to remove all the
unnecessary services and use the fetch code where it belongs.
This commit also adds testing infrastructure for intercepting client calls
to AbstractQueryTestCase to support GET calls in query tests.
2015-09-11 09:07:18 +02:00
Colin Goodheart-Smithe
cac72c9207 Query Refactoring: Refactor of GeoPolygonQuery
Moving the query building functionality from the parser to the builders
new toQuery() method analogous to other recent query refactorings.

Relates to 

PR goes against the query-refactoring branch
2015-09-11 07:54:00 +01:00
Masaru Hasegawa
5ae00a6129 Take initializing shards into consideration during awareness allocation
It makes decision consistent.
Fixes 
2015-09-11 13:13:36 +09:00
Ryan Ernst
f3142e0c93 Merge pull request from rjernst/test_jar_cleanup
Reorganize sharing of constants for mock cluster info service
2015-09-10 21:10:29 -07:00
Jason Tedor
b5b22d4b01 Remove and forbid use of com.google.common.collect.Queues
This commit removes and now forbids all uses of
com.google.common.collect.Queues across the codebase. This is one of
many steps in the eventual removal of Guava as a dependency.

Relates 
2015-09-10 23:59:59 -04:00
Jason Tedor
65353b8a32 Remove and forbid use of com.google.common.collect.ImmutableSortedSet
This commit removes and now forbids all uses of
com.google.common.collect.ImmutableSortedSet across the codebase. This
is one of many steps in the eventual removal of Guava as a dependency.

Relates 
2015-09-10 23:33:55 -04:00
Robert Muir
af2df9aef6 Update monitor probe tests for java 9: this stuff is no longer accessible 2015-09-10 23:18:58 -04:00
Ryan Ernst
4a9f5bcf1d Tests: Reorganize sharing of constants for mock cluster info service
The MockInternalClusterInfoService depends on a constant and helper
method from actual test cases. This moves the constant and helper method
into the mock itself. Without this change, the classes put into the test
jar are not completely useable on their own (since this mock is in the
test jar, but the test cases are not).
2015-09-10 20:08:04 -07:00
Robert Muir
6ba4d132df Disable some exception serialization asserts on java 9, likely StackTraceElement has new stuff in it and we have to think about what to do 2015-09-10 22:45:53 -04:00
Robert Muir
552d5b0f65 Add java 9 support for bootclasspath to JvmInfo (it throws UOE otherwise which is not properly handled) 2015-09-10 22:15:27 -04:00
Robert Muir
c663d9b6a5 Merge branch 'jarhell2.0' into 1702090 2015-09-10 22:07:37 -04:00
Robert Muir
7aeff9f526 fix merge 2015-09-10 22:07:20 -04:00
Robert Muir
dd208002c9 Merge branch 'master' into 1702090 2015-09-10 21:41:07 -04:00
Robert Muir
6f59e3fca1 cutover jarhell to parsing classpath all the time 2015-09-10 21:31:25 -04:00
Jason Tedor
5d4d34ab16 Remove and forbid use of c.g.c.b.Preconditions#checkNotNull
This commit removes and now forbids all uses of
com.google.common.base.Preconditions#checkNotNull across the codebase.
This is one of many steps in the eventual removal of Guava as a
dependency.

Relates 
2015-09-10 17:57:00 -04:00
Robert Muir
c1f2fc76c2 Upgrade lucene to r1702090
The semantics of the `boost` parameter for `function_score` changed. This is
due to the fact that Lucene now requires that query boosts and top-level boosts
are applied the same way.
2015-09-10 23:36:43 +02:00
Nik Everett
e4981968ad [search] Limit the size of the result window
Requesting a million hits, or page 100,000 is always a bad idea, but users
may not be aware of this. This adds a per-index limit on the maximum size +
from that can be requested which defaults to 10,000.

This should not interfere with deep-scrolling.

Closes 
2015-09-10 15:38:29 -04:00
Robert Muir
6cdcc805cc hack jython tests to work on windows
windows needs access to the "root" holding the jar file (see https://github.com/int3/jython/blob/master/src/org/python/core/PySystemState.java#L571-L616)

Its different on windows, because when canonicalizing the file (case sensitivity), it needs access to the file.

Closes 
2015-09-10 14:59:23 -04:00
Ryan Ernst
ca8867a334 Test: Fix tests looking for old indexes to resolve against the correct dir 2015-09-10 11:07:02 -07:00
Lee Hinman
b21d3d2fb5 [TEST] Additional logging for testDelayedUnassignedScheduleReroute
Relates to 
2015-09-10 11:44:09 -06:00
Ryan Ernst
47de1bd923 Merge pull request from rjernst/simplify_bwc_path
Move static bwc indexes to a shared location
2015-09-10 10:31:50 -07:00
Jason Tedor
3bd1d38176 Remove and forbid use of com.google.common.collect.Sets
This commit removes and now forbids all uses of
com.google.common.collect.Sets across the codebase. This is one of many
steps in the eventual removal of Guava as a dependency.

Relates 
2015-09-10 11:22:09 -04:00
Martijn van Groningen
ab0847e0df parent/child: several cleanups
* Dropped ScoreType in favour of Lucene's ScoreMode
* Removed `score_type` option from `has_child` and `has_parent` queries in favour for the already existing `score_mode` option.
* Removed the score mode `sum` in favour for the already existing `total` score mode. (`sum` doesn't exist in Lucene's ScoreMode class)
* If `max_children` is set to `0` it now really means that zero children are allowed to match.
2015-09-10 17:15:41 +02:00
Nik Everett
60a2dd7020 Merge pull request from nik9000/fix_nodeattrs_test
Fix test for _cat/nodeattrs
2015-09-10 11:12:01 -04:00
Nik Everett
56c3471851 Fix test for _cat/nodeattrs
Adds a node attribute to all test runs and uses the attribute to test
`_cat/nodeattrs`.

Note that its quite possible create an impressively slow regex while doing
this and you have to be careful. See comment in commit for more if curious.

Closes 
2015-09-10 10:50:51 -04:00
Colin Goodheart-Smithe
53d29e51e0 Aggregations: Pipeline Aggregations at the root of the agg tree are now validated
Previously PipelineAggregatorFactory's at the root to the agg tree (top-level aggs) were not validated. This commit adds a call to PipelineAggregatoFactory.validate() for that case.

Closes 
2015-09-10 15:44:52 +01:00
Nik Everett
3839d15ea0 Merge branch 'pr/13130' 2015-09-10 10:13:01 -04:00
javanna
2e68738429 adapt to upstream changes 2015-09-10 16:04:55 +02:00
javanna
56b3db6ba3 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/AndQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/AndQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/FQueryFilterParser.java
	core/src/main/java/org/elasticsearch/index/query/FilteredQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/FilteredQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/IndicesQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/LimitQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/LimitQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MultiMatchQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/QueryFilterBuilder.java
	core/src/main/java/org/elasticsearch/index/query/QueryFilterParser.java
	core/src/main/java/org/elasticsearch/index/query/QueryParseContext.java
	core/src/main/java/org/elasticsearch/index/query/ScriptQueryParser.java
	core/src/main/java/org/elasticsearch/indices/query/IndicesQueriesRegistry.java
	core/src/main/java/org/elasticsearch/search/sort/GeoDistanceSortParser.java
	core/src/main/java/org/elasticsearch/search/sort/SortParseElement.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchShortCircuitBenchmark.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
2015-09-10 15:52:28 +02:00
Martijn van Groningen
a0fea6d365 set query cache to null 2015-09-10 15:19:12 +02:00
André Carvalho
9802c38fa7 [Stats] Adds counter for failed indexing requests
Adds the stats to the _cat/indices, _cat/shards, and the _stats apis.

Closes 
2015-09-10 09:03:24 -04:00
Britta Weber
c10f116a84 [test] don't catch AssertionError from indexRandom(). just index and catch MapperParsingException 2015-09-10 14:51:23 +02:00