7433 Commits

Author SHA1 Message Date
Ryan Ernst
cd6e3f4cea Merge branch 'master' into keystore 2017-01-06 09:32:08 -08:00
Ryan Ernst
42ebfe7bdb fix NPE 2017-01-06 09:11:07 -08:00
Tim B
b9c2c2f6f0 Move IfConfig.logIfNecessary call into bootstrap (#22455)
This is related to #22116. A logIfNecessary() call makes a call to
NetworkInterface.getInterfaceAddresses() requiring SocketPermission
connect privileges. By moving this to bootstrap the logging call can be
made before installing the SecurityManager.
2017-01-06 11:10:53 -06:00
Simon Willnauer
79093e1663 Ensure shrunk indices carry over version information from its source (#22469)
Today when an index is shrunk the version information is not carried over
from the source to the target index. This can cause major issues like mapping
incompatibilities for instance if an index from a previous major version is shrunk.

This commit ensures that all version information from the soruce index is preserved
when a shrunk index is created.

Closes #22373
2017-01-06 16:36:43 +01:00
Simon Willnauer
56082d1028 add more tests for RemoteClusterService 2017-01-06 12:35:18 +01:00
Simon Willnauer
418ec62bfb Merge branch 'master' into feature/multi_cluster_search 2017-01-06 10:24:40 +01:00
Ryan Ernst
eb596d7270 more renames 2017-01-06 01:03:45 -08:00
Ryan Ernst
6e406aed2d addressing more PR comments 2017-01-06 00:49:18 -08:00
javanna
5daf46286e remove ParseFieldMatcher usages from InternalSearchHit 2017-01-05 19:33:04 +01:00
javanna
975fee402a remove ParseFieldMatcher usages from suggesters 2017-01-05 19:33:04 +01:00
javanna
13dcb8ccbe remove ParseFieldMatcher usages from IngestMetadata 2017-01-05 19:33:04 +01:00
javanna
d60e9bddd0 remove ParseFieldMatcher usages from IndexGraveyard 2017-01-05 19:33:04 +01:00
javanna
d87a30647b remove ParseFieldMatcher usages from SearchAfterBuilder 2017-01-05 19:33:04 +01:00
javanna
723bdc4549 remove ParseFieldMatcher usages from FetchSourceContext 2017-01-05 19:33:04 +01:00
javanna
6102523033 remove ParseFieldMatcher usages from Script parsing code 2017-01-05 19:33:04 +01:00
javanna
6b9a8db069 fix unchecked generics warnings in ObjectParser 2017-01-05 19:33:04 +01:00
javanna
1f7960aa52 ObjectParser to no longer require ParseFieldMatcherSupplier as its Context
ParseFieldMatcher as well as ParseFieldMatcherSupplier will be soon removed, hence the ObjectParser's context doesn't need to be a ParseFieldMatcherSupplier anymore. That will allow to remove ParseFieldMatcherSupplier's implementations, little by little.
2017-01-05 19:33:04 +01:00
javanna
9394792392 remove unused ParseFieldMatcher imports/arguments 2017-01-05 19:33:04 +01:00
Yannick Welsch
182e8115de [TEST] Fix IndexRecoveryIT.testDisconnectsDuringRecovery
The test currently checks that the recovering shard is not failed when it is not a primary relocation that has moved past the finalization step.
Checking if it has moved past that step is done by intercepting the request between the replication source and the target and checking if it has seen
then WAIT_FOR_CLUSTERSTATE action as this is the next action that is called after finalization. This action can, however, occur only after the shard was
already failed, and thus trip the assertion. This commit changes the check to look out for the FINALIZE action, independently of whether it succeeded or not.
2017-01-05 19:12:21 +01:00
Yannick Welsch
cfc106d721 Don't close store under CancellableThreads (#22434)
#22325 changed the recovery retry logic to use unique recovery ids. The change also introduced an issue, however, which made it possible for the shard store to be closed under CancellableThreads, triggering assertions in the node locking logic. This commit limits the use of CancellableThreads only to the part where we wait on the old recovery target to be closed.
2017-01-05 18:11:58 +01:00
Simon Willnauer
349ea0f9b6 cut over to use : instead of | for cross cluster search 2017-01-05 17:03:12 +01:00
Simon Willnauer
dca54734ac add basic docs 2017-01-05 16:10:34 +01:00
Simon Willnauer
0183b0c5a8 More cleanups 2017-01-05 15:23:55 +01:00
Simon Willnauer
1ef7115bbd Add javadocs to TransportActionProxy 2017-01-05 14:13:08 +01:00
Simon Willnauer
7b95c2f54c document and test concurrent remote cluster node discovery 2017-01-05 14:07:56 +01:00
Adrien Grand
97f3a9bd79 Relax LiveVersionMapTests.testRamBytesUsed.
With Java9's new restrictions we cannot compute ram usage as accurately as
before. See https://issues.apache.org/jira/browse/LUCENE-7595.
2017-01-05 11:33:24 +01:00
Simon Willnauer
80bf01d3c0 Merge branch 'master' into feature/multi_cluster_search 2017-01-05 08:00:03 +01:00
Simon Willnauer
a5daa5d3a2 Execute low level handshake in #openConnection (#22440)
Today we execute the low level handshake on the TCP layer in #connectToNode.
If #openConnection is used directly, which is truly expert, no handshake is executed
which allows connecting to nodes that are not necessarily compatible. This change
moves the handshake to #openConnection to prevent bypassing this logic.
2017-01-05 07:32:53 +01:00
Ryan Ernst
bf51522788 Add 5.3 version 2017-01-04 15:00:43 -08:00
Simon Willnauer
daf1f53c39 Make RemoteClusterConnectionIT a unit test 2017-01-04 21:36:53 +01:00
Ali Beyad
9d422c1c34 IndicesService handles all exceptions during index deletion (#22433)
Previously, we could run into a situation where attempting to delete an
index due to a cluster state update would cause an unhandled exception
to bubble up to the ClusterService and cause the cluster state applier
to fail. The result of this situation is that the cluster state never
gets updated on the ClusterService because the exception happens before
all cluster state appliers have completed and the ClusterService only
updates the cluster state once all cluster state appliers have
successfully completed.

All other methods on IndicesService properly handle all exceptions and
not just IOExceptions, but there were two instances with respect to
index deletion where only IOExceptions where handled by the
IndicesService. If any other exception occurred during these delete
operations, the exception would be bubbled up to the ClusterService,
causing the aforementioned issues.

This commit ensures all methods in IndicesService properly capture all
types of Exceptions, so that the ClusterService manages to update the
cluster state, even in the presence of shard creation/deletion failures.

Note that the lack of updating the cluster state in the presence of such
exceptions can have many unintended consequences, one of them being
the tripping of the assertion in IndicesClusterStateService#removeUnallocatedIndices
where the assumption is that if there is an IndexService to remove with
an unassigned shard, then the index must exist in the cluster state, but if
the cluster state was never updated due to the aforementioned exceptions,
then the cluster state will not have the index in question.
2017-01-04 13:16:49 -06:00
Adrien Grand
f8998fece5 Upgrade to lucene-6.4.0-snapshot-084f7a0. (#22413) 2017-01-04 19:03:52 +01:00
Simon Willnauer
e642965804 Cleanup lots of code, add javadocs and tests 2017-01-04 17:26:00 +01:00
Simon Willnauer
dd0331144a [TEST] Also register replica node in node map 2017-01-04 13:31:59 +01:00
Simon Willnauer
31499a1248 handle nodes that are not connected early in AbstractSearchAsyncAction 2017-01-04 11:23:33 +01:00
Jim Ferenczi
360ce532eb Implement stats for geo_point and geo_shape field (#22391)
Currently `geo_point` and `geo_shape` field are treated as `text` field by the field stats API and we
try to extract the min/max values with MultiFields.getTerms.
This is ok in master because a `geo_point` field is always a Point field but it can cause problem in 5.x (and 2.x) because the legacy
 `geo_point` are indexed as terms.
 As a result the min and max are extracted and then printed in the FieldStats output using BytesRef.utf8ToString
 which can throw an IndexOutOfBoundException since it's not valid UTF8 strings.
 This change ensure that we never try to extract min/max information from a `geo_point` field.
 It does not add a new type for geo points in the fieldstats API so we'll continue to use `text` for this kind of field.
 This PR is targeted to master even though we could only commit this change to 5.x. I think it's cleaner to have it in master too before we make any decision on
  https://github.com/elastic/elasticsearch/pull/21947.

Fixes #22384
2017-01-04 10:42:22 +01:00
Ryan Ernst
4e4a40df7a feedback 2017-01-03 15:42:38 -08:00
Jason Tedor
c6ddff757e Cleanup some comments in IndexShard.java
This commit cleans up the comments in IndexShard related to sequence numbers, making
them uniform in their formatting and taking advantage of the line-length
limit of 140 characters.
2017-01-03 15:10:38 -05:00
Jason Tedor
9a65d2008e Cleanup comments in GlobalCheckpointService.java
This commit cleans up the comments in GlobalCheckpointService, making
them uniform in their formatting and taking advantage of the line-length
limit of 140 characters.
2017-01-03 12:23:33 -05:00
Jason Tedor
64888ab1d3 Cleanup comments in SequenceNumbersService.java
This commit cleans up the comments in SequenceNumbersService, making
them uniform in their formatting and taking advantage of the line-length
limit of 140 characters.
2017-01-03 12:06:43 -05:00
Ali Beyad
6f242920d3 [TEST] only check node decisions if not in the AWAITING_INFO state 2017-01-03 11:39:40 -05:00
Simon Willnauer
422cd1ef77 Add support for proxy nodes
this commit adds full support for proxy nodes on the search layer.
This allows to connection only to a small set of nodes on a remote cluster
to exectue the search. The nodes will proxy the request to the correct node in the
cluster while the coordinting node doesn't need to be connected to the target node.
2017-01-03 17:24:32 +01:00
javanna
ee4dde46d3 Remove ParseFieldMatcher usages from Aggregator 2017-01-03 15:52:32 +01:00
javanna
8b8ff8b9e2 Remove ParseFieldMatcher usages from SearchService 2017-01-03 15:52:32 +01:00
javanna
6329a98a97 Remove ParseFieldMatcher usages from SearchContext 2017-01-03 15:52:32 +01:00
javanna
77f4152a18 Remove ParseFieldMatcher usages from a couple of Rest Actions 2017-01-03 15:52:32 +01:00
javanna
0d67891a64 Remove ParseFieldMatcher usages from QueryParsers#parseRewriteMethod 2017-01-03 15:52:32 +01:00
javanna
40540b3f3f Remove unused QueryParsers#setRewriteMethod 2017-01-03 15:52:32 +01:00
javanna
648ed46f01 Remove ParseFieldMatcher usages from MoreLikeThisQueryBuilder & MultiMatchQueryBuilder 2017-01-03 15:52:32 +01:00
javanna
c06d00dce1 Remove ParseFieldMatcher usage from SearchRequest 2017-01-03 15:52:32 +01:00