Commit Graph

27783 Commits

Author SHA1 Message Date
Boaz Leskes a474a13911 testRerouteRecovery should use assertBusy when checking recovery counters
The test check that the number of outgoing/incoming recoveries of a shard is 0 after recoveries were done. Sadly that is not guaranteed by the current recovery logic as we decrement the counters only when all references to the relevant RecoveryTarget object have been released. This may happen in an async fashion to the recovery completion which causes the test to fail. I looked at options to change the recovery logic to have the recovery counters decrease before the recovery is done *under normally circumstances* but I don't see a clean way to do it. Since it won't give hard guarantees anyway I opted to add assertBusy to the test

Closes #24669
2017-05-16 11:09:52 +02:00
Ryan Ernst d966034298 Scripting: Deprecate index lookup (#24691)
This commit adds a deprecation warning if `_index` is used in scripts.
It is emitted each time a script is invoked, but not per document. There
is no test because constructing a LeafIndexLookup is quite difficult,
but the deprecation warning does show up in IndexLookupIT, there is just
no way to assert warnings in integ tests.

relates #19359
2017-05-16 01:41:29 -07:00
Simon Willnauer f9cfe86320 Make RemoteClusterConnectionTests more robust against cancelable threads aborts
Today we assert hart if failure listeners are invoked more than once. Yet, this
can happen if we cancel the execution since the caller and the handler will get
the exception on the cancelable threads and will notify the listener concurrently
if timinig allows. This commit relaxes the assertion towards handling multiple
invocations with `ExecutionCancelledException`

Closes #24010
Closes #24179
Closes vagnerclementino/elasticsearch/#98
2017-05-16 10:01:17 +02:00
Ryan Ernst 6ce597a378 Scripts: Convert template script engines to return String instead of BytesReference (#24447)
Template script engines (mustache, the only one) currently return a
BytesReference that users must know is utf8 encoded. This commit
modifies all callers and mustache to have the template engine return
String. This is much simpler, and does not require decoding in order to
use (for example, in ingest).
2017-05-15 22:37:31 -07:00
Ryan Ernst 548a5c1386 Docs: Add migration note about .yaml and .json removal (#24689)
relates #19391
relates #24633
2017-05-15 13:42:28 -07:00
Jack Conradson 43292979fd Add New Security Script Settings (#24637)
Settings are simplified to allowed_types and allowed_contexts.  If a setting is not specified the default is to enable all for that setting.
2017-05-15 13:37:46 -07:00
Ryan Ernst 2e6dc04025 Re-enable centos-6 2017-05-15 13:00:24 -07:00
Ryan Ernst f1fd5350c9 Test: Fix how packaging test generates expected plugins 2017-05-15 12:58:11 -07:00
Igor Motov 243635222a Move ReindexAction class to core (#24684)
This class is also needed for plugins to use reindex functionality.

Relates to #24578
2017-05-15 14:28:59 -04:00
Vlad Holubiev 557390d7d1 Fix typo in example (grades_count -> types_count) (#24635)
Looks like `doc.grade` was used for examples before. But not anymore - https://www.elastic.co/guide/en/elasticsearch/reference/2.4/search-aggregations-metrics-valuecount-aggregation.html
2017-05-15 14:08:46 -04:00
Jason Tedor 92ba969804 Remove Jython hacks
We had a hack in setting up permissions for tests to support testing
the lang-python plugin. We also had a hack to prevent Log4j from
loading a shaded version of Jansi provided by Jython. This plugin has
been removed so these hacks are no longer necessary.

Relates #24681
2017-05-15 12:40:24 -04:00
Luca Cavanna 563e7ddc83 Pass over _routing value with more_like_this items to be retrieved (#24679)
When retrieving documents to extract terms from as part of a more like this query, the _routing value can be set, yet it gets lost. That leads to not being able to retrieve the documents, hence more_like_this used to return no matches all the time.

Closes #23699
2017-05-15 18:15:42 +02:00
Luca Cavanna e14ba81ac1 Make ObjectParser support string to boolean conversion (#24668)
We generally accept string values when a boolean is expected. We've been doing that in our parsing code, but we missed that bit when moving parsing code to ObjectParser, which throws an error instead. This commit makes ObjectParser parse also string values into booleans. It throws an error in case the value is not `true` or `false`.

Closes #21802
2017-05-15 18:15:20 +02:00
Tanguy Leroux 5fb04fa603 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-05-15 17:08:26 +02:00
Tanguy Leroux dcb63ab8bc Share XContent rendering code in significant terms aggregations (#24677)
The rendering methods in String and Long Significant String aggregations
 and buckets are very similar. They can be factored out in the
 InternalSignificantTerms class an InternalMappedSignificantTerms class.
2017-05-15 16:52:02 +02:00
Christoph Büscher 60505c9100 Add parsing for InternalFilters aggregation (#24648)
This adds parsing to the InternalFilters aggregation.
2017-05-15 15:26:35 +02:00
Tanguy Leroux 578223f679 Make SignificantTerms.Bucket an interface rather than an abstract class (#24670)
This commit changes SignificantTerms.Bucket so that it is not an
abstract class anymore but an interface. It will be easier for the Java
High Level Rest Client to provide its own implementation of
SignificantTerms and SignificantTerms.Bucket. Also, it is now more
coherent with the others aggregations.
2017-05-15 15:14:07 +02:00
Christoph Büscher 0b688a8733 Small improvement in InternalAggregationTestCase test setup after changes in master (#24675) 2017-05-15 15:06:01 +02:00
Jason Tedor bd5aee8cfa Split disruption test suite
The disruption tests sit in a single test suite which causes these tests
to be single-threaded. We can split this test suite into multiple suites
(logically, of course) enabling them to be run in parallel reducing the
total run time of all integration tests in core. This commit splits the
discovery with service disruptions test suite into three suites
 - master disruptions
 - discovery disruptions
 - cluster disruptions

The last one could probably be better named, it is meant to represent
performing actions in the cluster (indexing, failing a shard, etc.)
while a disruption is taking place.

Relates #24662
2017-05-15 07:47:42 -04:00
Christoph Büscher 42e8d4b761 Merge branch 'master' into feature/client_aggs_parsing
Conflicts:
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/filter/InternalFilterTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/global/InternalGlobalTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/InternalMissingTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/InternalNestedTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/InternalReverseNestedTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/sampler/InternalSamplerTests.java
	modules/parent-join/src/test/java/org/elasticsearch/join/aggregations/InternalChildrenTests.java
	test/framework/src/main/java/org/elasticsearch/search/aggregations/InternalSingleBucketAggregationTestCase.java
2017-05-15 12:25:07 +02:00
Christoph Büscher bb59ee51b0 Revert changing the InternalSampler type constant (#24667) 2017-05-15 11:49:47 +02:00
Kunal Kapoor fdb6cd8088 Deprecate use of + in index names (#24585)
Use of '+' in index names is implicit. There is no need to support it. This commit deprecates support for it.

Closes #24515
2017-05-15 11:26:51 +02:00
Yannick Welsch bd364c56a8 Mute packagingTest on CentOS 6
Relates #24645
2017-05-15 11:01:35 +02:00
Koen De Groote f185b69e04 Replace manual copying an array or collection with static methods calls (#24657) 2017-05-15 09:35:48 +02:00
Ryan Ernst 17f8d2debd Settings: Remove support for yaml and json config files (#24664)
This commit removes the deprecated support for .yaml and .json files. If
the files still exist, the node will fail to start, indicating the file
must be converted or renamed.

closes #19391
2017-05-14 17:45:04 -07:00
Ryan Ernst e6535bc771 Settings: Update settings deprecation from yml to yaml (#24663)
This converts the deprecation added for 5.5 from .yml to .yaml. Note
that this deprecation has not been released yet.

relates #19391
2017-05-14 17:40:50 -07:00
Jason Tedor 4e21a33689 Clarify disabling swap in docs
Our strong recommendation is disabling swap over any other alternative
to avoid the JVM from landing on disk. This commit clarifies the docs in
this regard.
2017-05-12 16:09:52 -04:00
Jason Tedor 5da940532d Remove Netty logging hack (#24653)
Netty removed a logging guarded we added to prevent a scary logging
message. We added a hack to work around this. They've added the guard
back, so we can remove the hack now.
2017-05-12 16:05:13 -04:00
Jason Tedor 458129a85a Upgrade to Netty 4.1.11.Final
This commit upgrades the Netty dependency from 4.1.10.Final to
4.1.11.Final.

Relates #24652
2017-05-12 15:53:51 -04:00
Christoph Büscher 322a4c18dc Fix msearch rest test using typed_keys 2017-05-12 21:11:21 +02:00
Jim Ferenczi e804817ce4 #24638 Add the parent-join plugin in the prebuilt transport client by default 2017-05-12 17:25:34 +02:00
Jim Ferenczi 4291880a29 [Tests] #24638 Fix rest high level client docs build by restoring hasChild and hasParent in the tests 2017-05-12 17:03:26 +02:00
Tanguy Leroux b9d2ecc3ea Add parsing methods to Range aggregations (#24583) 2017-05-12 16:52:47 +02:00
Tanguy Leroux f8df2a22e9 SniffNodesSampler should close connection after handling responses (#24632)
With the current implementation, SniffNodesSampler might close the
current connection right after a request is sent but before the response
is correctly handled. This causes to timeouts in the transport client
when the sniffing is activated.

closes #24575
closes #24557
2017-05-12 16:38:50 +02:00
Koen De Groote 878ae8eb3c Size lists in advance when known
When constructing an array list, if we know the size of the list in
advance (because we are adding objects to it derived from another list),
we should size the array list to the appropriate capacity in advance (to
avoid resizing allocations). This commit does this in various places.

Relates #24439
2017-05-12 10:36:13 -04:00
Simon Willnauer e8e2ccdcf5 Call onClose listener in a finally block 2017-05-12 16:32:29 +02:00
Jim Ferenczi 279a18a527 Add parent-join module (#24638)
* Add parent-join module

This change adds a new module named `parent-join`.
The goal of this module is to provide a replacement for the `_parent` field but as a first step this change only moves the `has_child`, `has_parent` queries and the `children` aggregation to this module.
These queries and aggregations are no longer in core but they are deployed by default as a module.

Relates #20257
2017-05-12 15:58:06 +02:00
Tanguy Leroux 29a5694bb7 Add parsing method to GeoHashGrid aggregation (#24589) 2017-05-12 15:44:39 +02:00
Simon Willnauer be2a6ce80b Notify onConnectionClosed rather than onNodeDisconnect to prune transport handlers (#24639)
Today we prune transport handlers in TransportService when a node is disconnected.
This can cause connections to starve in the TransportService if the connection is
opened as a short living connection ie. without sharing the connection to a node
via registering in the transport itself. This change now moves to pruning based
on the connections cache key to ensure we notify handlers as soon as the connection
is closed for all connections not just for registered connections.

Relates to #24632
Relates to #24575
Relates to #24557
2017-05-12 15:40:40 +02:00
Yannick Welsch 04e08f5e49 Simplify Discovery interface (#24608)
- Removes clusterState, getInitialClusterState and getMinimumMasterNodes methods from Discovery interface.
- Sets PingContextProvider in ZenPing constructor
- Renames state in ZenDiscovery to committedState
2017-05-12 14:08:14 +02:00
Dimitris Athanasiou b7976bd536 [TEST] Temporarily disable the secure fixture for hdfs tests (#24643)
This keeps failing the build so I am temporarily disabling it
until #24636 gets merged.
2017-05-12 12:58:30 +01:00
Christoph Büscher ee1303a9fc Merge branch 'master' into feature/client_aggs_parsing 2017-05-12 11:53:38 +02:00
Christoph Büscher f06467b6a0 [Tests] Add unit tests for InternalFilters (#24624)
Relates to #22278
2017-05-12 11:48:44 +02:00
Christoph Büscher 744b1afcb2 Merge branch 'master' into feature/client_aggs_parsing
Conflicts:
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/InternalHistogramTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/DoubleTermsTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/LongTermsTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/StringTermsTests.java
2017-05-12 10:55:05 +02:00
Byron Voorbach 70e14d5e3f Fix Source filtering in new field collapsing feature
Update ValueType of FetchSourceContext to OBJECT_ARRAY_BOOLEAN_OR_STRING to be in line with other source_filtering implementations
2017-05-12 09:38:16 +02:00
Koen De Groote 220bd2de6e Make a few class level fields static
This commit converts some final constant instance fields to class
fields.
2017-05-11 22:44:31 -04:00
Jim Ferenczi 7b7e15023a Add rest test for sliced scroll (#24630) 2017-05-12 00:07:24 +02:00
Nik Everett 4d86679f5a Disable reindex-from-old tests in windows
In windows we can't reliable git the pid so we skip the
reindex-from-remote tests from old versions of elasticsearch. This
is OK because we aren't really testing windows here anyway. It isn't
great, but should be safe.
2017-05-11 17:36:03 -04:00
Nik Everett a40c3a99c9 Reindex: don't duplicate _source parameter (#24629)
If the request asks for the `_source` stored field then don't
duplicate it when forcing the `_source` parameter to onto the
request for reindex-from-remote from versions before 1.0.

Closes #24628
2017-05-11 16:30:06 -04:00
Ryan Ernst f477a6472d Settings: Deprecate settings in .yml and .json (#24059)
This commit adds a deprecation warning when elasticsearch.yml or
elasticsearch.json is read during startup.

relates #19391
2017-05-11 13:11:18 -07:00