Commit Graph

8191 Commits

Author SHA1 Message Date
javanna 7a3e38eb8e Merge branch 'master' into feature/client_aggs_parsing 2017-05-22 12:25:14 +02:00
Luca Cavanna 726e6c45ab Move getType to Aggregation interface (#24822)
Given that both InternalAggregation and ParsedAggregation have this method, it makes sense to move it to the interface they both implement.
2017-05-22 10:49:23 +02:00
olcbean e08e92d934 Deleting a document from a non-existing index creates the should not auto create it, unless using EXTERNAL* versioning (#24518)
Currently a `delete document` request against a non-existing index actually **creates** this index.

With this change the `delete document` no longer creates the previously non-existing index and throws an `index_not_found` exception instead.

However as discussed in https://github.com/elastic/elasticsearch/pull/15451#issuecomment-165772026, if an external version is explicitly used, the current behavior is preserved and the index is still created and the document is marked for deletion.

Fixes #15425
2017-05-22 10:00:22 +03:00
Ryan Ernst 679ec99fad Scripting: Simplify search method on script service (#24817)
This commit is a simple cleanup to remove an unnecessary extra method on
ScriptService which was only used in 3 places. There is now only one
search method.
2017-05-20 23:46:53 -07:00
Ryan Ernst 2de748859f Scripting: Remove "inline script enabled" on script engines (#24815)
ScriptEngine implementations have an overridable method to indicate they
are safe to use as inline scripts. Since groovy was removed fro 6.0,
there are no longer any implementations which used the default false
value. Furthermore, the value was not actually read anywhere. This
commit removes the method. The ScriptEngineRegistry was also no longer
necessary as it only was used to build a map from language to engine.
2017-05-20 12:01:25 -07:00
Jason Tedor 3666092099 Remove unnecessary method from index shard
This commit removes a convenience method from index shard that is used
at exactly one call site. This method is used to callback a listener
when an operation is on too old of a primary term. Since it is only used
at one call site, we simply inline the method.
2017-05-20 09:44:09 -04:00
Jason Tedor 4cd70cf986 Block older operations on primary term transition
Today a replica learns of a new primary term via a cluster state update
and there is not a clean transition between the older primary term and
the newer primary term. This commit modifies this situation so that:
 - a replica shard learns of a new primary term via replication
   operations executed under the mandate of the new primary
 - when a replica shard learns of a new primary term, it blocks
   operations on older terms from reaching the engine, with a clear
   transition point between the operations on the older term and the
   operations on the newer term

This work paves the way for a primary/replica sync on primary
promotion. Future work will also ensure a clean transition point on a
promoted primary, and prepare a replica shard for a sync with the
promoted primary.

Relates #24779
2017-05-19 16:17:22 -04:00
Jason Tedor cb7a8d5876 Add version 5.3.3 constant
This commit adds the version 5.3.3 constant to the master branch.
2017-05-19 15:18:14 -04:00
javanna db0490343e Merge branch 'master' into feature/client_aggs_parsing 2017-05-19 18:17:06 +02:00
Nik Everett b9ea579633 Allow plugins to register pre-configured tokenizers (#24751)
Allows plugins to register pre-configured tokenizers. Much
of the decisions are the same as those in #24223, #24572,
and #24223. This only migrates the lowercase tokenizer but
I figure that is a good start because it proves out the features.
2017-05-19 12:07:04 -04:00
Luca Cavanna ae73670257 Mark SearchSourceBuilder as ToXContentObject (#24795)
SearchSourceBuilder#toXContent prints out a complete object, hence it should implement ToXContentObject
2017-05-19 18:00:58 +02:00
Nicholas Knize deb7caf4d3 Upgrade to lucene-7.0.0-snapshot-a0aef2f
This commit upgrades master to a current lucene snapshot with commit id a0aef2f.
2017-05-19 10:20:55 -05:00
Jim Ferenczi d241c4898e Removes parent child fielddata specialization (#24737)
This change removes the field data specialization needed for the parent field and replaces it with
a simple DocValuesIndexFieldData. The underlying global ordinals are retrieved via a new function called
IndexOrdinalsFieldData#getOrdinalMap.
The children aggregation is also modified to use a simple WithOrdinals value source rather than the deleted WithOrdinals.Parent.

Relates #20257
2017-05-19 17:11:23 +02:00
Simon Willnauer b17d23dc99 Obey lock order if working with store to get metadata snapshots (#24787)
Today when we get a metadata snapshot from the index shard we ensure
that if there is no engine started on the shard that we lock the index
writer before we go and fetch the store metadata. Yet, if we concurrently
recover that shard, recovery finalization might fail since it can't acquire
the IW lock on the directory. This is mainly due to the wrong order of aquiring
the IW lock and the metadata lock. Fetching store metadata without a started engine
should block on the metadata lock in Store.java but since IndexShard locks the writer
first we get into a failed recovery dance especially in test. In production
this is less of an issue since we rarely get into this siutation if at all.

Closes #24481
2017-05-19 15:36:39 +02:00
Luca Cavanna 0b8da31ccf SearchResponse#fromXContent to not require START_OBJECT as current token (#24794)
The method should rather advance one token and only then require a START_OBJECT as the current token. This allows to parse given a parser that's at the beginning of the response, where the initial/current token is null.
2017-05-19 15:24:29 +02:00
Tanguy Leroux ad3658a8a3 [Test] Remove ParsedAggregationTests (#24791)
Now the Java High Level Rest Client has tests to parse all aggregations,
 this test is not needed anymore. We have better tests like
 AggregationsTests and sub classes of InternalAggregationTestCase.

 Related to #23965
2017-05-19 13:56:58 +02:00
Christoph Büscher 05acc43505 Fix InternalExtendedStatsTests setUp 2017-05-19 13:37:31 +02:00
Tanguy Leroux 91d7bf474f Remove compareTerm() method in parsed Significant Terms aggregations
This method has been removed in core (see #24714)
2017-05-19 13:20:45 +02:00
Tanguy Leroux 83aa00b3f6 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-05-19 13:13:00 +02:00
Tanguy Leroux 55af1f7a2b [Test] Remove leftover in InternalRangeTests 2017-05-19 13:12:01 +02:00
Tanguy Leroux 4c34ea8fc8 Remove //norelease and cleans up somet aggregations tests (#24789) 2017-05-19 12:46:37 +02:00
Yannick Welsch 7054d24f0d Clear responsibilities for PublishClusterStateAction and ZenDiscovery (#24772)
This commit moves some functionality from PublishClusterStateAction to ZenDiscovery, which allows each class to focus on it's core competencies:
- PendingStatesQueue is now solely managed by ZenDiscovery (no shared access by both PublishClusterStateAction and ZenDiscovery)
- Validation logic is handled exclusively by ZenDiscovery
2017-05-19 09:34:23 +02:00
Jack Conradson 1196dfb6bb Remove Deprecated Script Settings (#24756)
Removes all fine-grained script settings replaced by scripts.types_allowed and scripts.contexts_allowed.
2017-05-18 13:32:46 -07:00
Jason Tedor fe83df2a50 Refactor update shard logic for primaries
This commit is a simple refactoring of the update shard logic for
primaries. Namely, there was some duplicated code here that was annoying
to have to read twice so it is now collapsed with this commit.
2017-05-18 15:15:45 -04:00
Boaz Leskes 3d5154191c TransportShardBulkAction: s/tha that/that/ 2017-05-18 21:15:17 +02:00
Boaz Leskes ff374f1d0d TransportShardBulkAction: remove hard version assertion and improve todo comment
We have decided not to force a future version upgrade to deal with this todo. Rather, we'll keep the code until its in our way / the opportunity arises to deal with it.
2017-05-18 20:59:08 +02:00
Nik Everett 82d2c7a142 Remove vagrant testing versions (#24754)
Now that we generate the versions list from Versions.java we can
drop the list of versions maintained for vagrant testing. One nice
thing that the vagrant testing did was to check if the list of
versions was out of date. This moves that test to the core
project.
2017-05-18 09:33:13 -04:00
Tanguy Leroux 96dac4c670 Remove the unused SignificantTerms.compareTerm() method (#24714)
This method is not used and not tested. While it exists it forces
implementations of the interface to implement it while it's unused.
2017-05-18 14:57:12 +02:00
Koen De Groote 905eb422f6 Use StringBuilder to construct a String instead of relying on appending where possible (#24753)
This PR revolves around places in the code where introducing a StringBuilder might make the construction
of a String easier to follow and also, maybe avoid a case where the compiler's very safe way of introducing 
StringBuilder instead of String might not always be optimal for performance.
2017-05-18 12:02:29 +02:00
Simon Willnauer 831c497638 [TEST] Fix compatibilty tests for wire compatibility based on Version.java 2017-05-18 11:21:50 +02:00
Christoph Büscher f658a8d137 DateHistogram: Fix 'extended_bounds' with 'offset' (#23789)
This fixes a bug in the 'date_histogram' aggregation that can happen when using 'extended_bounds'
together with some 'offset' parameter. Offsets should be applied after rounding the extended bounds 
and also be applied when adding empty buckets during the reduce phase in InternalDateHistogram.

Closes #23776
2017-05-18 11:07:48 +02:00
Tanguy Leroux eeef2e6c31 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-05-18 09:43:57 +02:00
Tanguy Leroux f10391e16e Fix checkstyle violation in ParsedScriptedMetric 2017-05-18 09:41:04 +02:00
Tanguy Leroux 055875392e Add parsing method for Top Hits aggregation (#24717)
Related to #23331
2017-05-18 09:37:46 +02:00
Tanguy Leroux 25fceb8c0f Add parsing method for binary range aggregation (#24706)
Related to #23331
2017-05-18 09:24:33 +02:00
Koen De Groote d744d77f61 Fix String concatenation within a StringBuilder append chain
This commit replaces String concatenation within a StringBuilder append chain by using explicit append calls.
2017-05-18 08:51:30 +02:00
Ryan Ernst 26e2e933f5 Scripting: Remove native scripts (#24726)
Native scripts have been replaced in documentation by implementing
a ScriptEngine and they were deprecated in 5.5.0. This commit
removes the native script infrastructure for 6.0.

closes #19966
2017-05-17 14:49:24 -07:00
Ryan Ernst 463fe2f4d4 Scripting: Remove file scripts (#24627)
This commit removes file scripts, which were deprecated in 5.5.

closes #21798
2017-05-17 14:42:25 -07:00
Ryan Ernst f8a48badcf Settings: Remove shared setting property (#24728)
Shared settings were added intially to allow the few common settings
names across aws plugins. However, in 6.0 these settings have been
removed. The last use was in netty, but since 6.0 also has the netty 3
modules removed, there is no longer a need for the shared property. This
commit removes the shared setting property.
2017-05-17 13:14:12 -07:00
Luca Cavanna da669f0554 Add fromXContent method to SearchResponse (#24720)
SearchResponse#fromXContent allows to parse a search response, including search hits, aggregations, suggestions and profile results. Only the aggs that we can parse today are supported (which means all of them but a couple that are left to support). SearchResponseTests reuses the existing test infra to randomize aggregations, suggestions and profile response.

Relates to #23331
2017-05-17 20:28:33 +02:00
Jay Modi 7ce0b1b7c6 Allow SearchOperationListeners to validate a search context (#24650)
This commit adds a new method to the SearchOperationListener that allows implementers to validate
the SearchContext immediately after it is retrieved from the active contexts. The listener may
throw a runtime exception if it deems the SearchContext is not valid and that the use of the context
should be terminated.
2017-05-17 13:07:57 -04:00
Lee Hinman 01bcbd868d [TEST] Fix TransportReplicationActionTests.testRetryOnReplica for replica request (#24745)
* [TEST] Fix TransportReplicationActionTests.testRetryOnReplica for replica request

We were improperly testing that it was a `ConcreteShardRequest` instead of a
`ConcreteReplicaRequest`. This adds that change and also ensures that the
checkpoint is retrievable from the request.

* Fix line-length
2017-05-17 10:58:50 -06:00
Christoph Büscher 9fc9db26fd Add parsing for InternalScriptedMetric aggregation (#24738) 2017-05-17 18:55:57 +02:00
javanna ce7326eb88 Merge branch 'master' into feature/client_aggs_parsing 2017-05-17 17:59:00 +02:00
Simon Willnauer 2ccc223ff7 Fix Version based BWC and set correct minCompatVersion (#24732)
Approaching the release of 6.0 we need to sort out the usage of
`Version#minimumCompatibilityVersion` which was still set to 5.0.0.
Now this change moves it to the latest released version of 5.x (5.4 at this point)
to ensure we are compatible with the latest minor of the previous major. This change
also removes all the `_UNRELEASED` from the versions that where released and drops versions
that were never released and are not expected to be released (bugfixes in minors that are not
the latest in the previous major).
2017-05-17 17:27:09 +02:00
Nik Everett f56d8676cf Fix GetTermVectorsIT
It was assuming that payloads were available in a context where
they weren't.

Relates to #24716
2017-05-17 09:32:07 -04:00
Jim Ferenczi 67c41d2e77 Fix ExpandSearchPhase when response contains no hits (#24688)
This change skips the expand search phase entirely when there is no search hits in the response.
2017-05-17 14:15:40 +02:00
Simon Willnauer cf846af0e5 Fix `_field_caps` serialization in order to support cross cluster search (#24722)
Today the `_field_caps` API doesn't implement its request serialization
correctly since indices and indices options are not serialized at all.
This will likely break with all transport clients etc. and if this request
must be send across the network. This commit fixes this and adds correct
handling if we have only remote indices to prevent the inclusion of
all local indices.
2017-05-17 14:02:45 +02:00
George Papadrosou 8a4d8909a1 Fix ArrayIndexOutOfBoundsException when no ranges are specified in the query (#23241)
* Fix ArrayIndexOutOfBoundsException in Range Aggregation when no ranges are specified in the query

* Revert "Fix ArrayIndexOutOfBoundsException in Range Aggregation when no ranges are specified in the query"

This reverts commit ad57d8feb3577a64b37de28c6f3df96a3a49fe93.

* Fix range aggregation out of bounds exception when there are no ranges in a range or date_range query

* Fix range aggregation out of bounds exception when there are no ranges in the query

This fix is applied to range queries, date range queries, ip range queries and geo distance aggregation queries
2017-05-17 10:34:01 +01:00
Simon Willnauer 11ea588aee Move BWC versin to 5.5 after backport
Relates to #24678
2017-05-16 21:01:19 +02:00