Commit Graph

25812 Commits

Author SHA1 Message Date
Nik Everett 1eddff822a Centralize remaining XContentParser creation in tests
Now we're ready to add `NamedXContentRegistry`.
2016-12-14 11:52:09 -05:00
Tanguy Leroux a8b238ebad Add equals() and hashCode() methods to ReplicationResponse (#22168)
This commit adds the equals() and hashCode() methods to ReplicationResponse.ShardInfo and ReplicationResponse.ShardInfo.Failure classes.
2016-12-14 17:37:52 +01:00
Tanguy Leroux 135f60502c Add parsing method for ElasticsearchException (#22143)
ElasticsearchException is used in various response objects like IndexResponse, DeleteResponse or BulkItemResponse.Failure. It would be helpful to the High Level Rest Client to be able to parse these exceptions back.

This commit adds the fromXContent() method to the ElasticsearchException object. This method does not return the original (wrapped or unwrapped) exception but always returns a ElasticsearchException that serves as a simple POJO for all types of exceptions. The parsed ElasticsearchException's message will be composed of the original exception type (ex: illegal_argument_exception) concatenated with the original reason to help users/clients to known and handle the error.
2016-12-14 16:11:44 +01:00
Yannick Welsch a511fb9ce6 Prefer joining node with conflicting transport address when becoming master (#22134)
PR #22049 changed the node update logic to never remove nodes from the cluster state when the cluster state is not published. This led to an issue electing a master (#22120) based on nodes with same transport address (but different node id) as previous nodes. The joining nodes should take precedence over conflicting ones. Note that this only applies to the action of becoming master. If a master is established and a node joins an existing master, it will be rejected if there is another node with same transport address.
2016-12-14 15:04:24 +01:00
Isabel Drost-Fromm c18f032b08 Fix typo in command for checking single doc file
The recommended command line option for running the doc check on a single documentation file contained a typo.

h/t to @nik9000 for finding the typo.
2016-12-14 13:44:57 +01:00
Isabel Drost-Fromm ea9f11965d Merge branch 'master' into feature/rank-eval 2016-12-14 12:47:12 +01:00
Jim Ferenczi 23d11ed729 Fix numeric terms aggregations with includes/excludes and minDocCount=0 (#22141)
For minDocCount=0 the numeric terms aggregator should also check the includes/excludes when buckets with empty count are added to the result.
This change fixes this bug and adds a test for it.

Fixes #22140
2016-12-14 12:37:11 +01:00
David Pilato 320dacd6d2 Update script metric aggregation for 5.0
From 5.0, we are now using painless.
2016-12-14 11:44:10 +01:00
David Pilato 11a6248344 Update script query doc for 5.1
From 5.1, we changed the order of Script class ctor.

Related to https://github.com/elastic/elasticsearch/pull/21321#issuecomment-266432519
2016-12-14 11:36:55 +01:00
David Pilato 87a016a155 Update search template doc for 5.1
From 5.1, we can't use anymore `ScriptService.ScriptType` but `ScriptType`.

Related to https://github.com/elastic/elasticsearch/pull/21136#issuecomment-266429243
2016-12-14 11:36:32 +01:00
Clinton Gormley 571b20137a Document using round-robin DNS for discovery (#21810)
* Document using round-robin DNS for discovery

Closes #8427

* Improve wording around round robin DNS

* Included jason's suggestion
2016-12-14 11:29:39 +01:00
Adrien Grand 149ef74b26 Fix `missing` on aggs on `boolean` fields. (#22135)
The creation of the `ValuesSource` used to pass `DateTimeZone.UTC` as a time
zone all the time in case of empty fields in spite of the fact that all doc
value formats but the date one reject this parameter.

This commit centralizes the creation of the `ValuesSource` and adds unit tests
to it.

Closes #22009
2016-12-14 10:03:09 +01:00
Daniel Mitterdorfer 7e5058037b Enable strict duplicate checks for JSON content
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default. This ensures that JSON keys are always unique. While this has
a performance impact, benchmarking has indicated that the typical drop in
indexing throughput is around 1 - 2%.

As a last resort, we allow users to still disable strict duplicate checks
by setting `-Des.json.strict_duplicate_detection=false` which is
intentionally undocumented.

Closes #19614
2016-12-14 09:35:53 +01:00
Nik Everett 49bdd29f91 Consolidate more parser creation into ESTestCase
This will make it easier to add the forthcoming required argument,
`NamedXContentRegistry`.
2016-12-13 20:28:41 -05:00
Jason Tedor 0d195f1afa Avoid corruption when deserializing booleans
Today we write 0x00 or 0x01 for false or true when serializing a boolean
(and 0x02 for null when serializing an optional boolean) but we
deserialize any non-zero byte to true (except when deserializing an
optional boolean in which case we deserialize 0x02 to null, 0x01 to
true, and any other non-zero byte to false). This too easily allows
corruption into the stream. Instead, we should mark the stream as
corrupted and stop deserializing. This catches when we try to
deserialize something as a boolean that is not a boolean.

Relates #22152
2016-12-13 20:10:05 -05:00
Jason Tedor 510ad7b9c7 Add shutdown hook for closing CLI commands
This commit enables CLI commands to be closeable and installs a runtime
shutdown hook to ensure that if the JVM shuts down (as opposed to
aborting) the close method is called.

It is not enough to wrap uses of commands in main methods in
try-with-resources blocks as these will not run if, say, the virtual
machine is terminated in response to SIGINT, or system shutdown event.

Relates #22126
2016-12-13 19:10:11 -05:00
Nik Everett 872984d21a Continue consolidating `XContentParser` construction in tests (#22145)
Consolidate more parser creation in tests

Moves more parser creation in tests to the `createParser` methods
in `ESTestCase`.
2016-12-13 17:22:39 -05:00
Simon Willnauer b7bcb5bb3a [TEST] Skip term / int partitioning tests in bwc tests pre 5.2.0 2016-12-13 22:20:44 +01:00
Tal Levy f56097b57a Fixes GrokProcessor's ignorance of named-captures with same name. (#22131)
Grok was originally ignoring potential matches to named-capture groups
larger than one. For example, If you had two patterns containing the
same named field, but only the second pattern matched, it would fail to
pick this up.

This PR fixes this by exploring all potential places where a
named-capture was used and chooses the first one that matched.

Fixes #22117.
2016-12-13 13:19:55 -08:00
Jason Tedor 9ad49faa86 Fix for loop that is bad and should feel bad
This commit fixes a for loop that reverses the order of shard stats
coming off the wire, and is really hard to read anyway (with the
post-increment in the loop initializer).

Relates #22150
2016-12-13 15:27:42 -05:00
Simon Willnauer 7a9b667e98 Introduce a low level protocol handshake (#22094)
Today we rely on the version that the API user passes in together with the DiscoveryNode. This commit introduces a low level handshake where nodes exchange their version to be used with the transport protocol that is executed every time a connection to a node is established. This, on the one hand allows to change the wire protocol based on the version we are talking to even without a full cluster restart. Today we would need to carry on a BWC layer across major versions but with a handshake we can rely on the fact that the latest version of the previous minor executes a handshake and uses the latest protocol version across all communication with the N+1 version nodes.

This change is yet fully backwards compatible, a followup PR will remove the BWC in 6.0 once this has been back-ported to the 5.x branch
2016-12-13 21:06:23 +01:00
Nik Everett ce86405394 Start to centralize creation of XContentParser in tests (#22096)
Starts to centralize creation of the `XContentParser` in
`protected final` methods on `ESTestCase`. The idea is to enable
adding `NamedXContentRegistry` relatively easily by giving tests
a single place they can override to define the
`NamedXContentRegistry`. Since `NamedXContentRegistry` doesn't
exist yet neither does the override point.

This doesn't attempt to migrate all the tests to calling the
new methods to build the parsers. I wanted to make this so we
could review the concept and then I'll merge a followup to
migrate the tests.
2016-12-13 11:22:15 -05:00
markharwood 4c6d17a176 Added tests for toXContent and fromXContent for IncludeExclude class.
New REST test revealed an issue with inconsistent hashing in partitioned
term tests which is also fixed in this change.

Closes #22102
2016-12-13 15:23:09 +00:00
Clinton Gormley 28397c9594 Added version 2.4.3 plus bwc indices 2016-12-13 16:15:52 +01:00
javanna 478cd78047 Remove unused import from RequestLogger
Not only was StringJoiner unused, it's also a class only available in java 1.8, which is a problem given that the REST client has minimum java required set to 1.7
2016-12-13 15:01:51 +01:00
Isabel Drost-Fromm eeff9fb100 Adjust docs to reflect removed template endpoint
The dedicated template endpoint for rank_eval queries was removed, reflect this in the docs as well.
2016-12-13 13:59:11 +01:00
Isabel Drost-Fromm 58342d4c9a Add checks to RankEvalSpec to safe guard against missing parameters. (#22026)
Add checks to RankEvalSpec to safe guard against missing parameters.

Fail early in case no metric is supplied, no rated requests are supplied or the search source builder is missing but no template is supplied neither.

Add stricter checks around rank eval request parsing: Fail if in a rated request we see both, a verbatim request as well as request
template parameters.


Relates to #21260
2016-12-13 11:21:57 +01:00
Isabel Drost-Fromm 6d1a658106 Fix compile issues after merging in master. 2016-12-13 11:16:53 +01:00
Isabel Drost-Fromm b2e8455745 Merge branch 'master' into feature/rank-eval 2016-12-13 10:45:22 +01:00
Adrien Grand 049fd3991c Remove `AggregationContext`. (#22124)
This class is just a wrapper around `SearchContext`, so let's use
`SearchContext` directly. The change is mechanical, except the
`ValuesSourceConfig` class, where I moved the logic to get a `ValuesSource`
given a config.
2016-12-13 09:09:40 +01:00
Lee Hinman a4e8b5d952 Throw an exception on unrecognized "match_mapping_type"
When using dynamic templates, ES will now throw an exception if a
`match_mapping_type` is used that doesn't correspond to an actual type.

Relates to #17285
2016-12-12 09:59:48 -07:00
Simon Willnauer b667ff46c4 Allow plugins to install bootstrap checks (#22110)
Plugins also have the need to provide better OOTB experience by configuring
defaults unless the plugin is used in _production_ mode. This change exposes
the bootstrap check infrastructure as part of the plugin API to allow plugins
to specify / install their own bootstrap checks if necessary.
2016-12-12 17:35:00 +01:00
Matt Sicker 32fcb7220e Clarify first search API explanation of parameters (#21958)
Added info about the sort parameter and included the pretty parameter in the example.
2016-12-12 15:00:53 +01:00
yutaono abf718832d add missing url-access-control link (#21960) 2016-12-12 14:56:52 +01:00
Peter Dyson 7904028fef Merge pull request #22105 from geekpete/master
Fix Fuziness typo in multimatch error message and associated tests
2016-12-12 23:13:20 +10:00
Luca Cavanna 6d987a9b69 Remove support for empty queries (#22092)
Our query DSL supports empty queries (`{}`), which have a different meaning depending on the query that holds it, either ignored, match_all or match_none. We deprecated the support for empty queries in 5.0, where we log a deprecation warning wherever they are used.

The way we supported it once we moved query parsing to the coordinating node was having an Optional<QueryBuilder> return type in all of our parse methods (called fromXContent). See #17624. The central place for this was QueryParseContext#parseInnerQueryBuilder. We can now remove all the optional return types and simply throw an exception whenever an empty query is found.
2016-12-12 12:37:12 +01:00
Luca Cavanna 9be778c5e5 Warn log deprecation warnings received from server (#21895)
The warnings get printed out in a single line e.g. WARNING: request [DELETE http://localhost:9200/index/type/_api] returned 3 warnings:[this is warning number 0],[this is warning number 1],[this is warning number 2]
2016-12-12 12:11:42 +01:00
Luca Cavanna a42bee5d60 Don't output empty ext object in SearchSourceBuilder#toXContent (#22093)
We shouldn't output an empty ext object if no ext sections have been set to the SearchSourceBuilder.

Closes #20969
2016-12-12 12:10:23 +01:00
Luca Cavanna 73cf002293 Un-deprecate fuzzy query (#22088)
When we decided to deprecate and remove fuzzy query in #15760, we didn't realize we would take away the possibililty for uses to use a fuzzy query as part of a span query, which is not possible using match query. This means we have to go back and un-deprecate fuzzy query, which will not be removed.

Closes #15760
2016-12-12 12:09:16 +01:00
David Pilato b0ab3237d3 Merge branch 'fix/22053-azure-readonly' 2016-12-12 11:59:01 +01:00
Peter Dyson d339814f77 Fix Fuziness typo in multimatch error message and associated tests 2016-12-12 11:33:31 +10:00
Artur Nowosielski 726f5dccc0 Rewrite filter queries in FiltersAggregationBuilder (#22076)
Queries must be rewritten before the query phase executes otherwise non-executable queries like `wrapper` query or `terms`  will fail or queries that require resources like script service can't access these service unless rewritten.

Relates to #21303
2016-12-11 14:37:12 +01:00
Masaru Hasegawa 3df2a086d4 Resolve index names in indices_boost
This change allows specifying alias/wildcard expression in indices_boost.
And added another format for specifying indices_boost. It accepts array of index name and boost pair.
If an index is included in multiple aliases/wildcard expressions, the first match will be used.
With new format, old format is marked as deprecated.

Closes #4756
2016-12-11 21:41:49 +09:00
Simon Willnauer 20ff703e07 Fix IncludeExclude parsing
`include` / `exclude` in terms / sig-terms aggs seems completely broken
and massively untested. This commit makes the TermsTests pass again that
randomly use `include` / `exclude`. This class must be tested individually
and we need real integ tests that use xcontent that use this feature.
2016-12-11 09:55:53 +01:00
javanna e6b10ca4db Restore proper bwcVersion in qa/backwards gradle build file 2016-12-10 21:11:05 +01:00
javanna 6003cbfb64 fix typos in qa/backwards gradle build file 2016-12-10 21:09:49 +01:00
Nik Everett 7a74a41a0c Fix test for changed message
Message is only changed in 5.2.0 so we shouldn't assert on it
if we're running with any nodes less than that version.
2016-12-10 10:35:14 -05:00
Yannick Welsch 68f30cae0c [TEST] Unmute testRestoreUnsupportedSnapshots 2016-12-10 13:49:47 +01:00
Yannick Welsch 4831632a6f [TEST] Handle legacy snapshots as if they don't exist anymore
An earlier commit removed BWC for pre-5.0 snapshots, which also meant removing the capability to load pre-5.0 snapshots. In 6.0, such snapshots are now
invisible and must be treated by the BWC tests in that way.
2016-12-10 13:49:47 +01:00
Yannick Welsch 4ad85c38c3 Throw NoSuchFileException to correctly adhere to readBlob contract
URLBlobContainer can in certain situations throw a FileNotFoundException. To fulfill the contract of the readBlob method it should throw a NoSuchFileException instead when the given blob cannot be found.
2016-12-10 13:49:47 +01:00