Commit Graph

39487 Commits

Author SHA1 Message Date
Tal Levy fbfb4ff8c7 Merge branch 'master' into index-lifecycle 2018-06-01 11:44:38 -07:00
Colin Goodheart-Smithe 39844be808 Adds an explain API endpoint (#31005)
* Adds an explain API endpoint

This endpoint can be used to explain the current lifecycle state of an
index

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/action/ExplainLifecycleAction.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/action/IndexExplainResponse.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/action/TransportExplainLifecycleAction.java
x-pack/plugin/src/test/resources/rest-api-spec/test/index_lifecycle/20_m
ove_to_step.yml

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/XPackClien
tPlugin.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/action/ExplainLifecycleAction.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/action/IndexExplainResponse.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/RandomStepInfo.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/action/ExplainLifecycleRequestTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/action/ExplainLifecycleResponseTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/action/IndexExplainResponseTests.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycle.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/action/RestExplainLifecycleAction.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/action/TransportExplainLifecycleAction.java
x-pack/plugin/index-lifecycle/src/test/java/org/elasticsearch/xpack/inde
xlifecycle/ExecuteStepsUpdateTaskTests.java
x-pack/plugin/index-lifecycle/src/test/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycleRunnerTests.java
x-pack/plugin/src/test/resources/rest-api-spec/api/xpack.index_lifecycle
.explain_lifecycle.json
x-pack/plugin/src/test/resources/rest-api-spec/test/index_lifecycle/20_m
ove_to_step.yml
x-pack/plugin/src/test/resources/rest-api-spec/test/index_lifecycle/30_e
xplain_lifecycle.yml

* Adds tests for explain API

* Addresses Review comments and fixes REST tests

* Removes RequestBuilder from ExplainLifecycleAction
2018-06-01 11:39:47 -07:00
lipsill a8c643e833 [Docs] Fix a typo in Create Index naming limitation (#30891) 2018-06-01 18:15:24 +02:00
Jason Tedor 4522b57e07
Introduce client feature tracking (#31020)
This commit introduces the ability for a client to communicate to the
server features that it can support and for these features to be used in
influencing the decisions that the server makes when communicating with
the client. To this end we carry the features from the client to the
underlying stream as we carry the version of the client today. This
enables us to enhance the logic where we make protocol decisions on the
basis of the version on the stream to also make protocol decisions on
the basis of the features on the stream. With such functionality, the
client can communicate to the server if it is a transport client, or if
it has, for example, X-Pack installed. This enables us to support
rolling upgrades from the OSS distribution to the default distribution
without breaking client connectivity as we can now elect to serialize
customs in the cluster state depending on whether or not the client
reports to us using the feature capabilities that it can under these
customs. This means that we would avoid sending a client pieces of the
cluster state that it can not understand. However, we want to take care
and always send the full cluster state during node-to-node communication
as otherwise we would end up with different understanding of what is in
the cluster state across nodes depending on which features they reported
to have. This is why when deciding whether or not to write out a custom
we always send the custom if the client is not a transport client and
otherwise do not send the custom if the client is transport client that
does not report to have the feature required by the custom.

Co-authored-by: Yannick Welsch <yannick@welsch.lu>
2018-06-01 11:45:35 -04:00
Alan Woodward b8fda588f4
Ensure that index_prefixes settings cannot be changed (#30967) 2018-06-01 15:17:35 +01:00
Sohaib Iftikhar 11887fa54a REST high-level client: add delete ingest pipeline API (#30865)
Relates to #27205
2018-06-01 14:13:41 +02:00
Colin Goodheart-Smithe 0f12ecc95f
Fixes checkstyle error
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/LifecyclePolicySecurityClient.java
2018-06-01 11:40:59 +01:00
Dimitris Athanasiou 9141108334
[ML][TEST] Fix bucket count assertion in all tests in ModelPlotsIT (#31026)
This fixes the last remaining test that was missed in #30717.

Closes #30715
2018-06-01 10:51:12 +01:00
Yannick Welsch e1649b8669
Allow rollup job creation only if cluster is x-pack ready (#30963)
Otherwise we could end up with persistent tasks metadata in the cluster that some of the nodes
might not understand in case where the cluster is during rolling upgrade from the default 6.2 to the
default 6.3 distribution.

Follow-up to #30743
2018-06-01 10:47:53 +02:00
Colin Goodheart-Smithe fc98f09289
Fixes compile errors caused by merge
Specifically because `RequestBuilder`s have been removed from `Action`s
2018-06-01 09:14:17 +01:00
Colin Goodheart-Smithe 830198fa00
Merge branch 'master' into index-lifecycle 2018-06-01 09:04:22 +01:00
Yannick Welsch fb671adfd6
Fix interoperability with < 6.3 transport clients (#30971)
With the default distribution changing in 6.3, clusters might now contain custom metadata that a
pure OSS transport client cannot deserialize. As this can break transport clients when accessing
the cluster state or reroute APIs, we've decided to exclude any custom metadata that the transport
client might not be able to deserialize. This will ensure compatibility between a < 6.3 transport
client and a 6.3 default distribution cluster. Note that this PR only covers interoperability with older
clients, another follow-up PR will cover full interoperability for >= 6.3 transport clients where we will
make it possible again to get the custom metadata from the cluster state.

Relates to #30731
2018-06-01 10:02:57 +02:00
Jim Ferenczi 0791f93dbd
Add an option to split keyword field on whitespace at query time (#30691)
This change adds an option named `split_queries_on_whitespace` to the `keyword`
field type. When set to true full text queries (`match`, `multi_match`, `query_string`, ...) that target the field will split the input on whitespace to build the query terms. Defaults to `false`.
Closes #30393
2018-06-01 09:47:03 +02:00
Christoph Büscher cea3c28b5b
[Tests] Fix alias names in PutIndexTemplateRequestTests (#30960)
The randomized alias names could contain unicode controll charactes that don't
survive an xContent rendering and parsing roundtrip when using the YAML xContent
type. This fix filters the randomized unicode string for control characters to
avoid this particular problem.

Closes #30911
2018-06-01 09:45:04 +02:00
Sohaib Iftikhar 80d20a9010 REST high-level client: add get ingest pipeline API (#30847)
Relates to #27205
2018-06-01 08:55:43 +02:00
Luca Cavanna 70749e01c4
Cross Cluster Search: preserve remote status code (#30976)
In case an error is returned when calling search_shards on a remote
cluster, which will lead to throwing an exception in the coordinating
 node, we should make sure that the status code returned by the
 coordinating node is the same as the one returned by the remote
 cluster. Up until now a 500 - Internal Server Error was always
 returned. This commit changes this behaviour so that for instance if an
 index is not found, which causes an 404, a 404 is also returned by the
 coordinating node to the client.

 Closes #27461
2018-06-01 08:53:53 +02:00
Luca Cavanna 31351ab880
High-level client: list tasks failure to not lose nodeId (#31001)
This commit reworks testing for `ListTasksResponse` so that random
fields insertion can be tested and xcontent equivalence can be checked
too. Proper exclusions need to be configured, and failures need to be
tested separately. This helped finding a little problem, whenever there
is a node failure returned, the nodeId was lost as it was never printed
out as part of the exception toXContent.
2018-06-01 08:53:24 +02:00
Lisa Cawley 7c74318580
[DOCS] Fixes links (#31011) 2018-05-31 17:02:18 -07:00
Nik Everett 283d1bb2cf Watcher: Give test a little more time
Changes watcher's integration tests to wait 30 seconds when starting
watcher rather than 10 seconds because this build failed when starting
took 12 seconds:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.3+periodic/222/console
2018-05-31 18:11:16 -04:00
Ioannis Kakavas 3004b9eeec
Reuse expiration date of trial licenses (#30950)
* Retain the expiryDate for trial licenses

While updating the license signature to the new license spec retain
the trial license expiration date to that of the existing license.

Resolves #30882
2018-05-31 23:05:58 +03:00
Julie Tibshirani cd0a375414
Remove unused query methods from MappedFieldType. (#30987)
* Remove MappedFieldType#nullValueQuery, as it is now unused.
* Remove MappedFieldType#queryStringTermQuery, as it is never overridden.
2018-05-31 12:47:52 -07:00
Tim Brooks 4f66b9a27c
Transport client: Don't validate node in handshake (#30737)
This is related to #30141. Right now in the transport client we open a
temporary node connection and take the node information. This node
information is used to open a permanent connection that is used for the
client. However, we continue to use the configured transport address.
If the configured transport address is a load balancer, you might
connect to a different node for the permanent connection. This causes
the handshake validation to fail. This commit removes the handshake
validation for the transport client when it simple node sample mode.
2018-05-31 13:14:28 -06:00
Albert Zaharovits d9ab1469c9
[DOCS] Clarify not all PKCS12 usable as truststores (#30750)
Although elasticsearch-certutil generates PKCS#12
files which are usable as both keystore and truststore
this is uncommon in practice. Settle these expectations
for the users following our security guides.
2018-05-31 21:49:27 +03:00
Nik Everett b225f5e5c6
HLRest: Allow caller to set per request options (#30490)
This modifies the high level rest client to allow calling code to
customize per request options for the bulk API. You do the actual
customization by passing a `RequestOptions` object to the API call
which is set on the `Request` that is generated by the high level
client. It also makes the `RequestOptions` a thing in the low level
rest client. For now that just means you use it to customize the
headers and the `httpAsyncResponseConsumerFactory` and we'll add
node selectors and per request timeouts in a follow up.

I only implemented this on the bulk API because it is the first one
in the list alphabetically and I wanted to keep the change small
enough to review. I'll convert the remaining APIs in a followup.
2018-05-31 13:59:52 -04:00
Michael Basnight d826cb36c3
Remove version read/write logic in Verify Response (#30879)
Since master will always communicate with a >=6.4 node, the logic for
checking if the node is 6.4 and conditionally reading and writing based
on that can be removed from master. This logic will stay in 6.x as it is
the bridge to the cleaner response in master. This also unmutes the
failing test due to this bwc change.

Closes #30807
2018-05-31 12:10:01 -05:00
Lisa Cawley 3960a7a310
[DOCS] Update readme for testing x-pack code snippets (#30696) 2018-05-31 09:32:22 -07:00
Ioannis Kakavas 532641ef30
Ensure intended key is selected in SamlAuthenticatorTests (#30993)
* Ensure that a purposefully wrong key is used

Uses a specific keypair for tests that require a purposefully wrong
keypair instead of selecting one randomly from the same pull from
which the correct one is selected. Entropy is low because of the
small space and the same key can be randomly selected as both the
correct one and the wrong one, causing the tests to fail.
The purposefully wrong key is also used in 
testSigningKeyIsReloadedForEachRequest and needs to be cleaned
up afterwards so the rest of the tests don't use that for signing.

Resolves #30970
2018-05-31 18:09:32 +03:00
Ryan Ernst 46e8d97813
Core: Remove RequestBuilder from Action (#30966)
This commit removes the RequestBuilder generic type from Action. It was
needed to be used by the newRequest method, which in turn was used by
client.prepareExecute. Both of these methods are now removed, along with
the existing users of prepareExecute constructing the appropriate
builder directly.
2018-05-31 16:15:00 +02:00
Jim Ferenczi 0f5e570184
Deprecates indexing and querying a context completion field without context (#30712)
This change deprecates completion queries and documents without context that target a
context enabled completion field. Querying without context degrades the search
performance considerably (even when the number of indexed contexts is low).
This commit targets master but the deprecation will take place in 6.x and the functionality
will be removed in 7 in a follow up.

Closes #29222
2018-05-31 16:09:48 +02:00
Luca Cavanna 63f3a61134
Refactor Sniffer and make it testable (#29638)
This commit reworks the Sniffer component to simplify it and make it possible to test it.

In particular, it no longer takes out the host that failed when sniffing on failure, but rather relies on whatever the cluster returns. This is the result of some valid comments from #27985. Taking out one single host is too naive, hard to test and debug.

A new Scheduler abstraction is introduced to abstract the tasks scheduling away and make it possible to plug in any test implementation and take out timing aspects when testing.

Concurrency aspects have also been improved, synchronized methods are no longer required. At the same time, we were able to take #27697 and #25701 into account and fix them, especially now that we can more easily add tests.

Last but not least, unit tests are added for the Sniffer component, long overdue.

Closes #27697
Closes #25701
2018-05-31 16:05:09 +02:00
Colin Goodheart-Smithe 976935967b
Merge branch 'master' into index-lifecycle 2018-05-31 14:29:00 +01:00
Christoph Büscher 4777d8a2df
[Docs] Fix typo in Min Aggregation reference (#30899) 2018-05-31 15:05:03 +02:00
David Roberts 0ff2c605b6 [CI] Mute Ml rolling upgrade test for mixed cluster too
It can fail in either the mixed cluster or the upgraded cluster,
so it needs to be muted in both.

Tracked by #30982
2018-05-31 11:17:18 +01:00
Tanguy Leroux c41574376f
Make AllocatedPersistentTask.isCompleted() protected (#30949)
This commit changes the isCompleted() method to be protected so that
classes that extends AllocatedPersistentTask can use it.

Related to #30858
2018-05-31 09:19:05 +02:00
Igor Motov 8e4ab82e3d [CI] Mute Ml rolling upgrade tests
Tracked by #30982
2018-05-30 21:30:18 -04:00
Jason Tedor 735f232e03
Fix license on AcitveDirectorySIDUtil (#30972)
This code is from an Apache 2.0 licensed codebase and when we imported
it into our codebase it carried the Apache 2.0 license as well. However,
during the migration of the X-Pack codebase from the internal private
repository to the elastic/elasticsearch repository, the migration tool
mistakently changed the license on this source file from the Apache 2.0
license to the Elastic license. This commit addresses this mistake by
reapplying the Apache 2.0 license.
2018-05-30 15:28:25 -04:00
Christoph Büscher 0a5d46ef3c
[Test] Prefer ArrayList over Vector (#30965)
Replaces some occurances of Vector class with ArrayList in
tests of the rank-eval module.
2018-05-30 21:11:49 +02:00
Igor Motov 1da26a42cb [CI] Mute HttpSecretsIntegrationTests#testWebhookAction test
Tracked by #30094
2018-05-30 14:29:06 -04:00
Nhat Nguyen b834254862 Mute FlushIT tests
We have identified the source causing these tests failed.
This commit mutes them again until we have a proper fix.

Relates #29392
2018-05-30 14:23:53 -04:00
markharwood facbb2b2dc
Add “took” timing info to response for _msearch/template API (#30961)
Add “took” timing info to response for _msearch/template API
Closes #30957
2018-05-30 17:57:28 +01:00
Michael Basnight b716b08197
Add Verify Repository High Level REST API (#30934)
This commit adds Verify Repository, the associated docs and tests for
the high level REST API client. A few small changes to the Verify
Repository Response went into the commit as well.

Relates #27205
2018-05-30 11:10:00 -05:00
Igor Motov 96b4dae221 [CI] Mute SamlAuthenticatorTests testIncorrectSigningKeyIsRejected
Tracked by #30970
2018-05-30 11:29:17 -04:00
lcawl 94e1bdba33 [DOCS] Fixes kibana security file location 2018-05-30 07:47:20 -07:00
Igor Motov cdbbbafb9f
SQL: Remove log4j and joda from JDBC dependencies (#30938)
More cleanup of JDBC driver project

Relates to #29856
2018-05-30 09:57:32 -04:00
Jim Ferenczi e852eb0548 Revert accidentally pushed changes in NoriAnalysisTests 2018-05-30 15:34:55 +02:00
Jim Ferenczi 532b91ffa6 Fix composite agg serialization error
Fix serialization after backport

Relates #29465
2018-05-30 14:22:48 +02:00
Christoph Büscher 1ea9f11b03
Change ScriptException status to 400 (bad request) (#30861)
Currently failures to compile a script usually lead to a ScriptException, which
inherits the 500 INTERNAL_SERVER_ERROR from ElasticsearchException if it does
not contain another root cause. Instead, this should be a 400 Bad Request error.
This PR changes this more generally for script compilation errors by changing 
ScriptException to return 400 (bad request) as status code.

Closes #12315
2018-05-30 14:00:07 +02:00
Nik Everett 7c5abc0a6b Fix synced flush docs
They had some copy and paste errors that failed the docs build.
2018-05-30 07:51:12 -04:00
Sohaib Iftikhar 2c27c58718 REST high-level client: add synced flush API (2) (#30650)
Adds the synced flush API to the high level REST client.

Relates to #27205.
2018-05-30 07:32:52 -04:00
Jim Ferenczi f582418ada Fix missing option serialization after backport
Relates #29465
2018-05-30 12:55:31 +02:00