Commit Graph

39228 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe f4fad07113
Merge branch 'master' into index-lifecycle 2018-05-16 11:43:26 +01:00
Colin Goodheart-Smithe a75b8adce5
Refactors ClientHelper to combine header logic (#30620)
* Refactors ClientHelper to combine header logic

This change removes all the `*ClientHelper` classes which were
repeating logic between plugins and instead adds
`ClientHelper.executeWithHeaders()` and
`ClientHelper.executeWithHeadersAsync()` methods to centralise the
logic for executing requests with stored security headers.

* Removes Watcher headers constant
2018-05-16 11:38:24 +01:00
David Kyle 16f5a515f3
[ML] Wait for ML indices in rolling upgrade tests (#30615) 2018-05-16 09:52:25 +01:00
Alexander Reelsen 6ec5e4408c
Watcher: Ensure secrets integration tests also run triggered watch (#30478)
When the encrpytion of sensitive date is enabled, test that a
scheduled watch is executed as expected and produces the correct value
from a secret in the basic auth header.
2018-05-16 10:32:22 +02:00
Yannick Welsch 3161386e2f
Move allocation awareness attributes to list setting (#30626)
Allows the setting to be specified using proper array syntax, for example:
"cluster.routing.allocation.awareness.attributes": [ "foo", "bar", "baz" ]

Closes #30617
2018-05-16 09:57:22 +02:00
lukens 9434f25ee3 [Docs] Update code snippet in has-child-query.asciidoc (#30510)
Changed `InetSocketTransportAddress` to `TransportAddress`, as that 
seems to be  the thing now.
2018-05-16 09:25:48 +02:00
Colin Goodheart-Smithe 79be4d2976
Adds step info to ClusterStateWaitSteps (#30609)
The `ClusterStateWaitStep.isConditionMet()` method now returns a
`Result` object which contains a boolean for if the condition is met
and an `ToXContentObject` to provide information in the case where the
condition is not met.
If the condition is not met, the step information is stored in the
cluster state
2018-05-16 08:15:23 +01:00
Tal Levy 8c12a59c5e Merge branch 'master' into index-lifecycle 2018-05-15 22:41:23 -07:00
Ioannis Kakavas 2b09e90237
Replace custom reloadable Key/TrustManager (#30509)
Make SSLContext reloadable

This commit replaces all customKeyManagers and TrustManagers 
(ReloadableKeyManager,ReloadableTrustManager, 
EmptyKeyManager, EmptyTrustManager) with instances of 
X509ExtendedKeyManager and X509ExtendedTrustManager. 
This change was triggered by the effort to allow Elasticsearch to 
run in a FIPS-140 environment. In JVMs running in FIPS approved 
mode, only SunJSSE TrustManagers and KeyManagers can be used. 
Reloadability is now ensured by a volatile instance of SSLContext
in SSLContectHolder.
SSLConfigurationReloaderTests use the reloadable SSLContext to
initialize HTTP Clients and Servers and use these for testing the
key material and trust relations.
2018-05-16 08:32:13 +03:00
Vladimir Dolzhenko fe3e0257ae
Allow date math for naming newly-created snapshots (#7939) (#30479)
Allow date math for naming newly-created snapshots (#7939)
2018-05-16 07:23:25 +02:00
Nik Everett 9b47e0508b Fix compilation of test framework tests
We accidentally broke the compilation of the test frameworks tests. All
better now.
2018-05-15 22:56:41 -04:00
Michael Basnight b94bc70aee
Add Create Repository High Level REST API (#30501)
This commit adds Create Repository, the associated docs and tests
for the high level REST API client. A few small changes to the
PutRepository Request and Response went into the commit as well.
2018-05-15 21:21:11 -05:00
Jason Tedor 25c823da09
Skip shard deprecation messages in REST tests (#30630)
A 6.x node can send a deprecation message that the default number of
shards will change from five to one in 7.0.0. In a mixed cluster,
whether or not a create index request sees five or one shard and
produces a deprecation message depends on the version of the master
node. This means that during BWC tests a test can see this deprecation
message depending on the version of the master node. In 6.x when we
introduced this deprecation message we assumed that whereever we see
this deprecation message is expected. However, in a mixed cluster test
we need a similar mechanism but it would only apply if the version of
the master node is earlier than 7.0.0. This commit takes advantage of a
recent change to expose the version of the master node to do sections of
REST tests. With this in hand, we can skip asserting on the deprecation
message if the version of the master node is before 7.0.0 and otherwise
seeing that deprecation message would be completely unexpected.
2018-05-15 21:07:32 -04:00
Ryan Ernst c7d82b378b
Build: Add task interdependencies for ssl configuration (#30633)
This commit fixes the tasks creating ssl certs for tests to have correct
dependsOn to ensure the right tasks are run before tests run.
2018-05-15 16:09:15 -07:00
Tim Brooks 99b9ab58e2
Add nio http server transport (#29587)
This commit is related to #28898. It adds an nio driven http server
transport. Currently it only supports basic http features. Cors,
pipeling, and read timeouts will need to be added in future PRs.
2018-05-15 16:37:14 -06:00
Nik Everett 6695d11a97
Switch many QA projects to use new style requests (#30574)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the the old requests. This
changes many calls in the `qa` projects to use the new version.
2018-05-15 17:48:47 -04:00
Jason Tedor abc06d5b79
Expose master version in REST test context (#30623)
This commit exposes the master version to the REST test context. This
will be needed in a follow-up where the master version will be used to
determine whether or not a certain warning header is expected.
2018-05-15 17:26:43 -04:00
Nik Everett 869b639d14
QA: System property to override distribution (#30591)
This configures all `qa` projects to use the distribution contained in
the `tests.distribution` system property if it is set. The goal is to
create a simple way to run tests against the default distribution which
has x-pack basic features enabled while not forcing these tests on all
contributors. You run these tests by doing something like:

```
./gradlew -p qa -Dtests.distribution=zip check
```

or

```
./gradlew -p qa -Dtests.distribution=zip bwcTest
```

x-pack basic *shouldn't* get in the way of any of these tests but
nothing is ever perfect so this we have to disable a few when running
with the zip distribution.
2018-05-15 17:16:16 -04:00
Lee Hinman 2cb71d0947
Refactor IndicesOptions to not be byte-based (#30586)
* Refactor IndicesOptions to not be byte-based

This refactors IndicesOptions to be enum/enummap based rather than using a byte
as a bitmap for each of the options. This is necessary because we'd like to add
additional options, but we ran out of bits.

Backwards compatibility is kept for earlier versions so the option serialization
does not change the options.

Relates sort of to #30188
2018-05-15 15:03:08 -06:00
Julie Tibshirani 4f9dd37169
Add support for search templates to the high-level REST client. (#30473) 2018-05-15 13:07:58 -07:00
Costin Leau 03dd2ab499 SQL: eliminate disabled tests 2018-05-15 22:49:05 +03:00
Costin Leau 09329eb84f
SQL: Verify GROUP BY ordering on grouped columns (#30585)
Due to the way composite aggregation works, ordering in GROUP BY can be
applied only through grouped columns which now the analyzer verifier
enforces.

Fix 29900
2018-05-15 22:46:46 +03:00
Ryan Ernst 21b9170dec
Security: Remove SecurityLifecycleService (#30526)
This commit removes the SecurityLifecycleService, relegating its former
functions of listening for cluster state updates to SecurityIndexManager
and IndexAuditTrail.
2018-05-15 12:13:24 -07:00
lcawl 5894e3574f [DOCS] Restores 7.0.0 release notes and highlights 2018-05-15 08:48:41 -07:00
Julie Tibshirani ab0be394e9
Remove assert statements from field caps documentation. (#30601)
Reorganize the test in `SearchDocumentationIT` so the assertions aren't shown in the generated documentation.
2018-05-15 08:37:50 -07:00
Albert Zaharovits 801973fa9f
Repository GCS plugin new client library (#30168)
This does away with the deprecated `com.google.api-client:google-api-client:1.23`
and replaces it with `com.google.cloud:google-cloud-storage:1.28.0`.
It also changes security permissions for the repository-gcs plugin.
2018-05-15 18:22:58 +03:00
Nhat Nguyen d1c28c60fc
HLRestClient: Follow-up for put index template api (#30592)
This commit addresses some comments given after the original PR was in.

Follow-up #30400
2018-05-15 08:06:58 -04:00
Ioannis Kakavas 17d65c1f06 Unmute IndexUpgradeIT tests
After additional fixes to the Jenkins job, these tests can now be
unmuted.

Closes #30430
2018-05-15 13:48:15 +03:00
javanna e1d675c690 [DOCS] Remove references to changelog and to highlights
highlights reference the changelog and it currently breaks the docs.

This aligns changes in master with the ones made in other branches.
2018-05-15 12:42:15 +02:00
Simon Willnauer b50cf3c6b0
Side-step pending deletes check (#30571)
When we split/shrink an index we open several IndexWriter instances
causeing file-deletes to be pending on windows. This subsequently fails
when we open an IW to bootstrap the index history due to pending deletes.
This change sidesteps the check since we know our history goes forward
in terms of files and segments.

Closes #30416
2018-05-15 11:51:54 +02:00
javanna 098b3b7fb4 [DOCS] Remove references to removed changelog 2018-05-15 11:47:56 +02:00
Yannick Welsch af2b9dd779 Revert "Mute ML upgrade test (#30458)"
This reverts commit 4b36ea7433.
2018-05-15 11:20:57 +02:00
Colin Goodheart-Smithe 8059aa7d55
Merge branch 'master' into index-lifecycle 2018-05-15 09:59:55 +01:00
David Roberts 1de5a3180a [ML] Adjust BWC version following backport of #30125 2018-05-15 09:51:56 +01:00
srini-raman 0592b685b9 [Docs] Improve section detailing translog usage (#30573) 2018-05-15 10:43:57 +02:00
Christoph Büscher bf2fb210cc
[Tests] Relax allowed delta in extended_stats aggregation (#30569)
The order in which double values are added in java can give different results
for the sum, so we need to allow a certain delta in the test assertions. The
current value was still a bit too low, resulting in rare test failures. This
change increases the allowed margin of error by a factor of ten.
2018-05-15 10:35:16 +02:00
David Roberts 50c34b2a9b
[ML] Reverse engineer Grok patterns from categorization results (#30125)
This change adds a grok_pattern field to the GET categories API
output in ML. It's calculated using the regex and examples in the
categorization result, and applying a list of candidate Grok
patterns to the bits in between the tokens that are considered to
define the category.

This can currently be considered a prototype, as the Grok patterns
it produces are not optimal. However, enough people have said it
would be useful for it to be worthwhile exposing it as experimental
functionality for interested parties to try out.
2018-05-15 09:02:38 +01:00
Tim Vernum 7dd816e77c Update build file due to doc file rename
A file with uncoverted snippets was change as part of 7f47ff9, but
build.gradle was not updated to reflect the rename.
2018-05-15 14:28:29 +10:00
Jason Tedor 0f85c6429c
Remove the changelog (#30593)
We are starting over on the changelog with a different approach. This
commit removes the existing incarnation of the changelog to remove
confusion that we need to continue adding entries to it.
2018-05-14 21:57:08 -04:00
Tim Brooks 848f240926
Fix issue with finishing handshake in ssl driver (#30580)
This is fixing an issue that has come up in some builds. In some
scenarios I see an assertion failure that we are trying to move to
application mode when we are not in handshake mode. What I think is
happening is that we are in handshake mode and have received the
completed handshake message AND an application message. While reading in
handshake mode we switch to application mode. However, there is still
data to be consumed so we attempt to continue to read in handshake mode.
This leads to us attempting to move to application mode again throwing
an assertion.

This commit fixes this by immediatly exiting the handshake mode read
method if we are not longer in handshake mode. Additionally if we swap
modes during a read we attempt to read with the new mode to see if there
is data that needs to be handled.
2018-05-14 19:19:53 -06:00
Tim Vernum 6517ac98eb
Fail if reading from closed KeyStoreWrapper (#30394)
In #28255 the implementation of the elasticsearch.keystore was changed
to no longer be built on top of a PKCS#12 keystore. A side effect of
that change was that calling getString or getFile on a closed
KeyStoreWrapper ceased to throw an exception, and would instead return
a value consisting of all 0 bytes.

This change restores the previous behaviour as closely as possible.
It is possible to retrieve the _keys_ from a closed keystore, but any
attempt to get or set the entries will throw an IllegalStateException.
2018-05-15 09:57:34 +10:00
Ioannis Kakavas 15790e1b56 Silence IndexUpgradeIT test failures. (#30430) 2018-05-15 02:14:35 +03:00
Lisa Cawley 21d67d1bd7
[DOCS] Adds release highlight pages (#30590) 2018-05-14 15:49:00 -07:00
Nik Everett 9881bfaea5
Docs: Document how to rebuild analyzers (#30498)
Adds documentation for how to rebuild all the built in analyzers and
tests for that documentation using the mechanism added in #29535.

Closes #29499
2018-05-14 18:40:54 -04:00
lcawl 7f47ff9fcd [DOCS] Fixes title capitalization in security content 2018-05-14 15:36:03 -07:00
Nik Everett 69481b4059
LLRest: Add equals and hashcode tests for Request (#30584)
Adds tests for the `Request` object's equals and hashcode to remove a
`TODO` and because we use the `equals` method in other testing.
2018-05-14 18:27:36 -04:00
Igor Motov 56d32bc8b2
SQL: Extract SQL request and response classes (#30457)
Extracts SQL request and response classes. This is the first step
towards creation of a small minimal dependencies jdbc driver.

Relates #29856
2018-05-14 16:43:29 -04:00
Jason Tedor 4e33443690
Adjust versions for resize copy settings (#30578)
Now that the change to deprecate copy settings and disallow it being
explicitly set to false is backported, this commit adjusts the BWC
versions in master.
2018-05-14 16:41:25 -04:00
Jack Conradson 1b0e6ee89f
Deprecate Empty Templates (#30194)
Deprecate the use of empty templates.  Bug fix allows empty
templates/scripts to be loaded on start up for upgrades/restarts, 
but empty templates can no longer be created.
2018-05-14 13:32:09 -07:00
Lisa Cawley 7928270610
[DOCS] Fix realm setting names (#30499) 2018-05-14 13:13:26 -07:00