Commit Graph

39638 Commits

Author SHA1 Message Date
Ben Abrams 909a18add7 Only set vm.max_map_count if greater than default (#31512)
So the issue here is that we want to avoid setting vm.max_map_count if
it is already equal to the desired value (the bootstrap check requires
262144). The reason we want to avoid this is because in some use-cases
using sysctl to set this will fail. In this case, we want to enable
users to set this value externally and then allow that to cause using
sysctl to set the value to be skipped so that cases where using sysctl
will fail to no longer fail.
2018-06-27 23:38:30 -04:00
Tim Brooks 9ac81a1322
Add Get Snapshots High Level REST API (#31537)
With this commit we add the get snapshots API to the Java high level
REST client.

Relates #27205
2018-06-27 18:04:34 -06:00
Nik Everett d0c276c456
QA: Merge query-builder-bwc to restart test (#30979)
Merges the `query-builder-bwc` qa project into the
`full-cluster-restart` qa project, saving a cluster starts on every
build and *many* cluster starts on `./gradlew bwcTests`.
2018-06-27 16:37:04 -04:00
DeDe Morton 50e60a510d
Update reindex.asciidoc (#31626) 2018-06-27 12:46:29 -07:00
Nik Everett b8907b04b1
Docs: Skip xpack snippet tests if no xpack (#31619)
Skips tests the require xpack if we run the doc build without xpack. So
this should work:
```
./gradlew -p docs check -Dtests.distribution=oss-zip
```

This is implemented by detecting parts of the doc that look like:
```
[testenv="basic"]
```

Relates to #30665
2018-06-27 15:41:44 -04:00
Vladimir Dolzhenko a368195965 mute CreateSnapshotRequestTests 2018-06-27 21:36:28 +02:00
Nik Everett ce1d0f5afd HLRest: Fix test for explain API
The test could sometimes generate an empty array of random stored fields
to fetch which would cause the test to fail. It is fine not to allow
empty here because we already randomize whether or not to generate the
stored fields param anyway.
2018-06-27 14:04:14 -04:00
Yannick Welsch 7520025d7d [TEST] Fix RemoteClusterConnectionTests
With PR #31574 we now ensure that connections are established under the proper
thread context. There is a test in RemoteClusterConnectionTests, however,
that shuts down the service while connecting. With the above change, a new kind
of exception can occur that the test is unaware of.
2018-06-27 18:57:15 +02:00
Jack Conradson 61eefc84f3
Add Create Snapshot to High-Level Rest Client (#31215)
Added support to the high-level rest client for the create snapshot API call. This required 
several changes to toXContent which may need to be cleaned up in a later PR. Also 
added several parsers for fromXContent to be able to retrieve appropriate responses 
along with tests.
2018-06-27 09:30:10 -07:00
Yannick Welsch 01623f66de
Remove legacy MetaDataStateFormat (#31603)
Removes the legacy (pre-1.5) legacy MetaDataStateFormat.
2018-06-27 17:36:58 +02:00
Yu 616703b880 Add explain API to high-level REST client (#31387)
Relates to #27205
2018-06-27 15:20:56 +02:00
Yannick Welsch b7246199db
Preserve thread context when connecting to remote cluster (#31574)
Establishing remote cluster connections uses a queue to coordinate multiple concurrent connect
attempts. Connect attempts can be initiated by user triggered searches as well as by system events 
(e.g. when nodes disconnect). Multiple such concurrent events can lead to the connectListener of
one event to be called under the thread context of another connect attempt. This can lead to the
situation as seen in #31462 where the connect listener is executed under the system context, which 
breaks when fetching the search shards from the remote cluster.

Closes #31462
2018-06-27 10:46:42 +02:00
Piotr Prądzyński 4fc833b1de Unify headers for full text queries
Relates #31599
2018-06-27 10:11:14 +02:00
Piotr Prądzyński f6c64a048d Remove redundant 'minimum_should_match'
Relates #31600
2018-06-27 10:11:07 +02:00
Andrei Stefan 400db4f37d
JDBC driver prepared statement set* methods (#31494)
Added setObject functionality and tests for it
2018-06-27 10:05:39 +03:00
Luca Cavanna a35b5341c4
[TEST] call yaml client close method from test suite (#31591)
We added a way to close the yaml test client with #31575.
Such close method also needs to be called from the test suite though for
the additional clients to be closed.
2018-06-27 08:23:53 +02:00
Armin Braun 13e1cf6191
ingest: Add ignore_missing property to foreach filter (#22147) (#31578) 2018-06-26 20:04:41 +02:00
Julie Tibshirani 26a927a120
Fix a formatting issue in the docvalue_fields documentation. (#31563) 2018-06-26 10:15:56 -07:00
Alpar Torok db90905405 reduce log level at gradle configuration time 2018-06-26 18:16:24 +03:00
Luca Cavanna 823a9d34da
[TEST] Close additional clients created while running yaml tests (#31575)
We recently introduced a mechanism that allows to specify a node
selector as part of do sections (see #31471). When a node selector that
is not the default one is configured, a new client will be initialized
with the same properties as the default one, but with the specified
node selector. This commit improves such mechanism but also closing
the additional clients being created and adding equals/hashcode impl to
the custom node selector as they are cached into a map.
2018-06-26 16:56:35 +02:00
Alexander Reelsen 8a6d062180
Docs: Clarify sensitive fields watcher encryption (#31551)
Clarify the scope of encrypting sensitive settings in watcher, which fields
are encrypted and if users can have their own encrypted fields.
2018-06-26 16:24:28 +02:00
Alexander Reelsen a72dc9e8fc
Watcher: Remove never executed code (#31135)
The removed code snippet was never executed, as the version was never set and
thus always -1, after parsing the watch. With the changes done in
c9d77d20fd this logic would not have
worked correctly anyway.
2018-06-26 16:14:40 +02:00
Alpar Torok 08b8d11e30
Add support for switching distribution for all integration tests (#30874)
* remove left-over comment

* make sure of the property for plugins

* skip installing modules if these exist in the distribution

* Log the distrbution being ran

* Don't allow running with integ-tests-zip passed externally

* top level x-pack/qa can't run with oss distro

* Add support for matching objects in lists

Makes it possible to have a key that points to a list and assert that a
certain object is present in the list. All keys have to be present and
values have to match. The objects in the source list may have additional
fields.

example:
```
  match:  { 'nodes.$master.plugins': { name: ingest-attachment }  }
```

* Update plugin and module tests to work with other distributions

Some of the tests expected that the integration tests will always be ran
with  the `integ-test-zip` distribution so that there will be no other
plugins loaded.

With this change, we check for the presence of the plugin without
assuming exclusivity.

* Allow modules to run on other distros as well

To match the behavior of tets.distributions

* Add and use a new `contains` assertion

Replaces the  previus changes that caused `match` to do a partial match.

* Implement PR review comments
2018-06-26 06:49:03 -07:00
Igor Motov 544e473908
Improve robustness of geo shape parser for malformed shapes (#31449)
Ensures that malformed geoshapes are more reliably ignored if
"ignore_malformed" is set to true instead of failing the entire
document.

Closes #31428
2018-06-26 06:44:10 -07:00
Nik Everett 232c71b6bf
QA: Create xpack yaml features (#31403)
This creates a YAML test "features" that indices if the cluster being
tested has xpack installed (`xpack`) or if it does *not* have xpack
installed (`no_xpack`). It uses those features to centralize skipping
a few tests that fail if xpack is installed.

The plan is to use this in a followup to skip docs tests that require
xpack when xpack is not installed. We *plan* to use the declaration
of required license level on the docs page to generate the required
`skip`.

Closes #30933.
2018-06-26 09:26:48 -04:00
Sohaib Iftikhar ca4c857a90 Improve test times for tests using `RandomObjects::addFields` (#31556)
Currently RandomObjects::addFields can potentially generate a large number of fields This commit decreases the chances that a new object or array is added as a new branch of an object, which lowers the probability of ending up with very big documents generated. It also reduces the number of documents generated for the SimulatePipelineResponseTests from 10 to 5 to reduce the testing time required for parsing.
2018-06-26 12:39:53 +02:00
Tanguy Leroux be9292cac6
[Test] Add full cluster restart test for Rollup (#31533)
This pull request adds a full cluster restart test for a Rollup job. 
The test creates and starts a Rollup job on the cluster and checks 
that the job already exists and is correctly started on the upgraded 
cluster.

This test allows to test that the persistent task state is correctly 
parsed from the cluster state after the upgrade, as the status field 
has been renamed to state in #31031.

The test undercovers a ClassCastException that can be thrown in 
the RollupIndexer when the timestamp as a very low value that fits 
into an integer. When it's the case, the value is parsed back as an 
Integer instead of Long object and (long) position.get(rollupFieldName) 
fails.
2018-06-26 10:07:25 +02:00
Jason Tedor 424be02728
Enhance thread context uniqueness assertion
This commit enhances the assertion message when de-duplicating values on
a thread context header so that if this assertion trips we can see the
values and their correpsonding unique values to understand why this
assertion might trip.
2018-06-26 00:01:45 -04:00
Sue Gallagher 357a07e7a2
[DOCS] Fix heading format errors (#31483)
* [DOCS] Fix heading format errors. Closes #31327

* [DOCS] Fix heading format errors. Closes #31327
2018-06-25 17:25:32 -07:00
Tal Levy c6012ed575
fix writeIndex evaluation for aliases (#31562)
AliasOrIndex.Alias#writeIndex was returning a write index when
an alias was pointing to only one index, regardless whether `is_write_index` was
set to `false`. This fixes that so that there is no write index in such a case
that an alias points to only one index with `is_write_index=false`.
2018-06-25 17:17:16 -07:00
Igor Motov 237650e9c0
Add x-opaque-id to search slow logs (#31539)
Add x-opaque-id to search slow logs only. Indexing slow log and audit
logs will be handled as separate PRs.

Relates #31521
2018-06-25 12:20:27 -07:00
Alexander Reelsen bb1d4aaf17
Watcher: Fix put watch action (#31524)
If no version is specified when putting a watch, the index API should be
used instead of the update API, so that the whole watch gets overwritten
instead of being merged with the existing one.

Merging only happens when a version is specified, so that credentials can be omitted, which is important for the watcher UI.
2018-06-25 18:25:34 +02:00
Michael Basnight adfcea2af6
Add package pre-install check for java binary (#31343)
The package installation relies on java being in the path. If java is
not in the path, the tests fail at post-install time. This commit adds a
pre-install check to validate that java exists, and if it fails, the
package is never installed, and thus keeps a system clean, rather than
aborting at post-install and leaving behind a mess.

Closes #29665
2018-06-25 10:54:39 -05:00
Christoph Büscher 86ab3a2d1a
Reduce number of raw types warnings (#31523)
A first attempt to reduce the number of raw type warnings, 
most of the time by using the unbounded wildcard.
2018-06-25 15:59:03 +02:00
Jonathan Little 8e4768890a Migrate scripted metric aggregation scripts to ScriptContext design (#30111)
* Migrate scripted metric aggregation scripts to ScriptContext design #29328

* Rename new script context container class and add clarifying comments to remaining references to params._agg(s)

* Misc cleanup: make mock metric agg script inner classes static

* Move _score to an accessor rather than an arg for scripted metric agg scripts

This causes the score to be evaluated only when it's used.

* Documentation changes for params._agg -> agg

* Migration doc addition for scripted metric aggs _agg object change

* Rename "agg" Scripted Metric Aggregation script context variable to "state"

* Rename a private base class from ...Agg to ...State that I missed in my last commit

* Clean up imports after merge
2018-06-25 12:01:33 +01:00
Vladimir Dolzhenko 8b698f0bce
turn GetFieldMappingsResponse to ToXContentObject (#31544) 2018-06-25 10:12:31 +02:00
Tom Callahan 629c376f02
Close xcontent parsers (partial) (#31513)
Partial pass at closing XContentParsers in server.  This mostly involved adding try-with-resources statements around the usage of XContentParsers.
2018-06-24 15:39:56 -04:00
Jack Conradson 9efb0fe9ba Ingest Attachment: Upgrade Tika to 1.18 (#31252)
Fixes ES from hanging when a bad zip file is loaded through Tika.
2018-06-24 11:08:45 -07:00
olcbean e401710f6e TEST: Correct the assertion arguments order (#31540)
This commit corrects the order of the assertion arguments in HLRest client.
2018-06-23 22:14:34 -04:00
Vladimir Dolzhenko b7ef75fed6
Add get field mappings to High Level REST API Client (#31423)
Add get field mappings to High Level REST API Client
Relates to #27205
2018-06-23 09:39:17 +02:00
Lisa Cawley b6cc6fc2bc
[DOCS] Updates Watcher examples for code testing (#31152) 2018-06-22 18:09:37 -07:00
Nhat Nguyen 51151027cd TEST: Add bwc recovery tests with synced-flush index
Although the master branch does not affect by #31482, it's helpful to
have BWC tests that verify the peer recovery with a synced-flush index.
This commit adds the bwc tests from #31506 to the master branch.

Relates #31482
Relates #31506
2018-06-22 20:14:06 -04:00
Lisa Cawley 638b9fd88c
[DOCS] Move sql to docs (#31474) 2018-06-22 15:40:25 -07:00
Lisa Cawley eb81a305ae
[DOCS] Move monitoring to docs folder (#31477) 2018-06-22 15:39:34 -07:00
Ryan Ernst 7a150ec06d
Core: Combine doExecute methods in TransportAction (#31517)
TransportAction currently contains 2 doExecute methods, one which takes
a the task, and one that does not. The latter is what some subclasses
implement, while the first one just calls the latter, dropping the given
task. This commit combines these methods, in favor of just always
assuming a task is present.
2018-06-22 15:03:01 -07:00
Vladimir Dolzhenko f04c579203
IndexShard should not return null stats (#31528)
IndexShard should not return null stats - empty stats or AlreadyCloseException if it's closed is better
2018-06-22 21:08:11 +02:00
Vladimir Dolzhenko 7313a987f4
fix repository update with the same settings but different type (#31458)
fix repository update with the same settings but different type
2018-06-22 17:44:13 +02:00
Armin Braun 3c42bfad4e
Fix Mockito trying to mock IOException that isn't thrown by method (#31433) (#31527) 2018-06-22 17:24:27 +02:00
Luca Cavanna 16e4e7a7cf
Node selector per client rather than per request (#31471)
We have made node selectors configurable per request, but all 
of other language clients don't allow for that.
A good reason not to do so, is that having a different node selector 
per request breaks round-robin. This commit makes NodeSelector 
configurable only at client initialization. It also improves the docs 
on this matter, important given that a single node selector can still 
affect round-robin.
2018-06-22 17:15:29 +02:00
Ryan Ernst 59e7c6411a
Core: Combine messageRecieved methods in TransportRequestHandler (#31519)
TransportRequestHandler currently contains 2 messageReceived methods,
one which takes a Task, and one that does not. The first just delegates
to the second. This commit changes all existing implementors of
TransportRequestHandler to implement the version which takes Task, thus
allowing the class to be a functional interface, and eliminating the
need to throw exceptions when a task needs to be ensured.
2018-06-22 07:36:03 -07:00