Commit Graph

519 Commits

Author SHA1 Message Date
Yannick Welsch 11f6983d69
HLRC: Add delete watch action ()
Adds the "delete watch" API to the High-Level Rest Client.

Relates 
2018-07-31 10:29:22 +02:00
Igor Motov 29c802f88e
Rest HL client: Add put license action ()
In the HL REST client we replace the License object with a string, because of 
complexity of this class. It is also not really needed on the client side since 
end-users are not interacting with the license besides passing it as a string 
to the server.

Relates 
2018-07-24 11:19:07 -07:00
Daniel Mitterdorfer 73a38895fd
Add Restore Snapshot High Level REST API
With this commit we add the restore snapshot API to the Java high level
REST client.

Relates 
Relates 
2018-07-24 16:17:09 +02:00
Christoph Büscher 59cf600e03
Register ERR metric with NamedXContentRegistry ()
This adds the ERR metric to the provided xContent parsers in the module and the
high level rest client registry. Also adding integration tests to make sure the
metric is correctly registered and usable from the client.
2018-07-24 16:05:43 +02:00
Christoph Büscher fe6bb75eb4
Rename ranking evaluation `quality_level` to `metric_score` ()
The notion of "quality" is an overloaded term in the search ranking evaluation 
context. Its usually used to decribe certain levels of "good" vs. "bad" of a 
seach result with respect to the users information need. We currently report the 
result of the ranking evaluation as `quality_level` which is a bit missleading.
This changes the response parameter name to `metric_score` which fits better.
2018-07-23 22:25:02 +02:00
Christoph Büscher ff87b7aba4
Remove unnecessary warning supressions () 2018-07-23 11:31:04 +02:00
Luca Cavanna 1390a849e1
[TEST] improve REST high-level client naming conventions check ()
Check the deprecated methods are effectively deprecated.
Also compare the class rather than their names when checking argument types.
2018-07-21 10:07:08 +02:00
Christoph Büscher 5cbd9ad177
Rename ranking evaluation response section ()
Currently the ranking evaluation response contains a 'unknown_docs' section 
for each search use case in the evaluation set. It contains document ids for 
results in the search hits that currently don't have a quality rating.
This change renames it to `unrated_docs`, which better reflects its purpose.
2018-07-20 11:43:46 +02:00
Alexander Reelsen 202894b832
Rest HL client: Add put watch action ()
Relates 

This implementation behaves like the current transport client, that you basically cannot configure a Watch POJO representation as an argument to the put watch API, but only a bytes reference. You can use the the `WatchSourceBuilder` from the `org.elasticsearch.plugin:x-pack-core` dependency to build watches.

This commit also changes the license type to trial, so that watcher is available in high level rest client tests.

/cc @hub-cap
2018-07-19 10:40:54 +02:00
Alpar Torok 38e2e1d553
Detect and prevent configuration that triggers a Gradle bug ()
* Detect and prevent configuration that triggers a Gradle bug

As we found in , this can lead to a lot of wasted time as it's not
immediatly obvius what's going on.
Givent how many projects we have it's getting increasingly easier to run
into .
2018-07-19 06:46:58 +00:00
Nik Everett 1b97652a4c
Build: Move shadow customizations into common code ()
Moves the customizations to the build to produce nice shadow jars and
javadocs into common build code, mostly BuildPlugin with a little into
the root build.gradle file. This means that any project that applies the
shadow plugin will automatically be set up just like the high level rest
client:
* The non-shadow jar will not be built
* The shadow jar will not have a "classifier"
* Tests will run against the shadow jar
* Javadoc will include all of the shadowed classes
* Service files in `META-INF/services` will be merged
2018-07-17 14:20:41 -04:00
Luca Cavanna 049966a829
Check that client methods match API defined in the REST spec ()
We have been encountering name mismatches between API defined in our
REST spec and method names that have been added to the high-level REST
client. We should check this automatically to prevent furher mismatches,
and correct all the current ones.

This commit adds a test for this and corrects the issues found by it.
2018-07-17 11:26:28 +02:00
Ioannis Kakavas 9e529d9d58
Enable testing in FIPS140 JVM ()
Ensure our tests can run in a FIPS JVM

JKS keystores cannot be used in a FIPS JVM as attempting to use one
in order to init a KeyManagerFactory or a TrustManagerFactory is not
allowed.( JKS keystore algorithms for private key encryption are not
FIPS 140 approved)
This commit replaces JKS keystores in our tests with the
corresponding PEM encoded key and certificates both for key and trust
configurations.
Whenever it's not possible to refactor the test, i.e. when we are
testing that we can load a JKS keystore, etc. we attempt to
mute the test when we are running in FIPS 140 JVM. Testing for the
JVM is naive and is based on the name of the security provider as
we would control the testing infrastrtucture and so this would be
reliable enough.
Other cases of tests being muted are the ones that involve custom
TrustStoreManagers or KeyStoreManagers, null TLS Ciphers and the
SAMLAuthneticator class as we cannot sign XML documents in the
way we were doing. SAMLAuthenticator tests in a FIPS JVM can be
reenabled with precomputed and signed SAML messages at a later stage.

IT will be covered in a subsequent PR
2018-07-17 10:54:10 +03:00
Yogesh Gaikwad 6717df3c2d
[Test] Modify assert statement for ssl handshake ()
There have been changes in error messages for `SSLHandshakeException`.
This has caused a couple of failures in our tests.
This commit modifies test verification to assert on exception type of
class `SSLHandshakeException`.
There was another issue in Java11 which caused NPE. The bug has now
been fixed on Java11 - early access build 22.
Bug Ref: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8206355
Enable the skipped tests due to this bug.

Closes 
2018-07-17 15:43:03 +10:00
Christoph Büscher 3587d8872e
[Tests] Fix failure due to changes exception message ()
Java 11 seems to get more verbose on the ClassCastException we check for in
SearchDocumentationIT. This changes the test from asserting the exact exception
message to only checking the two classes involved are part of the message.

Closes 
2018-07-16 11:22:42 +02:00
Jack Conradson 42ca520377
Clean Up Snapshot Create Rest API ()
Make SnapshotInfo and CreateSnapshotResponse parsers lenient for backwards compatibility.  Remove extraneous fields from CreateSnapshotRequest toXContent.
2018-07-13 13:07:26 -07:00
Colin Goodheart-Smithe 0edb096eb4 Adds a new auto-interval date histogram ()
* Adds a new auto-interval date histogram

This change adds a new type of histogram aggregation called `auto_date_histogram` where you can specify the target number of buckets you require and it will find an appropriate interval for the returned buckets. The aggregation works by first collecting documents in buckets at second interval, when it has created more than the target number of buckets it merges these buckets into minute interval bucket and continues collecting until it reaches the target number of buckets again. It will keep merging buckets when it exceeds the target until either collection is finished or the highest interval (currently years) is reached. A similar process happens at reduce time.

This aggregation intentionally does not support min_doc_count, offest and extended_bounds to keep the already complex logic from becoming more complex. The aggregation accepts sub-aggregations but will always operate in `breadth_first` mode deferring the computation of sub-aggregations until the final buckets from the shard are known. min_doc_count is effectively hard-coded to zero meaning that we will insert empty buckets where necessary.

Closes 

* Adds documentation

* Added sub aggregator test

* Fixes failing docs test

* Brings branch up to date with master changes

* trying to get tests to pass again

* Fixes multiBucketConsumer accounting

* Collects more buckets than needed on shards

This gives us more options at reduce time in terms of how we do the
final merge of the buckeets to produce the final result

* Revert "Collects more buckets than needed on shards"

This reverts commit 993c782d117892af9a3c86a51921cdee630a3ac5.

* Adds ability to merge within a rounding

* Fixes nonn-timezone doc test failure

* Fix time zone tests

* iterates on tests

* Adds test case and documentation changes

Added some notes in the documentation about the intervals that can bbe
returned.

Also added a test case that utilises the merging of conseecutive buckets

* Fixes performance bug

The bug meant that getAppropriate rounding look a huge amount of time
if the range of the data was large but also sparsely populated. In
these situations the rounding would be very low so iterating through
the rounding values from the min key to the max keey look a long time
(~120 seconds in one test).

The solution is to add a rough estimate first which chooses the
rounding based just on the long values of the min and max keeys alone
but selects the rounding one lower than the one it thinks is
appropriate so the accurate method can choose the final rounding taking
into account the fact that intervals are not always fixed length.

Thee commit also adds more tests

* Changes to only do complex reduction on final reduce

* merge latest with master

* correct tests and add a new test case for 10k buckets

* refactor to perform bucket number check in innerBuild

* correctly derive bucket setting, update tests to increase bucket threshold

* fix checkstyle

* address code review comments

* add documentation for default buckets

* fix typo
2018-07-13 13:08:35 -04:00
Ryan Ernst 2c3ea43f45
HLRC: Add xpack usage api ()
This commit adds the _xpack/usage api to the high level rest client.
Currently in the transport api, the usage data is exposed in a limited
fashion, at most giving one level of helper methods for the inner keys
of data, but then exposing thos subobjects as maps of objects. Rather
than making parsers for every set of usage data from each feature, this
PR exposes the entire set of usage data as a map of maps.
2018-07-13 09:33:27 -07:00
Christoph Büscher 69c2b2d802 [Tests] Mute failing SearchDocumentationIT#testSearchRequestAggregations 2018-07-13 10:56:47 +02:00
Martijn van Groningen ae5c70e603
muted tests due to 2018-07-12 11:51:33 +02:00
James Baiera 5bcdff73d7
Add Snapshots Status API to High Level Rest Client ()
This PR adds the Snapshots Status API to the Snapshot Client, as 
well as additional documentation for the status api.
2018-07-11 12:07:31 -04:00
Nik Everett eda6d182b5
Switch low level rest tests to new style Requests ()
In  we added `Request` object flavored requests to the low level
REST client and in  we deprecated the old `performRequest`s. This
changes all calls in the `client/rest` project to use the new versions.
2018-07-11 09:48:47 -04:00
Nik Everett aa6a1c5ca0
Switch high level rest tests to new style requests ()
In  we added `Request` object flavored requests to the low level
REST client and in  we deprecated the old `performRequest`s. This
changes all calls in the `client/rest-high-level` project to use the new
versions.
2018-07-11 09:18:04 -04:00
Nik Everett 4d83a0dd5a
HLREST: Bundle the x-pack protocol project ()
The `:x-pack:protocol` project is an implementation detail shared by the
xpack projects and the high level rest client and really doesn't deserve
its own maven coordinants and published javadoc. This change bundles
`:x-pack:protocol` into the high level rest client.

Relates to 
2018-07-10 17:59:15 -04:00
Nik Everett dcbb1154bf
HLRest: Move xPackInfo() to xPack().info() ()
Originally I put the X-Pack info object into the top level rest client
object. I did that because we thought we'd like to squash `xpack` from
the name of the X-Pack APIs now that it is part of the default
distribution. We still kind of want to do that, but at least for now we
feel like it is better to keep the high level rest client aligned with
the other language clients like C# and Python. This shifts the X-Pack
info API to align with its json spec file.

Relates to 
2018-07-10 13:01:28 -04:00
Nik Everett fb27f3e7f0
HLREST: Add x-pack-info API ()
This is the first x-pack API we're adding to the high level REST client
so there is a lot to talk about here!

= Open source

The *client* for these APIs is open source. We're taking the previously
Elastic licensed files used for the `Request` and `Response` objects and
relicensing them under the Apache 2 license.

The implementation of these features is staying under the Elastic
license. This lines up with how the rest of the Elasticsearch language
clients work.

= Location of the new files

We're moving all of the `Request` and `Response` objects that we're
relicensing to the `x-pack/protocol` directory. We're adding a copy of
the Apache 2 license to the root fo the `x-pack/protocol` directory to
line up with the language in the root `LICENSE.txt` file. All files in
this directory will have the Apache 2 license header as well. We don't
want there to be any confusion. Even though the files are under the
`x-pack` directory, they are Apache 2 licensed.

We chose this particular directory layout because it keeps the X-Pack
stuff together and easier to think about.

= Location of the API in the REST client

We've been following the layout of the rest-api-spec files for other
APIs and we plan to do this for the X-Pack APIs with one exception:
we're dropping the `xpack` from the name of most of the APIs. So
`xpack.graph.explore` will become `graph().explore()` and
`xpack.license.get` will become `license().get()`.

`xpack.info` and `xpack.usage` are special here though because they
don't belong to any proper category. For now I'm just calling
`xpack.info` `xPackInfo()` and intend to call usage `xPackUsage` though
I'm not convinced that this is the final name for them. But it does get
us started.

= Jars, jars everywhere!

This change makes the `xpack:protocol` project a `compile` scoped
dependency of the `x-pack:plugin:core` and `client:rest-high-level`
projects. I intend to keep it a compile scoped dependency of
`x-pack:plugin:core` but I intend to bundle the contents of the protocol
jar into the `client:rest-high-level` jar in a follow up. This change
has grown large enough at this point.

In that followup I'll address javadoc issues as well.

= Breaking-Java

This breaks that transport client by a few classes around. We've
traditionally been ok with doing this to the transport client.
2018-07-08 11:03:56 -04:00
Tim Brooks a5f5ea8422
Get snapshot rest client cleanups ()
This is a followup to . It makes a number of changes requested by
a review that came after the PR was merged. These are mostly cleanups
and doc improvements.
2018-07-05 17:19:46 -06:00
Sohaib Iftikhar 09e8ac8167 REST high-level client: add get index API ()
Also added master_timeout parameter for the indices.get spec

Relates to 
2018-07-05 13:52:25 -04:00
Alan Woodward 1d114071da
Add analyze API to high-level rest client () 2018-07-03 15:57:02 +01:00
Sohaib Iftikhar a5fd4a7709 Implemented XContent serialisation for GetIndexResponse ()
This PR does the server side work for adding the Get Index API to the REST
high-level-client, namely moving resolving default settings to the
transport action. A follow up would be the client side changes.
2018-07-03 08:08:50 -04:00
Sohaib Iftikhar c55d11f8b5 rest-high-level: added get cluster settings ()
Relates to 
2018-07-02 13:25:17 -04:00
Konrad Beiske 2971dd56ca Enable setting client path prefix to / ()
Some proxies require all requests to have paths starting with / since
there are no relative paths at the HTTP connection level. Elasticsearch
assumes paths are absolute. In order to run rest tests against a cluster
behind such a proxy, set the system property
tests.rest.client_path_prefix to /.
2018-07-01 13:42:03 -04:00
Alpar Torok 200e1f45f2
Fix gradle4.8 deprecation warnings ()
* remove explicit wrapper task

It's created by Gradle and triggers a deprecation warning
Simplify configuration

* Upgrade shadow plugin to get rid of Gradle deprecation

* Move compile configuration to base plugin

Solves Gradle deprecation warning from earlier Gradle versions

* Enable stable publishing in the Gradle build

* Replace usage of deprecated property

* bump Gradle version in build compare
2018-06-29 19:17:19 +00:00
Armin Braun 7a76e3a4fa
TEST: Fix test task invocation () 2018-06-29 10:06:36 +02:00
Tim Brooks 3fce13fd28 Fix syntax errors in get-snapshots docs ()
This is related to . It fixes two syntax errors that are breaking
the docs build.
2018-06-28 08:40:33 -05:00
markharwood 09dd19a403
Add MultiSearchTemplate support to High Level Rest client ()
Add MultiSearchTemplate support to High Level Rest client.
Addresses part of 
2018-06-28 14:05:26 +01:00
Luca Cavanna 48cfb9b0db
Add test for low-level client round-robin behaviour () 2018-06-28 15:00:36 +02:00
Tim Brooks cff7be16fb
Fix missing get-snapshots docs reference
This is related to . It fixes a missing docs references in
get_snapshots.asciidoc.
2018-06-28 05:04:04 -06:00
Alpar Torok 8557bbab28
Upgrade gradle wrapper to 4.8 ()
* Move to Gradle 4.8 RC1

* Use latest version of plugin

The current does not work with Gradle 4.8 RC1

* Switch to Gradle GA

* Add and configure build compare plugin

* add work-around for https://github.com/gradle/gradle/issues/5692

* work around https://github.com/gradle/gradle/issues/5696

* Make use of Gradle build compare with reference project

* Make the manifest more compare friendly

* Clear the manifest in compare friendly mode

* Remove animalsniffer from buildscript classpath

* Fix javadoc errors

* Fix doc issues

* reference Gradle issues in comments

* Conditionally configure build compare

* Fix some more doclint issues

* fix typo in build script

* Add sanity check to make sure the test task was replaced

Relates to . It seems like Gradle has an inconsistent behavior and
the taks is not always replaced.

* Include number of non conforming tasks in the exception.

* No longer replace test task, create implicit instead

Closes . The issue has full context in comments.

With this change the `test` task becomes nothing more than an alias for `utest`.
Some of the stand alone tests that had a `test` task now have `integTest`, and a
few of them that used to have `integTest` to run multiple tests now only
have `check`.
This will also help separarate unit/micro tests from integration tests.

* Revert "No longer replace test task, create implicit instead"

This reverts commit f1ebaf7d93e4a0a19e751109bf620477dc35023c.

* Fix replacement of the test task

Based on information from  replace the task taking
into account the task providres.
Closes .

* Only apply build comapare plugin if needed

* Make sure test runs before integTest

* Fix doclint aftter merge

* PR review comments

* Switch to Gradle 4.8.1 and remove workaround

* PR review comments

* Consolidate task ordering
2018-06-28 08:13:21 +03:00
Tim Brooks 9ac81a1322
Add Get Snapshots High Level REST API ()
With this commit we add the get snapshots API to the Java high level
REST client.

Relates 
2018-06-27 18:04:34 -06:00
Nik Everett ce1d0f5afd HLRest: Fix test for explain API
The test could sometimes generate an empty array of random stored fields
to fetch which would cause the test to fail. It is fine not to allow
empty here because we already randomize whether or not to generate the
stored fields param anyway.
2018-06-27 14:04:14 -04:00
Jack Conradson 61eefc84f3
Add Create Snapshot to High-Level Rest Client ()
Added support to the high-level rest client for the create snapshot API call. This required 
several changes to toXContent which may need to be cleaned up in a later PR. Also 
added several parsers for fromXContent to be able to retrieve appropriate responses 
along with tests.
2018-06-27 09:30:10 -07:00
Yu 616703b880 Add explain API to high-level REST client ()
Relates to 
2018-06-27 15:20:56 +02:00
Luca Cavanna 823a9d34da
[TEST] Close additional clients created while running yaml tests ()
We recently introduced a mechanism that allows to specify a node
selector as part of do sections (see ). When a node selector that
is not the default one is configured, a new client will be initialized
with the same properties as the default one, but with the specified
node selector. This commit improves such mechanism but also closing
the additional clients being created and adding equals/hashcode impl to
the custom node selector as they are cached into a map.
2018-06-26 16:56:35 +02:00
Christoph Büscher 86ab3a2d1a
Reduce number of raw types warnings ()
A first attempt to reduce the number of raw type warnings, 
most of the time by using the unbounded wildcard.
2018-06-25 15:59:03 +02:00
olcbean e401710f6e TEST: Correct the assertion arguments order ()
This commit corrects the order of the assertion arguments in HLRest client.
2018-06-23 22:14:34 -04:00
Vladimir Dolzhenko b7ef75fed6
Add get field mappings to High Level REST API Client ()
Add get field mappings to High Level REST API Client
Relates to 
2018-06-23 09:39:17 +02:00
Ryan Ernst 7a150ec06d
Core: Combine doExecute methods in TransportAction ()
TransportAction currently contains 2 doExecute methods, one which takes
a the task, and one that does not. The latter is what some subclasses
implement, while the first one just calls the latter, dropping the given
task. This commit combines these methods, in favor of just always
assuming a task is present.
2018-06-22 15:03:01 -07:00
Luca Cavanna 16e4e7a7cf
Node selector per client rather than per request ()
We have made node selectors configurable per request, but all 
of other language clients don't allow for that.
A good reason not to do so, is that having a different node selector 
per request breaks round-robin. This commit makes NodeSelector 
configurable only at client initialization. It also improves the docs 
on this matter, important given that a single node selector can still 
affect round-robin.
2018-06-22 17:15:29 +02:00
Sohaib Iftikhar eade161894 REST high-level client: add simulate pipeline API ()
relates to 
2018-06-22 09:59:04 +02:00