Commit Graph

469 Commits

Author SHA1 Message Date
Ryan Ernst 4f9332ee16
Core: Remove ThreadPool from base TransportAction ()
Most transport actions don't need the node ThreadPool. This commit
removes the ThreadPool as a super constructor parameter for
TransportAction. The actions that do need the thread pool then have a
member added to keep it from their own constructor.
2018-06-21 11:25:26 -07:00
Nik Everett 6f3e97f2b7 Test: Skip assertion on windows
Windows doesn't provide consistent exception messages when it can't
connect so skip the exception message assertion on windows.

Closes 
2018-06-21 13:24:39 -04:00
Daniel Mitterdorfer 90d62e6e4d
Add Delete Snapshot High Level REST API
With this commit we add the delete snapshot API to the Java high level
REST client.

Relates 
Relates 
2018-06-20 16:42:50 +02:00
Ryan Ernst 401800d958
Core: Remove index name resolver from base TransportAction ()
Most transport actions don't need to resolve index names. This commit
removes the index name resolver as a super constructor parameter for
TransportAction. The actions that do need the resolver then have a
member added to keep the resolver from their own constructor.
2018-06-19 17:06:09 -07:00
Vladimir Dolzhenko 04e4e44409
Add get stored script and delete stored script to high level REST API ()
Add get stored script and delete stored script to high level REST API

Relates to 
2018-06-19 14:21:11 +02:00
Ryan Ernst e67aa96c81
Core: Combine Action and GenericAction ()
Since , Action no longer has anything but a call to the
GenericAction super constructor. This commit renames GenericAction
into Action, thus eliminating the Action class. Additionally, this
commit removes the Request generic parameter of the class, since
it was unused.
2018-06-18 23:53:04 +02:00
Sohaib Iftikhar c4f8df3ad6 REST high-level client: add validate query API ()
Adds the validate query API to the high level rest client.
2018-06-18 09:59:29 -04:00
Nik Everett 856936c286
REST Client: NodeSelector for node attributes ()
Add a `NodeSelector` so that users can filter the nodes that receive
requests based on node attributes.

I believe we'll need this to backport  and we want it anyway.

I also added a bash script to help with rebuilding the sniffer parsing
test documents.
2018-06-15 08:04:54 -04:00
Nik Everett 045f76d67f LLClient: Fix assertion on windows
In windows the exception message is ever so slightly differant than in
Linux and OSX. That is fine. We'll just catch either.
2018-06-15 08:03:53 -04:00
Christoph Büscher a0d6c19e75
Add details section for dcg ranking metric ()
While the other two ranking evaluation metrics (precicion and reciprocal rank)
already provide a more detailed output for how their score is calculated, the
discounted cumulative gain metric (dcg) and its normalized variant are lacking
this until now. Its not really clear which level of detail might be useful for
debugging and understanding the final metric calculation, but this change adds a
`metric_details` section to REST output that contains some information about the
evaluation details.
2018-06-15 11:56:16 +02:00
Tanguy Leroux 4d7447cb5e
Reenable Checkstyle's unused import rule () 2018-06-14 09:52:46 +02:00
Luca Cavanna 24163d10b7
REST hl client: cluster health to default to cluster level ()
With  we added support for the cluster health API to the
high-level REST client. The transport client does not support the level
parameter, and it always returns all the info needed for shards level
rendering. We have maintained that behaviour when adding support for
cluster health to the high-level REST client, to ease migration, but the
correct thing to do is to default the high-level REST client to
`cluster` level, which is the same default as when going through the
Elasticsearch REST layer.
2018-06-13 15:06:13 +02:00
Luca Cavanna bd51b6b96a
REST hl client: adjust wait_for_active_shards param in cluster health ()
The default wait_for_active_shards is NONE for cluster health, which
differs from all the other API in master, hence we need to make sure to
set the parameter whenever it differs from NONE (0). The test around
this also had a bug, which is why this was not originally uncovered.

Relates to 
2018-06-12 21:00:33 +02:00
Luca Cavanna 92eb324776
REST high-level Client: remove deprecated API methods ()
This commit removes all the API methods that accept a `Header` varargs
argument, in favour of the newly introduced API methods that accept a
`RequestOptions` argument.

Relates to 
2018-06-12 21:00:06 +02:00
Van0SS d5e8a5cd69 REST high-level client: add Cluster Health API ()
Relates to 
2018-06-12 13:34:06 +02:00
Luca Cavanna 113c1916ee
SyncedFlushResponse to implement ToXContentObject ()
The response currently implements ToXContentFragment although the only time it's used
it is supposed to print out a complete object rather than a fragment. Note that this
is the client version of the response, used only in the high-level client.
2018-06-12 10:27:51 +02:00
olcbean 7d7ead95b2 Add Get Aliases API to the high-level REST client ()
Given the weirdness of the response returned by the get alias API, we went for a client specific response, which allows us to hold the error message, exception and status returned as part of the response together with aliases. See  .

Relates to 
2018-06-12 10:26:17 +02:00
Nik Everett 0d9b78834f
LLClient: Support host selection ()
Allows users of the Low Level REST client to specify which hosts a
request should be run on. They implement the  `NodeSelector` interface
or reuse a built in selector like `NOT_MASTER_ONLY` to chose which nodes
are valid. Using it looks like:
```
Request request = new Request("POST", "/foo/_search");
RequestOptions options = request.getOptions().toBuilder();
options.setNodeSelector(NodeSelector.NOT_MASTER_ONLY);
request.setOptions(options);
...
```

This introduces a new `Node` object which contains a `HttpHost` and the
metadata about the host. At this point that metadata is just `version`
and `roles` but I plan to add node attributes in a followup. The
canonical way to **get** this metadata is to use the `Sniffer` to pull
the information from the Elasticsearch cluster.

I've marked this as "breaking-java" because it breaks custom
implementations of `HostsSniffer` by renaming the interface to
`NodesSniffer` and by changing it from returning a `List<HttpHost>` to a
`List<Node>`. It *shouldn't* break anyone else though.

Because we expect to find it useful, this also implements `host_selector`
support to `do` statements in the yaml tests. Using it looks a little
like:

```
---
"example test":
  - skip:
      features: host_selector
  - do:
      host_selector:
        version: " - 7.0.0" # same syntax as skip
      apiname:
        something: true
```

The `do` section parses the `version` string into a host selector that
uses the same version comparison logic as the `skip` section. When the
`do` section is executed it passed the off to the `RestClient`, using
the `ElasticsearchHostsSniffer` to sniff the required metadata.

The idea is to use this in mixed version tests to target a specific
version of Elasticsearch so we can be sure about the deprecation
logging though we don't currently have any examples that need it. We do,
however, have at least one open pull request that requires something
like this to properly test it.

Closes 
2018-06-11 17:07:27 -04:00
Nhat Nguyen 99e04582de
HLRest: Add get index templates API ()
Relates 
2018-06-11 11:06:28 -04:00
Tanguy Leroux bf58660482
Remove all unused imports and fix CRLF ()
The X-Pack opening and the recent other refactorings left a lot of 
unused imports in the codebase. This commit removes them all.
2018-06-11 15:12:12 +02:00
Jason Tedor 65c107b47d
Fix unknown licenses ()
The goal of this commit is to address unknown licenses when producing
the dependencies info report. We have two different checks that we run
on licenses. The first check is whether or not we have stashed a copy of
the license text for a dependency in the repository. The second is to
map every dependency to a license type (e.g., BSD 3-clause). The problem
here is that the way we were handling licenses in the second check
differs from how we handle licenses in the first check. The first check
works by finding a license file with the name of the artifact followed
by the text -LICENSE.txt. Yet in some cases we allow mapping an artifact
name to another name used to check for the license (e.g., we map
lucene-.* to lucene, and opensaml-.* to shibboleth. The second check
understood the first way of looking for a license file but not the
second way. So in this commit we teach the second check about the
mappings from artifact names to license names. We do this by copying the
configuration from the dependencyLicenses task to the dependenciesInfo
task and then reusing the code from the first check in the second
check. There were some other challenges here though. For example,
dependenciesInfo was checking too many dependencies. For now, we should
only be checking direct dependencies and leaving transitive dependencies
from another org.elasticsearch artifact to that artifact (we want to do
this differently in a follow-up). We also want to disable
dependenciesInfo for projects that we do not publish, users only care
about licenses they might be exposed to if they use our assembled
products. With all of the changes in this commit we have eliminated all
unknown licenses. A follow-up will enforce that when we add a new
dependency it does not get mapped to unknown, these will be forbidden in
the future. Therefore, with this change and earlier changes are left
having no unknown licenses and two custom licenses; custom here means it
does not map to an SPDX license type. Those two licenses are xz and
ldapsdk. A future change will not allow additional custom licenses
unless they are explicitly whitelisted. This ensures that if a new
dependency is added it is mapped to an SPDX license or mapped to custom
because it does not have an SPDX license.
2018-06-09 07:28:41 -04:00
Paul Sanwald e82e5cc2e8
high level REST api: cancel task ()
* Initial commit of rest high level exposure of cancel task

* fix javadocs

* address some code review comments

* update branch to use tasks namespace instead of cluster

* High-level client: list tasks failure to not lose nodeId

This commit reworks testing for `ListTasksResponse` so that random
fields insertion can be tested and xcontent equivalence can be checked
too. Proper exclusions need to be configured, and failures need to be
tested separately. This helped finding a little problem, whenever there
is a node failure returned, the nodeId was lost as it was never printed
out as part of the exception toXContent.

* added comment

* merge from master

* re-work CancelTasksResponseTests to separate XContent failure cases from non-failure cases

* remove duplication of logic in parser creation

* code review changes

* refactor TasksClient to support RequestOptions

* add tests for parent task id

* address final PR review comments, mostly formatting and such
2018-06-07 14:02:23 -07:00
Jason Tedor d8c0a39c15
Remove vestiges of animal sniffer ()
We no longer need animal sniffer because we use JDK functionality
(introduced in JDK 9) to target older versions of the JDK for
compilation. This functionality means that the JDK handles the problem
of ensuring that we do not use JDK APIs from the version that we are
compiling from that are not available in the version that we are
compiling to. A previous commit removed this for the REST client (where
we target JDK 7) but a few traces were left behind.
2018-06-07 17:00:22 -04:00
Jason Tedor 94be9b471f
Rename elasticsearch-core to core ()
This commit renames :libs:elasticsearch-core to :libs:core.
2018-06-07 16:50:21 -04:00
Luca Cavanna be4a101ea1
Add high-level client methods that accept RequestOptions ()
With  we have introduced a new way to provide request options
whenever sending a request using the high-level REST client. Before you
could provide headers as the last argument varargs of each API method,
now you can provide `RequestOptions` that in the future will allow to
provide more options which can be specified per request.

This commit deprecates all of the client methods that accept a `Header`
varargs argument in favour of new methods that accept `RequestOptions`
instead. For some API we don't even go through deprecation given that
they were not released since they were added, hence in that case we can
just move them to the new method.
2018-06-06 23:17:45 +02:00
Lee Hinman b22a055bcf
Add get mappings support to high-level rest client ()
This adds support for the get mappings API to the high level rest client.

Relates to 
2018-06-04 14:31:08 -06:00
Sohaib Iftikhar 3809450bf3 Move pipeline APIs to ingest namespace () 2018-06-04 10:34:55 +02:00
Sohaib Iftikhar 11887fa54a REST high-level client: add delete ingest pipeline API ()
Relates to 
2018-06-01 14:13:41 +02:00
Sohaib Iftikhar 80d20a9010 REST high-level client: add get ingest pipeline API ()
Relates to 
2018-06-01 08:55:43 +02:00
Nik Everett b225f5e5c6
HLRest: Allow caller to set per request options ()
This modifies the high level rest client to allow calling code to
customize per request options for the bulk API. You do the actual
customization by passing a `RequestOptions` object to the API call
which is set on the `Request` that is generated by the high level
client. It also makes the `RequestOptions` a thing in the low level
rest client. For now that just means you use it to customize the
headers and the `httpAsyncResponseConsumerFactory` and we'll add
node selectors and per request timeouts in a follow up.

I only implemented this on the bulk API because it is the first one
in the list alphabetically and I wanted to keep the change small
enough to review. I'll convert the remaining APIs in a followup.
2018-05-31 13:59:52 -04:00
Ryan Ernst 46e8d97813
Core: Remove RequestBuilder from Action ()
This commit removes the RequestBuilder generic type from Action. It was
needed to be used by the newRequest method, which in turn was used by
client.prepareExecute. Both of these methods are now removed, along with
the existing users of prepareExecute constructing the appropriate
builder directly.
2018-05-31 16:15:00 +02:00
Luca Cavanna 63f3a61134
Refactor Sniffer and make it testable ()
This commit reworks the Sniffer component to simplify it and make it possible to test it.

In particular, it no longer takes out the host that failed when sniffing on failure, but rather relies on whatever the cluster returns. This is the result of some valid comments from . Taking out one single host is too naive, hard to test and debug.

A new Scheduler abstraction is introduced to abstract the tasks scheduling away and make it possible to plug in any test implementation and take out timing aspects when testing.

Concurrency aspects have also been improved, synchronized methods are no longer required. At the same time, we were able to take  and  into account and fix them, especially now that we can more easily add tests.

Last but not least, unit tests are added for the Sniffer component, long overdue.

Closes 
Closes 
2018-05-31 16:05:09 +02:00
Michael Basnight b716b08197
Add Verify Repository High Level REST API ()
This commit adds Verify Repository, the associated docs and tests for
the high level REST API client. A few small changes to the Verify
Repository Response went into the commit as well.

Relates 
2018-05-30 11:10:00 -05:00
Sohaib Iftikhar 2c27c58718 REST high-level client: add synced flush API (2) ()
Adds the synced flush API to the high level REST client.

Relates to .
2018-05-30 07:32:52 -04:00
Luca Cavanna eaee530778
Move list tasks under Tasks namespace ()
Our API spec define the tasks API as e.g. tasks.list, meaning that they belong to their own namespace. This commit moves them from the cluster namespace to their own namespace.

Relates to 
2018-05-29 10:54:41 +02:00
Sohaib Iftikhar 5a97423b7a REST high-level client: add put ingest pipeline API ()
REST high-level client: add put ingest pipeline API

Adds the put ingest pipeline API to the high level rest client.
2018-05-24 19:02:26 -04:00
Luca Cavanna a17d6cab98
Replace Request#setHeaders with addHeader ()
Adding headers rather than setting them all at once seems more
user-friendly and we already do it in a similar way for parameters
(see Request#addParameter).
2018-05-22 20:32:30 +02:00
Luca Cavanna 0d37ac4e8c
[TEST] remove endless wait in RestClientTests ()
This commit adds a max wait timeout of one second to all the latch.await
calls made in RestClientTests. It also makes clearer that the `onSuccess`
listener method will never be called given that the underlying http
client is mocked and makes sure that `latch.countDown` is always called
2018-05-22 20:31:36 +02:00
Michael Basnight c6be3b4e5a
Add Delete Repository High Level REST API ()
This commit adds Delete Repository, the associated docs and tests for
the high level REST API client. It also cleans up a seemingly innocuous
line in the RestDeleteRepositoryAction and some naming in SnapshotIT.

Relates 
2018-05-21 19:52:21 -05:00
Jason Tedor d68c44b76c
Default copy settings to true and deprecate on the REST layer ()
This commit defaults the copy_settings REST parameter to the shrink and
split APIs to true, and deprecates the parameter.
2018-05-18 10:12:08 -04:00
Tim Vernum 8a89306aaf
Preserve REST client auth despite 401 response ()
The default behaviour for Apache HTTP client is to mimic the standard
browser behaviour of clearing the authentication cache (for a given
host) if that host responds with 401.
This behaviour is appropriate in a interactive browser environment
where the user is given the opportunity to provide alternative
credentials, but it is not the preferred behaviour for the ES REST
client.

X-Pack may respond with a 401 status if a request is made before the
node/cluster has recovered sufficient state to know how to handle the
provided authentication credentials - for example the security index
need to be recovered before we can authenticate native users.
In these cases the correct behaviour is to retry with the same
credentials (rather than discarding those credentials).
2018-05-17 18:27:18 +10:00
Van0SS 4478f10a2a Rest High Level client: Add List Tasks ()
This change adds a `listTasks` method to the high level java
ClusterClient which allows listing running tasks through the 
task management API.

Related to 
2018-05-16 13:31:37 +02:00
Michael Basnight b94bc70aee
Add Create Repository High Level REST API ()
This commit adds Create Repository, the associated docs and tests
for the high level REST API client. A few small changes to the
PutRepository Request and Response went into the commit as well.
2018-05-15 21:21:11 -05:00
Julie Tibshirani 4f9dd37169
Add support for search templates to the high-level REST client. () 2018-05-15 13:07:58 -07:00
Julie Tibshirani ab0be394e9
Remove assert statements from field caps documentation. ()
Reorganize the test in `SearchDocumentationIT` so the assertions aren't shown in the generated documentation.
2018-05-15 08:37:50 -07:00
Nik Everett 69481b4059
LLRest: Add equals and hashcode tests for Request ()
Adds tests for the `Request` object's equals and hashcode to remove a
`TODO` and because we use the `equals` method in other testing.
2018-05-14 18:27:36 -04:00
javanna df852fbdd9 Fix non existing javadocs link in RestClientTests 2018-05-14 20:24:00 +02:00
Jason Tedor 4a4e3d70d5
Default to one shard ()
This commit changes the default out-of-the-box configuration for the
number of shards from five to one. We think this will help address a
common problem of oversharding. For users with time-based indices that
need a different default, this can be managed with index templates. For
users with non-time-based indices that find they need to re-shard with
the split API in place they no longer need to resort only to
reindexing.

Since this has the impact of changing the default number of shards used
in REST tests, we want to ensure that we still have coverage for issues
that could arise from multiple shards. As such, we randomize (rarely)
the default number of shards in REST tests to two. This is managed via a
global index template. However, some tests check the templates that are
in the cluster state during the test. Since this template is randomly
there, we need a way for tests to skip adding the template used to set
the number of shards to two. For this we add the default_shards feature
skip. To avoid having to write our docs in a complicated way because
sometimes they might be behind one shard, and sometimes they might be
behind two shards we apply the default_shards feature skip to all docs
tests. That is, these tests will always run with the default number of
shards (one).
2018-05-14 12:22:35 -04:00
Nik Everett 41148e4bb1
Docs: Update HighLevelRestClient migration docs ()
The High Level REST Client's documentation suggested that users should
use the Low Level REST Client for index management activities. This
change removes that suggestion because the high level REST client
supports those APIs now.

This also changes the examples in the migration docs to that still use
the Low Level REST Client to use the non-deprecated varieats of
`performRequest`.
2018-05-14 11:11:27 -04:00
Nik Everett b8bf480742
Clients: Switch to new performRequest ()
Switch several calls in the client projects from the deprecated
`performRequest` calls to the new version.
2018-05-14 10:37:53 -04:00