OpenSearch/docs/java-rest/low-level/usage.asciidoc

323 lines
14 KiB
Plaintext
Raw Normal View History

[[java-rest-low-usage]]
=== Getting started
[[java-rest-low-usage-maven]]
==== Maven Repository
The low-level Java REST client is hosted on
http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.elasticsearch.client%22[Maven
Central]. The minimum Java version required is `1.7`.
The low-level REST client is subject to the same release cycle as
elasticsearch. Replace the version with the desired client version, first
released with `5.0.0-alpha4`. There is no relation between the client version
and the elasticsearch version that the client can communicate with. The
low-level REST client is compatible with all elasticsearch versions.
[[java-rest-low-usage-maven-maven]]
===== Maven configuration
Here is how you can configure the dependency using maven as a dependency manager.
Add the following to your `pom.xml` file:
["source","xml",subs="attributes"]
--------------------------------------------------
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>rest</artifactId>
<version>{version}</version>
</dependency>
--------------------------------------------------
[[java-rest-low-usage-maven-gradle]]
===== Gradle configuration
Here is how you can configure the dependency using gradle as a dependency manager.
Add the following to your `build.gradle` file:
["source","groovy",subs="attributes"]
--------------------------------------------------
dependencies {
2016-11-23 11:00:41 -05:00
compile 'org.elasticsearch.client:rest:{version}'
}
--------------------------------------------------
[[java-rest-low-usage-dependencies]]
==== Dependencies
The low-level Java REST client internally uses the
http://hc.apache.org/httpcomponents-asyncclient-dev/[Apache Http Async Client]
to send http requests. It depends on the following artifacts, namely the async
http client and its own transitive dependencies:
- org.apache.httpcomponents:httpasyncclient
- org.apache.httpcomponents:httpcore-nio
- org.apache.httpcomponents:httpclient
- org.apache.httpcomponents:httpcore
- commons-codec:commons-codec
- commons-logging:commons-logging
[[java-rest-low-usage-initialization]]
==== Initialization
A `RestClient` instance can be built through the corresponding
`RestClientBuilder` class, created via `RestClient#builder(HttpHost...)`
static method. The only required argument is one or more hosts that the
client will communicate with, provided as instances of
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpHost.html[HttpHost]
as follows:
[source,java]
--------------------------------------------------
RestClient restClient = RestClient.builder(
new HttpHost("localhost", 9200, "http"),
new HttpHost("localhost", 9201, "http")).build();
--------------------------------------------------
The `RestClient` class is thread-safe and ideally has the same lifecycle as
the application that uses it. It is important that it gets closed when no
longer needed so that all the resources used by it get properly released,
as well as the underlying http client instance and its threads:
[source,java]
--------------------------------------------------
restClient.close();
--------------------------------------------------
`RestClientBuilder` also allows to optionally set the following configuration
parameters while building the `RestClient` instance:
`setDefaultHeaders`:: default headers that need to be sent with each request,
to prevent having to specify them with each single request
`setMaxRetryTimeoutMillis`:: the timeout that should be honoured in case
multiple attempts are made for the same request. The default value is 10
seconds, same as the default socket timeout. In case the socket timeout is
customized, the maximum retry timeout should be adjusted accordingly
`setFailureListener`:: a listener that gets notified every time a node
fails, in case actions need to be taken. Used internally when sniffing on
failure is enabled
`setRequestConfigCallback`:: callback that allows to modify the default
request configuration (e.g. request timeouts, authentication, or anything that
the https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/config/RequestConfig.Builder.html[`org.apache.http.client.config.RequestConfig.Builder`]
allows to set)
`setHttpClientConfigCallback`:: callback that allows to modify the http client
configuration (e.g. encrypted communication over ssl, or anything that the
http://hc.apache.org/httpcomponents-asyncclient-dev/httpasyncclient/apidocs/org/apache/http/impl/nio/client/HttpAsyncClientBuilder.html[`org.apache.http.impl.nio.client.HttpAsyncClientBuilder`]
allows to set)
[[java-rest-low-usage-requests]]
==== Performing requests
Once the `RestClient` has been created, requests can be sent by calling one of
the available `performRequest` or `performRequestAsync` method variants.
The `performRequest` methods are synchronous and they return the `Response`
directly, meaning that the client will block and wait for a response to be returned.
The `performRequestAsync` variants, which return `void` and accept an extra
`ResponseListener` as an argument, are executed asynchronously. The provided
listener will be notified upon completion or failure.
[source,java]
--------------------------------------------------
// Synchronous variants
Response performRequest(String method, String endpoint,
Header... headers)
throws IOException;
Response performRequest(String method, String endpoint,
Map<String, String> params, Header... headers)
throws IOException;
Response performRequest(String method, String endpoint,
Map<String, String> params,
HttpEntity entity,
Header... headers)
throws IOException;
Response performRequest(String method, String endpoint,
Map<String, String> params,
HttpEntity entity,
Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries (#21378) * Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries Turns out that AbstractAsyncResponseConsumer from apache async http client is stateful and cannot be reused across multiple requests. The failover mechanism was mistakenly reusing that same instance, which can be provided by users, across retries in case nodes are down or return 5xx errors. The downside is that we have to change the signature of two public methods, as HttpAsyncResponseConsumer cannot be provided directly anymore, rather its factory needs to be provided which is going to be used to create one instance of the consumer per request attempt. Up until now we tested our RestClient against multiple nodes only in a mock environment, where we don't really send http requests. In that scenario we can verify that retries etc. work properly but the interaction with the http client library in a real scenario is different and can catch other problems. With this commit we also add an integration test that sends requests to multiple hosts, and some of them may also get stopped meanwhile. The specific test for pathPrefix was also removed as pathPrefix is now randomly applied by default, hence implicitly tested. Moved also a small test method that checked the validity of the path argument to the unit test RestClientSingleHostTests. Also increase default buffer limit to 100MB and make it required in default consumer The default buffer limit used to be 10MB but that proved not to be high enough for scroll requests (see reindex from remote). With this commit we increase the limit to 100MB and make it a bit more visibile in the consumer factory.
2016-11-08 10:42:42 -05:00
HttpAsyncResponseConsumerFactory responseConsumerFactory,
Header... headers)
throws IOException;
// Asynchronous variants
void performRequestAsync(String method, String endpoint,
ResponseListener responseListener,
Header... headers);
void performRequestAsync(String method, String endpoint,
Map<String, String> params,
ResponseListener responseListener,
Header... headers);
void performRequestAsync(String method, String endpoint,
Map<String, String> params,
HttpEntity entity,
ResponseListener responseListener,
Header... headers);
void performRequestAsync(String method, String endpoint,
Map<String, String> params,
HttpEntity entity,
Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries (#21378) * Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries Turns out that AbstractAsyncResponseConsumer from apache async http client is stateful and cannot be reused across multiple requests. The failover mechanism was mistakenly reusing that same instance, which can be provided by users, across retries in case nodes are down or return 5xx errors. The downside is that we have to change the signature of two public methods, as HttpAsyncResponseConsumer cannot be provided directly anymore, rather its factory needs to be provided which is going to be used to create one instance of the consumer per request attempt. Up until now we tested our RestClient against multiple nodes only in a mock environment, where we don't really send http requests. In that scenario we can verify that retries etc. work properly but the interaction with the http client library in a real scenario is different and can catch other problems. With this commit we also add an integration test that sends requests to multiple hosts, and some of them may also get stopped meanwhile. The specific test for pathPrefix was also removed as pathPrefix is now randomly applied by default, hence implicitly tested. Moved also a small test method that checked the validity of the path argument to the unit test RestClientSingleHostTests. Also increase default buffer limit to 100MB and make it required in default consumer The default buffer limit used to be 10MB but that proved not to be high enough for scroll requests (see reindex from remote). With this commit we increase the limit to 100MB and make it a bit more visibile in the consumer factory.
2016-11-08 10:42:42 -05:00
HttpAsyncResponseConsumerFactory responseConsumerFactory,
ResponseListener responseListener,
Header... headers);
--------------------------------------------------
[[java-rest-low-usage-requests-arguments]]
===== Request Arguments
The following are the arguments accepted by the different methods:
`method`:: the http method or verb
`endpoint`:: the request path, which identifies the Elasticsearch API to
call (e.g. `/_cluster/health`)
`params`:: the optional parameters to be sent as querystring parameters
`entity`:: the optional request body enclosed in an
`org.apache.http.HttpEntity` object
Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries (#21378) * Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries Turns out that AbstractAsyncResponseConsumer from apache async http client is stateful and cannot be reused across multiple requests. The failover mechanism was mistakenly reusing that same instance, which can be provided by users, across retries in case nodes are down or return 5xx errors. The downside is that we have to change the signature of two public methods, as HttpAsyncResponseConsumer cannot be provided directly anymore, rather its factory needs to be provided which is going to be used to create one instance of the consumer per request attempt. Up until now we tested our RestClient against multiple nodes only in a mock environment, where we don't really send http requests. In that scenario we can verify that retries etc. work properly but the interaction with the http client library in a real scenario is different and can catch other problems. With this commit we also add an integration test that sends requests to multiple hosts, and some of them may also get stopped meanwhile. The specific test for pathPrefix was also removed as pathPrefix is now randomly applied by default, hence implicitly tested. Moved also a small test method that checked the validity of the path argument to the unit test RestClientSingleHostTests. Also increase default buffer limit to 100MB and make it required in default consumer The default buffer limit used to be 10MB but that proved not to be high enough for scroll requests (see reindex from remote). With this commit we increase the limit to 100MB and make it a bit more visibile in the consumer factory.
2016-11-08 10:42:42 -05:00
`responseConsumerFactory`:: the optional factory that is used to create an
http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/apidocs/org/apache/http/nio/protocol/HttpAsyncResponseConsumer.html[`org.apache.http.nio.protocol.HttpAsyncResponseConsumer`]
Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries (#21378) * Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries Turns out that AbstractAsyncResponseConsumer from apache async http client is stateful and cannot be reused across multiple requests. The failover mechanism was mistakenly reusing that same instance, which can be provided by users, across retries in case nodes are down or return 5xx errors. The downside is that we have to change the signature of two public methods, as HttpAsyncResponseConsumer cannot be provided directly anymore, rather its factory needs to be provided which is going to be used to create one instance of the consumer per request attempt. Up until now we tested our RestClient against multiple nodes only in a mock environment, where we don't really send http requests. In that scenario we can verify that retries etc. work properly but the interaction with the http client library in a real scenario is different and can catch other problems. With this commit we also add an integration test that sends requests to multiple hosts, and some of them may also get stopped meanwhile. The specific test for pathPrefix was also removed as pathPrefix is now randomly applied by default, hence implicitly tested. Moved also a small test method that checked the validity of the path argument to the unit test RestClientSingleHostTests. Also increase default buffer limit to 100MB and make it required in default consumer The default buffer limit used to be 10MB but that proved not to be high enough for scroll requests (see reindex from remote). With this commit we increase the limit to 100MB and make it a bit more visibile in the consumer factory.
2016-11-08 10:42:42 -05:00
callback instance per request attempt. Controls how the response body gets
streamed from a non-blocking HTTP connection on the client side. When not
provided, the default implementation is used which buffers the whole response
body in heap memory, up to 100 MB
`responseListener`:: the listener to be notified upon asynchronous
request success or failure
`headers`:: optional request headers
[[java-rest-low-usage-responses]]
==== Reading responses
The `Response` object, either returned by the synchronous `performRequest` methods or
received as an argument in `ResponseListener#onSuccess(Response)`, wraps the
response object returned by the http client and exposes the following information:
`getRequestLine`:: information about the performed request
`getHost`:: the host that returned the response
`getStatusLine`:: the response status line
`getHeaders`:: the response headers, which can also be retrieved by name
though `getHeader(String)`
`getEntity`:: the response body enclosed in an
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpEntity.html[`org.apache.http.HttpEntity`]
object
When performing a request, an exception is thrown (or received as an argument
in `ResponseListener#onFailure(Exception)` in the following scenarios:
`IOException`:: communication problem (e.g. SocketTimeoutException etc.)
`ResponseException`:: a response was returned, but its status code indicated
an error (not `2xx`). A `ResponseException` originates from a valid
http response, hence it exposes its corresponding `Response` object which gives
access to the returned response.
NOTE: A `ResponseException` is **not** thrown for `HEAD` requests that return
a `404` status code because it is an expected `HEAD` response that simply
denotes that the resource is not found. All other HTTP methods (e.g., `GET`)
throw a `ResponseException` for `404` responses unless the `ignore` parameter
contains `404`. `ignore` is a special client parameter that doesn't get sent
to Elasticsearch and contains a comma separated list of error status codes.
It allows to control whether some error status code should be treated as an
expected response rather than as an exception. This is useful for instance
with the get api as it can return `404` when the document is missing, in which
case the response body will not contain an error but rather the usual get api
response, just without the document as it was not found.
[[java-rest-low-usage-example]]
==== Example requests
Here are a couple of examples:
[source,java]
--------------------------------------------------
Response response = restClient.performRequest("GET", "/",
Collections.singletonMap("pretty", "true"));
System.out.println(EntityUtils.toString(response.getEntity()));
//index a document
HttpEntity entity = new NStringEntity(
"{\n" +
" \"user\" : \"kimchy\",\n" +
" \"post_date\" : \"2009-11-15T14:12:12\",\n" +
" \"message\" : \"trying out Elasticsearch\"\n" +
"}", ContentType.APPLICATION_JSON);
Response indexResponse = restClient.performRequest(
"PUT",
"/twitter/tweet/1",
Collections.<String, String>emptyMap(),
entity);
--------------------------------------------------
IMPORTANT: The `ContentType` that you specify for the `HttpEntity` is important
because it will be used to set the `Content-Type` header so that Elasticsearch
can properly parse the content.
Note that the low-level client doesn't expose any helper for json marshalling
and un-marshalling. Users are free to use the library that they prefer for that
purpose.
The underlying Apache Async Http Client ships with different
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpEntity.html[`org.apache.http.HttpEntity`]
implementations that allow to provide the request body in different formats
(stream, byte array, string etc.). As for reading the response body, the
`HttpEntity#getContent` method comes handy which returns an `InputStream`
reading from the previously buffered response body. As an alternative, it is
possible to provide a custom
http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/apidocs/org/apache/http/nio/protocol/HttpAsyncResponseConsumer.html[`org.apache.http.nio.protocol.HttpAsyncResponseConsumer`]
that controls how bytes are read and buffered.
The following is a basic example of how async requests can be sent:
[source,java]
--------------------------------------------------
int numRequests = 10;
final CountDownLatch latch = new CountDownLatch(numRequests);
for (int i = 0; i < numRequests; i++) {
restClient.performRequestAsync(
"PUT",
"/twitter/tweet/" + i,
Collections.<String, String>emptyMap(),
//assume that the documents are stored in an entities array
entities[i],
new ResponseListener() {
@Override
public void onSuccess(Response response) {
System.out.println(response);
latch.countDown();
}
@Override
public void onFailure(Exception exception) {
latch.countDown();
}
}
);
}
//wait for all requests to be completed
latch.await();
--------------------------------------------------
[[java-rest-low-usage-logging]]
==== Logging
The Java REST client uses the same logging library that the Apache Async Http
Client uses: https://commons.apache.org/proper/commons-logging/[Apache Commons Logging],
which comes with support for a number of popular logging implementations. The
java packages to enable logging for are `org.elasticsearch.client` for the
client itself and `org.elasticsearch.client.sniffer` for the sniffer.
The request tracer logging can also be enabled to log every request and
corresponding response in curl format. That comes handy when debugging, for
instance in case a request needs to be manually executed to check whether it
still yields the same response as it did. Enable trace logging for the `tracer`
package to have such log lines printed out. Do note that this type of logging is
expensive and should not be enabled at all times in production environments,
but rather temporarily used only when needed.