Stated that node client is unsupported, removed node client config info, and edited the transport client config section. Closes elastic/elasticsearch#906

Original commit: elastic/x-pack-elasticsearch@9bbd6726a8
This commit is contained in:
debadair 2015-07-17 14:22:29 -07:00
parent 8e5b321cc7
commit 91786f4662
1 changed files with 105 additions and 341 deletions

View File

@ -1,250 +1,24 @@
=== Using Elasticsearch Java Clients with Shield
Elasticsearch supports two types of Java clients: _Node Client_ and _Transport Client_.
Shield supports the Java http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html[transport client] for Elasticsearch. The transport client uses the same transport protocol that the cluster nodes use for inter-node communication. It is very efficient as it does not have to marshall and unmarshall JSON requests like a typical REST client.
The _Node Client_ is a cluster node that joins the cluster and receives all the cluster events, in the same manner as
any other cluster node. Node clients cannot be allocated shards, and therefore cannot hold data. Node clients are not
eligible for election as a master node in the cluster. For more information about node clients, see the
http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/node-client.html[following section].
Unlike the _Node Client_, the _Transport Client_ is not a node in the cluster. Yet it uses the same transport protocol
the cluster nodes use for inter-node communication and is therefore considered to be very efficient as it bypasses the
process of un/marshalling of request from/to JSON which you typically have in REST based clients (read more about
http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html[_Transport Client_]).
Shield supports both clients. This section provides configuration instructions for these clients.
[float]
==== Node Client
WARNING: While _Node Clients_ may work with Shield, since these are actual nodes in the cluster, they require access
to a breadth of cluster management internal APIs. Additionally, just like all other nodes in the cluster,
_Node Clients_ require the License plugin to be installed and access to Shield configuration files that contain
sensitive data. For this reason, _Node Clients_ should be considered as unsafe clients. If you choose to use
these clients, make sure you treat them in the same way you treat any other node in your cluster. Your
application should sit next to the cluster within the same security zone.
There are several steps for setting up this client:
. Set the appropriate dependencies for you project
. Duplicate <<ref-shield-files, configuration files>> for authentication
. Configure the authentication token
. (Optional) If SSL/TLS is enabled, set up the keystore, then create and import the appropriate certificates.
[float]
===== Java project dependencies
If you plan on using the Node Client, you first need to make sure the Shield jar files (`elasticsearch-shield-1.3.0.jar`,
`automaton-1.11-8.jar`, `unboundid-ldapsdk-2.3.8.jar`) and the License jar file (`elasticsearch-license-1.0.0.jar`) are
in the classpath. You can either download the distributions, extract the jar files manually and include them in your
classpath, or you can pull them out of the Elasticsearch Maven repository.
[float]
===== Maven Example
The following snippet shows the configuration you will need to include in your project's `pom.xml` file:
[source,xml]
--------------------------------------------------------------
<project ...>
<repositories>
<!-- add the elasticsearch repo -->
<repository>
<id>elasticsearch-releases</id>
<url>http://maven.elasticsearch.org/releases</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
...
</repositories>
...
<dependencies>
<!-- add the Shield jar as a dependency -->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-shield</artifactId>
<version>1.3.0</version>
</dependency>
<!-- add the License jar as a dependency -->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-license-plugin</artifactId>
<version>1.0.0</version>
<scope>runtime</scope>
</dependency>
...
</dependencies>
...
</project>
--------------------------------------------------------------
[float]
===== Gradle Example
If you are using Gradle, then you will need to add the dependencies to your `build.gradle` file:
[source,groovy]
--------------------------------------------------------------
repositories {
/* ... Any other repositories ... */
// Add the Elasticsearch Maven Repository
maven {
url "http://maven.elasticsearch.org/releases"
}
}
dependencies {
// Provide the Shield jar on the classpath for compilation and at runtime
// Note: Many projects can use the Shield jar as a runtime dependency
compile "org.elasticsearch:elasticsearch-shield:1.3.0"
/* ... */
// Provide the License jar on the classpath at runtime (not needed for compilation)
runtime "org.elasticsearch:elasticsearch-license-plugin:1.0.0"
}
--------------------------------------------------------------
It is also possible to manually download the http://maven.elasticsearch.org/releases/org/elasticsearch/elasticsearch-shield/1.3.0/elasticsearch-shield-1.3.0.jar[Shield jar]
and the http://maven.elasticsearch.org/releases/org/elasticsearch/elasticsearch-license-plugin/1.0.0/elasticsearch-license-plugin-1.0.0.jar[License jar]
files from our Maven repository.
[float]
===== Duplicate Shield Configuration Files
The _Node Client_ will authenticate requests before sending the requests to the cluster. To do this, copy the `users`,
`users_roles`, `roles.yml`, and `system_key` files from the <<ref-shield-files,Shield configuration files>> to a place
accessible to the node client. These files should be stored on the filesystem in a folder with restricted access as they
contain sesnitive data. This can be configured with the following settings:
* `shield.authc.realms.esusers.files.users`
* `shield.authc.realms.esusers.files.users_roles`
* `shield.authz.store.files.roles`
* `shield.system_key.file`
[source, java]
------------------------------------------------------------------------------------------------------
import static org.elasticsearch.node.NodeBuilder.*;
...
Node node = nodeBuilder().client(true).settings(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("discovery.zen.ping.multicast.enabled", false)
.putArray("discovery.zen.ping.unicast.hosts", "localhost:9300", "localhost:9301")
.put("shield.authc.realms.esusers.type", "esusers")
.put("shield.authc.realms.esusers.files.users", "/Users/es/config/shield/users")
.put("shield.authc.realms.esusers.files.users_roles", "/Users/es/config/shield/users_roles")
.put("shield.authz.store.files.roles", "/Users/es/config/shield/roles.yml")
.put("shield.system_key.file", "/Users/es/config/shield/system_key"))
...
.node();
------------------------------------------------------------------------------------------------------
Additionally, if you are using LDAP or Active Directory authentication then you will need to specify that configuration
in the settings when configuring the node or provide a `elasticsearch.yml` in the classpath with the appropriate settings.
[float]
===== Configuring Authentication Token
The authentication token can be configured in two ways - globally or per-request. When setting it up globally, the
values of the username and password are configured in the client's settings:
[source,java]
------------------------------------------------------------------------------------------------------
import static org.elasticsearch.node.NodeBuilder.*;
import static org.elasticsearch.shield.authc.support.UsernamePasswordToken.basicAuthHeaderValue;
...
Node node = nodeBuilder().client(true).settings(ImmutableSettings.builder()
...
.put("shield.user", "test_user:changeme"))
...
.node();
Client client = node.client();
------------------------------------------------------------------------------------------------------
Once the client is created as above, the `shield.user` setting is translated to a request header in the standard HTTP
basic authentication form `Authentication base64("test_user:changeme")` which will be sent with every request executed.
To skip the global configuration of the token, manually set the authentication token header on every request:
[source,java]
------------------------------------------------------------------------------------------------------
import org.elasticsearch.shield.authc.support.SecuredString;
import static org.elasticsearch.node.NodeBuilder.*;
import static org.elasticsearch.shield.authc.support.UsernamePasswordToken.basicAuthHeaderValue;
...
String token = basicAuthHeaderValue("test_user", new SecuredString("changeme".toCharArray()));
Node node = nodeBuilder().client(true).settings(ImmutableSettings.builder()
...
.node();
Client client = node.client();
client.prepareSearch().putHeader("Authorization", token).get();
------------------------------------------------------------------------------------------------------
The example above executes a search request and manually adds the authentication token as a header on it.
[float]
===== Setting up SSL
Authenticating to the cluster requires proof that a node client is trusted as part of the cluster. This is done through
standard PKI and SSL. A client node creates a private key and an associated certificate. The cluster Certificate
Authority signs the certificate. A Client node authenticates during the SSL connection setup by presenting the signed
certificate, and proving ownership of the private key. All of these setup steps are described in
<<private-key, Securing Nodes>>.
In addition, the node client acts like a node, authenticating locally any request made. Copies of the files `users`,
`users_roles`, `roles.yml` , and `system_key` need to be made available to the node client.
After following the steps in <<private-key, Securing Nodes>>, configuration for a node client with Shield might look
like this:
[source, java]
------------------------------------------------------------------------------------------------------
import static org.elasticsearch.node.NodeBuilder.*;
...
Node node = nodeBuilder().client(true).settings(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("discovery.zen.ping.multicast.enabled", false)
.putArray("discovery.zen.ping.unicast.hosts", "localhost:9300", "localhost:9301")
.put("shield.ssl.keystore.path", "/Users/es/node_client/node_client.jks")
.put("shield.ssl.keystore.password", "password")
.put("shield.transport.ssl", "true")
.put("shield.authc.realms.esusers.type", "esusers")
.put("shield.authc.realms.esusers.files.users", "/Users/es/config/shield/users")
.put("shield.authc.realms.esusers.files.users_roles", "/Users/es/config/shield/users_roles")
.put("shield.authz.store.files.roles", "/Users/es/config/shield/roles.yml")
.put("shield.system_key.file", "/Users/es/config/shield/system_key"))
...
.node();
------------------------------------------------------------------------------------------------------
NOTE: Using the Java Node Client with Shield is not recommended or supported.
[float]
[[transport-client]]
==== Transport Client
==== Configuring the Transport Client to work with Shield
If you plan on using the Transport Client over SSL/TLS you first need to make sure the Shield jar file
(`elasticsearch-shield-2.0.jar`) is in the classpath. You can either download the Shield distribution, extract the jar
files manually and include them in your classpath, or you can pull them out of the Elasticsearch Maven repository.
To use the transport client with Shield, you need to:
NOTE: Unlike the _Node Client_, the _Transport Client_ is not acting as a node in the cluster, and therefore
**does not** require the License plugin to be installed.
[float]
===== Maven Example
The following snippet shows the configuration you will need to include in your project's `pom.xml` file:
[[java-transport-client-role]]
. Configure a user with the privileges required to start the transport client. A default
`transport_client` role is defined in `roles.yml` that grants the `cluster:monitor/nodes/info` cluster permission. The transport client uses the node info API to fetch information about the nodes in the cluster. If the client is configured to use sniffing, you need to add the
`cluster:monitor/state` cluster permission to the `transport_client` role.
. Add the Shield JAR file (`elasticsearch-shield-2.0.0.jar`) to your CLASSPATH. You can download the Shield distribution and extract the JAR file manually or you can get it from the http://maven.elasticsearch.org/releases/org/elasticsearch/elasticsearch-shield/2.0.0/elasticsearch-shield-2.0.0.jar[Elasticsearch Maven repository].
+
If you are using Maven, you need to add the Shield JAR file as a dependency in your project's `pom.xml` file:
+
[source,xml]
--------------------------------------------------------------
<project ...>
@ -270,7 +44,7 @@ The following snippet shows the configuration you will need to include in your p
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-shield</artifactId>
<version>1.3.0</version>
<version>2.0.0</version>
</dependency>
...
</dependencies>
@ -278,12 +52,9 @@ The following snippet shows the configuration you will need to include in your p
</project>
--------------------------------------------------------------
[float]
===== Gradle Example
If you are using Gradle, then you will need to add the dependencies to your `build.gradle` file:
+
If you are using Gradle, you need to add the Shield JAR file as a dependency in your `build.gradle` file:
+
[source,groovy]
--------------------------------------------------------------
repositories {
@ -298,138 +69,129 @@ repositories {
dependencies {
// Provide the Shield jar on the classpath for compilation and at runtime
// Note: Many projects can use the Shield jar as a runtime dependency
compile "org.elasticsearch:elasticsearch-shield:1.3.0"
compile "org.elasticsearch:elasticsearch-shield:2.0.0"
/* ... */
}
--------------------------------------------------------------
It is also possible to manually download the http://maven.elasticsearch.org/releases/org/elasticsearch/elasticsearch-shield/1.3.0/elasticsearch-shield-1.3.0.jar[Shield jar]
file from our Maven repository.
TIP: Even if you are not planning on using the client over SSL/TLS, it is still worth having the Shield jar file in
the classpath as it provides various helpful utilities, such as the `UsernamePasswordToken` class for generating
basic-auth tokens and the `ShieldClient` that <<shield-client,exposes an API>> to clear realm caches.
[[java-transport-client-role]]
Before setting up the client itself, you need to make sure you have a user with sufficient privileges to start
the transport client. The transport client uses Elasticsearch's node info API to fetch information about the
nodes in the cluster. For this reason, the authenticated user of the transport client must have the
`cluster:monitor/nodes/info` cluster permission. Furthermore, if the client is configured to use sniffing, the
`cluster:monitor/state` cluster permission is required.
TIP: `roles.yml` ships with a predefined `transport_client` role. By default it is configured to only grant the
`cluster:monitor/nodes/info` cluster permission. You can use this role and assign it to any user
that will be attached to a transport client.
Setting up the transport client is similar to the Node client except authentication files do not need to be configured.
Without SSL, it is as easy as setting up the authentication token on the request, similarly to how they're set up with
the _Node Client_:
. Set up the transport client. At a minimum, you must configure `shield.user` to include the name and password of your transport client user in your requests. The following snippet configures the user credentials globally--every request submitted with this client includes the `transport_client_user` credentials in its headers.
+
[source,java]
------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------
import org.elasticsearch.client.transport.TransportClient;
...
TransportClient client = new TransportClient(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("shield.user", "test_user:changeme")
.put("shield.user", "transport_client_user:changeme")
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
------------------------------------------------------------------------------------------------------
WARNING: Configuring a Transport Client without SSL will send passwords in plaintext.
When using SSL for transport client communication, a few more settings are required. By default, Shield requires client
authentication for secured transport communication. This means that every client would need to have a certificate signed
by a trusted CA. The client authentication can be disabled through the use of a <<separating-node-client-traffic, client
specific transport profile>>.
Configuration required for SSL when using client authentication:
-------------------------------------------------------------------------------------------------
+
WARNING: If you configure a transport client without SSL, passwords are sent in plaintext.
+
You can also add an `Authorization` header to each request. If you've configured global authorization credentials, the `Authorization` header overrides the global authentication credentials. This is useful when an application has multiple users who access Elasticsearch using the same client. You can set the global token to a user that only has the `transport_client` role, and add the `transport_client` role to the individual users.
+
For example, the following snippet adds the `Authorization` header to a search request:
+
[source,java]
------------------------------------------------------------------------------------------------------
import org.elasticsearch.client.transport.TransportClient;
...
TransportClient client = new TransportClient(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("shield.user", "test_user:changeme")
.put("shield.ssl.keystore.path", "/path/to/client.jks")
.put("shield.ssl.keystore.password", "password")
.put("shield.transport.ssl", "true"))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
------------------------------------------------------------------------------------------------------
NOTE: The `client.jks` keystore needs to contain the client's signed CA certificate and the CA certificate.
Configuration required for SSL without client authentication:
[source,java]
------------------------------------------------------------------------------------------------------
import org.elasticsearch.client.transport.TransportClient;
...
TransportClient client = new TransportClient(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("shield.user", "test_user:changeme")
.put("shield.ssl.truststore.path", "/path/to/truststore.jks")
.put("shield.ssl.truststore.password", "password")
.put("shield.transport.ssl", "true"))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
------------------------------------------------------------------------------------------------------
NOTE: The `truststore.jks` truststore needs to contain the certificate of the CA that has signed the Elasticsearch nodes'
certificates. If you are using a public CA that is already trusted by the Java runtime, then you can omit
`shield.ssl.truststore.path` and `shield.ssl.truststore.password`.
In the above code snippets, we set up a _Transport Client_ and configured the authentication token globally. Meaning,
that every request executed with this client will include this token in its headers.
The global configuration of the token *must be* set to some user with the privileges in the default `transport_client`
role, as described earlier. The global authentication token may also be overridden by adding a `Authorization` header on
each request. This is useful when an application uses multiple users to access Elasticsearch via the same client. When
operating in this mode, it is best to set the global token to a user that only has the `transport_client` role. The
following example directly sets the authentication token on the request when executing a search.
[source,java]
------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
import org.elasticsearch.shield.authc.support.SecuredString;
import org.elasticsearch.client.transport.TransportClient;
import static org.elasticsearch.shield.authc.support.UsernamePasswordToken.basicAuthHeaderValue;
...
String token = basicAuthHeaderValue("test_user", new SecuredString("changeme".toCharArray()));
TransportClient client = new TransportClient(ImmutableSettings.builder()
.put("shield.user", "transport_client_user:changeme")
...
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
String token = basicAuthHeaderValue("test_user", new SecuredString("changeme".toCharArray()));
client.prepareSearch().putHeader("Authorization", token).get();
------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
. Enable SSL to authenticate clients and encrypt communications. To enable SSL, you need to:
.. Configure the client's keystore path and password. Client authentication requires every
client to have a certification signed by a trusted CA.
+
NOTE: Client authentication is enabled by default. For information about disabling client authentication, see <<disabling-client-auth, Disabling Client Authentication>>.
+
[source,java]
--------------------------------------------------------------------------------------------------
import org.elasticsearch.client.transport.TransportClient;
...
TransportClient client = new TransportClient(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("shield.user", "transport_client_user:changeme")
.put("shield.ssl.keystore.path", "/path/to/client.jks") (1)
.put("shield.ssl.keystore.password", "password")
...
--------------------------------------------------------------------------------------------------
+
(1) The `client.jks` keystore must contain the client's signed CA certificate and the CA certificate.
+
.. Enable the SSL transport by setting `shield.transport.ssl` to `true` in the client configuration.
+
[source,java]
--------------------------------------------------------------------------------------------------
import org.elasticsearch.client.transport.TransportClient;
...
TransportClient client = new TransportClient(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("shield.user", "transport_client_user:changeme")
.put("shield.ssl.keystore.path", "/path/to/client.jks") (1)
.put("shield.ssl.keystore.password", "password")
.put("shield.transport.ssl", "true"))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
--------------------------------------------------------------------------------------------------
[float]
===== Anonymous Access
[[disabling-client-auth]]
===== Disabling Client Authentication
added[1.1.0]
If you want to disable client authentication, you can use a client-specific transport protocol. For more information, <<separating-node-client-traffic, Separating Node to Node and Client Traffic>>.
If Shield enables <<anonymous-access,anonymous access>>, the `shield.user` setting may be dropped and all requests will
be executed under the anonymous user (with the exception of the requests on which the `Authorization` header is explicitly
set, as shown above). For this to work, please make sure the anonymous user is configured with sufficient roles that have
the same privileges as described <<java-transport-client-role,above>>.
If you are not using client authentication and sign the Elasticsearch node certificates with your own CA, you need to set the truststore path and password in the client configuration:
[source,java]
------------------------------------------------------------------------------------------------------
import org.elasticsearch.client.transport.TransportClient;
...
TransportClient client = new TransportClient(ImmutableSettings.builder()
.put("cluster.name", "myClusterName")
.put("shield.user", "test_user:changeme")
.put("shield.ssl.truststore.path", "/path/to/truststore.jks") (1)
.put("shield.ssl.truststore.password", "password")
.put("shield.transport.ssl", "true"))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
------------------------------------------------------------------------------------------------------
(1) The `truststore.jks` truststore must contain the certificate of the CA that signed the Elasticsearch node certificates.
NOTE: If you are using a public CA that is already trusted by the Java runtime, you to not need to set the `shield.ssl.truststore.path` and `shield.ssl.truststore.password`.
[float]
[[connecting-anonymously]]
===== Connecting Anonymously added[1.1.0]
To enable the transport client to connect anonymously, you must assign the anonymous user the privileges defined in the <<java-transport-client-role,transport_client>> role. Anonymous access must also be enabled, of course. For more information, see <<anonymous-access,Enabling Anonymous Access>>.
[float]
[[shield-client]]
==== Shield Client
Shield exposes its own API to the user which is accessible by the `ShieldClient` class. The purpose of this API
is to manage all Shield related aspects. While at the moment it only exposes an operation for clearing up the
realm caches, the plan is to extend this API in the future.
Shield exposes its own API through the `ShieldClient` class. At the moment, this API only exposes one operation, for clearing the realm caches. `ShieldClient` is a wrapper around the existing clients (any client class implementing `org.elasticsearch.client.Client`).
`ShieldClient` is a wrapper around the existing clients (any client class implementing `org.elasticsearch.client.Client`.
The following example shows how one can clear up Shield's realm caches using the `ShieldClient`:
The following example shows how you can clear Shield's realm caches using `ShieldClient`:
[source,java]
------------------------------------------------------------------------------------------------------
@ -440,9 +202,11 @@ Client client = ... // create the client (either transport or node)
ShieldClient shieldClient = new ShieldClient(client);
ClearRealmCacheResponse response = shieldClient.authc().prepareClearRealmCache()
.realms("ldap1", "ad1")
.realms("ldap1", "ad1") (1)
.usernames("rdeniro")
.get();
------------------------------------------------------------------------------------------------------
In the example above, we clear the caches of two realms - `ldap1` and `ad1` - for the `rdeniro` user.
(1) Clears the `ldap1` and `ad1` realm caches for the `rdeniro` user.