honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[DOCS] Migrated security topics from x-pack repo to x-pack-elasticsearch. 

Original commit: elastic/x-pack-elasticsearch@e54aa1fd0a
This commit is contained in:
debadair 2017-04-06 18:29:29 -07:00
parent e804d78be5
commit ee5ab5b1d2
40 changed files with 6386 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

432
docs/en/security/auditing.asciidoc Normal file
View File

@ -0,0 +1,432 @@
[[auditing]]
== Auditing Security Events
You can enable auditing to keep track of security-related events such as
authentication failures and refused connections. Logging these events enables you
to monitor your cluster for suspicious activity and provides evidence in the
event of an attack.
[IMPORTANT]
============================================================================
Audit logs are **disabled** by default. To enable this functionality, you
must set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`.
============================================================================
{Security} provides two ways to persist audit logs:
* The <<audit-log-output, `logfile`>> output, which persists events to
a dedicated `<clustername>_access.log` file on the host's file system.
* The <<audit-index, `index`>> output, which persists events to an Elasticsearch index.
The audit index can reside on the same cluster, or a separate cluster.
By default, only the `logfile` output is used when enabling auditing.
To facilitate browsing and analyzing the events, you can also enable
indexing by setting `xpack.security.audit.outputs` in `elasticsearch.yml`:
[source,yaml]
----------------------------
xpack.security.audit.outputs: [ index, logfile ]
----------------------------
The `index` output type should be used in conjunction with the `logfile`
output type Because it is possible for the `index` output type to lose
messages if the target index is unavailable, the `access.log` should be
used as the official record of events.
NOTE: Audit events are batched for indexing so there is a lag before
events appear in the index. You can control how frequently batches of
events are pushed to the index by setting
`xpack.security.audit.index.flush_interval` in `elasticsearch.yml`.
[float]
[[audit-event-types]]
=== Audit Event Types
Each request may generate multiple audit events.
The following is a list of the events that can be generated:
|======
| `anonymous_access_denied` | | | Logged when a request is denied due to a missing
authentication token.
| `authentication_success` | | | Logged when a user successfully authenticates.
| `authentication_failed` | | | Logged when the authentication token cannot be
matched to a known user.
| `realm_authentication_failed` | | | Logged for every realm that fails to present a valid
authentication token. `<realm>` represents the
realm type.
| `access_denied` | | | Logged when an authenticated user attempts to execute
an action they do not have the necessary
<<security-reference, privilege>> to perform.
| `access_granted` | | | Logged when an authenticated user attempts to execute
an action they have the necessary privilege to perform.
When the `system_access_granted` event is included, all system
(internal) actions are also logged. The default setting does
not log system actions to avoid cluttering the logs.
| `run_as_granted` | | | Logged when an authenticated user attempts to <<run-as-privilege, run as>>
another user that they have the necessary privileges to do.
| `run_as_denied` | | | Logged when an authenticated user attempts to <<run-as-privilege, run as>>
another user action they do not have the necessary
<<security-reference, privilege>> to do so.
| `tampered_request` | | | Logged when {security} detects that the request has
been tampered with. Typically relates to `search/scroll`
requests when the scroll ID is believed to have been
tampered with.
| `connection_granted` | | | Logged when an incoming TCP connection passes the
<<ip-filtering, IP Filter>> for a specific
profile.
| `connection_denied` | | | Logged when an incoming TCP connection does not pass the
<<ip-filtering, IP Filter>> for a specific
profile.
|======
[float]
[[audit-event-attributes]]
=== Audit Event Attributes
The following table shows the common attributes that can be associated with every event.
.Common Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `timestamp` | When the event occurred.
| `node_name` | The name of the node.
| `node_host_name` | The hostname of the node.
| `node_host_address` | The IP address of the node.
| `layer` | The layer from which this event originated: `rest`, `transport` or `ip_filter`
| `event_type` | The type of event that occurred: `anonymous_access_denied`,
`authentication_failed`, `access_denied`, `access_granted`,
`connection_granted`, `connection_denied`, `tampered_request`,
`run_as_granted`, `run_as_denied`.
|======
The following tables show the attributes that can be associated with each type of event.
The log level determines which attributes are included in a log entry.
.REST anonymous_access_denied Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_address` | The IP address from which the request originated.
| `uri` | The REST endpoint URI.
| `request_body` | The body of the request, if enabled.
|======
.REST authentication_success Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `user` | The authenticated user.
| `realm` | The realm that authenticated the user.
| `uri` | The REST endpoint URI.
| `params` | The REST URI query parameters.
| `request_body` | The body of the request, if enabled.
|======
.REST authentication_failed Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_address` | The IP address from which the request originated.
| `principal` | The principal (username) that failed authentication.
| `uri` | The REST endpoint URI.
| `request_body` | The body of the request, if enabled.
|======
.REST realm_authentication_failed Attributes
[cols="2,7",options="header"]
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_address` | The IP address from which the request originated.
| `principal` | The principal (username) that failed authentication.
| `uri` | The REST endpoint URI.
| `request_body` | The body of the request, if enabled.
| `realm` | The realm that failed to authenticate the user.
NOTE: A separate entry is logged for each
consulted realm.
|======
.Transport anonymous_access_denied Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_type` | Where the request originated: `rest` (request
originated from a REST API request), `transport`
(request was received on the transport channel),
`local_node` (the local node issued the request).
| `origin_address` | The IP address from which the request originated.
| `action` | The name of the action that was executed.
| `request` | The type of request that was executed.
| `indices` | A comma-separated list of indices this request
pertains to (when applicable).
|======
.Transport authentication_success Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_type` | Where the request originated: `rest` (request
originated from a REST API request), `transport`
(request was received on the transport channel),
`local_node` (the local node issued the request).
| `origin_address` | The IP address from which the request originated.
| `user` | The authenticated user.
| `realm` | The realm that authenticated the user.
| `action` | The name of the action that was executed.
| `request` | The type of request that was executed.
|======
.Transport authentication_failed Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_type` | Where the request originated: `rest` (request
originated from a REST API request), `transport`
(request was received on the transport channel),
`local_node` (the local node issued the request).
| `origin_address` | The IP address from which the request originated.
| `principal` | The principal (username) that failed authentication.
| `action` | The name of the action that was executed.
| `request` | The type of request that was executed.
| `indices` | A comma-separated list of indices this request
pertains to (when applicable).
|======
.Transport realm_authentication_failed Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_type` | Where the request originated: `rest` (request
originated from a REST API request), `transport`
(request was received on the transport channel),
`local_node` (the local node issued the request).
| `origin_address` | The IP address from which the request originated.
| `principal` | The principal (username) that failed authentication.
| `action` | The name of the action that was executed.
| `request` | The type of request that was executed.
| `indices` | A comma-separated list of indices this request
pertains to (when applicable).
| `realm` | The realm that failed to authenticate the user.
NOTE: A separate entry is logged for each
consulted realm.
|======
.Transport access_granted Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_type` | Where the request originated: `rest` (request
originated from a REST API request), `transport`
(request was received on the transport channel),
`local_node` (the local node issued the request).
| `origin_address` | The IP address from which the request originated.
| `principal` | The principal (username) that failed authentication.
| `action` | The name of the action that was executed.
| `request` | The type of request that was executed.
| `indices` | A comma-separated list of indices this request
pertains to (when applicable).
|======
.Transport access_denied Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_type` | Where the request originated: `rest` (request
originated from a REST API request), `transport`
(request was received on the transport channel),
`local_node` (the local node issued the request).
| `origin_address` | The IP address from which the request originated.
| `principal` | The principal (username) that failed authentication.
| `action` | The name of the action that was executed.
| `request` | The type of request that was executed.
| `indices` | A comma-separated list of indices this request
relates to (when applicable).
|======
.Transport tampered_request Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_type` | Where the request originated: `rest` (request
originated from a REST API request), `transport`
(request was received on the transport channel),
`local_node` (the local node issued the request).
| `origin_address` | The IP address from which the request originated.
| `principal` | The principal (username) that failed to authenticate.
| `action` | The name of the action that was executed.
| `request` | The type of request that was executed.
| `indices` | A comma-separated list of indices this request
pertains to (when applicable).
|======
.IP Filter connection_granted Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_address` | The IP address from which the request originated.
| `transport_profile` | The transport profile the request targeted.
| `rule` | The <<ip-filtering, IP filtering>> rule that granted
the request.
|======
.IP Filter connection_denied Attributes
[cols="2,7",options="header"]
|======
| Attribute | Description
| `origin_address` | The IP address from which the request originated.
| `transport_profile` | The transport profile the request targeted.
| `rule` | The <<ip-filtering, IP filtering>> rule that denied
the request.
|======
[float]
[[audit-log-output]]
=== Logfile Audit Output
The `logfile` audit output is the default output for auditing. It writes data to
the `<clustername>_access.log` file in the logs directory.
[float]
[[audit-log-entry-format]]
=== Log Entry Format
The format of a log entry is:
[source,txt]
----------------------------------------------------------------------------
[<timestamp>] [<local_node_info>] [<layer>] [<entry_type>] <attribute_list>
----------------------------------------------------------------------------
`<timestamp>` :: When the event occurred. You can configure the
timestamp format in `log4j2.properties`.
`<local_node_info>` :: Information about the local node that generated
the log entry. You can control what node information
is included by configuring the
<<audit-log-entry-local-node-info, local node info settings>>.
`<layer>` :: The layer from which this event originated:
`rest`, `transport` or `ip_filter`.
`<entry_type>` :: The type of event that occurred: `anonymous_access_denied`,
`authentication_failed`, `access_denied`, `access_granted`,
`connection_granted`, `connection_denied`.
`<attribute_list>` :: A comma-separated list of key-value pairs that contain
data pertaining to the event. Formatted as
`attr1=[val1], attr2=[val2]`. See <<audit-event-attributes,
Audit Entry Attributes>> for the attributes that can be included
for each type of event.
[float]
[[audit-log-settings]]
=== Logfile Output Settings
The events and some other information about what gets logged can be
controlled using settings in the `elasticsearch.yml` file.
.Audited Event Settings
[cols="4,^2,4",options="header"]
|======
| Name | Default | Description
| `xpack.security.audit.logfile.events.include` | `access_denied`, `access_granted`, `anonymous_access_denied`, `authentication_failed`, `connection_denied`, `tampered_request`, `run_as_denied`, `run_as_granted` | Includes the specified events in the output.
| `xpack.security.audit.logfile.events.exclude` | | Excludes the specified events from the output.
| `xpack.security.audit.logfile.events.emit_request_body`| false | Include or exclude the request body from REST requests
on certain event types such as `authentication_failed`.
|======
IMPORTANT: No filtering is performed when auditing, so sensitive data may be
audited in plain text when including the request body in audit events.
[[audit-log-entry-local-node-info]]
.Local Node Info Settings
[cols="4,^2,4",options="header"]
|======
| Name | Default | Description
| `xpack.security.audit.logfile.prefix.emit_node_name` | true | Include or exclude the node's name
from the local node info.
| `xpack.security.audit.logfile.prefix.emit_node_host_address` | false | Include or exclude the node's IP address
from the local node info.
| `xpack.security.audit.logfile.prefix.emit_node_host_name` | false | Include or exclude the node's host name
from the local node info.
|======
[[logging-file]]
You configure also configure how the logfile is written in the `log4j2.properties`
file located in `CONFIG_DIR/x-pack`. By default, audit information is appended to the
`<clustername>_access.log` file located in the standard Elasticsearch `logs` directory
(typically located at `$ES_HOME/logs`). The file rolls over on a daily basis.
[float]
[[audit-index]]
=== Index Audit Output
In addition to logging to a file, you can store audit logs in Elasticsearch
rolling indices. These indices can be either on the same cluster, or on a
remote cluster. You configure the following settings in
`elasticsearch.yml` to control how audit entries are indexed. To enable
this output, you need to configure the setting `xpack.security.audit.outputs`
in the `elasticsearch.yml` file:
[source,yaml]
----------------------------
xpack.security.audit.outputs: [ index, logfile ]
----------------------------
.Audit Log Indexing Configuration
[options="header"]
|======
| Attribute | Default Setting | Description
| `xpack.security.audit.index.bulk_size` | `1000` | Controls how many audit events are batched into a single write.
| `xpack.security.audit.index.flush_interval` | `1s` | Controls how often buffered events are flushed to the index.
| `xpack.security.audit.index.rollover` | `daily` | Controls how often to roll over to a new index:
`hourly`, `daily`, `weekly`, or `monthly`.
| `xpack.security.audit.index.events.include` | `anonymous_access_denied`, `authentication_failed`, `realm_authentication_failed`, `access_granted`, `access_denied`, `tampered_request`, `connection_granted`, `connection_denied`, `run_as_granted`, `run_as_denied` | The audit events to be indexed. See <<audit-event-types, Audit Entry Types>> for the complete list.
| `xpack.security.audit.index.events.exclude` | | The audit events to exclude from indexing.
| `xpack.security.audit.index.events.emit_request_body`| false | Include or exclude the request body from REST requests
on certain event types such as `authentication_failed`.
|======
IMPORTANT: No filtering is performed when auditing, so sensitive data may be
audited in plain text when including the request body in audit events.
[float]
==== Audit Index Settings
You can also configure settings for the indices that the events are stored in.
These settings are configured in the `xpack.security.audit.index.settings` namespace
in `elasticsearch.yml`. For example, the following configuration sets the
number of shards and replicas to 1 for the audit indices:
[source,yaml]
----------------------------
xpack.security.audit.index.settings:
index:
number_of_shards: 1
number_of_replicas: 1
----------------------------
[float]
==== Forwarding Audit Logs to a Remote Cluster
To index audit events to a remote Elasticsearch cluster, you configure
the following `xpack.security.audit.index.client` settings.
.Remote Audit Log Indexing Configuration
[options="header"]
|======
| Attribute | Description
| `xpack.security.audit.index.client.hosts` | Comma-separated list of `host:port` pairs. These hosts
should be nodes in the remote cluster.
| `xpack.security.audit.index.client.cluster.name` | The name of the remote cluster.
| `xpack.security.audit.index.client.xpack.security.user` | The `username:password` pair to use to authenticate with
the remote cluster.
|======
You can pass additional settings to the remote client by specifying them in the
`xpack.security.audit.index.client` namespace. For example, to allow the remote
client to discover all of the nodes in the remote cluster you can specify the
`client.transport.sniff` setting:
[source,yaml]
----------------------------
xpack.security.audit.index.client.transport.sniff: true
----------------------------

265
docs/en/security/authentication.asciidoc Normal file
View File

@ -0,0 +1,265 @@
[[setting-up-authentication]]
== Setting Up User Authentication
Authentication identifies an individual. To gain access to restricted resources,
a user must prove their identity, via passwords, credentials, or some other
means (typically referred to as authentication tokens).
You can use the native support for managing and authenticating users, or
integrate with external user management systems such as LDAP and Active
Directory. For information about managing native users,
see <<managing-native-users, Managing Native Users>>.
[float]
[[built-in-users]]
=== Built-in Users
{security} provides built-in user credentials to help you get up and running.
These users have a fixed set of privileges and the default password `changeme`.
Please read <<reset-built-in-user-passwords,Reset Built-in User Passwords>> and
<<disabling-default-password, Disable Default Password Functionality>> below.
.{security} Built-in Users
|========
| Name | Description
| `elastic` | A built-in <<built-in-roles-superuser, _superuser_>>.
| `kibana` | The user Kibana uses to connect and communicate with Elasticsearch.
| `logstash_system` | The user Logstash uses when storing monitoring information in Elasticsearch.
|========
[float]
[[reset-built-in-user-passwords]]
==== Reset Built-in User Passwords
[IMPORTANT]
=============================================================================
You must reset the default passwords for all built-in users, and then
<<disabling-default-password, disable default password support>>.
You can update passwords from the *Management > Users* UI in Kibana or with the
<<security-api-reset-user-password, Reset Password API>>:
[source,js]
---------------------------------------------------------------------
PUT _xpack/security/user/elastic/_password
{
"password": "elasticpassword"
}
---------------------------------------------------------------------
// CONSOLE
[source,js]
---------------------------------------------------------------------
PUT _xpack/security/user/kibana/_password
{
"password": "kibanapassword"
}
---------------------------------------------------------------------
// CONSOLE
[source,js]
---------------------------------------------------------------------
PUT _xpack/security/user/logstash_system/_password
{
"password": "logstashpassword"
}
---------------------------------------------------------------------
// CONSOLE
Once the `kibana` user password is reset, you need to update the Kibana server
with the new password by setting `elasticsearch.password` in the
`kibana.yml` configuration file:
[source,yaml]
-----------------------------------------------
elasticsearch.password: kibanapassword
-----------------------------------------------
The `logstash_system` user is used internally within Logstash when
<<monitoring-logstash-settings, monitoring>> is enabled for logstash
If you wish to enable this feature in Logstash, then you need to update the Logstash
configuration with the new password by setting `xpack.monitoring.elasticsearch.password` in
the `logstash.yml` configuration file:
[source,yaml]
----------------------------------------------------------
xpack.monitoring.elasticsearch.password: logstashpassword
----------------------------------------------------------
If you have upgraded from an older version of elasticsearch/x-pack,
the `logstash_system` user may have defaulted to _disabled_ for security reasons.
Once the password has been changed, you can enable the user via the following API call:
[source,js]
---------------------------------------------------------------------
PUT _xpack/security/user/logstash_system/_enable
---------------------------------------------------------------------
// CONSOLE
=============================================================================
[float]
[[disabling-default-password]]
==== Disable Default Password Functionality
[IMPORTANT]
=============================================================================
The default password of `changeme` is provided as a convenience that allows you to quickly
setup your Elasticsearch stack. It should not be used when running in production.
Once you have changed the password for the built-in users, you must disable default password support
by setting `xpack.security.authc.accept_default_password` to `false`.
A {ref}/bootstrap-checks.html[bootstrap check] will prevent your cluster from operating in production
mode until you make this configuration change.
=============================================================================
=== How Authentication Works
Authentication in {security} is handled by one or more authentication services
called _realms_. A _realm_ is used to resolve and authenticate users based on
authentication tokens. {security} provides the following built-in realms:
_native_::
An internal realm where users are stored in a dedicated Elasticsearch index.
This realm supports an authentication token in the form of username and password,
and is available by default when no realms are explicitly configured. See
<<native-realm>>.
_ldap_::
A realm that uses an external LDAP server to authenticate the
users. This realm supports an authentication token in the form of username and
password, and requires explicit configuration in order to be used. See
<<ldap-realm>>.
_active_directory_::
A realm that uses an external Active Directory Server to authenticate the
users. With this realm, users are authenticated by usernames and passwords.
See <<active-directory-realm>>.
_pki_::
A realm that authenticates users using Public Key Infrastructure (PKI). This
realm works in conjunction with SSL/TLS and identifies the users through the
Distinguished Name (DN) of the client's X.509 certificates. See <<pki-realm>>.
_file_::
An internal realm where users are defined in files stored on each node in the
Elasticsearch cluster. This realm supports an authentication token in the form
of username and password, and is always available. See <<file-realm>>.
{security} also supports custom realms. If you need to integrate with another
authentication system, you can build a custom realm plugin. For more information,
see <<custom-realms, Integrating with Other Authentication Systems>>.
Realms live within a _realm chain_. It is essentially a prioritized list of
configured realms (typically of various types). The order of the list determines
the order in which the realms will be consulted. During the authentication process,
{security} will consult and try to authenticate the request one realm at a time.
Once one of the realms successfully authenticates the request, the authentication
is considered to be successful and the authenticated user will be associated
with the request (which will then proceed to the authorization phase). If a realm
cannot authenticate the request, the next in line realm in the chain will be
consulted. If all realms in the chain could not authenticate the request, the
authentication is then considered to be unsuccessful and an authentication error
will be returned (as HTTP status code `401`).
NOTE: Some systems (e.g. Active Directory) have a temporary lock-out period after
several successive failed login attempts. If the same username exists in
multiple realms, unintentional account lockouts are possible. For more
information, please see <<trouble-shoot-active-directory, here>>.
The default realm chain contains the `native` and `file` realms. To explicitly,
configure a realm chain, you specify the chain in `elasticsearch.yml`. When you
configure a realm chain, only the realms you specify are used for authentication.
To use the `native` and `file` realms, you must include them in the chain.
The following snippet configures a realm chain that includes the `file` and
`native` realms, as well as two LDAP realms and an Active Directory realm.
[source,yaml]
----------------------------------------
xpack.security.authc:
realms:
file:
type: file
order: 0
native:
type: native
order: 1
ldap1:
type: ldap
order: 2
enabled: false
url: 'url_to_ldap1'
...
ldap2:
type: ldap
order: 3
url: 'url_to_ldap2'
...
ad1:
type: active_directory
order: 4
url: 'url_to_ad'
----------------------------------------
As can be seen above, each realm has a unique name that identifies it and each
realm type dictates its own set of required and optional settings. That said,
there are three settings that are common to all realms:
[cols=",^,",options="header"]
|=========
| Setting | Required | Description
| `type` | true | Identifies the type of the realm. The realm type
determines what other settings the realms should be
configured with. The type can be one of: `native`,
`ldap`, `active_directory`, `pki`, `file`, or in case
of a custom realm, the type name that identifies it.
| `order` | false | A numeric value representing the priority/index of
the realm within the realm chain. This will determine
the order by which the realms will be consulted
during authentication, with lower order being consulted
first.
| `enabled` | false | When set to `false` the realm will be disabled and
will not be added to the realm chain. This is useful
for debugging purposes as it enables you to remove
a realm from the chain without deleting and losing
its configuration.
|=========
Realm types can roughly be classified in two categories:
Internal:: Realms that are internal to Elasticsearch and don't require any
communication with external parties. They are fully managed by
{security}. There can only be a maximum of one configured realm
per internal realm type. {security} provides two internal realm
types: `native` and `file`.
External:: Realms that require interaction with parties/components external to
Elasticsearch, typically, with enterprise grade identity management
systems. Unlike internal realms, there can be as many external realms
as one would like - each with its own unique name and configuration.
{security} provides three external realm types: `ldap`,
`active_directory` and `pki`.
include::authentication/anonymous-access.asciidoc[]
include::authentication/native-realm.asciidoc[]
include::authentication/ldap-realm.asciidoc[]
include::authentication/active-directory-realm.asciidoc[]
include::authentication/pki-realm.asciidoc[]
include::authentication/file-realm.asciidoc[]
include::authentication/custom-realm.asciidoc[]
include::authentication/user-cache.asciidoc[]

336
docs/en/security/authentication/active-directory-realm.asciidoc Normal file
View File

@ -0,0 +1,336 @@
[[active-directory-realm]]
=== Active Directory User Authentication
You can configure {security} to communicate with Active Directory to authenticate
users. To integrate with Active Directory, you configure an `active_directory`
realm and map Active Directory users and groups to {security} roles in the
<<mapping-roles, role mapping file>>.
To protect passwords, communications between Elasticsearch and the Active Directory
server should be encrypted using SSL/TLS. Clients and nodes that connect via
SSL/TLS to the Active Directory server need to have the Active Directory server's
certificate or the server's root CA certificate installed in their keystore or
truststore. For more information about installing certificates, see
<<active-directory-ssl>>.
==== Configuring an Active Directory Realm
{security} uses LDAP to communicate with Active Directory, so `active_directory`
realms are similar to <<ldap-realm, `ldap` realms>>. Like LDAP directories,
Active Directory stores users and groups hierarchically. The directory's
hierarchy is built from containers such as the _organizational unit_ (`ou`),
_organization_ (`o`), and _domain controller_ (`dc`).
The path to an entry is a _Distinguished Name_ (DN) that uniquely identifies a
user or group. User and group names typically have attributes such as a
_common name_ (`cn`) or _unique ID_ (`uid`). A DN is specified as a string, for
example `"cn=admin,dc=example,dc=com"` (white spaces are ignored).
{security} only supports Active Directory security groups. You cannot map
distribution groups to roles.
NOTE: When you use Active Directory for authentication, the username entered by
the user is expected to match the `sAMAccountName` or `userPrincipalName`,
not the common name.
To configure an `active_directory` realm:
. Add a realm configuration of type `active_directory` to `elasticsearch.yml`
under the `xpack.security.authc.realms` namespace. At a minimum, you must set the realm
`type` to `active_directory` and specify the Active Directory `domain_name`. To
use SSL/TLS for secured communication with the Active Directory server, you must
also set the `url` attribute and specify the `ldaps` protocol and secure port
number. If you are configuring multiple realms, you should also explicitly set
the `order` attribute to control the order in which the realms are consulted
during authentication. See <<ad-settings, Active Directory Realm Settings>>
for all of the options you can set for an `active_directory` realm.
+
NOTE: Binding to Active Directory fails if the domain name is not mapped in DNS.
If DNS is not being provided by a Windows DNS server, add a mapping for
the domain in the local `/etc/hosts` file.
+
For example, the following realm configuration configures {security} to connect
to `ldaps://example.com:636` to authenticate users through Active Directory.
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
active_directory:
type: active_directory
order: 0 <1>
domain_name: ad.example.com
url: ldaps://ad.example.com:636 <2>
unmapped_groups_as_roles: true <3>
------------------------------------------------------------
<1> The realm order controls the order in which the configured realms are checked
when authenticating a user.
<2> If you don't specify the URL, it defaults to `ldap:<domain_name>:389`.
<3> When this option is enabled, Active Directory groups are automatically mapped
to roles of the same name.
+
IMPORTANT: When you configure realms in `elasticsearch.yml`, only the
realms you specify are used for authentication. If you also want to use the
`native` or `file` realms, you must include them in the realm chain.
. Restart Elasticsearch.
===== Multiple Domain Support
When authenticating users across multiple domains in a forest, there are a few minor
differences in the configuration and the way that users will authenticate. The `domain_name`
setting should be set to the forest root domain name. The `url` setting also needs to
be set as you will need to authenticate against the Global Catalog, which uses a different
port and may not be running on every Domain Controller.
For example, the following realm configuration configures {security} to connect to specific
Domain Controllers on the Global Catalog port with the domain name set to the forest root.
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
active_directory:
type: active_directory
order: 0
domain_name: example.com <1>
url: ldaps://dc1.ad.example.com:3269, ldaps://dc2.ad.example.com:3269 <2>
load_balance:
type: "round_robin" <3>
------------------------------------------------------------
<1> The `domain_name` is set to the name of the root domain in the forest.
<2> The `url` value used in this example has URLs for two different Domain Controllers,
which are also Global Catalog servers. Port 3268 is the default port for unencrypted
communication with the Global Catalog; port 3269 is the default port for SSL connections.
The servers that are being connected to can be in any domain of the forest as long as
they are also Global Catalog servers.
<3> A load balancing setting is provided to indicate the desired behavior when choosing
the server to connect to.
In this configuration, users will need to use either their full User Principal
Name (UPN) or their Down-Level Logon Name. A UPN is typically a concatenation of
the username with `@<DOMAIN_NAME` such as `johndoe@ad.example.com`. The Down-Level
Logon Name is the NetBIOS domain name, followed by a `\` and the username, such as
`AD\johndoe`. Use of Down-Level Logon Name requires a connection to the regular LDAP
ports (389 or 636) in order to query the configuration container to retrieve the
domain name from the NetBIOS name.
[[ad-load-balancing]]
===== Load Balancing and Failover
The `load_balance.type` setting can be used at the realm level to configure how
{security} should interact with multiple Active Directory servers. Two modes of
operation are supported: failover and load balancing
.Load Balancing and Failover Types
|=======================
| Type | | | Description
| `failover` | | | The URLs specified are used in the order that they are
specified. The first server that can be connected to will
be used for all subsequent connections. If a connection to
that server fails then the next server that a connection
can be established to will be used for subsequent connections.
| `dns_failover` | | | In this mode of operation, only a single URL may be specified.
This URL must contain a DNS name. The system will be queried
for all IP addresses that correspond to this DNS name.
Connections to the Active Directory server will always be
tried in the order in which they were retrieved. This differs
from `failover` in that there is no reordering of the list
and if a server has failed at the beginning of the list, it
will still be tried for each subsequent connection.
| `round_robin` | | | Connections will continuously iterate through the list of
provided URLs. If a server is unavailable, iterating through
the list of URLs will continue until a successful connection
is made.
| `dns_round_robin` | | | In this mode of operation, only a single URL may be specified.
This URL must contain a DNS name. The system will be queried
for all IP addresses that correspond to this DNS name.
Connections will continuously iterate through the list of
addresses. If a server is unavailable, iterating through the
list of URLs will continue until a successful connection is
made.
|=======================
[[ad-settings]]
===== Active Directory Realm Settings
[cols="4,^3,10"]
|=======================
| Setting | Required | Description
| `type` | yes | Indicates the realm type. Must be set to `active_directory`.
| `order` | no | Indicates the priority of this realm within the realm chain.
Realms with a lower order are consulted first. Although not
required, we recommend explicitly setting this value when
you configure multiple realms. Defaults to `Integer.MAX_VALUE`.
| `enabled` | no | Indicates whether this realm is enabled or disabled. Enables
you to disable a realm without removing its configuration.
Defaults to `true`.
| `domain_name` | yes | Specifies the domain name of the Active Directory. {security}
uses the domain name to derive the LDAP URL and `user_search_dn`
if those fields are not specified.
| `url` | no/yes | Specifies an LDAP URL of the form `ldap[s]://<server>:<port>`.
{security} attempts to authenticate against this URL. If the
URL is not specified, it is derived from the `domain_name`,
assuming an unencrypted connection to port 389. For example,
`ldap://<domain_name>:389`. This settings is required when
connecting using SSL/TLS or via a custom port.
| `load_balance.type` | no | The behavior to use when there are multiple LDAP URLs defined.
For supported values see <<ad-load-balancing>>.
| `load_balance.cache_ttl` | no | When using `dns_failover` or `dns_round_robin` as the load
balancing type, this setting controls the amount of time to
cache DNS lookups. Defaults to `1h`.
| `user_search.base_dn` | no | Specifies the context to search for the user. Defaults to the
root of the Active Directory domain.
| `user_search.scope` | no | Specifies whether the user search should be `sub_tree` (default),
`one_level`, or `base`. `sub_tree` searches all objects contained
under `base_dn`. `one_level` only searches users directly
contained within the `base_dn`. `base` specifies that the
`base_dn` is a user object and that it is the only user considered.
| `user_search.filter` | no | Specifies a filter to use to lookup a user given a username.
The default filter looks up `user` objects with either
`sAMAccountName` or `userPrincipalName`. If specified, this
must be a valid LDAP user search filter, for example
`(&(objectClass=user)(sAMAccountName={0}))`. For more
information, see https://msdn.microsoft.com/en-us/library/aa746475(v=vs.85).aspx[Search Filter Syntax].
| `group_search.base_dn` | no | Specifies the context to search for groups in which the user
has membership. Defaults to the root of the Active Directory
domain.
| `group_search.scope` | no | Specifies whether the group search should be `sub_tree` (default),
`one_level` or `base`. `sub_tree` searches all objects contained
under `base_dn`. `one_level` searches for groups directly
contained within the `base_dn`. `base` specifies that the
`base_dn` is a group object and that it is the only group considered.
| `unmapped_groups_as_roles` | no | Specifies whether the names of any unmapped Active Directory
groups should be used as role names and assigned to the user.
Defaults to `false`.
| `files.role_mapping` | no | Specifies the path and file name of the
<<ad-role-mapping, YAML role mapping configuration file>>.
Defaults to `CONF_DIR/x-pack/role_mapping.yml`,
where `CONF_DIR` is `ES_HOME/config` (zip/tar installations)
or `/etc/elasticsearch` (package installations).
| `follow_referrals` | no | Specifies whether {security} should follow referrals returned
by the Active Directory server. Referrals are URLs returned by
the server that are to be used to continue the LDAP operation
(such as `search`). Defaults to `true`.
| `ssl.key` | no | Specifies the path to the PEM encoded private key to use if the Active Directory
server requires client authentication. `ssl.key` and `ssl.keystore.path` may not be used at the
same time.
| `ssl.key_passphrase` | no | Specifies the passphrase to decrypt the PEM encoded private key if it is encrypted.
| `ssl.certificate` | no | Specifies the path to the PEM encoded certificate (or certificate chain) that goes with the key
if the Active Directory server requires client authentication.
| `ssl.certificate_authorities`| no | Specifies the paths to the PEM encoded certificate authority certificates that
should be trusted. `ssl.certificate_authorities` and `ssl.trustsore.path` may not be used at
the same time.
| `ssl.keystore.path` | no | The path to the Java Keystore file that contains a private key and certificate. `ssl.key` and
`ssl.keystore.path` may not be used at the same time.
| `ssl.keystore.password` | no | The password to the keystore.
| `ssl.keystore.key_password`| no | The password for the key in the keystore. Defaults to the keystore password.
| `ssl.truststore.path` | no | The path to the Java Keystore file that contains the certificates to trust.
`ssl.certificate_authorities` and `ssl.trustsore.path` may not be used at the same time.
| `ssl.truststore.password` | no | The password to the truststore.
| `ssl.verification_mode` | no | Specifies the type of verification to be performed when
connecting to an Active Directory server using `ldaps`. When
set to `full`, the hostname or IP address used in the `url`
must match one of the names in the certificate or the
connection will not be allowed. Due to their potential security impact,
`ssl` settings are not exposed via the
{ref}/cluster-nodes-info.html#cluster-nodes-info[nodes info API].
Values are `none`, `certificate`, and `full`. Defaults to `full`.
| `ssl.supported_protocols` | no | Specifies the supported protocols for TLS/SSL.
| `ssl.cipher_suites` | no | Specifies the cipher suites that should be supported when communicating
with the Active Directory server.
| `cache.ttl` | no | Specifies the time-to-live for cached user entries. A user's
credentials are cached for this period of time. Specify the
time period using the standard Elasticsearch
{ref}/common-options.html#time-units[time units].
Defaults to `20m`.
| `cache.max_users` | no | Specifies the maximum number of user entries that can be
stored in the cache at one time. Defaults to 100,000.
| `cache.hash_algo` | no | Specifies the hashing algorithm that is used for the
cached user credentials.
See <<cache-hash-algo, Cache hash algorithms>> for the
possible values. (Expert Setting).
|=======================
[[mapping-roles-ad]]
==== Mapping Active Directory Users and Groups to Roles
An integral part of a realm authentication process is to resolve the roles
associated with the authenticated user. Roles define the privileges a user has
in the cluster.
Since with the `active_directory` realm the users are managed externally in the
Active Directory server, the expectation is that their roles are managed there
as well. In fact, Active Directory supports the notion of groups, which often
represent user roles for different systems in the organization.
The `active_directory` realm enables you to map Active Directory users and groups
to roles in the role mapping file stored on each node. You specify users and
groups using their distinguished names (DNs). For example, the following mapping
configuration maps the Active Directory `admins` group to both the `monitoring`
and `user` roles, maps the `users` group to the `user` role and maps the `John Doe`
user to the `user` role.
[source, yaml]
------------------------------------------------------------
monitoring: <1>
- "cn=admins,dc=example,dc=com" <2>
user:
- "cn=users,dc=example,dc=com" <3>
- "cn=admins,dc=example,dc=com"
- "cn=John Doe,cn=contractors,dc=example,dc=com" <4>
------------------------------------------------------------
<1> The name of the role.
<2> The Active Directory distinguished name (DN) of the `admins` group.
<3> The Active Directory distinguished name (DN) of the `users` group.
<4> The Active Directory distinguished name (DN) of the user `John Doe`.
For more information, see <<mapping-roles, Mapping Users and Groups to Roles>>.
[[active-directory-ssl]]
==== Setting up SSL Between Elasticsearch and Active Directory
To protect the user credentials that are sent for authentication, it's highly
recommended to encrypt communications between Elasticsearch and your Active
Directory server. Connecting via SSL/TLS ensures that the identity of the Active
Directory server is authenticated before {security} transmits the user
credentials, and the usernames and passwords are encrypted in transit.
To encrypt communications between Elasticsearch and Active Directory:
. Configure each node to trust certificates signed by the CA that signed your
Active Directory server certificates. The following example demonstrates how to trust a CA certificate,
`cacert.pem`, located within the {xpack} configuration directory:
+
[source,shell]
--------------------------------------------------
xpack:
security:
authc:
realms:
active_directory:
type: active_directory
order: 0
domain_name: ad.example.com
url: ldaps://ad.example.com:636
ssl:
certificate_authorities: [ "CONFIG_DIR/x-pack/cacert.pem" ]
--------------------------------------------------
+
The CA cert must be a PEM encoded certificate.
. Set the `url` attribute in the realm configuration to specify the LDAPS protocol
and the secure port number. For example, `url: ldaps://ad.example.com:636`.
. Restart Elasticsearch.
NOTE: By default, when you configure {security} to connect to Active Directory
using SSL/TLS, {security} attempts to verify the hostname or IP address
specified with the `url` attribute in the realm configuration with the
values in the certificate. If the values in the certificate and realm
configuration do not match, {security} does not allow a connection to the
Active Directory server. This is done to protect against man-in-the-middle
attacks. If necessary, you can disable this behavior by setting the
<<ssl-tls-settings, `ssl.verification_mode`>> property to `none`.

30
docs/en/security/authentication/anonymous-access.asciidoc Normal file
View File

@ -0,0 +1,30 @@
[[anonymous-access]]
=== Enabling Anonymous Access
Incoming requests are considered to be _anonymous_ if no authentication token
can be extracted from the incoming request. By default, anonymous requests are rejected and an authentication error is returned (status code `401`).
To enable anonymous access, you assign one or more roles to anonymous
users in the `elasticsearch.yml` configuration file. For example, the following
configuration assigns anonymous users `role1` and `role2`:
[source,yaml]
----------------------------------------
xpack.security.authc:
anonymous:
username: anonymous_user <1>
roles: role1, role2 <2>
authz_exception: true <3>
----------------------------------------
<1> The username/principal of the anonymous user. Defaults to
`_es_anonymous_user` if not specified.
<2> The roles to associate with the anonymous user. If no roles are specified, anonymous access is disabled--anonymous requests will be rejected and return an authentication error.
<3> When `true`, a 403 HTTP status code is returned if the anonymous user
does not have the permissions needed to perform the requested action and the
user will NOT be prompted to provide credentials to access the requested
resource. When `false`, a 401 HTTP status code is returned if the anonymous user
does not have the necessary permissions and the user is prompted for
credentials to access the requested resource. If you are using anonymous access
in combination with HTTP, you might need to set `authz_exception` to `false`
if your client does not support preemptive basic authentication. Defaults to
`true`.

107
docs/en/security/authentication/custom-realm.asciidoc Normal file
View File

@ -0,0 +1,107 @@
[[custom-realms]]
=== Integrating with Other Authentication Systems
If you are using an authentication system that is not supported out-of-the-box
by {security}, you can create a custom realm to interact with it to authenticate
users. You implement a custom realm as an {xpack} extension.
[[implementing-custom-realm]]
==== Implementing a Custom Realm
Sample code that illustrates the structure and implementation of a custom realm
is provided in the https://github.com/elastic/shield-custom-realm-example[custom-realm-example]
repository on GitHub. You can use this code as a starting point for creating your
own realm.
To create a custom realm, you need to:
. Extend `org.elasticsearch.xpack.security.authc.Realm` to communicate with your
authentication system to authenticate users.
. Implement the `org.elasticsearch.xpack.security.authc.Realm.Factory` interface in
a class that will be used to create the custom realm.
. Extend `org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler` to
handle authentication failures when using your custom realm.
To package your custom realm as a plugin:
. Implement an extension class for your realm that extends
`org.elasticsearch.xpack.extensions.XPackExtension`. There you need to
override one or more of the following methods:
+
[source,java]
----------------------------------------------------
@Override
public Map<String, Factory> getRealms() {
...
}
----------------------------------------------------
+
The `getRealms` method is used to provide a map of type names to the `Factory` that
will be used to create the realm.
+
[source,java]
----------------------------------------------------
@Override
public AuthenticationFailureHandler getAuthenticationFailureHandler() {
...
}
----------------------------------------------------
+
The `getAuthenticationFailureHandler` method is used to optionally provide a
custom `AuthenticationFailureHandler`, which will control how X-Pack responds
in certain authentication failure events.
+
[source,java]
----------------------------------------------------
@Override
public Collection<String> getRestHeaders() {
...
}
----------------------------------------------------
+
The `getRestHeaders` method returns a collection of header names that should be
copied from the request into the `ThreadContext` where they can be accessed by
the realm.
+
[source,java]
----------------------------------------------------
@Override
public List<String> getSettingsFilter() {
...
}
----------------------------------------------------
+
The `getSettingsFilter` method returns a list of setting names that should be
filtered from the settings APIs as they may contain sensitive credentials.
. Create a build configuration file for the plugin; Gradle is our recommendation.
. Create a `x-pack-extension-descriptor.properties` descriptor file for the
extension.
. Bundle all in a single zip file.
[[using-custom-realm]]
==== Using a Custom Realm to Authenticate Users
To use a custom realm:
. Install the realm extension on each node in the cluster. You run
`bin/x-pack/extension` with the `install` sub-command and specify the URL
pointing to the zip file that contains the extension. For example:
+
[source,shell]
----------------------------------------
bin/x-pack/extension install file:///<path>/my-realm-1.0.zip
----------------------------------------
. Add a realm configuration of the appropriate realm type to `elasticsearch.yml`
under the `xpack.security.authc.realms` namespace. The options you can set depend
on the settings exposed by the custom realm. At a minimum, you must set the realm
`type` to the type defined by the extension. If you are configuring multiple
realms, you should also explicitly set the `order` attribute to control the
order in which the realms are consulted during authentication.
+
IMPORTANT: When you configure realms in `elasticsearch.yml`, only the
realms you specify are used for authentication. If you also want to use the
`native` or `file` realms, you must include them in the realm chain.
. Restart Elasticsearch.

292
docs/en/security/authentication/file-realm.asciidoc Normal file
View File

@ -0,0 +1,292 @@
[[file-realm]]
=== File-based User Authentication
You can manage and authenticate users with the built-in `file` internal realm.
With the `file` realm users are defined in local files on each node in the cluster.
IMPORTANT: As the administrator of the cluster, it is your responsibility to
ensure the same users are defined on every node in the cluster.
{security} does not deliver any mechanism to guarantee this.
The `file` realm is primarily supported to serve as a fallback/recovery realm. It
is mostly useful in situations where all users locked themselves out of the system
(no one remembers their username/password). In this type of scenarios, the `file`
realm is your only way out - you can define a new `admin` user in the `file` realm
and use it to log in and reset the credentials of all other users.
IMPORTANT: When you configure realms in `elasticsearch.yml`, only the
realms you specify are used for authentication. To use the
`file` realm as a fallback, you must include it in the realm chain.
To define users, {security} provides the <<managing-file-users, users>> command-line
tool. This tool enables you to add and remove users, assign user roles and manage
user passwords.
==== Configuring a File Realm
The `file` realm is added to the realm chain by default. You don't need to
explicitly configure a `file` realm to manage users with the `users` tool.
Like other realms, you can configure options for a `file` realm in the
`xpack.security.authc.realms` namespace in `elasticsearch.yml`.
To configure an `file` realm:
. Add a realm configuration of type `file` to `elasticsearch.yml` under the
`xpack.security.authc.realms` namespace. At a minimum, you must set the realm `type` to
`file`. If you are configuring multiple realms, you should also explicitly set
the `order` attribute. See <<file-realm-settings>> for all of the options you can set
for a `file` realm.
+
For example, the following snippet shows a `file` realm configuration that sets
the `order` to zero so the realm is checked first:
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
file1:
type: file
order: 0
------------------------------------------------------------
. Restart Elasticsearch.
[[file-realm-settings]]
===== File Realm Settings
[cols="4,^3,10"]
|=======================
| Setting | Required | Description
| `type` | yes | Indicates the realm type. Must be set to `file`.
| `order` | no | Indicates the priority of this realm within the
realm chain. Realms with a lower order are
consulted first. Although not required, we
recommend explicitly setting this value when you
configure multiple realms. Defaults to
`Integer.MAX_VALUE`.
| `enabled` | no | Indicates whether this realm is enabled or
disabled. Enables you to disable a realm without
removing its configuration. Defaults to `true`.
| `cache.ttl` | no | Specifies the time-to-live for cached user entries.
A user's credentials are cached for this period of
time. Specify the time period using the standard
Elasticsearch {ref}/common-options.html#time-units[time units].
Defaults to `20m`.
| `cache.max_users` | no | Specifies the maximum number of user entries that
can be stored in the cache at one time. Defaults
to 100,000.
| `cache.hash_algo` | no | Specifies the hashing algorithm that is used for
the cached user credentials. See <<cache-hash-algo,
Cache hash algorithms>> for the possible values.
(Expert Setting).
|=======================
[[managing-file-users]]
==== Managing Users
The `users` command-line tool is located in `ES_HOME/bin/x-pack` and enables
several administrative tasks for managing users:
* <<file-realm-add-user, Adding users>>
* <<file-realm-list-users, Listing users and roles>>
* <<file-realm-manage-passwd, Managing user passwords>>
* <<file-realm-manage-roles, Managing users' roles>>
* <<file-realm-remove-user, Removing users>>
[[file-realm-add-user]]
===== Adding Users
Use the `useradd` sub-command to add a user to your local node.
NOTE: To ensure that Elasticsearch can read the user and role information at
startup, run `users useradd` as the same user you use to run Elasticsearch.
Running the command as root or some other user will update the permissions
for the `users` and `users_roles` files and prevent Elasticsearch from
accessing them.
[source,shell]
----------------------------------------
bin/x-pack/users useradd <username>
----------------------------------------
A username must be at least 1 character and no longer than 30 characters. The
first character must be a letter (`a-z` or `A-Z`) or an underscore (`_`).
Subsequent characters can be letters, underscores (`_`), digits (`0-9`), or any
of the following symbols `@`, `-`, `.` or `$`.
You can specify the user's password at the command-line with the `-p` option.
When this option is absent, the command prompts you for the password. Omit the
`-p` option to keep plaintext passwords out of the terminal session's command
history.
[source,shell]
----------------------------------------------------
bin/x-pack/users useradd <username> -p <secret>
----------------------------------------------------
Passwords must be at least 6 characters long.
You can define a user's roles with the `-r` option. This option accepts a
comma-separated list of role names to assign to the user.
[source,shell]
-------------------------------------------------------------------
bin/x-pack/users useradd <username> -r <comma-separated list of role names>
-------------------------------------------------------------------
The following example adds a new user named `jacknich` to the `file` realm. The
password for this user is `theshining`, and this user is associated with the
`network` and `monitoring` roles.
[source,shell]
-------------------------------------------------------------------
bin/x-pack/users useradd jacknich -p theshining -r network,monitoring
-------------------------------------------------------------------
For valid role names please see <<valid-role-name, Role Definitions>>.
[[file-realm-list-users]]
===== Listing Users
Use the `list` sub-command to list the users registered with the `file` realm
on the local node.
[source, shell]
----------------------------------
bin/x-pack/users list
rdeniro : admin
alpacino : power_user
jacknich : monitoring,network
----------------------------------
Users are in the left-hand column and their corresponding roles are listed in
the right-hand column.
The `list <username>` sub-command lists a specific user. Use this command to
verify that a user was successfully added to the local `file` realm.
[source,shell]
-----------------------------------
bin/x-pack/users list jacknich
jacknich : monitoring,network
-----------------------------------
[[file-realm-manage-passwd]]
===== Managing User Passwords
Use the `passwd` sub-command to reset a user's password. You can specify the new
password directly with the `-p` option. When `-p` option is omitted, the tool
will prompt you to enter and confirm a password in interactive mode.
[source,shell]
--------------------------------------------------
bin/x-pack/users passwd <username>
--------------------------------------------------
[source,shell]
--------------------------------------------------
bin/x-pack/users passwd <username> -p <password>
--------------------------------------------------
[[file-realm-manage-roles]]
===== Assigning Users to Roles
Use the `roles` sub-command to manage the roles of a particular user. The `-a`
option adds a comma-separated list of roles to a user. The `-r` option removes
a comma-separated list of roles from a user. You can combine adding and removing
roles within the same command to change a user's roles.
[source,shell]
------------------------------------------------------------------------------------------------------------
bin/x-pack/users roles <username> -a <commma-separate list of roles> -r <comma-separated list of roles>
------------------------------------------------------------------------------------------------------------
The following command removes the `network` and `monitoring` roles from user
`jacknich` and adds the `user` role:
[source,shell]
------------------------------------------------------------
bin/x-pack/users roles jacknich -r network,monitoring -a user
------------------------------------------------------------
Listing the user displays the new role assignment:
[source,shell]
---------------------------------
bin/x-pack/users list jacknich
jacknich : user
---------------------------------
[[file-realm-remove-user]]
===== Deleting Users
Use the `userdel` sub-command to delete a user.
[source,shell]
--------------------------------------------------
bin/x-pack/users userdel <username>
--------------------------------------------------
==== A Look Under the Hood
All the data about the users for the `file` realm is stored in two files, `users`
and `users_roles`. Both files are located in `CONFIG_DIR/x-pack/` and are read
on startup.
By default, {security} checks these files for changes every 5 seconds. You can
change this default behavior by changing the `resource.reload.interval.high` setting in
the `elasticsearch.yml` file (as this is a common setting in Elasticsearch,
changing its value may effect other schedules in the system).
[IMPORTANT]
==============================
These files are managed locally by the node and are **not** managed
globally by the cluster. This means that with a typical multi-node cluster,
the exact same changes need to be applied on each and every node in the
cluster.
A safer approach would be to apply the change on one of the nodes and have the
`users` and `users_roles` files distributed/copied to all other nodes in the
cluster (either manually or using a configuration management system such as
Puppet or Chef).
==============================
While it is possible to modify these files directly using any standard text
editor, we strongly recommend using the `bin/x-pack/users` command-line tool
to apply the required changes.
[float]
[[users-file]]
===== The `users` File
The `users` file stores all the users and their passwords. Each line in the
`users` file represents a single user entry consisting of the username and
**hashed** password.
[source,bash]
----------------------------------------------------------------------
rdeniro:$2a$10$BBJ/ILiyJ1eBTYoRKxkqbuDEdYECplvxnqQ47uiowE7yGqvCEgj9W
alpacino:$2a$10$cNwHnElYiMYZ/T3K4PvzGeJ1KbpXZp2PfoQD.gfaVdImnHOwIuBKS
jacknich:$2a$10$GYUNWyABV/Ols/.bcwxuBuuaQzV6WIauW6RdboojxcixBq3LtI3ni
----------------------------------------------------------------------
NOTE: {security} uses `bcrypt` to hash the user passwords.
[float]
[[users_defining-roles]]
==== The `users_roles` File
The `users_roles` file stores the roles associated with the users, as in the
following example:
[source,shell]
--------------------------------------------------
admin:rdeniro
power_user:alpacino,jacknich
user:jacknich
--------------------------------------------------
Each row maps a role to a comma-separated list of all the users that are
associated with that role.

406
docs/en/security/authentication/ldap-realm.asciidoc Normal file
View File

@ -0,0 +1,406 @@
[[ldap-realm]]
=== LDAP User Authentication
You can configure {security} to communicate with a Lightweight Directory Access
Protocol (LDAP) server to authenticate users. To integrate with LDAP, you
configure an `ldap` realm and map LDAP groups to user roles in the
<<mapping-roles, role mapping file>>.
To protect passwords, communications between Elasticsearch and the LDAP server
should be encrypted using SSL/TLS. Clients and nodes that connect via SSL/TLS to
the LDAP server need to have the LDAP server's certificate or the server's root
CA certificate installed in their _keystore_ or _truststore_. For more information
about installing certificates, see <<ldap-ssl>>.
==== Configuring an LDAP Realm
LDAP stores users and groups hierarchically, similar to the way folders are
grouped in a file system. An LDAP directory's hierarchy is built from containers
such as the _organizational unit_ (`ou`), _organization_ (`o`), and
_domain controller_ (`dc`).
The path to an entry is a _Distinguished Name_ (DN) that uniquely identifies a
user or group. User and group names typically have attributes such as a
_common name_ (`cn`) or _unique ID_ (`uid`). A DN is specified as a string,
for example `"cn=admin,dc=example,dc=com"` (white spaces are ignored).
The `ldap` realm supports two modes of operation, a user search mode
and a mode with specific templates for user DNs. See
<<ldap-settings, LDAP Realm Settings>> for all of the options you can set for an
`ldap` realm.
[[ldap-user-search]]
===== User Search Mode
LDAP user search is the most common mode of operation. In this mode, a specific
user with permission to search the LDAP directory is used to search for the
authenticating user DN based on its username and an LDAP attribute. Once found,
the user will be authenticated by attempting to bind to the LDAP server using the
found DN and the provided password.
To configure an `ldap` Realm with User Search:
. Add a realm configuration of type `ldap` to `elasticsearch.yml` under the
`xpack.security.authc.realms` namespace. At a minimum, you must set the realm `type`
to `ldap`, specify the `url` of the LDAP server, and set `user_search.base_dn`
to the container DN where the users are searched for. If you are configuring
multiple realms, you should also explicitly set the `order` attribute to control
the order in which the realms are consulted during authentication. See
<<ldap-settings, LDAP Realm Settings>> for all of the options you can set for an
`ldap` realm.
+
For example, the following snippet shows an LDAP realm configured with a user search:
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
ldap1:
type: ldap
order: 0
url: "ldaps://ldap.example.com:636"
bind_dn: "cn=ldapuser, ou=users, o=services, dc=example, dc=com"
bind_password: changeme
user_search:
base_dn: "dc=example,dc=com"
attribute: cn
group_search:
base_dn: "dc=example,dc=com"
files:
role_mapping: "CONFIG_DIR/x-pack/role_mapping.yml"
unmapped_groups_as_roles: false
------------------------------------------------------------
+
IMPORTANT: When you configure realms in `elasticsearch.yml`, only the
realms you specify are used for authentication. If you also want to use the
`native` or `file` realms, you must include them in the realm chain.
. Restart Elasticsearch
===== User DN Templates Mode
If your LDAP environment uses a few specific standard naming conditions for
users, you can use User DN templates to configure the realm. The advantage of
this method is that a search does not have to be performed to find the user DN.
However, multiple bind operations might be needed to find the correct user DN.
To configure an `ldap` Realm with User Search:
. Add a realm configuration of type `ldap` to `elasticsearch.yml` in the
`xpack.security.authc.realms` namespace. At a minimum, you must set the realm `type` to
`ldap`, specify the `url` of the LDAP server, and specify at least one template
with the `user_dn_templates` option. If you are configuring multiple realms, you
should also explicitly set the `order` attribute to control the order in which
the realms are consulted during authentication. See <<ldap-settings, LDAP Realm Settings>>
for all of the options you can set for an `ldap` realm.
+
For example, the following snippet shows an LDAP realm configured with User DN templates:
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
ldap1:
type: ldap
order: 0
url: "ldaps://ldap.example.com:636"
user_dn_templates:
- "cn={0}, ou=users, o=marketing, dc=example, dc=com"
- "cn={0}, ou=users, o=engineering, dc=example, dc=com"
group_search:
base_dn: "dc=example,dc=com"
files:
role_mapping: "/mnt/elasticsearch/group_to_role_mapping.yml"
unmapped_groups_as_roles: false
------------------------------------------------------------
. Restart Elasticsearch
[[ldap-load-balancing]]
===== Load Balancing and Failover
The `load_balance.type` setting can be used at the realm level to configure how
{security} should interact with multiple LDAP servers. {security} supports both
failover and load balancing modes of operation.
.Load Balancing and Failover Types
|=======================
| Type | | | Description
| `failover` | | | The URLs specified are used in the order that they are specified.
The first server that can be connected to will be used for all
subsequent connections. If a connection to that server fails then
the next server that a connection can be established to will be
used for subsequent connections.
| `dns_failover` | | | In this mode of operation, only a single URL may be specified.
This URL must contain a DNS name. The system will be queried for
all IP addresses that correspond to this DNS name. Connections to
the LDAP server will always be tried in the order in which they
were retrieved. This differs from `failover` in that there is no
reordering of the list and if a server has failed at the beginning
of the list, it will still be tried for each subsequent connection.
| `round_robin` | | | Connections will continuously iterate through the list of provided
URLs. If a server is unavailable, iterating through the list of
URLs will continue until a successful connection is made.
| `dns_round_robin` | | | In this mode of operation, only a single URL may be specified. This
URL must contain a DNS name. The system will be queried for all IP
addresses that correspond to this DNS name. Connections will
continuously iterate through the list of addresses. If a server is
unavailable, iterating through the list of URLs will continue until
a successful connection is made.
|=======================
[[ldap-settings]]
===== LDAP Realm Settings
.Common LDAP Realm Settings
[cols="4,^3,10"]
|=======================
| Setting | Required | Description
| `type` | yes | Indicates the realm type. Must be set to `ldap`.
| `order` | no | Indicates the priority of this realm within the realm
chain. Realms with a lower order are consulted first.
Although not required, we recommend explicitly
setting this value when you configure multiple realms.
Defaults to `Integer.MAX_VALUE`.
| `enabled` | no | Indicates whether this realm is enabled or disabled.
Enables you to disable a realm without removing its
configuration. Defaults to `true`.
| `url` | yes | Specifies one or more LDAP URLs of the form of
`ldap[s]://<server>:<port>`. Multiple URLs can be
defined using a comma separated value or array syntax:
`[ "ldaps://server1:636", "ldaps://server2:636" ]`.
`ldaps` and `ldap` URL protocols cannot be mixed in
the same realm.
| `load_balance.type` | no | The behavior to use when there are multiple LDAP URLs
defined. For supported values see
<<ldap-load-balancing, LDAP load balancing and failover types>>.
| `load_balance.cache_ttl` | no | When using `dns_failover` or `dns_round_robin` as the
load balancing type, this setting controls the amount of time
to cache DNS lookups. Defaults to `1h`.
| `user_group_attribute` | no | Specifies the attribute to examine on the user for group
membership. The default is `memberOf`. This setting will
be ignored if any `group_search` settings are specified.
| `group_search.base_dn` | no | Specifies a container DN to search for groups in which
the user has membership. When this element is absent,
Security searches for the attribute specified by
`user_group_attribute` set on the user to determine
group membership.
| `group_search.scope` | no | Specifies whether the group search should be
`sub_tree`, `one_level` or `base`. `one_level` only
searches objects directly contained within the
`base_dn`. The default `sub_tree` searches all objects
contained under `base_dn`. `base` specifies that the
`base_dn` is a group object, and that it is the only
group considered.
| `group_search.filter` | no | Specifies a filter to use to lookup a group. If not
set, the realm searches for `group`,
`groupOfNames`, `groupOfUniqueNames`, or `posixGroup` with the
attributes `member`, `memberOf`, or `memberUid`. Any instance of
`{0}` in the filter is replaced by the user
attribute defined in `group_search.user_attribute`
| `group_search.user_attribute` | no | Specifies the user attribute that is fetched and
provided as a parameter to the filter. If not set,
the user DN is passed to the filter.
| `unmapped_groups_as_roles` | no | Specifies whether the names of any unmapped LDAP groups
should be used as role names and assigned to the user.
Defaults to `false`.
| `timeout.tcp_connect` | no | Specifies the TCP connect timeout period for establishing an
LDAP connection. An `s` at the end indicates seconds, or `ms`
indicates milliseconds. Defaults to `5s` (5 seconds).
| `timeout.tcp_read` | no | Specifies the TCP read timeout period after establishing an LDAP connection.
An `s` at the end indicates seconds, or `ms` indicates milliseconds.
Defaults to `5s` (5 seconds).
| `timeout.ldap_search` | no | Specifies the LDAP Server enforced timeout period for an LDAP search.
An `s` at the end indicates seconds, or `ms` indicates milliseconds.
Defaults to `5s` (5 seconds).
| `files.role_mapping` | no | Specifies the path and file name for the
<<ldap-role-mapping, YAML role mapping configuration file>>.
Defaults to `ES_HOME/config/x-pack/role_mapping.yml`.
| `follow_referrals` | no | Specifies whether {security} should follow referrals
returned by the LDAP server. Referrals are URLs returned by
the server that are to be used to continue the LDAP operation
(e.g. search). Defaults to `true`.
| `ssl.key` | no | Specifies the path to the PEM encoded private key to use if the LDAP
server requires client authentication. `ssl.key` and `ssl.keystore.path`
may not be used at the same time.
| `ssl.key_passphrase` | no | Specifies the passphrase to decrypt the PEM encoded private key if it is encrypted.
| `ssl.certificate` | no | Specifies the path to the PEM encoded certificate (or certificate chain) that goes with the
key if the LDAP server requires client authentication.
| `ssl.certificate_authorities` | no | Specifies the paths to the PEM encoded certificate authority certificates that
should be trusted. `ssl.certificate_authorities` and `ssl.trustsore.path` may not be used
at the same time.
| `ssl.keystore.path` | no | The path to the Java Keystore file that contains a private key and certificate. `ssl.key` and
`ssl.keystore.path` may not be used at the same time.
| `ssl.keystore.password` | no | The password to the keystore.
| `ssl.keystore.key_password` | no | The password for the key in the keystore. Defaults to the keystore password.
| `ssl.truststore.path` | no | The path to the Java Keystore file that contains the certificates to trust.
`ssl.certificate_authorities` and `ssl.trustsore.path` may not be used at the same time.
| `ssl.truststore.password` | no | The password to the truststore.
| `ssl.verification_mode` | no | Specifies the type of verification to be performed when
connecting to a LDAP server using `ldaps`. When
set to `full`, the hostname or IP address used in the `url`
must match one of the names in the certificate or the
connection will not be allowed. Due to their potential security impact,
`ssl` settings are not exposed via the
{ref}/cluster-nodes-info.html#cluster-nodes-info[nodes info API].
Values are `none`, `certificate`, and `full`. Defaults to `full`.
| `ssl.supported_protocols` | no | Specifies the supported protocols for SSL/TLS.
| `ssl.cipher_suites` | no | Specifies the cipher suites that should be supported when communicating
with the LDAP server.
| `cache.ttl` | no | Specifies the time-to-live for cached user entries. A
user's credentials are cached for this period of time.
Specify the time period using the standard Elasticsearch
{ref}/common-options.html#time-units[time units].
Defaults to `20m`.
| `cache.max_users` | no | Specifies the maximum number of user entries that can be
stored in the cache at one time. Defaults to 100,000.
| `cache.hash_algo` | no | Specifies the hashing algorithm that is used for the
cached user credentials. See
<<cache-hash-algo, Cache hash algorithms>> for the possible
values. (Expert Setting).
|=======================
.User Search Mode Settings
|=======================
| Setting | Required | Description
| `bind_dn` | no | The DN of the user that is used to bind to the LDAP
and perform searches. If not specified, an anonymous
bind is attempted. Due to its potential security
impact, `bind_dn` is not exposed via the
{ref}/cluster-nodes-info.html#cluster-nodes-info[nodes info API].
| `bind_password` | no | The password for the user that is used to bind to the
LDAP. Due to its potential security impact,
`bind_password` is not exposed via the
{ref}/cluster-nodes-info.html#cluster-nodes-info[nodes info API].
| `user_search.base_dn` | yes | Specifies a container DN to search for users.
| `user_search.scope` | no | The scope of the user search. Valid values are `sub_tree`,
`one_level` or `base`. `one_level` only searches objects
directly contained within the `base_dn`. `sub_tree` searches
all objects contained under `base_dn`. `base` specifies
that the `base_dn` is the user object, and that it is the
only user considered. Defaults to `sub_tree`.
| `user_search.attribute` | no | Specifies the attribute to match with the username presented
to. Defaults to `uid`.
| `user_search.pool.enabled` | no | Enables or disables connection pooling for user search. When
disabled a new connection is created for every search. The
default is `true`.
| `user_search.pool.size` | no | Specifies the maximum number of connections to the LDAP
server to allow in the connection pool. Defaults to `20`.
| `user_search.pool.initial_size` | no | The initial number of connections to create to the LDAP
server on startup. Defaults to `0`. Values greater than `0`
could cause startup failures if the LDAP server is down.
| `user_search.pool.health_check.enabled` | no | Enables or disables a health check on LDAP connections in
the connection pool. Connections are checked in the
background at the specified interval. Defaults to `true`.
| `user_search.pool.health_check.dn` | no/yes | Specifies the distinguished name to retrieve as part of
the health check. Defaults to the value of `bind_dn`.
This setting is required when `bind_dn` is not configured.
| `user_search.pool.health_check.interval` | no | How often to perform background checks of connections in
the pool. Defaults to `60s`.
|=======================
.User Templates Mode Settings
[cols="4,^3,10"]
|=======================
| Setting | Required | Description
| `user_dn_templates` | yes | Specifies the DN template that replaces the
user name with the string `{0}`. This element
is multivalued, allowing for multiple user
contexts.
|=======================
NOTE: If any settings starting with `user_search` are specified, the
`user_dn_templates` the settings are ignored.
[[mapping-roles-ldap]]
==== Mapping LDAP Groups to Roles
An integral part of a realm authentication process is to resolve the roles
associated with the authenticated user. Roles define the privileges a user has
in the cluster.
Since with the `ldap` realm the users are managed externally in the LDAP server,
the expectation is that their roles are managed there as well. If fact, LDAP
supports the notion of groups, which often represent user roles for different
systems in the organization.
The `ldap` realm enables you to map LDAP groups to roles in the role mapping
file stored on each node. When a user authenticates with LDAP, the privileges
for that user are the union of all privileges defined by the roles assigned to
the set of groups that the user belongs to.
You specify groups using their distinguished names. For example, the following
mapping configuration maps the LDAP `admins` group to both the `monitoring` and
`user` roles, and maps the `users` group to the `user` role.
[source, yaml]
------------------------------------------------------------
monitoring: <1>
- "cn=admins,dc=example,dc=com" <2>
user:
- "cn=users,dc=example,dc=com" <3>
- "cn=admins,dc=example,dc=com"
------------------------------------------------------------
<1> The name of the mapped role.
<2> The LDAP distinguished name (DN) of the `admins` group.
<3> The LDAP distinguished name (DN) of the `users` group.
For more information, see <<mapping-roles, Mapping Users and Groups to Roles>>.
[[ldap-ssl]]
==== Setting up SSL Between Elasticsearch and LDAP
To protect the user credentials that are sent for authentication, it's highly
recommended to encrypt communications between Elasticsearch and your LDAP server.
Connecting via SSL/TLS ensures that the identity of the LDAP server is
authenticated before {security} transmits the user credentials and the contents
of the connection are encrypted.
To encrypt communications between Elasticsearch and your LDAP server:
. Configure the realm's SSL settings on each node to trust certificates signed by the CA that signed your
LDAP server certificates. The following example demonstrates how to trust a CA certificate,
`cacert.pem`, located within the {xpack} configuration directory:
+
[source,shell]
--------------------------------------------------
xpack:
security:
authc:
realms:
ldap1:
type: ldap
order: 0
url: "ldaps://ldap.example.com:636"
ssl:
certificate_authorities: [ "CONFIG_DIR/x-pack/cacert.pem" ]
--------------------------------------------------
+
The CA cert must be a PEM encoded certificate.
+
[NOTE]
===============================
You can also specify the individual server certificates rather than the CA
certificate, but this is only recommended if you have a single LDAP server
or the certificates are self-signed.
===============================
. Set the `url` attribute in the realm configuration to specify the LDAPS
protocol and the secure port number. For example, `url: ldaps://ldap.example.com:636`.
. Restart Elasticsearch.
NOTE: By default, when you configure {security} to connect to an LDAP server
using SSL/TLS, {security} attempts to verify the hostname or IP address
specified with the `url` attribute in the realm configuration with the
values in the certificate. If the values in the certificate and realm
configuration do not match, {security} does not allow a connection to the
LDAP server. This is done to protect against man-in-the-middle attacks. If
necessary, you can disable this behavior by setting the
`ssl.verification_mode` property to `none`.

73
docs/en/security/authentication/migrate-tool.asciidoc Normal file
View File

@ -0,0 +1,73 @@
[[migrate-tool]]
==== Migrating File-based Users and Roles to the Native Realm
From 5.0 onward, you should use the `native` realm to manage roles and local
users. To migrate existing file-based users and roles to the native realm, use
the `migrate` tool that's included with the X-Pack plugin.
NOTE: When migrating from Shield 2.x, the `migrate` tool should be run prior
to upgrading to ensure all roles can be migrated as some may be in a deprecated
format that {xpack} cannot read. The `migrate` tool is available in Shield
2.4.0 and higher.
The `migrate` tool loads the existing file-based users and roles and calls the
user and roles APIs to add them to the native realm. You can migrate all users
and roles, or specify the ones you want to migrate. Users and roles that
already exist in the `native` realm are not replaced or overridden. If
the names you specify with the `--users` and `--roles` options don't
exist in the `file` realm, they are skipped.
Run the migrate tool after you install the X-Pack plugin. For example:
[source, sh]
----------------------------------------------------------------------
$ bin/x-pack/migrate native -U http://localhost:9200 -u elastic -p changeme
-n lee,foo -r role1,role2,role3,role4,foo
starting migration of users and roles...
importing users from [/home/es/config/shield/users]...
found existing users: [test_user, joe3, joe2]
migrating user [lee]
{"user":{"created":true}}
no user [foo] found, skipping
importing roles from [/home/es/config/shield/roles.yml]...
found existing roles: [marvel_user, role_query_fields, admin_role, role3, admin,
remote_marvel_agent, power_user, role_new_format_name_array, role_run_as,
logstash, role_fields, role_run_as1, role_new_format, kibana4_server, user,
transport_client, role1.ab, role_query]
migrating role [role1]
{"role":{"created":true}}
migrating role [role2]
{"role":{"created":true}}
role [role3] already exists, skipping
no role [foo] found, skipping
users and roles imported.
----------------------------------------------------------------------
[[migrate-tool-options]]
The `native` subcommand supports the following options:
`-U`, `--url`::
Endpoint URL of the Elasticsearch cluster to which you want to migrate the
file-based users and roles. Required.
`-u`, `--username`::
Username to use for authentication.
`-p`, `--password`::
Password to use for authentication.
`-n`, `--users`::
Comma-separated list of the users you want to migrate. If not specified, all
users are migrated.
`-r`, `--roles`::
Comma-separated list of the roles you want to migrate. If not specified, all
roles are migrated.
Additionally, the `-E` flag can be used to specify additional settings. For example
to specify a different configuration directory, the command would look like:
[source, sh]
----------------------------------------------------------------------
$ bin/x-pack/migrate native -U http://localhost:9200 -u elastic -p changeme -Epath.conf=/etc/elasticsearch
----------------------------------------------------------------------

196
docs/en/security/authentication/native-realm.asciidoc Normal file
View File

@ -0,0 +1,196 @@
[[native-realm]]
=== Native User Authentication
The easiest way to manage and authenticate users is with the internal `native`
realm. You can use the REST APIs or Kibana to add and remove users, assign user roles, and
manage user passwords.
[[native-realm-configuration]]
[float]
==== Configuring a Native Realm
The native realm is added to the realm chain by default. You don't need to
explicitly configure a native realm to manage users through the REST APIs.
IMPORTANT: When you configure realms in `elasticsearch.yml`, only the
realms you specify are used for authentication. To use the
`native` realm as a fallback, you must include it in the realm chain.
You can, however, configure options for the `native` realm in the
`xpack.security.authc.realms` namespace in `elasticsearch.yml`. Explicitly
configuring a native realm enables you to set the order in which it appears in
the realm chain, temporary disable the realm, and control its cache options.
To configure a native realm:
. Add a realm configuration of type `native` to `elasticsearch.yml` under the
`xpack.security.authc.realms` namespace. At a minimum, you must set the realm
`type` to `native`. If you are configuring multiple realms, you should also
explicitly set the `order` attribute. See <<native-settings, Native Realm Settings>>
for all of the options you can set for the `native` realm.
+
For example, the following snippet shows a `native` realm configuration that
sets the `order` to zero so the realm is checked first:
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
native1:
type: native
order: 0
------------------------------------------------------------
. Restart Elasticsearch.
[[native-settings]]
.Native Realm Settings
[cols="4,^3,10"]
|=======================
| Setting | Required | Description
| `type` | yes | Indicates the realm type. Must be set to `native`.
| `order` | no | Indicates the priority of this realm within
the realm chain. Realms with a lower order
are consulted first. Although not required,
we recommend explicitly setting this value
when you configure multiple realms. Defaults
to `Integer.MAX_VALUE`.
| `enabled` | no | Indicates whether this realm is enabled or
disabled. When set to `false`, the realm is
not added to the realm chain and therefore
is inactive. Defaults to `true`.
| `cache.ttl` | no | Specifies the time-to-live for cached user
entries. A user's credentials are cached for
this period of time. Specify the time period
using the standard Elasticsearch
{ref}/common-options.html#time-units[time units].
Defaults to `20m`.
| `cache.max_users` | no | Specifies the maximum number of user entries
that can be cached at any given time. Defaults
to 100,000.
| `cache.hash_algo` | no | Specifies the hashing algorithm that is used
for the cached user credentials. See
<<cache-hash-algo, Cache hash algorithms>>
for the possible values. (Expert Setting)
|=======================
[[managing-native-users]]
==== Managing Native Users
You manage users in the `native` realm through the <<security-api-users, user API>>.
[[migrating-from-file]]
NOTE: To migrate file-based users to the `native` realm, use the
<<migrate-tool, migrate>> tool.
[float]
[[native-add]]
===== Adding Users
To add a user, submit a PUT or POST request to the `/_xpack/security/user/<username>`
endpoint. A username must be at least 1 character long and no longer than 30
characters. The first character must be a letter (`a-z` or `A-Z`) or an
underscore (`_`). Subsequent characters can be letters, underscores (`_`),
digits (`0-9`), or any of the following symbols `@`, `-`, `.` or `$`.
[source,js]
--------------------------------------------------
POST /_xpack/security/user/jacknich
{
"password" : "j@rV1s", <1>
"roles" : [ "admin", "other_role1" ], <2>
"full_name" : "Jack Nicholson", <3>
"email" : "jacknich@example.com", <4>
"metadata" : { <5>
"intelligence" : 7
},
"enabled": true <6>
}
--------------------------------------------------
// CONSOLE
<1> You must specify a password when adding a user. Passwords must be at least 6
characters long.
<2> You must assign at least one role to the user. The roles determine the user's
access permissions.
<3> The user's full name. Optional.
<4> The user's email address. Optional.
<5> Arbitrary metadata you want to associate with the user. Optional.
<6> Specifies whether the user should be enabled. Optional with a default of true.
[float]
[[native-list]]
===== Retrieving Users
To retrieve all users, submit a GET request to the `/_xpack/security/user` endpoint:
[source,js]
--------------------------------------------------
GET /_xpack/security/user
--------------------------------------------------
// CONSOLE
// TEST[continued]
To retrieve particular users, specify the users as a comma-separated list:
[source,js]
--------------------------------------------------
GET /_xpack/security/user/jacknich,rdeniro
--------------------------------------------------
// CONSOLE
// TEST[continued]
An object is returned holding the found users, each keyed by the relevant
username. Note that user passwords are not included.
[source,js]
--------------------------------------------------
{
"jacknich" : {
"username": "jacknich",
"roles" : [ "admin", "other_role1" ],
"full_name" : "Jack Nicholson",
"email" : "jacknich@example.com",
"enabled" : true,
"metadata" : {
"intelligence" : 7
}
}
}
--------------------------------------------------
// TESTRESPONSE
[float]
[[native-delete]]
===== Deleting Users
To delete a user, submit a DELETE request to the `/_xpack/security/user/<username>`
endpoint:
[source,js]
--------------------------------------------------
DELETE /_xpack/security/user/jacknich
--------------------------------------------------
// CONSOLE
// TEST[continued]
If the user is successfully deleted, the request returns `{"found": true}`.
Otherwise, `found` is set to false.
[source,js]
--------------------------------------------------
{
"found" : true
}
--------------------------------------------------
// TESTRESPONSE

132
docs/en/security/authentication/pki-realm.asciidoc Normal file
View File

@ -0,0 +1,132 @@
[[pki-realm]]
=== PKI User Authentication
You can configure {security} to use Public Key Infrastructure (PKI) certificates
to authenticate users. This requires clients to present X.509 certificates. To
use PKI, you configure a PKI realm, enable client authentication on the desired
network layers (transport or http), and map the Distinguished Names (DNs) from
the user certificates to {security} roles in the <<mapping-roles, role mapping file>>.
You can also use a combination of PKI and username/password authentication. For
example, you can enable SSL/TLS on the transport layer and define a PKI realm to
require transport clients to authenticate with X.509 certificates, while still
authenticating HTTP traffic using username and password credentials. You can also set
`xpack.security.transport.ssl.client_authentication` to `optional` to allow clients without
certificates to authenticate with other credentials.
IMPORTANT: You must enable SSL/TLS and enabled client authentication to use PKI.
For more information, see <<ssl-tls, Setting Up SSL/TLS on a Cluster>>.
==== PKI Realm Configuration
Like other realms, you configure options for a `pki` realm under the
`xpack.security.authc.realms` namespace in `elasticsearch.yml`.
To configure `pki` realm:
. Add a realm configuration of type `pki` to `elasticsearch.yml` under the
`xpack.security.authc.realms` namespace. At a minimum, you must set the realm `type` to
`pki`. If you are configuring multiple realms, you should also explicitly set
the `order` attribute. See <<pki-settings>> for all of the options you can set
for a `pki` realm.
+
For example, the following snippet shows the most basic `pki` realm configuration:
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
pki1:
type: pki
------------------------------------------------------------
+
With this configuration, any certificate trusted by the SSL/TLS layer is accepted
for authentication. The username is the common name (CN) extracted from the DN
of the certificate.
+
IMPORTANT: When you configure realms in `elasticsearch.yml`, only the
realms you specify are used for authentication. If you also want to use the
`native` or `file` realms, you must include them in the realm chain.
+
If you want to use something other than the CN of the DN as the username, you
can specify a regex to extract the desired username. For example, the regex in
the following configuration extracts the email address from the DN:
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
pki1:
type: pki
username_pattern: "EMAILADDRESS=(.*?)(?:,|$)"
------------------------------------------------------------
+
You can also specify which truststore to use for authentication. This is useful
when the SSL/TLS layer trusts clients with certificates that are signed by a
different CA than the one that signs your users' certificates. To specify the
location of the truststore, specify the `truststore.path` option:
+
[source, yaml]
------------------------------------------------------------
xpack:
security:
authc:
realms:
pki1:
type: pki
truststore:
path: "/path/to/pki_truststore.jks"
password: "changeme"
------------------------------------------------------------
. Restart Elasticsearch.
[[pki-settings]]
===== PKI Realm Settings
[cols="4,^3,10"]
|=======================
| Setting | Required | Description
| `type` | yes | Indicates the realm type. Must be set to `pki`.
| `order` | no | Indicates the priority of this realm within the realm
chain. Realms with a lower order are consulted first.
Although not required, we recommend explicitly
setting this value when you configure multiple realms.
Defaults to `Integer.MAX_VALUE`.
| `enabled` | no | Indicates whether this realm is enabled or disabled.
Enables you to disable a realm without removing its
configuration. Defaults to `true`.
| `username_pattern` | no | Specifies the regular expression pattern used to extract
the username from the certificate DN. The first match
group is used as the username. Defaults to `CN=(.*?)(?:,\|$)`.
| `truststore.path` | no | The path to the truststore. Defaults to the path
defined by <<ssl-tls-settings,SSL/TLS settings>>.
| `truststore.password` | no/yes | Specifies the password for the truststore. Must be
provided if `truststore.path` is set.
| `truststore.algorithm` | no | Specifies the algorithm used for the truststore.
Defaults to `SunX509`.
| `files.role_mapping` | no | Specifies the <<security-files-location,location>>
for the <<pki-role-mapping, YAML role mapping configuration file>>.
Defaults to `CONFIG_DIR/x-pack/role_mapping.yml`.
|=======================
[[assigning-roles-pki]]
==== Mapping Roles for PKI Users
You map roles for PKI users in the role mapping file stored on each node. You
identify a user by the distinguished name in their certificate. For example, the
following mapping configuration maps `John Doe` to the `user` role:
[source, yaml]
------------------------------------------------------------
user: <1>
- "cn=John Doe,ou=example,o=com" <2>
------------------------------------------------------------
<1> The name of a role.
<2> The distinguished name (DN) of a PKI user.
For more information, see <<mapping-roles, Mapping Users and Groups to Roles>>.

60
docs/en/security/authentication/user-cache.asciidoc Normal file
View File

@ -0,0 +1,60 @@
[[controlling-user-cache]]
=== Controlling the User Cache
User credentials are cached in memory on each node to avoid connecting to a
remote authentication service or hitting the disk for every incoming request.
You can configure characteristics of the user cache with the `cache.ttl`,
`cache.max_users`, and `cache.hash_algo` realm settings.
NOTE: PKI realms do not use the user cache.
The cached user credentials are hashed in memory. By default, {security} uses a
salted `sha-256` hash algorithm. You can use a different hashing algorithm by
setting the `cache_hash_algo` setting to any of the following:
[[cache-hash-algo]]
.Cache hash algorithms
|=======================
| Algorithm | | | Description
| `ssha256` | | | Uses a salted `sha-256` algorithm (default).
| `md5` | | | Uses `MD5` algorithm.
| `sha1` | | | Uses `SHA1` algorithm.
| `bcrypt` | | | Uses `bcrypt` algorithm with salt generated in 10 rounds.
| `bcrypt4` | | | Uses `bcrypt` algorithm with salt generated in 4 rounds.
| `bcrypt5` | | | Uses `bcrypt` algorithm with salt generated in 5 rounds.
| `bcrypt6` | | | Uses `bcrypt` algorithm with salt generated in 6 rounds.
| `bcrypt7` | | | Uses `bcrypt` algorithm with salt generated in 7 rounds.
| `bcrypt8` | | | Uses `bcrypt` algorithm with salt generated in 8 rounds.
| `bcrypt9` | | | Uses `bcrypt` algorithm with salt generated in 9 rounds.
| `noop`,`clear_text` | | | Doesn't hash the credentials and keeps it in clear text in
memory. CAUTION: keeping clear text is considered insecure
and can be compromised at the OS level (for example through
memory dumps and using `ptrace`).
|=======================
[[cache-eviction-api]]
==== Evicting Users from the Cache
{security} exposes a <<security-api-clear-cache, Clear Cache API>> you can use
to force the eviction of cached users. For example, the following request evicts
all users from the `ad1` realm:
[source, js]
------------------------------------------------------------
$ curl -XPOST 'http://localhost:9200/_xpack/security/realm/ad1/_clear_cache'
------------------------------------------------------------
To clear the cache for multiple realms, specify the realms as a comma-separated
list:
[source, js]
------------------------------------------------------------
$ curl -XPOST 'http://localhost:9200/_xpack/security/realm/ad1,ad2/_clear_cache'
------------------------------------------------------------
You can also evict specific users:
[source, java]
------------------------------------------------------------
$ curl -XPOST 'http://localhost:9200/_xpack/security/realm/ad1/_clear_cache?usernames=rdeniro,alpacino'
------------------------------------------------------------

394
docs/en/security/authorization.asciidoc Normal file
View File

@ -0,0 +1,394 @@
[[authorization]]
== Configuring Role-based Access Control
{security} introduces the concept of _authorization_ to Elasticsearch.
Authorization is the process of determining whether the user behind an incoming
request is allowed to execute it. This process takes place once a request is
successfully authenticated and the user behind the request is identified.
[[roles]]
[float]
=== Roles, Permissions and Privileges
The authorization process revolves around the following 5 constructs:
_Secured Resource_::
A resource to which access is restricted. Indices/aliases, documents, fields,
users and the Elasticsearch cluster itself are all examples of secured objects.
_Privilege_::
A named group representing one or more actions that a user may execute against a
secured resource. Each secured resource has its own sets of available privileges.
For example, `read` is an index privilege that represents all actions that enable
reading the indexed/stored data. For a complete list of available privileges
see <<security-privileges>>.
_Permissions_::
A set of one or more privileges against a secured resource. Permissions can
easily be described in words, here are few examples:
* `read` privilege on the `products` index
* `manage` privilege on the cluster
* `run_as` privilege on `john` user
* `read` privilege on documents that match query X
* `read` privilege on `credit_card` field
_Role_::
A named sets of permissions
_User_::
The authenticated user.
A secure Elasticsearch cluster manages the privileges of users through _roles_.
A role has a unique name and identifies a set of permissions that translate to
privileges on resources. A user can be associated with an arbitrary number of
roles. The total set of permissions that a user has is therefore defined by
union of the permissions in all its roles.
As an administrator, you will need to define the roles that you want to use,
then assign users to the roles. These can be assigned to users in a number of
ways depending on the realms by which the users are authenticated.
[[built-in-roles]]
=== Built-in Roles
{security} applies a default role to all users, including <<anonymous-access, anonymous users>>. The default role enables users to access the authenticate
endpoint, change their own passwords, and get information about themselves.
{security} also provides a set of built-in roles you can explicitly assign
to users. These roles have a fixed set of privileges and cannot be updated.
[[built-in-roles-superuser]]
`superuser`::
Grants full access to the cluster, including all indices and data. A user with
the `superuser` role can also manage users and roles and <<run-as-privilege, impersonate>> any other user in the system. Due to the permissive nature of
this role, take extra care when assigning it to a user.
[[built-in-roles-transport-client]]
`transport_client`::
Grants the privileges required to access the cluster through the Java Transport Client. The Java Transport Client fetches information about the nodes in the
cluster using the _Node Liveness API_ and the _Cluster State API_ (when
sniffing is enabled). Assign your users this role if they use the
Transport Client.
+
NOTE: Using the Transport Client effectively means the users are granted access
to the cluster state. This means users can view the metadata over all indices,
index templates, mappings, node and basically everything about the cluster.
However, this role does not grant permission to view the data in all indices.
[[built-in-roles-kibana-user]]
`kibana_user` ::
Grants the minimum privileges required for any user of Kibana. This role grants
access to the Kibana indices and grants monitoring privileges for the cluster.
[[built-in-roles-monitoring-user]]
`monitoring_user` ::
Grants the minimum privileges required for any user of Monitoring other than those
required to use Kibana. This role grants access to the monitoring indices. Monitoring
users should also be assigned the `kibana_user` role.
[[built-in-roles-reporting-user]]
`reporting_user` ::
Grants the specific privileges required for users of Reporting other than those
required to use Kibana. This role grants access to the reporting indices. Reporting
users should also be assigned the `kibana_user` role and a role that grants them
access to the data that will be used to generate reports with.
[[built-in-roles-remote-monitoring-agent]]
`remote_monitoring_agent` ::
Grants the minimum privileges required for a remote monitoring agent to write data
into this cluster.
[[built-in-roles-ingest-user]]
`ingest_admin` ::
Grants access to manage *all* index templates and *all* ingest pipeline configurations.
+
NOTE: This role does *not* provide the ability to create indices; those privileges
must be defined in a separate role.
[[built-in-roles-kibana-system]]
`kibana_system` ::
Grants access necessary for the Kibana system user to read from and write to the Kibana indices
and check the availability of the Elasticsearch cluster.
+
NOTE: This role should not be assigned to users as the granted permissions may
change between releases.
[[built-in-roles-logstash-system]]
`logstash_system` ::
Grants access necessary for the Logstash system user to send data to Elasticsearch.
+
NOTE: This role should not be assigned to users as the granted permissions may
change between releases.
[[defining-roles]]
=== Defining Roles
A role is defined by the following JSON structure:
[source,js]
-----
{
"name": "...", <1>
"run_as": [ ... ] <2>
"cluster": [ ... ], <3>
"indices": [ ... ] <4>
}
-----
<1> The role name, also used as the role ID.
<2> A list of usernames the owners of this role can <<run-as-privilege, impersonate>>.
<3> A list of cluster privileges. These privileges define the
cluster level actions users with this role are able to execute. This field
is optional (missing `cluster` privileges effectively mean no cluster level
permissions).
<4> A list of indices permissions entries. This field is optional (missing `indices`
privileges effectively mean no index level permissions).
[[valid-role-name]]
NOTE: A valid role name must be at least 1 character and no longer than 30
characters. It must begin with a letter (`a-z`) or an underscore (`_`).
Subsequent characters can be letters, underscores (`_`), digits (`0-9`) or
any of the following symbols `@`, `-`, `.` or `$`
The following describes the structure of an indices permissions entry:
[source,js]
-------
{
"names": [ ... ], <1>
"privileges": [ ... ], <2>
"field_security" : { ... }, <3>
"query": "..." <4>
}
-------
<1> A list of indices (or index name patterns) to which the permissions in this
entry apply.
<2> The index level privileges the owners of the role have on the associated
indices (those indices that are specified in the `name` field)
<3> Specification for document fields the owners of the role have read access to.
See <<field-and-document-access-control>> for details.
<4> A search query that defines the documents the owners of the role have read
access to. A document within the associated indices must match this query
in order for it to be accessible by the owners of the role.
[TIP]
==============================================================================
When specifying index names, you can use indices and aliases with their full
names or regular expressions that refer to multiple indices.
* Wildcard (default) - simple wildcard matching where `*` is a placeholder
for zero or more characters, `?` is a placeholder for a single character
and `\` may be used as an escape character.
* Regular Expressions - A more powerful syntax for matching more complex
patterns. This regular expression is based on Lucene's regexp automaton
syntax. To enable this syntax, it must be wrapped within a pair of
forward slashes (`/`). Any pattern starting with `/` and not ending with
`/` is considered to be malformed.
.Example Regular Expressions
[source,yaml]
------------------------------------------------------------------------------
"foo-bar": # match the literal `foo-bar`
"foo-*": # match anything beginning with "foo-"
"logstash-201?-*": # ? matches any one character
"/.*-201[0-9]-.*/": # use a regex to match anything containing 2010-2019
"/foo": # syntax error - missing final /
------------------------------------------------------------------------------
==============================================================================
The following snippet shows an example definition of a `clicks_admin` role:
[source,js]
-----------
{
"run_as": [ "clicks_watcher_1" ]
"cluster": [ "monitor" ],
"indices": [
{
"names": [ "events-*" ],
"privileges": [ "read" ],
"field_security" : {
"grant" : [ "category", "@timestamp", "message" ]
},
"query": "{\"match\": {\"category\": \"click\"}}"
}
]
}
-----------
Based on the above definition, users owning the `clicks_admin` role can:
* Impersonate the `clicks_watcher_1` user and execute requests on its behalf.
* Monitor the Elasticsearch cluster
* Read data from all indices prefixed with `events-`
* Within these indices, only read the events of the `click` category
* Within these document, only read the `category`, `@timestamp` and `message`
fields.
TIP: For a complete list of available <<security-privileges, cluster and indices privileges>>
There are two available mechanisms to define roles: using the _Role Management APIs_
or in local files on the Elasticsearch nodes.
[float]
[[roles-management-ui]]
=== Role Management UI
If you are a Kibana user, make sure to <<installing-xpack, install {xpack} in Kibana>>.
This enables you to easily manage users and roles from within Kibana. To manage roles,
log in to Kibana and go to *Management / Elasticsearch / Roles*.
[float]
[[roles-management-api]]
=== Role Management API
The _Role Management APIs_ enable you to add, update, remove and retrieve roles
dynamically. When you use the APIs to manage roles in the `native` realm, the
roles are stored in an internal Elasticsearch index.
[[roles-api-add]]
==== Adding a Role
To add a role, submit a PUT or POST request to the `/_xpack/security/role/<name>`
endpoint.
[source,js]
--------------------------------------------------
POST /_xpack/security/role/clicks_admin
{
"run_as": [ "clicks_watcher_1" ],
"cluster": [ "monitor" ],
"indices": [
{
"names": [ "events-*" ],
"privileges": [ "read" ],
"field_security" : {
"grant" : [ "category", "@timestamp", "message" ]
},
"query": "{\"match\": {\"category\": \"click\"}}"
}
]
}
--------------------------------------------------
// CONSOLE
// TESTSETUP
NOTE: This API can also be used for updating role definitions.
[[roles-api-list]]
==== List Role
To retrieve all roles, submit a GET request to the `/_xpack/security/role` endpoint:
[source,js]
--------------------------------------------------
GET /_xpack/security/role
--------------------------------------------------
// CONSOLE
To retrieve particular roles, specify the roles as a comma-separated list:
[source,js]
--------------------------------------------------
GET /_xpack/security/role/clicks_admin
--------------------------------------------------
// CONSOLE
Response:
[source,js]
--------------------------------------------------
{
"clicks_admin": {
"run_as": [ "clicks_watcher_1" ],
"cluster": [ "monitor" ],
"indices": [
{
"names": [ "events-*" ],
"privileges": [ "read" ],
"field_security" : {
"grant" : [ "category", "@timestamp", "message" ]
},
"query": "{\"match\": {\"category\": \"click\"}}"
}
],
"metadata": { },
"transient_metadata": {
"enabled": true
}
}
}
--------------------------------------------------
// TESTRESPONSE
NOTE: If single role is requested, that role is returned as the response. When
requesting multiple roles, an object is returned holding the found roles,
each keyed by the relevant role name.
[[roles-api-delete]]
==== Deleting a Role
To delete a role, submit a DELETE request to the `/_xpack/security/role/<name>`
endpoint:
[source,js]
--------------------------------------------------
DELETE /_xpack/security/role/clicks_admin
--------------------------------------------------
// CONSOLE
[float]
[[roles-management-file]]
=== File-based Role Management
Apart from the _Role Management APIs_ roles can also be defined in local
`roles.yml` file located in `CONFIG_DIR/x-pack`. This is a YAML file where each
role definition is keyed by its name.
[IMPORTANT]
==============================
If the same role name is used in the `roles.yml` file and through the
_Role Management APIs_, the role found in the file will be used.
==============================
While the _Role Management APIs_ is the preferred mechanism to define roles,
using the `roles.yml` file becomes useful if you want to define fixed roles that
no one (beside an administrator having physical access to the Elasticsearch nodes)
would be able to change.
[IMPORTANT]
==============================
The `roles.yml` file is managed locally by the node and is not globally by the
cluster. This means that with a typical multi-node cluster, the exact same
changes need to be applied on each and every node in the cluster.
A safer approach would be to apply the change on one of the nodes and have the
`roles.yml` distributed/copied to all other nodes in the cluster (either
manually or using a configuration management system such as Puppet or Chef).
==============================
The following snippet shows an example of the `roles.yml` file configuration:
[source,yaml]
-----------------------------------
click_admins:
run_as: [ 'clicks_watcher_1' ]
cluster: [ 'monitor' ]
indices:
- names: [ 'events-*' ]
privileges: [ 'read' ]
field_security:
grant: ['category', '@timestamp', 'message' ]
query: '{"match": {"category": "click"}}'
-----------------------------------
{security} continuously monitors the `roles.yml` file and automatically picks
up and apples any changes to it.
include::authorization/alias-privileges.asciidoc[]
include::authorization/mapping-roles.asciidoc[]
include::authorization/field-and-document-access-control.asciidoc[]
include::authorization/run-as-privilege.asciidoc[]

101
docs/en/security/authorization/alias-privileges.asciidoc Normal file
View File

@ -0,0 +1,101 @@
[[securing-aliases]]
=== Granting Privileges for Indices & Aliases
Elasticsearch allows to execute operations against {ref}/indices-aliases.html[index aliases],
which are effectively virtual indices. An alias points to one or more indices,
holds metadata and potentially a filter. {security} treats aliases and indices
the same. Privileges for indices actions are granted on specific indices or
aliases. In order for an indices action to be authorized, the user that executes
it needs to have permissions for that action on all the specific indices or
aliases that the request relates to.
Let's look at an example. Assuming we have an index called `2015`, an alias that
points to it called `current_year`, and a user with the following role:
[source,js]
--------------------------------------------------
{
"names" : [ "2015" ],
"privileges" : [ "read" ]
}
--------------------------------------------------
// NOTCONSOLE
The user attempts to retrieve a document from `current_year`:
[source,shell]
-------------------------------------------------------------------------------
GET /current_year/event/1
-------------------------------------------------------------------------------
// CONSOLE
// TEST[s/^/PUT 2015\n{"aliases": {"current_year": {}}}\nPUT 2015\/event\/1\n{}\n/]
The above request gets rejected, although the user has `read` privilege on the
concrete index that the `current_year` alias points to. The correct permission
would be as follows:
[source,js]
--------------------------------------------------
{
"names" : [ "current_year" ],
"privileges" : [ "read" ]
}
--------------------------------------------------
// NOTCONSOLE
[float]
==== Managing aliases
Unlike creating indices, which requires the `create_index` privilege, adding,
removing and retrieving aliases requires the `manage` permission. Aliases can be
added to an index directly as part of the index creation:
[source,shell]
-------------------------------------------------------------------------------
PUT /2015
{
"aliases" : {
"current_year" : {}
}
}
-------------------------------------------------------------------------------
// CONSOLE
or via the dedicated aliases api if the index already exists:
[source,shell]
-------------------------------------------------------------------------------
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "2015", "alias" : "current_year" } }
]
}
-------------------------------------------------------------------------------
// CONSOLE
// TEST[s/^/PUT 2015\n/]
The above requests both require the `manage` privilege on the alias name as well
as the targeted index, as follows:
[source,js]
--------------------------------------------------
{
"names" : [ "20*", "current_year" ],
"privileges" : [ "manage" ]
}
--------------------------------------------------
// NOTCONSOLE
The index aliases api also allows also to delete aliases from existing indices.
The privileges required for such a request are the same as above. Both index and
alias need the `manage` permission.
[float]
==== Filtered aliases
Aliases can hold a filter, which allows to select a subset of documents that can
be accessed out of all the documents that the physical index contains. These
filters are not always applied and should not be used in place of
<<document-level-security, document level security>>.

443
docs/en/security/authorization/field-and-document-access-control.asciidoc Normal file
View File

@ -0,0 +1,443 @@
[[field-and-document-access-control]]
=== Setting Up Field and Document Level Security
You can control access to data within an index by adding field and document level
security permissions to a role. Field level security permissions restrict access
to particular fields within a document. Document level security permissions
restrict access to particular documents within an index.
NOTE: Document and field level security is currently meant to operate with
read-only privileged accounts. Users with document and field level
security enabled for an index should not perform write operations.
A role can define both field and document level permissions on a per-index basis.
A role that doesn’t specify field level permissions grants access to ALL fields.
Similarly, a role that doesn't specify document level permissions grants access
to ALL documents in the index.
[IMPORTANT]
=====================================================================
When assigning users multiple roles, be careful that you don't inadvertently
grant wider access than intended. Each user has a single set of field level and
document level permissions per index. When you assign a user multiple roles,
the permissions are ORed together. This means if you assign one role that
restricts access to particular fields in an index, and another that doesn't
specify any field level access restrictions for that index, the user will have
access to all fields. The same is true for document level permissions.
For example, let's say `role_a` only grants access to the `address`
field of the documents in `index1`, but doesn't specify any document
restrictions. Conversely, `role_b` limits access to a subset of the documents
in `index1`, but doesn't specify any field restrictions. If you assign a user
both roles, `role_a` gives the user access to all documents and `role_b` gives
the user access to all fields.
If you need to restrict access to both documents and fields, consider splitting
documents by index instead.
=====================================================================
[[field-level-security]]
==== Field Level Security
To enable field level security, you specify the fields that each role can access
as part of the indices permissions in a role definition. This binds field level
security to a well defined set of indices (and potentially a set of
<<document-level-security, documents>>).
The following role definition grants read access only to the `category`,
`@timestamp`, and `message` fields in all the `events-*` indices.
[source,js]
--------------------------------------------------
{
"indices": [
{
"names": [ "events-*" ],
"privileges": [ "read" ],
"field_security" : {
"grant" : [ "category", "@timestamp", "message" ]
}
}
]
}
--------------------------------------------------
To allow access to the `_all` meta field, you must explicitly list it as an
allowed field. Access to the following meta fields is always allowed: `_id`,
`_type`, `_parent`, `_routing`, `_timestamp`, `_ttl`, `_size` and `_index`. If
you specify an empty list of fields, only these meta fields are accessible.
NOTE: Omitting the fields entry entirely disables field-level security.
You can also specify field expressions. For example, the following
example grants read access to all fields starting with `event_` prefix:
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"grant" : [ "event_*" ]
}
}
]
}
--------------------------------------------------
Use the dot notations to refer to nested fields in more complex documents. For
example, assuming the following document:
[source,js]
--------------------------------------------------
{
"customer": {
"handle": "Jim",
"email": "jim@mycompany.com",
"phone": "555-555-5555"
}
}
--------------------------------------------------
The following role definition only allows access to the customer `handle` field:
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"grant" : [ "customer.handle" ]
}
}
]
}
--------------------------------------------------
This is where wildcard support shines. For example, use `customer.*` to only
enable read access to the `customer` data:
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"grant" : [ "customer.*" ]
}
}
]
}
--------------------------------------------------
Similar to granting field permissions the permission to access fields can be denied with the following syntax:
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"grant" : [ "*"],
"except": [ "customer.handle" ]
}
}
]
}
--------------------------------------------------
The following rules apply:
Absence of "field_security" in a role is equivalent to * access.
Denied fields may only be provided if permission has been granted explicitly to other fields. The exceptions given must be a subset of the
fields that permissions have been granted to.
Denied and granted fields defined implies access to all granted fields except those which match the pattern in denied fields. Example:
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"except": [ "customer.handle" ],
"grant" : [ "customer.*" ]
}
}
]
}
--------------------------------------------------
In the above example all fields with the prefix "customer." are allowed except for "customer.handle".
An empty array for grant (eg. "grant" : []) means that no fields are granted access to.
===== Field Level Security and Roles
When a user has several roles that specify field level permissions then the resulting field level permissions per index are the union
of the individual role permissions.
For example if these two roles are merged:
[source,js]
--------------------------------------------------
{
// role 1
...
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"grant": [ "a.*" ],
"except" : [ "a.b*" ]
}
}
]
}
{
// role 2
...
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"grant": [ "a.b*" ],
"except" : [ "a.b.c*" ]
}
}
]
}
--------------------------------------------------
Then the resulting permission would be equal to:
[source,js]
--------------------------------------------------
{
// role 1 + role 2
...
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"field_security" : {
"grant": [ "a.*" ],
"except" : [ "a.b.c*" ]
}
}
]
}
--------------------------------------------------
[[document-level-security]]
==== Document Level Security
Document level security restricts the documents that users have read access to.
To enable document level security, you specify a query that matches all the
accessible documents as part of the indices permissions within a role definition.
This binds document level security to a well defined set of indices.
Enabling document level security restricts which documents can be accessed from any document based read API.
To enable document level security, you use a query to specify the documents that each role can access in the `roles.yml` file.
You specify the document query with the `query` option. The document query is associated with a particular index or index pattern and
operates in conjunction with the privileges specified for the indices.
The following role definition grants read access only to documents that
belong to the `click` category within all the `events-*` indices.
[source,js]
--------------------------------------------------
{
"indices": [
{
"names": [ "events-*" ],
"privileges": [ "read" ],
"query": "{\"match\": {\"category\": \"click\"}}"
}
]
}
--------------------------------------------------
NOTE: Omitting the `query` entry entirely disables document level security for
the respective indices permission entry.
The specified `query` expects the same format as if it was defined in the
search request and supports ELasticsearch's full {ref}/query-dsl.html[Query DSL].
For example, the following role grants read access to all indices, but restricts
access to documents whose `department_id` equals `12`.
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "*" ],
"privileges" : [ "read" ],
"query" : {
"term" : { "department_id" : 12 }
}
}
]
}
--------------------------------------------------
NOTE: `query` also accepts queries written as string values
[[templating-role-query]]
===== Templating a Role Query
You can use Mustache templates in a role query to insert the username of the
current authenticated user into the role. Like other places in Elasticsearch
that support templating or scripting, you can specify inline, stored,
or file based templates and define custom parameters. You access the current
authenticated user's details through the `_user` parameter.
For example, the following role query uses a template to insert the username
of the current authenticated user:
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "my_index" ],
"privileges" : [ "read" ],
"query" : {
"template" : {
"inline" : {
"term" : { "acl.username" : "{{_user.username}}" }
}
}
}
}
]
}
--------------------------------------------------
You can access the following information through the `_user` variable:
[options="header"]
|======
| Property | Description
| `_user.username` | The username of the current authenticated user.
| `_user.full_name` | If specified, the full name of the current authenticated user.
| `_user.email` | If specified, the email of the current authenticated user.
| `_user.roles` | If associated, a list of the role names of the current authenticated user.
| `_user.metadata` | If specified, a hash holding custom metadata of the current authenticated user.
|======
You can also access custom user metadata. For example, if you maintain a
`group_id` in your user metadata, you can apply document level security
based on the `group.id` field in your documents:
[source,js]
--------------------------------------------------
{
"indices" : [
{
"names" : [ "my_index" ],
"privileges" : [ "read" ],
"query" : {
"template" : {
"inline" : {
"term" : { "group.id" : "{{_user.metadata.group_id}}" }
}
}
}
}
]
}
--------------------------------------------------
[[set-security-user-processor]]
===== Set Security User Ingest Processor
If an index is being shared by many small users it makes sense put all these users into the same index as having a
dedicated index or shard per user is too wasteful. In order to guarantee that a user only read its own documents it
makes sense to set up document level security. In order to use document level security for this each document must have
the username or role name associated with it, so that it can be queried by the document level security's role query.
This is where the `set_security_user` ingest processor can help.
NOTE: You need to make sure to use unique ids for each user that uses the same index, because document level security
doesn't apply on write APIs and you can overwrite other users' documents. This ingest processor just adds
properties of the current authenticated user to the documents being indexed.
The `set_security_user` processor attaches user related details (`username`, `roles`, `email`, `full_name` and `metadata` )
from the current authenticated user to the current document by pre-processed by ingest.
So when indexing data with an ingest pipeline then user details get automatically attached with the document:
[source,js]
--------------------------------------------------
PUT shared-logs/log/1?pipeline=my_pipeline_id
{
...
}
--------------------------------------------------
Read the {ref}/ingest.html[ingest docs] for more information
about setting up a pipeline and other processors.
[[set-security-user-options]]
.Set Security User Options
[options="header"]
|======
| Name | Required | Default | Description
| `field` | yes | - | The field to store the user information into.
| `properties` | no | [`username`, `roles`, `email`, `full_name`, `metadata`] | Controls what user related properties are added to the `field`.
|======
Example config that adds all user details of the current authenticated user to the `user` field to all documents being
processed by this pipeline:
[source,js]
--------------------------------------------------
{
"processors" : [
{
"set_security_user": {
"field": "user"
}
}
]
}
--------------------------------------------------
[[multiple-roles-dls-fls]]
==== Multiple Roles with Document and Field Level Security
A user can have many roles and each role can define different permissions on the
same index. It is important to understand the behavior of Document and Field Level
security in this scenario.
Document level security will take into account each role held by the user, and
combine each document level security query for a given index with an "OR". This
means that only one of the role queries must match for a document to be returned.
For example, if a role grants access to an index without document level security
and another grants access with document level security, document level security
will not be applied; the user with both roles will have access to all of the
documents in the index.
Field level security will take into account each role the user has and combine
all of the fields listed into a single set for each index. For example, if a
role grants access to an index without field level security and another grants
access with field level security, field level security will not be applied for
that index; the user with both roles will have access to all of the fields in
in the index.

67
docs/en/security/authorization/mapping-roles.asciidoc Normal file
View File

@ -0,0 +1,67 @@
[[mapping-roles]]
=== Mapping Users and Groups to Roles
If you authenticate users with the `native` or `file` realms, you can manage
role assignment user the <<managing-native-users, User Management APIs>> or the
<<managing-file-users, file-realm>> command-line tool respectively.
For other types of realms, you configure role mappings for users and groups in a
YAML file and copy it to each node in the cluster. Tools like Puppet or Chef can
help with this.
By default, role mappings are stored in `CONF_DIR/x-pack/role_mapping.yml`, where
`CONF_DIR` is `ES_HOME/config` (zip/tar installations) or `/etc/elasticsearch`
(package installations). To specify a different location, you configure the
`files.role_mapping` realm settings in `elasticsearch.yml`. This setting enable
you to use a different set of mappings for each realm type:
|=====
| `xpack.security.authc.ldap.files.role_mapping` | | | The location of the role mappings for LDAP realms.
| `xpack.security.authc.active_directory.files.role_mapping` | | | The location of the role mappings for Active Directory realms.
| `xpack.security.authc.pki.files.role_mapping` | | | The location of the role mappings for PKI realms.
|=====
Within the role mapping file, the security roles are keys and groups and users
are values. The mappings can have a many-to-many relationship. When you map roles
to groups, the roles of a user in that group are the combination of the roles
assigned to that group and the roles assigned to that user.
[[ad-role-mapping]]
The available roles are either added using the <<roles-management-api, Role Management APIs>>
or defined in the <<roles-management-file, roles file>>. To specify users and
groups in the role mappings, you use their _Distinguished Names_ (DNs). A DN is
a string that uniquely identifies the user or group, for example
`"cn=John Doe,cn=contractors,dc=example,dc=com"`.
NOTE: {security} only supports Active Directory security groups. You cannot map
distribution groups to roles.
[[ldap-role-mapping]]
For example, the following snippet maps the `admins` group to the `monitoring`
role and maps the `John Doe` user, the `users` group, and the `admins` group to
the `user` role.
[source, yaml]
------------------------------------------------------------
monitoring: <1>
- "cn=admins,dc=example,dc=com" <2>
user:
- "cn=John Doe,cn=contractors,dc=example,dc=com" <3>
- "cn=users,dc=example,dc=com"
- "cn=admins,dc=example,dc=com"
------------------------------------------------------------
<1> The name of a {security} role.
<2> The distinguished name of an LDAP group or an Active Directory security group.
<3> The distinguished name of an LDAP or Active Directory user.
[[pki-role-mapping]]
PKI realms only support mapping users to roles, as there is no notion of a group
in PKI. For example:
[source, yaml]
------------------------------------------------------------
monitoring:
- "cn=Admin,ou=example,o=com"
user:
- "cn=John Doe,ou=example,o=com"
------------------------------------------------------------

31
docs/en/security/authorization/run-as-privilege.asciidoc Normal file
View File

@ -0,0 +1,31 @@
[[run-as-privilege]]
=== Submitting Requests on Behalf of Other Users
{security} supports a permission that enables an authenticated user to submit
requests on behalf of other users. If your application already authenticates
users, you can use the _run as_ mechanism to restrict data access according to
{security} permissions without having to re-authenticate each user through.
To "run as" (impersonate) another user, you must be able to retrieve the user from
the realm you use to authenticate. Both the internal `native` and `file` realms
support this out of the box. The LDAP realm however must be configured to enable
user search. For more information, see <<ldap-user-search, Configuring an LDAP Realm with User Search>>.
To submit requests on behalf of other users, you need to have the `run_as`
permission. For example, the following role grants permission to submit request
on behalf of `jacknich` or `redeniro`:
[source,js]
---------------------------------------------------
{
"run_as" : [ "jacknich", "rdeniro" ]
}
---------------------------------------------------
To submit a request as another user, you specify the user in the
`es-security-runas-user` request header. For example:
[source,shell]
---------------------------------------------------
curl -H "es-security-runas-user: jacknich" -u es_admin -XGET 'http://localhost:9200/'
---------------------------------------------------

118
docs/en/security/getting-started.asciidoc Normal file
View File

@ -0,0 +1,118 @@
[[security-getting-started]]
== Getting Started with Security
To secure a cluster, you must install {xpack} on every node in the
cluster. Basic authentication is enabled by default--to communicate
with the cluster, you must specify a username and password.
Unless you <<anonymous-access, enable anonymous access>>, all
requests that don't include a user name and password are rejected.
{security} provides a built-in `elastic` superuser you can use
to start setting things up. This `elastic` user has full access
to the cluster, including all indices and data, so make sure
you change the default password and protect the `elastic` user
credentials accordingly.
To get started with {security}:
. <<installing-xpack, Install X-Pack>> and start Elasticsearch and Kibana.
. Change the passwords of the built in `kibana`, `logstash_system` and `elastic` users:
+
[source,shell]
----------------------------------------------------------
curl -XPUT -u elastic 'localhost:9200/_xpack/security/user/elastic/_password' -H "Content-Type: application/json" -d '{
"password" : "elasticpassword"
}'
curl -XPUT -u elastic 'localhost:9200/_xpack/security/user/kibana/_password' -H "Content-Type: application/json" -d '{
"password" : "kibanapassword"
}'
curl -XPUT -u elastic 'localhost:9200/_xpack/security/user/logstash_system/_password' -H "Content-Type: application/json" -d '{
"password" : "logstashpassword"
}'
----------------------------------------------------------
// NOTCONSOLE
+
NOTE: The default password for the `elastic` user is `changeme`.
. Set up roles and users to control access to Elasticsearch and Kibana.
For example, to grant _John Doe_ full access to all indices that match
the pattern `events*` and enable him to create visualizations and dashboards
for those indices in Kibana, you could create an `events_admin` role and
and assign the role to a new `johndoe` user.
+
[source,shell]
----------------------------------------------------------
curl -XPOST -u elastic 'localhost:9200/_xpack/security/role/events_admin' -H "Content-Type: application/json" -d '{
"indices" : [
{
"names" : [ "events*" ],
"privileges" : [ "all" ]
},
{
"names" : [ ".kibana*" ],
"privileges" : [ "manage", "read", "index" ]
}
]
}'
curl -XPOST -u elastic 'localhost:9200/_xpack/security/user/johndoe' -H "Content-Type: application/json" -d '{
"password" : "userpassword",
"full_name" : "John Doe",
"email" : "john.doe@anony.mous",
"roles" : [ "events_admin" ]
}'
----------------------------------------------------------
// NOTCONSOLE
[[enable-message-authentication]]
. Enable message authentication to verify that messages are not tampered with or corrupted in transit:
.. Run the `syskeygen` tool from `ES_HOME` without any options:
+
[source, shell]
----------------
bin/x-pack/syskeygen
----------------
+
This creates a system key file in `CONFIG_DIR/x-pack/system_key`.
.. Copy the generated system key to the rest of the nodes in the cluster.
+
IMPORTANT: The system key is a symmetric key, so the same key must be on every
node in the cluster.
[[enable-auditing]]
. Enable Auditing to keep track of attempted and successful interactions with
your Elasticsearch cluster:
+
--
.. Add the following setting to `elasticsearch.yml` on all nodes in your cluster:
+
[source,yaml]
----------------------------
xpack.security.audit.enabled: true
----------------------------
.. Restart Elasticsearch.
By default, events are logged to a dedicated `elasticsearch-access.log` file in
`ES_HOME/logs`. You can also store the events in an Elasticsearch index for
easier analysis and control what events are logged. For more information, see
<<auditing, Configuring Auditing>>.
--
[[moving-on]]
IMPORTANT: Once you get these basic security measures in place, we strongly
recommend that you secure communications to and from nodes by
configuring your cluster to use <<ssl-tls, SSL/TLS encryption>>.
Nodes that do not have encryption enabled send passwords in plain
text!
Depending on your security requirements, you might also want to:
* Integrate with <<ldap-realm, LDAP>> or <<active-directory-realm, Active Directory>>,
or <<pki-realm, require certificates>> for authentication.
* Use <<ip-filtering, IP Filtering>> to allow or deny requests from particular
IP addresses or address ranges.

145
docs/en/security/how-security-works.asciidoc Normal file
View File

@ -0,0 +1,145 @@
[[how-security-works]]
== How Security Works
An Elasticsearch cluster is typically made out of many moving parts. There are
the Elasticsearch nodes that form the cluster, and often Logstash instances,
Kibana instances, Beats agents an clients, all communicating with the it.
It should not come as a surprise that securing such clusters has many facets and
layers.
{security} provides the means to secure the Elastic cluster on several levels:
* User authentication
* Authorization and Role Based Access Control (a.k.a RBAC)
* Node/Client Authentication and Channel Encryption
* Auditing
[float]
=== User Authentication
User authentication is the process of identifying the users behind the requests
that hit the cluster and verifying that indeed they are who they claim to be. The
authentication process is handled by one or more authentication services called
_realms_. {security} provides the following built-in realms:
|======
| `native` | | | An internal realm where users are stored in a dedicated
Elasticsearch index. With this realm, users are
authenticated by usernames and passwords. The users
are managed via the <<security-api-users,User Management API>>.
| `ldap` | | | A realm that uses an external LDAP server to authenticate
the users. With this realm, users are authenticated by
usernames and passwords.
| `active_directory` | | | A realm that uses an external Active Directory Server
to authenticate the users. With this realm, users
are authenticated by usernames and passwords.
| `pki` | | | A realm that authenticates users using Public Key
Infrastructure (PKI). This realm works in conjunction
with SSL/TLS and identifies the users through the
Distinguished Name (DN) of the client's X.509
certificates.
| `file` | | | An internal realm where users are defined in files
stored on each node in the Elasticsearch cluster.
With this realm, users are authenticated by usernames
and passwords. The users are managed via
<<managing-file-users,dedicated tools>> that are
provided by {xpack} on installation.
|======
If none of the built-in realms meets your needs, you can also build your own
custom realm and plug it into {xpack}.
When {security} is enabled, depending on the realms you've configured, you will
need to attach your user credentials to the requests sent to Elasticsearch. For
example, when using realms that support usernames and passwords you can simply
attach {wikipedia}/Basic_access_authentication[basic auth] header to the requests.
For more information on user authentication see <<setting-up-authentication>>
[float]
=== Authorization
The authorization process takes place once a request is authenticated and the
User behind the request is identified. Authorization is the process of determining
whether the user behind an incoming request is allowed to execute it. Naturally,
this process takes place right after an successful authentication - when the
user identity is known.
The authorization process revolves around the following 5 constructs:
_Secured Resource_::
A resource to which access is restricted. Indices/aliases, documents, fields,
users and the Elasticsearch cluster itself are all examples of secured objects.
_Privilege_::
A named group representing one or more actions that a user may execute against a
secured resource. Each secured resource has its own sets of available privileges.
For example, `read` is an index privilege that represents all actions that enable
reading the indexed/stored data. For a complete list of available privileges
see <<security-privileges>>.
_Permissions_::
A set of one or more privileges against a secured resource. Permissions can
easily be described in words, here are few examples:
* `read` privilege on the `products` index
* `manage` privilege on the cluster
* `run_as` privilege on `john` user
* `read` privilege on documents that match query X
* `read` privilege on `credit_card` field
_Role_::
A named sets of permissions
_User_::
The authenticated user.
A secure Elasticsearch cluster manages the privileges of users through _roles_.
A role has a unique name and identifies a set of permissions that translate to
privileges on resources. A user can be associated with an arbitrary number of
roles. The total set of permissions that a user has is therefore defined by
union of the permissions in all its roles.
Roles can be assigned to users in a number of ways depending on the realms by
which the users are authenticated.
For more information on user authentication see <<authorization>>
[float]
=== Node/Client Authentication and Channel Encryption
{security} supports configuring SSL/TLS for securing the communication channels
to, from and within the cluster. This support accounts for:
* Encryption of data transmitted over the wires
* Certificate based node authentication - preventing unauthorized nodes/clients
from establishing a connection with the cluster.
For more information, see <<encrypting-communications, Encrypting Communications>>.
{security} also enables you to <<ip-filtering, configure IP Filters>> which can
be seen as a light mechanism for node/client authentication. With IP Filtering
you can restrict the nodes and clients that can connect to the cluster based
on their IP addresses. The IP filters configuration provides whitelisting
and blacklisting of IPs, subnets and DNS domains.
[float]
=== Auditing
When dealing with any secure system, it is critical to have a audit trail
mechanism set in place. Audit trails log various activities/events that occur in
the system, enabling you to analyze and back track past events when things go
wrong (e.g. security breach).
{security} provides such audit trail functionality for all nodes in the cluster.
You can configure the audit level which accounts for the type of events that are
logged. These events include failed authentication attempts, user access denied,
node connection denied, and more.
For more information on auditing see <<auditing>>.

118
docs/en/security/index.asciidoc Normal file
View File

@ -0,0 +1,118 @@
[[xpack-security]]
= Securing Elasticsearch and Kibana
:imagesdir: images/security
[partintro]
--
{security} enables you to easily secure a cluster. With Security,
you can password-protect your data as well as implement more advanced security
measures such as encrypting communications, role-based access control,
IP filtering, and auditing. This guide describes how to configure the security
features you need, and interact with your secured cluster.
Security protects Elasticsearch clusters by:
* <<preventing-unauthorized-access, Preventing unauthorized access>>
with password protection, role-based access control, and IP filtering.
* <<preserving-data-integrity, Preserving the integrity of your data>>
with message authentication and SSL/TLS encryption.
* <<maintaining-audit-trail, Maintaining an audit trail>>
so you know who's doing what to your cluster and the data it stores.
[float]
[[preventing-unauthorized-access]]
=== Preventing Unauthorized Access
To prevent unauthorized access to your Elasticsearch cluster, you must have a
way to _authenticate_ users. This simply means that you need a way to validate
that a user is who they claim to be. For example, you have to make sure only
the person named _Kelsey Andorra_ can sign in as the user `kandorra`. X-Pack
Security provides a standalone authentication mechanism that enables you to
quickly password-protect your cluster. If you're already using <<ldap-realm, LDAP>>,
<<active-directory-realm, Active Directory>>, or <<pki-realm, PKI>> to manage
users in your organization, {security} is able to integrate with those
systems to perform user authentication.
In many cases, simply authenticating users isn't enough. You also need a way to
control what data users have access to and what tasks they can perform. {security}
enables you to _authorize_ users by assigning access _privileges_ to _roles_,
and assigning those roles to users. For example, this
<<authorization,role-based access control>> mechanism (a.k.a RBAC) enables
you to specify that the user `kandorra` can only perform read operations on the
`events` index and can't do anything at all with other indices.
{security} also supports <<ip-filtering, IP-based authorization>>. You can
whitelist and blacklist specific IP addresses or subnets to control network-level
access to a server.
[float]
[[preserving-data-integrity]]
=== Preserving Data Integrity
A critical part of security is keeping confidential data confidential.
Elasticsearch has built-in protections against accidental data loss and
corruption. However, there's nothing to stop deliberate tampering or data
interception. {security} preserves the integrity of your data by
<<ssl-tls, encrypting communications>> to and from nodes and
<<enable-message-authentication, authenticating messages>> to verify that they
have not been tampered with or corrupted in transit during node-to-node
communication. For even greater protection, you can increase the
<<ciphers, encryption strength>> and
<<separating-node-client-traffic, separate client traffic from node-to-node communications>>.
[float]
[[maintaining-audit-trail]]
=== Maintaining an Audit Trail
Keeping a system secure takes vigilance. By using {security} to maintain
an audit trail, you can easily see who is accessing your cluster and what they're
doing. By analyzing access patterns and failed attempts to access your cluster,
you can gain insights into attempted attacks and data breaches. Keeping an
auditable log of the activity in your cluster can also help diagnose operational
issues.
[float]
=== Where to Go Next
* <<security-getting-started, Getting Started>>
steps through how to install and start using Security for basic authentication.
* <<how-security-works, How Security Works>>
provides more information about how Security supports user authentication,
authorization, and encryption.
* <<tribe-clients-integrations>>
shows you how to interact with an Elasticsearch cluster protected by
X-Pack Security.
* <<security-reference, Reference>>
provides detailed information about the access privileges you can grant to
users, the settings you can configure for Security in `elasticsearch.yml`,
and the files where Security configuration information is stored.
[float]
=== Have Comments, Questions, or Feedback?
Head over to our {security-forum}[Security Discussion Forum]
to share your experience, questions, and suggestions.
--
include::getting-started.asciidoc[]
include::how-security-works.asciidoc[]
include::authentication.asciidoc[]
include::authorization.asciidoc[]
include::auditing.asciidoc[]
include::securing-communications.asciidoc[]
include::using-ip-filtering.asciidoc[]
include::tribe-clients-integrations.asciidoc[]
include::reference.asciidoc[]

87
docs/en/security/limitations.asciidoc Normal file
View File

@ -0,0 +1,87 @@
[[security-limitations]]
== Security Limitations
[float]
=== Plugins
Elasticsearch's plugin infrastructure is extremely flexible in terms of what can
be extended. While it opens up Elasticsearch to a wide variety of (often custom)
additional functionality, when it comes to security, this high extensibility level
comes at a cost. We have no control over the third-party plugins' code (open
source or not) and therefore we cannot guarantee their compliance with {security}.
For this reason, third-party plugins are not officially supported on clusters
with {security} enabled.
[float]
=== Changes in Index Wildcard Behavior
Elasticsearch clusters with {security} enabled apply the `/_all` wildcard, and
all other wildcards, to the indices that the current user has privileges for, not
the set of all indices on the cluster.
While creating or retrieving aliases by providing wildcard expressions for alias names, if there are no existing authorized aliases
that match the wildcard expression provided an IndexNotFoundException is returned.
[float]
=== Multi Document APIs
Multi get and multi term vectors API throw IndexNotFoundException when trying to access non existing indices that the user is
not authorized for. By doing that they leak information regarding the fact that the index doesn't exist, while the user is not
authorized to know anything about those indices.
[float]
=== Filtered Index Aliases
Aliases containing filters are not a secure way to restrict access to individual
documents, due to the limitations described in <<alias-limitations, Index and Field Names Can Be Leaked When Using Aliases>>.
{security} provides a secure way to restrict access to documents through the
<<field-and-document-access-control, document-level security>> feature.
[float]
=== Field and Document Level Security Limitations
When a user's role enables document or field level security for an index:
* The user cannot perform write operations:
** The update API isn't supported.
** Update requests included in bulk requests aren't supported.
* The request cache is disabled for search requests.
When a user's role enables document level security for an index:
* Document level security isn't applied for APIs that aren't document based.
An example is the field stats API.
* Document level security doesn't affect global index statistics that relevancy
scoring uses. So this means that scores are computed without taking the role
query into account. Note that documents not matching with the role query are
never returned.
* The `has_child` and `has_parent` queries aren't supported as query in the
role definition. The `has_child` and `has_parent` queries can be used in the
search API with document level security enabled.
* Any query that makes remote calls to fetch data to query by isn't supported.
The following queries aren't supported:
** The `terms` query with terms lookup isn't supported.
** The `geo_shape` query with indexed shapes isn't supported.
** The `percolate` query isn't supported.
[float]
[[alias-limitations]]
=== Index and Field Names Can Be Leaked When Using Aliases
Calling certain Elasticsearch APIs on an alias can potentially leak information
about indices that the user isn't authorized to access. For example, when you get
the mappings for an alias with the `_mapping` API, the response includes the
index name and mappings for each index that the alias applies to. Similarly, the
response to a `_field_stats` request includes the name of the underlying index,
rather than the alias name.
Until this limitation is addressed, avoid index and field names that contain
confidential or sensitive information.
[float]
=== LDAP Realm
The <<ldap-realm, LDAP Realm>> does not currently support the discovery of nested
LDAP Groups. For example, if a user is a member of `group_1` and `group_1` is a
member of `group_2`, only `group_1` will be discovered. However, the
<<active-directory-realm, Active Directory Realm>> *does* support transitive
group membership.

12
docs/en/security/reference.asciidoc Normal file
View File

@ -0,0 +1,12 @@
[[security-reference]]
== Reference
* <<security-privileges, Security Privileges>>
* <<security-settings, Security Settings>>
* <<security-files, Security Files>>
* <<security-api, Security API>>
include::reference/privileges.asciidoc[]
// include::reference/settings.asciidoc[]
include::reference/files.asciidoc[]

41
docs/en/security/reference/files.asciidoc Normal file
View File

@ -0,0 +1,41 @@
[[security-files]]
=== Security Files
The {security} uses the following files:
* `CONFIG_DIR/x-pack/roles.yml` defines the roles in use on the cluster
(read more <<roles-management-file, here>>).
* `CONFIG_DIR/x-pack/users` defines the users and their hashed passwords for
the <<file-realm,`file` realm>>.
* `CONFIG_DIR/x-pack/users_roles` defines the user roles assignment for the
the <<file-realm, `file` realm>>.
* `CONFIG_DIR/x-pack/role_mapping.yml` defines the role assignments for a
Distinguished Name (DN) to a role. This allows for LDAP and Active Directory
groups and users and PKI users to be mapped to roles (read more
<<mapping-roles, here>>).
* `CONFIG_DIR/x-pack/log4j2.properties` contains audit information (read more
<<logging-file, here>>).
* `CONFIG_DIR/x-pack/system_key` holds a cluster secret key that's used to
authenticate messages during node to node communication. For more information,
see <<enable-message-authentication, Enabling Message Authentication>>.
[[security-files-location]]
IMPORTANT: Any files that {security} uses must be stored in the Elasticsearch
configuration directory. Elasticsearch runs with restricted permissions
and is only permitted to read from the locations configured in the
directory layout for enhanced security.
Several of these files are in the YAML format. When you edit these files, be
aware that YAML is indentation-level sensitive and indentation errors can lead
to configuration errors. Avoid the tab character to set indentation levels, or
use an editor that automatically expands tabs to spaces.
Be careful to properly escape YAML constructs such as `:` or leading exclamation
points within quoted strings. Using the `|` or `>` characters to define block
literals instead of escaping the problematic characters can help avoid problems.

93
docs/en/security/reference/privileges.asciidoc Normal file
View File

@ -0,0 +1,93 @@
[[security-privileges]]
=== Security Privileges
This section lists the privileges that you can assign to a role.
[[privileges-list-cluster]]
==== Cluster Privileges
[horizontal]
`all`::
All cluster administration operations, like snapshotting, node shutdown/restart,
settings update, rerouting, or managing users and roles.
`monitor`::
All cluster read-only operations, like cluster health & state, hot threads, node
info, node & cluster stats, snapshot/restore status, pending cluster tasks.
`manage`::
Builds on `monitor` and adds cluster operations that change values in the cluster.
This includes snapshotting,updating settings, and rerouting. This privilege does
not include the ability to manage security.
`manage_security`::
All security related operations such as CRUD operations on users and roles and
cache clearing.
`manage_index_templates`::
All operations on index templates.
`manage_pipeline`::
All operations on ingest pipelines.
`transport_client`::
All privileges necessary for a transport client to connect.
[[privileges-list-indices]]
==== Indices Privileges
[horizontal]
`all`::
Any action on an index
`monitor`::
All actions that are required for monitoring (recovery, segments info, index stats
& status).
`manage`::
All `monitor` privileges plus index administration (aliases, analyze, cache clear,
close, delete, exists, flush, mapping, open, force merge, refresh, settings,
search shards, templates, validate).
`view_index_metadata`::
Read-only access to index metadata (aliases, aliases exists, get index, exists, field mappings,
mappings, search shards, type exists, validate, warmers, settings). This
privilege is primarily available for use by <<kibana-roles, Kibana users>>.
`read`::
Read only access to actions (count, explain, get, mget, get indexed scripts,
more like this, multi percolate/search/termvector, percolate, scroll,
clear_scroll, search, tv). Also grants access to the update mapping
action.
`index`::
Privilege to index and update documents. Also grants access to the update
mapping action.
`create`::
Privilege to index documents. Also grants access to the update mapping
action.
`delete`::
Privilege to delete documents.
`write`::
Privilege to perform all write operations to documents, which includes the
permission to index, update, and delete documents as well as performing bulk
operations. Also grants access to the update mapping action.
`delete_index`::
Privilege to delete an index.
`create_index`::
Privilege to create an index. A create index request may contain aliases to be
added to the index once created. In that case the request requires the `manage`
privilege as well, on both the index and the aliases names.
==== Run As Privilege
The `run_as` permission enables an authenticated user to submit requests on
behalf of another user. The value can be a user name or a comma-separated list
of user names. (You can also specify users as an array of strings or a YAML
sequence.) For more information, see
<<run-as-privilege, Submitting Requests on Behalf of Other Users>>.

329
docs/en/security/release-notes.asciidoc Normal file
View File

@ -0,0 +1,329 @@
[[security-release-notes]]
== Shield Release Notes (Pre-5.0)
[float]
[[update-roles]]
=== Updated Role Definitions
The default role definitions in the `roles.yml` file may need to be changed to ensure proper interoperation with other
applications such as Monitoring and Kibana. Any role changes are stored in `roles.yml.new` when you upgrade. We recommend copying the following changes to your `roles.yml` file.
* The `kibana4` role now grants access to the Field Stats API.
* The permission on all the roles are updated to the verbose format to make it easier to enable field level and document level security. The `transport_client` role has been updated to work with Elasticsearch 2.0.0.
The `marvel_user` role has been updated to work with Monitoring 2.0 and a `remote_marvel_agent` role has been added. The `kibana3` and `marvel_agent` roles have been removed.
* `kibana` role added that defines the minimum set of permissions necessary for the Kibana 4 server.
* `kibana4` role updated to work with new features in Kibana 4 RC1
[float]
[[security-change-list]]
=== Change List
[float]
==== 2.4.2
November 22, 2016
.Bug Fixes
* Users with `manage` or `manage_security` cluster privileges can now access the `.security` index if they have the appropriate index
privileges.
.Breaking Changes
* Shield on tribe nodes now requires `tribe.on_conflict` to prefer one of the clusters.
[float]
==== 2.4.0
August 31, 2016
.Breaking Changes
* The `monitor` cluster privilege now grants access to the GET `/_license` API
[float]
==== 2.3.5
August 3, 2016
.Bug Fixes
* Fixed a license problem that was preventing tribe nodes from working with
Shield.
[float]
==== 2.3.4
July 7, 2016
.Bug Fixes
* The `default` transport profile SSL settings now override the `shield.ssl.*`
settings properly.
* Fixed a memory leak that occured when indices were deleted or closed.
[float]
==== 2.3.3
May 18, 2016
.Bug Fixes
* Fixed the `/_shield/realm/{realms}/_cache/clear` REST endpoint. This endpoint is deprecated and `/_shield/realm/{realms}/_clear_cache` should be used going forward.
[float]
==== 2.3.2
April 26, 2016
.Bug Fixes
* Date math expressions in index names are now resolved before attempting to authorize access to the indices.
* Fixed an issue where active directory realms did not work unless the url setting was configured.
* Enabled `_cat/indices` to be used when Shield is installed.
[float]
==== 2.3.1
April 4, 2016
.Bug Fixes
* Fixed an issue that could prevent nodes from joining the cluster.
[float]
==== 2.3.0
March 30, 2016
.New Features
* <<native-realm,Native realm>> with support for <<security-api-users,user management APIs>>.
* <<security-api-roles,Role management APIs>> have been added.
.Bug Fixes
* When evaluating permissions for multiple roles that have document level security enabled for the same index, Shield performed an `AND`
on the queries, which is not consistent with how role privileges work in Shield. This has been changed to an `OR` relationship and may
affect the behavior of existing roles; please ensure you are not relying on the `AND` behavior of document level security queries.
* When evaluation permissions for user that has roles with and without document level security (and/or field level security), the roles that
granted unrestricted access were not being applied properly and the user's access was still being restricted.
.Enhancements
* Added new <<security-privileges, privileges>> to simplify access control.
[float]
==== 2.2.1
March 15, 2016
.Bug Fixes
* Enable <<field-and-document-access-control,document and field level security>> by default.
* Fix issues with <<enable-message-authentication,message authentication>> on certain JDKs that do not support cloning message
authentication codes.
* Built in <<setting-up-authentication, realms>> no longer throw an exception if the `Authorization` header does not contain a basic
authentication token.
* Ensure each tribe client node has the same shield configuration as defined in the settings.
[float]
==== 2.2.0
February 2, 2016
.New Features
* Shield plugin for Kibana: Secures user sessions and enables users to log in and out of Kibana.
For information about installing the Shield plugin, see <<kibana, Using Kibana with Shield>>.
.Bug Fixes
* Update requests (including within bulk requests) are blocked when document
and field level security is enabled
[float]
==== 2.1.2
February 2, 2016
.Enhancements
* Adds support for Elasticssearch 2.1.2
[float]
==== 2.1.1
December 17, 2015
.Bug Fixes
* Disable the request cache when <<document-level-security, document level security>> is in use for a search request.
* Fix startup failures when using auditing and <<audit-log-entry-local-node-info, enabling network information output>>.
* Updated the `kibana4` role to include the Field Stats API.
[float]
==== 2.1.0
November 24, 2015
.Breaking Changes
* Same as 2.0.1. <<field-and-document-access-control, Document and Field Level Security>> is now disabled by default. Set `shield.dls_fls.enabled` to `true` in `elasticsearch.yml` to enable it. You cannot submit `_bulk` update requests when document and field level security is enabled.
.Enhancements
* Adds support for Elasticsearch 2.1.0.
[float]
==== 2.0.2
December 16, 2015
.Bug Fixes
* Disable the request cache when <<document-level-security, document level security>> is in use for a search request.
[float]
==== 2.0.1
November 24, 2015
.Breaking Changes
* <<field-and-document-access-control, Document and Field Level Security>> is now disabled by default. Set `shield.dls_fls.enabled` to `true` in `elasticsearch.yml` to enable it. You cannot submit `_bulk` update requests when document and field level security is enabled.
.Enhancement
* Adds support for Elasticsearch 2.0.1.
[float]
==== 2.0.0
October 28, 2015
.Breaking Changes
* All files that Shield uses must be kept in the <<security-files-location, configuration directory>> due to the enhanced security of Elasticsearch 2.0.
* The network format has been changed from all previous versions of Shield and a full cluster restart is required to upgrade to Shield 2.0.
.New Features
* <<field-and-document-access-control, Document and Field Level Security>> support has been added and can be
configured per role.
* Support for <<custom-realms, custom authentication realms>> has been added, allowing Shield to integrate with more authentication sources and methods.
* <<run-as-privilege, User impersonation support>> has also been added, which allows a user to send a request to Elasticsearch that will be run
with the specified user's permissions.
.Bug Fixes
* <<auditing, Auditing>> now captures requests from nodes using a different system key as tampered requests.
* The <<audit-index, index output for auditing>> stores the type of request when available.
* `esusers` and `syskeygen` work when spaces are in the Elasticsearch installation path.
* Fixed a rare issue where authentication fails even when the username and password are correct.
[float]
==== 1.3.3
.Bug Fixes
* Fixed a rare issue where authentication fails even when the username and password are correct.
* The <<audit-index, index output for auditing>> stores the type of request when available.
.Enhancements
* Tampered requests with a bad header are now audited.
[float]
==== 1.3.2
August 10, 2015
.Bug Fixes
* When using the <<ldap-user-search,LDAP user search>> mechanism, connection errors during startup no longer cause the node to stop.
* The <<security-api-clear-cache,Clear Cache API>> no longer generates invalid JSON.
* The <<audit-index,index output for auditing>> starts properly when forwarding the audit events to a remote cluster and uses
the correct user to index the audit events.
[float]
==== 1.3.1
July 21, 2015
.Bug Fixes
* Fixes <<enable-message-authentication,message authentication>> serialization to work with Shield 1.2.1 and earlier.
** NOTE: if you are upgrading from Shield 1.3.0 or Shield 1.2.2 a {ref-17}/setup-upgrade.html#restart-upgrade[cluster restart upgrade]
will be necessary. When upgrading from other versions of Shield, follow the normal upgrade procedure.
[float]
==== 1.3.0
June 24, 2015
.Breaking Changes
* The `sha2` and `apr1` hashing algorithms have been removed as options for the <<cache-hash-algo,`cache.hash_algo` setting>>.
If your existing Shield installation uses either of these options, remove the setting and use the default `ssha256`
algorithm.
* The `users` file now only supports `bcrypt` password hashing. All existing passwords stored using the `esusers` tool
have been hashed with `bcrypt` and are not affected.
.New Features
* <<pki-realm,PKI Realm>>: Adds Public Key Infrastructure (PKI) authentication through the use of X.509 certificates in place of
username and password credentials.
* <<auditing, Index Output for Audit Events>>: An index based output has been added for storing audit events in an Elasticsearch index.
.Enhancements
* TLS 1.2 is now the default protocol.
* Clients that do not support pre-emptive basic authentication can now support both anonymous and authenticated access
by specifying the `shield.authc.anonymous.authz_exception` <<anonymous-access,setting>> with a value of `false`.
* Reduced logging for common SSL exceptions, such as a client closing the connection during a handshake.
.Bug Fixes
* The `esusers` and `syskeygen` tools now work correctly with environment variables in the RPM and DEB installation
environment files `/etc/sysconfig/elasticsearch` and `/etc/default/elasticsearch`.
* Default ciphers no longer include `TLS_DHE_RSA_WITH_AES_128_CBC_SHA`.
[float]
==== 1.2.3
July 21, 2015
.Bug Fixes
* Fixes <<enable-message-authentication,message authentication>> serialization to work with Shield 1.2.1 and earlier.
** NOTE: if you are upgrading from Shield 1.2.2 a {ref-17}/setup-upgrade.html#restart-upgrade[cluster restart upgrade]
will be necessary. When upgrading from other versions of Shield, follow the normal upgrade procedure.
[float]
==== 1.2.2
June 24, 2015
.Bug Fixes
* The `esusers` tool no longer warns about missing roles that are properly defined in the `roles.yml` file.
* The period character, `.`, is now allowed in usernames and role names.
* The {ref-17}/query-dsl-terms-filter.html#_caching_19[terms filter lookup cache] has been disabled to ensure all requests
are properly authorized. This removes the need to manually disable the terms filter cache.
* For LDAP client connections, only the protocols and ciphers specified in the `shield.ssl.supported_protocols` and
`shield.ssl.ciphers` <<ssl-tls-settings,settings>> will be used.
* The auditing mechanism now logs authentication failed events when a request contains an invalid authentication token.
[float]
==== 1.2.1
April 29, 2015
.Bug Fixes
* Several bug fixes including a fix to ensure that {ref}/disk-allocator.html[Disk-based Shard Allocation]
works properly with Shield
[float]
==== 1.2.0
March 24, 2015
.Enhancements
* Adds support for Elasticsearch 1.5
[float]
==== 1.1.1
April 29, 2015
.Bug Fixes
* Several bug fixes including a fix to ensure that {ref}/disk-allocator.html[Disk-based Shard Allocation]
works properly with Shield
[float]
==== 1.1.0
March 24, 2015
.New Features
* LDAP:
** Add the ability to bind as a specific user for LDAP searches, which removes the need to specify `user_dn_templates`.
This mode of operation also makes use of connection pooling for better performance. Please see <<ldap-user-search, ldap user search>>
for more information.
** User distinguished names (DNs) can now be used for <<ldap-role-mapping, role mapping>>.
* Authentication:
** <<anonymous-access, Anonymous access>> is now supported (disabled by default).
* IP Filtering:
** IP Filtering settings can now be <<dynamic-ip-filtering,dynamically updated>> using the {ref}/cluster-update-settings.html[Cluster Update Settings API].
.Enhancements
* Significant memory footprint reduction of internal data structures
* Test if SSL/TLS ciphers are supported and warn if any of the specified ciphers are not supported
* Reduce the amount of logging when a non-encrypted connection is opened and `https` is being used
* Added the <<kibana-roles, `kibana_server` role>>, which is a role that contains the minimum set of permissions required for the Kibana 4 server.
* In-memory user credential caching hash algorithm defaults now to salted SHA-256 (see <<cache-hash-algo, Cache hash algorithms>>
.Bug Fixes
* Filter out sensitive settings from the settings APIs
[float]
==== 1.0.2
March 24, 2015
.Bug Fixes
* Filter out sensitive settings from the settings APIs
* Significant memory footprint reduction of internal data structures
[float]
==== 1.0.1
February 13, 2015
.Bug Fixes
* Fixed dependency issues with Elasticsearch 1.4.3 and (Lucene 4.10.3 that comes with it)
* Fixed bug in how user roles were handled. When multiple roles were defined for a user, and one of the
roles only had cluster permissions, not all privileges were properly evaluated.
* Updated `kibana4` permissions to be compatible with Kibana 4 RC1
* Ensure the mandatory `base_dn` settings is set in the `ldap` realm configuration

23
docs/en/security/securing-communications.asciidoc Normal file
View File

@ -0,0 +1,23 @@
[[encrypting-communications]]
== Encrypting Communications
Elasticsearch nodes store data that may be confidential. Attacks on the data may
come from the network. These attacks could include sniffing of the data,
manipulation of the data, and attempts to gain access to the server and thus the
files storing the data. Securing your nodes with the procedures below helps to
reduce risk from network-based attacks.
This section shows how to:
* Encrypt traffic to, from and within an Elasticsearch cluster using SSL/TLS,
* Require nodes to authenticate as they join the cluster using SSL certificates, and
* Make it more difficult for remote attackers to issue any commands to Elasticsearch.
The authentication of new nodes helps prevent a rogue node from joining the
cluster and receiving data through replication.
include::securing-communications/setting-up-ssl.asciidoc[]
include::securing-communications/enabling-cipher-suites.asciidoc[]
include::securing-communications/separating-node-client-traffic.asciidoc[]

24
docs/en/security/securing-communications/enabling-cipher-suites.asciidoc Normal file
View File

@ -0,0 +1,24 @@
[[ciphers]]
=== Enabling Cipher Suites for Stronger Encryption
The TLS and SSL protocols use a cipher suite that determines the strength of
encryption used to protect the data. You may want to increase the strength of
encryption used when using a Oracle JVM; the IcedTea OpenJDK ships without these
restrictions in place. This step is not required to successfully use encrypted
communication.
The _Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy
Files_ enable the use of additional cipher suites for Java in a separate JAR file
that you need to add to your Java installation. You can download this JAR file
from Oracle's http://www.oracle.com/technetwork/java/javase/downloads/index.html[download page].
The _JCE Unlimited Strength Jurisdiction Policy Files`_ are required for
encryption with key lengths greater than 128 bits, such as 256-bit AES encryption.
After installation, all cipher suites in the JCE are available for use. To enable
the use of stronger cipher suites with {security}, configure the `cipher_suites`
parameter. See the <<ssl-tls-settings, Configuration Parameters for TLS/SSL>>
section of this document for specific parameter information.
NOTE: The _JCE Unlimited Strength Jurisdiction Policy Files_ must be installed
on all nodes in the cluster to establish an improved level of encryption
strength.

78
docs/en/security/securing-communications/separating-node-client-traffic.asciidoc Normal file
View File

@ -0,0 +1,78 @@
[[separating-node-client-traffic]]
=== Separating node-to-node and client traffic
Elasticsearch has the feature of so called {ref}/modules-transport.html#_tcp_transport_profiles[TCP transport profiles]
that allows it to bind to several ports and addresses. {security} extends on this
functionality to enhance the security of the cluster by enabling the separation
of node-to-node transport traffic from client transport traffic. This is important
if the client transport traffic is not trusted and could potentially be malicious.
To separate the node-to-node traffic from the client traffic, add the following
to `elasticsearch.yml`:
[source, yaml]
--------------------------------------------------
transport.profiles.client: <1>
port: 9500-9600 <2>
shield:
type: client <3>
--------------------------------------------------
<1> `client` is the name of this example profile
<2> The port range that will be used by transport clients to communicate with
this cluster
<3> Categorizes the profile as a `client`. This accounts for additional security
filters by denying request attempts on for internal cluster operations
(e.g shard level actions and ping requests) from this profile.
If supported by your environment, an internal network can be used for node-to-node
traffic and public network can be used for client traffic by adding the following
to `elasticsearch.yml`:
[source, yaml]
--------------------------------------------------
transport.profiles.default.bind_host: 10.0.0.1 <1>
transport.profiles.client.bind_host: 1.1.1.1 <2>
--------------------------------------------------
<1> The bind address for the network that will be used for node-to-node communication
<2> The bind address for the network used for client communication
If separate networks are not available, then <<ip-filtering, IP Filtering>> can
be enabled to limit access to the profiles.
The TCP transport profiles also allow for enabling SSL on a per profile basis.
This is useful if you have a secured network for the node-to-node communication,
but the client is on an unsecured network. To enable SSL on a client profile when
SSL is disabled for node-to-node communication, add the following to
`elasticsearch.yml`:
[source, yaml]
--------------------------------------------------
transport.profiles.client.xpack.security.ssl.enabled: true <1>
--------------------------------------------------
<1> This enables SSL on the client profile. The default value for this setting
is the value of `xpack.security.transport.ssl.enabled`.
When using SSL for transport, a different set of certificates can also be used
for the client traffic by adding the following to `elasticsearch.yml`:
[source, yaml]
--------------------------------------------------
transport.profiles.client.xpack.security.ssl.truststore:
path: /path/to/another/truststore
password: changeme
transport.profiles.client.xpack.security.ssl.keystore:
path: /path/to/another/keystore
password: changeme
--------------------------------------------------
To change the default behavior that requires certificates for transport clients,
set the following value in the `elasticsearch.yml` file:
[source, yaml]
--------------------------------------------------
transport.profiles.client.xpack.security.ssl.client_authentication: no
--------------------------------------------------
This setting keeps certificate authentication active for node-to-node traffic,
but removes the requirement to distribute a signed certificate to transport
clients. Please see the <<transport-client, Transport Client>> section.

258
docs/en/security/securing-communications/setting-up-ssl.asciidoc Normal file
View File

@ -0,0 +1,258 @@
[[ssl-tls]]
=== Setting Up SSL/TLS on a Cluster
{security} enables you to encrypt traffic to, from and within your Elasticsearch
cluster. Connections are secured using Transport Layer Security (TLS), which is
commonly referred to as "SSL".
WARNING: Clusters that do not have encryption enabled send all data in plain text
including passwords.
To enable encryption, you need to perform the following steps on each node in
the cluster:
. <<installing-node-certificates, Generate a private key and X.509 certificate>>.
. <<configure-ssl, Configure the node>> to:
.. Identify itself using its signed certificate.
.. Enable SSL on the transport and HTTP layers.
. Restart Elasticsearch.
[[installing-node-certificates]]
==== Node Certificates
TLS requires X.509 certificates to perform encryption and authentication of the application
that is being communicated with. In order for the communication between nodes to be truly
secure, the certificates must be validated. The recommended approach for validating
certificate authenticity in a Elasticsearch cluster is to trust the certificate authority (CA)
that signed the certificate. By doing this, as nodes are added to your cluster they just need
to use a certificate signed by the same CA and the node is automatically allowed to join the
cluster. Additionally, it is recommended that the certificates contain subject alternative
names (SAN) that correspond to the node's ip address and dns name so that hostname verification
can be performed.
In order to simplify the process of generating certificates for the Elastic Stack, a command
line tool, `certgen` has been included with {xpack}. This tool takes care of the generating
a CA and signing certificates with the CA. `certgen` can be used interactively or in a silent
mode through the use of an input file. The `certgen` tool also supports generation of certificate
signing requests (CSR), so that a commercial or organization specific CA may be used to sign
the certificates.
NOTE: If you choose not to use the `certgen`, the certificates that you obtain must allow for both
`clientAuth` and `serverAuth` if the extended key usage extension is present. The certificates
need to be in PEM format. Although not required, it is highly recommended that the certificate contain
the dns name(s) and/or ip address(es) of the node so that hostname verification may be used.
[[generating-signed-certificates]]
===== Generating Certificates with `certgen`
The `certgen` tool can be used to generate a CA and signed certificates for your nodes. The tool
can be used interactively:
[listing]
....
bin/x-pack/certgen
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL in the Elastic stack. Depending on the command
line option specified, you may be prompted for the following:
* The path to the output file
* The output file is a zip file containing the signed certificates and
private keys for each instance. If a Certificate Authority was generated,
the certificate and private key will also be included in the output file.
* Information about each instance
* An instance is any piece of the Elastic Stack that requires a SSL certificate.
Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
may all require a certificate and private key.
* The minimum required value for each instance is a name. This can simply be the
hostname, which will be used as the Common Name of the certificate. A full
distinguished name may also be used.
* IP addresses and DNS names are optional. Multiple values can be specified as a
comma separated string. If no IP addresses or DNS names are provided, you may
disable hostname verification in your SSL configuration.
* Certificate Authority private key password
* The password may be left empty if desired.
Let's get started...
Please enter the desired output file [/home/es/config/x-pack/certificate-bundle.zip]:
Enter instance name: node01
Enter name for directories and files [node01]:
Enter IP Addresses for instance (comma-separated if more than one) []: 10.10.0.1
Enter DNS names for instance (comma-separated if more than one) []: node01.mydomain.com,node01
Would you like to specify another instance? Press 'y' to continue entering instance information: y
Enter instance name: node02
Enter name for directories and files [node02]:
Enter IP Addresses for instance (comma-separated if more than one) []: 10.10.0.2
Enter DNS names for instance (comma-separated if more than one) []: node02.mydomain.com
Would you like to specify another instance? Press 'y' to continue entering instance information:
Certificates written to /Users/jmodi/dev/tmp/elasticsearch-5.0.0-alpha5-SNAPSHOT/config/x-pack/certificate-bundle.zip
This file should be properly secured as it contains the private keys for all
instances and the certificate authority.
After unzipping the file, there will be a directory for each instance containing
the certificate and private key. Copy the certificate, key, and CA certificate
to the configuration directory of the Elastic product that they will be used for
and follow the SSL configuration instructions in the product guide.
For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
....
The usage of `certgen` above generates a zip file with the CA certificate, private key, two signed certificates and keys
in PEM format for `node01` and `node02`.
[[generating-csr]]
===== Generating Certificate Signing Requests with `certgen`
When using a commercial or organization specific CA, the `certgen` tool may be used to generate
certificate signing requests (CSR) for the nodes in your cluster:
[listing]
....
bin/x-pack/certgen -csr
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL in the Elastic stack. Depending on the command
line option specified, you may be prompted for the following:
* The path to the output file
* The output file is a zip file containing the certificate signing requests
and private keys for each instance.
* Information about each instance
* An instance is any piece of the Elastic Stack that requires a SSL certificate.
Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
may all require a certificate and private key.
* The minimum required value for each instance is a name. This can simply be the
hostname, which will be used as the Common Name of the certificate. A full
distinguished name may also be used.
* IP addresses and DNS names are optional. Multiple values can be specified as a
comma separated string. If no IP addresses or DNS names are provided, you may
disable hostname verification in your SSL configuration.
Let's get started...
Please enter the desired output file [/home/es/config/x-pack/csr-bundle.zip]:
Enter instance name: node01
Enter name for directories and files [node01]:
Enter IP Addresses for instance (comma-separated if more than one) []: 10.10.0.1
Enter DNS names for instance (comma-separated if more than one) []: node01.mydomain.com,node01
Would you like to specify another instance? Press 'y' to continue entering instance information: y
Enter instance name: node02
Enter name for directories and files [node02]:
Enter IP Addresses for instance (comma-separated if more than one) []: 10.10.0.2
Enter DNS names for instance (comma-separated if more than one) []: node02.mydomain.com
Would you like to specify another instance? Press 'y' to continue entering instance information:
Certificate signing requests written to /Users/jmodi/dev/tmp/elasticsearch-5.0.0-alpha5-SNAPSHOT/config/x-pack/csr-bundle.zip
This file should be properly secured as it contains the private keys for all
instances.
After unzipping the file, there will be a directory for each instance containing
the certificate signing request and the private key. Provide the certificate
signing requests to your certificate authority. Once you have received the
signed certificate, copy the signed certificate, key, and CA certificate to the
configuration directory of the Elastic product that they will be used for and
follow the SSL configuration instructions in the product guide.
....
The usage of `certgen` above generates a zip file with two CSRs and private
keys. The CSRs should be provided to the CA in order to obtain the signed
certificates. The signed certificates will need to be in PEM format in order to
be used.
===== Using `certgen` in Silent Mode
`certgen` supports a silent mode of operation to enable easier batch operations. In order
to use this mode, a YAML file containing the information about the instances needs to be
created matching the format shown below:
[source, yaml]
--------------------------------------------------
instances:
- name: "node1" <1>
ip: <2>
- "192.0.2.1"
dns: <3>
- "node1.mydomain.com"
- name: "node2"
ip:
- "192.0.2.2"
- "198.51.100.1"
- name: "node3"
- name: "node4"
dns:
- "node4.mydomain.com"
- "node4.internal"
- name: "CN=node5,OU=IT,DC=mydomain,DC=com"
filename: "node5" <4>
--------------------------------------------------
<1> The name of the instance. This can be a simple string value or can be a Distinguished Name (DN). This is the only required field.
<2> An optional array of strings that represent IP Addresses for this instance. Both IPv4 and IPv6 values are allowed. The values will
be added as Subject Alternative Names.
<3> An optional array of strings that represent DNS names for this instance. The values will be added as Subject Alternative Names.
<4> The filename to use for this instance. This name will be the name of the directory in the zip file that this instance's files will
stored in and it will used be used in the naming of the files within the directory. This filename should not have an extension. Note: If
the `name` provided for the instance does not represent a valid filename, then the `filename` field must be present.
With the YAML file ready, the `certgen` tool can be used to generate certificates or certificate signing requests. Simply pass the file's
path to `certgen` using the `-in` option. For example:
[source, sh]
--------------------------------------------------
bin/x-pack/certgen -in instances.yml <1>
--------------------------------------------------
<1> Generates a CA certificate and private key in addition to certificates and private keys for the instances
contained in the YAML file. The other options to the tool can be specified in addition to the `-in` option. For all of the available
options, run `bin/x-pack/certgen -h`.
[[enable-ssl]]
==== Enabling SSL in the Node Configuration
Once you have the signed certificate, private key, and CA certificate you need to
modify the node configuration to enable SSL.
[[configure-ssl]]
To enable SSL, make the following changes in `elasticsearch.yml`:
. Specify the location of the node's keystore and the password(s) needed to
access the node's certificate. For example:
+
--
[source, yaml]
--------------------------------------------------
xpack.ssl.key: /home/es/config/x-pack/node01.key <1>
xpack.ssl.certificate: /home/es/config/x-pack/node01.crt <2>
xpack.ssl.certificate_authorities: [ "/home/es/config/x-pack/ca.crt" ] <3>
--------------------------------------------------
<1> The full path to the node key file. This must be a location within the
Elasticsearch configuration directory.
<2> The full path to the node certificate. This must be a location within the
Elasticsearch configuration directory.
<3> An array of paths to the CA certificates that should be trusted. These paths
must be a location within the Elasticsearch configuration directory.
--
. Enable SSL on the transport networking layer to ensure that communication
between nodes is encrypted:
+
[source, yaml]
--------------------------------------------------
xpack.security.transport.ssl.enabled: true
--------------------------------------------------
+
. Enable SSL on the HTTP layer to ensure that communication between HTTP clients
and the cluster is encrypted:
+
[source, yaml]
--------------------------------------------------
xpack.security.http.ssl.enabled: true
--------------------------------------------------
+
. Restart Elasticsearch.
NOTE: All SSL related node settings that are considered to be highly sensitive
and therefore are not exposed via the
{ref}/cluster-nodes-info.html#cluster-nodes-info[nodes info API].

42
docs/en/security/tribe-clients-integrations.asciidoc Normal file
View File

@ -0,0 +1,42 @@
[[tribe-clients-integrations]]
== Tribe, Clients and Integrations
When using a {ref}/modules-tribe.html[Tribe Nodes] you need to take extra steps to secure its communication
with the connected clusters
* <<tribe-node-configuring, Tribe Node Security>>
You will need to update the configuration for several clients to work with a
secured cluster:
* <<java-clients, Java Clients>>
* <<http-clients, HTTP Clients>>
{security} enables you to secure you Elasticsearch cluster. But Elasticsearch
itself is only one product within the Elastic Stack. It is often the case that
other products in the stack are connected to the cluster and therefore need to
be secured as well, or at least communicate with the cluster in a secured way:
* <<hadoop, Apache Hadoop>>
* <<logstash, Logstash>>
* <<kibana, Kibana>>
* <<secure-monitoring, Monitoring>>
include::tribe-clients-integrations/tribe.asciidoc[]
include::tribe-clients-integrations/java.asciidoc[]
include::tribe-clients-integrations/http.asciidoc[]
include::tribe-clients-integrations/hadoop.asciidoc[]
include::tribe-clients-integrations/logstash.asciidoc[]
include::tribe-clients-integrations/beats.asciidoc[]
include::tribe-clients-integrations/kibana.asciidoc[]
include::tribe-clients-integrations/monitoring.asciidoc[]
include::tribe-clients-integrations/reporting.asciidoc[]

187
docs/en/security/tribe-clients-integrations/beats.asciidoc Normal file
View File

@ -0,0 +1,187 @@
[[beats]]
=== Beats and Security
To send data to a secured cluster through the `elasticsearch` output,
a Beat needs to authenticate as a user who can manage index templates,
monitor the cluster, create indices, and read, and write to the indices
it creates.
If encryption is enabled on the cluster, you also need to enable HTTPS in the
Beat configuration.
In addition to configuring authentication credentials for the Beat itself, you
need to grant authorized users permission to access the indices it creates.
[float]
[[beats-basic-auth]]
==== Configuring Authentication Credentials for a Beat
When sending data to a secured cluster through the `elasticsearch`
output, a Beat must either provide basic authentication credentials
or present a client certificate.
To configure authentication credentials for a Beat:
. Create a role that has the `manage_index_templates` and
`monitor` cluster privileges, and `read`, `write`, and `create_index`
privileges for the indices the Beat creates. You can create roles from the
**Management / Roles** UI in Kibana or through the `role` API.
For example, the following request creates a `packetbeat_writer` role:
+
[source, sh]
---------------------------------------------------------------
POST _xpack/security/role/packetbeat_writer
{
"cluster": ["manage_index_templates", "monitor"],
"indices": [
{
"names": [ "packetbeat-*" ], <1>
"privileges": ["write","create_index"]
}
]
}
---------------------------------------------------------------
<1> If you use a custom Packetbeat index pattern, specify that pattern
instead of the default `packetbeat-*` pattern.
. Assign the writer role to the user the Beat is going to use to
connect to Elasticsearch:
.. To authenticate as a native user, create a user for the Beat
to use internally and assign it the writer role. You can create
users from the **Management / Users** UI in Kibana or through the
`user` API. For example, the following request creates a
`packetbeat_internal` user that has the `packetbeat_writer` role:
+
[source, sh]
---------------------------------------------------------------
POST /_xpack/security/user/packetbeat_internal
{
"password" : "changeme",
"roles" : [ "packetbeat_writer"],
"full_name" : "Internal Packetbeat User"
}
---------------------------------------------------------------
.. To authenticate using PKI authentication, assign the writer role
to the internal Beat user in the <<mapping-roles,`role_mapping.yml`>>
configuration file. Specify the user by the distinguished name that
appears in its certificate.
+
[source, yaml]
---------------------------------------------------------------
packetbeat_writer:
- "cn=Internal Packetbeat User,ou=example,o=com"
---------------------------------------------------------------
. Configure authentication credentials for the `elasticsearch` output
in the Beat configuration file:
.. To use basic authentication, configure the `username` and `password`
settings. For example, the following Packetbeat output configuration
uses the native `packetbeat_internal` user to connect to Elasticsearch:
+
[source,js]
--------------------------------------------------
output.elasticsearch:
hosts: ["localhost:9200"]
index: "packetbeat"
username: "packetbeat_internal"
password: "changeme"
--------------------------------------------------
.. To use PKI authentication, configure the `certificate` and
`key` settings:
+
[source,js]
--------------------------------------------------
output.elasticsearch:
hosts: ["localhost:9200"]
index: "packetbeat"
ssl.certificate: "/etc/pki/client/cert.pem" <1>
ssl.key: "/etc/pki/client/cert.key"
--------------------------------------------------
<1> The distinguished name (DN) in the certificate must be mapped to
the writer role in the `role_mapping.yml` configuration file on each
node in the Elasticsearch cluster.
[float]
[[beats-user-access]]
==== Granting Users Access to Beats Indices
To enable users to access the indices a Beat creates, grant them `read` and
`view_index_metadata` privileges on the Beat indices:
. Create a role that has the `read` and `view_index_metadata`
privileges for the Beat indices. You can create roles from the
**Management > Roles** UI in Kibana or through the `role` API.
For example, the following request creates a `packetbeat_reader`
role:
+
[source, sh]
---------------------------------------------------------------
POST _xpack/security/role/packetbeat_reader
{
"indices": [
{
"names": [ "packetbeat-*" ], <1>
"privileges": ["read","view_index_metadata"]
}
]
}
---------------------------------------------------------------
<1> If you use a custom Packetbeat index pattern, specify that pattern
instead of the default `packetbeat-*` pattern.
. Assign your users the reader role so they can access the Beat indices:
.. If you're using the `native` realm, you can assign roles with the
**Management > Users** UI in Kibana or through the `user` API. For
example, the following request grants `packetbeat_user`
the `packetbeat_reader` role:
+
[source, sh]
---------------------------------------------------------------
POST /_xpack/security/user/packetbeat_user
{
"password" : "changeme",
"roles" : [ "packetbeat_reader"],
"full_name" : "Packetbeat User"
}
---------------------------------------------------------------
.. If you're using the LDAP, Active Directory, or PKI realms, you
assign the roles in the <<mapping-roles,`role_mapping.yml`>> configuration
file. For example, the following snippet grants `Packetbeat User`
the `packetbeat_reader` role:
+
[source, yaml]
---------------------------------------------------------------
packetbeat_reader:
- "cn=Packetbeat User,dc=example,dc=com"
---------------------------------------------------------------
[float]
[[beats-tls]]
===== Configuring Beats to use Encrypted Connections
If encryption is enabled on the Elasticsearch cluster, you need to
connect to Elasticsearch via HTTPS. If the CA that signed your node certificates
is not in the host system's trusted certificate authorities list, you also need
to add the path to the `.pem` file that contains your CA's certificate to the
Beat configuration.
To configure a Beat to connect to Elasticsearch via HTTPS, add the `https` protocol
to all host URLs:
[source,js]
--------------------------------------------------
output.elasticsearch:
hosts: ["https://localhost:9200"] <1>
index: "packetbeat"
ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] <2>
--------------------------------------------------
<1> Specify the `https` protocol to connect the Elasticsearch cluster.
<2> Specify the path to the local `.pem` file that contains your Certificate
Authority's certificate. This is generally only needed if you use your
own CA to sign your node certificates.

23
docs/en/security/tribe-clients-integrations/hadoop.asciidoc Normal file
View File

@ -0,0 +1,23 @@
[[hadoop]]
=== ES-Hadoop and Security
Elasticsearch for Apache Hadoop ("ES-Hadoop") is capable of using HTTP basic and
PKI authentication and/or TLS/SSL when accessing an Elasticsearch cluster. For
full details please refer to the ES-Hadoop documentation, in particular the
`Security` section.
For authentication purposes, select the user for your ES-Hadoop client (for
maintenance purposes it is best to create a dedicated user). Then, assign that
user to a role with the privileges required by your Hadoop/Spark/Storm job.
Configure ES-Hadoop to use the user name and password through the
`es.net.http.auth.user` and `es.net.http.auth.pass` properties.
If PKI authentication is enabled, setup the appropriate `keystore` and `truststore`
instead through `es.net.ssl.keystore.location` and `es.net.truststore.location`
(and their respective `.pass` properties to specify the password).
For secured transport, enable SSL/TLS through the `es.net.ssl` property by
setting it to `true`. Depending on your SSL configuration (keystore, truststore, etc...)
you might need to set other parameters as well - please refer to the
http://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html[ES-Hadoop] documentation,
specifically the `Configuration` and `Security` chapters.

62
docs/en/security/tribe-clients-integrations/http.asciidoc Normal file
View File

@ -0,0 +1,62 @@
[[http-clients]]
=== HTTP/REST Clients and Security
{security} works with standard HTTP {wikipedia}/Basic_access_authentication[basic authentication]
headers to authenticate users. Since Elasticsearch is stateless, this header must
be sent with every request:
[source,shell]
--------------------------------------------------
Authorization: Basic <TOKEN> <1>
--------------------------------------------------
<1> The `<TOKEN>` is computed as `base64(USERNAME:PASSWORD)`
[float]
==== Client examples
This example uses `curl` without basic auth to create an index:
[source,shell]
-------------------------------------------------------------------------------
curl -XPUT 'localhost:9200/idx'
-------------------------------------------------------------------------------
[source,js]
-------------------------------------------------------------------------------
{
"error": "AuthenticationException[Missing authentication token]",
"status": 401
}
-------------------------------------------------------------------------------
Since no user is associated with the request above, an authentication error is
returned. Now we'll use `curl` with basic auth to create an index as the
`rdeniro` user:
[source,shell]
---------------------------------------------------------
curl --user rdeniro:taxidriver -XPUT 'localhost:9200/idx'
---------------------------------------------------------
[source,js]
---------------------------------------------------------
{
"acknowledged": true
}
---------------------------------------------------------
[float]
==== Client Libraries over HTTP
For more information about how to use {security} with the language specific clients
please refer to
https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch-transport#authentication[Ruby],
http://elasticsearch-py.readthedocs.org/en/master/#ssl-and-authentication[Python],
https://metacpan.org/pod/Search::Elasticsearch::Cxn::HTTPTiny#CONFIGURATION[Perl],
http://www.elastic.co/guide/en/elasticsearch/client/php-api/current/_security.html[PHP],
http://nest.azurewebsites.net/elasticsearch-net/security.html[.NET],
http://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/auth-reference.html[Javascript]
////
Groovy - TODO link
////

259
docs/en/security/tribe-clients-integrations/java.asciidoc Normal file
View File

@ -0,0 +1,259 @@
[[java-clients]]
=== Java Client and Security
{security} supports the Java http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html[transport client] for Elasticsearch.
The transport client uses the same transport protocol that the cluster nodes use
for inter-node communication. It is very efficient as it does not have to marshall
and unmarshall JSON requests like a typical REST client.
NOTE: Using the Java Node Client with secured clusters is not recommended or
supported.
[float]
[[transport-client]]
==== Configuring the Transport Client to work with a Secured Cluster
To use the transport client with a secured cluster, you need to:
[[java-transport-client-role]]
. Configure a user with the privileges required to start the transport client.
A default `transport_client` role is built-in to {xpack} that grants the
appropriate cluster permissions for the transport client to work with the secured
cluster. The transport client uses the _Nodes Info API_ to fetch information about
the nodes in the cluster.
. Add the {xpack} transport JAR file to your CLASSPATH. You can download the {xpack}
distribution and extract the JAR file manually or you can get it from the
https://artifacts.elastic.co/maven/org/elasticsearch/client/x-pack-transport/{version}/x-pack-transport-{version}.jar[Elasticsearch Maven repository].
+
As with any dependency, you will also need its transitive dependencies. Refer to the
https://artifacts.elastic.co/maven/org/elasticsearch/client/x-pack-transport/{version}/x-pack-transport-{version}.pom[X-Pack POM file
for your version] when downloading for offline usage.
--
If you are using Maven, you need to add the {xpack} JAR file as a dependency in
your project's `pom.xml` file:
[source,xml]
--------------------------------------------------------------
<project ...>
<repositories>
<!-- add the elasticsearch repo -->
<repository>
<id>elasticsearch-releases</id>
<url>https://artifacts.elastic.co/maven</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
...
</repositories>
...
<dependencies>
<!-- add the x-pack jar as a dependency -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>x-pack-transport</artifactId>
<version>{version}</version>
</dependency>
...
</dependencies>
...
</project>
--------------------------------------------------------------
If you are using Gradle, you need to add the {xpack} JAR file as a dependency in
your `build.gradle` file:
[source,groovy]
--------------------------------------------------------------
repositories {
/* ... Any other repositories ... */
// Add the Elasticsearch Maven Repository
maven {
url "https://artifacts.elastic.co/maven"
}
}
dependencies {
compile "org.elasticsearch.client:x-pack-transport:{version}"
/* ... */
}
--------------------------------------------------------------
--
. Set up the transport client. At a minimum, you must configure `xpack.security.user` to
include the name and password of your transport client user in your requests. The
following snippet configures the user credentials globally--every request
submitted with this client includes the `transport_client_user` credentials in
its headers.
+
[source,java]
-------------------------------------------------------------------------------------------------
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
...
TransportClient client = new PreBuiltXPackTransportClient(Settings.builder()
.put("cluster.name", "myClusterName")
.put("xpack.security.user", "transport_client_user:changeme")
...
.build())
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
-------------------------------------------------------------------------------------------------
+
WARNING: If you configure a transport client without SSL, passwords are sent in
clear text.
+
You can also add an `Authorization` header to each request. If you've configured
global authorization credentials, the `Authorization` header overrides the global
authentication credentials. This is useful when an application has multiple users
who access Elasticsearch using the same client. You can set the global token to
a user that only has the `transport_client` role, and add the `transport_client`
role to the individual users.
+
For example, the following snippet adds the `Authorization` header to a search
request:
+
[source,java]
--------------------------------------------------------------------------------------------------
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.xpack.security.authc.support.SecuredString;
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
import static org.elasticsearch.xpack.security.authc.support.UsernamePasswordToken.basicAuthHeaderValue;
...
TransportClient client = new PreBuiltXPackTransportClient(Settings.builder()
.put("cluster.name", "myClusterName")
.put("xpack.security.user", "transport_client_user:changeme")
...
.build())
.build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300))
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9301))
String token = basicAuthHeaderValue("test_user", new SecuredString("changeme".toCharArray()));
client.filterWithHeader(Collections.singletonMap("Authorization", token))
.prepareSearch().get();
--------------------------------------------------------------------------------------------------
. Enable SSL to authenticate clients and encrypt communications. To enable SSL,
you need to:
.. Configure the paths to the client's key and certificate in addition to the certificate authorities.
Client authentication requires every client to have a certification signed by a trusted CA.
+
NOTE: Client authentication is enabled by default. For information about
disabling client authentication, see <<disabling-client-auth, Disabling Client Authentication>>.
+
[source,java]
--------------------------------------------------------------------------------------------------
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
...
TransportClient client = new PreBuiltXPackTransportClient(Settings.builder()
.put("cluster.name", "myClusterName")
.put("xpack.security.user", "transport_client_user:changeme")
.put("xpack.ssl.key", "/path/to/client.key")
.put("xpack.ssl.certificate", "/path/to/client.crt")
.put("xpack.ssl.certificate_authorities", "/path/to/ca.crt")
...
.build());
--------------------------------------------------------------------------------------------------
+
.. Enable the SSL transport by setting `xpack.security.transport.ssl.enabled` to `true` in the
client configuration.
+
[source,java]
--------------------------------------------------------------------------------------------------
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
...
TransportClient client = new PreBuiltXPackTransportClient(Settings.builder()
.put("cluster.name", "myClusterName")
.put("xpack.security.user", "transport_client_user:changeme")
.put("xpack.ssl.key", "/path/to/client.key")
.put("xpack.ssl.certificate", "/path/to/client.crt")
.put("xpack.ssl.certificate_authorities", "/path/to/ca.crt")
.put("xpack.security.transport.ssl.enabled", "true")
...
.build())
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300))
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9301))
--------------------------------------------------------------------------------------------------
[float]
[[disabling-client-auth]]
===== Disabling Client Authentication
If you want to disable client authentication, you can use a client-specific
transport protocol. For more information see <<separating-node-client-traffic, Separating Node to Node and Client Traffic>>.
If you are not using client authentication and sign the Elasticsearch node
certificates with your own CA, you need to provide the path to the CA
certificate in your client configuration.
[source,java]
------------------------------------------------------------------------------------------------------
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
...
TransportClient client = new PreBuiltXPackTransportClient(Settings.builder()
.put("cluster.name", "myClusterName")
.put("xpack.security.user", "test_user:changeme")
.put("xpack.ssl.certificate_authorities", "/path/to/ca.crt")
.put("xpack.security.transport.ssl.enabled", "true")
...
.build())
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
------------------------------------------------------------------------------------------------------
NOTE: If you are using a public CA that is already trusted by the Java runtime,
you do not need to set the `xpack.ssl.certificate_authorities`.
[float]
[[connecting-anonymously]]
===== Connecting Anonymously
To enable the transport client to connect anonymously, you must assign the
anonymous user the privileges defined in the <<java-transport-client-role,transport_client>>
role. Anonymous access must also be enabled, of course. For more information,
see <<anonymous-access,Enabling Anonymous Access>>.
[float]
[[security-client]]
==== Security Client
{security} exposes its own API through the `SecurityClient` class. To get a hold
of a `SecurityClient` you'll first need to create the `XPackClient`, which is a
wrapper around the existing Elasticsearch clients (any client class implementing
`org.elasticsearch.client.Client`).
The following example shows how you can clear {security}'s realm caches using
the `SecurityClient`:
[source,java]
------------------------------------------------------------------------------------------------------
Client client = ... // create the transport client
XPackClient xpackClient = new XPackClient(client);
SecurityClient securityClient = xpackClient.security();
ClearRealmCacheResponse response = securityClient.authc().prepareClearRealmCache()
.realms("ldap1", "ad1") <1>
.usernames("rdeniro")
.get();
------------------------------------------------------------------------------------------------------
<1> Clears the `ldap1` and `ad1` realm caches for the `rdeniro` user.

197
docs/en/security/tribe-clients-integrations/kibana.asciidoc Normal file
View File

@ -0,0 +1,197 @@
[[kibana]]
=== Kibana and Security
[[using-kibana-with-security]]
Kibana users have to log in when {security} is enabled on your cluster. You
configure {security} roles for your Kibana users to control what data those users
can access. You also need to configure credentials for the
Kibana server so the requests it submits to Elasticsearch on the user's
behalf can be authenticated.
To prevent user passwords from being sent in the clear, you must configure
Kibana to encrypt communications between the browser and the Kibana server.
If are encrypting traffic to and from the nodes in your Elasticsearch cluster,
you must also configure Kibana to connect to Elasticsearch via HTTPS.
With {security} enabled, if you load a Kibana dashboard that accesses data in an
index that you are not authorized to view, you get an error that indicates the
index does not exist. {security} do not currently provide a way to control which
users can load which dashboards.
IMPORTANT: Support for tribe nodes in Kibana was added in v5.2.
To use Kibana with {security}:
. Configure the password for the built-in `kibana` user. The Kibana server submits
requests as this user to access the cluster monitoring APIs and the `.kibana` index.
The server does _not_ need access to user indices.
+
By default, the `kibana` user password is set to `changeme`. Change this password
through the reset password API:
+
[source,shell]
--------------------------------------------------------------------------------
PUT /_xpack/security/user/kibana/_password
{
"password" : "s0m3th1ngs3cr3t"
}
--------------------------------------------------------------------------------
// CONSOLE
+
Once you change the password, you need to specify it with the `elasticsearch.password`
property in `kibana.yml`:
+
[source,yaml]
--------------------------------------------------------------------------------
elasticsearch.password: "s0m3th1ngs3cr3t"
--------------------------------------------------------------------------------
[[kibana-roles]]
. Assign the `kibana_user` role to grant Kibana users the privileges they
need to use Kibana.
+
IMPORTANT: You also need to grant Kibana users access to the
indices that they will be working with in Kibana.
+
** If you're using the `native` realm, you can assign roles using the
<<managing-native-users, User Management API>>. For example, the following
creates a user named `jacknich` and assigns it the `kibana_user` role:
+
[source,js]
--------------------------------------------------------------------------------
POST /_xpack/security/user/jacknich
{
"password" : "t0pS3cr3t",
"roles" : [ "kibana_user" ]
}
--------------------------------------------------------------------------------
// CONSOLE
** If you are using an LDAP or Active Directory realm, you can either assign
roles on a per user basis, or assign roles to groups of users. By default, role
mappings are stored in <<mapping-roles, `CONFIGDIR/x-pack/role_mapping.yml`>>.
For example, the following snippet assigns the `kibana_user` role to the
group named `admins` and the user named Jack Nicholson:
+
[source,yaml]
--------------------------------------------------------------------------------
kibana_user:
- "cn=admins,dc=example,dc=com"
- "cn=Jack Nicholson,dc=example,dc=com"
--------------------------------------------------------------------------------
[[configure-kibana-cert]]
. Configure Kibana to encrypt communications between the browser and the Kibana
server:
.. Generate a server certificate for Kibana. You must either set the certificate's
`subjectAltName` to the hostname, fully-qualified domain name (FQDN), or IP
address of the Kibana server, or set the CN to the Kibana server's hostname
or FQDN. Using the server's IP address as the CN does not work.
.. Set the `server.ssl.key` and `server.ssl.cert` properties in `kibana.yml`:
+
[source,yaml]
--------------------------------------------------------------------------------
server.ssl.key: /path/to/your/server.key
server.ssl.cert: /path/to/your/server.crt
--------------------------------------------------------------------------------
+
Once you enable SSL encryption between the browser and the Kibana server,
access Kibana via HTTPS. For example, `https://localhost:5601`.
+
NOTE: You must enable SSL encryption between the browser and the Kibana
server to use Kibana with {security} enabled. If {security} is configured to
encrypt connections to Elasticsearch, you must also <<configure-kibana-ssl,
configure Kibana to connect to Elasticsearch via HTTPS>>.
[[configure-kibana-ssl]]
. If you have enabled SSL encryption in {security}, configure Kibana to connect
to Elasticsearch via HTTPS:
.. Specify the HTTPS protocol in the `elasticsearch.url` setting in the Kibana
configuration file, `kibana.yml`:
+
[source,yaml]
--------------------------------------------------------------------------------
elasticsearch.url: "https://<your_elasticsearch_host>.com:9200"
--------------------------------------------------------------------------------
.. If you are using your own CA to sign certificates for Elasticsearch, set the
`elasticsearch.ssl.ca` setting in `kibana.yml` to specify the location of the PEM
file.
+
[source,yaml]
--------------------------------------------------------------------------------
elasticsearch.ssl.ca: /path/to/your/cacert.pem
--------------------------------------------------------------------------------
. Install {xpack} into Kibana to secure user sessions and enable users
to log in and out of Kibana:
.. Run the following command in your Kibana installation directory.
+
[source,console]
--------------------------------------------------------------------------------
bin/kibana-plugin install x-pack
--------------------------------------------------------------------------------
+
[NOTE]
=============================================================================
To perform an offline install, download the {xpack} zip file from
https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-{version}.zip[
+https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-{version}.zip+]
(https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-{version}.zip.sha1[sha1])
and run:
["source","sh",subs="attributes"]
---------------------------------------------------------
bin/kibana-plugin install file:///path/to/file/x-pack-{version}.zip
---------------------------------------------------------
=============================================================================
.. Set the `xpack.security.encryptionKey` property in the `kibana.yml` configuration file.
You can use any text string that is 32 characters or longer as the encryption key.
+
[source,yaml]
--------------------------------------------------------------------------------
xpack.security.encryptionKey: "something_at_least_32_characters"
--------------------------------------------------------------------------------
.. To change the default session duration, set the `xpack.security.sessionTimeout` property
in the `kibana.yml` configuration file. By default, sessions will stay active until the
browser is closed. The timeout is specified in milliseconds. For example, set the timeout
to 600000 to expire sessions after 10 minutes:
+
[source,yaml]
--------------------------------------------------------------------------------
xpack.security.sessionTimeout: 600000
--------------------------------------------------------------------------------
. Restart Kibana and verify that you can log in as a user. If you are running
Kibana locally, go to `https://localhost:5601` and enter the credentials for a
user you've assigned a Kibana user role. For example, you could log in as the
`jacknich` user created above.
+
image::kibana-login.jpg["Kibana Login",link="images/kibana-login.jpg"]
+
NOTE: This must be a user who has been assigned the `kibana_user` role.
Kibana server credentials should only be used internally by the
Kibana server.
[float]
[[security-ui-settings]]
===== Kibana {security} UI Settings
[options="header"]
|======
| Name | Default | Description
| `xpack.security.encryptionKey` | - | An arbitrary string of 32 characters or more used to encrypt credentials in a
cookie. It is crucial that this key is not exposed to
users of Kibana. Required.
| `xpack.security.sessionTimeout` | `1800000` (30 minutes) | Sets the session duration (in milliseconds).
| `xpack.security.cookieName` | `"sid"` | Sets the name of the cookie used for the session.
| `xpack.security.secureCookies` | `false` | Sets the `secure` flag of the session cookie. Is set
to `true` if `server.ssl.cert` and `server.ssl.key`
are set. Set this to `true` if SSL is configured
outside of Kibana (for example, you are routing
requests through a load balancer or proxy).
|======

219
docs/en/security/tribe-clients-integrations/logstash.asciidoc Normal file
View File

@ -0,0 +1,219 @@
[[logstash]]
=== Logstash and Security
The Logstash Elasticsearch plugins (
{logstash-ref}/plugins-outputs-elasticsearch.html[output],
{logstash-ref}/plugins-inputs-elasticsearch.html[input],
{logstash-ref}/plugins-filters-elasticsearch.html[filter]
and <<monitoring-logstash-settings, monitoring>>)
support authentication and encryption over HTTP.
To use Logstash with a secured cluster, you need to configure authentication
credentials for Logstash. Logstash throws an exception and the processing
pipeline is halted if authentication fails.
If encryption is enabled on the cluster, you also need to enable SSL in the
Logstash configuration.
If you wish to monitor your logstash instance with x-pack monitoring, and store
the monitoring data in a secured elasticsearch cluster, you must configure Logstash
with a username and password for a user with the appropriate permissions.
In addition to configuring authentication credentials for Logstash, you need
to grant authorized users permission to access the Logstash indices.
[float]
[[ls-http-auth-basic]]
==== Configuring Logstash to use Basic Authentication
Logstash needs to be able to manage index templates, create indices,
and write and delete documents in the indices it creates.
To set up authentication credentials for Logstash:
. Create a `logstash_writer` role that has the `manage_index_templates` cluster
privilege, and the `write`, `delete`, and `create_index` privileges for the
Logstash indices. You can create roles from the **Management > Roles** UI in
Kibana or through the `role` API:
+
[source, sh]
---------------------------------------------------------------
POST _xpack/security/role/logstash_writer
{
"cluster": ["manage_index_templates", "monitor"],
"indices": [
{
"names": [ "logstash-*" ], <1>
"privileges": ["write","delete","create_index"]
}
]
}
---------------------------------------------------------------
<1> If you use a custom Logstash index pattern, specify that pattern
instead of the default `logstash-*` pattern.
. Create a `logstash_internal` user and assign it the `logstash_writer` role.
You can create users from the **Management > Users** UI in Kibana or through
the `user` API:
+
[source, sh]
---------------------------------------------------------------
POST _xpack/security/user/logstash_internal
{
"password" : "changeme",
"roles" : [ "logstash_writer"],
"full_name" : "Internal Logstash User"
}
---------------------------------------------------------------
. Configure Logstash to authenticate as the `logstash_internal` user you just
created. You configure credentials separately for each of the Elasticsearch
plugins in your Logstash `.conf` file. For example:
+
[source,js]
--------------------------------------------------
input {
...
user => logstash_internal
password => changeme
}
filter {
...
user => logstash_internal
password => changeme
}
output {
elasticsearch {
...
user => logstash_internal
password => changeme
}
--------------------------------------------------
[float]
[[ls-user-access]]
==== Granting Users Access to the Logstash Indices
To access the indices Logstash creates, users need the `read` and
`view_index_metadata` privileges:
. Create a `logstash_reader` role that has the `read and `view_index_metadata`
privileges for the Logstash indices. You can create roles from the
**Management > Roles** UI in Kibana or through the `role` API:
+
[source, sh]
---------------------------------------------------------------
POST _xpack/security/role/logstash_reader
{
"indices": [
{
"names": [ "logstash-*" ], <1>
"privileges": ["read","view_index_metadata"]
}
]
}
---------------------------------------------------------------
<1> If you use a custom Logstash index pattern, specify that pattern
instead of the default `logstash-*` pattern.
. Assign your Logstash users the `logstash_reader` role. You can create
and manage users from the **Management > Users** UI in Kibana or through
the `user` API:
+
[source, sh]
---------------------------------------------------------------
POST _xpack/security/user/logstash_user
{
"password" : "changeme",
"roles" : [ "logstash_reader"],
"full_name" : "Kibana User"
}
---------------------------------------------------------------
[float]
[[ls-http-auth-pki]]
===== Configuring the elasticsearch Output to use PKI Authentication
The `elasticsearch` output supports PKI authentication. To use an X.509
client-certificate for authentication, you configure the `keystore` and
`keystore_password` options in your Logstash `.conf` file:
[source,js]
--------------------------------------------------
output {
elasticsearch {
...
keystore => /path/to/keystore.jks
keystore_password => realpassword
truststore => /path/to/truststore.jks <1>
truststore_password => realpassword
}
}
--------------------------------------------------
<1> If you use a separate truststore, the truststore path and password are
also required.
[float]
[[ls-http-ssl]]
===== Configuring Logstash to use TLS Encryption
If TLS encryption is enabled on the Elasticsearch cluster, you need to
configure the `ssl` and `cacert` options in your Logstash `.conf` file:
[source,js]
--------------------------------------------------
output {
elasticsearch {
...
ssl => true
cacert => '/path/to/cert.pem' <1>
}
}
--------------------------------------------------
<1> The path to the local `.pem` file that contains the Certificate
Authority's certificate.
[float]
[[ls-monitoring-user]]
===== Configuring Logstash Monitoring
If you wish to ship Logstash <<monitoring-logstash-settings, monitoring>>
data to a secure cluster, Logstash must be configured with a username and password.
X-Pack security comes preconfigured with a `logstash_system` user for this purpose.
This user has the minimum permissions necessary for the monitoring function, and
_should not_ be used for any other purpose - it is specifically _not intended_ for
use within a Logstash pipeline.
By default, the `logstash_system` user password is set to `changeme`.
Change this password through the reset password API:
[source,js]
---------------------------------------------------------------------
PUT _xpack/security/user/logstash_system/_password
{
"password": "t0p.s3cr3t"
}
---------------------------------------------------------------------
// CONSOLE
Then configure the user and password in your `logstash.yml` configuration file:
[source,yaml]
----------------------------------------------------------
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: t0p.s3cr3t
----------------------------------------------------------
If you initially installed an older version of X-Pack, and then upgraded, then
the `logstash_system` user may have defaulted to disabled for security reasons.
You can enable the user with the following API call:
[source,js]
---------------------------------------------------------------------
PUT _xpack/security/user/logstash_system/_enable
---------------------------------------------------------------------
// CONSOLE

181
docs/en/security/tribe-clients-integrations/monitoring.asciidoc Normal file
View File

@ -0,0 +1,181 @@
[[secure-monitoring]]
=== Monitoring and Security
<<xpack-monitoring, {monitoring}>> consists of two components: an agent
that you install on on each Elasticsearch and Logstash node, and a Monitoring UI
in Kibana. The monitoring agent collects and indexes metrics from the nodes
and you visualize the data through the Monitoring dashboards in Kibana. The agent
can index data on the same Elasticsearch cluster, or send it to an external
monitoring cluster.
To use {monitoring} with {security} enabled, you need to
<<kibana, set up Kibana to work with {security}>> and create at least one user
for the Monitoring UI. If you are using an external monitoring cluster, you also
need to configure a user for the monitoring agent and configure the agent to use
the appropriate credentials when communicating with the monitoring cluster.
[float]
[[monitoring-ui-users]]
==== Setting Up Monitoring UI Users
When {security} is enabled, Kibana users are prompted to log in when they access
the UI. To use the Monitoring UI, a user must have access to the Kibana indices
and permission to read from the monitoring indices.
You set up Monitoring UI users on the cluster where the monitoring data is being
stored. To grant all of the necessary permissions, assign the user the
`monitoring_user` and `kibana_user` roles:
* If you're using the `native` realm, you can assign roles through Kibana or
with the <<managing-native-users, User Management API>>. For example, the following
command creates a user named `jacknich` and assigns him the `kibana_user` and
`monitoring_user` roles:
+
[source,js]
--------------------------------------------------------------------------------
POST /_xpack/security/user/jacknich
{
"password" : "t0pS3cr3t",
"roles" : [ "kibana_user", "monitoring_user" ]
}
--------------------------------------------------------------------------------
* If you are using an LDAP or Active Directory realm, you can either assign roles
on a per user basis, or assign roles to groups of users. By default, role mappings
are configured in <<mapping-roles, `config/x-pack/role_mapping.yml`>>. For example,
the following snippet assigns the user named Jack Nicholson to the `kibana_user`
and `monitoring_user` roles:
+
[source,yaml]
--------------------------------------------------------------------------------
kibana_user:
- "cn=Jack Nicholson,dc=example,dc=com"
monitoring_user:
- "cn=Jack Nicholson,dc=example,dc=com"
--------------------------------------------------------------------------------
[float]
[[configuring-monitoring-agent-security]]
==== Configuring Monitoring Agent to Communicate with a {security}-Enabled Monitoring Cluster
To configure the monitoring agent to communicate with a secured monitoring cluster:
. Configure a user on the monitoring cluster who has the `remote_monitoring_agent`
role, which is <<[[built-in-roles-remote-monitoring-agent]], built-in to {xpack}>>.
For example:
+
[source,js]
--------------------------------------------------------------------------------
POST /_xpack/security/user/agent-user
{
"password" : "t0pS3cr3t",
"roles" : [ "remote_monitoring_agent" ]
}
--------------------------------------------------------------------------------
+
. On each node in the cluster being monitored, configure a Monitoring HTTP exporter
in `elasticsearch.yml` and restart Elasticsearch. In the exporter configuration,
you need to:
+
--
.. Set the `type` to `http`.
.. Specify the location of the monitoring cluster in the `host` setting.
.. Provide the agent user credentials with the `username` and `password` settings.
For example:
[source,yaml]
--------------------------------------------------
xpack.monitoring.exporters:
id1:
type: http
host: ["http://es-mon1:9200", "http://es-mon2:9200"]
auth:
username: agent-user
password: password
--------------------------------------------------
If SSL/TLS is enabled on the monitoring cluster:
.. Specify the HTTPS protocol when setting the monitoring server host.
.. Include the CA certificate in each node's trusted certificates in order to verify
the identities of the nodes in the monitoring cluster.
To add a CA certificate to an Elasticsearch node's trusted certificates, you
can specify the location of the PEM encoded certificate with the
`certificate_authorities` setting:
[source,yaml]
--------------------------------------------------
xpack.monitoring.exporters:
id1:
type: http
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
auth:
username: agent-user
password: password
ssl:
certificate_authorities: [ "/path/to/ca.crt" ]
id2:
type: local
--------------------------------------------------
Alternatively, you can configure trusted certificates using a truststore
(a Java Keystore file that contains the certificates):
[source,yaml]
--------------------------------------------------
xpack.monitoring.exporters:
id1:
type: http
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
auth:
username: agent-user
password: password
ssl:
truststore.path: /path/to/file
truststore.password: password
id2:
type: local
--------------------------------------------------
--
. On each Logstash node being monitored, update `logstash.yml` to:
+
--
.. Specify the location of the monitoring cluster and provide credentials
for the agent user:
[source,yaml]
--------------------------------------------------
xpack.monitoring.elasticsearch.url: ["http://es-mon-1:9200", "http://es-mon2:9200"]
xpack.monitoring.elasticsearch.username: "remote_monitor"
xpack.monitoring.elasticsearch.password: "changeme"
--------------------------------------------------
.. If SSL/TLS is enabled on the monitoring cluster:
* Specify the HTTPS protocol when setting the `elasticsearch.url`.
* Include the CA certificate in each node's trusted certificates in order to verify
the identities of the nodes in the monitoring cluster.
To add a CA certificate to an node's trusted certificates, you
can specify the location of the PEM encoded certificate with the
`xpack.monitoring.elasticsearch.ssl.ca` setting:
[source,yaml]
--------------------------------------------------
xpack.monitoring.elasticsearch.ssl.ca: [ "/path/to/ca.crt" ]
--------------------------------------------------
Alternatively, you can configure trusted certificates using a truststore
(a Java Keystore file that contains the certificates):
[source,yaml]
--------------------------------------------------
xpack.monitoring.elasticsearch.ssl.truststore.path: /path/to/file
xpack.monitoring.elasticsearch.ssl.truststore.password: changeme
--------------------------------------------------
--

44
docs/en/security/tribe-clients-integrations/reporting.asciidoc Normal file
View File

@ -0,0 +1,44 @@
[[secure-reporting]]
=== Reporting and Security
Reporting operates by creating and updating documents in Elasticsearch in
response to user actions in Kibana.
To use Reporting with {security} enabled, you need to <<kibana, set up Kibana
to work with {security}>>. If you are automatically generating reports with
<<xpack-alerting, {watcher}>>, you also need to configure {watcher} to trust the
Kibana server's certificate. For more information, see <<securing-reporting,
Securing Reporting>>.
[[reporting-app-users]]
To enable users to generate reports, assign them the built in `reporting_user`
and `kibana_user` roles:
* If you're using the `native` realm, you can assign roles through
**Management / Users** UI in Kibana or with the `user` API. For example,
the following request creates a `reporter` user that has the
`reporting_user` and `kibana_user` roles:
+
[source, sh]
---------------------------------------------------------------
POST /_xpack/security/user/reporter
{
"password" : "changeme",
"roles" : ["kibana_user", "reporting_user"],
"full_name" : "Reporting User"
}
---------------------------------------------------------------
* If you are using an LDAP or Active Directory realm, you can either assign
roles on a per user basis, or assign roles to groups of users. By default, role
mappings are configured in <<mapping-roles, `config/shield/role_mapping.yml`>>.
For example, the following snippet assigns the user named Bill Murray the
`kibana_user` and `reporting_user` roles:
+
[source,yaml]
--------------------------------------------------------------------------------
kibana_user:
- "cn=Bill Murray,dc=example,dc=com"
reporting_user:
- "cn=Bill Murray,dc=example,dc=com"
--------------------------------------------------------------------------------

109
docs/en/security/tribe-clients-integrations/tribe.asciidoc Normal file
View File

@ -0,0 +1,109 @@
[[tribe-node-configuring]]
=== Tribe Nodes and Security
{ref}/modules-tribe.html[Tribe nodes] act as a federated client across multiple
clusters. When using tribe nodes with secured clusters, all clusters must have
{security} enabled and share the same security configuration (users, roles,
user-role mappings, SSL/TLS CA). The tribe node itself also must be configured
to grant access to actions and indices on all of the connected clusters, as
security checks on incoming requests are primarily done on the tribe node
itself.
IMPORTANT: Support for tribe nodes in Kibana was added in v5.2.
To use a tribe node with secured clusters:
. Install {xpack} on the tribe node and every node in each connected cluster.
. Enable <<enable-message-authentication, message authentication>> globally.
Generate a system key on one node and copy it to the tribe node and every other
node in each of the connected clusters.
+
IMPORTANT: For message authentication to work properly across multiple clusters,
the tribe node and all of the connected clusters must share the same
system key. {security} reads the system key from `CONFIG_DIR/x-pack/system_key`.
. Enable encryption globally. To encrypt communications, you must enable
<<ssl-tls,enable SSL/TLS>> on every node.
+
TIP: To simplify SSL/TLS configuration, use the same certificate authority to
generate certificates for all connected clusters.
. Configure the tribe in the tribe node's `elasticsearch.yml` file. You must
specify each cluster that is a part of the tribe and configure discovery and
encryption settings per cluster. For example, the following configuration adds
two clusters to the tribe:
+
[source,yml]
-----------------------------------------------------------
tribe:
on_conflict: prefer_cluster1 <1>
c1: <2>
cluster.name: cluster1
discovery.zen.ping.unicast.hosts: [ "cluster1-node1:9300", "cluster1-node2:9300"]
xpack.ssl.key: /home/es/config/x-pack/es-tribe-01.key
xpack.ssl.certificate: /home/es/config/x-pack/es-tribe-01.crt
xpack.ssl.certificate_authorities: [ "/home/es/config/x-pack/ca.crt" ]
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true
c2:
cluster.name: cluster2
discovery.zen.ping.unicast.hosts: [ "cluster2-node1:9300", "cluster2-node2:9300"]
xpack.ssl.key: /home/es/config/x-pack/es-tribe-01.key
xpack.ssl.certificate: /home/es/config/x-pack/es-tribe-01.crt
xpack.ssl.certificate_authorities: [ "/home/es/config/x-pack/ca.crt" ]
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true
-----------------------------------------------------------
<1> Results are returned from the preferred cluster if the named index exists
in multiple clusters. A preference is *required* when using {security} on
a tribe node.
<2> An arbitrary name that represents the connection to the cluster.
. Configure the same index privileges for your users on all nodes, including the
tribe node. The nodes in each cluster must grant access to indices in other
connected clusters as well as their own.
+
For example, let's assume `cluster1` and `cluster2` each have a indices `index1`
and `index2`. To enable a user to submit a request through the tribe node to
search both clusters:
+
--
.. On the tribe node and both clusters, <<defining-roles, define a `tribe_user` role>>
that has read access to `index1` and `index2`:
+
[source,yaml]
-----------------------------------------------------------
tribe_user:
indices:
'index*': search
-----------------------------------------------------------
.. Assign the `tribe_user` role to a user on the tribe node and both clusters.
For example, run the following command on each node to create `my_tribe_user`
and assign the `tribe_user` role:
+
[source,shell]
-----------------------------------------------------------
./bin/shield/users useradd my_tribe_user -p password -r tribe_user
-----------------------------------------------------------
+
NOTE: Each cluster needs to have its own users with admin privileges.
You cannot perform administration tasks such as create index through
the tribe node, you must send the request directly to the appropriate
cluster.
--
. To enable selected users to retrieve merged cluster state information
for the tribe from the tribe node, grant them the cluster
<<privileges-list-cluster, `monitor` privilege>> on the tribe node. For example,
you could create a `tribe_monitor` role that assigns the `monitor` privilege:
+
[source,yaml]
-----------------------------------------------------------
tribe_monitor:
cluster: monitor
-----------------------------------------------------------
. Start the tribe node. If you've made configuration changes to the nodes in the
connected clusters, they also need to be restarted.

229
docs/en/security/troubleshooting.asciidoc Normal file
View File

@ -0,0 +1,229 @@
[[security-troubleshooting]]
== {security} Troubleshooting
[float]
=== `settings`
Some settings are not returned via the nodes settings API::
+
--
This is intentional. Some of the settings are considered to be highly
sensitive: all `ssl` settings, ldap `bind_dn`, `bind_password`).
For this reason, we filter these settings and do not expose them via
the nodes info API rest endpoint. You can also define additional
sensitive settings that should be hidden using the
`xpack.security.hide_settings` setting. For example, this snippet
hides the `url` settings of the `ldap1` realm and all settings of the
`ad1` realm.
[source, yaml]
------------------------------------------
xpack.security.hide_settings: xpack.security.authc.realms.ldap1.url, xpack.security.authc.realms.ad1.*
------------------------------------------
--
[float]
=== `users`
I configured the appropriate roles and the users, but I still get an authorization exception::
+
--
Verify that the role names associated with the users match the roles defined in the `roles.yml` file. You
can use the `users` tool to list all the users. Any unknown roles are marked with `*`.
[source, shell]
------------------------------------------
bin/xpack/users list
rdeniro : admin
alpacino : power_user
jacknich : monitoring,unknown_role* <1>
------------------------------------------
<1> `unknown_role` was not found in `roles.yml`
--
ERROR: extra arguments [...] were provided::
+
--
This error occurs when the `users` tool is parsing the input and finds unexepected arguments. This can happen when there
are special characters used in some of the arguments. For example, on Windows systems the `,` character is considered
a parameter separator; in other words `-r role1,role2` is translated to `-r role1 role2` and the `users` tool only recognizes
`role1` as an expected parameter. The solution here is to quote the parameter: `-r "role1,role2"`.
--
[[trouble-shoot-active-directory]]
[float]
=== Active Directory
Certain users are being frequently locked out of Active Directory::
+
--
Check your realm configuration; realms are checked serially, one after another. If your Active Directory realm is being checked before other realms and there are usernames
that appear in both Active Directory and another realm, a valid login for one realm may be causing failed login attempts in another realm.
For example, if `UserA` exists in both Active Directory and a file realm, and the Active Directory realm is checked first and
file is checked second, an attempt to authenticate as `UserA` in the file realm would first attempt to authenticate
against Active Directory and fail, before successfully authenticating against the `file` realm. Because authentication is
verified on each request, the Active Directory realm would be checked - and fail - on each request for `UserA` in the `file`
realm. In this case, while the authentication request completed successfully, the account on Active Directory would have received
several failed login attempts, and that account may become temporarily locked out. Plan the order of your realms accordingly.
Also note that it is not typically necessary to define multiple Active Directory realms to handle domain controller failures. When using Microsoft DNS, the DNS entry for the domain should always point to an available domain controller.
--
[float]
=== LDAP
I can authenticate to LDAP, but I still get an authorization exception::
+
--
A number of configuration options can cause this error.
|======================
|_group identification_ |
Groups are located by either an LDAP search or by the "memberOf" attribute on
the user. Also, If subtree search is turned off, it will search only one
level deep. See the <<ldap-settings, LDAP Settings>> for all the options.
There are many options here and sticking to the defaults will not work for all
scenarios.
| _group to role mapping_|
Either the `role_mapping.yml` file or the location for this file could be
misconfigured. See <<security-files, Security Files>> for more.
|_role definition_|
The role definition may be missing or invalid.
|======================
To help track down these possibilities, add the following lines to the end of the `log4j2.properties` configuration file in the
`CONFIG_DIR`:
[source,properties]
----------------
logger.authc.name = org.elasticsearch.xpack.security.authc
logger.authc.level = DEBUG
----------------
A successful authentication should produce debug statements that list groups and role mappings.
--
[float]
=== Encryption & Certificates
`curl` on the Mac returns a certificate verification error even when the `--cacert` option is used::
+
--
Apple's integration of `curl` with their keychain technology disables the `--cacert` option.
See http://curl.haxx.se/mail/archive-2013-10/0036.html for more information.
You can use another tool, such as `wget`, to test certificates. Alternately, you can add the certificate for the
signing certificate authority MacOS system keychain, using a procedure similar to the one detailed at the
http://support.apple.com/kb/PH14003[Apple knowledge base]. Be sure to add the signing CA's certificate and not the server's certificate.
--
[float]
==== SSLHandshakeException causing connections to fail
A `SSLHandshakeException` will cause a connection to a node to fail and indicates that there is a configuration issue. Some of the
common exceptions are shown below with tips on how to resolve these issues.
`java.security.cert.CertificateException: No name matching node01.example.com found`::
+
--
Indicates that a client connection was made to `node01.example.com` but the certificate returned did not contain the name `node01.example.com`.
In most cases, the issue can be resolved by ensuring the name is specified during <<generating-signed-certificates, certificate creation>>.
Another scenario is when the environment does not wish to use DNS names in certificates at all. In this scenario, all settings
in `elasticsearch.yml` should only use IP addresses including the `network.publish_host` setting.
--
`java.security.cert.CertificateException: No subject alternative names present`::
+
--
Indicates that a client connection was made to an IP address but the returned certificate did not contain any `SubjectAlternativeName` entries.
IP addresses are only used for hostname verification if they are specified as a `SubjectAlternativeName` during
<<generating-signed-certificates, certificate creation>>. If the intent was to use IP addresses for hostname verification, then the certificate
will need to be regenerated with the appropriate IP address.
--
`javax.net.ssl.SSLHandshakeException: null cert chain` and `javax.net.ssl.SSLException: Received fatal alert: bad_certificate`::
+
--
The `SSLHandshakeException` above indicates that a self-signed certificate was returned by the client that is not trusted
as it cannot be found in the `truststore` or `keystore`. The `SSLException` above is seen on the client side of the connection.
--
`sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target` and `javax.net.ssl.SSLException: Received fatal alert: certificate_unknown`::
+
--
The `SunCertPathBuilderException` above indicates that a certificate was returned during the handshake that is not trusted.
This message is seen on the client side of the connection. The `SSLException` above is seen on the server side of the
connection. The CA certificate that signed the returned certificate was not found in the `keystore` or `truststore` and
needs to be added to trust this certificate.
--
[float]
==== Other SSL/TLS related exceptions
The are other exceptions related to SSL that may be seen in the logs. Below you will find some common exceptions and their
meaning.
WARN: received plaintext http traffic on a https channel, closing connection::
+
--
Indicates that there was an incoming plaintext http request. This typically occurs when an external applications attempts
to make an unencrypted call to the REST interface. Please ensure that all applications are using `https` when calling the
REST interface with SSL enabled.
--
`org.elasticsearch.common.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:`::
+
--
Indicates that there was incoming plaintext traffic on an SSL connection. This typically occurs when a node is not
configured to use encrypted communication and tries to connect to nodes that are using encrypted communication. Please
verify that all nodes are using the same setting for `xpack.security.transport.ssl.enabled`.
--
`java.io.StreamCorruptedException: invalid internal transport message format, got`::
+
--
Indicates an issue with data received on the transport interface in an unknown format. This can happen when a node with
encrypted communication enabled connects to a node that has encrypted communication disabled. Please verify that all
nodes are using the same setting for `xpack.security.transport.ssl.enabled`.
--
`java.lang.IllegalArgumentException: empty text`::
+
--
The exception is typically seen when a `https` request is made to a node that is not using `https`. If `https` is desired,
please ensure the following setting is in `elasticsearch.yml`:
[source,yaml]
----------------
xpack.security.http.ssl.enabled: true
----------------
--
ERROR: unsupported ciphers [...] were requested but cannot be used in this JVM::
+
--
This error occurs when a SSL/TLS cipher suite is specified that cannot supported by the JVM that Elasticsearch is running
in. Security will try to use the specified cipher suites that are supported by this JVM. This error can occur when using
the Security defaults as some distributions of OpenJDK do not enable the PKCS11 provider by default. In this case, we
recommend consulting your JVM documentation for details on how to enable the PKCS11 provider.
Another common source of this error is requesting cipher suites that use encrypting with a key length greater than 128 bits
when running on an Oracle JDK. In this case, you will need to install the <<ciphers, JCE Unlimited Strength Jurisdiction Policy Files>>.
--
[float]
==== Internal Server Error in Kibana
If the Security plugin is enabled in Elasticsearch but disabled in Kibana, you must
still set `elasticsearch.username` and `elasticsearch.password` in `kibana.yml`.
Otherwise, Kibana cannot connect to Elasticsearch. In 5.1.1, this results in an
`UnhandledPromiseRejectionWarning` and Kibana displays an Internal Server Error.

143
docs/en/security/using-ip-filtering.asciidoc Normal file
View File

@ -0,0 +1,143 @@
[[ip-filtering]]
== Restricting Connections with IP Filtering
You can apply IP filtering to application clients, node clients, or transport
clients, in addition to other nodes that are attempting to join the cluster.
If a node's IP address is on the blacklist, {security} will still allow the
connection to Elasticsearch, but it will be dropped immediately, and no requests
will be processed.
NOTE: Elasticsearch installations are not designed to be publicly accessible
over the Internet. IP Filtering and the other security capabilities of
{security} do not change this condition.
[float]
=== Enabling IP filtering
{security} features an access control feature that allows or rejects hosts,
domains, or subnets.
You configure IP filtering by specifying the `xpack.security.transport.filter.allow` and
`xpack.security.transport.filter.deny` settings in in `elasticsearch.yml`. Allow rules
take precedence over the deny rules.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.allow: "192.168.0.1"
xpack.security.transport.filter.deny: "192.168.0.0/24"
--------------------------------------------------
The `_all` keyword can be used to deny all connections that are not explicitly
allowed.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.allow: [ "192.168.0.1", "192.168.0.2", "192.168.0.3", "192.168.0.4" ]
xpack.security.transport.filter.deny: _all
--------------------------------------------------
IP filtering configuration also support IPv6 addresses.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.allow: "2001:0db8:1234::/48"
xpack.security.transport.filter.deny: "1234:0db8:85a3:0000:0000:8a2e:0370:7334"
--------------------------------------------------
You can also filter by hostnames when DNS lookups are available.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.allow: localhost
xpack.security.transport.filter.deny: '*.google.com'
--------------------------------------------------
[float]
=== Disabling IP Filtering
Disabling IP filtering can slightly improve performance under some conditions.
To disable IP filtering entirely, set the value of the `xpack.security.transport.filter.enabled`
setting in the `elasticsearch.yml` configuration file to `false`.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.enabled: false
--------------------------------------------------
You can also disable IP filtering for the transport protocol but enable it for
HTTP only.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.enabled: false
xpack.security.http.filter.enabled: true
--------------------------------------------------
[float]
=== Specifying TCP transport profiles
{ref}/modules-transport.html#_tcp_transport_profiles[TCP transport profiles]
enable Elasticsearch to bind on multiple hosts. {security} enables you to apply
different IP filtering on different profiles.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.allow: 172.16.0.0/24
xpack.security.transport.filter.deny: _all
transport.profiles.client.xpack.security.filter.allow: 192.168.0.0/24
transport.profiles.client.xpack.security.filter.deny: _all
--------------------------------------------------
NOTE: When you do not specify a profile, `default` is used automatically.
[float]
=== HTTP Filtering
You may want to have different IP filtering for the transport and HTTP protocols.
[source,yaml]
--------------------------------------------------
xpack.security.transport.filter.allow: localhost
xpack.security.transport.filter.deny: '*.google.com'
xpack.security.http.filter.allow: 172.16.0.0/16
xpack.security.http.filter.deny: _all
--------------------------------------------------
[float]
[[dynamic-ip-filtering]]
==== Dynamically updating ip filter settings
In case of running in an environment with highly dynamic IP addresses like cloud
based hosting, it is very hard to know the IP addresses upfront when provisioning
a machine. Instead of changing the configuration file and restarting the node,
you can use the _Cluster Update Settings API_. For example:
[source,js]
--------------------------------------------------
PUT /_cluster/settings
{
"persistent" : {
"xpack.security.transport.filter.allow" : "172.16.0.0/24"
}
}
--------------------------------------------------
// CONSOLE
You can also dynamically disable filtering completely:
[source,js]
--------------------------------------------------
PUT /_cluster/settings
{
"persistent" : {
"xpack.security.transport.filter.enabled" : false
}
}
--------------------------------------------------
// CONSOLE
// TEST[continued]
NOTE: In order to avoid locking yourself out of the cluster, the default bound
transport address will never be denied. This means you can always SSH into
a system and use curl to apply changes.