433 lines
21 KiB
Plaintext
433 lines
21 KiB
Plaintext
[[auditing]]
|
|
== Auditing Security Events
|
|
|
|
You can enable auditing to keep track of security-related events such as
|
|
authentication failures and refused connections. Logging these events enables you
|
|
to monitor your cluster for suspicious activity and provides evidence in the
|
|
event of an attack.
|
|
|
|
[IMPORTANT]
|
|
============================================================================
|
|
Audit logs are **disabled** by default. To enable this functionality, you
|
|
must set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`.
|
|
============================================================================
|
|
|
|
{Security} provides two ways to persist audit logs:
|
|
|
|
* The <<audit-log-output, `logfile`>> output, which persists events to
|
|
a dedicated `<clustername>_access.log` file on the host's file system.
|
|
* The <<audit-index, `index`>> output, which persists events to an Elasticsearch index.
|
|
The audit index can reside on the same cluster, or a separate cluster.
|
|
|
|
By default, only the `logfile` output is used when enabling auditing.
|
|
To facilitate browsing and analyzing the events, you can also enable
|
|
indexing by setting `xpack.security.audit.outputs` in `elasticsearch.yml`:
|
|
|
|
[source,yaml]
|
|
----------------------------
|
|
xpack.security.audit.outputs: [ index, logfile ]
|
|
----------------------------
|
|
|
|
The `index` output type should be used in conjunction with the `logfile`
|
|
output type Because it is possible for the `index` output type to lose
|
|
messages if the target index is unavailable, the `access.log` should be
|
|
used as the official record of events.
|
|
|
|
NOTE: Audit events are batched for indexing so there is a lag before
|
|
events appear in the index. You can control how frequently batches of
|
|
events are pushed to the index by setting
|
|
`xpack.security.audit.index.flush_interval` in `elasticsearch.yml`.
|
|
|
|
[float]
|
|
[[audit-event-types]]
|
|
=== Audit Event Types
|
|
|
|
Each request may generate multiple audit events.
|
|
The following is a list of the events that can be generated:
|
|
|
|
|======
|
|
| `anonymous_access_denied` | | | Logged when a request is denied due to a missing
|
|
authentication token.
|
|
| `authentication_success` | | | Logged when a user successfully authenticates.
|
|
| `authentication_failed` | | | Logged when the authentication token cannot be
|
|
matched to a known user.
|
|
| `realm_authentication_failed` | | | Logged for every realm that fails to present a valid
|
|
authentication token. `<realm>` represents the
|
|
realm type.
|
|
| `access_denied` | | | Logged when an authenticated user attempts to execute
|
|
an action they do not have the necessary
|
|
<<security-reference, privilege>> to perform.
|
|
| `access_granted` | | | Logged when an authenticated user attempts to execute
|
|
an action they have the necessary privilege to perform.
|
|
When the `system_access_granted` event is included, all system
|
|
(internal) actions are also logged. The default setting does
|
|
not log system actions to avoid cluttering the logs.
|
|
| `run_as_granted` | | | Logged when an authenticated user attempts to <<run-as-privilege, run as>>
|
|
another user that they have the necessary privileges to do.
|
|
| `run_as_denied` | | | Logged when an authenticated user attempts to <<run-as-privilege, run as>>
|
|
another user action they do not have the necessary
|
|
<<security-reference, privilege>> to do so.
|
|
| `tampered_request` | | | Logged when {security} detects that the request has
|
|
been tampered with. Typically relates to `search/scroll`
|
|
requests when the scroll ID is believed to have been
|
|
tampered with.
|
|
| `connection_granted` | | | Logged when an incoming TCP connection passes the
|
|
<<ip-filtering, IP Filter>> for a specific
|
|
profile.
|
|
| `connection_denied` | | | Logged when an incoming TCP connection does not pass the
|
|
<<ip-filtering, IP Filter>> for a specific
|
|
profile.
|
|
|======
|
|
|
|
[float]
|
|
[[audit-event-attributes]]
|
|
=== Audit Event Attributes
|
|
|
|
The following table shows the common attributes that can be associated with every event.
|
|
|
|
.Common Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `timestamp` | When the event occurred.
|
|
| `node_name` | The name of the node.
|
|
| `node_host_name` | The hostname of the node.
|
|
| `node_host_address` | The IP address of the node.
|
|
| `layer` | The layer from which this event originated: `rest`, `transport` or `ip_filter`
|
|
| `event_type` | The type of event that occurred: `anonymous_access_denied`,
|
|
`authentication_failed`, `access_denied`, `access_granted`,
|
|
`connection_granted`, `connection_denied`, `tampered_request`,
|
|
`run_as_granted`, `run_as_denied`.
|
|
|======
|
|
|
|
The following tables show the attributes that can be associated with each type of event.
|
|
The log level determines which attributes are included in a log entry.
|
|
|
|
.REST anonymous_access_denied Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `uri` | The REST endpoint URI.
|
|
| `request_body` | The body of the request, if enabled.
|
|
|======
|
|
|
|
.REST authentication_success Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `user` | The authenticated user.
|
|
| `realm` | The realm that authenticated the user.
|
|
| `uri` | The REST endpoint URI.
|
|
| `params` | The REST URI query parameters.
|
|
| `request_body` | The body of the request, if enabled.
|
|
|======
|
|
|
|
.REST authentication_failed Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `principal` | The principal (username) that failed authentication.
|
|
| `uri` | The REST endpoint URI.
|
|
| `request_body` | The body of the request, if enabled.
|
|
|======
|
|
|
|
.REST realm_authentication_failed Attributes
|
|
[cols="2,7",options="header"]
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `principal` | The principal (username) that failed authentication.
|
|
| `uri` | The REST endpoint URI.
|
|
| `request_body` | The body of the request, if enabled.
|
|
| `realm` | The realm that failed to authenticate the user.
|
|
NOTE: A separate entry is logged for each
|
|
consulted realm.
|
|
|======
|
|
|
|
.Transport anonymous_access_denied Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_type` | Where the request originated: `rest` (request
|
|
originated from a REST API request), `transport`
|
|
(request was received on the transport channel),
|
|
`local_node` (the local node issued the request).
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `action` | The name of the action that was executed.
|
|
| `request` | The type of request that was executed.
|
|
| `indices` | A comma-separated list of indices this request
|
|
pertains to (when applicable).
|
|
|======
|
|
|
|
.Transport authentication_success Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_type` | Where the request originated: `rest` (request
|
|
originated from a REST API request), `transport`
|
|
(request was received on the transport channel),
|
|
`local_node` (the local node issued the request).
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `user` | The authenticated user.
|
|
| `realm` | The realm that authenticated the user.
|
|
| `action` | The name of the action that was executed.
|
|
| `request` | The type of request that was executed.
|
|
|======
|
|
|
|
.Transport authentication_failed Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_type` | Where the request originated: `rest` (request
|
|
originated from a REST API request), `transport`
|
|
(request was received on the transport channel),
|
|
`local_node` (the local node issued the request).
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `principal` | The principal (username) that failed authentication.
|
|
| `action` | The name of the action that was executed.
|
|
| `request` | The type of request that was executed.
|
|
| `indices` | A comma-separated list of indices this request
|
|
pertains to (when applicable).
|
|
|======
|
|
|
|
.Transport realm_authentication_failed Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_type` | Where the request originated: `rest` (request
|
|
originated from a REST API request), `transport`
|
|
(request was received on the transport channel),
|
|
`local_node` (the local node issued the request).
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `principal` | The principal (username) that failed authentication.
|
|
| `action` | The name of the action that was executed.
|
|
| `request` | The type of request that was executed.
|
|
| `indices` | A comma-separated list of indices this request
|
|
pertains to (when applicable).
|
|
| `realm` | The realm that failed to authenticate the user.
|
|
NOTE: A separate entry is logged for each
|
|
consulted realm.
|
|
|======
|
|
|
|
.Transport access_granted Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_type` | Where the request originated: `rest` (request
|
|
originated from a REST API request), `transport`
|
|
(request was received on the transport channel),
|
|
`local_node` (the local node issued the request).
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `principal` | The principal (username) that failed authentication.
|
|
| `action` | The name of the action that was executed.
|
|
| `request` | The type of request that was executed.
|
|
| `indices` | A comma-separated list of indices this request
|
|
pertains to (when applicable).
|
|
|======
|
|
|
|
.Transport access_denied Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_type` | Where the request originated: `rest` (request
|
|
originated from a REST API request), `transport`
|
|
(request was received on the transport channel),
|
|
`local_node` (the local node issued the request).
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `principal` | The principal (username) that failed authentication.
|
|
| `action` | The name of the action that was executed.
|
|
| `request` | The type of request that was executed.
|
|
| `indices` | A comma-separated list of indices this request
|
|
relates to (when applicable).
|
|
|======
|
|
|
|
.Transport tampered_request Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_type` | Where the request originated: `rest` (request
|
|
originated from a REST API request), `transport`
|
|
(request was received on the transport channel),
|
|
`local_node` (the local node issued the request).
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `principal` | The principal (username) that failed to authenticate.
|
|
| `action` | The name of the action that was executed.
|
|
| `request` | The type of request that was executed.
|
|
| `indices` | A comma-separated list of indices this request
|
|
pertains to (when applicable).
|
|
|======
|
|
|
|
.IP Filter connection_granted Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `transport_profile` | The transport profile the request targeted.
|
|
| `rule` | The <<ip-filtering, IP filtering>> rule that granted
|
|
the request.
|
|
|======
|
|
|
|
.IP Filter connection_denied Attributes
|
|
[cols="2,7",options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `origin_address` | The IP address from which the request originated.
|
|
| `transport_profile` | The transport profile the request targeted.
|
|
| `rule` | The <<ip-filtering, IP filtering>> rule that denied
|
|
the request.
|
|
|======
|
|
|
|
[float]
|
|
[[audit-log-output]]
|
|
=== Logfile Audit Output
|
|
|
|
The `logfile` audit output is the default output for auditing. It writes data to
|
|
the `<clustername>_access.log` file in the logs directory.
|
|
|
|
[float]
|
|
[[audit-log-entry-format]]
|
|
=== Log Entry Format
|
|
|
|
The format of a log entry is:
|
|
|
|
[source,txt]
|
|
----------------------------------------------------------------------------
|
|
[<timestamp>] [<local_node_info>] [<layer>] [<entry_type>] <attribute_list>
|
|
----------------------------------------------------------------------------
|
|
|
|
`<timestamp>` :: When the event occurred. You can configure the
|
|
timestamp format in `log4j2.properties`.
|
|
`<local_node_info>` :: Information about the local node that generated
|
|
the log entry. You can control what node information
|
|
is included by configuring the
|
|
<<audit-log-entry-local-node-info, local node info settings>>.
|
|
`<layer>` :: The layer from which this event originated:
|
|
`rest`, `transport` or `ip_filter`.
|
|
`<entry_type>` :: The type of event that occurred: `anonymous_access_denied`,
|
|
`authentication_failed`, `access_denied`, `access_granted`,
|
|
`connection_granted`, `connection_denied`.
|
|
`<attribute_list>` :: A comma-separated list of key-value pairs that contain
|
|
data pertaining to the event. Formatted as
|
|
`attr1=[val1], attr2=[val2]`. See <<audit-event-attributes,
|
|
Audit Entry Attributes>> for the attributes that can be included
|
|
for each type of event.
|
|
|
|
[float]
|
|
[[audit-log-settings]]
|
|
=== Logfile Output Settings
|
|
|
|
The events and some other information about what gets logged can be
|
|
controlled using settings in the `elasticsearch.yml` file.
|
|
|
|
.Audited Event Settings
|
|
[cols="4,^2,4",options="header"]
|
|
|======
|
|
| Name | Default | Description
|
|
| `xpack.security.audit.logfile.events.include` | `access_denied`, `access_granted`, `anonymous_access_denied`, `authentication_failed`, `connection_denied`, `tampered_request`, `run_as_denied`, `run_as_granted` | Includes the specified events in the output.
|
|
| `xpack.security.audit.logfile.events.exclude` | | Excludes the specified events from the output.
|
|
| `xpack.security.audit.logfile.events.emit_request_body`| false | Include or exclude the request body from REST requests
|
|
on certain event types such as `authentication_failed`.
|
|
|======
|
|
|
|
|
|
IMPORTANT: No filtering is performed when auditing, so sensitive data may be
|
|
audited in plain text when including the request body in audit events.
|
|
|
|
[[audit-log-entry-local-node-info]]
|
|
.Local Node Info Settings
|
|
[cols="4,^2,4",options="header"]
|
|
|======
|
|
| Name | Default | Description
|
|
| `xpack.security.audit.logfile.prefix.emit_node_name` | true | Include or exclude the node's name
|
|
from the local node info.
|
|
| `xpack.security.audit.logfile.prefix.emit_node_host_address` | false | Include or exclude the node's IP address
|
|
from the local node info.
|
|
| `xpack.security.audit.logfile.prefix.emit_node_host_name` | false | Include or exclude the node's host name
|
|
from the local node info.
|
|
|======
|
|
|
|
[[logging-file]]
|
|
You configure also configure how the logfile is written in the `log4j2.properties`
|
|
file located in `CONFIG_DIR/x-pack`. By default, audit information is appended to the
|
|
`<clustername>_access.log` file located in the standard Elasticsearch `logs` directory
|
|
(typically located at `$ES_HOME/logs`). The file rolls over on a daily basis.
|
|
|
|
[float]
|
|
[[audit-index]]
|
|
=== Index Audit Output
|
|
|
|
In addition to logging to a file, you can store audit logs in Elasticsearch
|
|
rolling indices. These indices can be either on the same cluster, or on a
|
|
remote cluster. You configure the following settings in
|
|
`elasticsearch.yml` to control how audit entries are indexed. To enable
|
|
this output, you need to configure the setting `xpack.security.audit.outputs`
|
|
in the `elasticsearch.yml` file:
|
|
|
|
[source,yaml]
|
|
----------------------------
|
|
xpack.security.audit.outputs: [ index, logfile ]
|
|
----------------------------
|
|
|
|
.Audit Log Indexing Configuration
|
|
[options="header"]
|
|
|======
|
|
| Attribute | Default Setting | Description
|
|
| `xpack.security.audit.index.bulk_size` | `1000` | Controls how many audit events are batched into a single write.
|
|
| `xpack.security.audit.index.flush_interval` | `1s` | Controls how often buffered events are flushed to the index.
|
|
| `xpack.security.audit.index.rollover` | `daily` | Controls how often to roll over to a new index:
|
|
`hourly`, `daily`, `weekly`, or `monthly`.
|
|
| `xpack.security.audit.index.events.include` | `anonymous_access_denied`, `authentication_failed`, `realm_authentication_failed`, `access_granted`, `access_denied`, `tampered_request`, `connection_granted`, `connection_denied`, `run_as_granted`, `run_as_denied` | The audit events to be indexed. See <<audit-event-types, Audit Entry Types>> for the complete list.
|
|
| `xpack.security.audit.index.events.exclude` | | The audit events to exclude from indexing.
|
|
| `xpack.security.audit.index.events.emit_request_body`| false | Include or exclude the request body from REST requests
|
|
on certain event types such as `authentication_failed`.
|
|
|======
|
|
|
|
IMPORTANT: No filtering is performed when auditing, so sensitive data may be
|
|
audited in plain text when including the request body in audit events.
|
|
|
|
[float]
|
|
==== Audit Index Settings
|
|
|
|
You can also configure settings for the indices that the events are stored in.
|
|
These settings are configured in the `xpack.security.audit.index.settings` namespace
|
|
in `elasticsearch.yml`. For example, the following configuration sets the
|
|
number of shards and replicas to 1 for the audit indices:
|
|
|
|
[source,yaml]
|
|
----------------------------
|
|
xpack.security.audit.index.settings:
|
|
index:
|
|
number_of_shards: 1
|
|
number_of_replicas: 1
|
|
----------------------------
|
|
|
|
[float]
|
|
==== Forwarding Audit Logs to a Remote Cluster
|
|
|
|
To index audit events to a remote Elasticsearch cluster, you configure
|
|
the following `xpack.security.audit.index.client` settings.
|
|
|
|
.Remote Audit Log Indexing Configuration
|
|
[options="header"]
|
|
|======
|
|
| Attribute | Description
|
|
| `xpack.security.audit.index.client.hosts` | Comma-separated list of `host:port` pairs. These hosts
|
|
should be nodes in the remote cluster.
|
|
| `xpack.security.audit.index.client.cluster.name` | The name of the remote cluster.
|
|
| `xpack.security.audit.index.client.xpack.security.user` | The `username:password` pair to use to authenticate with
|
|
the remote cluster.
|
|
|======
|
|
|
|
You can pass additional settings to the remote client by specifying them in the
|
|
`xpack.security.audit.index.client` namespace. For example, to allow the remote
|
|
client to discover all of the nodes in the remote cluster you can specify the
|
|
`client.transport.sniff` setting:
|
|
|
|
[source,yaml]
|
|
----------------------------
|
|
xpack.security.audit.index.client.transport.sniff: true
|
|
----------------------------
|