From 0a1e352c5d4b0f67c318145ea23473066e7e7d6d Mon Sep 17 00:00:00 2001 From: Albert Zaharovits Date: Mon, 22 Jan 2018 11:45:12 +0200 Subject: [PATCH] [DOCS] for audit filtering (elastic/x-pack-elasticsearch#3594) This documents the changes merged in elastic/x-pack-elasticsearch#3005 and elastic/x-pack-elasticsearch#3100 . Original commit: elastic/x-pack-elasticsearch@d1702f04807d58573af886ecf0f1f165351121d4 --- docs/en/security/auditing.asciidoc | 83 +++++++++++++++++++++++++++++- 1 file changed, 81 insertions(+), 2 deletions(-) diff --git a/docs/en/security/auditing.asciidoc b/docs/en/security/auditing.asciidoc index cd4510ae8cd..2fd6a1c3548 100644 --- a/docs/en/security/auditing.asciidoc +++ b/docs/en/security/auditing.asciidoc @@ -135,7 +135,6 @@ The log level determines which attributes are included in a log entry. .REST realm_authentication_failed Attributes [cols="2,7",options="header"] -[cols="2,7",options="header"] |====== | Attribute | Description | `origin_address` | The IP address from which the request originated. @@ -222,6 +221,7 @@ The log level determines which attributes are included in a log entry. `local_node` (the local node issued the request). | `origin_address` | The IP address from which the request originated. | `principal` | The principal (username) that passed authentication. +| `roles` | The set of roles granting permissions. | `action` | The name of the action that was executed. | `request` | The type of request that was executed. | `indices` | A comma-separated list of indices this request @@ -238,6 +238,7 @@ The log level determines which attributes are included in a log entry. `local_node` (the local node issued the request). | `origin_address` | The IP address from which the request originated. | `principal` | The principal (username) that failed authentication. +| `roles` | The set of roles granting permissions. | `action` | The name of the action that was executed. | `request` | The type of request that was executed. | `indices` | A comma-separated list of indices this request @@ -350,11 +351,89 @@ audited in plain text when including the request body in audit events. |====== [[logging-file]] -You configure also configure how the logfile is written in the `log4j2.properties` +You can also configure how the logfile is written in the `log4j2.properties` file located in `CONFIG_DIR/x-pack`. By default, audit information is appended to the `_access.log` file located in the standard Elasticsearch `logs` directory (typically located at `$ES_HOME/logs`). The file rolls over on a daily basis. +[float] +[[audit-log-ignore-policy]] +=== Logfile Audit Events Ignore Policies + +The comprehensive audit trail is necessary to ensure accountability. It offers tremendous +value during incident response and can even be required for demonstrating compliance. + +The drawback of an audited system is represented by the inevitable performance penalty incurred. +In all truth, the audit trail spends _I/O ops_ that are not available anymore for the user's queries. +Sometimes the verbosity of the audit trail may become a problem that the event type restrictions, +<>, will not alleviate. + +*Audit events ignore policies* are a finer way to tune the verbosity of the audit trail. +These policies define rules that match audit events which will be _ignored_ (read as: not printed). +Rules match on the values of attributes of audit events and complement the <> method. +Imagine the corpus of audit events and the policies chopping off unwanted events. + +IMPORTANT: When utilizing audit events ignore policies you are acknowledging potential +accountability gaps that could render illegitimate actions undetectable. +Please take time to review these policies whenever your system architecture changes. + +A policy is a named set of filter rules. Each filter rule applies to a single event attribute, +one of the `users`, `realms`, `roles` or `indices` attributes. The filter rule defines +a list of {ref}/query-dsl-regexp-query.html#regexp-syntax[Lucene regexp], *any* of which has to match the value of the audit +event attribute for the rule to match. +A policy matches an event if *all* the rules comprising it match the event. +An audit event is ignored, therefore not printed, if it matches *any* policy. All other +non-matching events are printed as usual. + +All policies are defined under the `xpack.security.audit.logfile.events.ignore_filters` +settings namespace. For example, the following policy named _example1_ matches +events from the _kibana_ or _admin_user_ principals **and** operating over indices of the +wildcard form _app-logs*_: + +[source,yaml] +---------------------------- +xpack.security.audit.logfile.events.ignore_filters: + example1: + users: ["kibana", "admin_user"] + indices: ["app-logs*"] +---------------------------- + +An audit event generated by the _kibana_ user and operating over multiple indices +, some of which do not match the indices wildcard, will not match. +As expected, operations generated by all other users (even operating only on indices that +match the _indices_ filter) will not match this policy either. + +Audit events of different types may have <>. +If an event does not contain an attribute for which some policy defines filters, the +event will not match the policy. +For example, the following policy named _example2_, will never match `authentication_success` or +`authentication_failed` events, irrespective of the user's roles, because these +event schemas do not contain the `role` attribute: + +[source,yaml] +---------------------------- +xpack.security.audit.logfile.events.ignore_filters: + example2: + roles: ["admin", "ops_admin_*"] +---------------------------- + +Likewise, any events of users with multiple roles, some of which do not match the +regexps will not match this policy. + +For completeness, although practical use cases should be sparse, a filter can match +a missing attribute of an event, using the empty string ("") or the empty list ([]). +For example, the following policy will match events that do not have the `indices` +attribute (`anonymous_access_denied`, `authentication_success` and other types) as well +as events over the _next_ index. + +[source,yaml] +---------------------------- +xpack.security.audit.logfile.events.ignore_filters: + example3: + indices: ["next", ""] +---------------------------- + + [float] [[audit-index]] === Index Audit Output