[DOCS] for audit filtering (elastic/x-pack-elasticsearch#3594)

This documents the changes merged in elastic/x-pack-elasticsearch#3005 and elastic/x-pack-elasticsearch#3100 .

Original commit: elastic/x-pack-elasticsearch@d1702f0480

docs/en/security/auditing.asciidoc

@@ -135,7 +135,6 @@ The log level determines  which attributes are included in a log entry.
 
 .REST realm_authentication_failed Attributes
 [cols="2,7",options="header"]
-[cols="2,7",options="header"]
 |======
 | Attribute         | Description
 | `origin_address`  | The IP address from which the request originated.
@@ -222,6 +221,7 @@ The log level determines  which attributes are included in a log entry.
                       `local_node` (the local node issued the request).
 | `origin_address`  | The IP address from which the request originated.
 | `principal`       | The principal (username) that passed authentication.
+| `roles`           | The set of roles granting permissions.
 | `action`          | The name of the action that was executed.
 | `request`         | The type of request that was executed.
 | `indices`         | A comma-separated list of indices this request
@@ -238,6 +238,7 @@ The log level determines  which attributes are included in a log entry.
                       `local_node` (the local node issued the request).
 | `origin_address`  | The IP address from which the request originated.
 | `principal`       | The principal (username) that failed authentication.
+| `roles`           | The set of roles granting permissions.
 | `action`          | The name of the action that was executed.
 | `request`         | The type of request that was executed.
 | `indices`         | A comma-separated list of indices this request
@@ -350,11 +351,89 @@ audited in plain text when including the request body in audit events.
 |======
 
 [[logging-file]]
-You configure also configure how the logfile is written in the `log4j2.properties`
+You can also configure how the logfile is written in the `log4j2.properties`
 file located in `CONFIG_DIR/x-pack`. By default, audit information is appended to the
 `<clustername>_access.log` file located in the standard Elasticsearch `logs` directory
 (typically located at `$ES_HOME/logs`). The file rolls over on a daily basis.
 
+[float]
+[[audit-log-ignore-policy]]
+=== Logfile Audit Events Ignore Policies
+
+The comprehensive audit trail is necessary to ensure accountability. It offers tremendous
+value during incident response and can even be required for demonstrating compliance.
+
+The drawback of an audited system is represented by the inevitable performance penalty incurred.
+In all truth, the audit trail spends _I/O ops_ that are not available anymore for the user's queries.
+Sometimes the verbosity of the audit trail may become a problem that the event type restrictions,
+<<audit-log-settings, defined by `include` and `exclude`>>, will not alleviate.
+
+*Audit events ignore policies* are a finer way to tune the verbosity of the audit trail.
+These policies define rules that match audit events which will be _ignored_ (read as: not printed).
+Rules match on the values of attributes of audit events and complement the <<audit-log-settings, include/exclude>> method.
+Imagine the corpus of audit events and the policies chopping off unwanted events.
+
+IMPORTANT: When utilizing audit events ignore policies you are acknowledging potential
+accountability gaps that could render illegitimate actions undetectable.
+Please take time to review these policies whenever your system architecture changes.
+
+A policy is a named set of filter rules. Each filter rule applies to a single event attribute,
+one of the `users`, `realms`, `roles` or `indices` attributes. The filter rule defines
+a list of {ref}/query-dsl-regexp-query.html#regexp-syntax[Lucene regexp], *any* of which has to match the value of the audit
+event attribute for the rule to match.
+A policy matches an event if *all* the rules comprising it match the event.
+An audit event is ignored, therefore not printed, if it matches *any* policy. All other
+non-matching events are printed as usual.
+
+All policies are defined under the `xpack.security.audit.logfile.events.ignore_filters`
+settings namespace. For example, the following policy named _example1_ matches
+events from the _kibana_ or _admin_user_ principals **and** operating over indices of the
+wildcard form _app-logs*_:
+
+[source,yaml]
+----------------------------
+xpack.security.audit.logfile.events.ignore_filters:
+  example1:
+    users: ["kibana", "admin_user"]
+    indices: ["app-logs*"]
+----------------------------
+
+An audit event generated by the _kibana_ user and operating over multiple indices
+, some of which do not match the indices wildcard, will not match.
+As expected, operations generated by all other users (even operating only on indices that
+match the _indices_ filter) will not match this policy either.
+
+Audit events of different types may have <<audit-event-attributes, different attributes>>.
+If an event does not contain an attribute for which some policy defines filters, the
+event will not match the policy.
+For example, the following policy named _example2_, will never match `authentication_success` or
+`authentication_failed` events, irrespective of the user's roles, because these
+event schemas do not contain the `role` attribute:
+
+[source,yaml]
+----------------------------
+xpack.security.audit.logfile.events.ignore_filters:
+  example2:
+    roles: ["admin", "ops_admin_*"]
+----------------------------
+
+Likewise, any events of users with multiple roles, some of which do not match the
+regexps will not match this policy.
+
+For completeness, although practical use cases should be sparse, a filter can match
+a missing attribute of an event, using the empty string ("") or the empty list ([]).
+For example, the following policy will match events that do not have the `indices`
+attribute (`anonymous_access_denied`, `authentication_success` and other types) as well
+as events over the _next_ index.
+
+[source,yaml]
+----------------------------
+xpack.security.audit.logfile.events.ignore_filters:
+  example3:
+    indices: ["next", ""]
+----------------------------
+
+
 [float]
 [[audit-index]]
 === Index Audit Output