187 lines
8.9 KiB
Plaintext
187 lines
8.9 KiB
Plaintext
[role="xpack"]
|
|
[[audit-log-output]]
|
|
=== Logfile audit output
|
|
|
|
The `logfile` audit output is the default output for auditing. It writes data to
|
|
the `<clustername>_audit.json` file in the logs directory. To maintain
|
|
compatibility with releases prior to 6.5.0, a `<clustername>_access.log` file
|
|
is also generated. They differ in the output format but the contents
|
|
are similar. For systems that are not ingesting the audit file for search or
|
|
analytics it is strongly recommended to keep only the newer format.
|
|
|
|
To turn off the deprecated output format, you can disable the logger in the
|
|
`log4j2.properties` file:
|
|
|
|
[source, properties]
|
|
--------------------------------------------------
|
|
# change info to off
|
|
# logger.xpack_security_audit_deprecated_logfile.level = info
|
|
logger.xpack_security_audit_deprecated_logfile.level = off
|
|
--------------------------------------------------
|
|
|
|
Alternatively, use the
|
|
{ref}/cluster-update-settings.html[cluster update settings API] to dynamically
|
|
configure the logger:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /_cluster/settings
|
|
{
|
|
"persistent": {
|
|
"logger.org.elasticsearch.xpack.security.audit.logfile.DeprecatedLoggingAuditTrail": "off"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
NOTE: If you overwrite the `log4j2.properties` and do not specify appenders for
|
|
any of the audit trails, audit events are forwarded to the root appender, which
|
|
by default points to the `elasticsearch.log` file.
|
|
|
|
|
|
[discrete]
|
|
[[audit-log-entry-format]]
|
|
=== Log entry format
|
|
|
|
The log entries in the `<clustername>_audit.json` file have the following format:
|
|
|
|
- Each log entry is a one line JSON document and each one is printed on a separate line.
|
|
- The fields of a log entry are ordered. However, if a field does not have a value it
|
|
will not be printed. The precise line pattern, together with the complete field
|
|
order, are specified in the `log4j2.properties` config file.
|
|
- The log entry does not contain nested inner JSON objects, i.e. the doc is flat.
|
|
- The field names follow a dotted notation to flatten inner objects.
|
|
- A field's value can be a string, a number or an array of strings.
|
|
- A field's value, a request body as well, will be escaped as per the JSON RFC 4627.
|
|
|
|
There is a list of <<audit-event-types, audit event types>> specifying the
|
|
set of fields for each sog entry type.
|
|
|
|
[discrete]
|
|
[[deprecated-audit-log-entry-format]]
|
|
=== Deprecated log entry format
|
|
|
|
The log entries in the `<clustername>_access.log` file have the following format:
|
|
|
|
[source,txt]
|
|
----------------------------------------------------------------------------
|
|
[<timestamp>] [<local_node_info>] [<layer>] [<entry_type>] <attribute_list>
|
|
----------------------------------------------------------------------------
|
|
|
|
`<timestamp>` :: When the event occurred. You can configure the
|
|
timestamp format in `log4j2.properties`.
|
|
`<local_node_info>` :: Information about the local node that generated
|
|
the log entry. You can control what node information
|
|
is included by configuring the
|
|
{ref}/auditing-settings.html#node-audit-settings[local node info settings].
|
|
`<layer>` :: The layer from which this event originated:
|
|
`rest`, `transport` or `ip_filter`.
|
|
`<entry_type>` :: The type of event that occurred: `anonymous_access_denied`,
|
|
`authentication_failed`, `access_denied`, `access_granted`,
|
|
`connection_granted`, `connection_denied`.
|
|
`<attribute_list>` :: A comma-separated list of key-value pairs that contain
|
|
data pertaining to the event. Formatted as
|
|
`attr1=[val1], attr2=[val2]`. See <<audit-event-attributes,
|
|
Audit Entry Attributes>> for the attributes that can be included
|
|
for each type of event.
|
|
|
|
[discrete]
|
|
[[audit-log-settings]]
|
|
=== Logfile output settings
|
|
|
|
The events and some other information about what gets logged can be
|
|
controlled using settings in the `elasticsearch.yml` file. See
|
|
{ref}/auditing-settings.html#event-audit-settings[Audited Event Settings] and
|
|
{ref}/auditing-settings.html#node-audit-settings[Local Node Info Settings].
|
|
|
|
IMPORTANT: No filtering is performed when auditing, so sensitive data may be
|
|
audited in plain text when including the request body in audit events.
|
|
|
|
[[logging-file]]
|
|
You can also configure how the logfile is written in the `log4j2.properties`
|
|
file located in `ES_PATH_CONF`. By default, audit information is appended to the
|
|
`<clustername>_audit.json` file located in the standard Elasticsearch `logs` directory
|
|
(typically located at `$ES_HOME/logs`). The file rolls over on a daily basis.
|
|
The deprecated logfile audit format (`<clustername>_access.log`) can be disabled
|
|
from the same `log4j2.properties` file (hint: look for the comment
|
|
instructing to set the log level to `off`). The deprecated format is a duplication
|
|
of information that is in place to assure backwards compatibility. If you are
|
|
not strict about the audit format it is strongly recommended to only use the
|
|
`<clustername>_audit.json` log appender.
|
|
|
|
[discrete]
|
|
[[audit-log-ignore-policy]]
|
|
=== Logfile audit events ignore policies
|
|
|
|
The comprehensive audit trail is necessary to ensure accountability. It offers tremendous
|
|
value during incident response and can even be required for demonstrating compliance.
|
|
|
|
The drawback of an audited system is represented by the inevitable performance penalty incurred.
|
|
In all truth, the audit trail spends _I/O ops_ that are not available anymore for the user's queries.
|
|
Sometimes the verbosity of the audit trail may become a problem that the event type restrictions,
|
|
<<audit-log-settings, defined by `include` and `exclude`>>, will not alleviate.
|
|
|
|
*Audit events ignore policies* are a finer way to tune the verbosity of the audit trail.
|
|
These policies define rules that match audit events which will be _ignored_ (read as: not printed).
|
|
Rules match on the values of attributes of audit events and complement the <<audit-log-settings, include/exclude>> method.
|
|
Imagine the corpus of audit events and the policies chopping off unwanted events.
|
|
|
|
IMPORTANT: When utilizing audit events ignore policies you are acknowledging potential
|
|
accountability gaps that could render illegitimate actions undetectable.
|
|
Please take time to review these policies whenever your system architecture changes.
|
|
|
|
A policy is a named set of filter rules. Each filter rule applies to a single event attribute,
|
|
one of the `users`, `realms`, `roles` or `indices` attributes. The filter rule defines
|
|
a list of {ref}/regexp-syntax.html[Lucene regexp], *any* of which has to match the value of the audit
|
|
event attribute for the rule to match.
|
|
A policy matches an event if *all* the rules comprising it match the event.
|
|
An audit event is ignored, therefore not printed, if it matches *any* policy. All other
|
|
non-matching events are printed as usual.
|
|
|
|
All policies are defined under the `xpack.security.audit.logfile.events.ignore_filters`
|
|
settings namespace. For example, the following policy named _example1_ matches
|
|
events from the _kibana_system_ or _admin_user_ principals **and** operating over indices of the
|
|
wildcard form _app-logs*_:
|
|
|
|
[source,yaml]
|
|
----------------------------
|
|
xpack.security.audit.logfile.events.ignore_filters:
|
|
example1:
|
|
users: ["kibana_system", "admin_user"]
|
|
indices: ["app-logs*"]
|
|
----------------------------
|
|
|
|
An audit event generated by the _kibana_system_ user and operating over multiple indices
|
|
, some of which do not match the indices wildcard, will not match.
|
|
As expected, operations generated by all other users (even operating only on indices that
|
|
match the _indices_ filter) will not match this policy either.
|
|
|
|
Audit events of different types may have <<audit-event-attributes, different attributes>>.
|
|
If an event does not contain an attribute for which some policy defines filters, the
|
|
event will not match the policy.
|
|
For example, the following policy named _example2_, will never match `authentication_success` or
|
|
`authentication_failed` events, irrespective of the user's roles, because these
|
|
event schemas do not contain the `role` attribute:
|
|
|
|
[source,yaml]
|
|
----------------------------
|
|
xpack.security.audit.logfile.events.ignore_filters:
|
|
example2:
|
|
roles: ["admin", "ops_admin_*"]
|
|
----------------------------
|
|
|
|
Likewise, any events of users with multiple roles, some of which do not match the
|
|
regexps will not match this policy.
|
|
|
|
For completeness, although practical use cases should be sparse, a filter can match
|
|
a missing attribute of an event, using the empty string ("") or the empty list ([]).
|
|
For example, the following policy will match events that do not have the `indices`
|
|
attribute (`anonymous_access_denied`, `authentication_success` and other types) as well
|
|
as events over the _next_ index.
|
|
|
|
[source,yaml]
|
|
----------------------------
|
|
xpack.security.audit.logfile.events.ignore_filters:
|
|
example3:
|
|
indices: ["next", ""]
|
|
----------------------------
|