476 lines
11 KiB
Plaintext
476 lines
11 KiB
Plaintext
[role="xpack"]
|
|
[testenv="basic"]
|
|
[[eql-syntax]]
|
|
== EQL syntax reference
|
|
++++
|
|
<titleabbrev>Syntax reference</titleabbrev>
|
|
++++
|
|
|
|
dev::[]
|
|
|
|
[IMPORTANT]
|
|
====
|
|
{es} supports a subset of EQL syntax. See <<eql-limitations>>.
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-basic-syntax]]
|
|
=== Basic syntax
|
|
|
|
EQL queries require an event category and a matching condition. The `where`
|
|
keyword connects them.
|
|
|
|
[source,eql]
|
|
----
|
|
event_category where condition
|
|
----
|
|
|
|
For example, the following EQL query matches `process` events with a
|
|
`process.name` field value of `svchost.exe`:
|
|
|
|
[source,eql]
|
|
----
|
|
process where process.name == "svchost.exe"
|
|
----
|
|
|
|
[discrete]
|
|
[[eql-syntax-event-categories]]
|
|
==== Event categories
|
|
|
|
In {es}, an event category is a valid, indexed value of the
|
|
<<eql-required-fields,event category field>>. You can set the event category
|
|
field using the `event_category_field` parameter of the EQL search API.
|
|
|
|
[discrete]
|
|
[[eql-syntax-match-any-event-category]]
|
|
===== Match any event category
|
|
|
|
To match events of any category, use the `any` keyword. You can also use the
|
|
`any` keyword to search for documents without a event category field.
|
|
|
|
For example, the following EQL query matches any documents with a
|
|
`network.protocol` field value of `http`:
|
|
|
|
[source,eql]
|
|
----
|
|
any where network.protocol == "http"
|
|
----
|
|
|
|
[discrete]
|
|
[[eql-syntax-conditions]]
|
|
==== Conditions
|
|
|
|
A condition consists of one or more criteria an event must match.
|
|
You can specify and combine these criteria using the following operators:
|
|
|
|
[discrete]
|
|
[[eql-syntax-comparison-operators]]
|
|
===== Comparison operators
|
|
|
|
[source,eql]
|
|
----
|
|
< <= == != >= >
|
|
----
|
|
|
|
[IMPORTANT]
|
|
====
|
|
Avoid using the equal operator (`==`) to perform exact matching on `text` field
|
|
values.
|
|
|
|
By default, {es} changes the values of <<text,`text`>> fields as part of
|
|
<<analysis, analysis>>. This can make finding exact matches for `text` field
|
|
values difficult.
|
|
|
|
To search `text` fields, consider using a <<eql-search-filter-query-dsl,query
|
|
DSL filter>> that contains a <<query-dsl-match-query,`match`>> query.
|
|
====
|
|
|
|
.*Definitions*
|
|
[%collapsible]
|
|
====
|
|
`<` (less than)::
|
|
Returns `true` if the value to the left of the operator is less than the value
|
|
to the right. Otherwise returns `false`.
|
|
|
|
`<=` (less than or equal) ::
|
|
Returns `true` if the value to the left of the operator is less than or equal to
|
|
the value to the right. Otherwise returns `false`.
|
|
|
|
`==` (equal)::
|
|
Returns `true` if the values to the left and right of the operator are equal.
|
|
Otherwise returns `false`.
|
|
|
|
`!=` (not equal)::
|
|
Returns `true` if the values to the left and right of the operator are not
|
|
equal. Otherwise returns `false`.
|
|
|
|
`>=` (greater than or equal) ::
|
|
Returns `true` if the value to the left of the operator is greater than or equal
|
|
to the value to the right. Otherwise returns `false`.
|
|
|
|
`>` (greater than)::
|
|
Returns `true` if the value to the left of the operator is greater than the
|
|
value to the right. Otherwise returns `false`.
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-syntax-logical-operators]]
|
|
===== Logical operators
|
|
|
|
[source,eql]
|
|
----
|
|
and or not
|
|
----
|
|
|
|
.*Definitions*
|
|
[%collapsible]
|
|
====
|
|
`and`::
|
|
Returns `true` only if the condition to the left and right _both_ return `true`.
|
|
Otherwise returns `false.
|
|
|
|
`or`::
|
|
Returns `true` if one of the conditions to the left or right `true`.
|
|
Otherwise returns `false.
|
|
|
|
`not`::
|
|
Returns `true` if the condition to the right is `false`.
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-syntax-lookup-operators]]
|
|
===== Lookup operators
|
|
|
|
[source,eql]
|
|
----
|
|
user.name in ("Administrator", "SYSTEM", "NETWORK SERVICE")
|
|
user.name not in ("Administrator", "SYSTEM", "NETWORK SERVICE")
|
|
----
|
|
|
|
.*Definitions*
|
|
[%collapsible]
|
|
====
|
|
`in`::
|
|
Returns `true` if the value is contained in the provided list.
|
|
|
|
`not in`::
|
|
Returns `true` if the value is not contained in the provided list.
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-syntax-math-operators]]
|
|
===== Math operators
|
|
|
|
[source,eql]
|
|
----
|
|
+ - * / %
|
|
----
|
|
|
|
.*Definitions*
|
|
[%collapsible]
|
|
====
|
|
`+` (add)::
|
|
Adds the values to the left and right of the operator.
|
|
|
|
`-` (Subtract)::
|
|
Subtracts the value to the right of the operator from the value to the left.
|
|
|
|
`*` (Subtract)::
|
|
Multiplies the values to the left and right of the operator.
|
|
|
|
`/` (Divide)::
|
|
Divides the value to the left of the operator by the value to the right.
|
|
|
|
`%` (modulo)::
|
|
Divides the value to the left of the operator by the value to the right. Returns only the remainder.
|
|
====
|
|
|
|
[[eql-divide-operator-float-rounding]]
|
|
[WARNING]
|
|
====
|
|
If both the dividend and divisor are integers, the divide (`\`) operation
|
|
_rounds down_ any returned floating point numbers to the nearest integer.
|
|
|
|
EQL queries in {es} should account for this rounding. To avoid rounding, convert
|
|
either the dividend or divisor to a float.
|
|
|
|
[%collapsible]
|
|
.**Example**
|
|
=====
|
|
The `process.args_count` field is a <<number,`long`>> integer field containing a
|
|
count of process arguments.
|
|
|
|
A user might expect the following EQL query to only match events with a
|
|
`process.args_count` value of `4`.
|
|
|
|
[source,eql]
|
|
----
|
|
process where ( 4 / process.args_count ) == 1
|
|
----
|
|
|
|
However, the EQL query matches events with a `process.args_count` value of `3`
|
|
or `4`.
|
|
|
|
For events with a `process.args_count` value of `3`, the divide operation
|
|
returns a float of `1.333...`, which is rounded down to `1`.
|
|
|
|
To match only events with a `process.args_count` value of `4`, convert
|
|
either the dividend or divisor to a float.
|
|
|
|
The following EQL query changes the integer `4` to the equivalent float `4.0`.
|
|
|
|
[source,eql]
|
|
----
|
|
process where ( 4.0 / process.args_count ) == 1
|
|
----
|
|
=====
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-syntax-strings]]
|
|
==== Strings
|
|
|
|
Strings are enclosed with double quotes (`"`) or single quotes (`'`).
|
|
|
|
[source,eql]
|
|
----
|
|
"hello world"
|
|
"hello world with 'substring'"
|
|
----
|
|
|
|
[discrete]
|
|
[[eql-syntax-wildcards]]
|
|
===== Wildcards
|
|
|
|
You can use the wildcard operator (`*`) within a string to match specific
|
|
patterns. You can use wildcards with the `==` (equal) or `!=` (not equal)
|
|
operators:
|
|
|
|
[source,eql]
|
|
----
|
|
field == "example*wildcard"
|
|
field != "example*wildcard"
|
|
----
|
|
|
|
[discrete]
|
|
[[eql-syntax-match-any-condition]]
|
|
===== Match any condition
|
|
|
|
To match events solely on event category, use the `where true` condition.
|
|
|
|
For example, the following EQL query matches any `file` events:
|
|
|
|
[source,eql]
|
|
----
|
|
file where true
|
|
----
|
|
|
|
To match any event, you can combine the `any` keyword with the `where true`
|
|
condition:
|
|
|
|
[source,eql]
|
|
----
|
|
any where true
|
|
----
|
|
|
|
[discrete]
|
|
[[eql-syntax-escaped-characters]]
|
|
===== Escaped characters
|
|
|
|
When used within a string, special characters, such as a carriage return or
|
|
double quote (`"`), must be escaped with a preceding backslash (`\`).
|
|
|
|
[source,eql]
|
|
----
|
|
"example \t of \n escaped \r characters"
|
|
----
|
|
|
|
.*Escape sequences*
|
|
[%collapsible]
|
|
====
|
|
[options="header"]
|
|
|====
|
|
| Escape sequence | Literal character
|
|
|`\n` | A newline (linefeed) character
|
|
|`\r` | A carriage return character
|
|
|`\t` | A tab character
|
|
|`\\` | A backslash (`\`) character
|
|
|`\"` | A double quote (`"`) character
|
|
|`\'` | A single quote (`'`) character
|
|
|====
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-syntax-raw-strings]]
|
|
===== Raw strings
|
|
|
|
Raw strings are preceded by a question mark (`?`) and treat backslashes (`\`) as
|
|
literal characters.
|
|
|
|
[source,eql]
|
|
----
|
|
?"String with a literal 'blackslash' \ character included"
|
|
----
|
|
|
|
You can escape single quotes (`'`) and double quotes (`"`) with a backslash, but
|
|
the backslash remains in the resulting string.
|
|
|
|
[source,eql]
|
|
----
|
|
?"\""
|
|
----
|
|
|
|
[NOTE]
|
|
====
|
|
Raw strings cannot contain only a single backslash. Additionally, raw strings
|
|
cannot end in an odd number of backslashes.
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-syntax-non-alpha-field-names]]
|
|
==== Non-alphanumeric field names
|
|
|
|
Field names containing non-alphanumeric characters, such as underscores (`_`),
|
|
dots (`.`), hyphens (`-`), or spaces, must be escaped using backticks (+++`+++).
|
|
|
|
[source,eql]
|
|
----
|
|
`my_field`
|
|
`my.field`
|
|
`my-field`
|
|
`my field`
|
|
----
|
|
|
|
[discrete]
|
|
[[eql-sequences]]
|
|
=== Sequences
|
|
|
|
You can use EQL sequences to describe and match an ordered series of events.
|
|
Each item in a sequence is an event category and event condition,
|
|
surrounded by square brackets. Events are listed in ascending chronological
|
|
order, with the most recent event listed last.
|
|
|
|
[source,eql]
|
|
----
|
|
sequence
|
|
[ event_category_1 where condition_1 ]
|
|
[ event_category_2 where condition_2 ]
|
|
...
|
|
----
|
|
|
|
.*Example*
|
|
[%collapsible]
|
|
====
|
|
The following EQL query matches this series of ordered events:
|
|
|
|
. Start with an event with:
|
|
+
|
|
--
|
|
* An event category of `file`
|
|
* A `file.extension` of `exe`
|
|
--
|
|
. Followed by an event with an event category of `process`
|
|
|
|
[source,eql]
|
|
----
|
|
sequence
|
|
[ file where file.extension == "exe" ]
|
|
[ process where true ]
|
|
----
|
|
====
|
|
|
|
You can use the `by` keyword with sequences to only match events that share the
|
|
same field values. If a field value should be shared across all events, you
|
|
can use `sequence by`.
|
|
|
|
[source,eql]
|
|
----
|
|
sequence by field_foo
|
|
[ event_category_1 where condition_1 ] by field_baz
|
|
[ event_category_2 where condition_2 ] by field_bar
|
|
...
|
|
----
|
|
|
|
.*Example*
|
|
[%collapsible]
|
|
====
|
|
The following sequence uses the `by` keyword to constrain matching events to:
|
|
|
|
* Events with the same `user.name` value
|
|
* `file` events with a `file.path` value equal to the following `process`
|
|
event's `process.path` value.
|
|
|
|
[source,eql]
|
|
----
|
|
sequence
|
|
[ file where file.extension == "exe" ] by user.name, file.path
|
|
[ process where true ] by user.name, process.path
|
|
----
|
|
|
|
Because the `user.name` field is shared across all events in the sequence, it
|
|
can be included using `sequence by`. The following sequence is equivalent to the
|
|
prior one.
|
|
|
|
[source,eql]
|
|
----
|
|
sequence by user.name
|
|
[ file where file.extension == "exe" ] by file.path
|
|
[ process where true ] by process.path
|
|
----
|
|
====
|
|
|
|
[discrete]
|
|
[[eql-functions]]
|
|
=== Functions
|
|
|
|
{es} supports several of EQL's built-in functions. You can use these functions
|
|
to convert data types, perform math, manipulate strings, and more.
|
|
|
|
For a list of supported functions, see <<eql-function-ref>>.
|
|
|
|
[TIP]
|
|
====
|
|
Using functions in EQL queries can result in slower search speeds. If you
|
|
often use functions to transform indexed data, you can speed up search by making
|
|
these changes during indexing instead. However, that often means slower index
|
|
speeds.
|
|
|
|
.*Example*
|
|
[%collapsible]
|
|
=====
|
|
An index contains the `file.path` field. `file.path` contains the full path to a
|
|
file, including the file extension.
|
|
|
|
When running EQL searches, users often use the `endsWith` function with the
|
|
`file.path` field to match file extensions:
|
|
|
|
[source,eql]
|
|
----
|
|
file where endsWith(file.path,".exe") or endsWith(file.path,".dll")
|
|
----
|
|
|
|
While this works, it can be repetitive to write and can slow search speeds. To
|
|
speed up search, you can do the following instead:
|
|
|
|
. <<indices-put-mapping,Add a new field>>, `file.extension`, to the index. The
|
|
`file.extension` field will contain only the file extension from the
|
|
`file.path` field.
|
|
. Use an <<ingest,ingest pipeline>> containing the <<grok-processor,`grok`>>
|
|
processor or another preprocessor tool to extract the file extension from the
|
|
`file.path` field before indexing.
|
|
. Index the extracted file extension to the `file.extension` field.
|
|
|
|
These changes may slow indexing but allow for faster searches. Users
|
|
can use the `file.extension` field instead of multiple `endsWith` function
|
|
calls:
|
|
|
|
[source,eql]
|
|
----
|
|
file where file.extension in ("exe", "dll")
|
|
----
|
|
=====
|
|
|
|
We recommend testing and benchmarking any indexing changes before deploying them
|
|
in production. See <<tune-for-indexing-speed>> and <<tune-for-search-speed>>.
|
|
====
|