2020-02-12 08:40:10 -05:00
|
|
|
[role="xpack"]
|
|
|
|
[testenv="basic"]
|
|
|
|
[[eql-search]]
|
|
|
|
== Run an EQL search
|
|
|
|
|
|
|
|
experimental::[]
|
|
|
|
|
|
|
|
To start using EQL in {es}, first ensure your event data meets
|
|
|
|
<<eql-requirements,EQL requirements>>. Then ingest or add the data to an {es}
|
|
|
|
index.
|
|
|
|
|
|
|
|
The following <<docs-bulk,bulk API>> request adds some example log data to the
|
|
|
|
`sec_logs` index. This log data follows the {ecs-ref}[Elastic Common Schema
|
|
|
|
(ECS)].
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
PUT sec_logs/_bulk?refresh
|
2020-03-02 10:08:03 -05:00
|
|
|
{"index":{"_index" : "sec_logs", "_id" : "1"}}
|
2020-02-12 08:40:10 -05:00
|
|
|
{ "@timestamp": "2020-12-07T11:06:07.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } }
|
2020-03-02 10:08:03 -05:00
|
|
|
{"index":{"_index" : "sec_logs", "_id" : "2"}}
|
|
|
|
{ "@timestamp": "2020-12-07T11:07:08.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "file" }, "file": { "accessed": "2020-12-07T11:07:08.000Z", "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe", "type": "file", "size": 16384 }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } }
|
|
|
|
{"index":{"_index" : "sec_logs", "_id" : "3"}}
|
2020-02-12 08:40:10 -05:00
|
|
|
{ "@timestamp": "2020-12-07T11:07:09.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "regsvr32.exe", "path": "C:\\Windows\\System32\\regsvr32.exe" } }
|
|
|
|
----
|
2020-03-02 10:08:03 -05:00
|
|
|
// TESTSETUP
|
2020-02-12 08:40:10 -05:00
|
|
|
|
|
|
|
You can now use the EQL search API to search this index using an EQL query.
|
|
|
|
|
|
|
|
The following request searches the `sec_logs` index using the EQL query
|
2020-03-02 09:26:23 -05:00
|
|
|
specified in the `query` parameter. The EQL query matches events with an
|
2020-02-12 08:40:10 -05:00
|
|
|
`event.category` of `process` that have a `process.name` of `cmd.exe`.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
GET sec_logs/_eql/search
|
|
|
|
{
|
2020-03-04 08:02:38 -05:00
|
|
|
"event_category_field": "event.category",
|
2020-03-02 15:40:05 -05:00
|
|
|
"timestamp_field": "@timestamp",
|
2020-03-02 09:26:23 -05:00
|
|
|
"query": """
|
2020-02-12 08:40:10 -05:00
|
|
|
process where process.name == "cmd.exe"
|
|
|
|
"""
|
|
|
|
}
|
|
|
|
----
|
|
|
|
|
|
|
|
Because the `sec_log` index follows the ECS, you don't need to specify the
|
2020-03-02 16:07:42 -05:00
|
|
|
timestamp field. The request uses the `@timestamp` field by default.
|
2020-03-02 10:08:03 -05:00
|
|
|
|
|
|
|
The API returns the following response containing the matching event:
|
|
|
|
|
|
|
|
[source,console-result]
|
|
|
|
----
|
|
|
|
{
|
|
|
|
"took": 3,
|
|
|
|
"timed_out": false,
|
|
|
|
"hits": {
|
|
|
|
"total": {
|
|
|
|
"value": 1,
|
|
|
|
"relation": "eq"
|
|
|
|
},
|
|
|
|
"events": [
|
|
|
|
{
|
|
|
|
"_index": "sec_logs",
|
|
|
|
"_type": "_doc",
|
|
|
|
"_id": "1",
|
2020-03-02 15:40:05 -05:00
|
|
|
"_score": null,
|
2020-03-02 10:08:03 -05:00
|
|
|
"_source": {
|
|
|
|
"@timestamp": "2020-12-07T11:06:07.000Z",
|
|
|
|
"agent": {
|
|
|
|
"id": "8a4f500d"
|
|
|
|
},
|
|
|
|
"event": {
|
|
|
|
"category": "process"
|
|
|
|
},
|
|
|
|
"process": {
|
|
|
|
"name": "cmd.exe",
|
|
|
|
"path": "C:\\Windows\\System32\\cmd.exe"
|
|
|
|
}
|
2020-03-02 15:40:05 -05:00
|
|
|
},
|
|
|
|
"sort" : [1607339167000]
|
2020-03-02 10:08:03 -05:00
|
|
|
}
|
|
|
|
]
|
|
|
|
}
|
|
|
|
}
|
|
|
|
----
|
|
|
|
// TESTRESPONSE[s/"took": 3/"took": $body.took/]
|
|
|
|
|
|
|
|
[discrete]
|
|
|
|
[[eql-search-specify-event-type-field]]
|
|
|
|
=== Specify an event type field
|
|
|
|
|
2020-03-04 08:02:38 -05:00
|
|
|
The EQL search API uses `event.category` as the required <<eql-required-fields,event
|
|
|
|
category field>> by default. You can use the `event_category_field` parameter to specify
|
|
|
|
another event category field.
|
2020-03-02 10:08:03 -05:00
|
|
|
|
|
|
|
For example, the following request specifies `file.type` as the event type
|
|
|
|
field.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
GET sec_logs/_eql/search
|
|
|
|
{
|
2020-03-04 08:02:38 -05:00
|
|
|
"event_category_field": "file.type",
|
2020-03-02 15:40:05 -05:00
|
|
|
"timestamp_field": "@timestamp",
|
2020-03-02 10:08:03 -05:00
|
|
|
"query": """
|
|
|
|
file where agent.id == "8a4f500d"
|
|
|
|
"""
|
|
|
|
}
|
|
|
|
----
|
|
|
|
|
|
|
|
[discrete]
|
|
|
|
[[eql-search-specify-timestamp-field]]
|
|
|
|
=== Specify a timestamp field
|
|
|
|
|
|
|
|
The EQL search API uses `@timestamp` as the required <<eql-required-fields,event
|
|
|
|
timestamp field>> by default. You can use the `timestamp_field` parameter to
|
|
|
|
specify another timestamp field.
|
|
|
|
|
|
|
|
For example, the following request specifies `file.accessed` as the event
|
|
|
|
timestamp field.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
GET sec_logs/_eql/search
|
|
|
|
{
|
|
|
|
"timestamp_field": "file.accessed",
|
2020-03-04 08:02:38 -05:00
|
|
|
"event_category_field": "event.category",
|
2020-03-02 10:08:03 -05:00
|
|
|
"query": """
|
|
|
|
file where (file.size > 1 and file.type == "file")
|
|
|
|
"""
|
|
|
|
}
|
|
|
|
----
|
|
|
|
|
|
|
|
[discrete]
|
|
|
|
[[eql-search-filter-query-dsl]]
|
|
|
|
=== Filter using query DSL
|
|
|
|
|
|
|
|
You can use the `filter` parameter to specify an additional query using
|
|
|
|
<<query-dsl,query DSL>>. This query filters the documents on which the EQL query
|
|
|
|
runs.
|
|
|
|
|
|
|
|
For example, the following request uses a `range` query to filter the `sec_logs`
|
|
|
|
index down to only documents with a `file.size` value greater than `1` but less
|
|
|
|
than `1000000` bytes. The EQL query in `query` parameter then runs on these
|
|
|
|
filtered documents.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
GET sec_logs/_eql/search
|
|
|
|
{
|
2020-03-04 08:02:38 -05:00
|
|
|
"event_category_field": "event.category",
|
2020-03-02 15:40:05 -05:00
|
|
|
"timestamp_field": "@timestamp",
|
2020-03-02 10:08:03 -05:00
|
|
|
"filter": {
|
|
|
|
"range" : {
|
|
|
|
"file.size" : {
|
|
|
|
"gte" : 1,
|
|
|
|
"lte" : 1000000
|
|
|
|
}
|
|
|
|
}
|
|
|
|
},
|
|
|
|
"query": """
|
|
|
|
file where (file.type == "file" and file.name == "cmd.exe")
|
|
|
|
"""
|
|
|
|
}
|
|
|
|
----
|