[role="xpack"] [testenv="basic"] [[eql-search]] == Run an EQL search dev::[] To start using EQL in {es}, first ensure your event data meets <>. You can then use the <> to search event data stored in one or more {es} indices. .*Example* [%collapsible] ==== To get started, ingest or add the data to an {es} index. The following <> request adds some example log data to the `sec_logs` index. This log data follows the {ecs-ref}[Elastic Common Schema (ECS)]. [source,console] ---- PUT /sec_logs/_bulk?refresh {"index":{"_index" : "sec_logs", "_id" : "1"}} { "@timestamp": "2020-12-06T11:04:05.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } } {"index":{"_index" : "sec_logs", "_id" : "2"}} { "@timestamp": "2020-12-06T11:04:07.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "file" }, "file": { "accessed": "2020-12-07T11:07:08.000Z", "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe", "type": "file", "size": 16384 }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } } {"index":{"_index" : "sec_logs", "_id" : "3"}} { "@timestamp": "2020-12-07T11:06:07.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } } {"index":{"_index" : "sec_logs", "_id" : "4"}} { "@timestamp": "2020-12-07T11:07:08.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "file" }, "file": { "accessed": "2020-12-07T11:07:08.000Z", "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe", "type": "file", "size": 16384 }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } } {"index":{"_index" : "sec_logs", "_id" : "5"}} { "@timestamp": "2020-12-07T11:07:09.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "regsvr32.exe", "path": "C:\\Windows\\System32\\regsvr32.exe" } } ---- // TESTSETUP [TIP] ===== You also can set up {beats-ref}/getting-started.html[{beats}], such as {auditbeat-ref}/auditbeat-getting-started.html[{auditbeat}] or {winlogbeat-ref}/winlogbeat-getting-started.html[{winlogbeat}], to automatically send and index your event data in {es}. See {beats-ref}/getting-started.html[Getting started with {beats}]. ===== You can now use the EQL search API to search this index using an EQL query. The following request searches the `sec_logs` index using the EQL query specified in the `query` parameter. The EQL query matches events with an `event.category` of `process` that have a `process.name` of `cmd.exe`. [source,console] ---- GET /sec_logs/_eql/search { "query": """ process where process.name == "cmd.exe" """ } ---- Because the `sec_log` index follows the ECS, you don't need to specify the required <> fields. The request uses the `event.category` and `@timestamp` fields by default. The API returns the following response containing the matching events. Events in the response are sorted by timestamp, converted to milliseconds since the https://en.wikipedia.org/wiki/Unix_time[Unix epoch], in ascending order. [source,console-result] ---- { "took": 60, "timed_out": false, "hits": { "total": { "value": 2, "relation": "eq" }, "events": [ { "_index": "sec_logs", "_type": "_doc", "_id": "1", "_score": null, "_source": { "@timestamp": "2020-12-06T11:04:05.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } }, "sort": [ 1607252645000 ] }, { "_index": "sec_logs", "_type": "_doc", "_id": "3", "_score": null, "_source": { "@timestamp": "2020-12-07T11:06:07.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } }, "sort": [ 1607339167000 ] } ] } } ---- // TESTRESPONSE[s/"took": 60/"took": $body.took/] ==== [discrete] [[eql-search-sequence]] === Search for a sequence of events Many query languages allow you to match single events. However, EQL's <> lets you match an ordered series of events. .*Example* [%collapsible] ==== The following EQL search request matches a sequence that: . Starts with an event with: + -- * An `event.category` of `file` * A `file.name` of `cmd.exe` -- . Followed by an event with: + -- * An `event.category` of `process` * A `process.name` that contains the substring `regsvr32` -- [source,console] ---- GET /sec_logs/_eql/search { "query": """ sequence [ file where file.name == "cmd.exe" ] [ process where stringContains(process.name, "regsvr32") ] """ } ---- The API returns the following response. Matching events in the `hits.sequences.events` property are sorted by <>, converted to milliseconds since the https://en.wikipedia.org/wiki/Unix_time[Unix epoch], in ascending order. [source,console-result] ---- { "took": 60, "timed_out": false, "hits": { "total": { "value": 1, "relation": "eq" }, "sequences": [ { "events": [ { "_index": "sec_logs", "_type": "_doc", "_id": "4", "_score": null, "_source": { "@timestamp": "2020-12-07T11:07:08.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "file" }, "file": { "accessed": "2020-12-07T11:07:08.000Z", "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe", "type": "file", "size": 16384 }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } }, "fields": { "@timestamp": [ "1607339228000" ] }, "sort": [ 1607339228000 ] }, { "_index": "sec_logs", "_type": "_doc", "_id": "5", "_score": null, "_source": { "@timestamp": "2020-12-07T11:07:09.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "regsvr32.exe", "path": "C:\\Windows\\System32\\regsvr32.exe" } }, "fields": { "@timestamp": [ "1607339229000" ] }, "sort": [ 1607339229000 ] } ] } ] } } ---- // TESTRESPONSE[s/"took": 60/"took": $body.took/] You can further constrain matching event sequences using the `by` keyword. The following EQL search request adds `by agent.id` to each event item. This ensures events matching the sequence share the same `agent.id` field value. [source,console] ---- GET /sec_logs/_eql/search { "query": """ sequence [ file where file.name == "cmd.exe" ] by agent.id [ process where stringContains(process.name, "regsvr32") ] by agent.id """ } ---- Because the `agent.id` field is shared across all events in the sequence, it can be included using `sequence by`. The following query is equivalent to the prior one. [source,console] ---- GET /sec_logs/_eql/search { "query": """ sequence by agent.id [ file where file.name == "cmd.exe" ] [ process where stringContains(process.name, "regsvr32") ] """ } ---- The API returns the following response. The `hits.sequences.join_keys` property contains the shared `agent.id` value for each matching event. [source,console-result] ---- { "took": 60, "timed_out": false, "hits": { "total": { "value": 1, "relation": "eq" }, "sequences": [ { "join_keys": [ "8a4f500d" ], "events": [ { "_index": "sec_logs", "_type": "_doc", "_id": "4", "_score": null, "_source": { "@timestamp": "2020-12-07T11:07:08.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "file" }, "file": { "accessed": "2020-12-07T11:07:08.000Z", "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe", "type": "file", "size": 16384 }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } }, "fields": { "@timestamp": [ "1607339228000" ] }, "sort": [ 1607339228000 ] }, { "_index": "sec_logs", "_type": "_doc", "_id": "5", "_score": null, "_source": { "@timestamp": "2020-12-07T11:07:09.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process" }, "process": { "name": "regsvr32.exe", "path": "C:\\Windows\\System32\\regsvr32.exe" } }, "fields": { "@timestamp": [ "1607339229000" ] }, "sort": [ 1607339229000 ] } ] } ] } } ---- // TESTRESPONSE[s/"took": 60/"took": $body.took/] ==== [discrete] [[eql-search-specify-event-category-field]] === Specify an event category field The EQL search API uses `event.category` as the required <> by default. You can use the `event_category_field` parameter to specify another event category field. .*Example* [%collapsible] ==== The following request specifies `file.type` as the event category field. [source,console] ---- GET /sec_logs/_eql/search { "event_category_field": "file.type", "query": """ file where agent.id == "8a4f500d" """ } ---- ==== [discrete] [[eql-search-specify-timestamp-field]] === Specify a timestamp field The EQL search API uses `@timestamp` as the required <> by default. You can use the `timestamp_field` parameter to specify another timestamp field. .*Example* [%collapsible] ==== The following request specifies `file.accessed` as the event timestamp field. [source,console] ---- GET /sec_logs/_eql/search { "timestamp_field": "file.accessed", "query": """ file where (file.size > 1 and file.type == "file") """ } ---- ==== [discrete] [[eql-search-filter-query-dsl]] === Filter using query DSL You can use the EQL search API's `filter` parameter to specify an additional query using <>. This query filters the documents on which the EQL query runs. .*Example* [%collapsible] ==== The following request uses a `range` query to filter the `sec_logs` index down to only documents with a `file.size` value greater than `1` but less than `1000000` bytes. The EQL query in `query` parameter then runs on these filtered documents. [source,console] ---- GET /sec_logs/_eql/search { "filter": { "range" : { "file.size" : { "gte" : 1, "lte" : 1000000 } } }, "query": """ file where (file.type == "file" and file.name == "cmd.exe") """ } ---- ==== [discrete] [[eql-search-case-sensitive]] === Run a case-sensitive EQL search By default, matching for EQL queries is case-insensitive. You can use the EQL search API's `case_sensitive` parameter to toggle case sensitivity on or off. .*Example* [%collapsible] ==== The following search request contains a query that matches `process` events with a `process.path` containing `System32`. Because the `case_sensitive` parameter is `true`, this query only matches `process.path` values containing `System32` with the exact same capitalization. A `process.path` value containing `system32` or `SYSTEM32` would not match this query. [source,console] ---- GET /sec_logs/_eql/search { "case_sensitive": true, "query": """ process where stringContains(process.path, "System32") """ } ---- ====