Commit Graph

178 Commits

Author SHA1 Message Date
Costin Leau a8911802d3 EQL: transform query AST into queryDSL (#52432)
(cherry picked from commit 94cef29df259319dfe2a3bf92d3f1a42d7e45781)
2020-02-25 17:53:59 +02:00
Aleksandr Maus a6f5b4bb78
Unmute EqlActionIT (#52757)
Related to https://github.com/elastic/elasticsearch/issues/52737
2020-02-25 10:22:07 -05:00
Aleksandr Maus b2cb38ccf5
EQL: Expand verification tests (#52664) (#52725)
* EQL: Expand verification tests (#52664)

Expand verification tests
Fix some error messaging consistency in EqlParser

Related to https://github.com/elastic/elasticsearch/issues/51873

* Adjust for 7.x compatibility
2020-02-25 07:19:33 -05:00
Mark Vieira 025352f0a4
Mute EqlActionIT 2020-02-24 16:06:30 -08:00
Aleksandr Maus a7bdb0b456
EQL: Add integration tests harness to test EQL feature parity with original implementation (#52248) (#52675)
The tests use the original test queries from
https://github.com/endgameinc/eql/blob/master/eql/etc/test_queries.toml
for EQL implementation correctness validation.
The file test_queries_unsupported.toml serves as a "blacklist" for the
queries that we do not support. Currently all of the queries are
blacklisted. Over the time the expectation is to eventually have an
empty "blacklist" when all of the queries are fully supported.

The tests use the original test vector from
https://raw.githubusercontent.com/endgameinc/eql/master/eql/etc/test_data.json.

Only one EQL and the response is stubbed for now to match the expected
output from that query. This part would need some tweaking after EQL is
fully wired.

Related to https://github.com/elastic/elasticsearch/issues/49581
2020-02-24 12:46:59 -05:00
Jay Modi f3f6ff97ee
Single instance of the IndexNameExpressionResolver (#52604)
This commit modifies the codebase so that our production code uses a
single instance of the IndexNameExpressionResolver class. This change
is being made in preparation for allowing name expression resolution
to be augmented by a plugin.

In order to remove some instances of IndexNameExpressionResolver, the
single instance is added as a parameter of Plugin#createComponents and
PersistentTaskPlugin#getPersistentTasksExecutor.

Backport of #52596
2020-02-21 07:50:02 -07:00
Igor Motov e5b21a3fc6
Add HLRC for EQL search (#52550)
Adds EQL HLRC client with the search method.

Relates to #51961
2020-02-21 08:44:08 -05:00
Costin Leau 5373a77fb9 QL: Extract common Failure class (#52281)
Shared across SQL and EQL

(cherry picked from commit 1aeda20d3ec3d6c885de03c6043dd1e8eab9f230)
2020-02-13 14:35:15 +02:00
Costin Leau 26900bfb05 EQL: Add infra for planning and query folding (#52065)
Actual folding not yet in place (TBD)

(cherry picked from commit d52b96f273a94c90e475a5035cd57baa086fb0c0)
2020-02-12 18:51:42 +02:00
Andrei Stefan a3ebacfcf3
52169 & 52172 7x backport (#52256)
* Extract common optimizer tests (#52169)

(cherry picked from commit e5ad72bc22e9ec0686ab582195f0032efcb880bf)

* Hook in the optimizer rules (#52172)

(cherry picked from commit 1f90d8cc56052fbf2af604e72f9f5ca73f5e75d5)
2020-02-12 11:20:03 +02:00
James Rodewig d68a4ec82e
[7.x] Permit EQL feature flag in release builds (#52201) (#52214)
7.x backport of #52201

Provides a path to set register the EQL feature flag in release builds.
This enables EQL in release builds so that release docs tests pass.

Release docs tests do not have infrastructure in place to only register
snippets from included portions of the docs, they instead include all
docs snippets.

Since EQL can not be enabled in release builds, this meant that the EQL
snippets fail in the release docs tests.

This adds the ability to enable EQL in the release docs tests. This
system property will be removed when EQL is ready for release.
2020-02-11 11:49:49 -05:00
Andrei Stefan 2f1631d9d0
Telemetry data initial implementation (#51715) (#52175)
(cherry picked from commit f1d1cceacaacf226fcd2459f34689843b822fe4b)
2020-02-11 09:15:47 +02:00
Jay Modi 3edadfefd0 RestHandlers declare handled routes (#52123)
This commit changes how RestHandlers are registered with the
RestController so that a RestHandler no longer needs to register itself
with the RestController. Instead the RestHandler interface has new
methods which when called provide information about the routes
(method and path combinations) that are handled by the handler
including any deprecated and/or replaced combinations.

This change also makes the publication of RestHandlers safe since they
no longer publish a reference to themselves within their constructors.

Closes #51622

Co-authored-by: Jason Tedor <jason@tedor.me>

Backport of #51950
2020-02-09 22:48:32 -07:00
Costin Leau bd6d9e063c EQL: Add missing commit messages for #51940
* EQL: Plug query params into the AstBuilder (#51886)

As the eventType is customizable, plug that into the parser based on the
given request.

(cherry picked from commit 5b4a3a3c07eacbc339cbd4c05a3621d056cc8d60)

* EQL: Add field resolution and verification (#51872)

Add basic field resolution inside the Analyzer and a basic Verifier to
check for any unresolved fields.

(cherry picked from commit 7087358ae2fb212811d480ec8641a46167946c82)

* EQL: Introduce basic execution pipeline (#51809)

Add main classes that form the 'execution' pipeline are added - most of
them have no functionality; the purpose of this PR is to add flesh out
the contract between the various moving parts so that work can start on
them independently.

(cherry picked from commit 9a1bae50a49af7fe8467b74b154c0d82c6bb9a19)

* EQL: Add AstBuilder to convert to QL tree (#51558)

* EQL: Add AstBuilder visitors
* EQL: Add tests for wildcards and sets
* EQL: Fix licensing
* EQL: Fix ExpressionTests.java license
* EQL: Cleanup imports
* EQL: PR feedback and remove LiteralBuilder
* EQL: Split off logical plan from expressions
* EQL: Remove stray import
* EQL: Add predicate handling for set checks
* EQL: Remove commented out dead code
* EQL: Remove wildcard test, wait until analyzer

(cherry picked from commit a462700f9c8e1fb977d62d42eb0077403b8fa98b)

* EQL grammar updates and tests (#49658)

* EQL: Additional tests and grammar updates
* EQL: Add backtick escaped identifiers
* EQL: Adding keywords to language
* EQL: Add checks for unsupported syntax
* EQL: Testing updates and PR feedback
* EQL: Add string escapes
* EQL: Cleanup grammar for identifier
* EQL: Remove tabs from .eql tests

(cherry picked from commit 6f1890bf2d52cabdfd1e7848fb481cf54b895f25)
2020-02-05 16:53:42 +02:00
Costin Leau 6ff0e411a8
EQL: backport updates to 7.x (#51940) 2020-02-05 16:45:58 +02:00
Ryan Ernst 21224caeaf Remove comparison to true for booleans (#51723)
While we use `== false` as a more visible form of boolean negation
(instead of `!`), the true case is implied and the true value does not
need to explicitly checked. This commit converts cases that have slipped
into the code checking for `== true`.
2020-01-31 16:35:43 -08:00
Aleksandr Maus d4f6f38150
EQL: Fix #51541: [CI] unknown setting [xpack.eql.enabled] in release-tests (#51699) (#51770)
Fixes #51541
Co-authored-by: Igor Motov <igor@motovs.org>
2020-01-31 15:14:27 -05:00
Aleksandr Maus 0d21d9e2c5
EQL: Enable QA/rest integration tests for snapshot builds only (#51624) (#51645)
* Related to #51541: [CI] unknown setting [xpack.eql.enabled] in release-tests
2020-01-29 16:38:52 -05:00
Aleksandr Maus eb1ed2a35f Compilation fixes for 7.x 2020-01-27 16:23:36 -05:00
Aleksandr Maus d8f1735e39 Add xpack.eql.enabled feature flag, disabled by default. Enabled only for integration tests. (#51370)
Related to https://github.com/elastic/elasticsearch/issues/49581
2020-01-27 15:15:22 -05:00
Costin Leau d049de5b72 EQL: import QL into EQL (#50904)
Link QL into the new build file
Remove duplicate classes and use the new ql package
Update Exception hierarchy on top of QlException
2020-01-27 15:13:22 -05:00
Igor Motov c184411456 EQL: Replace EqlSearchResponse.Hits parser with ObjectParser (#50925)
Replaces the existing hand-build Hits parser with a
ConstructingObjectParser version.

Relates to #49581
2020-01-27 15:13:09 -05:00
Igor Motov 88cc30c0d8 EQL: Remove list classes from EqlSearchResponse (#50870)
Removes unnecessary classes from EqlSearchResponse that just represent
lists of other elements.

Relates to #49581
2020-01-27 15:13:00 -05:00
Aleksandr Maus d715176c00 Add more Eql REST API validation integration tests, clean up request implementation (#50822) 2020-01-27 15:12:48 -05:00
Igor Motov 628083183f EQL: Make EqlSearchResponse immutable (#50810)
Refactors EqlSearchResponse to make it immutable

Relates to #49581
2020-01-27 15:12:07 -05:00
Aleksandr Maus 31d2d01e25 Correct search_after handling (#50629) 2020-01-27 15:11:51 -05:00
Aleksandr Maus 79875ce4d9 Initial EQL rest API implementation (#49768) 2020-01-27 15:11:41 -05:00
Costin Leau 10a16d15d1 Add draft EQL grammar and expression tree 2020-01-27 15:11:18 -05:00