Commit Graph

1646 Commits

Author SHA1 Message Date
Tianli Feng ae0dc2ff5b [TEST] Fix unit test failure in RestHighLevelClientTests (#36)
* fix testProvidedNamedXContents by modifying the assertion

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:10 -06:00
Nick Knize 6f2cbdf0b8 [PURIFY] Cleanup build and unblock HLRC integration tests (#33)
This commit cleans up the following:

* Remove unused imports
* Remove ILM settings in hlrc testCluster formation
* Comment out security users settings in ElasticsearchNode creation for build-tools tests

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:10 -06:00
Rabi Panda 171ce992e4 Fix compilation issues for tests. (#29)
Some of the tests that use the x-pack code are failing compilation. This PR cleans up the references to fix the issue.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:10 -06:00
Nick Knize d91ce2bd00 [PURIFY] remove all trace of x-pack transforms (#31)
This commit removes all trace of Elastic licensed transforms.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:09 -06:00
Nick Knize 3f168ac85c [PURIFY] Revert "Move data stream transport and rest action to xpack (#59593)" (#28)
This commit reverts commit 2a89e13e43. Relicensing data streams from OSS to XPack.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:09 -06:00
Nick Knize 82300bbe3d [PURIFY] remove all trace of x-pack monitoring (#22)
This commit removes all trace of Elastic licensed monitoring

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:09 -06:00
Nick Knize 70a07216bb [PURIFY] remove all trace of x-pack watcher (#21)
This commit removes all trace of Elastic licensed watcher

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize 4aee8e56db [PURIFY] remove all trace of x-pack index lifecycle management (#20)
This commit removes all trace of Elastic licensed ILM.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize 46f37bfa81 [PURIFY] remove all trace of x-pack migration (#19)
This commit removes all trace of Elastic licensed Migration

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize 3769b2c6a4 [PURIFY] remove all trace of x-pack rollups (#17)
This commit removes all trace of Elastic licensed rollups

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize fb7ff93c42 [PURIFY] remove all trace of x-pack security (#16)
This commit removes all trace of the security high level rest client and other reference to x-pack security

Co-authored-by: Rabi Panda <rabipanda@icloud.com>
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize 6036fe822e [PURIFY] Add InferenceConfig to org.elasticsearch.client.analytics (#15)
This commits adds InferenceConfig back to org.elasticsearch.client.analytics for use in InferencePipelineAggregationBuilder.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
Nick Knize b5cd4c417c [PURIFY] remove all trace of x-pack ml (#14)
This commit removes all trace of x-pack ml.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
Nick Knize 1cdbc63917 [PURIFY] remove all trace of x-pack graph (#13)
This commit removes all trace of Elastic licensed graph feature

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
Nick Knize 125958eb2c Remove the Elastic license file, all checks for this license and the license REST APIs. (#12)
Co-authored-by: Rabi Panda <rabipanda@icloud.com>
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
Nick Knize 0dd4f9f281 [PURIFY] remove all trace of x-pack ccr (#11)
This committ removes all trace of Elastic licensed CCR.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
Nick Knize 479dd4b42a [PURIFY] remove all trace of x-pack async-search (#10)
This commit removes all trace of Elastic licensed asyc-search

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
Nick Knize 204122aae4 [PURIFY] remove all trace of x-pack enrichment processor (#9)
This commit removes all trace of the Elastic licensed enrichment processor.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:06 -06:00
Nick Knize 3a52e9ddc1 [PURIFY] update build.gradle files to ensure build completes; gradle check fails (#7)
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:06 -06:00
Nick Knize 168f5e825f [PURIFY] remove all trace of x-pack eql (#5)
This commit removes all trace of EQL from the sanitized fork.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:06 -06:00
Jay Modi 55e79dd286
Mute IndicesClientIT.testDataStreams (#65859)
This commit mutes IndicesClientIT.testDataStreams as this test is
failing in CI intermittently.

Relates #60746
Relates #60461
2020-12-03 14:36:35 -07:00
Sylvain Wallez f7e840b012
[client] Add simple support for gzip compression
Adds a `RestClient.setCompressionEnabled()` setting that will gzip-
compress request bodies and add a `Accept-Encoding: gzip` header so
that the ES server can send compressed responses.
2020-11-20 07:38:21 +01:00
Sylvain Wallez b2475f9ccf
Fix parsing RareTerms aggregation response in RestHighLevelClient (#65144)
Backport of #64454

- Add LongRareTerms and StringRareTerms to the DefaultNamedXContents,
  ensure that the response of RareTerms aggregation can be parsed
  correctly.
- Add testSearchWithRareTermsAgg method to test the response of
  RareTerms aggregation can be parsed correctly.
- Add some test code to ensure the AggregationsTests can execute
  successfully.

Co-authored-by: bellengao <gbl_long@163.com>
2020-11-17 17:43:51 +01:00
Costin Leau 76e73fec79
EQL: Add option for returning results from the tail of the stream (#64869) (#65040)
Introduce option for specifying whether the results are returned from
the tail (end) of the stream or the head (beginning).
Improve sequencing algorithm by significantly eliminating the number
of in-flight sequences for spare datasets.
Refactor the sequence class by eliminating some of the redundant code.
Change matching behavior for tail sequences.
Return results based on their first entry ordinal instead of
insertion order (which was ordered on the last match ordinal).
Randomize results position inside test suite.

Close #58646

(cherry picked from commit e85d9d1bbee13ad408e789fd62efb30bc8d223f2)
(cherry picked from commit 452c674a10cdc16dced3cde7babf5d5a9d64a6d9)
2020-11-14 13:44:17 +02:00
Tanguy Leroux 09ff421d4f
Fix TaskIT (#64724) (#64848)
Closes #64056
2020-11-10 11:06:25 +01:00
Ryan Ernst 6be4acd509
Use no-kpi subdomains for tests downloading artifacts (#64502)
This commit converts build code that downloads distributions or other
artifacts to use the new no-kpi subdomain, and removes the formerly used
no-kpi header.
2020-11-07 11:30:52 -08:00
Andrei Dan 460c1364c0
Tests: fix testBulkProcessorConcurrentRequestsReadOnlyIndex (#64633) (#64657)
Executing a MultiGetRequest with no items will fail. This makes sure there
is always at least one item in the request.

(cherry picked from commit bd4703250fe331296b8613b277ea25c8bef1dcd9)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-11-06 09:10:51 +00:00
Armin Braun 1880bcdc09
Add REST Test for Snapshot Clone API (#63863) (#63881)
Adds snapshot clone REST tests and HLRC support for the API.
2020-10-20 09:48:03 +02:00
Ryland Herrick 7e8769a666
EQL: make allow_no_indices true by default (#63573) (#63645)
* Allow all indices options variants
Irrespective of allow_no_indices value, throw VerificationException when
there is no index validated

Co-authored-by: Andrei Stefan <astefan@users.noreply.github.com>
2020-10-14 03:41:04 +03:00
Przemysław Witek acbd48f834
[ML] Allow setting num_top_classes to a special value -1 (#63587) (#63602) 2020-10-13 13:57:50 +02:00
Przemysław Witek bd761cce1d
[ML] Validate that AucRoc has the data necessary to be calculated (#63302) (#63454) 2020-10-08 09:52:15 +02:00
Lisa Cawley 8f76c89cd3
[7.x][DOCS] Add feature_importance_baseline to get trained model API (#63279) (#63336)
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
2020-10-06 10:08:34 -07:00
Hendrik Muhs 058c55da6a [Transform] disallow field and script being empty for group sources (#63313)
fail validation earlier when field and script are both missing in a group source
2020-10-06 16:59:02 +02:00
Yang Wang 7969fbb4ab
Cache API key doc to reduce traffic to the security index (#59376) (#63319)
Getting the API key document form the security index is the most time consuing part
of the API Key authentication flow (>60% if index is local and >90% if index is remote).
This traffic is now avoided by caching added with this PR.

Additionally, we add a cache invalidator registry so that clearing of different caches will
be managed in a single place (requires follow-up PRs).
2020-10-06 23:49:23 +11:00
Andrei Stefan 76bba601ab
Remove case_sensitive request option (#63218) (#63244)
Make EQL case sensitive by default and adapt some of the string functions
Remove the case sensitive option from Between string function
Add case_insensitive option to term and wildcard queries usage

(cherry picked from commit 7550e0664c8c2f1f13519036c759b1e76345551f)
2020-10-05 22:04:42 +03:00
Armin Braun cf75abb021
Optimize XContentParserUtils.ensureExpectedToken (#62691) (#63253)
We only ever use this with `XContentParser` no need to make it inline
worse by forcing the lambda and hence dynamic callsite here.
=> Extraced the exception formatting code path that is likely very cold
to a separate method and removed the lambda usage in hot loops by simplifying
the signature here.
2020-10-05 19:08:32 +02:00
Benjamin Trent 1e63313c19
[ML] adds feature_importance_baseline object to model metadata (#63172) (#63237)
this adds the new field `feature_importance_baseline` and allows it to be optionally be included in the model's metadata.

Related to: https://github.com/elastic/ml-cpp/pull/1522
2020-10-05 09:33:38 -04:00
Costin Leau 8c4503bcc3 EQL: Change default indices options (#63192)
Ignore by default unavailable indices (same as ES) and verify that
allowNoIndices is set to false since at least one index is required
for validating the query.

Fix #62986

(cherry picked from commit fd75ac27223cd1b699b8d9c311dc401a39f9e0c8)
2020-10-05 14:21:56 +03:00
Benjamin Trent cfcf973259
[7.x] [ML] renames */inference* apis to */trained_models* (#63097) (#63136)
* [ML] renames */inference* apis to */trained_models* (#63097)

This commit renames all `inference` CRUD APIs to `trained_models`.

This aligns with internal terminology, documentation, and use-cases.
2020-10-02 07:34:28 -04:00
Benjamin Trent 535f8a434b
Revert "[ML] adding `baseline` field to total_feature_importance objects (#63098) (#63125)" (#63144)
This reverts commit 95242eccee.
2020-10-02 07:03:15 -04:00
Benjamin Trent 95242eccee
[ML] adding `baseline` field to total_feature_importance objects (#63098) (#63125)
This adds a new `baseline` field to the feature importance values. 

This field contains the baseline importance for a given feature and class.
2020-10-01 09:48:07 -04:00
Costin Leau a6b903b783 EQL: Remove unused classes from reponse API (#62134)
Remove Count class and related artifacts since that functionality is not
(yet) available.
Update parser name for better error reporting.

Fix #62131

(cherry picked from commit 060f500346788c4c5d0b3b9c045facec5d677d3d)
2020-09-30 15:45:30 +03:00
Przemysław Witek d677a2b8ee
[7.x] [ML] Implement AucRoc metric for classification - HLRC (#62304) (#63058) 2020-09-30 14:04:10 +02:00
Benjamin Trent 0b3af242d4
[ML] fixing classification feature importance parsing (#63003) (#63015)
Classification feature importance supports various types in the class name:
- string
- boolean
- numerical

The xcontent parsing on the server side and the HLRC side should support and test these types.
2020-09-29 10:54:35 -04:00
Dimitris Athanasiou 7f6c1ff5b4
[7.x][ML] Remove top level importance from classification inference results (#62486) (#62964)
As we have decided top level importance for classification is not useful,
it has been removed from the results from the training job. This commit
also removes them from inference.

Backport of #62486
2020-09-29 10:58:48 +03:00
Andrei Dan 25106ba58f
HLRC: add support for the wait_for_snapshot ILM action (#62333) (#62931)
(cherry picked from commit b8a10b3995669954f0e8c6b3512c50da6c76d48d)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-09-28 09:54:24 +01:00
Andrei Dan 3590a77b2b
HLRC: add support for the searchable_snapshot ILM action (#62323) (#62887)
(cherry picked from commit 681eb58718c4cce9ed18a835f4eadb06997e91a0)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-09-24 16:45:50 +01:00
Hendrik Muhs a70389015d [Transform] Return parsed count for get transform stats (#62809)
In case of more than 500 transforms, get and stats return paged results which can be requested using
page parameters. For >500 transforms count wasn't parsed out of the server response but taken from
size of the list of transforms.

The change also adds client/server hlrc tests and fixes a wrong type for count in get.

fixes #56245
2020-09-24 08:38:07 +02:00
Marios Trivyzas 1e72144847
EQL: Remove support for `=` for comparisons (#62756) (#62775)
Since `=` is rarely used and is undocumented we its support for
equality comparisons keeping `==` as the only option. `=` is now only
used for assignments like in `maxspan=10m`.

Closes: #62650
(cherry picked from commit ad5ae4d887b5c2feca2d0e874d7bdf738e3fd54e)
2020-09-22 20:56:04 +02:00
Benjamin Trent e163559e4c
[7.x] [ML] Add new include flag to GET inference/<model_id> API for model training metadata (#61922) (#62620)
* [ML] Add new include flag to GET inference/<model_id> API for model training metadata (#61922)

Adds new flag include to the get trained models API
The flag initially has two valid values: definition, total_feature_importance.
Consequently, the old include_model_definition flag is now deprecated.
When total_feature_importance is included, the total_feature_importance field is included in the model metadata object.
Including definition is the same as previously setting include_model_definition=true.

* fixing test

* Update x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ml/action/GetTrainedModelsRequestTests.java
2020-09-18 10:07:35 -04:00