Commit Graph

51649 Commits

Author SHA1 Message Date
Tanguy Leroux 07ad742b60 Enable prewarming by default for searchable snapshots (#56201)
Now searchable snapshots directories respect the repository 
rate limitations (#55952) we can enable prewarming by default 
for shards.
2020-05-06 10:18:34 +02:00
Rory Hunter b8b4ebd089 Add version 7.9.0 2020-05-06 09:12:14 +01:00
Tanguy Leroux 131a3911eb Replace BlobContainerWrapper by FilterBlobContainer (#56200)
A FilterBlobContainer class was introduced in #55952 and it delegates
 its behavior to a given BlobContainer while allowing to override 
only necessary methods.

This commit replaces the existing BlobContainerWrapper class from 
the test framework with the new FilterBlobContainer from core.
2020-05-06 10:05:43 +02:00
Luca Cavanna ef66018119 [DOCS] Async search: clarify behaviour when submit returns final results (#55934)
Closes #55636
2020-05-06 10:01:23 +02:00
Julie Tibshirani bd7a2d2b01 Mute the geogrid agg circuit breaker tests. 2020-05-05 18:09:07 -07:00
Julie Tibshirani dc738e34d2 Mute the mixed cluster 80_transform_jobs_crud test. 2020-05-05 17:58:17 -07:00
Julie Tibshirani 7c55db9b04 Mute TransformSurvivesUpgradeIT#testTransformRollingUpgrade. 2020-05-05 17:37:17 -07:00
Jake Landis a22690c9ca
[7.x] Ensure that the monitoring export exceptions are logged. (#56237) (#56251)
If an exception occurs while flushing a bulk the cause of the exception
can be lost. This commit ensures that cause of the exception is carried
forward and gets logged.
2020-05-05 19:24:26 -05:00
Julie Tibshirani 133ba2691f Make sure to mute all 80_transform_jobs_crud tests. 2020-05-05 17:07:59 -07:00
Julie Tibshirani 49de092b38 Mute RegressionIT.testTwoJobsWithSameRandomizeSeedUseSameTrainingSet. 2020-05-05 16:25:36 -07:00
Jake Landis f3721fa88c
[7.x] Prevent stack overflow for numerous grok patterns. (#55899) (#56065)
This was noticed for a pipeline that was defining hundreds of
grok patterns inline with a single grok processor.

The recursive call used to translate a Grok pattern to a regular
expression can overflow the stack. This commit converts that method 
to an iterative method. 

Co-authored-by: Przemko Robakowski <probakowski@users.noreply.github.com>
2020-05-05 16:52:56 -05:00
Bogdan Pintea 47250b14a4
SQL: Add BigDecimal support to JDBC (#56015) (#56220)
* SQL: Add BigDecimal support to JDBC (#56015)

* Introduce BigDecimal support to JDBC -- fetching

This commit adds support for the getBigDecimal() methods.

* Allow BigDecimal params in double range

A prepared statement will now accept a BigDecimal parameter as a proxy
for a double, if the conversion is lossless.

(cherry picked from commit e9a873ad7f387682e3472110b1d7c0514bd347c9)

* Fix compilation error

Dimond notation with anonymous inner classes not avail in Java8.
2020-05-05 23:19:36 +02:00
Bogdan Pintea f159fd8a20
Fix test on incompatible client versions (#56234) (#56241)
The incomatible client version test is changed to:
- iterate on all versions prior to the allowed one_s;
- format the exception message just as the server does it.

The defect stemed from the fact that the clients will not send a
version's qualifier, but just major.minor.revision, so the raised
error/exception_message won't contain it, while the test expected it.

(cherry picked from commit 4a81c8f7a1f4573e3be95f346d9fb18772b297ee)
2020-05-05 23:18:29 +02:00
Julie Tibshirani 63062ec7bd Mute ClassificationIT.testDependentVariableCardinalityTooHighButWithQueryMakesItWithinRange. 2020-05-05 13:48:35 -07:00
James Rodewig 8686200a32 [DOCS] EQL: Document `concat` function (#56239)
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-05-05 16:45:29 -04:00
Dan Hermann 6674f14fb3
[7.x] Get index includes parent data stream for backing indices (#56238) 2020-05-05 15:43:42 -05:00
James Rodewig dac4ed282e [DOCS] EQL: Add collapsible sections to EQL tutorial docs (#56235)
Adds collapsible sections to the snippet examples of the EQL tutorial
docs.

Also adds a leading slash to EQL API snippet examples.
2020-05-05 16:29:51 -04:00
James Rodewig e7df8b388e [DOCS] EQL: Add collapsible sections to EQL search API response (#56232)
Add collapsible sections to the response parameter docs
of the EQL search API.

Also clarifies some language regarding documents and
events.
2020-05-05 16:01:55 -04:00
Benjamin Trent e1c5ca421e
[7.x] [ML] lay ground work for handling >1 result indices (#55892) (#56192)
* [ML] lay ground work for handling >1 result indices (#55892)

This commit removes all but one reference to `getInitialResultsIndexName`. 
This is to support more than one result index for a single job.
2020-05-05 15:54:08 -04:00
Julie Tibshirani 793f265451 Mute SearchableSnapshotDirectoryTests.testIndexSearcher. 2020-05-05 12:29:05 -07:00
Ross Wolf 389082033e
EQL: Add concat function (#55193)
* EQL: Add concat function
* EQL: for loop spacing for concat
* EQL: return unresolved arguments to concat early
* EQL: Add concat integration tests
* EQL: Fix concat query fail test
* EQL: Add class for concat function testing
* EQL: Add concat integration tests
* EQL: Update concat() null behavior
2020-05-05 12:53:34 -06:00
Andrei Dan a7968a1a5e
[7.x] HLRC: document index template v2 and component template APIs (#56136) (#56225)
This documents the index template v2 and component template APIs in the
high level rest client.

(cherry picked from commit 9bcf89b1e27613ab8887ce611ec2b0d1356cba8b)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-05-05 19:51:54 +01:00
Mark Tozzi 33da086d7b
[7.x] Wire up DiversifiedAggregation (#56145) (#56222) 2020-05-05 13:11:49 -04:00
Bogdan Pintea 23c35e32f2
SQL: introduce a query builder for the Rest tests (#55094) (#56221)
* Introduce a query builder for the rest tests

The new BaseRestSqlTestCase.RequestObjectBuilder class is a helper class
to build REST request objects for the tests. Consequently, "manual" string
concatenation to form JSON is done away with.

The class mimics SqlQueryRequestBuilder API.

(cherry picked from commit c8363f04c029542c233a758e9286d33c51d9c0c4)
2020-05-05 18:55:41 +02:00
Tal Levy e4f2c3105d
Add geo_shape support for geotile_grid and geohash_grid (#55966) (#56228)
this commit adds aggregation support for the geo_shape field
type on geo*_grid aggregations.

it introduces a Tiler for both tiles and hashes that enables a new type of
ValuesSource to replace the GeoPoint's CellIdSource. This makes it possible
for the existing Aggregator to be re-used, so no new implementations of
the grid aggregators are added.
2020-05-05 09:54:14 -07:00
Tim Brooks 6a51017cb2
Upgrade netty to 4.1.49.Final (#56059) 2020-05-05 10:40:23 -06:00
Thiago Souza 1feb0a95b5 [DOCS] Correct get enrich policy API request example (#56207) 2020-05-05 12:37:54 -04:00
Benjamin Trent 641f598364
[Transform] fixes http status code when bad scripts are provided (#56117) (#56219)
Transforms should propagate up the search execution exception if one is returned when it does the test query. 

this allows transforms to return a `4xx` when the aggs are malformed but parseable. 

closes https://github.com/elastic/elasticsearch/issues/55994
2020-05-05 12:36:22 -04:00
Lee Hinman b77c0bbe26
[7.x] Validate V2 templates more strictly (#56170) (#56226)
Backports the following commits to 7.x:
 - Validate V2 templates more strictly (#56170)
2020-05-05 10:34:56 -06:00
Bogdan Pintea 0e5632dc3a
SQL: relax version lock between server and clients (#56148) (#56223)
* Relax version lock between ES/SQL and clients

Allow older-than-server clients to connect, if these are past or on a
certain min release.

(cherry picked from commit 108f907297542ce649aa7304060aaf0a504eb699)
2020-05-05 18:27:06 +02:00
James Rodewig cd3663e5fa
[DOCS] EQL: Document `match` function (#56134) 2020-05-05 12:03:02 -04:00
James Rodewig 44414acd3b
[DOCS] EQL: Document nested field support (#56138)
Notes that you cannot use EQL in ES to search the values of `nested`
fields or their sub-fields. However, indices containing `nested` field
mappings are otherwise supported.
2020-05-05 11:46:06 -04:00
James Rodewig a528319827
[DOCS] Remove invalid search API query parameters (#55884) (#56212)
This is a backport of #55884 with redirects removed.

Changes:

* Adds an abbreviated title for the search API page.
* Removes the following invalid query parameters:
  * `analyzer`
  * `analyze_wildcard`
  * `default_operator`
  * `df`
  * `lenient`
  * `suggest_mode`
  * `suggest_size`
* Replaces the URI search page's query parameter docs with a xref
* Updates the headings of several examples
2020-05-05 11:10:34 -04:00
William Brafford 3499fa917c
Deprecated xpack "enable" settings should be no-ops (#55416) (#56167)
The following settings are now no-ops:

* xpack.flattened.enabled
* xpack.logstash.enabled
* xpack.rollup.enabled
* xpack.slm.enabled
* xpack.sql.enabled
* xpack.transform.enabled
* xpack.vectors.enabled

Since these settings no longer need to be checked, we can remove settings
parameters from a number of constructors and methods, and do so in this
commit.

We also update documentation to remove references to these settings.
2020-05-05 10:40:49 -04:00
Igor Motov 94b349cd18
[7.x] Simplify the ValuesSourceRegistry structure (#56154) (#56197)
Follow up to #55747.
2020-05-05 10:37:02 -04:00
Rory Hunter fd91617b19 Fix typo in docs (#56190)
Some dashes were swapped for question marks in bf4c364. Swap them back.
2020-05-05 15:28:32 +01:00
James Rodewig cc1af50214
[DOCS] Add tip for upgrade assistant (#55027)
Our upgrade docs don't mention the upgrade assistant, which can be
helpful when migrating across major versions. The docs also don't
mention deprecation logs, which can highlight other functionality that
may change.

This adds a related tip admonition to the upgrade docs.
2020-05-05 10:09:41 -04:00
Tanguy Leroux b9636713b1
Searchable Snapshots should respect max_restore_bytes_per_sec (#55952) (#56199)
This commit changes searchable snapshots so that it now respects the 
repository's max_restore_bytes_per_sec setting when it downloads blobs.

Backport of #55952 for 7.x
2020-05-05 15:43:06 +02:00
David Roberts 7aa0daaabd
[7.x][ML] More advanced model snapshot retention options (#56194)
This PR implements the following changes to make ML model snapshot
retention more flexible in advance of adding a UI for the feature in
an upcoming release.

- The default for `model_snapshot_retention_days` for new jobs is now
  10 instead of 1
- There is a new job setting, `daily_model_snapshot_retention_after_days`,
  that defaults to 1 for new jobs and `model_snapshot_retention_days`
  for pre-7.8 jobs
- For days that are older than `model_snapshot_retention_days`, all
  model snapshots are deleted as before
- For days that are in between `daily_model_snapshot_retention_after_days`
  and `model_snapshot_retention_days` all but the first model snapshot
  for that day are deleted
- The `retain` setting of model snapshots is still respected to allow
  selected model snapshots to be retained indefinitely

Backport of #56125
2020-05-05 14:31:58 +01:00
James Rodewig 4dfdd46dc3 [DOCS] EQL: Remove case sensitivity from function docs (#55063)
Per #54411, we plan to handle case sensitivity via a parameter for the
EQL search API (with the possible exception of the `between` function).

This removes references and examples related to case sensitivity from
the EQL functions docs.
2020-05-05 09:26:49 -04:00
James Rodewig d04ac8b496 [DOCS] Correct `refresh` parm def for GET API (#56140) 2020-05-05 09:13:49 -04:00
James Rodewig bd17d098c4
[DOCS] Remove duplicate `index alias` link in glossary (#56143) 2020-05-05 09:07:19 -04:00
Hendrik Muhs faadb388da
mute mixed continuous transforms upgrade test (#56198)
mute transform upgrade test, see #56196
2020-05-05 14:40:50 +02:00
David Turner 40ea0eabd9 Forbid snapshot access on applier thread (#56044)
This commit strengthens the assertion about which threads may access a blob
store to exclude the cluster applier thread, since we no longer need to do so.

Relates #50999
2020-05-05 13:27:55 +01:00
Dimitris Athanasiou 2d7899c83c
[7.x][ML] Adjust DF Analytics process phases (#56107) (#56177)
As of elastic/ml-cpp#1179, the analytics process reports phases
depending on the analysis type. This commit adjusts the phases
of current analyses from `analyzing` to the following:

 - outlier_detection: [`computing_outlier`]
 - regression/classification: [`feature_selection`, `coarse_parameter_search`, `fine_tuning_parameters`, `final_training`]

Backport of #56107
2020-05-05 15:00:07 +03:00
Dimitris Athanasiou 75dadb7a6d
[7.x][ML] Add loss_function to regression (#56118) (#56187)
Adds parameters `loss_function` and `loss_function_parameter`
to regression.

Backport of #56118
2020-05-05 14:59:51 +03:00
Jason Tedor c38388c506
Fix compiling in TransportValidateQueryActionTests
This arose after a backport where we do not have the nicities of the
Java 11 diamond operator. This commit fixes it by adding the proper type
parameter.
2020-05-05 07:36:40 -04:00
Jason Tedor 410eb29937
Fix validate query listener invocation bug (#56157)
When the index we are validating a query does not exist, we try to send
back a response letting the client know that the index does not
exist. Yet, we accidentally fallthrough into the case that the
validation failed for some other reason. This means that we end up
notifying the channel twice. Sometimes the notification occurs after the
failure has been written out and the channel closed (so the second
invocation leads to a silent failed to write to a closed channel issue),
and sometimes the response does end up in the channel, creating garbled
responses to the client. This commit fixes that issue by avoiding the
fallthrough.
2020-05-05 07:26:02 -04:00
Hendrik Muhs e177a38504
[7.x][Transform] add throttling (#56007) (#56184)
add throttling to transform, throttling will slow down search requests by
delaying the execution based on a documents per second metric.

fixes #54862
2020-05-05 13:09:02 +02:00
Andrei Dan f569405fde
Enable simulate API tests in 7.8 (#55946)
As #55686 was backported the simulate index template api is no available
in 7.8.
2020-05-05 11:28:00 +01:00