Commit Graph

54124 Commits

Author SHA1 Message Date
Nhat Nguyen 9015b50e1b
Check docs limit before indexing on primary (#63273)
Today indexing to a shard with 2147483519 documents will fail that
shard. We should check the number of documents and reject the write
requests instead.

Closes #51136
2020-10-13 17:39:08 -04:00
James Rodewig 8455cf8ff0 [DOCS] Remove unneeded word in EQL docs 2020-10-13 13:57:41 -04:00
James Rodewig 086df0636e
[DOCS] EQL: Document multi-value field support (#63622) (#63633) 2020-10-13 12:44:48 -04:00
Lee Hinman 7371e51583
[7.10] Add DiscoveryNodeRole compatibility role for bwc tier serialization (#63581) (#63613)
Backports the following commits to 7.10:

    Add DiscoveryNodeRole compatibility role for bwc tier serialization (#63581)
2020-10-13 09:17:15 -06:00
Andras Palinkas f9c8ae5f77 SQL: Improve parser error message for `ESCAPE` (#63616)
Mentions the list of wildchars in case a wildchar is used as an
`ESCAPE` character.

Relates #63428

(cherry picked from commit 74cbcf871e9593b3640e382ae6845168fd14966b)
2020-10-13 10:47:06 -04:00
Andras Palinkas 4aabc050a0 SQL: Escaped wildcard (*) not accepted in LIKE (#63428)
For a query like `SELECT name FROM test WHERE name LIKE ''%c*'` ES SQL
generates an error. `*` is not a special character in a `LIKE` construct
and it's expected to not needing to be escaped, so the previous query
should work as is.
In the LIKE pattern any `*` character was treated as invalid character
and the usage of `%` or `_` was suggested instead. But `*` is a valid,
acceptable non-wildcard on the right side of the `LIKE` operator.

Fix: #55108
(cherry picked from commit 190d9fe3deb31aed0d8f312007360625d4fff217)
2020-10-13 10:47:03 -04:00
Armin Braun f70391c6cc
Fix Broken Snapshot State Machine in Corner Case (#63534) (#63608)
This fixes a gap in testing and a bug that can occur in various forms:
When we would start a snapshot or clone related to a shard that was done
snapshotting/cloning but its overall operation was not yet finalized
at the time of starting the operation, we would base the operation off of
the wrong generation. This would not cause a corrupted repo, but would
cause the operation to be `PARTIAL`.
This commit fixes the state machine to take into account the correct generation
in this case.

Closes #63498
2020-10-13 16:05:34 +02:00
James Rodewig 845ccc2264
[DOCS] Fix dup word in ShardRouting hashcode method. (#63452) (#63583)
Co-authored-by: Howard <danielhuang@tencent.com>
2020-10-13 09:05:19 -04:00
Andrei Dan b3aa575f0d
Add release 7.10 highlights (#63468)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Jay Modi <jaymode@users.noreply.github.com>
Co-authored-by: Adrien Grand <jpountz@gmail.com>
2020-10-13 13:25:08 +01:00
Rene Groeschke 26f58c8bdb
Fix build tools git tests on windows 7x (#63456)
* Add error logging for git commands in build-tools integTests
* Explicitly set user and email in git related func tests
2020-10-13 14:12:52 +02:00
Przemysław Witek acbd48f834
[ML] Allow setting num_top_classes to a special value -1 (#63587) (#63602) 2020-10-13 13:57:50 +02:00
Adrien Grand 652af26369
Add space efficiency release highlight to 7.10. (#63475)
This release highlight is brought in by Lucene 8.7.

Co-authored-by: Andrei Dan <andrei.dan@elastic.co>
2020-10-13 13:42:21 +02:00
István Zoltán Szabó 7139c1d7db
[DOCS] Adds AUC ROC classification metric to the API examples (#63563) (#63596) 2020-10-13 12:18:29 +02:00
Tanguy Leroux 8499924e51
InternalSnapshotsInfoService should also removed failed snapshot shard size infos (#63492) (#63592)
Relates #61906
2020-10-13 10:42:38 +02:00
Julie Tibshirani 9e52513c7b
Add support for missing value fetchers. (#63585)
This PR implements value fetching for the following field types:
* `text` phrase and prefix subfields
* `search_as_you_type`, plus its subfields
* `token_count`, which is implemented by fetching doc values

Supporting these types helps ensure that retrieving all fields through
`"fields": ["*"]` doesn't fail because of unsupported value fetchers.
2020-10-12 17:34:21 -07:00
David Roberts 3f210e2620 [ML] Load data streams plugin for ML internal cluster tests (#63560)
Now that deprecation logs get indexed to a data stream, if we
do not load the data stream plugin in our tests and any test
generates a deprecation log message then millions of exceptions
get logged, slowing down the tests to the extent that they can
fail.

This change loads the data streams plugin during the ML internal
cluster tests.  (It should already be present in external cluster
tests.)

Fixes #63548
2020-10-12 17:46:50 +01:00
Gordon Brown 5fe51b1bf4
Fix type name in testSystemIndexMetadataIsUpgraded (#63506)
This test used _doc as the mapping type name, which needs to be set
to doc for versions prior to 6.7.0. This commit fixes the test to use
the proper type name for the current BWC version.
2020-10-12 10:44:38 -06:00
James Rodewig 25ead6abb7
[DOCS] Fix typo in search profile docs (#63522) (#63559)
Co-authored-by: ondrejbardon <70531797+ondrejbardon@users.noreply.github.com>
2020-10-12 10:13:13 -04:00
James Rodewig 9c170706dd
[DOCS] EQL: Move to beta (#63284) (#63552) 2020-10-12 09:12:26 -04:00
Dimitris Athanasiou e1c418aac7
[7.10][ML] Validate dest pipeline exists on transform update (#63494) (#63549)
Adds validation that the dest pipeline exists when a transform
is updated. Refactors the pipeline check into the `SourceDestValidator`.

Fixes #59587

Backport of #63494
2020-10-12 15:41:35 +03:00
Tomas Della Vedova 8b07750a8b
Updated version string in yaml test (#63410) (#63533) 2020-10-12 11:19:36 +02:00
David Turner 6ddeaa4791 Third-party S3-compatible repos aren't unsupported (#63238)
Today in the `repository-s3` docs we say

> Other S3-compatible storage systems may also work with Elasticsearch,
> but these are not tested or supported.

Saying that they are explicitly not supported is a very strong
statement, implying that it is positively irresponsible to use anything
except S3 or Minio, even after extensive testing. S3-compatibility in
third-party systems has matured in recent years and users today report
success with a good number of them. In contrast, we effectively claim
support for any old NFS implementation when used with the
shared-filesystem repository.

This commit weakens this statement, removing the absolute claim of
unsupportedness and instead spelling out that the user is responsible
for ironing out any incompatibilities with the storage supplier.
2020-10-12 09:49:30 +01:00
Tim Brooks 56092b1a9f
Flush translog writer before adding new operation (#63505)
Currently we flush the Translog buffer when a new operation causes the
buffer to breach 1MB. This introduces a scenario where an exception is
thrown AFTER the writer has accepted the operation. To avoid this, this
commit flushes the Translog in an #add call before adding a new
operation.

This fixes #63299.
2020-10-09 10:02:55 -06:00
Ryan Ernst 01dde647a6
Remove glibc warning (#63447)
Now that the bundled jdk is switched to the Oracle JDK, the glibc
warning is no longer relevant. This commit removes the warning.
2020-10-08 14:45:43 -07:00
Lee Hinman 27f13267f0
Add release note highlights for data tiers (#63427)
This adds release note highlights for the data tiers formalization feature.

Relates to #60848
2020-10-08 14:39:09 -06:00
Julie Tibshirani ae2fc4118d Add factory methods for common value fetchers. (#63438)
This PR adds factory methods for the most common implementations:
* `SourceValueFetcher.identity` to pass through the source value untouched.
* `SourceValueFetcher.toString` to simply convert the source value to a string.
2020-10-08 12:14:53 -07:00
Julie Tibshirani a506705569 Small fixes to flattened field value fetching. (#63443)
* Remove FlatObjectFieldTypeTests, as it's redundant.
* Do not apply null_value when fetching root-level values.
* Remove a TODO in favor of opening an issue.
2020-10-08 11:52:54 -07:00
Julie Tibshirani c6b915c8e6 Make TextFieldMapper.FAST_PHRASE_SUFFIX private. 2020-10-08 11:45:53 -07:00
Dan Hermann f822bb732c
Fix failure in AppendProcessorTests.testAppendingToListWithDuplicatesDisallowed (#62842) (#63499) 2020-10-08 13:15:29 -05:00
Tanguy Leroux 943fcaf970
Simplify reroute counting in InternalSnapshotsInfoServiceTests (#63416) (#63491)
Closes #63352
2020-10-08 18:20:07 +02:00
Dan Hermann 0024cf33e3
Fix failure in AppendProcessorTests.testAppendingUniqueValueToScalar (#62453) (#63493) 2020-10-08 10:39:57 -05:00
István Zoltán Szabó fe67d055f5 [DOCS] Adds huber and msle metrics to Evaluate API example calls (#63414) 2020-10-08 17:09:39 +02:00
Dan Hermann 85cd8e4868
Fix failure in AppendProcessorTests.testAppendingToListWithDuplicatesDisallowed (#62385) (#63488) 2020-10-08 09:32:55 -05:00
Dan Hermann 85886e71c2
Handle error conditions when simulating ingest pipelines with verbosity enabled (#63327) (#63484) 2020-10-08 09:21:05 -05:00
Dan Hermann 25fc6746cc
[DOCS] allow_duplicates option for append processor (#62336) (#63486) 2020-10-08 09:01:43 -05:00
Przemyslaw Gomulka d7391bc040
[7.10] Fix incorrect use of Format.equals instead of matches backport#63462 #63463
closes #63459
backports #63462
2020-10-08 15:35:13 +02:00
James Rodewig 5e623522d2
[DOCS] Move searchable snapshots to beta (#63436) (#63478) 2020-10-08 09:27:52 -04:00
Christoph Büscher 517d3e4336 Mute DiskThresholdDeciderIT.testHighWatermarkNotExceeded 2020-10-08 15:14:50 +02:00
Benjamin Trent a9be4181c6
[ML] fix grabbing the doc value limit setting in _explain (#63402) (#63471)
Getting the doc value settings shouldn't use the API callers headers. We only use this value internally.
2020-10-08 08:53:26 -04:00
Mayya Sharipova e022b78198
Upgrade to lucene-8.7.0-snapshot-5c4168d (#63466)
This disables sort optim on _doc, which may still be unstable.
Backport for #63444
2020-10-08 08:20:43 -04:00
Costin Leau 2ab5f226c4 EQL: Avoid filtering on tiebreakers (#63415)
Do not filter by tiebreaker while searching sequence matches as
it's not monotonic and thus can filter out valid data.
Add handling for data 'near' the boundary that has the same timestamp
but different tie-breaker and thus can be just outside the window.

Fix #62781
Relates #63215

(cherry picked from commit 36f834600d4d9ded0fb7b1440274b2e597733770)
(cherry picked from commit 72a2ce825f3bfd13f87423ba7f3c739ea64c57f6)
2020-10-08 13:50:41 +03:00
Christoph Büscher 1f7b107ed1 Muting FullClusterRestartIT.testSystemIndexMetadataIsUpgraded 2020-10-08 12:02:23 +02:00
Christoph Büscher 564823b00f Muting parts of JavaJodaTimeDuellingTests 2020-10-08 11:50:47 +02:00
David Roberts a9d541561f [ML] Unmute DeleteExpiredDataIT.testDeleteExpiredDataNoThrottle (#63408)
This test appears to work again following the Lucene bug fix
that was integrated in #63395
2020-10-08 09:11:29 +01:00
Przemysław Witek bd761cce1d
[ML] Validate that AucRoc has the data necessary to be calculated (#63302) (#63454) 2020-10-08 09:52:15 +02:00
Adam Locke f4530580e7
[DOCS] Updating password usage for create user API (#63292) (#63440)
* Updating password usage.
* Add asterisk
* Add explanation of asterisk for settings that are only sometimes required.
* Clarify change password guidance
2020-10-07 16:21:22 -04:00
Luca Cavanna 659988a77f
Remove runtime fields (#63418)
We are not going to release runtime fields with 7.10, hence we are removing them from the 7.10 branch.
2020-10-07 20:39:41 +02:00
James Rodewig 58eaba9950
[DOCS] Add xref to data tiers content (#63426) (#63434) 2020-10-07 13:24:52 -04:00
Lisa Cawley 51e6274f1e
[DOCS] Add props for ILM searchable snapshot links (#63431) 2020-10-07 10:23:50 -07:00
Lisa Cawley edb6a5b674
[DOCS] Add props for searchable snapshots API links (#63420) (#63424) 2020-10-07 09:34:58 -07:00