Commit Graph

54845 Commits

Author SHA1 Message Date
Nik Everett 56605e4d9a Fixup reduceRandom tests (#65263)
In aa1ea96b8698aa12bed1c4e8d704882a2a639791 I made all
`testReduceRandom` tests for aggs mimick production more precisely.
More precisely, they pick the correct "lead" result when performing
partial reduction. This is great, but, sadly, some tests assumed that we
always reduced against the "first" aggregator. This fixes those tests.

Closes #65163
2020-11-20 13:10:34 -05:00
Nik Everett 956ae93fdb Better mimick real reduction in test
This changes has `InternalAggregationTestCase` performs partial
reductions to better mimick `InternalAggregations#reduce`. In
particular, this fixes a bug when we'd attempt a partial reduction of
aggregations that were entirely unmapped but use a mapped aggregation as
the "lead" of the reduction. This was possible in the test but isn't
possible in production. The reduction "lead" is unmapped if all agg
results being reduced are unmapped.

Closes #65118
2020-11-20 13:10:34 -05:00
James Rodewig feca22729c
[DOCS] Remove duplicated word in replica shard allocator comment (#65295) (#65317)
Co-authored-by: Howard <danielhuang@tencent.com>
2020-11-20 12:25:52 -05:00
István Zoltán Szabó e5629f5580
[DOCS] Makes the screenshot larger on the custom URLs page. (#65269) (#65298) 2020-11-20 15:24:32 +01:00
James Rodewig 99f3156646
[DOCS] Fix formatting issue in search explain docs (#65303) (#65313)
Co-authored-by: bellengao <gbl_long@163.com>
2020-11-20 09:21:29 -05:00
Sylvain Wallez f7e840b012
[client] Add simple support for gzip compression
Adds a `RestClient.setCompressionEnabled()` setting that will gzip-
compress request bodies and add a `Accept-Encoding: gzip` header so
that the ES server can send compressed responses.
2020-11-20 07:38:21 +01:00
James Rodewig d4e308a135
[DOCS] Document `xpack.http.proxy.scheme` setting (#65264) (#65287) 2020-11-19 15:44:12 -05:00
Jay Modi 893e1a5282
Fix date math hidden index resolution (#65278)
This commit updates the IndexAbstractionResolver so that hidden indices
are properly resolved when date math is in use and when we are checking
if the index is visible.

Closes #65157
Backport of #65236
2020-11-19 12:40:14 -07:00
James Rodewig b60d0711b7
[DOCS] ILM: Add alloc ex with multiple node attributes (#65266) (#65271) 2020-11-19 12:38:24 -05:00
István Zoltán Szabó a6a9950868
[7.10][DOCS] Adds UI related limitation to configuring aggs docs (#65190) 2020-11-19 14:57:33 +01:00
Nhat Nguyen 3989243a52 Stop renew retention leases when follow task fails (#65168)
If a shard follow-task hits a non-retryable error and stops, then we 
should also stop the retention-leases renewal process associated with
that follow-task.
2020-11-18 15:53:55 -05:00
DeDe Morton 2e079b1b9b
Fix links to Fleet overview (#65174) (#65233) 2020-11-18 11:04:07 -08:00
James Rodewig 46c55c562a
[DOCS] Remote reindex is not fwd compatible (#65207) (#65226) 2020-11-18 13:03:12 -05:00
Armin Braun 6953726a2b
Fix Allocation `include` Filter Docs (#65202) (#65223)
Fix documentation to match actual behavior of `include` type filters.

Closes #65113
2020-11-18 18:25:09 +01:00
James Rodewig b1dc869b0f
[DOCS] Clarify logger-package relationship (#65169) (#65212)
Updates the logging level docs to better clarify the relationship
between loggers and their Java packages.
2020-11-18 10:45:00 -05:00
James Rodewig c8dac8cff8
[DOCS] Format the ilm-migrate doc (#65182) (#65204)
Co-authored-by: bellengao <gbl_long@163.com>
2020-11-18 09:14:00 -05:00
Jim Ferenczi 9f3e3e2162 Fix "resource not found" exception on existing EQL async search (#65167)
This change fixes the initialization of the async results service
for the EQL get async action. The boolean that differentiates EQL
from normal _async_search request is set incorrectly, which results
in errors (404) when extending the keep alive of a running EQL search.

Fixes #65108
2020-11-18 09:10:31 +01:00
Costin Leau f089547b20 EQL: Fix aggressive/incorrect until policy in sequences (#65156)
The current until implementation in sequences is too optimistic, leading
to an aggressive match that discards correct data leading to invalid
results.
This commit addresses this issue and also unifies the until usage inside
TumblingWindow.
Further more it packs together the UntilGroup with SequenceGroup to
minimize memory usage and improve clean-up.

(cherry picked from commit de2724e92c732c66436939dbbedef93c9981b435)
(cherry picked from commit a60757756aae5f5abb31176fee972a7cdeac3649)
2020-11-18 09:34:33 +02:00
Julie Tibshirani 5495032b00 Remove unused method Analysis#isNoStopwords. 2020-11-17 16:34:33 -08:00
James Rodewig a8d3053128
[DOCS] Clarify ingest attachment example (#65143) (#65160) 2020-11-17 15:15:25 -05:00
James Rodewig 9471b6345e
[DOCS] Add note about NFS repositories (#65149) (#65162)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Rafi Estrada <62263912+TheRiffRafi@users.noreply.github.com>
2020-11-17 14:48:47 -05:00
Johnny Lim d67907d244
[DOCS] Fix typos (#65124)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-11-17 12:35:41 -05:00
Sylvain Wallez b2475f9ccf
Fix parsing RareTerms aggregation response in RestHighLevelClient (#65144)
Backport of #64454

- Add LongRareTerms and StringRareTerms to the DefaultNamedXContents,
  ensure that the response of RareTerms aggregation can be parsed
  correctly.
- Add testSearchWithRareTermsAgg method to test the response of
  RareTerms aggregation can be parsed correctly.
- Add some test code to ensure the AggregationsTests can execute
  successfully.

Co-authored-by: bellengao <gbl_long@163.com>
2020-11-17 17:43:51 +01:00
James Rodewig 196c2a52c1
[DOCS] EQL: Update docs for null tiebreakers (#65078) (#65138) 2020-11-17 10:42:10 -05:00
James Rodewig b54133399e
[DOCS] EQL: Document result_position param (#65075) (#65135) 2020-11-17 10:41:06 -05:00
Dimitris Athanasiou 197de8fe66
[7.10][ML] Increase timeout waiting for DFA jobs to finish in integ tests (#65126) (#65131)
It appears that occasionally 30 seconds are not enough for CI workers
to complete DFA jobs. In order to eliminate such failures we increase
the time we wait for DFA jobs to complete in integration tests to
60 seconds.

Fixes #64926

Backport of #65126
2020-11-17 16:46:17 +02:00
István Zoltán Szabó 93505cc05f
[DOCS] Adds recommendation about when to use chunking_config in manual mode. (#65060) (#65083) 2020-11-17 10:45:28 +01:00
Julie Tibshirani 3974c3b066 Move the shared fetch cache to highlighting. (#65105)
The cache is only used by highlighters, so it can be scoped to only the
highlighting context.
2020-11-16 18:54:32 -08:00
James Rodewig 23a1c64347 [DOCS] Fix typo 2020-11-16 21:35:15 -05:00
Jack Conradson 0beffcd405 Revert null-safe behavior to error at runtime instead of compiletime (#65099)
This reverts a change where null-safe was enhanced to cause a compile-time error instead of a run-
time error when the target value was a primitive type. The reason for the reversion is consistency 
across def/non-def types and versions. I've added a follow up issue to fix this behavior in general 
(#65098).
2020-11-16 10:45:38 -08:00
Mark Vieira afd12fddaf Remove reference to 7.9.4 relase which won't happen 2020-11-16 10:31:36 -08:00
Adam Locke 5a6239e916
[DOCS] Adding authorization_realms setting for OIDC (#64877) (#65094)
* Adding authorization_realms setting for OIDC.

* Fix typo

* Splitting note into separate notes.
2020-11-16 13:07:20 -05:00
Costin Leau 74fde15833 EQL: Allow null tiebreakers inside ordinals/sequences (#65033)
Align Ordinal comparator to consider nulls last (higher) in tiebreakers.
Add unit tests to Ordinal comparisons and criterion extraction.

Fix #64706

(cherry picked from commit 93dc883abd6b8855ff1618a574412b7f773b8ff5)
(cherry picked from commit 936e5f1a2cc29c1d5662cb8aa90c629af563a987)
2020-11-16 16:52:55 +02:00
James Rodewig 20bda8d23a
[DOCS] Document reloadable Watcher settings (#64998) (#65073) 2020-11-16 09:18:48 -05:00
James Rodewig e4054e4e92
[DOCS] Clarify diff between shards per node settings (#64875) (#65071)
Clarifies differences between the
`cluster.routing.allocation.total_shards_per_node` and
`cluster.max_shards_per_node` cluster settings.

Closes #51839

Co-authored-by: Gordon Brown <arcsech@gmail.com>
2020-11-16 09:18:11 -05:00
Przemysław Witek de668ab84b
[7.10] [ML] Extract dependent variable's mapping correctly in case of a multi-field (#63813) (#64287) 2020-11-16 10:34:58 +01:00
Costin Leau 9551cb3420 EQL: small improvements to the testing base class
Extract request settings into dedicated methods for easier adjustments

(cherry picked from commit 4f93591cc561c7f8ff7c2f070dd1180f209810b7)
(cherry picked from commit ff7e8427345c304f5a37612c870b48555484b692)
2020-11-14 16:40:48 +02:00
Costin Leau f7cc570c4f EQL: Re-enable correctness tests (#65041)
Enable previously disabled tests - only two type of queries remain
disabled: one that does pattern matching and another one for
case-insensitivity.

Fix #63742

(cherry picked from commit 20210cc43b34438c40b8b5aebf0aa2b8161c4104)
(cherry picked from commit 95d08f2c8d0aac52cc1ed470fa489c239ee25159)
2020-11-14 16:09:11 +02:00
Costin Leau 76e73fec79
EQL: Add option for returning results from the tail of the stream (#64869) (#65040)
Introduce option for specifying whether the results are returned from
the tail (end) of the stream or the head (beginning).
Improve sequencing algorithm by significantly eliminating the number
of in-flight sequences for spare datasets.
Refactor the sequence class by eliminating some of the redundant code.
Change matching behavior for tail sequences.
Return results based on their first entry ordinal instead of
insertion order (which was ordered on the last match ordinal).
Randomize results position inside test suite.

Close #58646

(cherry picked from commit e85d9d1bbee13ad408e789fd62efb30bc8d223f2)
(cherry picked from commit 452c674a10cdc16dced3cde7babf5d5a9d64a6d9)
2020-11-14 13:44:17 +02:00
Adam Locke 38b5b59862
Explicitly defining types for sources parameter (#65006) (#65022) 2020-11-12 17:08:42 -05:00
Alan Woodward 0e2a9b4ac7 Fix sparse vector test 2020-11-12 20:10:07 +00:00
Alan Woodward caf143f4a5
Unused boost parameter should not throw mapping exception (#64999) (#65014)
We were correctly dealing with boosts that had an effect, but mappers
that had a silently accepted but ignored boost parameter were throwing
an error instead of continuing to ignore the boost but emitting a
warning.

Fixes #64982
2020-11-12 19:28:32 +00:00
István Zoltán Szabó d9970fa764
[DOCS] Adds scroll_size maximum value to datafeeds API docs (#64986) (#65003) 2020-11-12 18:10:24 +01:00
Benjamin Trent b888f36388
[ML] fix custom feature processor extraction bugs around boolean fields and custom one_hot feature output order (#64937) (#65009)
This commit fixes two problems:

- When extracting a doc value, we allow boolean scalars to be used as input
- The output order of processed feature names is deterministic. Previous custom one hot fields used to be non-deterministic and thus could cause weird bugs.
2020-11-12 11:15:57 -05:00
Tanguy Leroux e40d7e02ea
Makes testCcrRepositoryFetchesSnapshotShardSizeFromIndexShardStoreStats more robust (#64976) (#64989)
Today this test fails because the sizes of the snapshot 
shards are only kept in a very short period of time in 
the InternalSnapshotsInfoService and are not 
guaranteed to exist once the shards are correctly 
assigned.

closes #64167
2020-11-12 15:38:38 +01:00
James Rodewig 5c5fd50f8d
[DOCS] Fix indentation (#64941) (#64964)
Co-authored-by: kemalizing <36174398+kemalizing@users.noreply.github.com>
2020-11-12 08:47:46 -05:00
Dimitris Athanasiou b5efaf6e3b
[7.10][ML] Protect against stack overflow while loading DFA data (#64947) (#64956)
If we encounter an exception during extracting data in a data
frame analytics job, we retry once. However, we were not catching
exceptions thrown from processing the search response. This may
result in an infinite loop that causes a stack overflow.

This commit fixes this problem.

Backport of #64947
2020-11-12 11:08:40 +02:00
debadair e4b77bcd38
[DOCS] Make system & hidden index info more prominent. (#64839) (#64960)
* [DOCS] Make system & hidden index info more prominent.

* Update docs/reference/api-conventions.asciidoc
2020-11-11 15:16:45 -08:00
Andras Palinkas a7301065d7
SQL: Fix the return type in the sign function (#64845) (#64968)
Fixes the inconsistency between the type of the object returned by the
`SIGN()/SIGNUM()` SQL functions and the specified `DataType`.

In the Class Sign, DataType is DataTypes.INTEGER. The source code is as
follows:

```
    public DataType dataType() {
        return DataTypes.INTEGER;
    }
```

But In the Class MathProcessor, the source code of SIGN((Object l),
Parameter and return value types are the same. Therefore, when using
double or float parameters to test, there is a little problem, the test
method is like the following curl :

```
curl -XPOST 127.0.0.1:9200/_sql -d "{\"query\":\"select  SIGN(1.0) \"}" \
  -H 'Content-Type: application/json'
```

The result is:

```
{"columns":[{"name":"SIGN(1.0)","type":"integer"}],"rows":[[1.0]]}
```

The result value is `1.0`, but the type is `integer`.

Signed-off-by: mantuliu <240951888@qq.com>
Co-authored-by: Marios Trivyzas <matriv@gmail.com>
(cherry picked from commits aa78301e71f, ced3c1281c7, 40e5b9b)
2020-11-11 17:07:17 -05:00
James Rodewig 4b1d66fc29
[DOCS] Add checksum links for plugin downloads (#64949) 2020-11-11 13:12:35 -05:00