Commit Graph

54948 Commits

Author SHA1 Message Date
Jay Modi 01d54d222b
Fix cat tasks api params in spec and handler (#66294)
This commit fixes the cat tasks api parameter specification and the
handler so that the parameters are consumed during request preparation.

Closes #59493
Backport of #66272
2020-12-14 13:44:09 -07:00
James Rodewig 170c976acc
[DOCS] Add `require_alias` to Bulk API (#66259) (#66266) 2020-12-14 11:21:33 -05:00
Jim Ferenczi 330de82d59 Fix composite aggregation on unsigned long (#65715)
This commit ensures that the after key is parsed with the doc value formatter.
This is needed for unsigned longs that uses shifted longs internally.

Closes #65685
2020-12-14 16:59:07 +01:00
James Rodewig 25f51f82ab
[DOCS] Add adaptive replica selection reference (#66232) (#66262)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: calm4wei <alferwei98@163.com>
2020-12-14 10:47:16 -05:00
Armin Braun 7e1fc6dc67
Adjust Cleanup Order of Internal State in SnapshotsService (#66225) (#66244)
In the assertion mentioned in the new comment we first get the `endingSnapshots`
and then check that we don't have a listener that isn't referred to by it so we need
to remove from the listers map before removing from `endingSnapshots` to avoid rare, random
assertion tripping here with concurrent repository operations.
2020-12-14 12:24:39 +01:00
Marios Trivyzas 416ea4fcdc
EQL: [Tests] New eql correctness data snapshot (#66238)
With the upcoming validation for type compatibility of the sequence
keys, several tests are failing because some fields that contain IP
data were previously mapped as keyword. Fixed the mapping and created a
new snaphost of the correctness data in the gcs bucket.

Relates to: #66183

(cherry picked from commit 7f638f661c5a5c57a4ea7d3d3e2ccf5c81ae92d1)
2020-12-14 10:34:14 +01:00
Nhat Nguyen cc674e7d74 Fix TranslogTests#testStats (#66227)
If creating the latest translog file and retrieving a translog stats 
happen within the same millisecond, then the earliestLastModifiedAge
will be zero.

Closes #66092
2020-12-13 10:44:27 -05:00
Nhat Nguyen 84d4e5bcb1
CCR should check historyUUID in every read request (#66220)
Today, CCR only checks the historyUUID of the leader shard when it has
operations to replicate. If the follower shard is already in-sync with
the leader shard, then CCR won't detect if the historyUUID of the leader
shard has been changed. While this is not an issue, it can annoy users
in the following situation:

The follower index is in-sync with the leader index

Users restore the leader index from snapshots

CCR won't detect the issue and report ok in its stats API

CCR suddenly stops working when users start indexing to the leader index

This commit makes sure that we always check historyUUID in every
read-request so we can detect and report the issue as soon as possible.

Backport of #65841
2020-12-12 12:25:13 -05:00
Simon 9bf6d055eb [Docs] Add option to reference stored templates for `_rank_eval` requests
The option to use templates already defined in the cluster is not explicitly stated in the docs.
This PR adds and example to the `rank_eval` documentation.
2020-12-10 15:12:29 +01:00
Dimitrios Liappis 95dbc6d448
[7.10] Mute fvh REST tests (backport of #66149) #66152
Relates #66147
2020-12-10 12:33:15 +02:00
Julie Tibshirani d707f26eb9 Ensure consistent hit order in fvh REST tests. 2020-12-09 18:15:34 -08:00
Julie Tibshirani b2d3c3f6f9
Fix bug where fvh fragments could be loaded from wrong doc (#66142)
This PR fixes a regression where fvh fragments could be loaded from the wrong
document _source.

Some `FragmentsBuilder` implementations contain a `SourceLookup` to load from
_source. The lookup should be positioned to load from the current hit document.
However, since `FragmentsBuilder` are cached and shared across hits, the lookup
is never updated to load from the new documents. This means we accidentally
load _source from a different document.

The regression was introduced in #60179, which started storing `SourceLookup`
on `FragmentsBuilder`.

Fixes #65533.
2020-12-09 17:52:58 -08:00
James Rodewig 30f1f5c25f
[DOCS] Fix `search_timeout` parameter docs (#66075) (#66126)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: bellengao <gbl_long@163.com>
2020-12-09 13:44:41 -05:00
Francisco Fernández Castaño 55246d8d9b [7.10] Bump version after 7.10.1 release 2020-12-09 16:11:29 +01:00
Rory Hunter 410fac5970 Explicitly retry failed docker pulls (#63946)
Closes #63869. Perform `docker pull` explicitly instead of as part of
`docker build`, and wrap it in a retry loop. This is an attempt to make
the build more resilient to transient errors.
2020-12-09 14:30:45 +00:00
James Rodewig 2e77523923
[DOCS] Remove 7.10.1 coming tag (#66053) 2020-12-09 09:12:16 -05:00
István Zoltán Szabó 4de6006c71
[DOCS] Adds note about data_counts values to Revert snapshot API docs. (#66085) (#66090) 2020-12-09 11:48:43 +01:00
David Turner 9dbfd66731 Clarify network interface setting (#66013)
Today we document the use of `_[networkInterface]_` to specify the
addresses of a network interface but do not spell out which parts of
this syntax should be taken literally and which are part of the
placeholder for the interface name.

This commit clarifies the docs.

NB this is a backport of just the docs changes from #66013.

Closes #65978.
2020-12-09 08:49:44 +00:00
Lee Hinman 8cbb9612d0
[7.10] Create AllocationDeciders in the main method of the ILM step (#65037) (8ac30f9a) (#66070)
Backports the following commits to 7.x:

    Create AllocationDeciders in the main method of the ILM step (#65037) (8ac30f9)
2020-12-08 16:56:25 -07:00
debadair 55a9977d30
[DOCS] Qualified Watcher topics to distinguish from Kib Alerting (#65908) (#66065)
* [DOCS] Qualified Watcher topics to distinguish from Kib Alerting

* Reverted actions anchor due to client links.

* Updated redirects.

* Fixed links.

* Roll back anchor changes

* Apply suggestions from code review

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

* Incorporated review comments.
2020-12-08 13:33:10 -08:00
Gordon Brown fb65fd8723
[7.10] Correctly determine defaults of settings which depend on other settings (#65989)
This commit adjusts the behavior when calculating the diff between two
`AbstractScopedSettings` objects, so that the default values of settings
whose default values depend on the values of other settings are
correctly calculated. Previously, when calculating the diff, the default
value of a depended setting would be calculated based on the default
value of the setting(s) it depends on, rather than the current value of
those settings.
2020-12-08 13:21:00 -07:00
Ernesto Reig 43fa468076
[DOCS] Change `ctx.payload.hits.total.value` to `ctx.payload.hits.total` (#65874)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-12-08 09:22:22 -05:00
Peter Dyson 691f85e5a3
[DOCS] Fix typo (#65993) 2020-12-08 08:48:00 -05:00
Tanguy Leroux 16fae5d66d
Also reroute after shard snapshot size fetch failure (#66008)
In #61906 we added the possibility for the master node to fetch
the size of a shard snapshot before allocating the shard to a
data node with enough disk space to host it. When merging
this change we agreed that any failure during size fetching
should not prevent the shard to be allocated.

Sadly it does not work as expected: the service only triggers
reroutes when fetching the size succeed but never when it
 fails. It means that a shard might stay unassigned until
another cluster state update triggers a new allocation
(as in #64372). More sadly, the test I wrote was wrong as
it explicitly triggered a reroute.

This commit changes the InternalSnapshotsInfoService
so that it also triggers a reroute when fetching the snapshot
shard size failed, ensuring that the allocation can move
forward by using an UNAVAILABLE_EXPECTED_SHARD_SIZE
shard size. This unknown shard size is kept around in the
snapshot info service until no corresponding unassigned
shards need the information.

Backport of #65436
2020-12-08 12:10:37 +01:00
István Zoltán Szabó 063db03d17
[7.10] [DOCS] Adds Working with transforms at scale to docs (#65726) (#65966) 2020-12-08 07:52:32 +01:00
Przemko Robakowski eaab5c65e0
Allow more legit cases in Metadata.Builder.validateDataStreams (#65791) (#65938)
This change simplifies logic and allow more legit cases in Metadata.Builder.validateDataStreams.
It will only show conflict on names that are in form of .ds-<data stream name>-<[0-9]+> and will allow any names like .ds-<data stream name>-something-else-<[0-9]+>.
This fixes problem with rollover when you have 2 data streams with names like a and a-b - currently if a-b has generation greater than a you won't be able to rollover a anymore.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-12-07 19:54:46 +01:00
James Rodewig 53b839fada
[DOCS] Fix wording for HTTP settings (#65964) (#65968) 2020-12-07 12:33:22 -05:00
David Turner 29a50357e1 Expand docs on disk-based shard allocation (#65668)
Today we document the settings used to control rebalancing and
disk-based shard allocation but there isn't really any discussion around
what these processes do so it's hard to know what, if any, adjustments
to make.

This commit adds some words to help folk understand this area better.
2020-12-07 14:56:55 +00:00
James Rodewig 4676cb30d0
[DOCS] Make data stream names consistent (#65920) (#65944) 2020-12-07 09:14:03 -05:00
James Rodewig 24b48230cc
[DOCS] EQL: Add diagrams for sequence matching (#65898) (#65940) 2020-12-07 08:39:41 -05:00
Przemysław Witek d562caf9b2
Fix compile errors in QuerierTests (#65935) 2020-12-07 13:27:36 +01:00
Bogdan Pintea 2ec53ea7c4 Abort sorting in case of local agg sort queue overflow (#65687)
In case the local agg sorter queue gets full and no limit has been provided,
the local sorter will now erroneously call the failure callback for every
single row in the original rowset that's left over the local queue limit
(instead for just the first one).  The failure response is dispatched in any
case, so this is relatively harmless.  The sorter continues iterating on the
original response fetching subsequent pages. In case of correct Elasticsearch
behaviour, this is also harmless, it'll just trigger a number of internal
exceptions. However, in case of a pagination defect in Elasticsearch (like
GH#65685, where the same search_after is returned), this will result in an
effective spin loop, potentially rendering eventually the node unresponsive.

This PR simply breaks both the inner loop iterating over the current unsorted
rowset, as well as the outer one, iterating over the left pages.

It also fixes an outdated documentation limitation.

(cherry picked from commit 638402c387faf79bba38fcc95f371a73146efc0b)
2020-12-07 11:32:41 +01:00
James Rodewig 1ed5a5633f
[DOCS] Fix typo (#65912) (#65914)
Co-authored-by: Toast <mrtoastcheng@gmail.com>
2020-12-05 10:21:02 -05:00
James Rodewig 8b8154594d
[DOCS] Correct the default value of `wait_for_completion` query param (#65800) (#65903)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: bellengao <gbl_long@163.com>
2020-12-04 17:14:50 -05:00
lcawl 5320083bed [DOCS] Add coming tag to release notes 2020-12-04 10:59:10 -08:00
James Rodewig 793eb48502
[DOCS] EQL: Document how sequence queries handle matches (#65794) (#65887)
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-12-04 09:57:08 -05:00
Francisco Fernández Castaño a5e65beab2
Add release notes for 7.10.1 (#65439)
* Add release notes for 7.10.1

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-12-04 12:18:21 +01:00
Jay Modi 55e79dd286
Mute IndicesClientIT.testDataStreams (#65859)
This commit mutes IndicesClientIT.testDataStreams as this test is
failing in CI intermittently.

Relates #60746
Relates #60461
2020-12-03 14:36:35 -07:00
Seth Michael Larson 5a89835025
Mark Task APIs as experimental in rest-api-spec 2020-12-03 15:11:21 -06:00
Nhat Nguyen 26d67c1662 Ensure notify when proxy connections disconnect (#65697)
TransportService doesn't respond to the pending requests of proxy
connections when the underlying connections get disconnected because
proxy connections do not override the getCacheKey method. Some CCS
requests would never be completed because of this bug.
2020-12-03 14:53:17 -05:00
Mike Barretta 4d6a501cec
Update inference-bucket-aggregation.asciidoc (#65833)
minor tweak to lineup a code example and add a missing word
2020-12-03 11:49:24 -05:00
James Rodewig 873f4995c5
[DOCS] Fix typo in histogram agg docs (#65822) (#65828) 2020-12-03 10:53:19 -05:00
leonseng 57dca5f44a
Add missing comma in sample payload for the watcher's pagerduty action 2020-12-03 08:29:23 -06:00
Christoph Büscher dc2444c631 Fix lucene_version_path in docs (#65809)
The `lucene_version_path` variable is used in urls pointing to the Lucene docs.
They use underscore separators for the major and minor versions there.
Correcting this since this has been lost in the latest update on the 7.x branch.
2020-12-03 14:51:55 +01:00
Mark Vieira 49d062efb0 Always use "elasticsearch" as Gradle project name (#65709) 2020-12-02 09:29:36 -08:00
Bhavya Gupta dcff07f717
Updated painless-walkthrough documentation (#65530)
Update documentation about regex note.
2020-12-02 09:23:12 -08:00
James Rodewig ff368304a5
[DOCS] Add cluster get settings API example (#65754) (#65758) 2020-12-02 10:56:51 -05:00
James Rodewig d56d2a399b
[DOCS] Remove inert component template file (#65749) (#65752) 2020-12-02 09:59:00 -05:00
James Rodewig 6ae0370dd7
[DOCS] Fix EQL syntax formatting (#65711) (#65744)
Co-authored-by: Howard <danielhuang@tencent.com>
2020-12-02 09:40:52 -05:00
Thiago Souza 5b422dd19d
[DOCS] Remove erroneous `flat_settings` query param (#65670) 2020-12-02 09:15:15 -05:00