Commit Graph

54675 Commits

Author SHA1 Message Date
Andrei Stefan 12ffe4a4b3
QL: handle IP type fields extraction with ignore_malformed property (#66622) (#66686)
Return null for any field that is present in the _ignored section of the
response, not only numerics and IPs.

(cherry picked from commit 106719f6af1eb2f06396161143a5f0185c5d1b54)
2020-12-21 16:07:03 +02:00
Julie Tibshirani d4039228ae Fix regressions around nested hits and disabled _source. (#66572)
This PR fixes two bugs that can arise when _source is disabled and we fetch nested documents:
* Fix exception when highlighting `inner_hits` with disabled _source.
* Fix exception in nested `top_hits` with disabled _source.
* Add more tests for highlighting `inner_hits`.
2020-12-18 15:21:37 -08:00
Gordon Brown df8c92cfef
Mute tests failing on Debian 8 due to memory reporting (#66648) 2020-12-18 15:27:07 -07:00
Adam Locke a4d5e36905
[DOCS] Clarifying TLS steps (#65326) (#66614)
* [DOCS] Clarifying TLS steps

* Fix file name

* Clarifying note based on reviewer feedback
2020-12-18 16:00:28 -05:00
AndyHunt66 e2228df04a
[DOCS] Fix typos (#66576) 2020-12-18 12:02:53 -05:00
Adam Locke 3b58f8fd5b
[DOCS] Adding headers in TOC for aggregation docs. (#66604) (#66608) 2020-12-18 12:00:20 -05:00
Andrei Dan 2620725297
Fix MANAGE_IDX_TEMPLATE privilege to allow `component_template/*` (#66514) (#66581)
(cherry picked from commit bcc28e0ab8e6883e14b23f93f428dee03b377a1d)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-12-18 11:17:16 +00:00
Albert Zaharovits 480561dbc3
Store and use only internal security headers (#66365)
For async searches (EQL included) the client's request headers were
erroneously stored in the .tasks index. This might expose the requesting
client's HTTP Authorization header. This PR fixes that by employing the
usual approach to store only the security-internal headers, which carry
the authentication result, instead of the original Authorization header,
which is commonly utilized to redo authentication for scheduled tasks.
2020-12-17 23:40:55 +02:00
James Rodewig 50aca62486
[DOCS] Fix rollup docs formatting (#66425) (#66547)
Co-authored-by: Howard <danielhuang@tencent.com>
2020-12-17 14:14:28 -05:00
Costin Leau 4cb3ee5b4e EQL: Fix early trimming of in-flight data (#66493)
Rework trimToLast to take into account an ordinal for last trimming so
instead of keeping the last entry in a stage, it keeps the last entry
before the given ordinal.
This takes care of the case where a dense stage that requires several
passes does not discard valid data from a previous sparse stage that go
beyond the current stage point.

(cherry picked from commit 4f55749072b39f89822bdd52c67998f7bed890a9)
(cherry picked from commit 6b61dfead88a144c6e85e384d47a24f0c1480c6b)
(cherry picked from commit cece81b5dee88b18e3e7ea189fc342ef53ea19f2)
2020-12-17 18:00:33 +02:00
James Rodewig 499f4a09e7
[DOCS] Fix S3 bucket names in S3 repo plugin docs (#66521) (#66529)
Co-authored-by: Inbar Shimshon <inbar.shimshon@elastic.co>
2020-12-17 10:27:57 -05:00
Inbar Shimshon c17b409237
Update repository-s3.asciidoc 2020-12-17 15:30:46 +02:00
Inbar Shimshon 713f97a20b
Update repository-s3.asciidoc 2020-12-17 15:12:28 +02:00
Inbar Shimshon 8a8c6aff45
Update repository-s3.asciidoc
Additionally I want to point out that in all of our examples we of bucket name we use ourselves underscores which is confusing since using "_" is not supported for AWS  
see https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-s3-bucket-naming-requirements.html


IMO this is a larger change throughout the docs that should be made - its something we know about for a while and mentioned here https://github.com/elastic/sdh-cloud/issues/1501#issuecomment-638122152 though also that should be updates because specifically underscore is not mentioned there.
2020-12-17 15:11:35 +02:00
Julie Tibshirani 24c0f01543 Ensure all query builder tests consider older versions. (#66401)
This PR removes outdated overrides in some tests that prevent them from testing
older index versions. Also removes an old comment + logic from
AggregatorFactoriesTests.
2020-12-16 11:57:01 -08:00
Benjamin Trent a370104535
[ML] change to only calculate model size on initial load to prevent slow cache promotions (#66451) (#66462)
When a value is promoted in the LRU cache, its weight is removed and added.

The LocalModel object was recalculating the model size for ever weight check, which caused a polynomial runtime.

This commit changes the model size to only be calculated in the LocalModel ctor.
2020-12-16 14:01:21 -05:00
asce0705 5000ec87ca Fix constructors of NoOpResult (#66269) 2020-12-16 13:37:49 -05:00
James Rodewig 5e5eb7bc08
[DOCS] Fix formatting (#66450) (#66455) 2020-12-16 11:40:03 -05:00
Gao Ruifeng a3757ab43e
[DOCS] Remove duplicate word (#66320) 2020-12-16 10:16:47 -05:00
Bogdan Pintea 176587ebc4
QL: Verify filter's condition type (backport of #66268) (#66408)
* SQL: Verify filter's condition type (#66268)

* Verify filter's condition type

This adds a check in the verifier to check if filter's condition is of a
boolean type and fail the request otherwise.

(cherry picked from commit 3aec1a3d99a3f4650ec8be014a97106320f0874a)
2020-12-15 23:24:11 +01:00
Julie Tibshirani 9fa80c1896 Fix failure in fvh REST tests. (#66192)
In general, we can't guarantee that a match_all query will return documents in
the order they were indexed. This PR adds an ID to each document to avoid
relying on document order.
2020-12-15 12:01:25 -08:00
David Turner c318e14fed Fix Log4J config example in docs (#66386)
We lost the `logger.transport.name` line in #65169 and I incorrectly
extrapolated from what was left and mangled it further in #66318. This
commit fixes things.
2020-12-15 17:43:30 +00:00
David Turner fd97285b32 Give a less dangerous example of logger config (#66318)
Today the docs use `logger.org.elasticsearch.transport: TRACE` as the
example for adjusting the logger config. This is a dangerous thing to
suggest since that's one of the most verbose loggers we have. An
accidental copy-and-paste of this example into a busy cluster can
cause damage.

This commit suggests `logger.org.elasticsearch.discovery: DEBUG`
instead, which is much more benign.

It also corrects the order of the levels and notes that `DEBUG` and
`TRACE` are only for expert use.
2020-12-15 17:05:53 +00:00
Adam Locke 24c612e256
[DOCS] Add description for node info settings. (#66362) (#66376) 2020-12-15 11:45:18 -05:00
James Rodewig 937114c934
[DOCS] Note `tar.gz` does not include `systemd` (#66298) (#66345) 2020-12-15 10:13:50 -05:00
James Rodewig 1fe3cd5eaf
[DOCS] Fix ingest node.roles example (#66287) (#66340) 2020-12-15 08:59:02 -05:00
James Rodewig e9382487a3
[DOCS] Fix `search.max_buckets` default (#66311) (#66313) 2020-12-15 08:18:02 -05:00
Jay Modi 01d54d222b
Fix cat tasks api params in spec and handler (#66294)
This commit fixes the cat tasks api parameter specification and the
handler so that the parameters are consumed during request preparation.

Closes #59493
Backport of #66272
2020-12-14 13:44:09 -07:00
James Rodewig 170c976acc
[DOCS] Add `require_alias` to Bulk API (#66259) (#66266) 2020-12-14 11:21:33 -05:00
Jim Ferenczi 330de82d59 Fix composite aggregation on unsigned long (#65715)
This commit ensures that the after key is parsed with the doc value formatter.
This is needed for unsigned longs that uses shifted longs internally.

Closes #65685
2020-12-14 16:59:07 +01:00
James Rodewig 25f51f82ab
[DOCS] Add adaptive replica selection reference (#66232) (#66262)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: calm4wei <alferwei98@163.com>
2020-12-14 10:47:16 -05:00
Armin Braun 7e1fc6dc67
Adjust Cleanup Order of Internal State in SnapshotsService (#66225) (#66244)
In the assertion mentioned in the new comment we first get the `endingSnapshots`
and then check that we don't have a listener that isn't referred to by it so we need
to remove from the listers map before removing from `endingSnapshots` to avoid rare, random
assertion tripping here with concurrent repository operations.
2020-12-14 12:24:39 +01:00
Marios Trivyzas 416ea4fcdc
EQL: [Tests] New eql correctness data snapshot (#66238)
With the upcoming validation for type compatibility of the sequence
keys, several tests are failing because some fields that contain IP
data were previously mapped as keyword. Fixed the mapping and created a
new snaphost of the correctness data in the gcs bucket.

Relates to: #66183

(cherry picked from commit 7f638f661c5a5c57a4ea7d3d3e2ccf5c81ae92d1)
2020-12-14 10:34:14 +01:00
Nhat Nguyen cc674e7d74 Fix TranslogTests#testStats (#66227)
If creating the latest translog file and retrieving a translog stats 
happen within the same millisecond, then the earliestLastModifiedAge
will be zero.

Closes #66092
2020-12-13 10:44:27 -05:00
Nhat Nguyen 84d4e5bcb1
CCR should check historyUUID in every read request (#66220)
Today, CCR only checks the historyUUID of the leader shard when it has
operations to replicate. If the follower shard is already in-sync with
the leader shard, then CCR won't detect if the historyUUID of the leader
shard has been changed. While this is not an issue, it can annoy users
in the following situation:

The follower index is in-sync with the leader index

Users restore the leader index from snapshots

CCR won't detect the issue and report ok in its stats API

CCR suddenly stops working when users start indexing to the leader index

This commit makes sure that we always check historyUUID in every
read-request so we can detect and report the issue as soon as possible.

Backport of #65841
2020-12-12 12:25:13 -05:00
Simon 9bf6d055eb [Docs] Add option to reference stored templates for `_rank_eval` requests
The option to use templates already defined in the cluster is not explicitly stated in the docs.
This PR adds and example to the `rank_eval` documentation.
2020-12-10 15:12:29 +01:00
Dimitrios Liappis 95dbc6d448
[7.10] Mute fvh REST tests (backport of #66149) #66152
Relates #66147
2020-12-10 12:33:15 +02:00
Julie Tibshirani d707f26eb9 Ensure consistent hit order in fvh REST tests. 2020-12-09 18:15:34 -08:00
Julie Tibshirani b2d3c3f6f9
Fix bug where fvh fragments could be loaded from wrong doc (#66142)
This PR fixes a regression where fvh fragments could be loaded from the wrong
document _source.

Some `FragmentsBuilder` implementations contain a `SourceLookup` to load from
_source. The lookup should be positioned to load from the current hit document.
However, since `FragmentsBuilder` are cached and shared across hits, the lookup
is never updated to load from the new documents. This means we accidentally
load _source from a different document.

The regression was introduced in #60179, which started storing `SourceLookup`
on `FragmentsBuilder`.

Fixes #65533.
2020-12-09 17:52:58 -08:00
James Rodewig 30f1f5c25f
[DOCS] Fix `search_timeout` parameter docs (#66075) (#66126)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: bellengao <gbl_long@163.com>
2020-12-09 13:44:41 -05:00
Francisco Fernández Castaño 55246d8d9b [7.10] Bump version after 7.10.1 release 2020-12-09 16:11:29 +01:00
Rory Hunter 410fac5970 Explicitly retry failed docker pulls (#63946)
Closes #63869. Perform `docker pull` explicitly instead of as part of
`docker build`, and wrap it in a retry loop. This is an attempt to make
the build more resilient to transient errors.
2020-12-09 14:30:45 +00:00
James Rodewig 2e77523923
[DOCS] Remove 7.10.1 coming tag (#66053) 2020-12-09 09:12:16 -05:00
István Zoltán Szabó 4de6006c71
[DOCS] Adds note about data_counts values to Revert snapshot API docs. (#66085) (#66090) 2020-12-09 11:48:43 +01:00
David Turner 9dbfd66731 Clarify network interface setting (#66013)
Today we document the use of `_[networkInterface]_` to specify the
addresses of a network interface but do not spell out which parts of
this syntax should be taken literally and which are part of the
placeholder for the interface name.

This commit clarifies the docs.

NB this is a backport of just the docs changes from #66013.

Closes #65978.
2020-12-09 08:49:44 +00:00
Lee Hinman 8cbb9612d0
[7.10] Create AllocationDeciders in the main method of the ILM step (#65037) (8ac30f9a) (#66070)
Backports the following commits to 7.x:

    Create AllocationDeciders in the main method of the ILM step (#65037) (8ac30f9)
2020-12-08 16:56:25 -07:00
debadair 55a9977d30
[DOCS] Qualified Watcher topics to distinguish from Kib Alerting (#65908) (#66065)
* [DOCS] Qualified Watcher topics to distinguish from Kib Alerting

* Reverted actions anchor due to client links.

* Updated redirects.

* Fixed links.

* Roll back anchor changes

* Apply suggestions from code review

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

* Incorporated review comments.
2020-12-08 13:33:10 -08:00
Gordon Brown fb65fd8723
[7.10] Correctly determine defaults of settings which depend on other settings (#65989)
This commit adjusts the behavior when calculating the diff between two
`AbstractScopedSettings` objects, so that the default values of settings
whose default values depend on the values of other settings are
correctly calculated. Previously, when calculating the diff, the default
value of a depended setting would be calculated based on the default
value of the setting(s) it depends on, rather than the current value of
those settings.
2020-12-08 13:21:00 -07:00
Ernesto Reig 43fa468076
[DOCS] Change `ctx.payload.hits.total.value` to `ctx.payload.hits.total` (#65874)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-12-08 09:22:22 -05:00
Peter Dyson 691f85e5a3
[DOCS] Fix typo (#65993) 2020-12-08 08:48:00 -05:00