Commit Graph

53217 Commits

Author SHA1 Message Date
Mark Tozzi ab8518fb5b
[7.x] Extensibility for Composite Agg #59648 (#60842) 2020-08-11 09:14:33 -04:00
Dan Hermann 839c6cdfc0
Un-mute data stream REST test (#60120) (#60939) 2020-08-11 08:10:04 -05:00
James Rodewig 4aae278d1d
[DOCS] Move post filter/rescore content to new page (#60903) (#60961) 2020-08-11 09:06:59 -04:00
David Kyle 18a65c5b9a
DFA Get Stats can return multiple responses if more than one error occurs (#60950)
If the search for get stats with multiple job Ids fails the listener is called for each failure. 
This change waits for all responses then returns the first error if there was one.
2020-08-11 10:28:05 +01:00
Rene Groeschke a5ef38ca40
Update gradle wrapper to 6.6 (#59909) (#60949) 2020-08-11 11:03:19 +02:00
Henning Andersen a0b54b53fc Rest high level ReindexIT fix (#60834)
ReindexIT would rethrottle any delete or update by query task, fixed to
more precisely match the task started by the test.

Closes #60811
2020-08-11 10:35:15 +02:00
Alan Woodward 54279212cf
Make MetadataFieldMapper extend ParametrizedFieldMapper (#59847) (#60924)
This commit cuts over all metadata field mappers to parametrized format.
2020-08-11 09:02:28 +01:00
Armin Braun 3e2dfc6eac
Remove GCS Bucket Exists Check (#60899) (#60914)
Same as https://github.com/elastic/elasticsearch/pull/43288 for GCS.
We don't need to do the bucket exists check before using the repo, that just needlessly
increases the necessary permissions for using the GCS repository.
2020-08-11 09:54:27 +02:00
Julie Tibshirani d51eae6e9f
Prevent loading 'fields' with stored fields disabled. (#60938)
Because the 'fields' option loads from _source (which is a stored field), it is
not possible to retrieve 'fields' when stored_fields are disabled.

This also fixes #60912, where setting stored_fields: _none_ prevented the
_ignored fields from being loaded and caused a parsing exception.
2020-08-10 15:40:27 -07:00
debadair 063518ca2b
[DOCS] Mention that inline scripts need to be enabled for Kibana (#60633) (#60798) 2020-08-10 13:28:59 -07:00
Nik Everett 0286d0a769
Move distance_feature query building into MFT (#60614) (#60846)
This moves the `distance_feature` query building out of
`DistanceFeatureQueryBuilder` and into subclasses of `MappedFieldType`.
Without this we don't have a chance of supporting this for runtime
fields. In general I'm not sad to see the `instanceof`s go.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-08-10 16:05:17 -04:00
James Rodewig 1b2a015734
[DOCS] Cross-link `copy_to` and search speed docs (#60926) (#60928) 2020-08-10 15:35:10 -04:00
Julie Tibshirani b216340f50
Make `FetchPhase` logic more readable. (#60779)
* Factor out FieldsVisitor#postProcess call.
* Swap logical order for normal and nested documents.
* Extract the method createStoredFieldsVisitor.
2020-08-10 11:04:54 -07:00
James Rodewig 877ecd5b66
[DOCS] Add PUT example to `Date math in index names` (#60908) (#60920)
Previously, all examples in this section were GET requests. This
demonstrates that other CRUD operations are also supported.
2020-08-10 12:46:10 -04:00
Nik Everett dfd502f9ca
Rework checking if a year is a leap year (#60585) (#60790)
This way is faster, saving about 8% on the microbenchmark that rounds to
the nearest month. That is in the hot path for `date_histogram` which is
a very popular aggregation so it seems worth it to at least try and
speed it up a little.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-08-10 12:45:34 -04:00
Benjamin Trent 66b3e89482
[ML] enable logging for test failures (#60902) (#60910) 2020-08-10 12:36:30 -04:00
Francisco Fernández Castaño 2a4fd8329b
Avoid a race condition while waiting for pre warm to finish on SearchableSnapshotDirectoryTests (#60906)
Backport of #60885. Closes #60813
2020-08-10 17:29:16 +02:00
Jim Ferenczi f30f1f04e2
Replace AggregatorTestCase#search with AggregatorTestCase#searchAndReduce (#60816)
This commit removes the ability to test the top level result of an aggregator
before it runs the final reduce. All aggregator tests that use AggregatorTestCase#search
are rewritten with AggregatorTestCase#searchAndReduce in order to ensure that we test
the final output (the one sent to the end user) rather than an intermediary result
that could be different.
This change also removes spurious commits triggered on top of a random index writer.
These commits slow down the tests and are redundant with the commits that the
random index writer performs.
2020-08-10 17:23:00 +02:00
David Roberts dd02e9f31a [TEST] Mute SearchableSnapshotActionIT testSearchableSnapshotForceMergesIndexToOneSegment (#60904)
Due to https://github.com/elastic/elasticsearch/issues/60901
2020-08-10 15:25:39 +01:00
James Rodewig 739097a56c
[DOCS] Move `min_score` docs to search API page (#60895) (#60896)
Reformats the `min_score` docs as a param definition on the
search API reference page.
2020-08-10 09:43:07 -04:00
Henning Andersen a155315ceb
Autoscaling decider and decision service (#59005) (#60884)
Split the autoscaling decider into a service and configuration
in order to enable having additional context information available
in the service. Added AutoscalingDeciderContext holding generic
information all deciders are expected to need. Implemented GET
_autoscaling/decision
2020-08-10 15:28:52 +02:00
James Rodewig 8a0f1d8746
[DOCS] Combine highlighting docs files (#60849) (#60892) 2020-08-10 09:05:49 -04:00
Dan Hermann 192dc9dd3d
[DOCS] Update get data stream API (#60862) 2020-08-10 08:03:17 -05:00
David Turner f44c28b595
Deprecate and ignore join timeout (#60872)
There is no point in timing out a join attempt any more once a cluster
is entirely in 7.x. Timing out and retrying with the same master is
pointless, and an in-flight join attempt to one master no longer blocks
attempts to join other masters. This commit deprecates this unnecessary
setting and removes its effect from the joining process.

Relates #60873 which removes this setting in master.
2020-08-10 13:57:41 +01:00
Andrei Dan 235e5ed3ea
[7.x] ILM: add force-merge step to searchable snapshots action (#60819) (#60882)
This adds a force-merge step to the searchable snapshot action, enabled by default,
but parameterizable using the `force_merge-index" optional boolean.

eg.
```
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "cold": {
        "actions": {
          "searchable_snapshot" : {
            "snapshot_repository" : "backing_repo",
            "force_merge_index": true
          }
        }
      }
    }
  }
}
```

(cherry picked from commit d0a17b2d35f1b083b574246bdbf3e1929471a4a9)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-08-10 13:45:11 +01:00
Rene Groeschke e03993d238
Make jdk download repo not consumable (#60875)
- fixes #60860
2020-08-10 14:22:42 +02:00
Martijn van Groningen 64bb082f9b
Improve error message for non append-only writes that target data stream (#60874)
Backport of #60809 to 7.x branch.

Closes #60581
2020-08-10 13:18:59 +02:00
David Kyle 6b2ddf4453
Fix typo in DataHistogramGroupByIT name (#60880) (#60883) 2020-08-10 11:55:01 +01:00
David Turner f168bdac7d Change transitive -> transient in ILM log message (#60871)
"Transitive" is technically ok here but it's an overloaded word and it's
not immediately clear which meaning is intended so this log message
always makes me do a double-take. I think both "transient" and
"transitory" are clearer, with "transient" being the usual choice.
2020-08-10 11:37:49 +01:00
Alan Woodward e8d9185045 Cut over IPFieldMapper to parametrized form (#60602)
This commit makes IpFieldMapper extend ParametrizedFieldMapper. It also
updates the IpFieldMapper docs to add the ignore_malformed parameter,
which was not previously documented.
2020-08-10 11:01:10 +01:00
David Turner a2d5bfca2f Even longer timeout for XPackRestIT (#60812)
This suite is still occasionally failing with a timeout on macOS.
Suggest further increasing this timeout until this suite is broken up.

Relates #58071
2020-08-10 10:26:21 +01:00
David Turner 1f49e0b9d0 Fix testRerouteOccursOnDiskPassingHighWatermark (#60869)
Sometimes this test would refresh the disk stats so quickly that it hit
the refresh rate limiter even though it was almost completely disabled.
This commit allows the rate limiter to be completely disabled.

Closes #60587
2020-08-10 09:39:44 +01:00
Rory Hunter 3bac6db466
Use explicit GID in 7.x when chrooting in Docker entrypoint (#60859)
Closes #60853. After upgrading to CentOS 8, the behaviour of chroot has
subtly changed. Now we have to explicitly set the GID in order to get
the previous behaviour of creating files with GID 0.
2020-08-08 10:26:17 +01:00
Rory Hunter 5c76978e80 Wait for ES in Docker to start before checking files (#60825)
Closes #54817.
2020-08-07 14:20:14 +01:00
Mark Vieira 8e9f8ba420
Mute DockerTests.test010Install 2020-08-06 16:45:41 -07:00
Ryan Ernst cf39a69be3
Exit gradle run task if any node dies (#60789)
The Elasticsearch `run` task in Gradle is backed by testclusters
spinning up a node and waiting indefinitely on output from that node.
However, this thread currently waits for an interrupt to exit. If
Elasticsearch fails to start, we hang indefinitely. This commit adds a
check after each poll of the process stdout to see if it is still alive,
and fails the build if it has died.

closes #60761
2020-08-06 15:54:43 -07:00
Ryan Ernst ddcfbec569
Add assert message for multiple lines in osprobe (#60796)
Several /proc files are expected to contain a single line. We assert on
this in tests, but the contents of the file are lost and the assertion
therefore lacks important information to debug why the file appeared to
have multiple lines. This commit dumps the contents of the file on
assertion failure.

relates #59284
2020-08-06 15:53:30 -07:00
Ryan Ernst 479f32906d
Cleanup vagrant setup (#60697)
Though bats tests were recently removed, there remains a few unnecessary
setups needed for those tests in our vagrant files, as well as CI setup.
Additionally, we no longer rely on the vagrant images setting a
JAVA_HOME, instead relying on DistroTestPlugin to pull the appropriate
jdk when testing with no-jdk distributions.
2020-08-06 15:37:56 -07:00
Ryan Ernst fc38af363e
Ensure latch is counted down when assertion trips (#60800)
The ReloadSecureSettingsIT uses latches to ensure coordination across
requests to the underlying in memory cluster. However, in the case of an
expected failure, if the assertion fails, the latch will never be
counted down, and will cause the test to hang indefinitely. This commit
ensures the latch is always counted down with a try/finally.

relates #51546
2020-08-06 15:33:46 -07:00
Jim Ferenczi 98119578a1 Disable sort optimization on search collapsing (#60838)
Collapse search queries that sort by a field can throw
an ArrayStoreException due to a bug in the [sort optimization](https://github.com/elastic/elasticsearch/pull/51852)
introduced in 7.7.0. Search collapsing were not supposed to
be eligible for this sort optimization so this change explicitly
filters them from this new feature.
2020-08-06 21:37:12 +02:00
James Rodewig 788778c139
[DOCS] Move inner hits content to separate page (#60840) (#60843)
Moves inner hits content from the deprecated 'Request Body Search'
chapter to a separate page.
2020-08-06 14:06:01 -04:00
James Rodewig a761985fab
[DOCS] Move script and stored fields content to search fields page (#60826) (#60835)
Changes:

* Moves `Retrieve selected fields` to its own page and adds a title abbreviation.
* Adds existing script and stored fields content to `Retrieve selected fields`
* Adds a xref for `Retrieve selected fields` to `Search your data`
* Adds related redirects and updates existing xrefs
2020-08-06 13:06:06 -04:00
Jim Ferenczi 14980ff97e Fix AOOBE when setting min_doc_count to 0 in significant_terms (#60823)
This commit fixes the computation of the subset size on empty buckets (doc count of 0).
The aggregator test refactoring in #60683 revealed this bug.
2020-08-06 18:57:09 +02:00
Mark Tozzi fb7c431d8d
[DOCS] Update snapshot repo usage (#60791) (#60831)
Clarify how to use our snapshot repository.  Several folks were confused about this just now, including myself.
2020-08-06 12:29:21 -04:00
James Rodewig ff4ea4720a
[DOCS] Update example data stream names (#60783) (#60820)
Uses `my-data-stream` in place of `logs` for data stream examples.
This provides a more intuitive experience for users that copy/paste
their own values into snippets.
2020-08-06 09:38:35 -04:00
David Turner 721198c29e Increase logging in testRerouteOccursOnDiskPassingHighWatermark (#60817)
Relates #60587
2020-08-06 14:08:09 +01:00
Armin Braun a2c7991e96
Fix CompressibleBytesOutputStreamTests (#60815) (#60822)
Since #60730 the `bytes` field can be `null`. This adds the missing `null` check to the test
override.

Closes #60814
2020-08-06 15:07:48 +02:00
Benjamin Trent bc17afc535
[7.x] [ML] have DELETE analytics ignore stats failures and clean up unused stats (#60776) (#60784)
* [ML] have DELETE analytics ignore stats failures and clean up unused stats (#60776)

When deleting an analytics configuration, the request MIGHT fail if
the .ml-stats index does not exist or is in strange state (shards unallocated).

Instead of making the request fail, we should log that we were unable to delete the stats docs and then
have them cleaned up in the 'delete_expire_data' janitorial process
2020-08-06 08:55:35 -04:00
Rory Hunter 69645ee4ff Upgrade Docker image from CentOS 7 to 8 2020-08-06 13:44:58 +01:00
Steven Goossens a62c5ae11b Upgrade Centos version in Dockerfile to 8 (#59019) 2020-08-06 13:44:58 +01:00