Commit Graph

45640 Commits

Author SHA1 Message Date
Alpar Torok ff64314b2b Simplify testclusters, don't allow cross project clusters (#40972)
* Simplify testclusters, don't allow cross project clusters
2019-04-12 12:41:52 +03:00
Alpar Torok bee892006a Use the built image in docker tests (#40314)
Instead of allowing docker-compose to rebuild it.
With this change we tag the image with a test label, and use that
in the testing as this is simpler that dealing with a dynamically
generated docker-compose file.
2019-04-12 12:34:55 +03:00
Martijn van Groningen d01c1f3ba0
Added enrich policy definition. (#41003)
Relates to #32789
2019-04-12 11:15:02 +02:00
Ignacio Vera 8af930c468
Improve error message when polygons contains twice the same point in no-consecutive position (#41051) (#41133)
When a polygon contains a self-intersection due to have twice the same point in no-consecutive position, the polygon builder tries to split the polygon. During the split one of the polygons become invalid as it is not closed and an error is thrown which is not related to the real issue.

We detect this situation now and throw a more meaningful error.
2019-04-12 09:16:33 +02:00
Martijn van Groningen b66ad34565
Merge remote-tracking branch 'es/7.x' into enrich-7.x 2019-04-12 08:24:59 +02:00
Hendrik Muhs 3df6798c4c Rollup/DataFrame: disallow partial results (#41114)
disallow partial results in rollup and data frame, after this change the client throws an error directly
replacing the previous runtime exception thrown, allowing better error handling in implementations.
2019-04-12 07:31:22 +02:00
Nhat Nguyen e9999dfa1d
Init global checkpoint after copy commit in peer recovery (#40823)
Today a new replica of a closed index does not have a safe commit
invariant when its engine is opened because we won't initialize the
global checkpoint on a recovering replica until the finalize step. With
this change, we can achieve that property by creating a new translog
with the global checkpoint from the primary at the end of phase 1.
2019-04-11 22:18:31 -04:00
Antonio Matarrese 79c7a57737 Use the breadth first collection mode for significant terms aggs. (#29042)
This helps avoid memory issues when computing deep sub-aggregations. Because it
should be rare to use sub-aggregations with significant terms, we opted to always
choose breadth first as opposed to exposing a `collect_mode` option.

Closes #28652.
2019-04-11 15:56:02 -07:00
Nhat Nguyen 0f496842fd Fix msu assertion in restore shard history test
Since #40249, we always reinitialize max_seq_no_of_updates to max_seq_no
when a promoting primary restores history regardless of whether it did
rollback previously or not.

Closes #40929
2019-04-11 18:44:13 -04:00
Nhat Nguyen 9c36ab4ab4 Adjust bwc version for flush parameter validation
Relates to #40213
2019-04-11 18:02:48 -04:00
Ryan Ernst 5cdd87deb7 Remove settings members from Node (#40811)
This commit removes the settings member variable from Node.
This member made it confusing which settings should actually be looked
at. Now all settings are accessed through the final environment.
2019-04-11 13:59:54 -07:00
David Turner b522de975d Move primary term from replicas proxy to repl op (#41119)
A small refactoring that removes the primaryTerm field from ReplicasProxy and
instead passes it directly in to the methods that need it. Relates #40706.
2019-04-11 21:19:27 +01:00
Lisa Cawley e120deb08f [DOCS] Fixes callout for Asciidoctor migration (#41127) 2019-04-11 12:06:10 -07:00
Benjamin Trent 05cf53934a
[ML] checking if p-tasks metadata is null before updating state (#41091) (#41123)
* [ML] checking if p-tasks metadata is null before updating state

* Adding test that validates fix

* removing debug println
2019-04-11 13:54:41 -05:00
Benjamin Trent 9e32e36799
[ML] fixing test related to #40963 (#41074) (#41116) 2019-04-11 11:19:56 -05:00
Armin Braun 233df6b73b
Make Transport Shard Bulk Action Async (#39793) (#41112)
This is a dependency of #39504

Motivation:
By refactoring `TransportShardBulkAction#shardOperationOnPrimary` to async, we enable using `DeterministicTaskQueue` based tests to run indexing operations. This was previously impossible since we were blocking on the `write` thread until the `update` thread finished the mapping update.
With this change, the mapping update will trigger a new task in the `write` queue instead.
This change significantly enhances the amount of coverage we get from `SnapshotResiliencyTests` (and other potential future tests) when it comes to tracking down concurrency issues with distributed state machines.

The logical change is effectively all in `TransportShardBulkAction`, the rest of the changes is then simply mechanically moving the caller code and tests to being async and passing the `ActionListener` down.

Since the move to async would've added more parameters to the `private static` steps in this logic, I decided to inline and dry up (between delete and update) the logic as much as I could instead of passing the listener + wait-consumer down through all of them.
2019-04-11 16:01:52 +02:00
James Rodewig 93d6766afe [DOCS] Improve docs for 'elasticsearch-keystore add-file' command (#41084) 2019-04-11 08:50:13 -04:00
Armin Braun 9cfe194131
Disable TestingConventions Check in Reindex Module (#41100) (#41110)
* This is currently failing with:

```
Test classes are not included in any enabled task (:modules:reindex:test):
  * org.elasticsearch.client.documentation.ReindexDocumentationIT
  * org.elasticsearch.index.reindex.ManyDocumentsIT
  * org.elasticsearch.index.reindex.ReindexClientYamlTestSuiteIT
  * org.elasticsearch.index.reindex.ReindexWithoutContentIT
  * org.elasticsearch.index.reindex.remote.ReindexFromOldRemoteIT
```

* Same fix as in #38546
2019-04-11 13:47:14 +02:00
Mark Vieira a6dc20a09d Create heap dump on OOME (#41031) 2019-04-11 13:41:35 +03:00
Henning Andersen b967a97f8e
Reindex from remote deprecation warning (#41005)
If a reindex from remote request contains an index name that is URL
escaped, we now issue a warning to be able to not support this in 8.0.
2019-04-11 12:09:53 +02:00
Dimitris Athanasiou 736c7285d4
[ML] Fix scroll size comparison in DatafeedUpdate.isNoop (#41056) (#41079)
Note this does not affect users as the method is only used in tests.
2019-04-11 11:29:51 +03:00
James Rodewig 999462f460 [DOCS] Document limits for JSON objects with `ignore_malformed` mapping setting (#40976) 2019-04-10 14:47:26 -04:00
James Rodewig a0b54a9a92 [DOCS] Restructure `ids` to new query docs format (#41077) 2019-04-10 14:36:50 -04:00
Jason Tedor 24446ceae0
Add packaging to cluster stats response (#41048)
This commit adds a packaging_types field to the cluster stats response
that outlines the build flavors and types present in a cluster.
2019-04-10 13:47:19 -04:00
Ryan Ernst 884c3fd7ca
Use separate tasks per bwc artifact (#40703) (#40826)
This commit changes the bwc builds from a single task for a branch to a
task for each bwc artifact. This reduces the bwc build time when only
needing a specific artifact, for example when running cluster restart
tests on a mac, the windows artifacts or rpm/debs are not needed.
2019-04-10 10:43:01 -07:00
Hendrik Muhs c37b127a07 fix a timing issue: isFinished is used for a busy loop in testing, (#41055)
test: ensure state is persisted before the isFinished is changed

fixes #41046
2019-04-10 18:47:34 +02:00
Zachary Tong e611334b2b Add 7.0.1 version constant 2019-04-10 11:32:53 -04:00
Martijn van Groningen 1eff8976a8
Deprecate AbstractHlrc* and AbstractHlrcStreamable* base test classes (#41014)
* moved hlrc parsing tests from xpack to hlrc module and removed dependency on hlrc from xpack core

* deprecated old base test class

* added deprecated jdoc tag

* split test between xpack-core part and hlrc part

* added lang-mustache test dependency, this previously came in via
hlrc dependency.

* added hlrc dependency on a qa module

* duplicated ClusterPrivilegeName class in xpack-core, since x-pack
core no longer has a dependency on hlrc.

* replace ClusterPrivilegeName usages with string literals

* moved tests to dedicated to hlrc packages in order to remove Hlrc part from the name and make sure to use imports instead of full qualified class where possible

* remove ESTestCase. from method invocation and use method directly,
because these tests indirectly extend from ESTestCase
2019-04-10 16:29:17 +02:00
Chris Earle 64a05e522f
Properly handle Monitoring exporters all disabled (#40920) (#41043)
When monitoring exporters are all disabled, which must be done
explicitly, _and_ monitoring collection is enabled, then
any call to `_xpack/monitoring/_bulk` will create a task that
never closes _and_ ES collection will stop happening because
a semaphore is never marked as completed.

This also simplifies the async `ExportBulk` code by removing the
third step (second async step, `close`) entirely because it was
entirely unnecessary by both implementations.
2019-04-10 10:09:28 -04:00
Christoph Büscher d00d3f4afa Mute DataFrameTransformCheckpointTests#testGetBehind 2019-04-10 15:55:09 +02:00
Zachary Tong 0a85d1fe03 Add missing "long form" 7.0 breaking changes (#41049)
These are breaking changes that were present in the release notes but the PR didn't include any "narrative long form" description for the docs.
2019-04-10 09:44:45 -04:00
Zachary Tong 9aa8ab58ee Add 7.0.0 Release Notes (#41029) 2019-04-10 09:44:45 -04:00
Przemysław Witek f5014ace64
[ML] Add validation that rejects duplicate detectors in PutJobAction (#40967) (#41072)
* [ML] Add validation that rejects duplicate detectors in PutJobAction

Closes #39704

* Add YML integration test for duplicate detectors fix.

* Use "== false" comparison rather than "!" operator.

* Refine error message to sound more natural.

* Put job description in square brackets in the error message.

* Use the new validation in ValidateJobConfigAction.

* Exclude YML tests for new validation from permission tests.
2019-04-10 15:43:35 +02:00
Dimitrios Liappis 799541e068
Mute DateTimeUnitTests.testConversion (#40738)
Due to #39617

Backport of  #40086
2019-04-10 16:37:16 +03:00
Ed Savage 722362e402 Mute MachineLearningIt#testDeleteExpiredData
Tracked in #41070
2019-04-10 13:07:53 +01:00
Albert Zaharovits adf3393a4e
Deprecate permission over aliases (#38059) (#41060)
This PR generates deprecation log entries for each Role Descriptor,
used for building a Role, when the Role Descriptor grants more privileges
for an alias compared to an index that the alias points to. This is done in
preparation for the removal of the ability to define privileges over aliases.
There is one log entry for each "role descriptor name"-"alias name" pair.
On such a notice, the administrator is expected to modify the Role Descriptor
definition so that the name pattern for index names does not cover aliases.

Caveats:
* Role Descriptors that are not used in any authorization process,
either because they are not mapped to any user or the user they are mapped to
is not used by clients, are not be checked.
* Role Descriptors are merged when building the effective Role that is used in
the authorization process. Therefore some Role Descriptors can overlap others,
so even if one matches aliases in a deprecated way, and it is reported as such,
it is not at risk from the breaking behavior in the current role mapping configuration
and index-alias configuration. It is still reported because it is a best practice to
change its definition, or remove offending aliases.
2019-04-10 15:02:33 +03:00
Costin Leau 38f471ae1c SQL: Change schema calls to empty set (#41034)
As empty string has a certain meaning, the JDBC driver returns an empty
set instead for better client compatibility.

Fix #41028

(cherry picked from commit 4cbafa585b7a514eb6c156606dd516324cd3980a)
2019-04-10 13:14:12 +03:00
Jim Ferenczi 4263a28039 Fix rewrite of inner queries in DisMaxQueryBuilder (#40956)
This commit implements missing rewrite for the DisMaxQueryBuilder.

Closes #40953
2019-04-10 11:38:16 +02:00
Adrien Grand ecbb588a14 Fix command line to run doc tests in docs/README. (#40959)
It runs `gradle` instead of `gradlew` and has an extra `\`.
2019-04-10 11:35:19 +02:00
Adrien Grand 683cf56982 Update headline of the "removal of types" doc page to match changes in 7.0. (#40868)
Currently it describes what broke in 6.0.
2019-04-10 11:34:29 +02:00
Hendrik Muhs f9018ab11b [ML-DataFrame] create checkpoints on every new run (#40725)
Use the checkpoint service to create a checkpoint on every new run. Expose checkpoints stats on _stats endpoint.
2019-04-10 09:14:11 +02:00
Martijn van Groningen 46b0fdae33
Add realistic hlrc request serialization test base class and (#40362)
changed hlrc ccr request tests to use AbstractRequestTestCase base class.

This way the request classes are tested in a more realistic setting.
Note this change also adds a test dependency on xpack core module.

Similar to #39844 but then for hlrc request serialization tests.

Removed iterators from hlrc parsing tests.
Use empty xcontent registries.

Relates to #39745
2019-04-10 08:00:01 +02:00
Andrei Stefan 0157ebf947 Increase the interval filtering for CURRENT_DATE/TODAY tests (#40999)
(cherry picked from commit a5911330aecee90da5401e468b9976f66a2556f0)
2019-04-10 08:35:09 +03:00
Jason Tedor bb6f060f74
Add log message to forget follower test
This commit adds a log message to help debug failures in a forget
follower test.
2019-04-09 23:33:29 -04:00
Jason Tedor 3aae98f922
Add debug logging for leases sync on recovery test
This commit adds some debug logging for a retention leases sync on
recovery test.
2019-04-09 22:59:22 -04:00
Julie Tibshirani d38214060e Mute ClusterDisruptionIT#testCannotJoinIfMasterLostDataFolder.
Tracked in #41047.
2019-04-09 17:36:21 -07:00
Jason Tedor c0f715b337
Fix Docker build when sourced from artifacts
This commit fixes an issue when the artifact used to build the Docker
image is sourced from artifacts.elastic.co. In particular, the artifact
was not downloaded to the proper location.
2019-04-09 20:33:15 -04:00
Julie Tibshirani 0d5f86a001 Mute RollupIndexerStateTests#testIndexing.
Tracked in #41046.
2019-04-09 17:17:04 -07:00
Julie Tibshirani 0702c72151 Mute DataFrameGetAndGetStatsIT#testGetPersistedStatsWithoutTask.
Tracked in #40963.
2019-04-09 16:39:16 -07:00
Julie Tibshirani 21c5d7e95f Mute CcrRetentionLeaseIT#testRetentionLeasesAreNotBeingRenewedAfterRecoveryCompletes.
Tracked in #39331.
2019-04-09 16:08:44 -07:00