Commit Graph

47557 Commits

Author SHA1 Message Date
James Rodewig d64c31e43d [DOCS] Rewrite cross-cluster seach docs (#45583) 2019-08-15 13:23:40 -04:00
Michael Basnight db57d2206a Prevent delete policy for active executing policy (#45472)
This commit adds a lock to the delete policy, in the same way that the
locking is done for policy execution. It also creates a test to exercise
the delete transport action, and modifies an existing test to provide a
common set of functions for saving and deleting policies.
2019-08-15 10:08:11 -05:00
James Rodewig c75fd40f2c [DOCS] Add diagrams to cross-cluster search documentation (#45569) 2019-08-15 11:00:25 -04:00
Alpar Torok 7119e54be5 Mute data frame tests on 7.x
Tracking in #45610 #45609
2019-08-15 17:07:53 +03:00
Alpar Torok 03a1645bc6 Use dynamic port ranges for ExternalTestCluster (#45601)
Moves methods added in #44213 and uses them to configure the port range
for `ExternalTestCluster` too.
These were still using `9300-9400` ( teh default ) and running into
races.
2019-08-15 16:40:12 +03:00
Michael Basnight 03f45dad57
Fix policy removal bug in delete policy (#45573)
The delete policy had a subtle bug in that it would still delete the
policy if pipelines were accessing it, after giving the client back an
error. This commit fixes that and ensures it does not happen by adding
verification in the test.
2019-08-15 13:20:59 +02:00
David Roberts d40f3718f2 [ML] Muting 5 SSLErrorMessageTests tests on Windows (#45602)
Due to https://github.com/elastic/elasticsearch/issues/45598
2019-08-15 11:05:00 +01:00
Armin Braun 1beea3588b
Make BlobStoreRepository Validation Read master.dat (#45546) (#45578)
* Fixing this for two reasons:
   1. Why not verify that the seed we wrote is actually there when we can
   2. The AWS S3 SDK started to log a bunch of WARN messages about not fully reading the stream now that we started to abuse the read blob as an `exists` check after removing that method from the blob container
2019-08-15 07:07:52 +02:00
debadair ad3d8c59bd Restructure the GS topics & add redirects. (#45527)
* Restructure the GS topics & add redirects.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/getting-started.asciidoc to fix tests
2019-08-14 16:56:31 -07:00
debadair 0f780f43b9 [DOCS] Streamlined install of 3 node cluster for GS tutorial (#45009)
* [DOCS] Revise GS intro and remove redundant conceptual content. Closes #43846.

* [DOCS] Incorporated feedback.

* [DOCS] Consolidated archive install for all platforms.

* [DOCS] Added powershell command.

* [DOCS] Added health check.
2019-08-14 16:36:59 -07:00
Benjamin Trent fde5dae387
[ML][Data Frame] adjusting change detection workflow (#45511) (#45580)
* [ML][Data Frame] adjusting change detection workflow

* adjusting for PR comment

* disallowing null as an argument value
2019-08-14 17:26:24 -05:00
Nick Knize 647a8308c3
[SPATIAL] Backport new ShapeFieldMapper and ShapeQueryBuilder to 7x (#45363)
* Introduce Spatial Plugin (#44389)

Introduce a skeleton Spatial plugin that holds new licensed features coming to 
Geo/Spatial land!

* [GEO] Refactor DeprecatedParameters in AbstractGeometryFieldMapper (#44923)

Refactor DeprecatedParameters specific to legacy geo_shape out of
AbstractGeometryFieldMapper.TypeParser#parse.

* [SPATIAL] New ShapeFieldMapper for indexing cartesian geometries (#44980)

Add a new ShapeFieldMapper to the xpack spatial module for
indexing arbitrary cartesian geometries using a new field type called shape.
The indexing approach leverages lucene's new XYShape field type which is
backed by BKD in the same manner as LatLonShape but without the WGS84
latitude longitude restrictions. The new field mapper builds on and
extends the refactoring effort in AbstractGeometryFieldMapper and accepts
shapes in either GeoJSON or WKT format (both of which support non geospatial
geometries).

Tests are provided in the ShapeFieldMapperTest class in the same manner
as GeoShapeFieldMapperTests and LegacyGeoShapeFieldMapperTests.
Documentation for how to use the new field type and what parameters are
accepted is included. The QueryBuilder for searching indexed shapes is
provided in a separate commit.

* [SPATIAL] New ShapeQueryBuilder for querying indexed cartesian geometry (#45108)

Add a new ShapeQueryBuilder to the xpack spatial module for
querying arbitrary Cartesian geometries indexed using the new shape field
type.

The query builder extends AbstractGeometryQueryBuilder and leverages the
ShapeQueryProcessor added in the previous field mapper commit.

Tests are provided in ShapeQueryTests in the same manner as
GeoShapeQueryTests and docs are updated to explain how the query works.
2019-08-14 16:35:10 -05:00
Chris Dean deab736aad
[DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552) (#45579)
* changes to chunk_size #41591

* update to chunk size to include ` `

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/modules/snapshots.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-s3.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* edits to fix passive voice
2019-08-14 15:59:36 -05:00
Mark Vieira 285f011bbb
Add build operating system as build scan tag (#45558) 2019-08-14 13:16:57 -07:00
Chris Dean caa2a7738f Revert "[DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552)"
This reverts commit 8fdbcd7395.
2019-08-14 15:14:10 -05:00
Armin Braun e0d84e7178
Clean up Callback Chains and Duplicate in SnapshotResiliencyTests (#45398) (#45563)
* It's in the title, follow up to #45233
* Flatten more listeners into `StepListener`
* Remove duplication from repo and index bootstrap and asserting that the steps execute successfully
2019-08-14 21:53:07 +02:00
Chris Dean 8fdbcd7395 [DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552)
* changes to chunk_size #41591

* update to chunk size to include ` `

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/modules/snapshots.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-s3.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* edits to fix passive voice
2019-08-14 14:15:22 -05:00
Michael Basnight fd57d3cb29 Fix test broken by policy rename 2019-08-14 13:57:47 -05:00
Mark Vieira e31212b0aa
Remove leftover debugging logic (#45556) 2019-08-14 11:52:03 -07:00
Michael Basnight 52a094b177 Fail delete policy if pipeline exists (#44438)
If a pipeline that refrences the policy exists, we should not allow the
policy to be deleted. The user will need to remove the processor from
the pipeline before deleting the policy. This commit adds a check to
ensure that the policy cannot be deleted if it is referenced by any
pipeline in the system.
2019-08-14 13:51:10 -05:00
Benjamin Trent 0c343d8443
[7.x] [ML][Transforms] adjusting stats.progress for cont. transforms (#45361) (#45551)
* [ML][Transforms] adjusting stats.progress for cont. transforms (#45361)

* [ML][Transforms] adjusting stats.progress for cont. transforms

* addressing PR comments

* rename fix

* Adjusting bwc serialization versions
2019-08-14 13:08:27 -05:00
Ryan Ernst 4af6d86c01
Rename vagrant project to os (#45509) (#45530)
The system level tests for our distributions have historically be run in
vagrant, and thus the name of the gradle project has been "vagrant".
However, as we move to running these tests in other environments (eg
GCP) the name vagrant no longer makes sense. This commit renames the
project to "os" (short for operating system), since these tests ensure
all of our distributions run correctly on our supported operating
systems.
2019-08-14 10:30:41 -07:00
James Rodewig 6904778a14 [DOCS] Rewrite `fuzzy` query docs (#42078) 2019-08-14 13:07:29 -04:00
Jim Ferenczi 79a1390935 Add mapper-extras and the RankFeatureQuery in the hlrc (#43713)
This change adds the support for the RankFeatureQuery in the HLRC by
providing an extra dependency on mapper-extras-client. It also removes
the dependency on lang-painless in mapper-extras which is not needed
anymore since the move of the vector field into a dedicated module.

Closes #43634
2019-08-14 18:41:39 +02:00
István Zoltán Szabó 8142ca82f1 [DOCS] Reformats nodes feature usage API (#45539)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-14 18:20:19 +02:00
István Zoltán Szabó 0db5af5f65 [DOCS] Reformats cluster remote info API (#45545)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-14 17:08:52 +02:00
Armin Braun 5f6bc6fc2d
Prevent Leaking Search Tasks on Exceptions in FetchSearchPhase and DfsQueryPhase (#45500) (#45540)
* If `counter.onResult` throws an exception we might leak a transport task because the failure is not handled as a phase failure (instead it bubbles up in the transport service eventually hitting the `onFailure` callback again and couting down the `counter` twice).

Co-authored-by: Jim Ferenczi <jim.ferenczi@elastic.co>
2019-08-14 14:49:38 +02:00
Przemysław Witek 2a70d38439
[7.x] HLRC for ml/data_frame/analytics/_estimate_memory_usage API (#45531) (#45537) 2019-08-14 13:45:17 +02:00
Martijn van Groningen 43b8ab607d
Improve naming of enrich policy fields. (#45494)
Renamed `enrich_key` to `match_field` and
renamed `enrich_values` to `enrich_fields`.

Relates #32789
2019-08-14 11:45:22 +02:00
Ioannis Kakavas 3eb0e59a83 Remove reference to `accept_default_password` (#45533)
`xpack.security.authc.accept_default_password` has not been
 used since 6.0 but we still referenced it in our docs.
2019-08-14 11:33:57 +03:00
Przemysław Witek df574e5168
[7.x] Implement ml/data_frame/analytics/_estimate_memory_usage API endpoint (#45188) (#45510) 2019-08-14 08:26:03 +02:00
Ryan Ernst 84bf98e9cd Fix bats distro test to build distributions (#45529)
The dependency on copying distributions was accidentally masked by an
earlier refactoring. This commit fixes the copyDistributions task to be
run before bats tests run.
2019-08-13 22:50:59 -07:00
Mark Vieira 1403a389f8
Fix remote cache misses for test tasks (#45521)
(cherry picked from commit 14486c73128feefc19d1c6a4cad4864d03ecaa9d)
2019-08-13 19:09:37 -07:00
Gordon Brown 3f5dab99c3
Properly set origin for SLM history store client (#45515)
The origin was not set properly for the SnapshotHistoryStore client,
resulting in errors when SLM was used when security was enabled.
2019-08-13 18:23:20 -06:00
Mark Vieira bba7017262
Restore check part 1 and 2 order mistakenly reversed in [#45098] (#45522) 2019-08-13 16:39:00 -07:00
Mark Vieira b17bf70e04
Fix remote cache misses for checkstyle tasks (#45512) 2019-08-13 16:03:11 -07:00
Nhat Nguyen 4fcf7bbd07 Do not hold writeLock while verifying Lucene/translog
We should not hold Engine#writeLock while executing
assertConsistentHistoryBetweenTranslogAndLuceneIndex
for this check might acquire Engine#readLock.

Relates #45461
2019-08-13 16:16:06 -04:00
Nhat Nguyen 24514275c7 Get max_seq_no after snapshot translog and Lucene (#45461)
We should capture max_seq_no after snapshotting translog and Lucene;
otherwise, that max_seq_no can be smaller some operation in translog or
Lucene. With this change, we also hold the Engine#writeLock during this
check so that no indexing can happen.

Closes #45454
2019-08-13 16:16:06 -04:00
Andrei Stefan adf8e20021
SQL: adds format parameter to range queries for constant date comparisons (#45503)
* Add format parameter to the range queries built for CURRENT_* functions
used in comparison conditions
* Use range queries for date fields equality/non-equality as well.

(cherry picked from commit c1e81e90f937ee5a002524d632bfce74d76962f9)
2019-08-13 23:04:30 +03:00
Ryan Ernst 80a3aeaef1
Split off upgrade and plugins bats tests (#45360) (#45508)
The bats tests currently require many additional artifacts to be built.
In addition to the current distributions, they need all the plugins to
be installed, as well as a randomly chosen bwc distribution. This commit
splits these two cases into their own bats task, so the dependencies do
not slow down other tasks like distroTests which do not need them.
2019-08-13 12:45:29 -07:00
Martijn van Groningen 452557cf2e
Validate policy name like an index name. (#45452)
The policy name is used to generate the enrich index name.
For this reason, a policy name should be validated in the same way
as index names.

Relates to #32789
2019-08-13 20:25:17 +02:00
Ryan Ernst 884f26a1dc
Make distro test plugin apply to the top level project (#45406) (#45481)
The distro test plugin was originally designed to be applied within each
subproject, per operating system we run in a VM with vagrant. However,
for efficiency, and also ease of having a single task to run in CI when
launching within individual OS VMs, having the "destructive" tasks in a
single place is more convenient. This commit reworks the distro test
plugin to be applied to the qa/vagrant project, which now creates only
the wrapper tasks in each of the subprojects for each vagrant VM.
2019-08-13 08:19:18 -07:00
Jack Conradson 7f550f2b29 Complete decoupling ANTLR AST from Painless AST (#45366)
This change removes the Reserved class used to track variables usages 
within the ANTLR grammar. That task is now performed by an existing pass 
"extractVariables" in the Painless AST. The Painless AST no longer has any 
dependencies on the ANTLR AST for state outside of the tree being built. 
This will simplify future refactoring and opens the possibility of alternate 
grammars.
2019-08-13 08:02:10 -07:00
Armin Braun 90803a5caf
Reenable Integ Tests in native-multi-node-tests (#45482) (#45496)
* Reenable Integ Tests in native-multi-node-tests

* The tests broken here were likely fixed by #45463 => let's reenable them and see if things run fine again
* Relates #45405, #45455
2019-08-13 15:55:54 +02:00
Mayya Sharipova 22ab389531 Clarify that FLS/DLS disable shard request cache (#45462) 2019-08-13 09:05:57 -04:00
James Rodewig 109358ca3f [DOCS] Reformat cat API titles (#45407) 2019-08-13 08:37:10 -04:00
James Rodewig df0df9402d [DOCS] Correct anchor for cat segments API examples 2019-08-13 08:30:20 -04:00
James Rodewig 2543e9eca5 [DOCS] Reformats cat shards API (#45392) 2019-08-13 08:30:14 -04:00
István Zoltán Szabó 0ff723852f [DOCS] Reformats cluster update settings API (#45337) 2019-08-13 14:24:18 +02:00
István Zoltán Szabó f4b001dc4b [DOCS] Adds heading between examples to better separate them. (#45499) 2019-08-13 13:55:15 +02:00