Commit Graph

47786 Commits

Author SHA1 Message Date
Jack Conradson e243bbdc2a Fix Watcher Examples in Painless (#45631)
This fixes the mappings and types required to run watcher and other 
examples. A new set of seat data will be updated and available for 
download to go with this change.
2019-08-16 09:08:19 -07:00
Mark Vieira 529946aa15
Rename system property to change bwc checkout behavior (#45574) 2019-08-16 08:54:04 -07:00
Andrey Ershov dbc90653dc transport.publish_address should contain CNAME (#45626)
This commit adds CNAME reporting for transport.publish_address same way
it's done for http.publish_address.

Relates #32806
Relates #39970

(cherry picked from commit e0a2558a4c3a6b6fbfc6cd17ed34a6f6ef7b15a9)
2019-08-16 17:42:00 +02:00
Alpar Torok 59d378fc7b Toggle flag to tail cluster log on failure
The flag was toggled by accident in a refactoring.
This PR configures it so that we tail the cluster log on failure again.
2019-08-16 16:51:04 +03:00
István Zoltán Szabó cab2eeeafe [DOCS] Reformats voting configuration exclusion API. (#45644) 2019-08-16 15:42:07 +02:00
István Zoltán Szabó 6350020401 [DOCS] Reformats cluster allocation explain API (#45603)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-16 15:39:07 +02:00
István Zoltán Szabó 95bd264449 [DOCS] Reformats nodes hot_threads API (#45597)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-16 15:35:29 +02:00
Luca Cavanna c31cddf27e
Update the schema for the REST API specification (#42346)
* Update the REST API specification

This patch updates the REST API spefication in JSON files to better encode deprecated entities,
to improve specification of URL paths, and to open up the schema for future extensions.

Notably, it changes the `paths` from a list of strings to a list of objects, where each
particular object encodes all the information for this particular path: the `parts` and the `methods`.

Among the benefits of this approach is eg. encoding the difference between using the `PUT` and `POST`
methods in the Index API, to either use a specific document ID, or let Elasticsearch generate one.

Also `documentation` becomes an object that supports an `url` and also a `description` which is a
new field.

* Adapt YAML runner to new REST API specification format

The logic for choosing the path to use when running tests has been
simplified, as a consequence of the path parts being listed under each
path in the spec. The special case for create and index has been removed.

Also the parsing code has been hardened so that errors are thrown earlier
when the structure of the spec differs from what expected, and their
error messages should be more helpful.
2019-08-16 14:40:00 +02:00
Andrei Stefan 30a0711777 Remove deprecated use of "interval" method, in favor of "fixedInterval". (#45501)
(cherry picked from commit 3fef65160f9e61883e9f8f7f345b814f945e2f4b)
2019-08-16 15:03:43 +03:00
Armin Braun d6a9edea16
Lower Limit for Maximum Message Size in TcpTransport (#44496) (#45635)
* Since we're buffering network reads to the heap and then deserializing them it makes no sense to buffer a message that is 90% of the heap size since we couldn't deserialize it anyway
* I think `30%` is a more reasonable guess here given that we can reasonably assume that the deserialized message will be larger than the serialized message itself and processing it will take additional heap as well
2019-08-16 12:27:54 +02:00
István Zoltán Szabó da03a5d8d1 [DOCS] Reformats task management API (#45549)
This PR updates the task management API to align with the new API reference template.
2019-08-16 10:23:32 +02:00
Alpar Torok 4a67645e5d Use dynamic ports for ESSingleNodeTestCase too
Extends #45601 to cover all tests.
2019-08-16 09:17:19 +03:00
Mark Vieira 1f6cd5c6e1
Fix backwards compatibility with pre-Java11 build-tools users (#45633) 2019-08-15 18:39:11 -07:00
James Rodewig a635eca5f8 Retitle and relocate cross-cluster search docs (#45608) 2019-08-15 16:28:04 -04:00
Jason Tedor ec4182590f
Use bundled JDK in Sys V init (#45593)
This commit addresses an issue when trying to using Elasticsearch on
systems with Sys V init and the bundled JDK was not being used. Instead,
we were still inadvertently trying to fallback on the path. This commit
removes that fallback as that is against our intentions for 7.x where we
only support the bundled JDK or an explicit JDK via JAVA_HOME.
2019-08-15 16:15:17 -04:00
Jason Tedor 6b72d42cb8
Read git revision directly from repository (#45547)
Today we shell out to git rev-parse to read the git revision. Forking
another process is slower than reading the revision directly. This
commit changes to directly read the git revision from the repository,
avoiding to fork another process.
2019-08-15 15:27:06 -04:00
Armin Braun a48242c371
Cleanup Redundant TransportLogger Instantiation (#43265) (#45629)
* This class' methods are all effectively `static` => make them `static` and stop instantiating it needlessly
2019-08-15 21:16:56 +02:00
Zachary Tong cd441f6906 Catch AllocatedTask registration failures (#45300)
When a persistent task attempts to register an allocated task locally,
this creates the Task object and starts tracking it locally.  If there
is a failure while initializing the task, this is handled by a catch
and subsequent error handling (canceling, unregistering, etc).

But if the task fails to be created because an exception is thrown
in the tasks ctor, this is uncaught and fails the cluster update
thread.  The ramification is that a persistent task remains in the
cluster state, but is unable to create the allocated task, and the
exception prevents other tasks "after" the poisoned task from starting
too.

Because the allocated task is never created, the cancellation tools
are not able to remove the persistent task and it is stuck as a
zombie in the CS.

This commit adds exception handling around the task creation,
and attempts to notify the master if there is a failure (so the
persistent task can be removed).  Even if this notification fails,
the exception handling means the rest of the uninitialized tasks
can proceed as normal.
2019-08-15 15:14:19 -04:00
Armin Braun 73e266b2fd
Fix Failures when Closing Indices in EsBlobStoreRepositoryIntegTestCase (#45532) (#45614)
* Same issue as in #44754 as far as I can see: in case of async translog persistence we randomly fail to close
* Closes #45335 
* Closes #45334
2019-08-15 19:45:17 +02:00
Armin Braun de58353722
Lower Painless Static Memory Footprint (#45487) (#45619)
* Painless generates a ton of duplicate strings and empty `Hashmap` instances wrapped as unmodifiable
* This change brings down the static footprint of Painless on an idle node by 20MB (after running the PMC benchmark against said node)
   * Since we were looking into ways of optimizing for smaller node sizes I think this is a worthwhile optimization
2019-08-15 19:41:45 +02:00
James Rodewig d64c31e43d [DOCS] Rewrite cross-cluster seach docs (#45583) 2019-08-15 13:23:40 -04:00
James Rodewig c75fd40f2c [DOCS] Add diagrams to cross-cluster search documentation (#45569) 2019-08-15 11:00:25 -04:00
Alpar Torok 7119e54be5 Mute data frame tests on 7.x
Tracking in #45610 #45609
2019-08-15 17:07:53 +03:00
Alpar Torok 03a1645bc6 Use dynamic port ranges for ExternalTestCluster (#45601)
Moves methods added in #44213 and uses them to configure the port range
for `ExternalTestCluster` too.
These were still using `9300-9400` ( teh default ) and running into
races.
2019-08-15 16:40:12 +03:00
David Roberts d40f3718f2 [ML] Muting 5 SSLErrorMessageTests tests on Windows (#45602)
Due to https://github.com/elastic/elasticsearch/issues/45598
2019-08-15 11:05:00 +01:00
Armin Braun 1beea3588b
Make BlobStoreRepository Validation Read master.dat (#45546) (#45578)
* Fixing this for two reasons:
   1. Why not verify that the seed we wrote is actually there when we can
   2. The AWS S3 SDK started to log a bunch of WARN messages about not fully reading the stream now that we started to abuse the read blob as an `exists` check after removing that method from the blob container
2019-08-15 07:07:52 +02:00
debadair ad3d8c59bd Restructure the GS topics & add redirects. (#45527)
* Restructure the GS topics & add redirects.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/getting-started.asciidoc to fix tests
2019-08-14 16:56:31 -07:00
debadair 0f780f43b9 [DOCS] Streamlined install of 3 node cluster for GS tutorial (#45009)
* [DOCS] Revise GS intro and remove redundant conceptual content. Closes #43846.

* [DOCS] Incorporated feedback.

* [DOCS] Consolidated archive install for all platforms.

* [DOCS] Added powershell command.

* [DOCS] Added health check.
2019-08-14 16:36:59 -07:00
Benjamin Trent fde5dae387
[ML][Data Frame] adjusting change detection workflow (#45511) (#45580)
* [ML][Data Frame] adjusting change detection workflow

* adjusting for PR comment

* disallowing null as an argument value
2019-08-14 17:26:24 -05:00
Nick Knize 647a8308c3
[SPATIAL] Backport new ShapeFieldMapper and ShapeQueryBuilder to 7x (#45363)
* Introduce Spatial Plugin (#44389)

Introduce a skeleton Spatial plugin that holds new licensed features coming to 
Geo/Spatial land!

* [GEO] Refactor DeprecatedParameters in AbstractGeometryFieldMapper (#44923)

Refactor DeprecatedParameters specific to legacy geo_shape out of
AbstractGeometryFieldMapper.TypeParser#parse.

* [SPATIAL] New ShapeFieldMapper for indexing cartesian geometries (#44980)

Add a new ShapeFieldMapper to the xpack spatial module for
indexing arbitrary cartesian geometries using a new field type called shape.
The indexing approach leverages lucene's new XYShape field type which is
backed by BKD in the same manner as LatLonShape but without the WGS84
latitude longitude restrictions. The new field mapper builds on and
extends the refactoring effort in AbstractGeometryFieldMapper and accepts
shapes in either GeoJSON or WKT format (both of which support non geospatial
geometries).

Tests are provided in the ShapeFieldMapperTest class in the same manner
as GeoShapeFieldMapperTests and LegacyGeoShapeFieldMapperTests.
Documentation for how to use the new field type and what parameters are
accepted is included. The QueryBuilder for searching indexed shapes is
provided in a separate commit.

* [SPATIAL] New ShapeQueryBuilder for querying indexed cartesian geometry (#45108)

Add a new ShapeQueryBuilder to the xpack spatial module for
querying arbitrary Cartesian geometries indexed using the new shape field
type.

The query builder extends AbstractGeometryQueryBuilder and leverages the
ShapeQueryProcessor added in the previous field mapper commit.

Tests are provided in ShapeQueryTests in the same manner as
GeoShapeQueryTests and docs are updated to explain how the query works.
2019-08-14 16:35:10 -05:00
Chris Dean deab736aad
[DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552) (#45579)
* changes to chunk_size #41591

* update to chunk size to include ` `

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/modules/snapshots.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-s3.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* edits to fix passive voice
2019-08-14 15:59:36 -05:00
Mark Vieira 285f011bbb
Add build operating system as build scan tag (#45558) 2019-08-14 13:16:57 -07:00
Chris Dean caa2a7738f Revert "[DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552)"
This reverts commit 8fdbcd7395.
2019-08-14 15:14:10 -05:00
Armin Braun e0d84e7178
Clean up Callback Chains and Duplicate in SnapshotResiliencyTests (#45398) (#45563)
* It's in the title, follow up to #45233
* Flatten more listeners into `StepListener`
* Remove duplication from repo and index bootstrap and asserting that the steps execute successfully
2019-08-14 21:53:07 +02:00
Chris Dean 8fdbcd7395 [DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552)
* changes to chunk_size #41591

* update to chunk size to include ` `

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/modules/snapshots.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-s3.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* edits to fix passive voice
2019-08-14 14:15:22 -05:00
Mark Vieira e31212b0aa
Remove leftover debugging logic (#45556) 2019-08-14 11:52:03 -07:00
Benjamin Trent 0c343d8443
[7.x] [ML][Transforms] adjusting stats.progress for cont. transforms (#45361) (#45551)
* [ML][Transforms] adjusting stats.progress for cont. transforms (#45361)

* [ML][Transforms] adjusting stats.progress for cont. transforms

* addressing PR comments

* rename fix

* Adjusting bwc serialization versions
2019-08-14 13:08:27 -05:00
Ryan Ernst 4af6d86c01
Rename vagrant project to os (#45509) (#45530)
The system level tests for our distributions have historically be run in
vagrant, and thus the name of the gradle project has been "vagrant".
However, as we move to running these tests in other environments (eg
GCP) the name vagrant no longer makes sense. This commit renames the
project to "os" (short for operating system), since these tests ensure
all of our distributions run correctly on our supported operating
systems.
2019-08-14 10:30:41 -07:00
James Rodewig 6904778a14 [DOCS] Rewrite `fuzzy` query docs (#42078) 2019-08-14 13:07:29 -04:00
Jim Ferenczi 79a1390935 Add mapper-extras and the RankFeatureQuery in the hlrc (#43713)
This change adds the support for the RankFeatureQuery in the HLRC by
providing an extra dependency on mapper-extras-client. It also removes
the dependency on lang-painless in mapper-extras which is not needed
anymore since the move of the vector field into a dedicated module.

Closes #43634
2019-08-14 18:41:39 +02:00
István Zoltán Szabó 8142ca82f1 [DOCS] Reformats nodes feature usage API (#45539)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-14 18:20:19 +02:00
István Zoltán Szabó 0db5af5f65 [DOCS] Reformats cluster remote info API (#45545)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-14 17:08:52 +02:00
Armin Braun 5f6bc6fc2d
Prevent Leaking Search Tasks on Exceptions in FetchSearchPhase and DfsQueryPhase (#45500) (#45540)
* If `counter.onResult` throws an exception we might leak a transport task because the failure is not handled as a phase failure (instead it bubbles up in the transport service eventually hitting the `onFailure` callback again and couting down the `counter` twice).

Co-authored-by: Jim Ferenczi <jim.ferenczi@elastic.co>
2019-08-14 14:49:38 +02:00
Przemysław Witek 2a70d38439
[7.x] HLRC for ml/data_frame/analytics/_estimate_memory_usage API (#45531) (#45537) 2019-08-14 13:45:17 +02:00
Ioannis Kakavas 3eb0e59a83 Remove reference to `accept_default_password` (#45533)
`xpack.security.authc.accept_default_password` has not been
 used since 6.0 but we still referenced it in our docs.
2019-08-14 11:33:57 +03:00
Przemysław Witek df574e5168
[7.x] Implement ml/data_frame/analytics/_estimate_memory_usage API endpoint (#45188) (#45510) 2019-08-14 08:26:03 +02:00
Ryan Ernst 84bf98e9cd Fix bats distro test to build distributions (#45529)
The dependency on copying distributions was accidentally masked by an
earlier refactoring. This commit fixes the copyDistributions task to be
run before bats tests run.
2019-08-13 22:50:59 -07:00
Mark Vieira 1403a389f8
Fix remote cache misses for test tasks (#45521)
(cherry picked from commit 14486c73128feefc19d1c6a4cad4864d03ecaa9d)
2019-08-13 19:09:37 -07:00
Gordon Brown 3f5dab99c3
Properly set origin for SLM history store client (#45515)
The origin was not set properly for the SnapshotHistoryStore client,
resulting in errors when SLM was used when security was enabled.
2019-08-13 18:23:20 -06:00
Mark Vieira bba7017262
Restore check part 1 and 2 order mistakenly reversed in [#45098] (#45522) 2019-08-13 16:39:00 -07:00