Commit Graph

47397 Commits

Author SHA1 Message Date
Jason Tedor 200579bfce
Simplify finding jps (#45677)
This commit simplifies how we find jps to instead use built-in Gradle
functionality for the same.
2019-08-16 18:33:46 -04:00
debadair 74de6d2c71 [DOCS] Fixed TOC heading level issue. (#45680) 2019-08-16 14:56:26 -07:00
Costin Leau 1cd58c8ea8 SQL: Break TextFormatter/Cursor dependency (#45613)
Improve the initialization and state passing of TextFormatter in CLI
and TEXT mode by leveraging the Page listener hook. Additionally
simplify the code inside RestSqlQueryAction.

(cherry picked from commit a56db2fa119cf9e8748723e19f1fc9f6a8afe5fc)
2019-08-17 00:16:08 +03:00
Costin Leau 96883dd028 SQL: Refactor away the cycle between Rowset and Cursor (#45516)
Improve encapsulation of pagination of rowsets by breaking the cycle
between cursor and associated rowset implementation, all logic now
residing inside each cursor implementation.

(cherry picked from commit be8fe0a0ce562fe732fae12a0b236b5731e4638c)
2019-08-17 00:16:05 +03:00
Gordon Brown ecb3ebd796
Clean SLM and ongoing snapshots in test framework (#45564)
Adjusts the cluster cleanup routine in ESRestTestCase to clean up SLM
test cases, and optionally wait for all snapshots to be deleted.

Waiting for all snapshots to be deleted, rather than failing if any are
in progress, is necessary for tests which use SLM policies because SLM
policies may be in the process of executing when the test ends.
2019-08-16 14:17:34 -06:00
Armin Braun c321272ae7
Mute testBiDirectionalIndexFollowing for #45641 (#45674)
* Muting #45641
2019-08-16 22:02:41 +02:00
Igor Motov 98c850c08b
Geo: Change order of parameter in Geometries to lon, lat 7.x (#45618)
Changes the order of parameters in Geometries from lat, lon to lon, lat
and moves all Geometry classes are moved to the
org.elasticsearch.geomtery package.

Backport of #45332

Closes #45048
2019-08-16 14:42:02 -04:00
Ryan Ernst 742213d710 Improve error message when index settings are not a map (#45588)
This commit adds an explicit error message when a create index request
contains a settings key that is not a json object. Prior to this change
the user would be given a ClassCastException with no explanation of what
went wrong.

closes #45126
2019-08-16 11:39:26 -07:00
Zachary Tong 50c65d05ba Move bucket reduction from Bucket to the InternalAgg (#45566)
The current idiom is to have the InternalAggregator find all the
buckets sharing the same key, put them in a list, get the first bucket
and ask that bucket to reduce all the buckets (including itself).

This a somewhat confusing workflow, and feels like the aggregator should
be reducing the buckets (since the aggregator owns the buckets), rather
than asking one bucket to do all the reductions.

This commit basically moves the `Bucket.reduce()` method to the
InternalAgg and renames it `reduceBucket()`.  It also moves the
`createBucket()` (or equivalent) method from the bucket to the
InternalAgg as well.
2019-08-16 13:59:00 -04:00
Jason Tedor 2ecd6a83f7
Clarify mixed license text (#45637)
This commit clarifies the mixed licensing in the elasticsearch
repository. A few points of note:
 - we clarify that all code is under the Apache License 2.0 unless noted
   otherwise
 - we clarify that code under the Elastic License is only in the x-pack
   directory
 - we clarify that when code is not under the Apache License 2.0 nor the
   Elastic License that it is Apache License 2.0 compatible
 - we clarify that OSS-builds do not package any code from the x-pack
   directory
2019-08-16 13:39:12 -04:00
Mark Vieira 0e375e6cd7
Disable test runner task when not running with testclusters (#45659) 2019-08-16 10:30:36 -07:00
Jack Conradson e243bbdc2a Fix Watcher Examples in Painless (#45631)
This fixes the mappings and types required to run watcher and other 
examples. A new set of seat data will be updated and available for 
download to go with this change.
2019-08-16 09:08:19 -07:00
Mark Vieira 529946aa15
Rename system property to change bwc checkout behavior (#45574) 2019-08-16 08:54:04 -07:00
Andrey Ershov dbc90653dc transport.publish_address should contain CNAME (#45626)
This commit adds CNAME reporting for transport.publish_address same way
it's done for http.publish_address.

Relates #32806
Relates #39970

(cherry picked from commit e0a2558a4c3a6b6fbfc6cd17ed34a6f6ef7b15a9)
2019-08-16 17:42:00 +02:00
Alpar Torok 59d378fc7b Toggle flag to tail cluster log on failure
The flag was toggled by accident in a refactoring.
This PR configures it so that we tail the cluster log on failure again.
2019-08-16 16:51:04 +03:00
István Zoltán Szabó cab2eeeafe [DOCS] Reformats voting configuration exclusion API. (#45644) 2019-08-16 15:42:07 +02:00
István Zoltán Szabó 6350020401 [DOCS] Reformats cluster allocation explain API (#45603)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-16 15:39:07 +02:00
István Zoltán Szabó 95bd264449 [DOCS] Reformats nodes hot_threads API (#45597)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-16 15:35:29 +02:00
Luca Cavanna c31cddf27e
Update the schema for the REST API specification (#42346)
* Update the REST API specification

This patch updates the REST API spefication in JSON files to better encode deprecated entities,
to improve specification of URL paths, and to open up the schema for future extensions.

Notably, it changes the `paths` from a list of strings to a list of objects, where each
particular object encodes all the information for this particular path: the `parts` and the `methods`.

Among the benefits of this approach is eg. encoding the difference between using the `PUT` and `POST`
methods in the Index API, to either use a specific document ID, or let Elasticsearch generate one.

Also `documentation` becomes an object that supports an `url` and also a `description` which is a
new field.

* Adapt YAML runner to new REST API specification format

The logic for choosing the path to use when running tests has been
simplified, as a consequence of the path parts being listed under each
path in the spec. The special case for create and index has been removed.

Also the parsing code has been hardened so that errors are thrown earlier
when the structure of the spec differs from what expected, and their
error messages should be more helpful.
2019-08-16 14:40:00 +02:00
Andrei Stefan 30a0711777 Remove deprecated use of "interval" method, in favor of "fixedInterval". (#45501)
(cherry picked from commit 3fef65160f9e61883e9f8f7f345b814f945e2f4b)
2019-08-16 15:03:43 +03:00
Armin Braun d6a9edea16
Lower Limit for Maximum Message Size in TcpTransport (#44496) (#45635)
* Since we're buffering network reads to the heap and then deserializing them it makes no sense to buffer a message that is 90% of the heap size since we couldn't deserialize it anyway
* I think `30%` is a more reasonable guess here given that we can reasonably assume that the deserialized message will be larger than the serialized message itself and processing it will take additional heap as well
2019-08-16 12:27:54 +02:00
István Zoltán Szabó da03a5d8d1 [DOCS] Reformats task management API (#45549)
This PR updates the task management API to align with the new API reference template.
2019-08-16 10:23:32 +02:00
Alpar Torok 4a67645e5d Use dynamic ports for ESSingleNodeTestCase too
Extends #45601 to cover all tests.
2019-08-16 09:17:19 +03:00
Mark Vieira 1f6cd5c6e1
Fix backwards compatibility with pre-Java11 build-tools users (#45633) 2019-08-15 18:39:11 -07:00
James Rodewig a635eca5f8 Retitle and relocate cross-cluster search docs (#45608) 2019-08-15 16:28:04 -04:00
Jason Tedor ec4182590f
Use bundled JDK in Sys V init (#45593)
This commit addresses an issue when trying to using Elasticsearch on
systems with Sys V init and the bundled JDK was not being used. Instead,
we were still inadvertently trying to fallback on the path. This commit
removes that fallback as that is against our intentions for 7.x where we
only support the bundled JDK or an explicit JDK via JAVA_HOME.
2019-08-15 16:15:17 -04:00
Jason Tedor 6b72d42cb8
Read git revision directly from repository (#45547)
Today we shell out to git rev-parse to read the git revision. Forking
another process is slower than reading the revision directly. This
commit changes to directly read the git revision from the repository,
avoiding to fork another process.
2019-08-15 15:27:06 -04:00
Armin Braun a48242c371
Cleanup Redundant TransportLogger Instantiation (#43265) (#45629)
* This class' methods are all effectively `static` => make them `static` and stop instantiating it needlessly
2019-08-15 21:16:56 +02:00
Zachary Tong cd441f6906 Catch AllocatedTask registration failures (#45300)
When a persistent task attempts to register an allocated task locally,
this creates the Task object and starts tracking it locally.  If there
is a failure while initializing the task, this is handled by a catch
and subsequent error handling (canceling, unregistering, etc).

But if the task fails to be created because an exception is thrown
in the tasks ctor, this is uncaught and fails the cluster update
thread.  The ramification is that a persistent task remains in the
cluster state, but is unable to create the allocated task, and the
exception prevents other tasks "after" the poisoned task from starting
too.

Because the allocated task is never created, the cancellation tools
are not able to remove the persistent task and it is stuck as a
zombie in the CS.

This commit adds exception handling around the task creation,
and attempts to notify the master if there is a failure (so the
persistent task can be removed).  Even if this notification fails,
the exception handling means the rest of the uninitialized tasks
can proceed as normal.
2019-08-15 15:14:19 -04:00
Armin Braun 73e266b2fd
Fix Failures when Closing Indices in EsBlobStoreRepositoryIntegTestCase (#45532) (#45614)
* Same issue as in #44754 as far as I can see: in case of async translog persistence we randomly fail to close
* Closes #45335 
* Closes #45334
2019-08-15 19:45:17 +02:00
Armin Braun de58353722
Lower Painless Static Memory Footprint (#45487) (#45619)
* Painless generates a ton of duplicate strings and empty `Hashmap` instances wrapped as unmodifiable
* This change brings down the static footprint of Painless on an idle node by 20MB (after running the PMC benchmark against said node)
   * Since we were looking into ways of optimizing for smaller node sizes I think this is a worthwhile optimization
2019-08-15 19:41:45 +02:00
James Rodewig d64c31e43d [DOCS] Rewrite cross-cluster seach docs (#45583) 2019-08-15 13:23:40 -04:00
James Rodewig c75fd40f2c [DOCS] Add diagrams to cross-cluster search documentation (#45569) 2019-08-15 11:00:25 -04:00
Alpar Torok 7119e54be5 Mute data frame tests on 7.x
Tracking in #45610 #45609
2019-08-15 17:07:53 +03:00
Alpar Torok 03a1645bc6 Use dynamic port ranges for ExternalTestCluster (#45601)
Moves methods added in #44213 and uses them to configure the port range
for `ExternalTestCluster` too.
These were still using `9300-9400` ( teh default ) and running into
races.
2019-08-15 16:40:12 +03:00
David Roberts d40f3718f2 [ML] Muting 5 SSLErrorMessageTests tests on Windows (#45602)
Due to https://github.com/elastic/elasticsearch/issues/45598
2019-08-15 11:05:00 +01:00
Armin Braun 1beea3588b
Make BlobStoreRepository Validation Read master.dat (#45546) (#45578)
* Fixing this for two reasons:
   1. Why not verify that the seed we wrote is actually there when we can
   2. The AWS S3 SDK started to log a bunch of WARN messages about not fully reading the stream now that we started to abuse the read blob as an `exists` check after removing that method from the blob container
2019-08-15 07:07:52 +02:00
debadair ad3d8c59bd Restructure the GS topics & add redirects. (#45527)
* Restructure the GS topics & add redirects.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/getting-started.asciidoc to fix tests
2019-08-14 16:56:31 -07:00
debadair 0f780f43b9 [DOCS] Streamlined install of 3 node cluster for GS tutorial (#45009)
* [DOCS] Revise GS intro and remove redundant conceptual content. Closes #43846.

* [DOCS] Incorporated feedback.

* [DOCS] Consolidated archive install for all platforms.

* [DOCS] Added powershell command.

* [DOCS] Added health check.
2019-08-14 16:36:59 -07:00
Benjamin Trent fde5dae387
[ML][Data Frame] adjusting change detection workflow (#45511) (#45580)
* [ML][Data Frame] adjusting change detection workflow

* adjusting for PR comment

* disallowing null as an argument value
2019-08-14 17:26:24 -05:00
Nick Knize 647a8308c3
[SPATIAL] Backport new ShapeFieldMapper and ShapeQueryBuilder to 7x (#45363)
* Introduce Spatial Plugin (#44389)

Introduce a skeleton Spatial plugin that holds new licensed features coming to 
Geo/Spatial land!

* [GEO] Refactor DeprecatedParameters in AbstractGeometryFieldMapper (#44923)

Refactor DeprecatedParameters specific to legacy geo_shape out of
AbstractGeometryFieldMapper.TypeParser#parse.

* [SPATIAL] New ShapeFieldMapper for indexing cartesian geometries (#44980)

Add a new ShapeFieldMapper to the xpack spatial module for
indexing arbitrary cartesian geometries using a new field type called shape.
The indexing approach leverages lucene's new XYShape field type which is
backed by BKD in the same manner as LatLonShape but without the WGS84
latitude longitude restrictions. The new field mapper builds on and
extends the refactoring effort in AbstractGeometryFieldMapper and accepts
shapes in either GeoJSON or WKT format (both of which support non geospatial
geometries).

Tests are provided in the ShapeFieldMapperTest class in the same manner
as GeoShapeFieldMapperTests and LegacyGeoShapeFieldMapperTests.
Documentation for how to use the new field type and what parameters are
accepted is included. The QueryBuilder for searching indexed shapes is
provided in a separate commit.

* [SPATIAL] New ShapeQueryBuilder for querying indexed cartesian geometry (#45108)

Add a new ShapeQueryBuilder to the xpack spatial module for
querying arbitrary Cartesian geometries indexed using the new shape field
type.

The query builder extends AbstractGeometryQueryBuilder and leverages the
ShapeQueryProcessor added in the previous field mapper commit.

Tests are provided in ShapeQueryTests in the same manner as
GeoShapeQueryTests and docs are updated to explain how the query works.
2019-08-14 16:35:10 -05:00
Chris Dean deab736aad
[DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552) (#45579)
* changes to chunk_size #41591

* update to chunk size to include ` `

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/modules/snapshots.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-s3.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* edits to fix passive voice
2019-08-14 15:59:36 -05:00
Mark Vieira 285f011bbb
Add build operating system as build scan tag (#45558) 2019-08-14 13:16:57 -07:00
Chris Dean caa2a7738f Revert "[DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552)"
This reverts commit 8fdbcd7395.
2019-08-14 15:14:10 -05:00
Armin Braun e0d84e7178
Clean up Callback Chains and Duplicate in SnapshotResiliencyTests (#45398) (#45563)
* It's in the title, follow up to #45233
* Flatten more listeners into `StepListener`
* Remove duplication from repo and index bootstrap and asserting that the steps execute successfully
2019-08-14 21:53:07 +02:00
Chris Dean 8fdbcd7395 [DOCS] - Updating chunk_size values to fix size value notation. Chunksize41591 (#45552)
* changes to chunk_size #41591

* update to chunk size to include ` `

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/modules/snapshots.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-azure.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/plugins/repository-s3.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* edits to fix passive voice
2019-08-14 14:15:22 -05:00
Mark Vieira e31212b0aa
Remove leftover debugging logic (#45556) 2019-08-14 11:52:03 -07:00
Benjamin Trent 0c343d8443
[7.x] [ML][Transforms] adjusting stats.progress for cont. transforms (#45361) (#45551)
* [ML][Transforms] adjusting stats.progress for cont. transforms (#45361)

* [ML][Transforms] adjusting stats.progress for cont. transforms

* addressing PR comments

* rename fix

* Adjusting bwc serialization versions
2019-08-14 13:08:27 -05:00
Ryan Ernst 4af6d86c01
Rename vagrant project to os (#45509) (#45530)
The system level tests for our distributions have historically be run in
vagrant, and thus the name of the gradle project has been "vagrant".
However, as we move to running these tests in other environments (eg
GCP) the name vagrant no longer makes sense. This commit renames the
project to "os" (short for operating system), since these tests ensure
all of our distributions run correctly on our supported operating
systems.
2019-08-14 10:30:41 -07:00
James Rodewig 6904778a14 [DOCS] Rewrite `fuzzy` query docs (#42078) 2019-08-14 13:07:29 -04:00