Commit Graph

47004 Commits

Author SHA1 Message Date
Ryan Ernst 03dd22b56c Add missing ZonedDateTime methods for joda compat layer (#44829)
While joda no longer exists in the apis for 7.x, the compatibility layer
still exists with helper methods mimicking the behavior of joda for
ZonedDateTime objects returned for date fields in scripts. This layer
was originally intended to be removed in 7.0, but is now likely to exist
for the lifetime of 7.x.

This commit adds missing methods from ChronoZonedDateTime to the compat
class. These methods were not part of joda, but are needed to act like a
real ZonedDateTime.

relates #44411
2019-07-25 11:45:57 -07:00
Lisa Cawley c9909b09b5 [DOCS] Adds command reference for elasticsearch-croneval (#43946) 2019-07-25 11:41:05 -07:00
Julie Tibshirani acb7f599a3 Fix an NPE when requesting inner hits and _source is disabled. (#44836)
This PR makes two changes to FetchSourceSubPhase when _source is disabled and
we're in a nested context:
* If no source filters are provided, return early to avoid an NPE.
* If there are source filters, make sure to throw an exception.

The behavior was chosen to match what currently happens in a non-nested context.
2019-07-25 10:38:00 -07:00
lcawl 6a60fd6d30 [DOCS] Fixes terms in HLRC data frame transform APIs (#44838) (#44865) 2019-07-25 10:08:38 -07:00
Nicholas Knize 48757da6e1 [GEO] Fix GeoShapeQueryBuilder to check for valid spatial relations
Refactor left out the spatial strategy check in GeoShapeQueryBuilder.relation
setter method. This commit adds that check back in.
2019-07-25 11:32:13 -05:00
Nick Knize 133f848e9f [Geo] Refactor GeoShapeQueryBuilder to derive from AbstractGeometryQueryBuilder (#44780)
Refactors GeoShapeQueryBuilder to derive from a new AbstractGeometryQueryBuilder that provides common parsing and build logic for spatial geometries. This will allow development of custom geometry queries by extending AbstractGeometryQueryBuilder preventing duplication of common spatial query logic.
2019-07-25 11:32:13 -05:00
Lisa Cawley a79adca7e3 [DOCS] Updates terms in anomaly detection job APIs (#44839) 2019-07-25 09:06:52 -07:00
István Zoltán Szabó 4a31c426e6 [DOCS] Adds allow no datafeeds query param to the GET, GET stats and STOP datafeed APIs (#44499) 2019-07-25 17:08:01 +02:00
Hendrik Muhs 2ca6306452 do not assert on indexer state (#44854)
remove the unreliable check for the state change

fixes #44813
2019-07-25 16:39:24 +02:00
David Roberts b2e969f4ba
[ML-DataFrame] Remove ID field from data frame indexer stats (#44848)
This is a followup to #44350. The indexer stats used to
be persisted standalone, but now are only persisted as
part of a state-and-stats document. During the review
of #44350 it was decided that we'll stick with this
design, so there will never be a need for an indexer
stats object to store its transform ID as it is stored
on the enclosing document. This PR removes the indexer
stats document ID.

Backport of #44768
2019-07-25 15:19:32 +01:00
Albert Zaharovits af937b14ae
SecurityIndexManager handle RuntimeEx while reading mapping (#44409)
Fixes exception handling while reading and parsing `.security-*`
mappings and templates.
2019-07-25 16:52:21 +03:00
Armin Braun 383d7b7713
Cleanup Dead Code in Index Creation (#44784) (#44822)
* Cleanup Dead Code in Index Creation
* This is all unused and the state of a create request is always `OPEN`
2019-07-25 10:50:04 +02:00
Przemysław Witek 53f409e5ae
Add result_type field to TimingStats and DatafeedTimingStats documents (#44812) (#44841) 2019-07-25 10:11:55 +02:00
Yannick Welsch e0d4544ef6 Close connection manager on current thread in RemoteClusterConnection (#44805)
The problem is that RemoteClusterConnection closes the connection manager asynchronously, which races with the threadpool being shutdown at the end of the test.

Closes #44339
Closes #44610
2019-07-25 09:34:41 +02:00
Tanguy Leroux 9944e193f9
[7.x] Clean up ShardFollowTasks for deleted indices (#44702) (#44790)
Deleting a follower index does not delete its ShardFollowTasks, potentially
leaving many persistent tasks in the cluster that cannot be allocated on
nodes and unnecessary fill the logs. This commit adds a cluster state listener
(ShardFollowTaskCleaner) that completes (with a failure) any persistent task
that refers to a non existent follower index.

I think that this bug has been introduced by #34404: before this change the
task would have been completed as failed and removed from the cluster state.

Backport of #44702 and #44801 on 7.x
2019-07-25 09:33:57 +02:00
Andrei Stefan fd74b63602 SQL: fix URI path being lost in case of hosted ES scenario (#44776)
(cherry picked from commit 06dea859e8fddada868941aaae15e83b4f64babe)
2019-07-25 10:27:51 +03:00
Andrei Stefan ee53f7e161 SQL: [Tests] Re-enable testDriverConfigurationWithSSLInURL test with more logging (#44800)
(cherry picked from commit 5b9ccd72e9a3bb65c8b7b06979a75cb795c17111)
2019-07-25 10:10:45 +03:00
Andrei Stefan 2633d11eb7
Switch from using docvalue_fields to extracting values from _source (#44062) (#44804)
* Switch from using docvalue_fields to extracting values from _source
where applicable. Doing this means parsing the _source and handling the
numbers parsing just like Elasticsearch is doing it when it's indexing
a document.
* This also introduces a minor limitation: aliases type of fields that
are NOT part of a tree of sub-fields will not be able to be retrieved
anymore. field_caps API doesn't shed any light into a field being an
alias or not and at _source parsing time there is no way to know if a
root field is an alias or not. Fields of the type "a.b.c.alias" can be
extracted from docvalue_fields, only if the field they point to can be
extracted from docvalue_fields. Also, not all fields in a hierarchy of
fields can be evaluated to being an alias.

(cherry picked from commit 8bf8a055e38f00df5f49c8d97f632f69d6e00c2c)
2019-07-25 10:02:41 +03:00
Igor Motov f9943a3e53 Geo: deprecate ShapeBuilder in QueryBuilders (#44715)
Removes unnecessary now timeline decompositions from shape builders
and deprecates ShapeBuilders in QueryBuilder in favor of libs/geo
shapes.

Relates to #40908
2019-07-24 14:27:58 -04:00
Ryan Ernst b3d8b39f29 Convert logging related gradle classes to java (#44771)
This commit converts the logging related classes (only used for vagrant)
to java from groovy.

relates #34459
2019-07-24 11:20:51 -07:00
Lisa Cawley 8bac13d5b2 [DOCS] Adds descriptions for put and start data frame transforms (#44724) 2019-07-24 11:12:56 -07:00
Enrico Zimuel a12be619f6 Fix URL documentation in API specs (#44487) 2019-07-24 10:33:15 -07:00
Lee Hinman fde6fcdfac Order ILM actions in policy definition documentation (#44773)
We already have a note that the order of actions is up to ILM for each
phase, this commit puts the actions in the same order as they will be
executed.

Resolves #41729
2019-07-24 11:28:38 -06:00
Gordon Brown 2ac54da60a
Fix swapped variables in error message (#44300)
The alias name and index were in the incorrect order in this error
message. This commit corrects the order.
2019-07-24 11:38:15 -04:00
David Turner 4cfd2fc6b2 Fix testFirstListElementsToCommaDelimitedStringReportsFirstElementsIfLong (#44785)
This test can fail (super-rarely) if it generates a list of length 11
containing a duplicate, because the `.distinct()` reduces the list length to 10
and then it is not abbreviated any more. This change generalises the test to
cover lists of any random length.
2019-07-24 16:10:41 +01:00
Armin Braun d8be9244f9
Fix Repository Cleanup Test Correctness (#44738) (#44751)
* The tests were creating the corruption and asserting its existence not on the repository base path but on a clean path.
As a result the consistency assertion on the repository wouldn't see the corruption ever an pass even if the cleanup was broken for repositories that have a non-root base path
2019-07-24 16:03:37 +02:00
Tanguy Leroux a8905ef142
[7.x] Add CloseIndexResponse to HLRC (#44349) (#44788)
The CloseIndexResponse was improved in #39687; this commit
exposes it in the HLRC.

Backport of #44349 to 7.x.
2019-07-24 15:51:01 +02:00
Andrei Stefan 04cb3aebd5 Use hasValue() methods from aggregations' InspectionHelpers (#44745)
Use InspectionHelper classes to decide if the aggregations should return null (in case there is no value) or the value itself.

(cherry picked from commit dafd7b039b0da072750e8f57e7572d24f7aad44a)
2019-07-24 16:14:45 +03:00
Adrien Grand 0418eb35a7
Add 7.3 release highlights. (#44153)
Add 7.3 release highlights.
2019-07-24 15:05:23 +02:00
James Rodewig ad7c164dd0 [DOCS] Rewrite `regexp` query (#42711) 2019-07-24 08:38:41 -04:00
Ioannis Kakavas bfb2e323e9
mute test (#44809)
see #44808
2019-07-24 15:00:50 +03:00
Dimitris Athanasiou 5453188cef [TEST] Mute SharedClusterSnapshotRestoreIT.testParallelRestoreOperationsFromSingleSnapshot
This was supposed to be muted in #44675 and its backports but that PR accidentally muted
another test.

Relates #44671
2019-07-24 14:28:09 +03:00
Armin Braun 4a3218551c
Fix ConnectionManagerTests (#44769) (#44789)
* In both fake connection validators we were potentially executing the listener twice. This lead to the situation that the locking via `connectionLock` that ensures that each listener is only executed once ever
would fail and the lister would run twice (in which case the listeners for that node are already `null` and we get an NPE)
* The fact that two different tests fail is due to the fact that we weren't safely shutting down the threadpool which meant the the task that trips the assertion (on the generic pool) would leak into the next test and fail it
* Closes #44758
2019-07-24 13:12:57 +02:00
Jason Tedor c329b454d9
Mark fields in SystemdPluginTests as final
These fields can be final, since they are set at construction, and
changing them after that could lead to some confusing test cases. This
commit allows the compiler to enforce that we never modify these values
during tests.
2019-07-24 17:16:50 +09:00
Przemysław Witek 26da573e94
[ML] [7.x] Only emit deprecation warning if there was actual change of a datafeed's job_id. (#44755)
* Only emit deprecation warning if there was actual change of a datafeed's job_id.
* Add @Deprecated annotation to DatafeedUpdate.Builder#setJobId method
2019-07-24 10:03:25 +02:00
Jason Tedor 58a4bad12f
Align assertion and enable check in systemd plugin
This commit more closely aligns the assertion that we are running in a
package distribution with disabling the systemd integration if somehow
we running on not a package distribution. This is, previously we had an
assertion that we are in a package distribution (RPM or Debian package)
but would disable the systemd integration if we are not on
Linux. Instead, we should disable the systemd integration if we are not
running in a package distribution. Because of our assertion, we expect
this to never hold, but we need a fallback for when this assertion is
violated and assertions are not enabled.
2019-07-24 16:34:42 +09:00
Jason Tedor 1e9c505e95
Avoid dumping the heap in Painless tests (#44782)
Well, we have a test here that intentionally causes an OutOfMemoryError,
to ensure that Painless handles it (I still strongly disagree with doing
this). This causes two things to happen: an OutOfMemoryError to be
dumped to the console, and the heap to be dumped to disk. This makes it
look like we had an OutOfMemoryError while running tests, and the tests
did not fail properly. This commit changes the tests configuration so
that we suppress the heap dump, which also causes the OutOfMemoryError
to no longer be dumped to the console.
2019-07-24 16:04:19 +09:00
Jason Tedor 4c77d5e2c7
Remove stale permissions from untrusted policy (#44783)
We have some old permissions lying around, granted to untrusted code
from the days of yore when we supported Groovy and Javascript
scripting. This commit removes these stale permissions.
2019-07-24 15:59:16 +09:00
Jason Tedor 659ebf6cfb
Notify systemd when Elasticsearch is ready (#44673)
Today our systemd service defaults to a service type of simple. This
means that systemd assumes Elasticsearch is ready as soon as the
ExecStart (bin/elasticsearch) process is forked off. This means that the
service appears ready long before it actually is, so before it is ready
to receive requests. It also means that services that want to depend on
Elasticsearch being ready to start can not as there is not a reliable
mechanism to determine this. This commit changes the service type to
notify. This requires that Elasticsearch sends a notification message
via libsystemd sd_notify method. This commit does that by using JNA to
invoke this native method. Additionally, we use this integration to also
notify systemd when we are stopping.
2019-07-24 14:04:36 +09:00
Jake Landis 27440b7692
update 7.2.1 release notes (#44774) 2019-07-23 16:33:31 -05:00
Armin Braun 818103ff1e
Fix testRetentionLeasesClearedOnRestore (#44754) (#44766)
* Fix this test randomly failing when running into async translog persistence edge case and failing to successfully close index
* Also, slightly improve debug logging on close failure
* Closes #44681
2019-07-23 21:29:07 +02:00
Igor Motov 9338fc8536 GEO: Switch to using GeoTestUtil to generate random geo shapes (#44635)
Switches to more robust way of generating random test geometries by
reusing lucene's GeoTestUtil. Removes duplicate random geometry
generators by moving them to the test framework.

Closes #37278
2019-07-23 14:30:41 -04:00
James Rodewig 9524d71159 [DOCS] Document // TEARDOWN test snippet. (#44762)
The `// TEARDOWN` test snippet was added with #34716. You can use this
snippet to end and clean up a test series started with `// TESTSETUP` or
`// TEST[setup:name]`.
2019-07-23 14:05:14 -04:00
David Roberts caf9411a72
[ML] Improve response format of data frame stats endpoint (#44743)
This change adjusts the data frame transforms stats
endpoint to return a structure that is easier to
understand.

This is a breaking change for clients of the data frame
transforms stats endpoint, but the feature is in beta so
stability is not guaranteed.

Backport of #44350
2019-07-23 18:00:50 +01:00
Benjamin Trent 6f53865fde
[ML][Data Frame] Fixes failure state tests and failure setting handling (#44645) (#44698)
* [ML][Data Frame] fixing flaky test

* adjusting frequency

* fixing tests

* addressing PR comments
2019-07-23 08:33:12 -05:00
Przemysław Witek 16c8e18013
Deprecate the ability to update datafeed's job_id. (#44691) (#44742) 2019-07-23 14:48:56 +02:00
Ioannis Kakavas 7f6e2aae82 Handle version parsing in RestClientBuilder (#44737)
Handle version parsing more leniently in RestClientBuilder for the
cases where TLS version pinning is required since we cannot use
JavaVersion here.
2019-07-23 13:50:07 +03:00
Ioannis Kakavas 4dd9238cc0 Mute testPooledMemoryChoiceOnNotSmallHeap 2019-07-23 13:16:22 +03:00
Armin Braun e5bd3ad0e9
Remove some Dead Code in o.e.transport (#44653) (#44734)
* None of this is used
2019-07-23 10:52:37 +02:00
David Turner ee23968f05 Ignore unknown fields if overriding node metadata (#44689)
The `elasticsearch-node override-version` command fails if it cannot read the
existing node metadata file. However, it reads this file strictly and fails if
there are any unknown fields, which means it will not be useful if we add
another field in future.

This commit adds leniency to this command, allowing it to ignore any unknown
fields and proceed with the downgrade. A downgrade is already unsafe, and the
user is already copiously warned about this, so being lenient in this case does
not make things much worse.
2019-07-23 08:54:58 +01:00