Commit Graph

47705 Commits

Author SHA1 Message Date
Armin Braun 2b85dcb201
Parallelize Repository Cleanup Actions (#46647) (#46714)
* Parallelize Repository Cleanup Actions

Deleting root blobs and unreferenced indices can safely happen in parallel,
no need to have both operations run sequentially when they preclude all other
repository operations.
2019-09-16 07:52:03 +02:00
Dimitris Athanasiou 63eb0d9081
[7.x][ML] Avoid marking data frame analytics task completed twice (#46721) (#46724)
When the stop API is called while the task is running there is
a chance the task gets marked completed twice. This may cause
undesired side effects, like indexing the progress document a second
time after the stop API has returned (the cause for #46705).

This commit adds a check that the task has not been completed before
proceeding to mark it so. In addition, when we update the task's state
we could get some warnings that the task was missing if the stop API
has been called in the meantime. We now check the errors are
`ResourceNotFoundException` and ignore them if so.

Closes #46705

Backports #46721
2019-09-15 17:25:26 +03:00
Hendrik Muhs e1842c0e5a
[7.x][Transforms] backport BWC tests for transforms crud (#46452)
backport 8.0 transform tests to 7.x
2019-09-14 13:06:48 +02:00
Mark Vieira 9774377959
Log actual checkout hash instead of generic refspec (#46637) 2019-09-13 16:07:39 -07:00
David Turner 272b0ecbdd Remove docs for proxy mode (#46677)
We added docs for proxy mode in #40281 but on reflection we should not be
documenting this setting since it does not play well with all proxies and we
can't recommend its use. This commit removes those docs and expands its Javadoc
instead.
2019-09-13 22:20:11 +01:00
Lisa Cawley c0a16047fa [DOCS] Updates links to reporting content (#46717) 2019-09-13 11:40:07 -07:00
James Rodewig 2831535cf9 [DOCS] Replace "// CONSOLE" comments with [source,console] (#46679) 2019-09-13 11:44:54 -04:00
James Rodewig 0def429bc1 [DOCS] Correct anchors and title for Update index settings API docs (#46707) 2019-09-13 09:45:52 -04:00
Nhat Nguyen cabff5a7cd Handle lower retaining seqno retention lease error (#46420)
We renew the CCR retention lease at a fixed interval, therefore it's
possible to have more than one in-flight renewal requests at the same
time. If requests arrive out of order, then the assertion is violated.

Closes #46416
Closes #46013
2019-09-13 08:50:19 -04:00
Dimitris Athanasiou 0bc8acaf5b
[7.x][ML] Create state index and alias before starting an analytics job (#46602) (#46648)
This is fixing a bug where if an analytics job is started before any
anomaly detection job is opened, we create an index after the state
write alias.

Instead, we should create the state index and alias before starting
an analytics job and this commit makes sure this is the case.

Backport of #46602
2019-09-13 10:34:12 +03:00
Lisa Cawley dae5b22bf8 [DOCS] Fixes link to Kibana security (#46690) 2019-09-12 16:30:43 -07:00
Nhat Nguyen e1a33c6283 Fix false positive out of sync warning in synced-flush (#46576)
Synced-flush consists of three steps: (1) force-flush on every active
copy; (2) check for ongoing indexing operations; (3) seal copies if
there's no change since step 1. If some indexing operations are
completed on the primary but not replicas, then Lucene commits from step
1 on replicas won't be the same as the primary's. And step 2 would pass
if it's executed when all pending operations are done. Once step 2
passes, we will incorrectly emit the "out of sync" warning message
although nothing wrong here.

Relates #28464
Relates #30244
2019-09-12 16:34:33 -04:00
Nhat Nguyen 5465c8d095 Increase timeout for relocation tests (#46554)
There's nothing wrong in the logs from these failures. I think 30
seconds might not be enough to relocate shards with many documents as CI
is quite slow. This change increases the timeout to 60 seconds for these
relocation tests. It also dumps the hot threads in case of timed out.

Closes #46526
Closes #46439
2019-09-12 16:34:01 -04:00
Przemysław Witek 5b1f6669ff
Do not wait for the old notifications index (".ml-notifications"). It is no longer used. (#46657) (#46666) 2019-09-12 21:47:25 +02:00
Zachary Tong e1e06c2589 Add version constant for 7.3.3 2019-09-12 13:50:40 -04:00
James Rodewig af7aba18d4 Fixed sample code for minhash (#46385)
The sample code is wrong. Field type is required for the sample field.
I guess the intention was to give the sample field the name ```fingerprint```, mapping it as ```text``` using the custom analyzer ```my_analyzer```
2019-09-12 13:29:44 -04:00
James Rodewig 3b26657335 [DOCS] Remove coming tags for 7.3.2 release (#46676) 2019-09-12 13:26:11 -04:00
Alpar Torok 9229b87bd2 Upgrade to Gradle 5.6 (#45005)
* Upgrade to Gradle 5.6.2
2019-09-12 16:18:41 +03:00
Jim Ferenczi 4407f3af1b Delay the creation of SubSearchContext to the FetchSubPhase (#46598)
This change delays the creation of the SubSearchContext for nested and parent/child inner_hits
to the fetch sub phase in order to ensure that a SearchContext can built entirely from a
QueryShardContext. This commit also adds a validation step to the inner hits builder that ensures that we fail the request early if the inner hits path is invalid.

Relates #46523
2019-09-12 14:52:15 +02:00
James Rodewig 60db630abd
[DOCS] Add missing mention of current version to snapshot docs (#46516) (#46658) 2019-09-12 08:47:22 -04:00
David Turner 5c85b0998b
Clarify that discovery ignores master-ineligibles (#44835)
The changes in #32006 mean that the discovery process can no longer use
master-ineligible nodes as a stepping-stone between master-eligible nodes.
This was normally an indication of a strange and possibly-fragile configuration
and was not recommended, but this commit adds a note to the breaking changes
docs to note that this kind of configuration is more obviously broken in recent
versions.
2019-09-12 11:07:34 +01:00
Alpar Torok 9f2c5632fe Add support for tests.jvm.argline in testclusters (#46540)
We have CI jobs that make use of this for testing alternative garbadge
collectors.
2019-09-12 12:24:10 +03:00
Luca Cavanna e57756492a Update http-core and http-client dependencies (#46549)
Relates to #45808
Closes #45577
2019-09-12 09:45:29 +02:00
Luca Cavanna de47ea2cf4 Add SearchRestCancellationIT
This test verifies automatic cancellation of search requests on connection close.
It was previously not present in 7.x as the http client was subject do a bug which
made testing cancellation of requests impossible. Now that the bug is fixed upstream,
we can also backport this test
2019-09-12 09:45:29 +02:00
Jilles van Gurp 030d43a76a Add a few notes on Cancellable to the LLRC and HLRC docs. (#45912)
Add a section to both the low level and high level client documentation on asynchronous usage and `Cancellable` added for #44802 

Co-Authored-By: Lee Hinman <dakrone@users.noreply.github.com>
2019-09-12 09:45:29 +02:00
Jilles van Gurp 60f40d7638 Expose the ability to cancel async requests in REST high-level client (#45688)
This commits makes all the async methods in the high level client return the `Cancellable` object that the low level client now exposes.

Relates to #45379 
Closes #44802
2019-09-12 09:45:29 +02:00
Luca Cavanna cfb186afaf Add support for cancelling async requests in low-level REST client (#45379)
The low-level REST client exposes a `performRequestAsync` method that
allows to send async requests, but today it does not expose the ability
to cancel such requests. That is something that the underlying apache
async http client supports, and it makes sense for us to expose.

This commit adds a return value to the `performRequestAsync` method,
which is backwards compatible. A `Cancellable` object gets returned,
which exposes a `cancel` public method. When calling `cancel`, the
on-going request associated with the returned `Cancellable` instance
will be cancelled by calling its `abort` method. This works throughout
multiple retries, though some special care was needed for the case where
`cancel` is called between different attempts (when one attempt has
failed and the consecutive one has not been sent yet).

Note that cancelling a request on the client side does not automatically 
translate to cancelling the server side execution of it. That needs to be 
specifically implemented, which is on the work for the search API (see #43332).

Relates to #44802
2019-09-12 09:45:28 +02:00
Henning Andersen 1c706656c2 Wait for events in translog stats tests (#46586)
When waiting for no initializing shards we also have to wait for events
when we have more than one node in the cluster. When the primary is
started, there is a short period of time, where neither the primary nor
any of the replicas are initializing.

Closes #46535
2019-09-12 09:43:41 +02:00
Lisa Cawley ec5592ed76 [DOCS] Adds missing icons to Watcher HLRC APIs (#46626) 2019-09-11 16:35:15 -07:00
Lisa Cawley 4e371a220b [DOCS] Adds missing icons to ILM HLRC APIs (#46633) 2019-09-11 15:46:19 -07:00
Lisa Cawley c3b6e6fe96 [DOCS] Adds missing icons to CCR HLRC APIs (#46631) 2019-09-11 15:37:20 -07:00
Lisa Cawley 8200010e0a [DOCS] Adds missing icons to Graph HLRC APIs (#46630) 2019-09-11 15:22:10 -07:00
Lisa Cawley 769f42bdd7 [DOCS] Add missing icons to security HLRC APIs (#46619) 2019-09-11 15:05:31 -07:00
Igor Motov 35cb93248d Geo: fix indexing of west to east linestrings crossing the antimeridian (#46601)
Fixes that way linestrings that are crossing the antimeridian are
indexed due to a normalization bug these lines were decomposed into
a line segment that was stretching entire globe.

Fixes #43775
2019-09-11 17:43:17 -04:00
Zachary Tong 6dc8ed5d57
[7.x Backport] Refactor AllocatedPersistentTask#init(), move rollup ctor logic (#46406)
This makes the AllocatedPersistentTask#init() method protected so that
implementing classes can perform their initialization logic there,
instead of the constructor.  Rollup's task is adjusted to use this
init method.

It also slightly refactors the methods to se a static logger in the
AllocatedTask instead of passing it in via an argument.  This is
simpler, logged messages come from the task instead of the
service, and is easier for tests
2019-09-11 17:00:28 -04:00
Jason Tedor ffeeb41066
Fix muting of translog stats on closed indices test
We do not allow multiple skip sections, so this commit comments out the
existing skip section.

Relates #46535
2019-09-11 16:30:43 -04:00
Alpar Torok 27da0ea603
Mute translog stats on closed indices REST test
This commit mutes a test of the translog stats on a closed indices with
soft-deletes enabled test. This test failure is being tracked in #46535.
2019-09-11 16:04:58 -04:00
James Rodewig f9bf10f2b6
[DOCS] Change "a SSL" to "an SSL" in the Java docs (#46524) (#46618) 2019-09-11 15:55:57 -04:00
Jason Tedor 941c215d52
Favor java.vendor.version over java.vendor (#46615)
In some cases (for example some AdoptOpenJDK builds), the java.vendor is
mistakenly populated as "Oracle Corporation" while the real value is
under "java.vendor.version". Since "java.vendor.version" is mandatory
since JDK 10, this commit changes to use "java.vendor.version" as the
favored system property to find the JVM vendor, and we fallback to
"java.vendor" if this is not populated (as happens in some Oracle
builds). Ugh.
2019-09-11 15:20:46 -04:00
Lisa Cawley e609011754 [DOCS] Add missing icons to rollup HLRC APIs (#46617) 2019-09-11 11:50:22 -07:00
Marios Trivyzas d956509394 SQL: Implement DATE_TRUNC function (#46473)
DATE_TRUNC(<truncate field>, <date/datetime>) is a function that allows
the user to truncate a timestamp to the specified field by zeroing out
the rest of the fields. The function is implemented according to the
spec from PostgreSQL: https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC

Closes: #46319
(cherry picked from commit b37e96712db1aace09f17b574eb02ff6b942a297)
2019-09-11 21:41:02 +03:00
Lisa Cawley fa54de7789 [DOCS] Add missing icons to transform HLRC APIs (#46616) 2019-09-11 11:22:58 -07:00
Ryan Ernst 86290cb3d9 Make reuse of sql test code explicit (#45884)
The sql project uses a common set of security tests, which are run in
subprojects. Currently these are shared through a shared directory, but
this is not setup correctly to ensure it is built before tests run. This
commit changes the test classes to be an artifact of the sql/qa/security
project and makes the test runner use the built artifact (a directory of
classes) for tests.

closes #45866
2019-09-11 10:56:07 -07:00
Ryan Ernst fa9327cdb9 Add more meaningful keystore version mismatch errors (#46291)
This commit changes the version bounds of keystore reading to give
better error messages when a user has a too new or too old format.

relates #44624
2019-09-11 09:55:19 -07:00
Thibault Richard 44c4412406 [DOCS] Fix backquote in the list of realm types (#46530) 2019-09-11 12:54:46 -04:00
Lee Hinman 09a9cefaa0 Handle partial failure retrieving segments in SegmentCountStep (#46556)
Since the `IndicesSegmentsRequest` scatters to all shards for the index,
it's possible that some of the shards may fail. This adds failure
handling and logging (since this is a best-effort step in the first
place) for this case.
2019-09-11 10:29:31 -06:00
Marios Trivyzas 0963e78164 SQL: Fix issue with common type resolution (#46565)
Many scalar functions try to find out the common type between their
arguments in order to set it as their return time, e.g.:
for `float + double` the common type which is set as the return type
of the + operation is `double`.

Previously, for data types TEXT and KEYWORD (string data types) there
was no common data type found and null was returned causing NPEs when
the function was trying to resolve the return data type.

Fixes: #46551
(cherry picked from commit 291017d69dfc810707c3c7c692f5a50af431b790)
2019-09-11 19:10:15 +03:00
James Rodewig 043471c643
[DOCS] Minor improvement to the nested aggregation docs (#46475) (#46604)
* Minor improvement to the nested aggregation docs

* The attributes name and resellers.name were rather confusing,
  especially since the first one was dynamically mapped and not shown
  in the documentation (you had to read the test to see it). This
  change introduces a unique name for the nested attribute and adds
  the example document to the documentation.
* Change the index name from "index" to something more speaking.

* Update docs/reference/aggregations/bucket/nested-aggregation.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/aggregations/bucket/nested-aggregation.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/aggregations/bucket/nested-aggregation.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-11 12:06:42 -04:00
Lee Hinman dcdfc382c7 Fix ILM start and stop doc tests (#46559)
This uses whatever the server retrieves, rather than hardcoded
"STOPPING" and "STOPPED" since the server may go to STOPPED before the
request is issued.

Resolves #46528
2019-09-11 09:54:41 -06:00
Lee Hinman 52d7b03b49 Wait for no snapshots in state in testRetentionWhileSnapshotIn… (#46573)
This commit adds a wait/check for all running snapshots to be cleared
before taking another snapshot. The previous snapshot was successful but
had not yet been cleared from the cluster state, so the second snapshot
failed due to a `ConcurrentSnapshotException`.

Resolves #46508
2019-09-11 09:47:01 -06:00