Commit Graph

46860 Commits

Author SHA1 Message Date
Armin Braun f028ab43ad
Don't Swallow Interrupt in TransportService#onRequestReceived (#44622) (#44627)
* We shouldn't just swallow the interrupt here quietly and keep going on the IO thread
   * Currently interrupt continues here just the same way an invocation of `acceptIncomingRequests` woudl have made things continue
* Relates #44610
2019-07-19 20:35:29 +02:00
Christoph Büscher eafe54c81c Fix AnalysisMode propagation in NamedAnalyzer (#44626)
NamedAnalyzer should return the same AnalysisMode than any custom analyzer it
wraps, otherwise AnalysisMode.ALL. This used to be only CustomAnalyzer in the
past, but with the introduction of the ReloadableCustomAnalyzer this needs to be
added as an option where the analysis mode gets propagated.

Closes #44625
2019-07-19 18:18:43 +02:00
David Turner c3b6ab4dec Fix docs line length
Followup to #44617 that fixes some unexpected wrapping.
2019-07-19 17:04:42 +01:00
David Turner 87f246f6bd Clarify repository-s3 defaulting to instance role (#44617)
Specifies more clearly that you can leave the client credential settings unset
if you want to use the instance or container role.
2019-07-19 16:11:23 +01:00
James Rodewig 1f5a2ac3ad [DOCS] Reformat `parent_id` query docs (#44449) 2019-07-19 11:03:45 -04:00
Nikita Glashenko 804476c35d Remove support for old translog checkpoint formats (#44280)
This commit removes support for the translog checkpoint format from versions
before 6.0.0 since 7.x versions are incompatible with indices from these
versions.

Relates #44720
Fixes #44210
2019-07-19 16:01:47 +01:00
James Rodewig 51aefbdd3d [DOCS] Reformat `has_parent` query docs (#44443) 2019-07-19 10:51:11 -04:00
James Rodewig d46545f729 [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
James Rodewig 76c7e3a05f [DOCS] Replace `_meta` with `metadata` for snapshot APIs. (#44596)
elastic/elasticsearch#41281 added custom metadata parameter to
snapshots. During review, the parameter name was changed from '_meta' to
'metadata,' but the documentation wasn't updated. This corrects the
documentation to use the 'metadata' name.
2019-07-19 08:40:57 -04:00
Przemyslaw Gomulka 597d2dfaf5
Add types field to slow logs in 7.x (#44592)
By mistake in 7.x types field was removed from slow logs. Types are
still present in that version, so this have to be present as a JSON
field
relates #41354
backport that was causing this #44178
2019-07-19 08:31:00 +02:00
Ryan Ernst 60785a9fa8
Convert several direct uses of Streamable to Writeable (#44586) (#44604)
This commit converts several utility classes that implement Streamable
to have StreamInput constructors. It also adds a default version of
readFrom to Streamable so that overriding to throw UOE is not necessary.

relates #34389
2019-07-18 21:25:44 -07:00
Julie Tibshirani 336364fefe
Convert more classes in 'server' to Writeable. (#44600)
* Convert GetTask*.
* Convert RemoteInfo*.
* Convert GetFieldMappings*.
* Convert ValidateQueryRequest*.
* Convert MainResponse*.
* Convert MultiGet*.
* Convert Update*.
* Add a missing call to parent constructors.

Relates to #34389.
2019-07-18 18:45:10 -07:00
Ryan Ernst 13f46aa801
Convert index and persistent actions/response to writeable (#44582) (#44601)
This commit converts several more classes from streamable to writeable
in server, mostly within the o.e.index and o.e.persistent packages.

relates #34389
2019-07-18 18:32:09 -07:00
Tal Levy 03f5084ac7
remove usages of #readOptionalStreamable, #readStreamableList. (#44578) (#44598)
This commit removes references to Streamable from StreamInput.

This is all a part of the effort to remove Streamable usage.

relates #34389.
2019-07-18 16:19:02 -07:00
Lisa Cawley 5a05bd76b4 [DOCS] Reformat info and deprecation APIs to use new API format (#44599) 2019-07-18 16:11:48 -07:00
Lee Hinman 3001f7941f Allow empty configuration for SLM policies (#44465)
* Allow empty configuration for SLM policies

When putting or updating a snapshot lifecycle policy it was not possible
to elide the `config` map. This commit makes the configuration optional,
the same way that it is when taking a snapshot.

Relates to #38461

* Add Objects.requireNonNull for required parts of the policy
2019-07-18 16:20:31 -06:00
Lee Hinman fe2ef66e45 Expose index age in ILM explain output (#44457)
* Expose index age in ILM explain output

This adds the index's age to the ILM explain output, for example:

```
{
  "indices" : {
    "ilm-000001" : {
      "index" : "ilm-000001",
      "managed" : true,
      "policy" : "full-lifecycle",
      "lifecycle_date" : "2019-07-16T19:48:22.294Z",
      "lifecycle_date_millis" : 1563306502294,
      "age" : "1.34m",
      "phase" : "hot",
      "phase_time" : "2019-07-16T19:48:22.487Z",
      ... etc ...
    }
  }
}
```

This age can be used to tell when ILM will transition the index to the
next phase, based on that phase's `min_age`.

Resolves #38988

* Expose age in getters and in HLRC
2019-07-18 15:33:45 -06:00
Ryan Ernst af093a4095
Convert ShardOperationFailedException to Writeable (#44532) (#44580)
This commit converts subclasses of ShardOperationFailedException to
implement ctors with StreamInput instead of readFrom. It also simplifies
IndicesShardStoresResponse.Failure to serialize its shardId after the
super data.

relates #34389
2019-07-18 13:29:19 -07:00
Benjamin Trent 3477f5ae04
muting test testBulkIndexFailuresCauseTaskToFail (#44594) 2019-07-18 15:03:50 -05:00
Benjamin Trent d5ca72740e
[ML][Data Frame] adjust onFinish audit frequency (#44450) (#44508) 2019-07-18 14:28:34 -05:00
Marios Trivyzas 8745d38dc8
Compile subset of build-tools classes with Java 8 target compatibility (#44560)
Backport of #43177 so that VersionProperties is Java 8 compatible and
can be used by https://github.com/elastic/elasticsearch-hadoop
to retrieve snapshot versions for Lucene.

(cherry picked from commit ec3ac9b62452f04ce44dea0a904a6e2b31dd8076)
2019-07-18 21:15:56 +03:00
Armin Braun 3b5038b837
Implement Eventually Consistent Mock Repository for SnapshotResiliencyTests (#40893) (#44570)
* Add eventually consistent mock repository for reproducing and testing AWS S3 blob store behavior
* Relates #38941
2019-07-18 17:54:54 +02:00
Lisa Cawley 8445c41004
[DOCS] Moves content to ML anomaly-detection folder (#44520) (#44530) 2019-07-18 08:44:52 -07:00
Tal Levy c8a8915b27
migrate rollup/monitoring/graph/watcher actions to Writeable (#44464) (#44538)
this commit migrates leftover actions from a few x-pack plugins
to the new Writeable.Reader infrastructure.

relates #34389.
2019-07-18 08:42:56 -07:00
Dimitrios Liappis 651b7a001f
Support log file rolling in jvm presence packaging tests (#44548)
Make JVM presence related packaging tests more resilient, by
also checking for entries in rolled over log files.

Closes #41436
Backport of #44423
2019-07-18 18:39:55 +03:00
Andrey Ershov ef6ddd15c6 Revert "Snapshot tool: S3 orphaned files
cleanup (#44551)"

This reverts commit 09edeeb3
2019-07-18 17:21:45 +02:00
Lisa Cawley 213af8411f [DOCS] Fixes query default value (#44572) 2019-07-18 08:18:58 -07:00
Nhat Nguyen 47cfc25fa0 Skip update if leader and follower settings identical (#44535)
If the setting on the follower and the leader are identical after
filtering out private and internal settings, then we should not call
update setting (on the follower) as there's nothing to change. Moreover,
this makes the ShardFollowTask abort as it considers
ActionRequestValidationException (caused by an empty update setting
request) as a fatal error.

Closes #44521
2019-07-18 11:08:45 -04:00
Lisa Cawley fe23dd1073 [DOCS] Clarifies data transform source definition (#44504) 2019-07-18 08:04:56 -07:00
Andrey Ershov 6f5327ba45 Fix BlobStoreTestUtil 2019-07-18 17:00:23 +02:00
Armin Braun 07cf2cb665
Add disable_chunked_encoding Setting to S3 Repo (#44052) (#44562)
* Add disable_chunked_encoding setting to S3 repo plugin to support S3 implementations that don't support chunked encoding
2019-07-18 16:57:56 +02:00
Lisa Cawley 94ffa5a93c [DOCS] Improves description of dest data frame transform object (#44509) 2019-07-18 07:45:55 -07:00
Andrey Ershov 09edeeb38e Snapshot tool: S3 orphaned files cleanup (#44551)
A tool to work with snapshots.
Co-authored by @original-brownbear.
This commit adds snapshot tool and the single command cleanup, that
cleans up orphaned files for S3.
Snapshot tool lives in x-pack/snapshot-tool.

(cherry picked from commit fc4aed44dd975d83229561090f957a95cc76b287)
2019-07-18 16:38:00 +02:00
James Rodewig 661f70183e [DOCS] Make Query DSL titles consistent (#43935) 2019-07-18 10:33:10 -04:00
David Turner 452f7f67a0
Defer reroute when starting shards (#44539)
Today we reroute the cluster as part of the process of starting a shard, which
runs at `URGENT` priority. In large clusters, rerouting may take some time to
complete, and this means that a mere trickle of shard-started events can cause
starvation for other, lower-priority, tasks that are pending on the master.

However, it isn't really necessary to perform a reroute when starting a shard,
as long as one occurs eventually. This commit removes the inline reroute from
the process of starting a shard and replaces it with a deferred one that runs
at `NORMAL` priority, avoiding starvation of higher-priority tasks.

Backport of #44433 and #44543.
2019-07-18 14:10:40 +01:00
Yogesh Gaikwad 4c95cc3223 skip repository-hdfs integTest in case of fips jvm (#44319)
The repository-hdfs runners need to be disabled it in fips mode.

Testing done for all the tasks, dynamic created and static (integTest, integTestHa, integSecureTest, integSecureHaTest)
2019-07-18 21:10:53 +10:00
Alan Woodward ec0a0a41db Remove type parameter from ParserContext (#44478)
ParserContext.getType() is never called, so we can remove it and tidy up
the callers as well.
2019-07-18 11:07:46 +01:00
Luca Cavanna a8a16e6b08 Associate sub-requests to their parent task in multi search API (#44492)
Multi search accepts multiple search requests and runs them as
independent requests, each one as part of their own search task. Today
they don't get associated though with their parent multi search task,
which would be useful to monitor which msearch a certain search was part
of, if any, and also to cancel all of the sub-requests in case the
parent msearch gets cancelled (though this will also require making 
the multi search task cancellable as a follow-up).
2019-07-18 11:58:30 +02:00
Christoph Büscher e9f257b4d9 Fix ReloadDetailsTests compilation 2019-07-18 11:55:22 +02:00
Christoph Büscher eed2db8947 Add stream serialization to ReloadAnalyzersResponse (#44420)
This change adds Writeable support to ReloadAnalyzersResponse that
is required when using the transport client in 7.x.

Closes #44383
2019-07-18 11:11:54 +02:00
maarab7 1375cc93a8 Fix parameter value for calling data.advanceExact (#44205)
While the code works perfectly well for a single segment, it returns the wrong values for multiple segments. E.g. If we have 500 docs in one segment and if we want to get the doc id = 280 then data.advanceExact(topDocs.scoreDocs[i].doc) works fine. If we have two segments, say, with first segment having docs 1-200 and the second segment having docs 201-500, then 280 is fetched from the second segment but is actually 480. Subtracting the docBase (280-200) takes us to the correct document which is 80 in the second segment and actually 280.
2019-07-18 10:55:10 +02:00
David Turner 7598e0186a
Harmonise indentation of cluster settings (#44540)
Today the long list of `BUILT_IN_CLUSTER_SETTINGS` is indented differently
between `master` and `7.x`. This sometimes makes backporting painful. This
commit adjusts the indentation of earlier branches to match that in `master`.
2019-07-18 09:50:53 +01:00
David Kyle 0fc091f166
Enable XLint warnings for ML (#44346)
Removes the warning suppression -Xlint:-deprecation,-rawtypes,-serial,-try,-unchecked.
Many warnings were unchecked warnings in the test code often because of the use of mocks.
These are suppressed with @SuppressWarning
2019-07-18 09:33:37 +01:00
Ryan Ernst edd26339c5
Convert remaining request classes in xpack core to writeable.reader (#44524) (#44534)
This commit converts all remaining classes extending ActionRequest
in xpack core to have a StreamInput constructor.

relates #34389
2019-07-18 01:11:45 -07:00
Alpar Torok 9f97319b3c Detect process third party audit being killed by OOM (#44269)
* Detect process third party audit being killed by OOM

It's very common for the third party audit to be killed by the OOM
killer when the system is running low on memory.
Since the forbidden APIs call is expected to fail, we were ignoring
these and incorrectly interpreting the partial output.
With this change we detect and provide a proper error message when this
happens.
2019-07-18 10:07:23 +03:00
Armin Braun 6565825a13
Avoid CharsRef Allocations in StreamInput (#44488) (#44519)
* Many messages deserialized from a `StreamInput` only contain short strings, some use-cases of instantiating a `StreamInput` don't deserialize any strings
  * Don't allocate `CharsRef` for small strings to save some allocations (especially on the IO threads)
  * Lazily allocate a larger `CharsRef` if needed for larger strings like we did before and have it live as long as the `StreamInput` like before as well
2019-07-18 08:52:37 +02:00
Tal Levy 38d2ada84f
deprecate Supplier<Response> constructors in HandledTransportAction (#44456) (#44533)
This commit deprecates all constructors of HandledTransportAction
that take in a Supplier instead of a Writeable.Reader for response
objects.

in addition to the deprecation, the following modules were updated to
leverage Writeable

- modules:ingest-common
- modules:lang-mustache

relates #34389.
2019-07-17 22:47:09 -07:00
Tal Levy a5ad59451c
migrate more ML actions off of using Request suppliers (#44462) (#44529)
many classes still use the Streamable constructors of HandledTransportAction,
this commit moves more of those classes to the new Writeable constructors.

relates #34389.
2019-07-17 20:28:29 -07:00
Tal Levy 075a3f0e99
remove usage of ActionType#(String) (#44459) (#44526)
this commit removes usage of the deprecated
constructor with a single argument and no Writeable.Reader.

The purpose of this is to reduce the boilerplate necessary for
properly implementing a new action, as well as reducing the
chances of using the incorrect super constructor while classes
are being migrated to Writeable

relates #34389.
2019-07-17 20:28:11 -07:00
Nhat Nguyen 51180af91d Make peer recovery send file chunks async (#44468)
Relates #44040
Relates #36195
2019-07-17 22:25:43 -04:00