43378 Commits

Author SHA1 Message Date
Tanguy Leroux
5cd9b518ac
Fix usages of XContentParserUtils.ensureExpectedToken() (#35530)
Some usages of XContentParserUtils.ensureExpectedToken() are inverting
the expected and actual tokens, resulting in wrong error messages.
2018-11-15 09:18:32 +01:00
Tal Levy
dc1821c707
explain geo_point mapping in geoip-processor (#29114)
simple docs change to add missing mapping explanation. Users may not be aware this is
a prerequisite for doing geo-queries on this enriched data.
2018-11-14 20:05:45 -08:00
Tal Levy
95a09ab2d6
[ILM] TEST: skip checking _ilm/explain response b/c flaky timing (#35562)
The docs are not resilient to timing issues where the ILM metadata is not set on newly 
created indices, so we shouldn't be so strict on the returned response
2018-11-14 16:45:06 -08:00
Deb Adair
91a38b8fe0 [DOCS] Fixed bad cross doc link. 2018-11-14 16:09:18 -08:00
Jay Modi
faa9523d19
Remove deprecated constructor from failure handler (#35565)
The DefaultAuthenticationFailureHandler has a deprecated constructor
that was present to prevent a breaking change to custom realm plugin
authors in 6.x. This commit removes the constructor and its uses.
2018-11-14 17:04:33 -07:00
debadair
38d6bb27f5
[DOCS] Added link to standalone ODBC driver doc. (#35526) 2018-11-14 15:23:28 -08:00
Tal Levy
5d0544d732
update move-to-step spec with proper docs and extra warning to clients (#35477) 2018-11-14 14:08:31 -08:00
Tal Levy
f70d4470f7
[ILM] getting started documentation (#35379) 2018-11-14 13:58:08 -08:00
Lee Hinman
8ea999e489
Include stack trace with ILM error in explain output (#35512)
This changes the stacktrace to be included with the ILM explain error when the
index is an on ERROR step.

Before:

```json
{
  "indices" : {
    "foo" : {
      "index" : "foo",
      "managed" : true,
      "policy" : "bad",
      "lifecycle_date_millis" : 1542131670601,
      "phase" : "warm",
      "phase_time_millis" : 1542131676335,
      "action" : "shrink",
      "action_time_millis" : 1542131676335,
      "step" : "ERROR",
      "step_time_millis" : 1542131676451,
      "failed_step" : "shrink",
      "step_info" : {
        "type" : "illegal_argument_exception",
        "reason" : "the number of target shards [13] must be less that the number of source shards [2]"
      },
      "phase_execution" : {
        "policy" : "bad",
        "phase_definition" : {
          "min_age" : "5s",
          "actions" : {
            "shrink" : {
              "number_of_shards" : 13
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1542131669839
      }
    }
  }
}
```

After

```
{
  "indices" : {
    "foo" : {
      "index" : "foo",
      "managed" : true,
      "policy" : "bad",
      "lifecycle_date_millis" : 1542131670601,
      "phase" : "warm",
      "phase_time_millis" : 1542131676335,
      "action" : "shrink",
      "action_time_millis" : 1542131676335,
      "step" : "ERROR",
      "step_time_millis" : 1542131676451,
      "failed_step" : "shrink",
      "step_info" : {
        "type" : "illegal_argument_exception",
        "reason" : "the number of target shards [13] must be less that the number of source shards [2]",
        "stack_trace" : "java.lang.IllegalArgumentException: the number of target shards [13] must be less that the number of source shards [2]\n\tat org.elasticsearch.cluster.metadata.IndexMetaData.selectShrinkShards(IndexMetaData.java:1509)\n\tat org.elasticsearch.action.admin.indices.shrink.TransportResizeAction.prepareCreateIndexRequest(TransportResizeAction.java:146)\n\tat org.elasticsearch.action.admin.indices.shrink.TransportResizeAction$1.onResponse(TransportResizeAction.java:104)\n\tat org.elasticsearch.action.admin.indices.shrink.TransportResizeAction$1.onResponse(TransportResizeAction.java:101)\n\tat org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:64)\n\tat org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:60)\n\tat org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.onCompletion(TransportBroadcastByNodeAction.java:383)\n\tat org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.onNodeResponse(TransportBroadcastByNodeAction.java:352)\n\tat org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction$1.handleResponse(TransportBroadcastByNodeAction.java:324)\n\tat org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction$1.handleResponse(TransportBroadcastByNodeAction.java:314)\n\tat org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1117)\n\tat org.elasticsearch.transport.TransportService$DirectResponseChannel.processResponse(TransportService.java:1198)\n\tat org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1178)\n\tat org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:54)\n\tat org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:417)\n\tat org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:391)\n\tat org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:251)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\n\tat org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:309)\n\tat org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)\n\tat org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:714)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:726)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\n"
      },
      "phase_execution" : {
        "policy" : "bad",
        "phase_definition" : {
          "min_age" : "5s",
          "actions" : {
            "shrink" : {
              "number_of_shards" : 13
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1542131669839
      }
    }
  }
}
```

Resolves #35498
2018-11-14 14:40:05 -07:00
Hendrik Muhs
fc774a3776
add ES 6.5.1 (on master) (#35549)
add ES 6.5.1
2018-11-14 21:08:21 +01:00
Tal Levy
30a713cc3f
[ILM] add docs explaining high-level concepts (#35322)
add docs explaining high-level concepts
2018-11-14 09:23:50 -08:00
Benjamin Trent
803eccec11
HLRC: Adding ML Update Filter API (#35522)
* HLRC: Adding ml get filters api

* HLRC: Adding ML Update Filter API
2018-11-14 11:13:11 -06:00
Tanguy Leroux
c8c8ce2374
Extract RunOnce into a dedicated class (#35489)
This commit extracts the static inner class RunOnce from 
WorkerBulkByScrollTaskState so that it can be reused at 
other places.
2018-11-14 17:33:04 +01:00
Thiago Souza
98e075227b [Doc] Correct jdbc url example (#35513)
Adds a missing `?` to separate the url parameters.
2018-11-14 17:30:23 +01:00
Alpar Torok
a3cb74190c Fix flaky gradle integration test 2018-11-14 17:21:25 +02:00
Zachary Tong
c7ea0343d2 [Rollup] Update versions after backport
Both serialization and yaml test skips
2018-11-14 09:56:45 -05:00
Tanguy Leroux
5b7446bb5f
Add Delete Privileges API to HLRC (#35454)
This commit adds the Delete Privileges API to the high level REST
client.

Related to #29827
2018-11-14 14:04:30 +01:00
Ed Savage
e7b7d52a6a
[HLRC][ML] Add ML get model snapshots API (#35487)
Relates #29827
2018-11-14 13:03:04 +00:00
Andrey Ershov
045fdd0d3b Merge master into zen2 2018-11-14 15:37:13 +03:00
Yannick Welsch
4cfdb0609e
Adapt InternalCluster#fullRestart to call onNodeStopped when all nodes are stopped (#35494)
Refactors and simplifies the logic around stopping nodes, making sure that for a full cluster restart
onNodeStopped is only called after the nodes are actually all stopped (and in particular not while
starting up some nodes again). This change also ensures that a closed node client is not being used
anymore (which required a small change to a test).

Relates to #35049
2018-11-14 13:24:56 +01:00
Alpar Torok
212c202709 Remove hard coded version string in tests
- The current version was hard coded in the test, causing it to fail as
we removed the qualifier
- also applied the base plugin to the root project to get the `clean`
task to work as expected. This was preventing the failure from
reproducing locally.
2018-11-14 14:14:03 +02:00
Jim Ferenczi
72504c2512
Do not recommend to use the _id field in search_after docs (#35370)
The documentation of `search_after` recommends to use the `_id`
field as a tiebreaker for the sort without warning against
the additional memory required. This change changes the recommandation
to use a copy of the `_id` field with doc_values enabled.
2018-11-14 10:50:31 +01:00
Alpar Torok
0ce4649e88
Manage dependencies for test clusters (#35304)
* Manage dependencies for test clusters

Create a configuration and add the distribution to it automatically.
A task is created and added as a dependency to any task that uses a test
cluster.
The task extracts all the zip archives ( only zip support for now )
in the configuration.
We do this only once because most tests mostly use the same distribution
and thus we can avoid extracting it multiple times.
With this we will be able to start the node from the same files which
will most of the time live in OS caches or COW if the
configuration requires it.
2018-11-14 11:22:00 +02:00
Tanguy Leroux
bbe50e7a86
Remove LoggingRunnable class (#35486)
This commit removes the unused LoggingRunnable class.
2018-11-14 10:12:25 +01:00
Alpar Torok
dc0e05f35a
Fix Eclipse configuration (#35452) 2018-11-14 11:11:46 +02:00
Alpar Torok
b49f461bf1
Swithc to jcenter and hope to have fewer network failures in builds (#35427)
* Swithc to jcenter

Jcenter suposedly operates on a CDN.
It should be faster and more reliable.
It's the default in Andorid Studio currently
( it switched from mavenCentral ).

This change is safe because jcenter is a superset of mavenCentral.

* update comment
2018-11-14 10:50:08 +02:00
Tanguy Leroux
31567cefb4
[RCI] Check blocks while having index shard permit in TransportReplicationAction (#35332)
Today, the TransportReplicationAction checks the global level blocks and 
the index level blocks before routing the operation to the primary, in the 
ReroutePhase, and it happens at the very beginning of the transport 
replication action execution. For the upcoming rework of the Close Index 
API and in order to deal with primary relocation, we'll need to also check 
for blocks before executing the operation on the primary (while holding a 
permit) but before routing to the new primary.

This pull request change the AsyncPrimaryAction so that it checks for 
replication action's blocks before executing the operation locally or before 
routing the primary action to the newly primary shard. The check is done 
while holding a PrimaryShardReference.

Related to #33888
2018-11-14 09:43:55 +01:00
Hendrik Muhs
5c84708ee5 test: expose error message on failure 2018-11-14 08:25:41 +01:00
Tim Vernum
231f6c1595
Formal support for "password_hash" in Put User (#35242)
For some time, the PutUser REST API has supported storing a pre-hashed
password for a user. The change adds validation and tests around that
feature so that it can be documented & officially supported.

It also prevents the request from containing both a "password" and a "password_hash".
2018-11-14 16:46:31 +11:00
Christoph Büscher
d8b1c23e1d
Remove Comparable interface from ScoreAccessor (#35519)
The way ScoreAccessor implements `compareTo()` is problematic because it doesn't
completely follow the Comparable contract, specificaly symmetry (if x is a
ScoreAccessor and y any Number then x.comparTo(y) works, but y.compareTo(x)
generally does not even compile). Fortunately we don't seem to use the fact that
ScoreAccessor is a Comparable anywhere, so we can simply remove it.
2018-11-14 05:58:05 +01:00
Christoph Büscher
603d1a470f
Add stop rollup job support to HL REST Client (#34702)
This change adds support for stoping a rollup job to the High Level REST Client.

Relates to #29827
2018-11-14 05:40:42 +01:00
David Turner
229637fd7e
[Zen2] Remove duplicate discovered peers (#35505)
Today the `PeerFinder` probes each address it obtains, identifies the node to
which it just connected, and then returns all such nodes. However, this can
lead to duplicates if a node manages to connect to another node via two
distinct addresses.  This causes bootstrapping to fail since
`BootstrapConfiguration#resolve` forbids duplicates.

This change alters the behaviour of the `PeerFinder` to remove duplicates in
this situation.
2018-11-13 22:30:36 +00:00
Zachary Tong
c346a0f027
[Rollup] Add wait_for_completion option to StopRollupJob API (#34811)
This adds a `wait_for_completion` flag which allows the user to block 
the Stop API until the task has actually moved to a stopped state, 
instead of returning immediately.  If the flag is set, a `timeout` parameter
can be specified to determine how long (at max) to block the API
call.  If unspecified, the timeout is 30s.

If the timeout is exceeded before the job moves to STOPPED, a
timeout exception is thrown.  Note: this is just signifying that the API
call itself timed out.  The job will remain in STOPPING and evenutally
flip over to STOPPED in the background.

If the user asks the API to block, we move over the the generic
threadpool so that we don't hold up a networking thread.
2018-11-13 16:37:17 -05:00
jaymode
634bc9d4ea
mute TasksClientDocumentationIT testCancelTasks
This commit mutes this test, which has failed twice in CI today.

See #35514
2018-11-13 14:35:05 -07:00
Benjamin Trent
b9eb5f7b63
HLRC: Adding ml get filters api (#35502)
* HLRC: Adding ml get filters api

* refactoring setId name
2018-11-13 14:53:32 -06:00
Colin Goodheart-Smithe
c934fb087a
Adds HLRC docs for put lifecycle policy (#35457)
* Adds HLRC docs for put lifecycle policy

* Adds link to docs in client javadocs

* Fixes checkstyle

* Make the documentation use the right ack response
2018-11-13 20:40:53 +00:00
Vladimir Dolzhenko
9728119b82 [CI] AllocationIdIT testFailedRecoveryOnAllocateStalePrimaryRequiresAnotherAllocateStalePrimary failure
Closes #35504
2018-11-13 20:57:50 +01:00
Julie Tibshirani
bc799e4a6f
Ignore warnings related to types deprecation in REST tests. (#35395) 2018-11-13 11:56:01 -08:00
David Turner
8e40a2bbe2
[Zen2] Introduce vote withdrawal (#35446)
If shutting down half or more of the master-eligible nodes, their votes must
first be explicitly withdrawn to ensure that the cluster doesn't lose its
quorum. This works via _voting tombstones_, stored in the cluster state, which
tell the reconfigurator to remove nodes from the voting configuration.

This change introduces voting tombstones to the cluster state, together with
transport APIs for adding and removing them, and makes use of these APIs in
`InternalTestCluster` to support tests which remove at least half of the
master-eligible nodes at once (e.g. shrinking from two master-eligible nodes to
one).
2018-11-13 19:32:32 +00:00
Lee Hinman
6cb2aef943
[ILM] Add documentation for error handling in ILM (#35426)
* [ILM] Add documentation for error handling in ILM

This adds some initial documentation for error handling and retrying failed
steps for index lifecycle management
2018-11-13 12:19:22 -07:00
Armin Braun
3db671e12a
MINOR: Remove Hack in TestCluster Task (#35485)
*  The logic that splits by `","` isn't necessary anymore since #35375 removed the hack that set the comma separated list for the unicast host
2018-11-13 20:26:42 +02:00
lcawl
2ac4ef7a69 [DOCS] Fixes broken link 2018-11-13 10:04:26 -08:00
Jason Tedor
8b2f22ee07
Introduce CCR overview (#35436)
This commit introduces a basic overview for cross-cluster replication to
the docs.

Co-authored-by: "lcawl <lcawley@elastic.co>"
2018-11-13 12:15:37 -05:00
Colin Goodheart-Smithe
73a563c915
Converts ILM HLRC APIs to use client Ack response (#35492) 2018-11-13 17:02:56 +00:00
David Turner
0e1a12122c Merge branch 'master' into zen2 2018-11-13 15:25:35 +00:00
David Turner
fbd3cab410
[Zen2] Remove AbstractComponent usage (#35483)
AbstractComponent was deprecated in #35140 and is looking like it will be
removed at some point by #34888. Today all it does is provide a logger. This
change removes the usages of AbstractComponent that live solely in the zen2
feature branch to avoid some future merge pain, and replaces it where necessary
with some directly-created loggers.
2018-11-13 15:20:49 +00:00
Luca Cavanna
0b7d18d733
HLRC: add support for get license basic/trial status API (#33176)
Relates to #29827
2018-11-13 15:52:59 +01:00
Igor Motov
a76ac5729d
Rest HL client: Add watcher stats API (#35185)
Relates to #29827
2018-11-13 04:47:35 -10:00
Jason Tedor
4b5fbada9f
Fix reference to get follower stats API
This commit fixes a reference to the CCR get follower stats API in the
CCR getting started docs.
2018-11-13 09:14:26 -05:00
Przemyslaw Gomulka
fd4cd80496
HLRC: migration api - upgrade (#34898)
Implement high level client for migration upgrade API. It should wrap
RestHighLevelClient and expose high level IndexUpgradeRequest (new),
IndexTaskResponse for submissions with wait_for_completion=false and
BulkByScrollResponse (already used) objects.

refers: #29827
2018-11-13 15:01:53 +01:00