Commit Graph

44106 Commits

Author SHA1 Message Date
jaymode 642e45e9e6
Fix setting openldap realm ssl config
This change fixes the setup of the SSL configuration for the test
openldap realm. The configuration was missing the realm identifier so
the SSL settings being used were just the default JDK ones that do not
trust the certificate of the idp fixture.

See #37591
2019-01-18 12:24:11 -07:00
Heather McCartney fc1c47649a Document the need for JAVA11_HOME (#37589)
This commit updates the contribution docs to include java11 as a requirement for building and testing Elasticsearch.
2019-01-18 11:16:50 -08:00
Igor Motov 54af8a4e7a
SQL: fix object extraction from sources (#37502)
Throws an exception if hit extractor tries to retrieve unsupported
object. For example, selecting "a" from `{"a": {"b": "c"}}` now throws
an exception instead of returning null.

Relates to #37364
2019-01-18 14:03:48 -05:00
Albert Zaharovits 633bd09be0 Nit in settings.gradle for Eclipse
Fixes ``./gradlew eclipse` failure introduced in 6d99e790b3
2019-01-18 19:50:43 +02:00
Jack Conradson de55b4dfd1
Add types deprecation to script contexts (#37554)
This adds deprecation to _type in the script contexts for ingest and update. 
This adds a DeprecationMap that wraps the ctx Map containing _type for these 
specific contexts.
2019-01-18 09:13:49 -08:00
Martijn van Groningen 6846666b6b
Add ccr follow info api (#37408)
* Add ccr follow info api

This api returns all follower indices and per follower index
the provided parameters at put follow / resume follow time and
whether index following is paused or active.

Closes #37127

* iter

* [DOCS] Edits the get follower info API

* [DOCS] Fixes link to remote cluster

* [DOCS] Clarifies descriptions for configured parameters
2019-01-18 16:37:21 +01:00
Yannick Welsch 377d96e376
Remove initial_master_nodes on node restart (#37580)
Some tests (e.g. testRestoreIndexWithShardsMissingInLocalGateway) were split-braining since
being switched to Zen2 because the bootstrap setting was left around when nodes got restarted
with data folders wiped.

The test in question here was starting one node (which autobootstrapped to that single node), then
another node. The first node was then shut down (after excluding it from the voting configuration),
its data folder wiped, and restarted. After restart, the node had an empty data folder yet
initial_master_nodes set to itself (i.e. same name). This made the node sometimes form a cluster of
its own, and not rejoin the existing cluster with the other node.
2019-01-18 16:36:42 +01:00
Michael Basnight 604422c6c5
Update Execute Watch to allow unknown fields (#37498)
ExecuteWatchResponse did not allow unknown fields. This commit fixes the
test and ConstructingObjectParser such that it does now allow unknown
fields. It also creates a new client side test for the response.

Relates #36938
2019-01-18 09:20:32 -06:00
Ioannis Kakavas 7597b7ce2b
Add validation for empty PutPrivilegeRequest (#37569)
Return an error to the user if the put privilege api is called with
an empty body (no privileges)

Resolves: #37561
2019-01-18 17:06:40 +02:00
Jason Tedor ed297b7369
Only update response headers if we have a new one (#37590)
Currently when adding a response header, we do some de-duplication, and
maybe drop the header on the floor if we have reached capacity. Yet, we
still update the thread local tracking the response headers. This is
really expensive because under the hood there is a shared reference that
we synchronize on. In the case of a request processed across many shards
in a tight loop, this contention can be detrimental to performance. We
can avoid updating the thread local in these cases though, when the
response header is duplicate of one that we have already seen, or when
it's dropped on the floor. This commit addresses these performance
issues by avoiding the unnecessary set.
2019-01-18 08:20:05 -05:00
Christoph Büscher 3a96608b3f
Remove more include_type_name and types from docs (#37601) 2019-01-18 14:11:18 +01:00
Tanguy Leroux 29d3a708da Fix BulkWithUpdatesIT and CloseIndexIT
As of today the Close Index API does its best to close indices,
but closing an index with ongoing recoveries might or might not
be acknowledged depending of the values of the max seq number
and global checkpoint at the time the
TransportVerifyShardBeforeClose action is executed.

These tests failed because they always expect that the index is
correctly closed on the first try, which is not always the case.
Instead we need to retry the closing until it succeed.

Closes #37571
2019-01-18 10:54:35 +01:00
David Turner 65e76b3f6f
Migrate RecoveryFromGatewayIT to Zen2 (#37520)
* Fixes `testTwoNodeFirstNodeCleared` by manipulating voting config exclusions.

* Removes `testRecoveryDifferentNodeOrderStartup` since state recovery is now
  handled entirely on the elected master, so the order in which the data nodes
  start is irrelevant.
2019-01-18 09:15:51 +00:00
David Turner 699d881739
Migrate IndicesExistsIT to Zen2 (#37526)
This test was actually passing, for the wrong reason: it asserts a
`MasterNotDiscoveredException` is thrown, expecting this to be due to a failure
to perform state recovery, but in fact it's thrown because the node is not
correctly bootstrapped.
2019-01-18 09:15:30 +00:00
Christoph Büscher 25aac4f77f
Remove `include_type_name` in asciidoc where possible (#37568)
The "include_type_name" parameter was temporarily introduced in #37285 to facilitate
moving the default parameter setting to "false" in many places in the documentation
code snippets. Most of the places can simply be reverted without causing errors.
In this change I looked for asciidoc files that contained the
"include_type_name=true" addition when creating new indices but didn't look
likey they made use of the "_doc" type for mappings. This is mostly the case
e.g. in the analysis docs where index creating often only contains settings. I
manually corrected the use of types in some places where the docs still used an
explicit type name and not the dummy "_doc" type.
2019-01-18 09:34:11 +01:00
Christoph Büscher 2f0e0b2426
Allow indices.get_mapping response parsing without types (#37492)
This change adds deprecation warning to the indices.get_mapping API in case the
"inlcude_type_name" parameter is set to "true" and changes the parsing code in
GetMappingsResponse to parse the type-less response instead of the one
containing types. As a consequence the HLRC client doesn't need to force
"include_type_name=true" any more and the GetMappingsResponseTests can be
adapted to the new format as well. Also removing some "include_type_name"
parameters in yaml test and docs where not necessary.
2019-01-18 09:33:36 +01:00
Mike Place 080c073618 Minor docs cleanup (#37595)
Not all terminals are accessed via SSH.
2019-01-18 09:28:48 +01:00
Armin Braun 62ddc8c776 Reenable UnicastZenPingTests#testSimplePings
* This was muted needlessly, the problem in #26701 only applies to `6.x`
* Relates #26701
2019-01-18 08:36:22 +01:00
Tim Brooks 978c818d0f
Use RestoreSnapshotRequest in CcrRepositoryIT
Commit #37535 removed an internal restore request in favor of the
RestoreSnapshotRequest. Commit #37449 added a new test that used the
internal restore request. This commit modifies the new test to use the
RestoreSnapshotRequest.
2019-01-17 15:31:27 -07:00
Tim Brooks b6f06a48c0
Implement follower rate limiting for file restore (#37449)
This is related to #35975. This commit implements rate limiting on the
follower side using a new class `CombinedRateLimiter`.
2019-01-17 14:58:46 -07:00
Armin Braun 381d035cd6
Remove Redundant RestoreRequest Class (#37535)
* Same as #37464 but for the restore side
2019-01-17 22:23:23 +01:00
Tal Levy a0c504e4a3
Create specific exception for when snapshots are in progress (#37550)
delete and close index actions threw IllegalArgumentExceptions
when attempting to run against an index that has a snapshot
in progress.

This change introduces a dedicated SnapshotInProgressException
for these scenarios. This is done to explicitly signal to clients that
this is the reason the action failed, and it is a retryable error.

relates to #37541.
2019-01-17 13:21:12 -08:00
James Baiera 5782a5bbbc Mute UnicastZenPingTests#testSimplePings
relates #26701
2019-01-17 15:13:09 -05:00
Lisa Cawley 6dcb3af4c8
[DOCS] Adds size limitation to the get datafeeds APIs (#37578) 2019-01-17 10:47:15 -08:00
Yannick Welsch 68de2edb14
Fix assertion at end of forceRefreshes (#37559)
This commit ensures that we only change refreshListeners to a list if we're actually adding
something to the list.
2019-01-17 19:18:47 +01:00
Yannick Welsch 6d64a2a901
Propagate Errors in executors to uncaught exception handler (#36137)
This is a continuation of #28667 and has as goal to convert all executors to propagate errors to the
uncaught exception handler. Notable missing ones were the direct executor and the scheduler. This
commit also makes it the property of the executor, not the runnable, to ensure this property. A big
part of this commit also consists of vastly improving the test coverage in this area.
2019-01-17 17:46:35 +01:00
Lisa Cawley a2d9c464b2
[DOCS] Adds limitation to the get jobs API (#37549) 2019-01-17 08:21:37 -08:00
Jake Landis 587034dfa7
Add set_priority action to ILM (#37397)
This commit adds a set_priority action to the hot, warm, and cold
phases for an ILM policy. This action sets the `index.priority`
on the managed index to allow different priorities between the
hot, warm, and cold recoveries.

This commit also includes the HLRC and documentation changes.

closes #36905
2019-01-17 09:55:36 -06:00
Nhat Nguyen 20ed3dd1a8
Make recovery source send operations non-blocking (#37503)
Relates #37458
2019-01-17 09:59:05 -05:00
Jim Ferenczi 4351a5e537
Allow field types to optimize phrase prefix queries (#37436)
This change adds a way to customize how phrase prefix queries should be created
on field types. The match phrase prefix query is exposed in field types in order
to allow optimizations based on the options set on the field.
For instance the text field uses the configured prefix field (if available) to
build a span near that mixes the original field and the prefix field on the last
position.
This change also contains a small refactoring of the match/multi_match query that
simplifies the interactions between the builders.

Closes #31921
2019-01-17 15:10:28 +01:00
Martijn van Groningen b85bfd3e17
Added fatal_exception field for ccr stats in monitoring mapping. (#37563) 2019-01-17 14:04:41 +01:00
Yannick Welsch d9fa4e4ada
Fix testRelocateWhileContinuouslyIndexingAndWaitingForRefresh (#37560)
This test failed because the refresh at the end of the test is not guaranteed to run before the
indexing is completed, and therefore there's no guarantee that the refresh will free all operations.
This triggers an assertion failure in the test clean-up, which asserts that there are no more pending
operations.
2019-01-17 13:59:09 +01:00
Martijn van Groningen 99b09845da
Moved ccr integration to the package with other ccr integration tests. 2019-01-17 13:57:56 +01:00
Yannick Welsch 6fe2d6da03 Mute TransportClientNodesServiceTests#testListenerFailures
Relates to #37567
2019-01-17 13:54:48 +01:00
Martijn van Groningen da799306a8
Decreased time out in test
Relates to #37378
2019-01-17 11:51:17 +01:00
Torgeir Thoresen 676e1b1a13 Fix erroneous docstrings for abstract bulk by scroll request (#37517) 2019-01-17 10:22:49 +01:00
Marios Trivyzas 1686c32ba9
SQL: Rename SQL type DATE to DATETIME (#37395)
* SQL: Rename SQL data type DATE to DATETIME

SQL data type DATE has only the date part (e.g.: 2019-01-14)
without any time information. Previously the SQL type DATE was
referring to the ES DATE which contains also the time part along
with TZ information. To conform with SQL data types the data type
`DATE` is renamed to `DATETIME`, since it includes also the time,
as a new runtime SQL `DATE` data type will be introduced down the road,
which only contains the date part and meets the SQL standard.

Closes: #36440

* Address comments
2019-01-17 10:17:58 +02:00
Przemyslaw Gomulka b6e5ccaf8a
Remove the AbstracLifecycleComponent constructor with Settings (#37523)
Adding the migration guide and removing the deprecated in 6.x
constructor

relates #35560
relates #34488
2019-01-17 09:10:09 +01:00
Jason Tedor 18a3e48a4a
Change file descriptor limit to 65535 (#37537)
Some systems default to a nofile ulimit of 65535. To reduce the pain of
deploying Elasticsearch to such systems, this commit lowers the required
limit from 65536 to 65535.
2019-01-16 17:19:12 -05:00
Nhat Nguyen 655103de58 Increase timeout for testAddNewReplicas
We flush quite often in testAddNewReplicas to create the safe index
commit with gaps in sequence numbers. This test is failing recently
because CI is too slow to complete 5 small flushes in 10 seconds.

This commit increases timeout for this test and also ensures to always
terminate the background indexing. The latter is to eliminate unrelated
failures if this test fails again.

Closes #37183
2019-01-16 13:17:10 -05:00
Marios Trivyzas ecf0de30ba
SQL: Lowercase the datatypes in validation error msgs (#37524)
To follow the ES convention display the datatypes in lowercase
in error messages thrown during validation if `IN` and conditional
functions.
2019-01-16 18:41:10 +02:00
Andrey Ershov 4e72f3c5c6
DedicatedClusterSnapshotRestoreIT to Zen2 (#37489)
All tests except testRestorePersistentSettings (renamed to
testExceptionWhenRestoringPersistentSettings) worked fine.
testExceptionWhenRestoringPersistentSettings re-written to use a custom
setting, because "minimum master node" setting is no longer available
in Zen2. It turns out there is no good replacement for "minimum master
node" setting for this test, that's why the custom setting is
introduced.

Unfortunately, there is #37485 bug and currently
RestoreService does not perform setting validation. That's why the
test is annotated with @AwaitsFix, the idea is to merge this commit and
then fix the issue and enable the test. (The test passes with a simple
fix, that adds a single line to RestoreService).
2019-01-16 11:14:16 -05:00
Marios Trivyzas 2cf4b1863f
SQL: Lowercase es data type (mapping) returned from SQL Commands (#37531)
To follow the ES convention, convert the es data type, returned as
column `mapping` from SQL Commands, to lowercase.

Fixes: #37521
2019-01-16 18:08:33 +02:00
Costin Leau 19d2c29ca6 Remove SYS CATALOGS leftover
Relates #37506
2019-01-16 17:28:06 +02:00
Costin Leau 1f76b5fc31
SQL: Describe aliases as views (#37496)
When reporting metadata, several clients have issues with the 'ALIAS'
type. To improve compatibility and be consistent with the ANSI SQL
expectations and because they are similar, aliases targets are now
reported as views.

Close #37422
2019-01-16 17:26:00 +02:00
Jack Conradson 3d8c04659c
Deprecate _type from LeafDocLookup (#37491)
* Deprecate _type from LeafDocLookup

* Response to PR comments.

* Response to PR comments.
2019-01-16 07:05:09 -08:00
Tim Brooks 0b5af276a8
Allow system privilege to execute proxied actions (#37508)
Currently all proxied actions are denied for the `SystemPrivilege`.
Unfortunately, there are use cases (CCR) where we would like to proxy
actions to a remote node that are normally performed by the
system context. This commit allows the system context to perform
proxy actions if they are actions that the system context is normally
allowed to execute.
2019-01-16 07:52:38 -07:00
Michael Basnight 86697f23a3
Update Put Watch to allow unknown fields (#37494)
PutWatchResponse did not allow unknown fields. This commit fixes the
test and ConstructingObjectParser such that it does now allow unknown
fields.

Relates #36938
2019-01-16 08:50:03 -06:00
Nhat Nguyen 0160ba2539 AwaitsFix testAddNewReplicas
Tracked at #37183
2019-01-16 09:48:35 -05:00
Andrei Stefan 659326fdd6
SQL: Add protocol tests and remove jdbc_type from drivers response (#37516) 2019-01-16 16:28:46 +02:00