Commit Graph

40070 Commits

Author SHA1 Message Date
Alexander Reelsen 736d2e8a65
Docs: Fix README upgrade mention (#32313)
The README still mentioned Elasticsearch 1.x. This commit links to the
documentation instead and properly formats the last paragraph.
2018-07-31 16:14:37 +02:00
Sohaib Iftikhar 4fa92cbf49 Changed ReindexRequest to use Writeable.Reader (#32401)
-- This is a pre-stage for adding the reindex API to the REST high-level-client
-- Follows the pattern set in #26315
2018-07-31 10:11:17 -04:00
Daniel Mitterdorfer adb93da974 Mute KerberosAuthenticationIT
Relates #32498
2018-07-31 15:55:52 +02:00
Paul Sanwald 6f93911955
Fix AutoIntervalDateHistogram.testReduce random failures (#32301)
1. Refactor the test to use the same roundings as the implementation.
2. Refactor the test verification logic to use `innerIntervals` when rounding.
2018-07-31 08:52:16 -04:00
Colm O'Shea 97b379e0d4 fix no=>not typo (#32463)
Found a tiny typo while reading the docs
2018-07-31 13:33:23 +01:00
Daniel Mitterdorfer 9703d06321 Mute QueryProfilerIT#testProfileMatchesRegular()
Relates #32492
2018-07-31 13:29:21 +02:00
Yannick Welsch 11f6983d69
HLRC: Add delete watch action (#32337)
Adds the "delete watch" API to the High-Level Rest Client.

Relates #29827
2018-07-31 10:29:22 +02:00
Luca Cavanna a3b272966d
High-level client: fix clusterAlias parsing in SearchHit (#32465)
When using cross-cluster search through the high-level REST client, the cluster alias from each search hit was not parsed correctly. It would be part of the index field initially, but overridden just a few lines later once setting the shard target (in case we have enough info to build it from the response). In any case, getClusterAlias returns `null` which is a bug.

With this change we rather parse back clusterAliases from the index name, set its corresponding field and properly handle the two possible cases depending on whether we can or cannot build the shard target object.
2018-07-31 09:41:51 +02:00
David Turner 8b57e2e5ba
Fix calculation of orientation of polygons (#27967)
The method for working out whether a polygon is clockwise or anticlockwise is
mostly correct but doesn't work in some rare cases such as the included test
case. This commit fixes that.
2018-07-31 08:25:21 +01:00
Yogesh Gaikwad d4ea440e37
[Kerberos] Add missing javadocs (#32469)
This commit adds missing javadocs and fixes few where
the build failed when using JDK 11 for compilation.

Closes#32461
2018-07-31 11:18:08 +10:00
Yogesh Gaikwad f0b36679ec
[Kerberos] Remove Kerberos bootstrap checks (#32451)
This commit removes Kerberos bootstrap checks as they were more
validation checks and better done in Kerberos realm constructor
than as bootstrap checks. This also moves the check
for one Kerberos realm per node to where we initialize realms.
This commit adds few validations which were missing earlier
like missing read permissions on keytab file or if it is directory
to throw exception with error message.
2018-07-31 10:59:36 +10:00
Tim Vernum d75efbcf68
Make get all app privs requires "*" permission (#32460)
The default behaviour for "GetPrivileges" is to get all application
privileges. This should only be allowed if the user has access to
the "*" application.
2018-07-31 09:07:47 +10:00
Nik Everett 4101fc4e3d
Switch security to new style Requests (#32290)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/plugin/security` project to use the new
versions.
2018-07-30 18:16:26 -04:00
Nik Everett 670630948b
Switch security spi example to new style Requests (#32341)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/security-example-spi-extension`
project to use the new versions.
2018-07-30 18:07:49 -04:00
Jack Conradson c69e62d96f
Painless: Add PainlessConstructor (#32447)
PainlessMethod was being used as both a method and a constructor, and while there are 
similarities, there are also some major differences. This allows the reflection objects to be 
stored reducing the number of other pieces of data stored in a PainlessMethod as they are 
now redundant. This temporarily increases some of the code in FunctionRef and 
PainlessDocGenerator as they now differentiate between constructors and methods, BUT 
is also makes the code more maintainable because there aren't checks in several places 
anymore to differentiate.
2018-07-30 14:46:24 -07:00
Tal Levy 1e0fcebfe1
update rollover to leverage write-alias semantics (#32216)
Rollover should not swap aliases when `is_write_index` is set to `true`.
Instead, both the new and old indices should have the rollover alias,
with the newly created index as the new write index

Updates Rollover to leverage the ability to preserve aliases and swap which is the write index.

Historically, Rollover would swap which index had the designated alias for writing documents against. This required users to keep a separate read-alias that enabled reading against both rolled over and newly created indices, whiles the write-alias was being re-assigned at every rollover.

With the ability for aliases to designate a write index, Rollover can be a bit more flexible with its use of aliases.

Updates include:

- Rollover validates that the target alias has a write index (the index that is being rolled over). This means that the restriction that aliases only point to one index is no longer necessary.
- Rollover explicitly (and atomically) swaps which index is the write-index by explicitly assigning the existing index to have `is_write_index: false` and have the newly created index have its rollover alias as `is_write_index: true`. This is only done when `is_write_index: true` on the write index. Default behavior of removing the alias from the rolled over index stays when `is_write_index` is not explicitly set

Relevant things that are staying the same:

- Rollover is rejected if there exist any templates that match the newly-created index and configure the rollover-alias
   - I think this existed to prevent the situation where an alias pointed to two indices for a short while. Although this can technically be relaxed, the specific cases that are safe are really particular and difficult to reason, so leaving the broad restriction sounds good
2018-07-30 14:32:55 -07:00
w-bonelli 072c0be8af Update Fuzzy Query docs to clarify default behavior re max_expansions (#30819)
Stating that the Fuzzy Query generates "all possible" matching terms is misleading, given that the query's default behavior is to generate a maximum of 50 matching terms.

(cherry picked from commit 345a0071a2a41fd7f80ae9ef8a39a2cb4991aedd)
2018-07-30 13:19:26 -07:00
Armin Braun cf7489899a
INGEST: Clean up Java8 Stream Usage (#32059)
* GrokProcessor: Rationalize the loop over the map to save allocations and indirection
* IngestDocument: Rationalize way we append to `List`
2018-07-30 21:25:30 +02:00
Ioannis Kakavas c2e3bebab9
Ensure KeyStoreWrapper decryption exceptions are handled (#32464)
* Ensure decryption related exceptions are handled

This commit ensures that all possible Exceptions in
KeyStoreWrapper#decrypt() are handled. More specifically, in the
case that a wrong password is used for secure settings, calling readX
on the DataInputStream that wraps the CipherInputStream can throw an
IOException. It also adds a test for loading a KeyStoreWrapper with
a wrong password.

Resolves #32411
2018-07-30 22:15:59 +03:00
Ryan Ernst 34d006f82a
Tests: Fix convert error tests to use fixed value (#32415)
The error tests for hex values previously used a random string of
digits, but this could be a valid hex value. This commit changes these
tests to use a fixed invalid hex value.

closes #32370
2018-07-30 10:00:55 -07:00
Boaz Leskes 0cae19c8d7
IndicesClusterStateService should replace an init. replica with an init. primary with the same aId (#32374)
In rare cases it is possible that a nodes gets an instruction to replace a replica
shard that's in `POST_RECOVERY` with a new initializing primary with the same allocation id.
This can happen by batching cluster states that include the starting of the replica, with
closing of the indices, opening it up again and allocating the primary shard to the node in
question. The node should then clean it's initializing replica and replace it with a new
initializing primary.

I'm not sure whether the test I added really adds enough value as existing tests found this. The main reason I added is to allow for simpler reproduction and to double check I fixed it. I'm open to discuss if we should keep.

Closes #32308
2018-07-30 16:24:41 +03:00
Luca Cavanna 9a4d0069f6
REST high-level client: parse back _ignored meta field (#32362)
`GetResult` and `SearchHit` have been adjusted to parse back the `_ignored` meta field whenever it gets printed out. Expanded the existing tests to make sure this is covered. Fixed also a small problem around highlighted fields in `SearchHitTests`.
2018-07-30 13:43:40 +02:00
David Roberts 14a4a740ac [CI] Mute DocumentSubsetReaderTests testSearch
Relates #32457
2018-07-30 09:04:17 +01:00
Nhat Nguyen 5b1ad8099b
TEST: testDocStats should always use forceMerge (#32450)
Due to the recent change in LUCENE-8263, we need to adjust the deletion
ration to between 10% to 33% to preserve the current behavior of the
test. However, we may need another refinement if soft-deletes is enabled
as the actual deletes are different because of delete tombstones.

This commit prefers to always execute forceMerge instead of adjusting
the deletion ratio so that this test can focus on testing docStats.

Closes #32449
2018-07-28 07:41:30 -04:00
Nhat Nguyen 6e98615cc1 TEST: Avoid deletion in FlushIT
Due to the recent change in LUCENE-8263, a merge can be triggered if the
deletion ration is higher than 33%. An in-progress merge can prevent a
synced-flush from issuing.

This commit avoids deletes by using different docIds.

Closes #32436
2018-07-27 23:14:24 -04:00
Nhat Nguyen 139631c77d AwaitsFix IndexShardTests#testDocStats
Relates #32449
2018-07-27 20:48:23 -04:00
Jack Conradson e9e1095596
Painless: Add method type to method. (#32441)
MethodType can be computed at compile-time rather than run-time. This removes the 
method that collects MethodType at run-time from a PainlessMethod since is it no longer 
necessary.
2018-07-27 14:23:37 -07:00
Jason Tedor 588db621ac
Remove reference to non-existent store type (#32418)
We removed the default_fs store type yet the docs still contain a
reference to them. This commit addresses that by removing this
reference, and changing a reference to this section of the docs to
instead refer to mmapfs.
2018-07-27 11:24:03 -04:00
javanna dcb5d24639 [TEST] Mute failing FlushIT test
See #32436
2018-07-27 17:10:29 +02:00
Jason Tedor 3ac57f0ba3
Fix ordering of bootstrap checks in docs (#32417)
In the section of the bootstrap checks docs for the maximum map count
check, we refer to max size virtual memory check and explicitly call out
the maximum size virtual memory check as being the previous
point. However, this is not correct as the previous point is currently
the max file size check. It does make sense for these two checks to be
proximate to each other in the docs so this commit reorders the checks
so that the maximum size virtual memory check indeed comes before the
maximum map count check. This makes the sense in the maximum map count
check correct.
2018-07-27 10:40:16 -04:00
javanna 7aa5365497 [TEST] Mute failing InternalEngineTests#testSeqNoAndCheckpoints 2018-07-27 14:41:32 +02:00
javanna 83d007e7be [TEST] Mute failing testConvertLongHexError
See #32370
2018-07-27 11:50:13 +02:00
Jim Ferenczi 5decb23687 bump lucene version after backport 2018-07-27 10:50:22 +02:00
Jim Ferenczi 53ff06e621
Upgrade to Lucene-7.5.0-snapshot-608f0277b0 (#32390)
The main highlight is the removal of the reclaim_deletes_weight in the TieredMergePolicy.
The es setting index.merge.policy.reclaim_deletes_weight is deprecated in this commit and the value is ignored. The new merge policy setting setDeletesPctAllowed should be added in a follow up.
2018-07-27 08:28:51 +02:00
Yogesh Gaikwad 9198bcf78e
[Kerberos] Avoid vagrant update on precommit (#32416)
This commit avoids dependency during compile on copy keytab to
be present in the generated sources so pre-commit does not
stall for updating vagrant box.

Closes#32387
2018-07-27 12:01:52 +10:00
Armin Braun 1628c833c7
TESTS: Move netty leak detection to paranoid level (#32354) 2018-07-26 21:36:49 +02:00
lcawl 67a884ec88 [DOCS] Fixes formatting of scope object in job resource 2018-07-26 12:28:26 -07:00
Jim Ferenczi 860f92fcdd
Copy missing segment attributes in getSegmentInfo (#32396)
The index sort and the attributes map of a segment are not copied
on committed segments that are not loaded by the internal or external searcher.
2018-07-26 20:29:27 +02:00
Jim Ferenczi 8e5f281b27
AbstractQueryTestCase should run without type less often (#28936)
This commit changes the randomization to always create an index with a type.
It also adds a way to create a query shard context that maps to an index with
no type registered in order to explicitely test cases where there is no type.
2018-07-26 20:29:05 +02:00
Armin Braun 57876bfeb9
INGEST: Fix Deprecation Warning in Script Proc. (#32407)
* Using short script form normalized to a map that used 'inline' instead of 'source' so a short form processor definition like:

```
{
 "script": "ctx.foo= 'bar'"
}
```

would always warn about the following deprecation:

```
  #! Deprecation: Deprecated field [inline] used, expected [source] 
```
2018-07-26 19:55:28 +02:00
Nik Everett 643235d46a
Switch x-pack/plugin to new style Requests (#32327)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/plugin` project to use the new versions.
2018-07-26 13:31:16 -04:00
Sandeep Kanabar 7ad16ffd84 Docs: Correcting a typo in tophits (#32359) 2018-07-26 13:30:01 -04:00
Nik Everett 0f783ce94b
Build: Stop double generating buildSrc pom (#32408)
When we added the `java-gradle-plugin` to `buildSrc` it added a second
task to generate the pom that duplicates the publishing work that we
configure in `BuildPlugin`. Not only does it dupliciate the pom, it
creates a pom that is missing things like `name` and `description` which
are required for publishing to maven central.

This change disables the duplicate pom generation.
2018-07-26 13:19:05 -04:00
Nhat Nguyen 0ed3458534 TEST: Avoid triggering merges in FlushIT
In testSyncedFlushSkipOutOfSyncReplicas, we reindex the extra documents
to all shards including the out-of-sync replica. However, reindexing to
that replica can trigger merges (due to the new deletes) which cause the
synced-flush failed. This test starts failing after we aggressively
trigger merges segments with a large number of deletes in LUCENE-8263.
2018-07-26 12:38:36 -04:00
Jack Conradson eb73dde7c8 Fix missing JavaDoc for @throws in several places in
KerberosTicketValidator.
2018-07-26 09:14:43 -07:00
Nik Everett 11a2189c6f
Switch x-pack full restart to new style Requests (#32294)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack:qa:full-cluster-restart` project to use
the new versions.
2018-07-26 12:11:31 -04:00
Tim Brooks 7a56df7c98
Release requests in cors handler (#32364)
There are two scenarios where a http request could terminate in the cors
handler. If that occurs, the requests need to be released. This commit
releases those requests.
2018-07-26 10:06:24 -06:00
Jack Conradson df579f8bce
Painless: Clean Up PainlessClass Variables (#32380)
Removes the variables name, clazz, and type as they are unnecessary. Renames 
staticMembers -> staticFields, members -> fields, getters -> getterMethodHandles, and 
setters -> setterMethodHandles.
2018-07-26 09:02:06 -07:00
Igor Motov 3b050194f9
Docs: Fix callouts in put license HL REST docs (#32363) 2018-07-26 09:00:35 -07:00
David Roberts 0afa265ac9
[ML] Consistent pattern for strict/lenient parser names (#32399)
Previously we had two patterns for naming of strict
and lenient parsers.

Some classes had CONFIG_PARSER and METADATA_PARSER,
and used an enum to pass the parser type to nested
parsers.

Other classes had STRICT_PARSER and LENIENT_PARSER
and used ternary operators to pass the parser type
to nested parsers.

This change makes all ML classes use the second of
the patterns described above.
2018-07-26 16:55:40 +01:00