Commit Graph

38976 Commits

Author SHA1 Message Date
Nhat Nguyen 4c130a1054 Re-enable FlushIT tests
These tests failed due to in flight operations on the primary shard.
Sadly, we don't have any clue on those ops. This commit unmutes
these tests and logs the acquirers when checking for ongoing ops.

1> [2018-05-02T23:10:32,145][INFO ][o.e.i.f.FlushIT          ] Third
seal: Total shards: [2], failed: [true], reason: [[1] ongoing operations
on primary], detail: []

Relates #29392
2018-05-11 22:23:52 -04:00
Jason Tedor 07b962f31a
Bump Gradle heap to 2 GB (#30535)
We are still seeing rare failures with the Gradle heap set to 1792m,
especially on machines with high core count. Given it appears we are
close to the needed threshold, this commit bumps the heap one more time
to 2 GB.
2018-05-11 14:30:36 -04:00
Nik Everett 069fec83a8
SQL: Use request flavored methods in tests (#30345)
Modifies the SQL tests to use the new `Request` object flavored methods
introduced onto the `RestClient` in #29623. We'd like to remove the old
methods eventually so we should stop using them.
2018-05-11 13:10:01 -04:00
David Turner 15df911e41
Suppress hdfsFixture if there are spaces in the path (#30302)
HDFS sets its thread-name format based partly on a URL-encoded version of the
path, but the URL-encoding of spaces as `%20` is interpreted as a field in the
formatted string of type `2`, which is nonsensical. This change simply skips 
these tests in this case.
2018-05-11 13:36:31 +01:00
Yannick Welsch 323bcd84a0
Delete temporary blobs before creating index file (#30528)
Fixes an (un-released) bug introduced in #30332.

Closes #30507
2018-05-11 14:34:11 +02:00
Alexander Reelsen ed2d427c48
Watcher: Remove TriggerEngine.getJobCount() (#30395)
Since adding back the per-watch statistics, we do not need to access
every trigger engine implementation to get the current total job count.
This commit removes the unused methods to do so.
2018-05-11 10:04:58 +02:00
David Kyle 0ff8cf48e8
[ML] Fix wire BWC for JobUpdate (#30512)
Fix wire BWC for the JobUpdate class

Hide JobUpdate internal fields from the REST request parser
2018-05-11 09:03:45 +01:00
Yannick Welsch cdcd4a1129
Use simpler write-once semantics for FS repository (#30435)
The writeBlob method for FsBlobContainer already opens the file with StandardOpenOption.CREATE_NEW, so there's no need for an extra blobExists(blobName) check.
2018-05-11 10:02:07 +02:00
Daniel Mitterdorfer 09cf530f4b
Derive max composite buffers from max content len
With this commit we determine the maximum number of buffers that Netty keeps
while accumulating one HTTP request based on the maximum content length (default
1500 bytes, overridable with the system property `es.net.mtu`). Previously, we
kept the default value of 1024 which is too small for bulk requests which leads
to unnecessary copies of byte buffers internally.

Relates #29448
2018-05-11 10:01:09 +02:00
Yannick Welsch fc870fdb4c
Use simpler write-once semantics for HDFS repository (#30439)
There's no need for an extra `blobExists()` call when writing a blob to the HDFS service. The writeBlob implementation for the HDFS repository already uses the `CreateFlag.CREATE` option on the file creation, which ensures that the blob that's uploaded does not already exist. This saves one network roundtrip.
2018-05-11 09:50:37 +02:00
Costin Leau 2594c1fb38
SQL: Improve correctness of SYS COLUMNS & TYPES (#30418)
Tweak the return data, in particular with regards for ODBC columns to
better align with the spec
Fix order for SYS TYPES and TABLES according to the JDBC/ODBC spec

Fix #30386
Fix #30521
2018-05-11 10:17:01 +03:00
Julie Tibshirani 73b08d937b Mute two tests in FlushIT with @AwaitsFix.
The issue is being tracked in #29392.
2018-05-10 23:16:39 -07:00
Ryan Ernst a82acac3e7 Fix incorrect template name in test case 2018-05-10 22:34:38 -07:00
Ryan Ernst e9d898a7f4
Build: Remove legacy bwc files from xpack (#30485)
These bwc build files were leftover from before xpack was opened.
2018-05-10 22:23:02 -07:00
Julie Tibshirani 6129d88e07 Mute UnicastZenPingTests#testSimplePings with @AwaitsFix.
This failure is being tracked in #28685.
2018-05-10 14:21:18 -07:00
Jay Modi 5039b9bcb3
Security: cleanup code in file stores (#30348)
This commit cleans up some code in the FileUserPasswdStore and the
FileUserRolesStore classes. The maps used in these classes are volatile
so we need to make sure that we don't perform multiple operations with
the map unless we are sure we are using a reference to the same map.

The maps are also never null, but there were a few null checks in the
code that were not needed. These checks have been removed.
2018-05-10 13:28:19 -06:00
Jay Modi f733de8e67
Security: fix TokenMetaData equals and hashcode (#30347)
The TokenMetaData equals method compared byte arrays using `.equals` on
the arrays themselves, which is the equivalent of an `==` check. This
means that a seperate byte[] with the same contents would not be
considered equivalent to the existing one, even though it should be.

The method has been updated to use `Array#equals` and similarly the
hashcode method has been updated to call `Arrays#hashCode` instead of
calling hashcode on the array itself.
2018-05-10 13:12:11 -06:00
Julie Tibshirani 66ef5550ce Mute two tests from SmokeTestWatcherWithSecurityClientYamlTestSuiteIT.
These tests are both in the file `watcher/stats/10_basic`, and have been
failing fairly frequently over the last month with a start-up issue.

The issue is being tracked in #30298.
2018-05-10 11:26:18 -07:00
Julie Tibshirani 1112fac206 Mute SharedClusterSnapshotRestoreIT#testSnapshotSucceedsAfterSnapshotFailure with @AwaitsFix.
The issue is being tracked in #30507.
2018-05-10 10:20:11 -07:00
Costin Leau 293ca92e93
SQL: Improve compatibility with MS query (#30516)
Support TABLE as a legacy argument for SYS TABLE commands

Fix #30398
2018-05-10 20:15:50 +03:00
Costin Leau 52580b5ca8
SQL: Fix parsing of dates with milliseconds (#30419)
Dates internally contain milliseconds (which appear when converting them
to Strings) however parsing does not accept them (and is being strict).
The parser has been changed so that Date is mandatory but the time
(including its fractions such as millis) are optional.

Fix #30002
2018-05-10 20:14:54 +03:00
Nhat Nguyen 519768b5d3
Upgrade to Lucene-7.4-snapshot-6705632810 (#30519)
This snapshot is to include LUCENE-8298 which allows DocValues updates
to reset a value. This is needed for the Lucene rollback work.
2018-05-10 12:31:45 -04:00
Paul Sanwald e79894aa52
add version compatibility from 6.4.0 after backport, see #30319 (#30390) 2018-05-10 12:27:44 -04:00
Ryan Ernst 8d1756cd12
Security: Simplify security index listeners (#30466)
This commit adds a general state listener to the SecurityIndexManager,
and replaces the existing health and up-to-date listeners with that. It
also moves helper methods relating to health to SecurityIndexManager
from SecurityLifecycleService.
2018-05-10 08:23:29 -07:00
Igor Motov 2a79d9234b
Add proper longitude validation in geo_polygon_query (#30497)
Fixes longitude validation in geo_polygon_query builder. The queries
with wrong longitude currently fail but only later during polygon
with quite complicated error message.

Fixes #30488
2018-05-10 11:14:08 -04:00
David Turner df17f85e14
Remove Discovery.AckListener.onTimeout() (#30514)
The MasterService takes responsibility for timeouts of the AckListeners that it
creates, and the rest of the Discovery subsystem is unaware of these timeouts,
so there's no need for this to appear in the Discovery.AckListener interface.

Also fix a typo in the name of DelegatingAckListener.
2018-05-10 15:27:38 +01:00
Jay Modi 37bb8f8075
Build: move generated-resources to build (#30366)
This commit moves the generated-resources directory to be within
the build directory for the openldap-tests and saml-idp-tests
projects. Both projects create a generated-resources directory that
should have been in the build directory but were instead at the same
level as the build directory.
2018-05-10 07:35:23 -06:00
Nik Everett 51fa8739ea
Reindex: Fold "with all deps" project into reindex (#30154)
This folds the `:qa:smoke-test-reindex-with-all-modules` project into
`:modules:reindex` by declaring the reindex's integration testing
cluster requires the `parent-join` and `lang-painless` plugins and then
moving all of the integration tests that depended on parent-join and
painless into reindex.

It saves us one cluster start up during the build at the cost of a
little of the reindex module's "purity". Since the reindex module *does*
have unit tests that test scripting without painless I'm fairly ok with
that.
2018-05-10 08:02:23 -04:00
Jason Tedor 596b262b55
Isolate REST client single host tests (#30504)
These tests are sharing the same server and client for every test. Yet,
we are seeing some tests fail with mysterious connection resets. It is
not clear what is happening but one theory is that the tests are
interfering with each other. This commit moves to use a separate server
and client per test.
2018-05-10 07:58:02 -04:00
Alpar Torok 84660ffd5f
Solve Gradle deprecation warnings around shadowJar (#30483)
* Solve Gradle deprecation warnings around shadowJar

- Apply workaround for: johnrengelman/shadow#336
- bump plugin to 2.0.4

Changes between 2.0.2 and 2.0.4 of the plugin:
```
477db40 12 days ago john.engelman@target.com Release 2.0.4
3e3da37 3 weeks ago john.engelman@target.com Remove internal Gradle API and annotation internal getters on shadow jar.
31e2380 3 weeks ago john.engelman@target.com Close input streams. Closes #364
f712cc8 3 weeks ago john.engelman@target.com Upgrade ASM to 6.1.1 to address perf issues. Closes #374
2f94b2b 3 weeks ago john.engelman@target.com next version
23bbf3d 7 weeks ago john.r.engelman@gmail.com Add some gradle versions. Update changelog for 2.0.3
7435c74 7 weeks ago john.r.engelman@gmail.com Merge pull request #367 from ttsiebzehntt/366-java10
325c002 7 weeks ago info@martinsadowski.de Update ASM to 6.1
94550e5 3 months ago john.r.engelman@gmail.com Merge pull request #356 from sgnewson/update-file-to-files
66b691e 4 months ago john.r.engelman@gmail.com Merge pull request #358 from 3flex/patch-1
14761b1 4 months ago 3flex@users.noreply.github.com fix markdown for User Guide URL in issue template
a3f6984 4 months ago newson@synopsys.com update inputs.file to inputs.files, to remove warning
```

closes #30389

* Improove comment as suggested
2018-05-10 12:49:41 +03:00
Ioannis Kakavas 4b319d7151
SAML: Process only signed data (#30420)
As conformance to best practices, this changes ensures that if a
SAML Response is signed, we verify the signature before processing
it any further. We were only checking the InResponseTo and
Destination attributes before potential signature validation but
there was no reason to do that up front either.
2018-05-10 11:42:22 +03:00
Jason Tedor bf2365d13b
Remove BWC repository test (#30500)
This commit removes a test that we can not restore from 1.x and 2.x
repository files. This test is not needed, the version of Elasticsearch
that this commit targets can not even read index files from those
versions.
2018-05-09 23:24:54 -04:00
Ryan Ernst bd24caccaf
Build: Remove xpack specific run task (#30487)
With the opening of xpack, we still retained a run task within
:x-pack:plugin. However, the root level run task also runs with the
default distribution. This change removes the extra run task inside
xpack in favor of using the root level task, and moves the
license/configuration code for run into the main run configuration.
2018-05-09 18:46:14 -07:00
Tal Levy 34f92df2d3 AwaitsFix IntegTestZipClientYamlTestSuiteIT#indices.split tests
there are two tests that have failed multiple times in one day on windows CI.

This commit AwaitsFixes them until their timeout issues are resolved.

tracking here: https://github.com/elastic/elasticsearch/issues/30503
2018-05-09 18:25:25 -07:00
Nik Everett b4502dbf74
LLClient: Add setJsonEntity (#30447)
Adds `Request#setJsonEntity(String)` which short circuits the process of
sending a json string which is super common.
2018-05-09 18:33:03 -04:00
Julie Tibshirani 9828e11709
Expose CommonStatsFlags directly in IndicesStatsRequest. (#30163)
This allows us to simplify the logic in a couple places where all flags need to be accessed.
2018-05-09 14:25:28 -07:00
Jack Conradson c87a3ea49b Silence IndexUpgradeIT test failures. (#30430) 2018-05-09 13:53:22 -07:00
Jason Tedor b33806ed15
Bump Gradle heap to 1792m (#30484)
After we stopped forking the compiler, some folks are running into out
of memory errors. This commit is a bump to the Gradle heap to workaround
these out of memory errors (until we can better understand their
source).
2018-05-09 14:29:29 -04:00
Mueed Chaudhry bf141a3fd1 [docs] add warning for read-write indices in force merge documentation (#28869) 2018-05-09 18:53:55 +02:00
Jason Tedor 4defaa4f2d
Avoid deadlocks in cache (#30461)
This commit avoids deadlocks in the cache by removing dangerous places
where we try to take the LRU lock while completing a future. Instead, we
block for the future to complete, and then execute the handling code
under the LRU lock (for example, eviction).
2018-05-09 11:52:38 -04:00
Jay Modi 143df3a51d
Test: remove hardcoded list of unconfigured ciphers (#30367)
This commit removes the hardcoded list of unconfigured ciphers in the
SslIntegrationTests. This list may include ciphers that are not
supported on certain JVMs. This list is replaced with code that
dynamically computes the set of ciphers that are not configured for
use by default.
2018-05-09 08:41:17 -06:00
Boaz Leskes 54122d8464 mute SplitIndexIT due to https://github.com/elastic/elasticsearch/issues/30416 2018-05-09 15:49:06 +02:00
Nik Everett f9dc86836d
Docs: Test examples that recreate lang analyzers (#29535)
We have a pile of documentation describing how to rebuild the built in
language analyzers and, previously, our documentation testing framework
made sure that the examples successfully built *an* analyzer but they
didn't assert that the analyzer built by the documentation matches the
built in anlayzer. Unsuprisingly, some of the examples aren't quite
right.

This adds a mechanism that tests that the analyzers built by the docs.
The mechanism is fairly simple and brutal but it seems to be working:
build a hundred random unicode sequences and send them through the
`_analyze` API with the rebuilt analyzer and then again through the
built in analyzer. Then make sure both APIs return the same results.
Each of these calls to `_anlayze` takes about 20ms on my laptop which
seems fine.
2018-05-09 09:23:10 -04:00
Yu 2228e6e663 BulkProcessor to retry based on status code (#29329)
Previously `BulkProcessor` retry logic was based on the exception type of the failed response (`EsRejectedExecutionException`). This commit changes it to be based on the returned status code. This allows us to reproduce the same retry behaviour when the `BulkProcessor` is used from the high-level REST client, which was previously not the case as we cannot rebuild the same exception type when parsing back the response. This change has no effect on the transport client.

Closes #28885
2018-05-09 14:27:58 +02:00
Michael Basnight 3b9c8204a6
Add GET Repository High Level REST API (#30362)
This commit adds the Snapshot Client with a first API call within it,
the get repositories call in snapshot/restore module. This also creates
a snapshot namespace for the docs, as well as get repositories docs.

Relates #27205
2018-05-09 07:25:23 -05:00
Boaz Leskes ad564240b1 add a comment explaining the need for RetryOnReplicaException on missing mappings 2018-05-09 14:19:50 +02:00
Yu 106bed90c7 Add `coordinating_only` node selector (#30313)
Today we can execute cluster API actions on only master, data or ingest nodes
using the `master:true`, `data:true` and `ingest:true` filters, but it is not
so easy to select coordinating-only nodes (i.e. those nodes that are neither
master nor data nor ingest nodes). This change fixes this by adding support for
a `coordinating_only` filter such that `coordinating_only:true` adds all
coordinating-only nodes to the set of selected nodes, and 
`coordinating_only:false` deletes them.

Resolves #28831.
2018-05-09 12:14:07 +01:00
Jason Tedor 92a0b0a00c
Stop forking groovyc (#30471)
This is a follow-up to our previous change to only fork javac if needed
to respect the Java compiler home (via JAVA_HOME). This commit makes the
same change for groovyc: we only fork groovyc if the JDK for Gradle is
not the JDK specified for the compiler (via JAVA_HOME).
2018-05-09 06:17:43 -04:00
Luca Cavanna ee6abef87b
Avoid setting connection request timeout (#30384)
We've been setting this value to 500ms in the default low-level REST
client configuration, misunderstanding the effect that it would have.
This proved very problematic, as it ends up causing `TimeoutException`
returned from the leased pool in some cases even for successful requests.

Closes #24069
2018-05-09 11:00:02 +02:00
Ke Li 0c6789bc72 Use date format in `date_range` mapping before fallback to default (#29310)
If the date format is not forced in query, use the format in mapping before 
fallback to the default format.

Closes #29282
2018-05-09 09:41:44 +02:00