Commit Graph

39043 Commits

Author SHA1 Message Date
Luca Cavanna 2f4212b80a
Fold RestGetAllSettingsAction in RestGetSettingsAction (#30561)
We currently have a separate endpoint for retrieving settings from all indices. We introduced such endpoint when removing comma-separated feature parsing for GetIndicesAction. The RestGetAllSettingsAction duplicates the code to print out the response that we already have in GetSettingsResponse (since it became a ToXContentObject), and uses the get index API internally instead of the get settings API, but the response is the same, hence we can fold get all settings and get settings in a single API, which is what this commit does.
2018-05-14 19:56:50 +02:00
Jason Tedor 4a4e3d70d5
Default to one shard (#30539)
This commit changes the default out-of-the-box configuration for the
number of shards from five to one. We think this will help address a
common problem of oversharding. For users with time-based indices that
need a different default, this can be managed with index templates. For
users with non-time-based indices that find they need to re-shard with
the split API in place they no longer need to resort only to
reindexing.

Since this has the impact of changing the default number of shards used
in REST tests, we want to ensure that we still have coverage for issues
that could arise from multiple shards. As such, we randomize (rarely)
the default number of shards in REST tests to two. This is managed via a
global index template. However, some tests check the templates that are
in the cluster state during the test. Since this template is randomly
there, we need a way for tests to skip adding the template used to set
the number of shards to two. For this we add the default_shards feature
skip. To avoid having to write our docs in a complicated way because
sometimes they might be behind one shard, and sometimes they might be
behind two shards we apply the default_shards feature skip to all docs
tests. That is, these tests will always run with the default number of
shards (one).
2018-05-14 12:22:35 -04:00
Ioannis Kakavas af10fd65e7 Unmute IndexUpgradeIT tests
The errors were caused because release tests would use a copy of
the public key that was formatted differently. The change to the
public key format was introduced in [1].

Release tests Jenkins job has now been updated to use the correct
key format depending on the branch they run on [2]

Closes #30430

[1] https://github.com/elastic/elasticsearch/pull/30251
[2] https://github.com/elastic/infra/pull/4944
2018-05-14 18:53:27 +03:00
Christoph Büscher cc93131318
Forbid expensive query parts in ranking evaluation (#30151)
Currently the ranking evaluation API accepts the full query syntax for
the queries specified in the evaluation set and executes them via multi
search. This potentially runs costly aggregations and suggestions too.
This change adds checks that forbid using aggregations, suggesters, 
highlighters and the explain and profile options in the queries that are 
run as part of the ranking evaluation since they are irrelevent in the 
context of this API.
2018-05-14 17:36:26 +02:00
Nik Everett 41148e4bb1
Docs: Update HighLevelRestClient migration docs (#30544)
The High Level REST Client's documentation suggested that users should
use the Low Level REST Client for index management activities. This
change removes that suggestion because the high level REST client
supports those APIs now.

This also changes the examples in the migration docs to that still use
the Low Level REST Client to use the non-deprecated varieats of
`performRequest`.
2018-05-14 11:11:27 -04:00
Nik Everett b8bf480742
Clients: Switch to new performRequest (#30543)
Switch several calls in the client projects from the deprecated
`performRequest` calls to the new version.
2018-05-14 10:37:53 -04:00
Zachary Tong 1a7110524f [TEST] Fix typo in MovAvgIT test
The second set of assertions was accidentally using the count's
moving average for the error delta in the value's moving average
assertion.  This fixes the typo, and unmutes the test.

Closes #29456
2018-05-14 13:38:36 +00:00
Alpar Torok 9a5555963b
Add missing dependencies on testClasses (#30527) 2018-05-14 16:06:56 +03:00
David Roberts 6a8aa99e3f [TEST] Mute ML test that needs updating to following ml-cpp changes
Relates #30399
2018-05-14 12:49:37 +01:00
Yannick Welsch c96f2d7bf7
Document woes between auto-expand-replicas and allocation filtering (#30531)
Relates to #2869
2018-05-14 12:14:37 +02:00
Martijn van Groningen 7b95470897
Moved tokenizers to analysis common module (#30538)
The following tokenizers were moved: classic, edge_ngram,
letter, lowercase, ngram, path_hierarchy, pattern, thai, uax_url_email and
whitespace.

Left keyword tokenizer factory in server module, because
normalizers directly depend on it.This should be addressed on a
follow up change.

Relates to #23658
2018-05-14 07:55:01 +02:00
Jason Tedor 901436148b Adjust copy settings versions
This commit adjusts the versions on the copy settings behavior now
that the default behavior is configured in 7.0.0.
2018-05-13 22:23:13 -04:00
Nhat Nguyen 73ec90f1b9 Mute ShrinkIndexIT suite
Relates #30416
2018-05-13 15:29:31 -04:00
Costin Leau 8dbe9198a1
SQL: SYS TABLES ordered according to *DBC specs (#30530)
To obey the *DBC specs, SYS TABLES returns information sorted by type
first and name second
2018-05-13 17:55:54 +03:00
Jason Tedor 593fdd40ed
Deprecate not copy settings and explicitly disallow (#30404)
We want copying settings to be the default behavior. This commit
deprecates not copying settings, and disallows explicitly not copying
settings. This gives users a transition path to the future default
behavior.
2018-05-13 10:30:05 -04:00
David Kyle 9dd629648d [ML] Improve state persistence log message 2018-05-12 09:20:08 +01:00
Ryan Ernst be8c094e8c
Build: Add mavenPlugin cluster configuration method (#30541)
This commit adds the ability to specify a plugin from maven for a
test cluster to use. Currently, only local projects may be used as
plugins, except when testing bwc, where the coordinates of the project
are used. However, that assumes all projects always keep the same
coordinates, or are even still plugins, which is no longer the case for
x-pack. The full cluster and rolling restart tests are changed to use
this new method when pulling x-pack versions before 6.3.0.
2018-05-11 21:58:18 -07:00
Nhat Nguyen 4c130a1054 Re-enable FlushIT tests
These tests failed due to in flight operations on the primary shard.
Sadly, we don't have any clue on those ops. This commit unmutes
these tests and logs the acquirers when checking for ongoing ops.

1> [2018-05-02T23:10:32,145][INFO ][o.e.i.f.FlushIT          ] Third
seal: Total shards: [2], failed: [true], reason: [[1] ongoing operations
on primary], detail: []

Relates #29392
2018-05-11 22:23:52 -04:00
Jason Tedor 07b962f31a
Bump Gradle heap to 2 GB (#30535)
We are still seeing rare failures with the Gradle heap set to 1792m,
especially on machines with high core count. Given it appears we are
close to the needed threshold, this commit bumps the heap one more time
to 2 GB.
2018-05-11 14:30:36 -04:00
Nik Everett 069fec83a8
SQL: Use request flavored methods in tests (#30345)
Modifies the SQL tests to use the new `Request` object flavored methods
introduced onto the `RestClient` in #29623. We'd like to remove the old
methods eventually so we should stop using them.
2018-05-11 13:10:01 -04:00
David Turner 15df911e41
Suppress hdfsFixture if there are spaces in the path (#30302)
HDFS sets its thread-name format based partly on a URL-encoded version of the
path, but the URL-encoding of spaces as `%20` is interpreted as a field in the
formatted string of type `2`, which is nonsensical. This change simply skips 
these tests in this case.
2018-05-11 13:36:31 +01:00
Yannick Welsch 323bcd84a0
Delete temporary blobs before creating index file (#30528)
Fixes an (un-released) bug introduced in #30332.

Closes #30507
2018-05-11 14:34:11 +02:00
Alexander Reelsen ed2d427c48
Watcher: Remove TriggerEngine.getJobCount() (#30395)
Since adding back the per-watch statistics, we do not need to access
every trigger engine implementation to get the current total job count.
This commit removes the unused methods to do so.
2018-05-11 10:04:58 +02:00
David Kyle 0ff8cf48e8
[ML] Fix wire BWC for JobUpdate (#30512)
Fix wire BWC for the JobUpdate class

Hide JobUpdate internal fields from the REST request parser
2018-05-11 09:03:45 +01:00
Yannick Welsch cdcd4a1129
Use simpler write-once semantics for FS repository (#30435)
The writeBlob method for FsBlobContainer already opens the file with StandardOpenOption.CREATE_NEW, so there's no need for an extra blobExists(blobName) check.
2018-05-11 10:02:07 +02:00
Daniel Mitterdorfer 09cf530f4b
Derive max composite buffers from max content len
With this commit we determine the maximum number of buffers that Netty keeps
while accumulating one HTTP request based on the maximum content length (default
1500 bytes, overridable with the system property `es.net.mtu`). Previously, we
kept the default value of 1024 which is too small for bulk requests which leads
to unnecessary copies of byte buffers internally.

Relates #29448
2018-05-11 10:01:09 +02:00
Yannick Welsch fc870fdb4c
Use simpler write-once semantics for HDFS repository (#30439)
There's no need for an extra `blobExists()` call when writing a blob to the HDFS service. The writeBlob implementation for the HDFS repository already uses the `CreateFlag.CREATE` option on the file creation, which ensures that the blob that's uploaded does not already exist. This saves one network roundtrip.
2018-05-11 09:50:37 +02:00
Costin Leau 2594c1fb38
SQL: Improve correctness of SYS COLUMNS & TYPES (#30418)
Tweak the return data, in particular with regards for ODBC columns to
better align with the spec
Fix order for SYS TYPES and TABLES according to the JDBC/ODBC spec

Fix #30386
Fix #30521
2018-05-11 10:17:01 +03:00
Julie Tibshirani 73b08d937b Mute two tests in FlushIT with @AwaitsFix.
The issue is being tracked in #29392.
2018-05-10 23:16:39 -07:00
Ryan Ernst a82acac3e7 Fix incorrect template name in test case 2018-05-10 22:34:38 -07:00
Ryan Ernst e9d898a7f4
Build: Remove legacy bwc files from xpack (#30485)
These bwc build files were leftover from before xpack was opened.
2018-05-10 22:23:02 -07:00
Julie Tibshirani 6129d88e07 Mute UnicastZenPingTests#testSimplePings with @AwaitsFix.
This failure is being tracked in #28685.
2018-05-10 14:21:18 -07:00
Jay Modi 5039b9bcb3
Security: cleanup code in file stores (#30348)
This commit cleans up some code in the FileUserPasswdStore and the
FileUserRolesStore classes. The maps used in these classes are volatile
so we need to make sure that we don't perform multiple operations with
the map unless we are sure we are using a reference to the same map.

The maps are also never null, but there were a few null checks in the
code that were not needed. These checks have been removed.
2018-05-10 13:28:19 -06:00
Jay Modi f733de8e67
Security: fix TokenMetaData equals and hashcode (#30347)
The TokenMetaData equals method compared byte arrays using `.equals` on
the arrays themselves, which is the equivalent of an `==` check. This
means that a seperate byte[] with the same contents would not be
considered equivalent to the existing one, even though it should be.

The method has been updated to use `Array#equals` and similarly the
hashcode method has been updated to call `Arrays#hashCode` instead of
calling hashcode on the array itself.
2018-05-10 13:12:11 -06:00
Julie Tibshirani 66ef5550ce Mute two tests from SmokeTestWatcherWithSecurityClientYamlTestSuiteIT.
These tests are both in the file `watcher/stats/10_basic`, and have been
failing fairly frequently over the last month with a start-up issue.

The issue is being tracked in #30298.
2018-05-10 11:26:18 -07:00
Julie Tibshirani 1112fac206 Mute SharedClusterSnapshotRestoreIT#testSnapshotSucceedsAfterSnapshotFailure with @AwaitsFix.
The issue is being tracked in #30507.
2018-05-10 10:20:11 -07:00
Costin Leau 293ca92e93
SQL: Improve compatibility with MS query (#30516)
Support TABLE as a legacy argument for SYS TABLE commands

Fix #30398
2018-05-10 20:15:50 +03:00
Costin Leau 52580b5ca8
SQL: Fix parsing of dates with milliseconds (#30419)
Dates internally contain milliseconds (which appear when converting them
to Strings) however parsing does not accept them (and is being strict).
The parser has been changed so that Date is mandatory but the time
(including its fractions such as millis) are optional.

Fix #30002
2018-05-10 20:14:54 +03:00
Nhat Nguyen 519768b5d3
Upgrade to Lucene-7.4-snapshot-6705632810 (#30519)
This snapshot is to include LUCENE-8298 which allows DocValues updates
to reset a value. This is needed for the Lucene rollback work.
2018-05-10 12:31:45 -04:00
Paul Sanwald e79894aa52
add version compatibility from 6.4.0 after backport, see #30319 (#30390) 2018-05-10 12:27:44 -04:00
Ryan Ernst 8d1756cd12
Security: Simplify security index listeners (#30466)
This commit adds a general state listener to the SecurityIndexManager,
and replaces the existing health and up-to-date listeners with that. It
also moves helper methods relating to health to SecurityIndexManager
from SecurityLifecycleService.
2018-05-10 08:23:29 -07:00
Igor Motov 2a79d9234b
Add proper longitude validation in geo_polygon_query (#30497)
Fixes longitude validation in geo_polygon_query builder. The queries
with wrong longitude currently fail but only later during polygon
with quite complicated error message.

Fixes #30488
2018-05-10 11:14:08 -04:00
David Turner df17f85e14
Remove Discovery.AckListener.onTimeout() (#30514)
The MasterService takes responsibility for timeouts of the AckListeners that it
creates, and the rest of the Discovery subsystem is unaware of these timeouts,
so there's no need for this to appear in the Discovery.AckListener interface.

Also fix a typo in the name of DelegatingAckListener.
2018-05-10 15:27:38 +01:00
Jay Modi 37bb8f8075
Build: move generated-resources to build (#30366)
This commit moves the generated-resources directory to be within
the build directory for the openldap-tests and saml-idp-tests
projects. Both projects create a generated-resources directory that
should have been in the build directory but were instead at the same
level as the build directory.
2018-05-10 07:35:23 -06:00
Nik Everett 51fa8739ea
Reindex: Fold "with all deps" project into reindex (#30154)
This folds the `:qa:smoke-test-reindex-with-all-modules` project into
`:modules:reindex` by declaring the reindex's integration testing
cluster requires the `parent-join` and `lang-painless` plugins and then
moving all of the integration tests that depended on parent-join and
painless into reindex.

It saves us one cluster start up during the build at the cost of a
little of the reindex module's "purity". Since the reindex module *does*
have unit tests that test scripting without painless I'm fairly ok with
that.
2018-05-10 08:02:23 -04:00
Jason Tedor 596b262b55
Isolate REST client single host tests (#30504)
These tests are sharing the same server and client for every test. Yet,
we are seeing some tests fail with mysterious connection resets. It is
not clear what is happening but one theory is that the tests are
interfering with each other. This commit moves to use a separate server
and client per test.
2018-05-10 07:58:02 -04:00
Alpar Torok 84660ffd5f
Solve Gradle deprecation warnings around shadowJar (#30483)
* Solve Gradle deprecation warnings around shadowJar

- Apply workaround for: johnrengelman/shadow#336
- bump plugin to 2.0.4

Changes between 2.0.2 and 2.0.4 of the plugin:
```
477db40 12 days ago john.engelman@target.com Release 2.0.4
3e3da37 3 weeks ago john.engelman@target.com Remove internal Gradle API and annotation internal getters on shadow jar.
31e2380 3 weeks ago john.engelman@target.com Close input streams. Closes #364
f712cc8 3 weeks ago john.engelman@target.com Upgrade ASM to 6.1.1 to address perf issues. Closes #374
2f94b2b 3 weeks ago john.engelman@target.com next version
23bbf3d 7 weeks ago john.r.engelman@gmail.com Add some gradle versions. Update changelog for 2.0.3
7435c74 7 weeks ago john.r.engelman@gmail.com Merge pull request #367 from ttsiebzehntt/366-java10
325c002 7 weeks ago info@martinsadowski.de Update ASM to 6.1
94550e5 3 months ago john.r.engelman@gmail.com Merge pull request #356 from sgnewson/update-file-to-files
66b691e 4 months ago john.r.engelman@gmail.com Merge pull request #358 from 3flex/patch-1
14761b1 4 months ago 3flex@users.noreply.github.com fix markdown for User Guide URL in issue template
a3f6984 4 months ago newson@synopsys.com update inputs.file to inputs.files, to remove warning
```

closes #30389

* Improove comment as suggested
2018-05-10 12:49:41 +03:00
Ioannis Kakavas 4b319d7151
SAML: Process only signed data (#30420)
As conformance to best practices, this changes ensures that if a
SAML Response is signed, we verify the signature before processing
it any further. We were only checking the InResponseTo and
Destination attributes before potential signature validation but
there was no reason to do that up front either.
2018-05-10 11:42:22 +03:00
Jason Tedor bf2365d13b
Remove BWC repository test (#30500)
This commit removes a test that we can not restore from 1.x and 2.x
repository files. This test is not needed, the version of Elasticsearch
that this commit targets can not even read index files from those
versions.
2018-05-09 23:24:54 -04:00
Ryan Ernst bd24caccaf
Build: Remove xpack specific run task (#30487)
With the opening of xpack, we still retained a run task within
:x-pack:plugin. However, the root level run task also runs with the
default distribution. This change removes the extra run task inside
xpack in favor of using the root level task, and moves the
license/configuration code for run into the main run configuration.
2018-05-09 18:46:14 -07:00