Commit Graph

152 Commits

Author SHA1 Message Date
Christoph Büscher 712473b558
[Docs] Replace InetSocketTransportAddress with TransportAdress (#30673)
The former class has been removed in 6.0, the documentation code
snippets should be updated accordingly.
2018-05-17 14:23:08 +02:00
David Roberts ef0daee850
[TEST] Account for increase in ML C++ memory usage (#30675)
Recent changes to the ML C++ have resulted in higher memory usage,
so fewer "by" fields can be analyzed in a given amount of model
memory.
2018-05-17 12:59:20 +01:00
Tim Vernum 9f824c4aa8
Add detailed assert message to IndexAuditUpgradeIT (#30669)
Print out the returned buckets if the size does not match the
expectation.
2018-05-17 21:36:13 +10:00
Ioannis Kakavas 35fa934971
Adjust fast forward for token expiration test (#30668)
Adjust fast forward for token expiration test

Adjusts the maximum fast forward time for token expiration tests
to be 5 seconds before actual token expiration so that the test
won't fail even when upperlimit is randomly selected.

Resolves: #30062
2018-05-17 14:10:49 +03:00
Alexander Reelsen 11d776ecf0
Watcher: Fix watch history template for dynamic slack attachments (#30172)
The part of the history template responsible for slack attachments had a
dynamic mapping configured which could lead to problems, when a string
value looking like a date was configured in the value field of an
attachment.

This commit fixes the template by setting this field always to text.
This also requires a change in the template numbering to be sure this
will be applied properly when starting watcher.
2018-05-17 11:57:54 +02:00
Ryan Ernst a4c9c2fa2a
Make xpack modules instead of a meta plugin (#30589)
This commit removes xpack from being a meta-plugin-as-a-module.
It also fixes a couple tests which were missing task dependencies, which
failed once the gradle execution order changed.
2018-05-16 15:35:57 -07:00
Dimitris Athanasiou 01bdfcde6f
[ML] DeleteExpiredDataAction should use client with origin (#30646)
This is an admin action that should be allowed to operate on
ML indices with full permissions.
2018-05-16 23:35:23 +03:00
lcawl 8ff9baeb82 [DOCS] Fixes list of unconverted snippets in build.gradle 2018-05-16 13:11:06 -07:00
lcawl 4f41018753 [DOCS] Reorganizes RBAC documentation 2018-05-16 12:22:52 -07:00
Igor Motov 7b6d0faec2
SQL: Remove dependency for server's version from JDBC driver (#30631)
Removes dependency for server's version from the JDBC driver code. This
should allow us to dramatically reduce driver's size by removing the
server dependency from the driver.

Relates #29856
2018-05-16 15:07:14 -04:00
jaymode d912fa4357
Test: increase search logging for LicensingTests
This commit increases the logging level around search to aid in
debugging failures in LicensingTests#testSecurityActionsByLicenseType
where we are seeing all shards failed error while trying to search the
security index.

See #30301
2018-05-16 11:59:49 -06:00
Van0SS 4478f10a2a Rest High Level client: Add List Tasks (#29546)
This change adds a `listTasks` method to the high level java
ClusterClient which allows listing running tasks through the 
task management API.

Related to #27205
2018-05-16 13:31:37 +02:00
Colin Goodheart-Smithe a75b8adce5
Refactors ClientHelper to combine header logic (#30620)
* Refactors ClientHelper to combine header logic

This change removes all the `*ClientHelper` classes which were
repeating logic between plugins and instead adds
`ClientHelper.executeWithHeaders()` and
`ClientHelper.executeWithHeadersAsync()` methods to centralise the
logic for executing requests with stored security headers.

* Removes Watcher headers constant
2018-05-16 11:38:24 +01:00
David Kyle 16f5a515f3
[ML] Wait for ML indices in rolling upgrade tests (#30615) 2018-05-16 09:52:25 +01:00
Alexander Reelsen 6ec5e4408c
Watcher: Ensure secrets integration tests also run triggered watch (#30478)
When the encrpytion of sensitive date is enabled, test that a
scheduled watch is executed as expected and produces the correct value
from a secret in the basic auth header.
2018-05-16 10:32:22 +02:00
Ioannis Kakavas 2b09e90237
Replace custom reloadable Key/TrustManager (#30509)
Make SSLContext reloadable

This commit replaces all customKeyManagers and TrustManagers 
(ReloadableKeyManager,ReloadableTrustManager, 
EmptyKeyManager, EmptyTrustManager) with instances of 
X509ExtendedKeyManager and X509ExtendedTrustManager. 
This change was triggered by the effort to allow Elasticsearch to 
run in a FIPS-140 environment. In JVMs running in FIPS approved 
mode, only SunJSSE TrustManagers and KeyManagers can be used. 
Reloadability is now ensured by a volatile instance of SSLContext
in SSLContectHolder.
SSLConfigurationReloaderTests use the reloadable SSLContext to
initialize HTTP Clients and Servers and use these for testing the
key material and trust relations.
2018-05-16 08:32:13 +03:00
Ryan Ernst c7d82b378b
Build: Add task interdependencies for ssl configuration (#30633)
This commit fixes the tasks creating ssl certs for tests to have correct
dependsOn to ensure the right tasks are run before tests run.
2018-05-15 16:09:15 -07:00
Tim Brooks 99b9ab58e2
Add nio http server transport (#29587)
This commit is related to #28898. It adds an nio driven http server
transport. Currently it only supports basic http features. Cors,
pipeling, and read timeouts will need to be added in future PRs.
2018-05-15 16:37:14 -06:00
Jason Tedor abc06d5b79
Expose master version in REST test context (#30623)
This commit exposes the master version to the REST test context. This
will be needed in a follow-up where the master version will be used to
determine whether or not a certain warning header is expected.
2018-05-15 17:26:43 -04:00
Costin Leau 03dd2ab499 SQL: eliminate disabled tests 2018-05-15 22:49:05 +03:00
Costin Leau 09329eb84f
SQL: Verify GROUP BY ordering on grouped columns (#30585)
Due to the way composite aggregation works, ordering in GROUP BY can be
applied only through grouped columns which now the analyzer verifier
enforces.

Fix 29900
2018-05-15 22:46:46 +03:00
Ryan Ernst 21b9170dec
Security: Remove SecurityLifecycleService (#30526)
This commit removes the SecurityLifecycleService, relegating its former
functions of listening for cluster state updates to SecurityIndexManager
and IndexAuditTrail.
2018-05-15 12:13:24 -07:00
Ioannis Kakavas 17d65c1f06 Unmute IndexUpgradeIT tests
After additional fixes to the Jenkins job, these tests can now be
unmuted.

Closes #30430
2018-05-15 13:48:15 +03:00
Yannick Welsch af2b9dd779 Revert "Mute ML upgrade test (#30458)"
This reverts commit 4b36ea7433.
2018-05-15 11:20:57 +02:00
David Roberts 1de5a3180a [ML] Adjust BWC version following backport of #30125 2018-05-15 09:51:56 +01:00
David Roberts 50c34b2a9b
[ML] Reverse engineer Grok patterns from categorization results (#30125)
This change adds a grok_pattern field to the GET categories API
output in ML. It's calculated using the regex and examples in the
categorization result, and applying a list of candidate Grok
patterns to the bits in between the tokens that are considered to
define the category.

This can currently be considered a prototype, as the Grok patterns
it produces are not optimal. However, enough people have said it
would be useful for it to be worthwhile exposing it as experimental
functionality for interested parties to try out.
2018-05-15 09:02:38 +01:00
Tim Vernum 7dd816e77c Update build file due to doc file rename
A file with uncoverted snippets was change as part of 7f47ff9, but
build.gradle was not updated to reflect the rename.
2018-05-15 14:28:29 +10:00
Tim Brooks 848f240926
Fix issue with finishing handshake in ssl driver (#30580)
This is fixing an issue that has come up in some builds. In some
scenarios I see an assertion failure that we are trying to move to
application mode when we are not in handshake mode. What I think is
happening is that we are in handshake mode and have received the
completed handshake message AND an application message. While reading in
handshake mode we switch to application mode. However, there is still
data to be consumed so we attempt to continue to read in handshake mode.
This leads to us attempting to move to application mode again throwing
an assertion.

This commit fixes this by immediatly exiting the handshake mode read
method if we are not longer in handshake mode. Additionally if we swap
modes during a read we attempt to read with the new mode to see if there
is data that needs to be handled.
2018-05-14 19:19:53 -06:00
Ioannis Kakavas 15790e1b56 Silence IndexUpgradeIT test failures. (#30430) 2018-05-15 02:14:35 +03:00
lcawl 7f47ff9fcd [DOCS] Fixes title capitalization in security content 2018-05-14 15:36:03 -07:00
Igor Motov 56d32bc8b2
SQL: Extract SQL request and response classes (#30457)
Extracts SQL request and response classes. This is the first step
towards creation of a small minimal dependencies jdbc driver.

Relates #29856
2018-05-14 16:43:29 -04:00
Lisa Cawley 7928270610
[DOCS] Fix realm setting names (#30499) 2018-05-14 13:13:26 -07:00
Lisa Cawley fa45c6c9a6
[DOCS] Fix path info for various security files (#30502) 2018-05-14 13:07:27 -07:00
Jason Tedor 4a4e3d70d5
Default to one shard (#30539)
This commit changes the default out-of-the-box configuration for the
number of shards from five to one. We think this will help address a
common problem of oversharding. For users with time-based indices that
need a different default, this can be managed with index templates. For
users with non-time-based indices that find they need to re-shard with
the split API in place they no longer need to resort only to
reindexing.

Since this has the impact of changing the default number of shards used
in REST tests, we want to ensure that we still have coverage for issues
that could arise from multiple shards. As such, we randomize (rarely)
the default number of shards in REST tests to two. This is managed via a
global index template. However, some tests check the templates that are
in the cluster state during the test. Since this template is randomly
there, we need a way for tests to skip adding the template used to set
the number of shards to two. For this we add the default_shards feature
skip. To avoid having to write our docs in a complicated way because
sometimes they might be behind one shard, and sometimes they might be
behind two shards we apply the default_shards feature skip to all docs
tests. That is, these tests will always run with the default number of
shards (one).
2018-05-14 12:22:35 -04:00
Ioannis Kakavas af10fd65e7 Unmute IndexUpgradeIT tests
The errors were caused because release tests would use a copy of
the public key that was formatted differently. The change to the
public key format was introduced in [1].

Release tests Jenkins job has now been updated to use the correct
key format depending on the branch they run on [2]

Closes #30430

[1] https://github.com/elastic/elasticsearch/pull/30251
[2] https://github.com/elastic/infra/pull/4944
2018-05-14 18:53:27 +03:00
David Roberts 6a8aa99e3f [TEST] Mute ML test that needs updating to following ml-cpp changes
Relates #30399
2018-05-14 12:49:37 +01:00
Costin Leau 8dbe9198a1
SQL: SYS TABLES ordered according to *DBC specs (#30530)
To obey the *DBC specs, SYS TABLES returns information sorted by type
first and name second
2018-05-13 17:55:54 +03:00
David Kyle 9dd629648d [ML] Improve state persistence log message 2018-05-12 09:20:08 +01:00
Ryan Ernst be8c094e8c
Build: Add mavenPlugin cluster configuration method (#30541)
This commit adds the ability to specify a plugin from maven for a
test cluster to use. Currently, only local projects may be used as
plugins, except when testing bwc, where the coordinates of the project
are used. However, that assumes all projects always keep the same
coordinates, or are even still plugins, which is no longer the case for
x-pack. The full cluster and rolling restart tests are changed to use
this new method when pulling x-pack versions before 6.3.0.
2018-05-11 21:58:18 -07:00
Nik Everett 069fec83a8
SQL: Use request flavored methods in tests (#30345)
Modifies the SQL tests to use the new `Request` object flavored methods
introduced onto the `RestClient` in #29623. We'd like to remove the old
methods eventually so we should stop using them.
2018-05-11 13:10:01 -04:00
Alexander Reelsen ed2d427c48
Watcher: Remove TriggerEngine.getJobCount() (#30395)
Since adding back the per-watch statistics, we do not need to access
every trigger engine implementation to get the current total job count.
This commit removes the unused methods to do so.
2018-05-11 10:04:58 +02:00
David Kyle 0ff8cf48e8
[ML] Fix wire BWC for JobUpdate (#30512)
Fix wire BWC for the JobUpdate class

Hide JobUpdate internal fields from the REST request parser
2018-05-11 09:03:45 +01:00
Costin Leau 2594c1fb38
SQL: Improve correctness of SYS COLUMNS & TYPES (#30418)
Tweak the return data, in particular with regards for ODBC columns to
better align with the spec
Fix order for SYS TYPES and TABLES according to the JDBC/ODBC spec

Fix #30386
Fix #30521
2018-05-11 10:17:01 +03:00
Ryan Ernst a82acac3e7 Fix incorrect template name in test case 2018-05-10 22:34:38 -07:00
Ryan Ernst e9d898a7f4
Build: Remove legacy bwc files from xpack (#30485)
These bwc build files were leftover from before xpack was opened.
2018-05-10 22:23:02 -07:00
Jay Modi 5039b9bcb3
Security: cleanup code in file stores (#30348)
This commit cleans up some code in the FileUserPasswdStore and the
FileUserRolesStore classes. The maps used in these classes are volatile
so we need to make sure that we don't perform multiple operations with
the map unless we are sure we are using a reference to the same map.

The maps are also never null, but there were a few null checks in the
code that were not needed. These checks have been removed.
2018-05-10 13:28:19 -06:00
Jay Modi f733de8e67
Security: fix TokenMetaData equals and hashcode (#30347)
The TokenMetaData equals method compared byte arrays using `.equals` on
the arrays themselves, which is the equivalent of an `==` check. This
means that a seperate byte[] with the same contents would not be
considered equivalent to the existing one, even though it should be.

The method has been updated to use `Array#equals` and similarly the
hashcode method has been updated to call `Arrays#hashCode` instead of
calling hashcode on the array itself.
2018-05-10 13:12:11 -06:00
Julie Tibshirani 66ef5550ce Mute two tests from SmokeTestWatcherWithSecurityClientYamlTestSuiteIT.
These tests are both in the file `watcher/stats/10_basic`, and have been
failing fairly frequently over the last month with a start-up issue.

The issue is being tracked in #30298.
2018-05-10 11:26:18 -07:00
Costin Leau 293ca92e93
SQL: Improve compatibility with MS query (#30516)
Support TABLE as a legacy argument for SYS TABLE commands

Fix #30398
2018-05-10 20:15:50 +03:00
Costin Leau 52580b5ca8
SQL: Fix parsing of dates with milliseconds (#30419)
Dates internally contain milliseconds (which appear when converting them
to Strings) however parsing does not accept them (and is being strict).
The parser has been changed so that Date is mandatory but the time
(including its fractions such as millis) are optional.

Fix #30002
2018-05-10 20:14:54 +03:00