Commit Graph

394 Commits

Author SHA1 Message Date
Simon Willnauer f5864c7291 Move away from `Settings#getAsMap()` (elastic/x-pack-elasticsearch#2661)
Relates to elastic/elasticsearch#26845

Original commit: elastic/x-pack-elasticsearch@0323ea07a5
2017-10-04 01:21:59 -06:00
Tal Levy 048418aca7 add dummy test to LicensingTribeIT for build to pass (elastic/x-pack-elasticsearch#2627)
The true purpose of this test is to introduce another test alongside
the original, so that the test suite passes even if the other test
is skipped due to the assumption it makes about `build.snapshot`.

Original commit: elastic/x-pack-elasticsearch@709d7a5dc5
2017-09-25 22:03:46 -07:00
Simon Willnauer e7b5702f50 Adopt settings cleanups from core (elastic/x-pack-elasticsearch#2605)
Relates to elastic/elasticsearch#26739

Original commit: elastic/x-pack-elasticsearch@dd13d099de
2017-09-25 12:27:07 +02:00
Tal Levy d9554955f2 make tribe-node-license validation assume snapshot context (elastic/x-pack-elasticsearch#2589)
Release tests were introduced that sets the `build.snapshot`
system property to `false` to mimic release builds. This invalidates
the hardcoded license signatures that were signed against the
integration test pub/priv keys. This commit modifies the
license-validation assertions to assume the test is running against
those test keys, and will be skipped/ignored when these assertions
fail (which should only occur with `build.snapshot=true`)

Original commit: elastic/x-pack-elasticsearch@871704a3af
2017-09-21 14:39:01 -07:00
Tanguy Leroux 70687fbef3 [Tests] Add packaging tests for SSL/TLS communication (elastic/x-pack-elasticsearch#2556)
This commit adds a packaging test that uses the certgen tool
to set up a two nodes cluster that uses encrypted communication.

relates elastic/x-pack-elasticsearch#2485

Original commit: elastic/x-pack-elasticsearch@6d2e3c5cd0
2017-09-21 10:12:07 +02:00
Tanguy Leroux 0aef18333f Add packaging test for bootstrap password setup (elastic/x-pack-elasticsearch#2509)
relates elastic/x-pack-elasticsearch#2388

Original commit: elastic/x-pack-elasticsearch@cc750155d0
2017-09-19 10:07:39 +02:00
Tim Brooks b3914afd30 Reenable TribeWithSecurityIT tests (elastic/x-pack-elasticsearch#2511)
This is related to elastic/x-pack-elasticsearch#1996. These tests were disabled during the bootstrap
password work. They can now be reenabled. Additionally, I made the test
password used in tests consistent.

Original commit: elastic/x-pack-elasticsearch@5b490c8231
2017-09-15 12:50:54 -06:00
Jay Modi 57de66476c Disable TLS by default (elastic/x-pack-elasticsearch#2481)
This commit adds back the ability to disable TLS on the transport layer and also disables TLS by
default to restore the 5.x behavior. The auto generation of key/cert and bundled CA certificate
have also been removed.

Relates elastic/x-pack-elasticsearch#2463

Original commit: elastic/x-pack-elasticsearch@abc66ec67d
2017-09-14 12:18:54 -06:00
Jay Modi aaa0510821 Run core's full cluster restart tests with x-pack (elastic/x-pack-elasticsearch#2433)
This change pulls in the o.e.u.FullClusterRestartIT class from core and runs it as part of the
x-pack full cluster restart tests.

Relates elastic/x-pack-elasticsearch#1629

Original commit: elastic/x-pack-elasticsearch@87da59485f
2017-09-08 13:33:33 -06:00
David Kyle 51603620ee Mute ML rolling upgrade tests. Awaits fix elastic/x-pack-elasticsearch#1760
Original commit: elastic/x-pack-elasticsearch@deaf060818
2017-09-07 14:25:52 +01:00
David Roberts c73d70491a [TEST] Fix error if named pipe already connected (elastic/x-pack-elasticsearch#2423)
On Windows a named pipe server must call ConnectNamedPipe() before using
a named pipe.  However, if the client has already connected then this
function returns a failure code, with detailed error code
ERROR_PIPE_CONNECTED.  The server must check for this, as it means the
connection will work fine.  The Java test that emulates what the C++
would do in production did not have this logic.

This was purely a test problem.  The C++ code used in production already
does the right thing.

relates elastic/x-pack-elasticsearch#2359

Original commit: elastic/x-pack-elasticsearch@e162887f28
2017-09-05 13:39:22 +01:00
David Roberts 500b4ac6b9 [TEST] Improve ML security tests (elastic/x-pack-elasticsearch#2417)
The changes made for elastic/x-pack-elasticsearch#2369 showed that the ML security tests were seriously
weakened by the decision to grant many "minimal" privileges to all users
involved in the tests.  A better solution is to override the auth header
such that a superuser runs setup actions and assertions that work by
querying raw documents in ways that an end user wouldn't.  Then the ML
endpoints can be called with the privileges provided by the ML roles and
nothing else.

Original commit: elastic/x-pack-elasticsearch@4de42d9e54
2017-09-05 10:49:41 +01:00
David Roberts 32b4c18ea3 [ML] Ensure internal client is used where appropriate (elastic/x-pack-elasticsearch#2415)
Implementation details of ML endpoints should be performed using the
internal client, so that the end user only requires permissions for
the public ML endpoints and does not need to know how they are
implemented.  This change fixes some instances where this rule was
not adhered to.

Original commit: elastic/x-pack-elasticsearch@01c8f5172c
2017-09-01 13:16:48 +01:00
Tim Vernum 57a07d6b5a Authorize on shard requests for bulk actions (elastic/x-pack-elasticsearch#2369)
* Add support for authz checks at on shard requests

* Add Rest Tests for authorization

* Bulk security - Only reject individual items, rather than a whole shard

* Sync with core change

* Grant "delete" priv in ML smoketest

This role had index and+bulk privileges but it also needs delete (in order to delete ML model-snapshots)

Original commit: elastic/x-pack-elasticsearch@830e89e652
2017-08-31 11:49:46 -04:00
David Kyle 91635608ef [ML] Rolling upgrade test job configurations with empty strings (elastic/x-pack-elasticsearch#2333)
Original commit: elastic/x-pack-elasticsearch@b61947cca7
2017-08-30 15:20:14 +01:00
Jason Tedor 5cd92ffbbf Remove extraneous newlines from keystore.bash
This commit removes some extraneous trailing newlines from
keystore.bash, the packaging test cases for the interaction between
installing X-Pack and the keystore.

Original commit: elastic/x-pack-elasticsearch@86250ecfbc
2017-08-28 21:09:47 -04:00
Jason Tedor fb7118fde2 Add packaging tests for keystore creation
This commit adds a packaging test that the keystore is created when
X-Pack is installed, and that it has the correct ownership and
permissions.

Relates elastic/x-pack-elasticsearch#2380

Original commit: elastic/x-pack-elasticsearch@27e181d2f6
2017-08-28 20:47:58 -04:00
Michael Basnight e18f04f3eb Revert "Use shaded rest client dependencies" (elastic/x-pack-elasticsearch#2352)
This reverts commit elastic/x-pack-elasticsearch@8605560232.

Relates elastic/elasticsearch#26367

Original commit: elastic/x-pack-elasticsearch@e4cd960504
2017-08-25 14:13:16 -05:00
Alexander Reelsen 9b0a1a34e0 Upgrade: Remove watcher/security upgrade checks (elastic/x-pack-elasticsearch#2338)
The checks are used for the 5.6 to 6.x transition, thus they do
not make sense to keep in 7.x.

Original commit: elastic/x-pack-elasticsearch@c6c6fa819e
2017-08-25 17:24:49 +02:00
David Kyle c6b6a5c804 Fix failing ML test after bucket count change (elastic/x-pack-elasticsearch#2351)
Original commit: elastic/x-pack-elasticsearch@c215ba1c16
2017-08-25 10:25:18 +01:00
Albert Zaharovits 026729e911 TOKEN_SERVICE_ENABLED_SETTING enabled if HTTP_SSL_ENABLED (elastic/x-pack-elasticsearch#2321)
`authc.token.enabled` is true unless `http.ssl.enabled` is `false` and `http.enabled` is `true`.

* TokenService default enabled if HTTP_ENABLED == false

* Fixed tests that need TokenService explicitly enabled

* [DOC] Default value for `xpack.security.authc.token.enabled`

Original commit: elastic/x-pack-elasticsearch@bd154d16eb
2017-08-23 13:21:30 +03:00
Alexander Reelsen 3f541fa556 Tests: Ensure watcher is started via awaitBusy in bwc test
Original commit: elastic/x-pack-elasticsearch@a8c0cf04c9
2017-08-22 00:39:11 +02:00
Alexander Reelsen 17980ab360 Tests: Remove randomized unsupported code upgrading two indices at once
Original commit: elastic/x-pack-elasticsearch@c86b87927d
2017-08-22 00:09:01 +02:00
Alexander Reelsen 27f39c615b Watcher: Create two index ugprade checks for watcher upgrade (elastic/x-pack-elasticsearch#2298)
As there are two indices to upgrade for watcher, it makes a lot of sense
to also have two upgrade checks.

There is one upgrader for the watches index, which deletes
old templates, adds the new one before and then does the reindexing.
Same for the triggered watches index.

This also means, that there will be two entries popping up in the kibana
UI.

Note: Each upgrade check checks if the other index (for the .watches
upgrade check the triggered watches index and vice versa) is already
upgraded and only if that is true, watcher is restarted.

relates elastic/x-pack-elasticsearch#2238

Original commit: elastic/x-pack-elasticsearch@2c92040ed6
2017-08-21 17:36:16 +02:00
Simon Willnauer 71827b70a0 Bump token service BWC version to 6.0.0-beta2
Original commit: elastic/x-pack-elasticsearch@ef688f02cb
2017-08-18 17:06:45 +02:00
Simon Willnauer ac9ab974f4 Ensure token service can boostrap itself without a pre-shared key (elastic/x-pack-elasticsearch#2240)
Today we require a pre-shared key to use the token service. Beside the
additional setup step it doesn't allow for key-rotation which is a major downside.

This change adds a TokenService private ClusterState.Custom that is used to distribute
the keys used to encrypt tokens. It also has the infrastructur to add automatic key
rotation which is not in use yet but included here to illustrate how it can work down
the road.

This is considered a prototype and requires additioanl integration testing. Yet, it's fully
BWC with a rolling / full cluster restart from a previous version (also from 5.6 to 6.x)
since if the password is set it will just use it instead of generating a new one.
Once we implement the automatic key rotation via the clusterstate we need to ensure that we are
fully upgraded before we do that.
Also note that the ClusterState.Custom is fully transient and will never be serialized to disk.

Original commit: elastic/x-pack-elasticsearch@1ae22f5d41
2017-08-18 14:23:43 +02:00
Simon Willnauer 724325f161 Fallback to `keystore.seed` as a bootstrap password if actual password is not present (elastic/x-pack-elasticsearch#2295)
Today we require the `bootstrap.password` to be present in the keystore in order to
bootstrap xpack. With the addition of `keystore.seed` we have a randomly generated password
per node to do the bootstrapping. This will improve the initial user experience significantly
since the user doesn't need to create a keystore and add a password, they keystore is created
automatically unless already present and is always created with this random seed.

Relates to elastic/elasticsearch#26253

Original commit: elastic/x-pack-elasticsearch@5a984b4fd8
2017-08-17 16:42:32 +02:00
Alexander Reelsen 6d30806996 Watcher: Improvements on the rolling restart tests (elastic/x-pack-elasticsearch#2286)
This improves the rolling restart tests (tests different paths in
different ways) and aligns the upgrade code with the 5.6 branch from

Relates elastic/x-pack-elasticsearch#2238

Original commit: elastic/x-pack-elasticsearch@01b0954558
2017-08-17 11:41:11 +02:00
Alexander Reelsen 0a86f00d7e Tests: Ensure responses are closed in watch backwards compat tests
The HTTP client has to have it's response entities closed, otherwise
it might block further requests because the underlying connection cannot
be reused.

Relates elastic/x-pack-elasticsearch#2004

Original commit: elastic/x-pack-elasticsearch@a24ecb9764
2017-08-16 16:29:52 +02:00
Alexander Reelsen 69b3ffa40a Tests: Remove useless wait time in watcher REST test
The rest test waited for the watch to run in the background, but there
were no guarantees that this really happened. Also it waited for five
seconds, instead of just executing the watch manually.

relates elastic/x-pack-elasticsearch#2255

Original commit: elastic/x-pack-elasticsearch@56765a649e
2017-08-14 11:52:26 +02:00
Alexander Reelsen 3ad2f5e9f5 Tests: Increase logging for watcher smoke tests
These tests have repeating but not reproducible failures,
where the stash is filled with a second PUT operation and the
watcher stats response does not match. Setting the log to trace
should shed some light on this.

As the smoke tests are only four tests this will not lead to a
log explosion.

Relates elastic/x-pack-elasticsearch#1513, elastic/x-pack-elasticsearch#1874

Original commit: elastic/x-pack-elasticsearch@5832dc7990
2017-08-11 15:29:52 +02:00
Alexander Reelsen 26c5766a0d Tests: Do not delete index templates on bwc tests
In order to run the bwc tests there is no need to delete the
index template at the end of a test run which results
in recreation of those due to all the cluster state listener.

Relates elastic/x-pack-elasticsearch#2228

Original commit: elastic/x-pack-elasticsearch@702d1c61ed
2017-08-11 14:11:50 +02:00
Alexander Reelsen 2d08477093 Tests: Fix upgrade tests by not running stats against arbitrary hosts
Original commit: elastic/x-pack-elasticsearch@1b858aad52
2017-08-07 13:27:36 +02:00
Alexander Reelsen 8f6874abf9 Tests: Increase logging in watcher upgrade tests to debug test failures
Original commit: elastic/x-pack-elasticsearch@380a8541dc
2017-08-07 11:31:16 +02:00
David Roberts 05cbe8dc0c [ML] Disallow creating a job against a closed results or state index (elastic/x-pack-elasticsearch#2186)
Previously if this was attempted you'd get an NPE (5.x) or hang (6.x).
Following this change you get an error message telling you what the
problem is.

relates elastic/x-pack-elasticsearch#2170

Original commit: elastic/x-pack-elasticsearch@ea12a9ff46
2017-08-07 08:53:12 +01:00
Jay Modi 8b0fb5eae8 Re-enable OpenLDAP tests and run against vagrant instance (elastic/x-pack-elasticsearch#2121)
This commit re-enables the OpenLDAP tests that were previously running against a one-off instance
in AWS but now run against a vagrant fixture. There were some IntegTests that would run against the
OpenLDAP instance randomly but with this change they no longer run against OpenLDAP. This is ok as
the functionality that is tested by these has coverage elsewhere.

relates elastic/x-pack-elasticsearch#1823

Original commit: elastic/x-pack-elasticsearch@ac9bc82297
2017-08-04 09:44:08 -06:00
Jay Modi a7d6138f83 Fix the building of the default URL for the setup password tool (elastic/x-pack-elasticsearch#2176)
This commit fixes the building of the default URL for the setup password tool so that a default
elasticsearch.yml file will still result in a succesful run of the tool.

relates elastic/x-pack-elasticsearch#2174

Original commit: elastic/x-pack-elasticsearch@2291b14875
2017-08-03 15:14:24 -06:00
Ryan Ernst 45a55d16cf Bump master version to 7.0.0-alpha1 (elastic/x-pack-elasticsearch#2135)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/25876

Original commit: elastic/x-pack-elasticsearch@c86ea25009
2017-08-01 15:48:04 -04:00
Alexander Reelsen 4bf5d9536a Tests: Remove @ClusterScope tests, create REST tests (elastic/x-pack-elasticsearch#2131)
Replacing integration tests with rest tests and unit tests, thus removing integration tests that require start of a new cluster. Removing unused testing methods

Original commit: elastic/x-pack-elasticsearch@265966d80c
2017-08-01 14:15:36 +02:00
Jason Tedor 5b1bf9a31e Remove max script compilation settings in tests
Some standalone tests set the max script compilation limit. However,
this setting is now set in the main cluster setup in core so it is no
longer needed here. This commit removes these obviated settings in
standalone tests.

Original commit: elastic/x-pack-elasticsearch@089328c8d7
2017-08-01 14:57:57 +09:00
Jason Tedor 50b8a56d34 Add packaging assertions for new scripts
This commit adds packaging assertions for recent script additions:
setup-passwords and x-pack-env.

Original commit: elastic/x-pack-elasticsearch@d83624f419
2017-07-30 09:32:39 +09:00
Jay Modi db4c00b565 Update the full cluster restart tests to be more generic (elastic/x-pack-elasticsearch#2107)
The full cluster restart tests are currently geared towards the 5.6 -> 6.0 upgrade and have some
issues when the versions are changed to 6.x -> 7.0. One issue is a real code issue in that the
security code always expects the mappings to have the same version as the version of the node, but
we no longer update the mappings on the security index during a rolling upgrade. We know look at
the index format to determine if the index is up to date.

Original commit: elastic/x-pack-elasticsearch@14c1c72ff6
2017-07-28 10:31:44 -06:00
Jason Tedor 826df4eb72 Fix migrate tool integration test
The command line flag --path.conf was removed yet this test was still
using it. This commit replaces the usage of this flag in this test with
the new mechanism.

Original commit: elastic/x-pack-elasticsearch@c37050894c
2017-07-28 21:33:54 +09:00
David Kyle bb360c1ff3 Mute MigrateToolIT::testRunMigrateTool
Original commit: elastic/x-pack-elasticsearch@48cb30169a
2017-07-28 11:19:08 +01:00
Michael Basnight 6a7e51d9c0 Use shaded rest client dependencies
This commit modifies all org.apache.http to use the shaded rest clients
org.elasticsearch.client.http packages. It also removes a few unused
licenses due to the change.

Relates elastic/elasticsearch#25780

Original commit: elastic/x-pack-elasticsearch@8605560232
2017-07-24 12:56:17 -05:00
Ali Beyad 5190a05b75 Improves error message on non-upgraded security index (elastic/x-pack-elasticsearch#2061)
This commit improves the error message in 6x if the security index has
not been upgraded, and warns the user that the native realm will not be
functional until the upgrade API is run.

Original commit: elastic/x-pack-elasticsearch@710b7634b4
2017-07-21 15:56:15 -04:00
jaymode 9e14bff948 Test: increase number of script compilations per minute to 20
This commit increases the number of script compilations per minute to 20 to account for security
and watcher upgrade scripts in addition to the scripts contained within cluster alerts.

Relates elastic/x-pack-elasticsearch#2041

Original commit: elastic/x-pack-elasticsearch@6c15cb9b89
2017-07-21 13:49:22 -06:00
Alexander Reelsen 139513fdd3 Tests: Fix WatchBackwardsCompatibilityIT temporarily
the upgrade API is lacking some functionality in a special case,
where triggered_watches exists, but .watches does not. This
deletes the triggered watches index manually until we integrated
this properly in the upgrade API to fix the tests

Original commit: elastic/x-pack-elasticsearch@e9d1b0d35d
2017-07-18 18:00:55 +02:00
Ali Beyad 37cc602aef Adds upgrade API functionality for security (elastic/x-pack-elasticsearch#2012)
This commit adds the upgrade API functionality and script for security.
It also enables previously muted tests that would fail due to the lack
of security upgrade features in testing cluster restarts and old
security index backward compatibility.

Original commit: elastic/x-pack-elasticsearch@4abe9f1263
2017-07-18 11:44:28 -04:00
Alexander Reelsen 8200b18e9f Tests: Allow to run BWC in isolation
The current tests were only able to finish successfully, if the earlier
tests were run as well, you could not run the restart test in isolation.

This commit ensures an upgrade is executed if needed.

Original commit: elastic/x-pack-elasticsearch@616ebbd6eb
2017-07-18 15:29:07 +02:00