Commit Graph

365 Commits

Author SHA1 Message Date
Alexander Reelsen 69b3ffa40a Tests: Remove useless wait time in watcher REST test
The rest test waited for the watch to run in the background, but there
were no guarantees that this really happened. Also it waited for five
seconds, instead of just executing the watch manually.

relates elastic/x-pack-elasticsearch#2255

Original commit: elastic/x-pack-elasticsearch@56765a649e
2017-08-14 11:52:26 +02:00
Alexander Reelsen 3ad2f5e9f5 Tests: Increase logging for watcher smoke tests
These tests have repeating but not reproducible failures,
where the stash is filled with a second PUT operation and the
watcher stats response does not match. Setting the log to trace
should shed some light on this.

As the smoke tests are only four tests this will not lead to a
log explosion.

Relates elastic/x-pack-elasticsearch#1513, elastic/x-pack-elasticsearch#1874

Original commit: elastic/x-pack-elasticsearch@5832dc7990
2017-08-11 15:29:52 +02:00
Alexander Reelsen 26c5766a0d Tests: Do not delete index templates on bwc tests
In order to run the bwc tests there is no need to delete the
index template at the end of a test run which results
in recreation of those due to all the cluster state listener.

Relates elastic/x-pack-elasticsearch#2228

Original commit: elastic/x-pack-elasticsearch@702d1c61ed
2017-08-11 14:11:50 +02:00
Alexander Reelsen 2d08477093 Tests: Fix upgrade tests by not running stats against arbitrary hosts
Original commit: elastic/x-pack-elasticsearch@1b858aad52
2017-08-07 13:27:36 +02:00
Alexander Reelsen 8f6874abf9 Tests: Increase logging in watcher upgrade tests to debug test failures
Original commit: elastic/x-pack-elasticsearch@380a8541dc
2017-08-07 11:31:16 +02:00
David Roberts 05cbe8dc0c [ML] Disallow creating a job against a closed results or state index (elastic/x-pack-elasticsearch#2186)
Previously if this was attempted you'd get an NPE (5.x) or hang (6.x).
Following this change you get an error message telling you what the
problem is.

relates elastic/x-pack-elasticsearch#2170

Original commit: elastic/x-pack-elasticsearch@ea12a9ff46
2017-08-07 08:53:12 +01:00
Jay Modi 8b0fb5eae8 Re-enable OpenLDAP tests and run against vagrant instance (elastic/x-pack-elasticsearch#2121)
This commit re-enables the OpenLDAP tests that were previously running against a one-off instance
in AWS but now run against a vagrant fixture. There were some IntegTests that would run against the
OpenLDAP instance randomly but with this change they no longer run against OpenLDAP. This is ok as
the functionality that is tested by these has coverage elsewhere.

relates elastic/x-pack-elasticsearch#1823

Original commit: elastic/x-pack-elasticsearch@ac9bc82297
2017-08-04 09:44:08 -06:00
Jay Modi a7d6138f83 Fix the building of the default URL for the setup password tool (elastic/x-pack-elasticsearch#2176)
This commit fixes the building of the default URL for the setup password tool so that a default
elasticsearch.yml file will still result in a succesful run of the tool.

relates elastic/x-pack-elasticsearch#2174

Original commit: elastic/x-pack-elasticsearch@2291b14875
2017-08-03 15:14:24 -06:00
Ryan Ernst 45a55d16cf Bump master version to 7.0.0-alpha1 (elastic/x-pack-elasticsearch#2135)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/25876

Original commit: elastic/x-pack-elasticsearch@c86ea25009
2017-08-01 15:48:04 -04:00
Alexander Reelsen 4bf5d9536a Tests: Remove @ClusterScope tests, create REST tests (elastic/x-pack-elasticsearch#2131)
Replacing integration tests with rest tests and unit tests, thus removing integration tests that require start of a new cluster. Removing unused testing methods

Original commit: elastic/x-pack-elasticsearch@265966d80c
2017-08-01 14:15:36 +02:00
Jason Tedor 5b1bf9a31e Remove max script compilation settings in tests
Some standalone tests set the max script compilation limit. However,
this setting is now set in the main cluster setup in core so it is no
longer needed here. This commit removes these obviated settings in
standalone tests.

Original commit: elastic/x-pack-elasticsearch@089328c8d7
2017-08-01 14:57:57 +09:00
Jason Tedor 50b8a56d34 Add packaging assertions for new scripts
This commit adds packaging assertions for recent script additions:
setup-passwords and x-pack-env.

Original commit: elastic/x-pack-elasticsearch@d83624f419
2017-07-30 09:32:39 +09:00
Jay Modi db4c00b565 Update the full cluster restart tests to be more generic (elastic/x-pack-elasticsearch#2107)
The full cluster restart tests are currently geared towards the 5.6 -> 6.0 upgrade and have some
issues when the versions are changed to 6.x -> 7.0. One issue is a real code issue in that the
security code always expects the mappings to have the same version as the version of the node, but
we no longer update the mappings on the security index during a rolling upgrade. We know look at
the index format to determine if the index is up to date.

Original commit: elastic/x-pack-elasticsearch@14c1c72ff6
2017-07-28 10:31:44 -06:00
Jason Tedor 826df4eb72 Fix migrate tool integration test
The command line flag --path.conf was removed yet this test was still
using it. This commit replaces the usage of this flag in this test with
the new mechanism.

Original commit: elastic/x-pack-elasticsearch@c37050894c
2017-07-28 21:33:54 +09:00
David Kyle bb360c1ff3 Mute MigrateToolIT::testRunMigrateTool
Original commit: elastic/x-pack-elasticsearch@48cb30169a
2017-07-28 11:19:08 +01:00
Michael Basnight 6a7e51d9c0 Use shaded rest client dependencies
This commit modifies all org.apache.http to use the shaded rest clients
org.elasticsearch.client.http packages. It also removes a few unused
licenses due to the change.

Relates elastic/elasticsearch#25780

Original commit: elastic/x-pack-elasticsearch@8605560232
2017-07-24 12:56:17 -05:00
Ali Beyad 5190a05b75 Improves error message on non-upgraded security index (elastic/x-pack-elasticsearch#2061)
This commit improves the error message in 6x if the security index has
not been upgraded, and warns the user that the native realm will not be
functional until the upgrade API is run.

Original commit: elastic/x-pack-elasticsearch@710b7634b4
2017-07-21 15:56:15 -04:00
jaymode 9e14bff948 Test: increase number of script compilations per minute to 20
This commit increases the number of script compilations per minute to 20 to account for security
and watcher upgrade scripts in addition to the scripts contained within cluster alerts.

Relates elastic/x-pack-elasticsearch#2041

Original commit: elastic/x-pack-elasticsearch@6c15cb9b89
2017-07-21 13:49:22 -06:00
Alexander Reelsen 139513fdd3 Tests: Fix WatchBackwardsCompatibilityIT temporarily
the upgrade API is lacking some functionality in a special case,
where triggered_watches exists, but .watches does not. This
deletes the triggered watches index manually until we integrated
this properly in the upgrade API to fix the tests

Original commit: elastic/x-pack-elasticsearch@e9d1b0d35d
2017-07-18 18:00:55 +02:00
Ali Beyad 37cc602aef Adds upgrade API functionality for security (elastic/x-pack-elasticsearch#2012)
This commit adds the upgrade API functionality and script for security.
It also enables previously muted tests that would fail due to the lack
of security upgrade features in testing cluster restarts and old
security index backward compatibility.

Original commit: elastic/x-pack-elasticsearch@4abe9f1263
2017-07-18 11:44:28 -04:00
Alexander Reelsen 8200b18e9f Tests: Allow to run BWC in isolation
The current tests were only able to finish successfully, if the earlier
tests were run as well, you could not run the restart test in isolation.

This commit ensures an upgrade is executed if needed.

Original commit: elastic/x-pack-elasticsearch@616ebbd6eb
2017-07-18 15:29:07 +02:00
Alexander Reelsen 443cfb94be Tests: Ensure waiting time between stopping and starting watcher (elastic/x-pack-elasticsearch#2008)
Otherwise we might run into race conditions that prevent a useful
start up again.

Those tests can be massively improved (no need to run against the real
master node), once the watcher BWC compatible stats are in.

relates elastic/x-pack-elasticsearch#2004

Original commit: elastic/x-pack-elasticsearch@52ca77809c
2017-07-18 10:19:33 +02:00
Tim Brooks 6d04eacdec Require elastic password be bootstrapped (elastic/x-pack-elasticsearch#1962)
This is related to elastic/x-pack-elasticsearch#1217. This commit requires that the elastic password
be bootstrapped for the user to be authenticated. As a result it removes
the special "setup" mode that allowed the user to be authenticated from
localhost.

Additionally, this commit updates the tests to work with this
functionality.

Original commit: elastic/x-pack-elasticsearch@d0d5d697a7
2017-07-13 19:59:50 -05:00
Igor Motov 4de6d9ebe5 Upgrade API: upgrade assistance shouldn't throw 404 on an empty cluster (elastic/x-pack-elasticsearch#1997)
When a user asks for upgrade information for all indices and there are no indices in the cluster, upgrade assistance should just return an empty response indicating that no indices require upgrade or reindexing. This commit also reverts the temporary fix in WatchBackwardsCompatibilityIT tests that was added as a workaround for this issue.

Original commit: elastic/x-pack-elasticsearch@2ea9707867
2017-07-13 17:01:50 -04:00
Alexander Reelsen eb118b365c Tests: Ignore 404 errors in watcher bwc tests
Original commit: elastic/x-pack-elasticsearch@8e583cf293
2017-07-13 13:41:51 +02:00
Alexander Reelsen 32bc0cd5d5 Tests: Add logging and error tracing to watcher bwc tests
Original commit: elastic/x-pack-elasticsearch@4a4b65bcb7
2017-07-13 11:08:48 +02:00
Ryan Ernst df41b8342d Convert put_template uses to put_script in rest tests
Original commit: elastic/x-pack-elasticsearch@5f5f52b955
2017-07-13 00:01:56 -07:00
Tim Vernum e4c8851a24 Convert Realm.authenticate to provide a richer result (elastic/x-pack-elasticsearch#1932)
This allows for messages to be returned, and distinguishes between 4 different results:
- I have authenticated the user
- I don't know how to authenticate that user. Try another realm.
- I tried to authenticate the user, but failed. Try another realm.
- I tried to authenticate the user, but failed. Fail the authentication attempt.

Original commit: elastic/x-pack-elasticsearch@f796949cfb
2017-07-13 14:24:08 +10:00
Simon Willnauer 84ee21ed26 Followup for elastic/elasticsearch#25658 (elastic/x-pack-elasticsearch#1984)
This is the xpack side fo elastic/elasticsearch#25658 which is mainly refactorings
of a ctor and added tests.

Original commit: elastic/x-pack-elasticsearch@d8e2a2a057
2017-07-12 22:19:34 +02:00
Alexander Reelsen e64cf23b13 Watcher: Start watcher on master node only with mixed versions (elastic/x-pack-elasticsearch#1983)
When there are data or master nodes in the cluster, that are older
than ES 6.0 alpha3, then watcher will only start on the master node.

Changed all transport actions to be master node actions, as there is
already a method to decide to run locally, which we can piggyback on.

Original commit: elastic/x-pack-elasticsearch@65cecb6d69
2017-07-12 20:58:47 +02:00
Jack Conradson 32dbfba0c2 Disallow lang to specified in requests where a stored script is used. (elastic/x-pack-elasticsearch#1949)
Requests that execute a stored script will no longer be allowed to specify the lang of the script. This information is stored in the cluster state making only an id necessary to execute against. Putting a stored script will still require a lang.

Original commit: elastic/x-pack-elasticsearch@926a7b2d86
2017-07-12 07:56:08 -07:00
Alexander Reelsen 14c12cfcde Revert "Tests: Disable all watcher BWC tests until fixed correctly"
This reverts commit elastic/x-pack-elasticsearch@8043ec1858.

Original commit: elastic/x-pack-elasticsearch@39180b095c
2017-07-11 00:40:05 +02:00
Alexander Reelsen 4dc13c3698 Tests: Disable all watcher BWC tests until fixed correctly
Original commit: elastic/x-pack-elasticsearch@8043ec1858
2017-07-10 23:12:19 +02:00
Igor Motov 37075bd201 Upgrade API: Add end-to-end bwc test for watcher upgrade (elastic/x-pack-elasticsearch#1939)
This test creates watches in old versions of elasticsearch, upgrades them after upgrading cluster to the latest version and then tests that they were upgraded correctly.

Original commit: elastic/x-pack-elasticsearch@b9d45eb2a5
2017-07-09 11:21:57 -04:00
Tim Brooks d95c365e64 Loosen setup mode restrictions for upgrade tests (elastic/x-pack-elasticsearch#1927)
This commit is related to elastic/x-pack-elasticsearch#1896. Currently setup mode means that the
password must be set post 6.0 for using x-pack. This interferes with
upgrade tests as setting the password fails without a properly
upgraded security index.

This commit loosens two aspects of the security.

1. The old default password will be accept in setup mode (requests
from localhost).
2. All request types can be submitted in setup mode.

Original commit: elastic/x-pack-elasticsearch@8a2a577038
2017-07-06 10:37:48 -05:00
Dimitrios Athanasiou 2e0560528f [TEST] Fix MlBasicMultiNodeIT after changing flush response
Relates elastic/x-pack-elasticsearch#1914

Original commit: elastic/x-pack-elasticsearch@5175bf64d9
2017-07-05 13:30:25 +01:00
Jay Modi a9707a461d Use a secure setting for the watcher encryption key (elastic/x-pack-elasticsearch#1831)
This commit removes the system key from master and changes watcher to use a secure setting instead
for the encryption key.

Original commit: elastic/x-pack-elasticsearch@5ac95c60ef
2017-06-29 14:58:35 -06:00
Tim Brooks f2cbe20ea0 Remove default passwords from reserved users (elastic/x-pack-elasticsearch#1665)
This is related to elastic/x-pack-elasticsearch#1217. This PR removes the default password of
"changeme" from the reserved users.

This PR adds special behavior for authenticating the reserved users. No
ReservedRealm user can be authenticated until its password is set. The
one exception to this is the elastic user. The elastic user can be
authenticated with an empty password if the action is a rest request
originating from localhost. In this scenario where an elastic user is
authenticated with a default password, it will have metadata indicating
that it is in setup mode. An elastic user in setup mode is only
authorized to execute a change password request.

Original commit: elastic/x-pack-elasticsearch@e1e101a237
2017-06-29 15:27:57 -05:00
Jay Modi f60c0f893c Test: add a basic rest test for CCS with non-matching remote index patterns (elastic/x-pack-elasticsearch#1866)
This commit adds a basic rest test to verify that security works with cross cluster search when a
remote pattern is provided and no remote indices match.

Relates elastic/elasticsearch#25436
relates elastic/x-pack-elasticsearch#1854

Original commit: elastic/x-pack-elasticsearch@e804d0bb12
2017-06-29 07:14:11 -06:00
Ali Beyad a68fb27a23 Upgrade security index to use only one (the default) index type (elastic/x-pack-elasticsearch#1780)
The .security index used several different types to differentiate the
documents added to the index (users, reserved-users, roles, etc).  Since
types are deprecated in 6.x, this commit changes the .security index
access layer to only use a single type and have all documents in the
index be of that single type.  To differentiate documents that may have
the same id (e.g. the same user name and role name), the appropriate
type of the document is prepended to the id.  For example, a user named
"jdoe" will now have the document id "user-jdoe".  

This commit also ensures that any native realm security index operations
that lead to auto creation of the security index first go through the process
of creating the internal security index (.security-v6) and creating the alias
.security to point to the internal index. 

Lastly, anytime the security index is accessed without having been
upgraded, an exception is thrown notifying the user to use the
upgrade API to upgrade the security index.

Original commit: elastic/x-pack-elasticsearch@cc0a474aed
2017-06-27 17:53:58 -04:00
Ryan Ernst 9b3fb66394 Settings: Add secure versions of SSL passphrases (elastic/x-pack-elasticsearch#1852)
This commit adds new settings for the ssl keystore (not the ES keystore)
passphrase settings. New setting names are used, instead of trying to
support the existing names in both yml and the ES keystore, so that
there does not need to be complicated logic between the two. Note that
the old settings remain the only way to set the ssl passphrases for the
transport client, but the Settings object for transport clients are
created in memory by users, so they are already as "secure" as having a
loaded ES keystore. Also note that in the long term future (6.x
timeframe?) these settings should be deprecated and the keys/certs
themselves should be moved into the ES keystore, so there will be no
need for separate keystores/passphrases.

relates elastic/elasticsearch#22475

Original commit: elastic/x-pack-elasticsearch@be5275fa3d
2017-06-27 10:15:12 -07:00
Jason Tedor c22494bcb7 Remove path.conf setting
This commit is a response to a change in core removing path.conf as a
valid setting.

Relates elastic/x-pack-elasticsearch#1844

Original commit: elastic/x-pack-elasticsearch@477a7eab71
2017-06-26 15:18:49 -04:00
Alexander Reelsen efc93c7246 Tests: Increase loglevel, deactivate ML/monitoring to unclutter logs
Relates elastic/x-pack-elasticsearch#1807

Original commit: elastic/x-pack-elasticsearch@a958f32a3c
2017-06-23 10:40:31 +02:00
Alexander Reelsen 1e7f61b4c8 Tests: Ensure watcher index templates are installed in REST tests (elastic/x-pack-elasticsearch#1784)
The current testing setup only checked if watcher was started, but it
also needs to check for the index template in order to be sure that
everything is set up correctly, before trying to put a watch.

relates elastic/x-pack-elasticsearch#1762

Original commit: elastic/x-pack-elasticsearch@3ed78b15a1
2017-06-20 14:17:36 +02:00
Nik Everett 1559f85c73 Remove assemble from build task when assemble removed
Removes the `assemble` task from the `build` task when we have
removed `assemble` from the project. We removed `assemble` from
projects that aren't published so our releases will be faster. But
That broke CI because CI builds with `gradle precommit build` and,
it turns out, that `build` includes `check` and `assemble`. With
this change CI will only run `check` for projects without an
`assemble`.

Original commit: elastic/x-pack-elasticsearch@d01b0df1d9
2017-06-16 17:19:47 -04:00
Nik Everett d526461bd2 Add basic full cluster restart tests for x-pack (elastic/x-pack-elasticsearch#1743)
Adds tests similar to `:qa:full-cluster-restart` for x-pack. You
run them with `gradle :x-pack:qa:full-cluster-restart:check`.

The actual tests are as basic as it gets: create a doc and load it,
shut down, upgrade to master, startup, and load it. Create a user
and load it, shut down, upgrade to master, startup, and load it.

Relates to elastic/x-pack-elasticsearch#1629

Original commit: elastic/x-pack-elasticsearch@8994bec8e7
2017-06-16 11:44:51 -04:00
Martijn van Groningen 8cc4f29f33 test: make sure analysis-common module is also available on the cluster 1 and cluster2 nodes.
Original commit: elastic/x-pack-elasticsearch@eef5d2b566
2017-06-15 23:25:45 +02:00
jaymode ee3d17adfd Test: fix security rolling upgrade tests that were failing
This commit fixes the default password migration tests that had been failing reproducibly. The
first fix skips tests using the set enabled api when running against a version prior to 5.1.2 as
this api would otherwise trip an assertion that the xcontent builder was not closed. The second
fix is to ensure we set the password field in the user object.

relates elastic/x-pack-elasticsearch#1529
relates elastic/x-pack-elasticsearch#1516

Original commit: elastic/x-pack-elasticsearch@2f9c804309
2017-06-15 14:50:26 -06:00
Martijn van Groningen 863755d2da Make sure that the module are installed too.
Original commit: elastic/x-pack-elasticsearch@f581d0902c
2017-06-15 22:03:18 +02:00
David Roberts b748da1880 [ML] Prevent time_field and control field name in analysis_config (elastic/x-pack-elasticsearch#1729)
In does not make sense for the time_field in the data_description to
be used as a by/over/partition field name, nor the summary_count_field,
categorization_field or as an influencer.  Therefore, configurations
where the time_field in the data_description is used in the
analysis_config are now rejected.

Additionally, it causes a problem communicating with the C++ code if
the control field name (which is '.') is used in the analysis_config,
so this is also rejected at the validation stage.

Relates elastic/x-pack-elasticsearch#1684

Original commit: elastic/x-pack-elasticsearch@e6750a2cda
2017-06-15 13:04:25 +01:00