Commit Graph

4215 Commits

Author SHA1 Message Date
Alexander Reelsen 37b0d52882 Watcher: Remove mock web server from square (elastic/elasticsearch#4221)
The latest release of the mock web server requires more security permissions, and
we dont need all the functionality anyway.

This introduces a small MockWebServer using the JDK internal HttpServer, yet fullfilling
all our needs and supporting SSL as well for testing.

The MockWebServer allows to enqueue responses and also requires you to enqueue as many responses
as you requests will be executed - there is no fallback at the moment.

SSL is also supported by passing an SSL context - for which the TestsSSLService is needed, which
makes the required methods public.

Original commit: elastic/x-pack-elasticsearch@55f4a172a2
2016-12-09 09:07:09 +01:00
Ryan Ernst b1846190af Remove uses of 2.0 prerelease constants
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#22004

Original commit: elastic/x-pack-elasticsearch@55433d760c
2016-12-08 23:11:20 -08:00
Jason Tedor e6d758b92b Bump version to 5.1.2
This commit bumps the version to 5.1.2.

Relates elastic/elasticsearch#4318

Original commit: elastic/x-pack-elasticsearch@a922dc7288
2016-12-08 18:43:16 -05:00
Tanguy Leroux 1f9871c31e [TEST] Mute OldMonitoringIndicesBackwardsCompatibilityTests
This tests fails because of multiple issues in monitoring service and exporters lifecycle. These issues are tracked in https://github.com/elastic/x-plugins/issues/4314.

Original commit: elastic/x-pack-elasticsearch@f461d98a4c
2016-12-08 10:48:39 +01:00
Areek Zillur b274ddc0da Log license type on license update (elastic/elasticsearch#4308)
Now when a cluster gets updated with a new license,
the license uid and type will be logged as info. Making
it easier to verify the current license mode of the cluster.

Original commit: elastic/x-pack-elasticsearch@c205cb42ae
2016-12-07 18:01:58 -05:00
Yannick Welsch 754fada43b Use autoMinMasterNodes for most of the license tests
There were some test failures as LicenseServiceClusterTests was not properly setting minimum_master_nodes while having autoMinMasterNodes disabled.

Original commit: elastic/x-pack-elasticsearch@2030db2424
2016-12-07 17:59:25 +01:00
jaymode 714bf929af test: LocalExporterTests should wait for exporters to be started
Original commit: elastic/x-pack-elasticsearch@978933f9d6
2016-12-07 09:22:05 -05:00
Jim Ferenczi fba54d02d7 fix compilation error due to a missing parameter in SortedSetDVOrdinalsIndexFieldData.
Original commit: elastic/x-pack-elasticsearch@f9803968d9
2016-12-07 11:33:29 +01:00
Boaz Leskes affdf10274 Remove `InternalTestCluster.startNode(s)Async` (elastic/elasticsearch#4198)
A companion PR to https://github.com/elastic/elasticsearch/pull/21846 where the above methods were removed. See ES PR for details.

With the concurrent starting the issues with licenses and time freeze became more apparent and I had to apply my suggestion to only freeze time once the license has been applied (as opposed to freeze on node start up). Since this also means that a node that starts up after the cluster time has been frozen need to also immediately freeze, it felt natural to use a `ServiceDisruptionScheme`. Although the name doesn't really make sense here, it all has all the logic we need. 

Original commit: elastic/x-pack-elasticsearch@5641742f60
2016-12-06 12:07:28 +01:00
Boaz Leskes 7a996da86d ci: don't use ssh to check elasticsearch repo for a sibling branch
ssh keys are not always available

Original commit: elastic/x-pack-elasticsearch@5c974724ea
2016-12-06 11:37:50 +01:00
Boaz Leskes 7b044a6e16 ci: remove unneeded AWAITSFIX comment
Original commit: elastic/x-pack-elasticsearch@726d26fcdb
2016-12-06 10:00:22 +01:00
Boaz Leskes 11ae30cf90 ci: properly accommodate for repo name to be part of `GIT_BRANCH`
previous commit missed a `*`

Original commit: elastic/x-pack-elasticsearch@33b9f32c62
2016-12-06 09:52:03 +01:00
Boaz Leskes 373c350920 ci: accommodate for repo name to be part of `GIT_BRANCH`
It seems that that variable is set by the git jenkins plugin and it's format is `origin/master`. See https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin#GitPlugin-Environmentvariables

Original commit: elastic/x-pack-elasticsearch@17f8ec1a9e
2016-12-06 09:43:28 +01:00
Ryan Ernst 7483abb226 Add fallback branch until CI configuration is fixed to pass GIT_BRANCH
Original commit: elastic/x-pack-elasticsearch@68d12b33e9
2016-12-05 16:34:08 -08:00
Ryan Ernst 13e427d83f Remove action filter response override (elastic/elasticsearch#4268)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21950

Original commit: elastic/x-pack-elasticsearch@0034a3384c
2016-12-05 16:14:10 -08:00
Ryan Ernst 75a72b34cb Build: Checkout relevant branch of elasticsearch for ci (elastic/elasticsearch#4270)
The ci script checks out elasticsearch for x-plugins to use. However, it
always checks out the master branch. This change makes the script look
for the source branch of a PR (falling back to GIT_BRANCH which is provided by jenkins for non PR CI jobs) and checks if that branch exists in elasticsearch. If it does not, it falls back to the target branch for the PR.

Original commit: elastic/x-pack-elasticsearch@73146bb3b3
2016-12-05 16:10:33 -08:00
Jay Modi ffeaea0663 Migrate authentication calls to be asynchronous
This commit migrates the actual authentication calls that are performed by internal realms
to be asynchronous so that we do not execute blocking calls on network threads. The realms
that used LDAP were especially bad as they can issue many different requests and/or open
multiple connections per authentication attempt.

The LdapRealm now uses the ThreadPool to fork a thread for authentication. This is necessary
because a LDAP bind operation is not allowed to be performed asynchronously and must be done
in a blocking manner. After the bind request has completed, all other requests can be done
asynchrnously. The LdapUtils class now provides asynchronous methods for searching and also
includes the handling of referral following as automated following is not supported by
UnboundID when using the asynchronous calls.

Original commit: elastic/x-pack-elasticsearch@dfb259e498
2016-12-05 10:07:31 -05:00
Tanguy Leroux 549629b1aa [Watcher] HistoryStore should hide secrets (elastic/elasticsearch#4256)
* [Watcher] HistoryStore should hide secrets

Some watcher actions like Jira stores the whole HTTP request in case of a failure. This is very helpful when something goes wrong, but it has to hide the password used for Basic authentication otherwise it will appear in the watch record in the .watcher-history index. In general, I would consider better to always hide secrets when storing something in the history index.

 This commit removes the XContentParams "hide secrets" set at the JiraIssue so that it is now set at the HistoryStore level.

 It also fixes test failures that can be reproduced with:
 gradle :x-plugins:elasticsearch:test -Dtests.seed=BA730F93E726AECC -Dtests.class=org.elasticsearch.xpack.notification.jira.JiraIssueTests -Dtests.method="testToXContent" -Dtests.security.manager=true -Dtests.locale=es -Dtests.timezone=Asia/Choibalsan

Original commit: elastic/x-pack-elasticsearch@d686b3b3ad
2016-12-05 15:30:48 +01:00
Jay Modi 9afb6dd4f2 monitoring: local exporter waits for in flight requests before retrying
The local exporter previously fired off asynchronous requests every time a cluster state was
observed that did not contain all of the required items for monitoring. This change adds a
flag so that monitoring can wait for the pending requests to complete before retrying. This
will reduce the number of duplicated log messages as well.

Additionally, the log message for adding modern aliases now contains the name of the indices.

Closes elastic/elasticsearch#3756

Original commit: elastic/x-pack-elasticsearch@727a0adfbe
2016-12-05 09:01:24 -05:00
jaymode 2b7c03848f test: disable xpack plugins on tribe client nodes
The license test for the tribe node disables xpack features but these settings
are not passed to the tribe client nodes, so they need to manually be added.

Original commit: elastic/x-pack-elasticsearch@7273817c47
2016-12-03 14:02:55 -05:00
Ryan Ernst 97336ea946 Fix rest filter tests to expect passthrough when security is disabled
Original commit: elastic/x-pack-elasticsearch@ea6adabb4d
2016-12-02 16:28:18 -08:00
Ryan Ernst dbbf470734 Fix line length to appease checkstyle
Original commit: elastic/x-pack-elasticsearch@35df54d342
2016-12-02 15:06:33 -08:00
Ryan Ernst 923926ef28 Convert security rest filter to rest handler wrapper (elastic/elasticsearch#4234)
* Convert security rest filter to rest handler wrapper

This is the xpack side of elastic/elasticsearchelastic/elasticsearch#21905

Original commit: elastic/x-pack-elasticsearch@38bfa771b6
2016-12-02 14:55:10 -08:00
Tanguy Leroux f1a4a2fb73 [Monitoring] Remove unused code in Collectors (elastic/elasticsearch#4240)
This commit removes some unnecessary code: collectors do not need to extend AbstractLifecycleComponent and do not need to be started/stopped. The extra
interface Collector is removed and AbstractCollector renamed to Collector.

Original commit: elastic/x-pack-elasticsearch@75893d66e3
2016-12-02 13:29:09 +01:00
Simon Willnauer 7f17896927 Revert "Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)"
This reverts commit elastic/x-pack@c6c0ffd5d9.

Original commit: elastic/x-pack-elasticsearch@8b7386fb99
2016-12-02 10:55:46 +01:00
Alexander Reelsen 7c04897392 Watcher: Compile scripts on each invocation (elastic/elasticsearch#4239)
Transform and condition scripts were only compiled on its initial creation, so
when a new watch is created or when the master node loads all the watches. However
changing a script (like a stored one) did not lead to any changes in the in memory
watch store and thus the old script was executed again.

We do however have a mechanism in Elasticsearch's ScriptService that already does some
caching, and should reuse that one.

Closes elastic/elasticsearch#4237

Original commit: elastic/x-pack-elasticsearch@477548e237
2016-12-02 10:36:05 +01:00
Alexander Reelsen 946d943868 Watcher: Removing unused upgradeSource boolean in watch parsing (elastic/elasticsearch#4202)
This seems to be a leftover from elastic/elasticsearch#4162

However the boolean parameters is completely unused already and
can be removed.

Original commit: elastic/x-pack-elasticsearch@3371b089d6
2016-12-02 10:34:11 +01:00
Simon Willnauer ace1a7e6af Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)
Original commit: elastic/x-pack-elasticsearch@07cecdbf00
2016-12-02 08:06:46 +01:00
Aaron Bull Schaefer 417ccdd7cc Merge pull request elastic/elasticsearch#4137 from dliappis/master
Allow defining gradle test params via cli argument

Original commit: elastic/x-pack-elasticsearch@3d312de1cc
2016-12-01 10:24:10 -08:00
Jay Modi e8836f99b0 test: add tests that exercise the security index during rolling upgrades
This commit adds basic tests that store a user and a role using the native API. The test checks
that the user and role can be used prior to starting the upgrade. The realm and roles caches are
also cleared to ensure the next authentication will require a read from the security index; this
ensures we are actually testing reads from the index.

Original commit: elastic/x-pack-elasticsearch@396862da94
2016-12-01 11:15:15 -05:00
Jay Modi 139073e8f7 security: improve migrate tool output and remove trappy config option
This commit improves the output of the migrate tool in cases when there are errors parsing entries
in the roles or users files. This is done through the use of a logger that delegates its output to
the terminal.

Additionally, the `-c` option has been removed. This option was used to set the configuration directory
but this should be handled one way only and that is through the use of the `-Epath.conf` setting.

Closes elastic/elasticsearch#3757
Closes elastic/elasticsearch#3758

Original commit: elastic/x-pack-elasticsearch@811e367766
2016-12-01 10:17:28 -05:00
Boaz Leskes d881e4d9ad properly deal with overriding parent's tearDown method
Original commit: elastic/x-pack-elasticsearch@0edd4b1e20
2016-12-01 13:18:36 +01:00
Boaz Leskes 9e3ae67423 HttpClientTests don't override parent tearDown
Original commit: elastic/x-pack-elasticsearch@3cf9f6f352
2016-12-01 13:15:25 +01:00
Boaz Leskes a4cec2316b Add before and after logging for unit tests
Currently we have these logs for integration tests only.

This adds the following log at the start:
```
logger.info("[{}]: before test", getTestName());
```

and this is logged at the end, but before any clean up done in sub classes

```
 logger.info("[{}]: after test", getTestName());
```

Original commit: elastic/x-pack-elasticsearch@2ca7296665
2016-12-01 12:56:38 +01:00
Martijn van Groningen 0701f7bb18 test: verify whether templates exist in an assertBusy(...)
It may take a few clicks before the templates are added.
Waiting for started doesn't guarantee that the index templates have been added.

Original commit: elastic/x-pack-elasticsearch@ee94b740a8
2016-12-01 11:04:11 +01:00
javanna 792a821d1a Adapt to indices query removal
Original commit: elastic/x-pack-elasticsearch@8f2d4c23c5
2016-11-30 18:23:19 +01:00
Jay Modi 217bd8add0 Add tests for DNS only hostname verification with SSL
This commit adds tests for DNS only hostname verification. This is a followup of elastic/elasticsearchelastic/elasticsearch#21828, which fixes issues with this type of hostname verification
due to some addresses losing the host information.

Original commit: elastic/x-pack-elasticsearch@8a63bb113d
2016-11-30 12:20:11 -05:00
Jason Tedor 16e3bb4587 Add version 5.1.1
This commit adapts x-plugins for the removal of the version constant for
5.1.0 and the addition of the version constant for 5.1.1 in core.

Relates elastic/elasticsearch#4223

Original commit: elastic/x-pack-elasticsearch@2fa92f0056
2016-11-30 11:14:47 -05:00
javanna 69218af73f Remove subrequests method from CompositeIndicesRequest
Original commit: elastic/x-pack-elasticsearch@c644204598
2016-11-30 15:03:42 +01:00
Adrien Grand 981648774f Remove usage of `indices.ttl.interval`. (elastic/elasticsearch#4207)
Remove usage of `indices.ttl.interval`.

Original commit: elastic/x-pack-elasticsearch@54c987daec
2016-11-30 10:11:10 +01:00
Adrien Grand 84be89861a Remove 2.x bwc indices. (elastic/elasticsearch#4206)
Version 6.0 will only support reading 5.0+ indices.

Original commit: elastic/x-pack-elasticsearch@167f5f209d
2016-11-30 10:10:30 +01:00
Jason Tedor 3da81aa922 Add version 5.0.3
This commit adds version 5.0.3 and the BWC indices for version 5.0.2.

Relates elastic/elasticsearch#4211

Original commit: elastic/x-pack-elasticsearch@a0c83a0b92
2016-11-29 18:48:45 -05:00
javanna f19be18b20 [TEST] add render search template qa test
Original commit: elastic/x-pack-elasticsearch@7febccbfb2
2016-11-29 23:45:52 +01:00
Luca Cavanna 34d6dc1db1 Categorize search template action as a composite indices request (elastic/elasticsearch#4209)
When we encounter a composite request, we authorize at first without looking at the indices, to see whether the action can be executed at all. We then rely on the action to delegate to an inner action per sub-request, which will be authorized based on the indices it refers to. The first step works great for the simulate mode of search template, as it doesn't involve any index. The second step will make sure that when search template involves a search, it will be authorized as a normal search request would, based on the indices it reads from.

Note that the wildcard expansion happens now on the search side, it doesn't have to happen when executing the first authorization step, hence SearchTemplateRequest doesn't have to implement IndicesRequest, only SearchRequest has to (which it does already).

Closes elastic/elasticsearch#4171

Original commit: elastic/x-pack-elasticsearch@d586bd90cb
2016-11-29 20:53:01 +01:00
Tanguy Leroux a414e3a7d9 [TEST] Waits for security template to be created in XPackRestIT
This commit applies the same fix merged in  elastic/elasticsearch#4179 for XDocsClientYamlTestSuiteIT. It adds a waitForSecurityTemplate() method in order to wait for the security-index-template to be created by the SecurityTemplateService.

Original commit: elastic/x-pack-elasticsearch@1476f30e2d
2016-11-29 14:05:58 +01:00
Dimitrios Liappis 9294827a91 Switch to /bin/bash
Since there is no hard requirement for a strict POSIX shell
implementation, we can switch to /bin/bash.

Clean up left over break statements and consistently use $() for command
substitutions.

Original commit: elastic/x-pack-elasticsearch@0a33dfed56
2016-11-29 14:44:56 +02:00
Tanguy Leroux 0673d6b3d6 [Test] Add back ThreadLeakLingering in OldMonitoringIndicesBackwardsCompatibilityTests
Also changes a bit how collection is stopped.

Original commit: elastic/x-pack-elasticsearch@e28f8bc11d
2016-11-28 16:10:28 +01:00
Jay Modi 637154cc6e Iterate over realms asynchonously
This commit moves the iteration of realms for authentication and user lookup to
be done in an asynchronous fashion. The existing blocking methods have been deprecated
to allow custom realm implementors time to switch. All internal realms implement the
asynchronous methods.

This PR is another step toward the full migration to async authentication, but does not
complete the work. Additional work is needed for the LDAP realms, which make blocking
network calls. These blocking calls will be handled in a follow-up PR.

See elastic/elasticsearch#3790

Original commit: elastic/x-pack-elasticsearch@a65a9b2bb4
2016-11-28 09:28:51 -05:00
Dimitrios Liappis 589e880a9c Make cli arg parsing compliant with requirements.
Remove duplicate definition of GRADLE_OPT_TEST and rename arguments as
discussed in
https://github.com/elastic/x-plugins/pull/4137#discussion_r89660620

Default to `check` if no argument is passed, but exit with error in an
unsupported parameter is provided.

Also ensure the pipefail option is only set when the executing shell
is *really* bash as opposed to POSIX sh which doesn't support it.

Original commit: elastic/x-pack-elasticsearch@64bd740295
2016-11-28 13:19:48 +02:00
Alexander Reelsen f265ab7cae Watcher: Throw exception if HttpClient response is not a HTTP response (elastic/elasticsearch#4154)
If the HTTP response is an invalid one, it is still logged as success.
This commit changes the behaviour, that if the response status code is
set to -1 (which means it could not be interpreted), than an IOException
is thrown and thus the execution will be marked as a failure.

Closes elastic/elasticsearch#4152

Original commit: elastic/x-pack-elasticsearch@5736fbe3c0
2016-11-28 11:29:28 +01:00