Commit Graph

1388 Commits

Author SHA1 Message Date
Tanguy Leroux f1a4a2fb73 [Monitoring] Remove unused code in Collectors (elastic/elasticsearch#4240)
This commit removes some unnecessary code: collectors do not need to extend AbstractLifecycleComponent and do not need to be started/stopped. The extra
interface Collector is removed and AbstractCollector renamed to Collector.

Original commit: elastic/x-pack-elasticsearch@75893d66e3
2016-12-02 13:29:09 +01:00
Simon Willnauer 7f17896927 Revert "Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)"
This reverts commit elastic/x-pack@c6c0ffd5d9.

Original commit: elastic/x-pack-elasticsearch@8b7386fb99
2016-12-02 10:55:46 +01:00
Alexander Reelsen 7c04897392 Watcher: Compile scripts on each invocation (elastic/elasticsearch#4239)
Transform and condition scripts were only compiled on its initial creation, so
when a new watch is created or when the master node loads all the watches. However
changing a script (like a stored one) did not lead to any changes in the in memory
watch store and thus the old script was executed again.

We do however have a mechanism in Elasticsearch's ScriptService that already does some
caching, and should reuse that one.

Closes elastic/elasticsearch#4237

Original commit: elastic/x-pack-elasticsearch@477548e237
2016-12-02 10:36:05 +01:00
Alexander Reelsen 946d943868 Watcher: Removing unused upgradeSource boolean in watch parsing (elastic/elasticsearch#4202)
This seems to be a leftover from elastic/elasticsearch#4162

However the boolean parameters is completely unused already and
can be removed.

Original commit: elastic/x-pack-elasticsearch@3371b089d6
2016-12-02 10:34:11 +01:00
Simon Willnauer ace1a7e6af Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)
Original commit: elastic/x-pack-elasticsearch@07cecdbf00
2016-12-02 08:06:46 +01:00
Jay Modi 139073e8f7 security: improve migrate tool output and remove trappy config option
This commit improves the output of the migrate tool in cases when there are errors parsing entries
in the roles or users files. This is done through the use of a logger that delegates its output to
the terminal.

Additionally, the `-c` option has been removed. This option was used to set the configuration directory
but this should be handled one way only and that is through the use of the `-Epath.conf` setting.

Closes elastic/elasticsearch#3757
Closes elastic/elasticsearch#3758

Original commit: elastic/x-pack-elasticsearch@811e367766
2016-12-01 10:17:28 -05:00
Boaz Leskes d881e4d9ad properly deal with overriding parent's tearDown method
Original commit: elastic/x-pack-elasticsearch@0edd4b1e20
2016-12-01 13:18:36 +01:00
Boaz Leskes 9e3ae67423 HttpClientTests don't override parent tearDown
Original commit: elastic/x-pack-elasticsearch@3cf9f6f352
2016-12-01 13:15:25 +01:00
Boaz Leskes a4cec2316b Add before and after logging for unit tests
Currently we have these logs for integration tests only.

This adds the following log at the start:
```
logger.info("[{}]: before test", getTestName());
```

and this is logged at the end, but before any clean up done in sub classes

```
 logger.info("[{}]: after test", getTestName());
```

Original commit: elastic/x-pack-elasticsearch@2ca7296665
2016-12-01 12:56:38 +01:00
Martijn van Groningen 0701f7bb18 test: verify whether templates exist in an assertBusy(...)
It may take a few clicks before the templates are added.
Waiting for started doesn't guarantee that the index templates have been added.

Original commit: elastic/x-pack-elasticsearch@ee94b740a8
2016-12-01 11:04:11 +01:00
javanna 792a821d1a Adapt to indices query removal
Original commit: elastic/x-pack-elasticsearch@8f2d4c23c5
2016-11-30 18:23:19 +01:00
Jay Modi 217bd8add0 Add tests for DNS only hostname verification with SSL
This commit adds tests for DNS only hostname verification. This is a followup of elastic/elasticsearchelastic/elasticsearch#21828, which fixes issues with this type of hostname verification
due to some addresses losing the host information.

Original commit: elastic/x-pack-elasticsearch@8a63bb113d
2016-11-30 12:20:11 -05:00
Jason Tedor 16e3bb4587 Add version 5.1.1
This commit adapts x-plugins for the removal of the version constant for
5.1.0 and the addition of the version constant for 5.1.1 in core.

Relates elastic/elasticsearch#4223

Original commit: elastic/x-pack-elasticsearch@2fa92f0056
2016-11-30 11:14:47 -05:00
javanna 69218af73f Remove subrequests method from CompositeIndicesRequest
Original commit: elastic/x-pack-elasticsearch@c644204598
2016-11-30 15:03:42 +01:00
Adrien Grand 981648774f Remove usage of `indices.ttl.interval`. (elastic/elasticsearch#4207)
Remove usage of `indices.ttl.interval`.

Original commit: elastic/x-pack-elasticsearch@54c987daec
2016-11-30 10:11:10 +01:00
Adrien Grand 84be89861a Remove 2.x bwc indices. (elastic/elasticsearch#4206)
Version 6.0 will only support reading 5.0+ indices.

Original commit: elastic/x-pack-elasticsearch@167f5f209d
2016-11-30 10:10:30 +01:00
Jason Tedor 3da81aa922 Add version 5.0.3
This commit adds version 5.0.3 and the BWC indices for version 5.0.2.

Relates elastic/elasticsearch#4211

Original commit: elastic/x-pack-elasticsearch@a0c83a0b92
2016-11-29 18:48:45 -05:00
Luca Cavanna 34d6dc1db1 Categorize search template action as a composite indices request (elastic/elasticsearch#4209)
When we encounter a composite request, we authorize at first without looking at the indices, to see whether the action can be executed at all. We then rely on the action to delegate to an inner action per sub-request, which will be authorized based on the indices it refers to. The first step works great for the simulate mode of search template, as it doesn't involve any index. The second step will make sure that when search template involves a search, it will be authorized as a normal search request would, based on the indices it reads from.

Note that the wildcard expansion happens now on the search side, it doesn't have to happen when executing the first authorization step, hence SearchTemplateRequest doesn't have to implement IndicesRequest, only SearchRequest has to (which it does already).

Closes elastic/elasticsearch#4171

Original commit: elastic/x-pack-elasticsearch@d586bd90cb
2016-11-29 20:53:01 +01:00
Tanguy Leroux a414e3a7d9 [TEST] Waits for security template to be created in XPackRestIT
This commit applies the same fix merged in  elastic/elasticsearch#4179 for XDocsClientYamlTestSuiteIT. It adds a waitForSecurityTemplate() method in order to wait for the security-index-template to be created by the SecurityTemplateService.

Original commit: elastic/x-pack-elasticsearch@1476f30e2d
2016-11-29 14:05:58 +01:00
Tanguy Leroux 0673d6b3d6 [Test] Add back ThreadLeakLingering in OldMonitoringIndicesBackwardsCompatibilityTests
Also changes a bit how collection is stopped.

Original commit: elastic/x-pack-elasticsearch@e28f8bc11d
2016-11-28 16:10:28 +01:00
Jay Modi 637154cc6e Iterate over realms asynchonously
This commit moves the iteration of realms for authentication and user lookup to
be done in an asynchronous fashion. The existing blocking methods have been deprecated
to allow custom realm implementors time to switch. All internal realms implement the
asynchronous methods.

This PR is another step toward the full migration to async authentication, but does not
complete the work. Additional work is needed for the LDAP realms, which make blocking
network calls. These blocking calls will be handled in a follow-up PR.

See elastic/elasticsearch#3790

Original commit: elastic/x-pack-elasticsearch@a65a9b2bb4
2016-11-28 09:28:51 -05:00
Alexander Reelsen f265ab7cae Watcher: Throw exception if HttpClient response is not a HTTP response (elastic/elasticsearch#4154)
If the HTTP response is an invalid one, it is still logged as success.
This commit changes the behaviour, that if the response status code is
set to -1 (which means it could not be interpreted), than an IOException
is thrown and thus the execution will be marked as a failure.

Closes elastic/elasticsearch#4152

Original commit: elastic/x-pack-elasticsearch@5736fbe3c0
2016-11-28 11:29:28 +01:00
Jason Tedor c4e890cba0 Adapt to unreleased versions change
Core has better support unreleased versions now, making maintenance of
these simpler. This commit adapts x-plugins to this change.

Relates elastic/elasticsearch#4168

Original commit: elastic/x-pack-elasticsearch@a5d8a2f7df
2016-11-23 15:49:53 -05:00
Alexander Reelsen d53dbe5283 Watcher: Clean up email server (elastic/elasticsearch#4163)
* Do not try bind to port range but use free ephemeral port
* Start a new email server in all tests, do not use static one
* Remove selection of username/password, as it was static anyway
* Remove Listener.Handle class, as it is not needed, when not running in static context

Original commit: elastic/x-pack-elasticsearch@8816cc25f6
2016-11-23 18:19:02 +01:00
Tanguy Leroux ddddee1e1f [Watcher] Increment watcher history template version (elastic/elasticsearch#4166)
This commit increments the version number of the watcher history template.

Original commit: elastic/x-pack-elasticsearch@1c86e781ca
2016-11-23 16:27:33 +01:00
Tanguy Leroux 52eb621309 [TEST] Fix OldMonitoringIndicesBackwardsCompatibilityTests
OldMonitoringIndicesBackwardsCompatibilityTests fails because it waits for more shards stats to be collected but that can only work if new indices are created in the meanwhile.

Original commit: elastic/x-pack-elasticsearch@003c28cf93
2016-11-23 15:16:22 +01:00
Tanguy Leroux a32f2096a6 Add mappings for Jira action (elastic/elasticsearch#4155)
This commit updates the watch_history.json file so that it includes mappings for the new Jira action. It also update the JiraIssue format so that it now includes the name of the account used to create the Jira issue. It also update the REST tests to check that Jira action result are searchable and hide the user's password.

Original commit: elastic/x-pack-elasticsearch@75888f7748
2016-11-23 11:53:06 +01:00
Martijn van Groningen 5d042fc1b8 updated testScriptConditionParserBadScript() test to face reality
Original commit: elastic/x-pack-elasticsearch@222537dbe2
2016-11-23 09:15:45 +01:00
Ryan Ernst 3f02111a92 Remove upgrade source test, no longer needed with groovy gone
Original commit: elastic/x-pack-elasticsearch@c4a6c87227
2016-11-22 23:11:28 -08:00
Ryan Ernst 1dc839bd98 Remove groovy scripting language (elastic/elasticsearch#4162)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21607

Original commit: elastic/x-pack-elasticsearch@125843e814
2016-11-22 22:45:15 -08:00
Jay Modi 7873bb73c4 add tests for concurrent user lookup
This commit adds tests to ensure that user lookup in caching realms works as expected. An unclear
contract in the Cache#computeIfAbsent method allowed for null values to be returned from this method
even if there should have been exception reported to the loader. This has been fixed in the cache
implementation and we add tests to verify that the caching of user lookups is done properly under
concurrent operations.

Closes elastic/elasticsearch#4054

Original commit: elastic/x-pack-elasticsearch@41567c6ed9
2016-11-22 15:16:49 -05:00
Simon Willnauer 7f28ca7fb7 Adopt changes to TransportClient constructor (elastic/elasticsearch#4141)
Follow-up for elastic/elasticsearchelastic/elasticsearch#21709

Original commit: elastic/x-pack-elasticsearch@40d3725ac2
2016-11-22 20:46:43 +01:00
jaymode 7821203422 test: add bwc index for 2.4.2
Original commit: elastic/x-pack-elasticsearch@f9fbd83b9c
2016-11-22 14:37:36 -05:00
Jay Modi 4239ba5415 allow reads of native users and roles when template version hasn't been updated
This change allows reads of our native users and roles when the template version has not been updated to
match the current version. This is useful for rolling upgrades where the nodes are also being actively
queried and/or indexed into. Without this, we can wreak havoc on a cluster by causing exceptions during
replication, which leads to shard failures. On nodes that match the version defined in the template,
write operations are allowed since we know that we are backwards compatible in terms of format but we
may have added new fields and shouldn't index them until the mappings and template have been updated.

As part of this, the rolling upgrade tests from core were used as the basis for a very basic set of tests
for doing a rolling upgrade with x-pack.

Closes elastic/elasticsearch#4126

Original commit: elastic/x-pack-elasticsearch@9be518ef00
2016-11-22 12:00:09 -05:00
Areek Zillur a9f3619b5a Enable merging license in tribe node (elastic/elasticsearch#4147)
Currently, a tribe node ignored underlying cluster licenses
due to inablity to select an appropriate license from  multiple
licenses. Now that tribe node supports merging custom metadata
(elasticsearch#elastic/elasticsearch#21552), we can enable license support in tribe
node.

Now, tribe node chooses license with the highest operation
mode from underlying cluster licenses. This commit also
adds integration tests for licensing to verify that:
 - autogenerated trial license propagates to tribe node
 - tribe node chooses the highest operation mode license
 - removing a license from underlying cluster license is
   removed from tribe

closes elastic/elasticsearch#3212

Original commit: elastic/x-pack-elasticsearch@b5c003decd
2016-11-22 11:42:51 -05:00
Tanguy Leroux c9b3de6f23 Make HttpRequest toXContent / parse method coherent (elastic/elasticsearch#4153)
This commit changes the toXContent() method so that it generates content that can be parsed again by the parse() method. Before this commit, the authentication of the HTTP request is rendered as:

{ "auth": {"username": "foo", "password": "bar" } }

but the parsing method expects the authentication type to be a root object:

{ "auth":  { "basic" :  {"username": "foo", "password": "bar" } } }

The toXContent method has been adapted to include the type of authentication in the generated content.

Original commit: elastic/x-pack-elasticsearch@b740466109
2016-11-22 11:24:06 +01:00
Tanguy Leroux efe6524253 Mute JiraIssueTests.toToXContent()
Until "https://github.com/elastic/x-plugins/pull/4153" is solved

Original commit: elastic/x-pack-elasticsearch@c7720faddf
2016-11-22 10:07:27 +01:00
Alexander Reelsen a709d78e5a Tests: Ensure mock webserver binds always on arbitrary port (elastic/elasticsearch#4139)
This removes the mock webserver trying to bind to a port range, just
try to bind to any port.

Also removed unneeded wrapper class in the process.

Original commit: elastic/x-pack-elasticsearch@19b6ac393a
2016-11-22 09:24:32 +01:00
Tanguy Leroux 4978d3a8e0 [TEST] Fix JiraIssueTests.testEquals()
This commit fixes a test bug were a random field is picked out of an empty collection.

Original commit: elastic/x-pack-elasticsearch@a42dff7257
2016-11-22 09:09:44 +01:00
Jason Tedor 158ab2e724 Die with dignity on the network layer
When a fatal error is thrown on the network layer, such an error never
makes its way to the uncaught exception handler. This prevents the node
from being torn down if an out of memory error or other fatal error is
thrown while handling HTTP or transport traffic. This commit adds logic
to ensure that such errors bubble their way up to the uncaught exception
handler, even though Netty tries really hard to swallow everything.

Original commit: elastic/x-pack-elasticsearch@f76757674f
2016-11-21 22:14:45 -05:00
Tanguy Leroux 7c1f4326fc [TEST] Fix OldMonitoringIndicesBackwardsCompatibilityTests
This commit aims to fix the regularly failing test OldMonitoringIndicesBackwardsCompatibilityTests. It enables/disables monitoring data collection using time interval "-1" as well as stopping AgentService instances directly and checks multiple times if indices are still present at the end of the test.

closes elastic/elasticsearch#3999

Original commit: elastic/x-pack-elasticsearch@8ac785061a
2016-11-21 16:15:42 +01:00
Tanguy Leroux 18478d63c2 Watcher: Add JIRA action (elastic/elasticsearch#4014)
closes elastic/elasticsearch#493

Original commit: elastic/x-pack-elasticsearch@6b7387d3e4
2016-11-21 10:52:55 +01:00
Tim Vernum 74b0a1e71a Record audit trail even if indicies-resolver throws exception (elastic/elasticsearch#4116)
* Record audit trail even if indicies-resolver throws exception

If the IndicesAndAliasesResolver throws an exception, treat is as "accessDenied" for the purpose of the audit-trail.
This can occur when an index request has a wildcard that doesn't match (and "allowNoIndices" is false)

Closes elastic/elasticsearch#3719

Original commit: elastic/x-pack-elasticsearch@ca6567e5ed
2016-11-21 09:18:49 +11:00
Tim Vernum ae6eb0d159 Enforce username validation on PUT security/user (elastic/elasticsearch#4118)
Validation was being applied within UsersTool but not via the REST API
This also causes some validation messages in PutUser to change.

Closes elastic/elasticsearch#4018

Original commit: elastic/x-pack-elasticsearch@8e02146dee
2016-11-18 12:33:45 +11:00
Simon Willnauer cc22888417 Stabelize RemoteIndexAuditTrailStartingTests and improve IndexAuditTrail (elastic/elasticsearch#4112)
RemoteIndexAuditTrailStartingTests has been failing quite often due to several
race conditions. This change simplifies the test (one node per cluster, local and
remote) and fixes some issues in IndexAuditTrail. The interrupt based shutdown has
been removed from the `QueueConsumer` and an additional health call has been added
to trigger another start attempt if the first one failed. This stabilize the test
that now run in 1 second rather than 1 minute.

Original commit: elastic/x-pack-elasticsearch@a1c8131cd2
2016-11-17 17:57:15 +01:00
Jason Tedor 7309f11f51 Respond to exposing of executor service
This commit responds to a chance in core which modified the interface
for ThreadPool#executor and ThreadPool#generic to return an executor
service rather than an executor.

Relates elastic/elasticsearch#4107

Original commit: elastic/x-pack-elasticsearch@7c7c6a3b90
2016-11-17 09:19:15 -05:00
Simon Willnauer 92040ef72e Remove netty_3 support from xpack (elastic/elasticsearch#4097)
This is a followup from elastic/elasticsearchelastic/elasticsearch#21590 and needs to be
committed first or at the same time since netty_3 is removed

Original commit: elastic/x-pack-elasticsearch@131d74dd6b
2016-11-17 12:44:24 +01:00
Adrien Grand a5e410ba59 Remove expectations about store throttling stats now that it has been removed.
Original commit: elastic/x-pack-elasticsearch@244afb234a
2016-11-17 10:13:29 +01:00
Adrien Grand 67cc424303 Remove call to the store throttling stats.
Store throttling has been removed in elastic/elasticsearchelastic/elasticsearch#21573.

Original commit: elastic/x-pack-elasticsearch@40bcb51101
2016-11-17 09:48:26 +01:00
Nik Everett 042072410c Rename some tests from IT to Tests (elastic/elasticsearch#4106)
If these tests were in `:core` then then `IT` would be an appropriate
suffix, but in xplugins they should have `Tests`. This moves them
from the `integTest` task to the `test` task which is a good idea
because it lets them run in parallel with the other tests. Naming
them `IT` means that they are run not in parallel and they are run
with a running instance of Elasticsearch which they don't need.

This cuts the build from 15.5 minutes to 13.5 minutes for me.

Original commit: elastic/x-pack-elasticsearch@7b4bcf8bc5
2016-11-16 17:15:34 -05:00