Commit Graph

554 Commits

Author SHA1 Message Date
Tanguy Leroux 233c64e942 Monitoring: Fix cleaner tests
Some changes in elastic/x-pack@d13557c517 change the testIgnoreTimestampedIndicesInOtherVersions method and it now sometime fails.
 This commit revert the previous behavior of the test and ensures that at least 1 index is cleaned up/deleted in each test.

Original commit: elastic/x-pack-elasticsearch@3c6acb4ff8
2016-05-12 12:19:10 +02:00
Robert Muir 3a2cfabc4d use painless syntax improvements in watcher tests and docs
Original commit: elastic/x-pack-elasticsearch@27ef31efac
2016-05-11 21:24:43 -04:00
Chris Earle 5c9d18fc34 Adding {index} option for _xpack/monitoring/_bulk
This adds it so that a system can specify "_data" as the index to index into the
data index (without having to know its name). _Not_ supplying an index will use
the timestamped index. Any other index name (including wrong case) is invalid.

Original commit: elastic/x-pack-elasticsearch@6eeadfb3c8
2016-05-11 15:05:06 -04:00
Jason Tedor 6a3aef32d6 Switch scripts to use bash
This commit switches the command-line scripts to use bash instead of sh
so that we can take advantage of features that bash provides like
arrays.

Relates elastic/elasticsearch#2214

Original commit: elastic/x-pack-elasticsearch@826c10ef04
2016-05-10 15:07:04 -04:00
Yannick Welsch ee406e9c3f Use immutable ShardRouting
Relates to elastic/elasticsearchelastic/elasticsearch#17821

Original commit: elastic/x-pack-elasticsearch@92a9581327
2016-05-10 19:07:53 +02:00
markharwood e7b70794b6 Prefix Graph REST endpoint with _xpack. Partial fix for issue 1760
Original commit: elastic/x-pack-elasticsearch@53ceb1c20d
2016-05-10 13:45:07 +01:00
Adrien Grand 7a6025784d Use Versions.MATCH_ANY rather than NOT_SET now that NOT_SET is gone.
Original commit: elastic/x-pack-elasticsearch@14694aea60
2016-05-10 08:23:32 +02:00
Daniel Mitterdorfer 828b1902c5 Merge remote-tracking branch 'danielmitterdorfer/free-request-bytes'
Original commit: elastic/x-pack-elasticsearch@91867432c2
2016-05-09 16:01:21 +02:00
Tanguy Leroux 8f097d24e4 Mute XPackRestIT.test {p0=bulk/10_basic/Bulk indexing of monitoring data}
Original commit: elastic/x-pack-elasticsearch@e3a889222e
2016-05-09 13:27:54 +02:00
Daniel Mitterdorfer c5aa644048 Free bytes reserved on request breaker
With this commit we free all bytes reserved on the request circuit breaker.

Relates elastic/elasticsearchelastic/elasticsearch#18144

Original commit: elastic/x-pack-elasticsearch@3986436b8c
2016-05-09 11:22:42 +02:00
Chris Earle 68728e6bee Use Strict version check for VersionCompatibilityTests
This changes the loose usage of onOrBefore to equals so that when we add beta1, this test fails
again.

Original commit: elastic/x-pack-elasticsearch@fe4f2cbdf0
2016-05-06 15:08:40 -04:00
Chris Earle fd62b2308e Updating with ES-side abstract method addition
Original commit: elastic/x-pack-elasticsearch@0d075b433b
2016-05-06 12:26:10 -04:00
Chris Earle 080000a595 Updating with array changed to list.
Original commit: elastic/x-pack-elasticsearch@552227458f
2016-05-06 12:26:10 -04:00
Chris Earle ec0a4646ea Add Failure Details to every NodesResponse (x-plugins side)
Original commit: elastic/x-pack-elasticsearch@9ffb88caaf
2016-05-06 12:26:10 -04:00
Jason Tedor 35121bc206 Add handshake to list of known handlers for tests
This commit adds internal:transport/handshake to the list of known
handlers for tests.

Closes elastic/elasticsearch#2183

Original commit: elastic/x-pack-elasticsearch@c4b415367b
2016-05-06 09:17:44 -04:00
Tanguy Leroux 330e427f40 Monitoring: Add Points stats to node and index segments stats
Original commit: elastic/x-pack-elasticsearch@7df2538068
2016-05-06 10:10:25 +02:00
Adrien Grand 5db861d14b Remove generics from QueryBuilder.
Related to elastic/elasticsearchelastic/elasticsearch#18133.

Original commit: elastic/x-pack-elasticsearch@bd8ca4f061
2016-05-06 09:06:30 +02:00
Adrien Grand 22c4fb9a32 Make compilation pass.
Note: tests are still failing.

Original commit: elastic/x-pack-elasticsearch@827d129876
2016-05-06 09:05:45 +02:00
Areek Zillur bd04cc9d1f Extend tribe integ test infra to test on master and client nodes
Original commit: elastic/x-pack-elasticsearch@5826fb4161
2016-05-05 15:28:04 -04:00
Areek Zillur 3f0acdd70e refactor tribe integ tests to test monitoring transport actions
Original commit: elastic/x-pack-elasticsearch@4c8735d4a8
2016-05-05 15:28:04 -04:00
Areek Zillur f808b251b5 Disable monitoring transport and rest actions on tribe node
Original commit: elastic/x-pack-elasticsearch@a9d97b4f64
2016-05-05 15:27:45 -04:00
Areek Zillur d9e9f7dfd0 Disable licensing services and management APIs for tribe node
closes elastic/elasticsearch#1426

Original commit: elastic/x-pack-elasticsearch@d8a312b1b5
2016-05-05 15:10:05 -04:00
Alexander Reelsen 2cd7c74bc7 Security: Replace `_shield/` urls with `_xpack/security` (elastic/elasticsearch#2174)
This changes the security endpoints to _xpack/security, fixes the rest api spec to also use
the xpack.security prefix and adds documentation and tests.

Original commit: elastic/x-pack-elasticsearch@7977575f0e
2016-05-04 21:42:11 +02:00
Jason Tedor ed26294916 Fix HttpExporterTemplateTests from string split
This commit fixes an issue in HttpExporterTemplateTests caused by the
migration from Strings#splitStringToArray to String#split. Namely, the
previous would split a string like "/x/y/z/" into { "x", "y", "z" } but
the former will split this into { "", "x", "y", "z" }. This commit
modifies the test logic to respond to this change.

Original commit: elastic/x-pack-elasticsearch@c567b17180
2016-05-04 11:29:50 -04:00
Jason Tedor 5b12eef2d3 Fix ObjectPath#evalContext edge cases
This commit fixes an issue that was introduced in ObjectPath#evalContext
when refactoring from Strings#splitStringToArray to
String#split. Namely, the former would return an empty array when
receiving a null or empty string as input but the latter will NPE on a
null string and return an array containing the empty string on an empty
string input.

Original commit: elastic/x-pack-elasticsearch@2f509f9fa0
2016-05-04 11:21:58 -04:00
Christoph Büscher 357f0178e9 Adapt to changes in QueryShardContext
Original commit: elastic/x-pack-elasticsearch@36b97cac75
2016-05-04 16:42:52 +02:00
Jason Tedor 4a1591f2a8 Remove Strings#splitStringToArray
This commit replaces the uses of Strings#splitStringToArray in favor of
String#split as this method has been removed from core.

Relates elastic/elasticsearch#2175

Original commit: elastic/x-pack-elasticsearch@97ec094fa0
2016-05-04 10:39:05 -04:00
Alexander Reelsen a243647ea1 Watcher: Move urls from _watcher to _xpack/watcher
This moves the watcher base URL to _xpack/watcher. This includes
code, tests, rest-api-spec and the documentation.

Relates elastic/elasticsearch#1760

Original commit: elastic/x-pack-elasticsearch@0a44aec022
2016-05-04 09:39:47 +02:00
Alexander Reelsen 1aedda3627 X-Pack: Create notification module
In order to move things from watcher to x-pack this created a notification module in x-pack.
This also means that the HTTPClient was moved up and settings have changed from
`xpack.watcher.http` to just `xpack.http`.

Further things done:

* Move http under o.e.xpack.common
* Moved secret service to o.e.xpack.common, initializing in XpackPlugin
* Moved text template to o.e.xpack.common.text
* Moved http client initialization into xpack plugin
* Renamed xpack.watcher.encrypt_sensitive_data setting, moved into Watch class
* Moved script service proxy to common

Original commit: elastic/x-pack-elasticsearch@41eb6e6946
2016-05-04 08:53:29 +02:00
Daniel Mitterdorfer 7eebacc884 Disable HTTP compression by default when HTTPS is enabled.
With elastic/elasticsearchelastic/elasticsearch#7309 we enable HTTP compression by
default. However, this can pose a security risk for HTTPS
traffic (e.g. BREACH attack). Hence, we disable HTTP compression
by default again if HTTPS enabled (note that this still allows the
user to explicitly enable HTTP compression if they want to).

Relates elastic/elaticsearchelastic/elasticsearch#7309

Original commit: elastic/x-pack-elasticsearch@8da100c9a5
2016-05-03 08:54:57 +02:00
Alexander Reelsen 23ebbed95a Watcher: Expose HTTP response headers in payload
This exposes the headers (all lower-cased) in the payload, so
that the can be accessed in the conditions.

Closes elastic/elasticsearch#1560

Original commit: elastic/x-pack-elasticsearch@c9b08558fe
2016-05-02 15:06:38 +02:00
Alexander Reelsen 74edbe6332 Watcher: Refactoring, move to org.elasticsearch.xpack
This refactors the org.elasticsearch.watcher over to
org.elasticsearch.xpack.watcher

This also adds all watcher actions to the KnownActionsTests,
as watcher actions had not been taken care of until here.

Original commit: elastic/x-pack-elasticsearch@a046dc7c6a
2016-05-02 10:58:34 +02:00
jaymode 773876caee security: ssl by default on the transport layer
This commit adds the necessary changes to make SSL work on the transport layer by default. A large
portion of the SSL configuration/settings was re-worked with this change. Some notable highlights
include support for PEM cert/keys, reloadable SSL configuration, separate HTTP ssl configuration, and
separate LDAP configuration.

The following is a list of specific items addressed:

* `SSLSettings` renamed to `SSLConfiguration`
* `KeyConfig` and `TrustConfig` abstractions created. These hide the details of how `KeyManager[]` and `TrustManager[]` are loaded. These are also responsible for settings validation (ie keystore password is not null)
* Configuration fallback is changed. Previously any setting would fallback to the "global" value (`xpack.security.ssl.*`). Now a keystore path, key path, ca paths, or truststore path must be specified otherwise the configuration for that key/trust will fallback to the global configuration. In other words if you want to change part of a keystore or truststore in a profile you need to supply all the information. This could be considered breaking if a user relied on the old fallback
* JDK trusted certificates (`cacerts`) are trusted by default (breaking change). This can be disabled via a setting.
* We now monitor the SSL files for changes and enable dynamic reloading of the configuration. This will make it easier for users when they are getting set up with certificates so they do not need to restart every time. This can be disabled via a setting
* LDAP realms can now have their own SSL configurations
* HTTP can now have its own SSL configuration
* SSL is enabled by default on the transport layer only. Hostname verification is enabled as well. On startup if no global SSL settings are present and SSL is configured to be used, we auto generate one based on the default CA that is shipped. This process includes a best effort attempt to generate the subject alternative names.
* `xpack.security.ssl.hostname_verification` is deprecated in favor of `xpack.security.ssl.hostname_verification.enabled`
* added Bouncy Castle info to NOTICE
* consolidated NOTICE and LICENSE files

Closes elastic/elasticsearch#14
Closes elastic/elasticsearch#34
Closes elastic/elasticsearch#1483
Closes elastic/elasticsearch#1933
Addresses security portion of elastic/elasticsearch#673

Original commit: elastic/x-pack-elasticsearch@7c359db90b
2016-04-29 12:50:07 -04:00
markharwood 29b996ea1d Test fix - graph test occasionally failed to fail on all shards due to random nature of indexing. Tightened test class logic to deal with partial failures.
Original commit: elastic/x-pack-elasticsearch@b2dcdd7600
2016-04-29 14:45:15 +01:00
jaymode de48b2426b change how audit user is compared, do not setDaemon, test cleanup
This commit makes a few modifications to the IndexAuditTrail class:

* Use `InternalAuditUser#is` to determine if the principal is the auditor when we have a user
and simply compare `InternalAuditUser#NAME` when only a string principal is available
* Remove the `Thread#setDaemon` call in the QueueConsumer as this thread should be terminated
as part of the shutdown of the node

In terms of tests, there are some issues and changes to how we test certain aspects. The muted tests
were not accurate since the tests immediately checked for the existence of an index and did not poll or
wait and this operation is asynchronous so the index could be created after the exists request was
executed. These tests were removed and a new class was added to test the muted behavior. In these
tests we override the audit trails implementation of a queue, which will set a flag to indicate a message
has been added to the queue. This is a synchronous operation so it can be checked immediately.

The other tests in the IndexAuditTrail tests remain but a few changes have been made to the execution.

* ensureYellow is called for the index we expect to be created before searching for documents
* the remote cluster is only setup at the beginning of the suite rather than before every test to ensure
quicker execution
* the maximum number of shards has been reduced to three since we do not really need up to 10 shards
for a single document

Original commit: elastic/x-pack-elasticsearch@501b6ce9da
2016-04-29 09:08:10 -04:00
Alexander Reelsen 27f0a68a28 X-Pack Notification: Settings refactoring, removed 'service'
The service part is now obsolete with moving to `xpack.notification`.

Original commit: elastic/x-pack-elasticsearch@a7907f24a5
2016-04-29 09:02:36 +02:00
Ryan Ernst 4be1266616 Fix xpack rest test with new xpack info output (timestamp instead of date)
Original commit: elastic/x-pack-elasticsearch@ccb89481cf
2016-04-28 11:46:31 -07:00
Ryan Ernst 09a0276a56 Merge pull request elastic/elasticsearch#2123 from rjernst/build_info
Build: use jar metadata instead of expecting a properties file for xpack build info

Original commit: elastic/x-pack-elasticsearch@a7238cf527
2016-04-28 08:58:52 -07:00
jaymode c39b3ba2fc security: add the proper behavior for the standard license
This change adds the proper behavior for the standard license which is:

* authentication is enabled but only the reserved, native and file realms are available
* authorization is enabled

Features that are disabled:

* auditing
* ip filtering
* custom realms
* LDAP, Active Directory, PKI realms

See elastic/elasticsearch#1263

Original commit: elastic/x-pack-elasticsearch@920c045bf1
2016-04-28 09:33:57 -04:00
markharwood 077599b63f X-plugin tests - added testing for Standard licence in graph plugin. See https://github.com/elastic/x-plugins/issues/1263
Original commit: elastic/x-pack-elasticsearch@6773ead0fc
2016-04-28 13:51:43 +01:00
jaymode 91943318bf security: cleanup authentication service
This commit removes duplicated code in the authentication service by combining
the authentication logic for rest and transport requests. As part of this we no longer
cache the authentication token since we put the user in the context and serialize the
user.

Additionally we now pass the thread context to the AuthenticationFailureHandler to
restore access to the headers and context.

Original commit: elastic/x-pack-elasticsearch@79e2375a13
2016-04-28 07:59:16 -04:00
jaymode 4f7dad8da2 security: handle null values for full name and email
This commit adds logic so that we properly handle null tokens for full name and
email.

Closes elastic/elasticsearch#1887

Original commit: elastic/x-pack-elasticsearch@e03188c29f
2016-04-28 07:41:27 -04:00
Alexander Reelsen 917101f7a3 Smoke Testing: Add smoke tester for licensing
In order to prevent shipping of RCs with the wrong license, this
smoke tester downloads the internal RC, installs x-pack and puts
a license in there.

if putting is successful, we can be sure, we got the right license.

Closes elastic/elasticsearch#2087

Original commit: elastic/x-pack-elasticsearch@021d228e29
2016-04-28 08:47:31 +02:00
Ryan Ernst 4d1f4a244a Build: use jar metadata instead of expecting a properties file for xpack
build info

There are many other things that should be cleaned up around this (eg
XpackInfoResponse.BuildInfo should not exist, it is the exact same as
what XPackBuild has), but this change gets the build info output working
again.

closes elastic/elasticsearch#2116

Original commit: elastic/x-pack-elasticsearch@0730daf031
2016-04-27 13:33:42 -07:00
jaymode f4f156b351 test: add awaits fix to FLS field stats tests
See elastic/elasticsearch#2120

Original commit: elastic/x-pack-elasticsearch@fc7950bf65
2016-04-27 13:55:59 -04:00
Alexander Reelsen 5d53080a1f Watcher: Remove build based property creation (elastic/elasticsearch#2107)
There we still left over files from the clean up PR to not use
build properties for the watcher templates.

Relates elastic/elasticsearch#2040

Original commit: elastic/x-pack-elasticsearch@b838d92124
2016-04-26 17:54:27 +02:00
Alexander Reelsen 3bbe5916d1 Fix compilation issue
Original commit: elastic/x-pack-elasticsearch@803275d634
2016-04-26 14:03:19 +02:00
Alexander Reelsen 91242f3a98 Tests: Increase logging for tests for randomly failing tests
Relates elastic/elasticsearch#2090

Original commit: elastic/x-pack-elasticsearch@4051354f45
2016-04-25 17:46:09 +02:00
jaymode c7ad6b9872 test: add a simple test for reserved realm authentication
See elastic/elasticsearch#2089

Original commit: elastic/x-pack-elasticsearch@1bede0a206
2016-04-25 07:34:14 -04:00
Alexander Reelsen b47d161b9e X-Pack: Porting watcher notifications to xpack notifications (elastic/elasticsearch#2056)
This mainly moves packages over to the x-pack directory and renames the settings
from `xpack.watcher.actions.` to `xpack.notification.`

Moved services include pagerduty, hipchat, slack and email.

Closes elastic/elasticsearch#1998

Original commit: elastic/x-pack-elasticsearch@40c16fe123
2016-04-22 15:57:34 +02:00