Commit Graph

538 Commits

Author SHA1 Message Date
Adrien Grand 5db861d14b Remove generics from QueryBuilder.
Related to elastic/elasticsearchelastic/elasticsearch#18133.

Original commit: elastic/x-pack-elasticsearch@bd8ca4f061
2016-05-06 09:06:30 +02:00
Adrien Grand 22c4fb9a32 Make compilation pass.
Note: tests are still failing.

Original commit: elastic/x-pack-elasticsearch@827d129876
2016-05-06 09:05:45 +02:00
Areek Zillur bd04cc9d1f Extend tribe integ test infra to test on master and client nodes
Original commit: elastic/x-pack-elasticsearch@5826fb4161
2016-05-05 15:28:04 -04:00
Areek Zillur 3f0acdd70e refactor tribe integ tests to test monitoring transport actions
Original commit: elastic/x-pack-elasticsearch@4c8735d4a8
2016-05-05 15:28:04 -04:00
Areek Zillur f808b251b5 Disable monitoring transport and rest actions on tribe node
Original commit: elastic/x-pack-elasticsearch@a9d97b4f64
2016-05-05 15:27:45 -04:00
Areek Zillur d9e9f7dfd0 Disable licensing services and management APIs for tribe node
closes elastic/elasticsearch#1426

Original commit: elastic/x-pack-elasticsearch@d8a312b1b5
2016-05-05 15:10:05 -04:00
Alexander Reelsen 2cd7c74bc7 Security: Replace `_shield/` urls with `_xpack/security` (elastic/elasticsearch#2174)
This changes the security endpoints to _xpack/security, fixes the rest api spec to also use
the xpack.security prefix and adds documentation and tests.

Original commit: elastic/x-pack-elasticsearch@7977575f0e
2016-05-04 21:42:11 +02:00
Jason Tedor ed26294916 Fix HttpExporterTemplateTests from string split
This commit fixes an issue in HttpExporterTemplateTests caused by the
migration from Strings#splitStringToArray to String#split. Namely, the
previous would split a string like "/x/y/z/" into { "x", "y", "z" } but
the former will split this into { "", "x", "y", "z" }. This commit
modifies the test logic to respond to this change.

Original commit: elastic/x-pack-elasticsearch@c567b17180
2016-05-04 11:29:50 -04:00
Jason Tedor 5b12eef2d3 Fix ObjectPath#evalContext edge cases
This commit fixes an issue that was introduced in ObjectPath#evalContext
when refactoring from Strings#splitStringToArray to
String#split. Namely, the former would return an empty array when
receiving a null or empty string as input but the latter will NPE on a
null string and return an array containing the empty string on an empty
string input.

Original commit: elastic/x-pack-elasticsearch@2f509f9fa0
2016-05-04 11:21:58 -04:00
Christoph Büscher 357f0178e9 Adapt to changes in QueryShardContext
Original commit: elastic/x-pack-elasticsearch@36b97cac75
2016-05-04 16:42:52 +02:00
Jason Tedor 4a1591f2a8 Remove Strings#splitStringToArray
This commit replaces the uses of Strings#splitStringToArray in favor of
String#split as this method has been removed from core.

Relates elastic/elasticsearch#2175

Original commit: elastic/x-pack-elasticsearch@97ec094fa0
2016-05-04 10:39:05 -04:00
Alexander Reelsen a243647ea1 Watcher: Move urls from _watcher to _xpack/watcher
This moves the watcher base URL to _xpack/watcher. This includes
code, tests, rest-api-spec and the documentation.

Relates elastic/elasticsearch#1760

Original commit: elastic/x-pack-elasticsearch@0a44aec022
2016-05-04 09:39:47 +02:00
Alexander Reelsen 1aedda3627 X-Pack: Create notification module
In order to move things from watcher to x-pack this created a notification module in x-pack.
This also means that the HTTPClient was moved up and settings have changed from
`xpack.watcher.http` to just `xpack.http`.

Further things done:

* Move http under o.e.xpack.common
* Moved secret service to o.e.xpack.common, initializing in XpackPlugin
* Moved text template to o.e.xpack.common.text
* Moved http client initialization into xpack plugin
* Renamed xpack.watcher.encrypt_sensitive_data setting, moved into Watch class
* Moved script service proxy to common

Original commit: elastic/x-pack-elasticsearch@41eb6e6946
2016-05-04 08:53:29 +02:00
Daniel Mitterdorfer 7eebacc884 Disable HTTP compression by default when HTTPS is enabled.
With elastic/elasticsearchelastic/elasticsearch#7309 we enable HTTP compression by
default. However, this can pose a security risk for HTTPS
traffic (e.g. BREACH attack). Hence, we disable HTTP compression
by default again if HTTPS enabled (note that this still allows the
user to explicitly enable HTTP compression if they want to).

Relates elastic/elaticsearchelastic/elasticsearch#7309

Original commit: elastic/x-pack-elasticsearch@8da100c9a5
2016-05-03 08:54:57 +02:00
Alexander Reelsen 23ebbed95a Watcher: Expose HTTP response headers in payload
This exposes the headers (all lower-cased) in the payload, so
that the can be accessed in the conditions.

Closes elastic/elasticsearch#1560

Original commit: elastic/x-pack-elasticsearch@c9b08558fe
2016-05-02 15:06:38 +02:00
Alexander Reelsen 74edbe6332 Watcher: Refactoring, move to org.elasticsearch.xpack
This refactors the org.elasticsearch.watcher over to
org.elasticsearch.xpack.watcher

This also adds all watcher actions to the KnownActionsTests,
as watcher actions had not been taken care of until here.

Original commit: elastic/x-pack-elasticsearch@a046dc7c6a
2016-05-02 10:58:34 +02:00
jaymode 773876caee security: ssl by default on the transport layer
This commit adds the necessary changes to make SSL work on the transport layer by default. A large
portion of the SSL configuration/settings was re-worked with this change. Some notable highlights
include support for PEM cert/keys, reloadable SSL configuration, separate HTTP ssl configuration, and
separate LDAP configuration.

The following is a list of specific items addressed:

* `SSLSettings` renamed to `SSLConfiguration`
* `KeyConfig` and `TrustConfig` abstractions created. These hide the details of how `KeyManager[]` and `TrustManager[]` are loaded. These are also responsible for settings validation (ie keystore password is not null)
* Configuration fallback is changed. Previously any setting would fallback to the "global" value (`xpack.security.ssl.*`). Now a keystore path, key path, ca paths, or truststore path must be specified otherwise the configuration for that key/trust will fallback to the global configuration. In other words if you want to change part of a keystore or truststore in a profile you need to supply all the information. This could be considered breaking if a user relied on the old fallback
* JDK trusted certificates (`cacerts`) are trusted by default (breaking change). This can be disabled via a setting.
* We now monitor the SSL files for changes and enable dynamic reloading of the configuration. This will make it easier for users when they are getting set up with certificates so they do not need to restart every time. This can be disabled via a setting
* LDAP realms can now have their own SSL configurations
* HTTP can now have its own SSL configuration
* SSL is enabled by default on the transport layer only. Hostname verification is enabled as well. On startup if no global SSL settings are present and SSL is configured to be used, we auto generate one based on the default CA that is shipped. This process includes a best effort attempt to generate the subject alternative names.
* `xpack.security.ssl.hostname_verification` is deprecated in favor of `xpack.security.ssl.hostname_verification.enabled`
* added Bouncy Castle info to NOTICE
* consolidated NOTICE and LICENSE files

Closes elastic/elasticsearch#14
Closes elastic/elasticsearch#34
Closes elastic/elasticsearch#1483
Closes elastic/elasticsearch#1933
Addresses security portion of elastic/elasticsearch#673

Original commit: elastic/x-pack-elasticsearch@7c359db90b
2016-04-29 12:50:07 -04:00
markharwood 29b996ea1d Test fix - graph test occasionally failed to fail on all shards due to random nature of indexing. Tightened test class logic to deal with partial failures.
Original commit: elastic/x-pack-elasticsearch@b2dcdd7600
2016-04-29 14:45:15 +01:00
jaymode de48b2426b change how audit user is compared, do not setDaemon, test cleanup
This commit makes a few modifications to the IndexAuditTrail class:

* Use `InternalAuditUser#is` to determine if the principal is the auditor when we have a user
and simply compare `InternalAuditUser#NAME` when only a string principal is available
* Remove the `Thread#setDaemon` call in the QueueConsumer as this thread should be terminated
as part of the shutdown of the node

In terms of tests, there are some issues and changes to how we test certain aspects. The muted tests
were not accurate since the tests immediately checked for the existence of an index and did not poll or
wait and this operation is asynchronous so the index could be created after the exists request was
executed. These tests were removed and a new class was added to test the muted behavior. In these
tests we override the audit trails implementation of a queue, which will set a flag to indicate a message
has been added to the queue. This is a synchronous operation so it can be checked immediately.

The other tests in the IndexAuditTrail tests remain but a few changes have been made to the execution.

* ensureYellow is called for the index we expect to be created before searching for documents
* the remote cluster is only setup at the beginning of the suite rather than before every test to ensure
quicker execution
* the maximum number of shards has been reduced to three since we do not really need up to 10 shards
for a single document

Original commit: elastic/x-pack-elasticsearch@501b6ce9da
2016-04-29 09:08:10 -04:00
Alexander Reelsen 27f0a68a28 X-Pack Notification: Settings refactoring, removed 'service'
The service part is now obsolete with moving to `xpack.notification`.

Original commit: elastic/x-pack-elasticsearch@a7907f24a5
2016-04-29 09:02:36 +02:00
Ryan Ernst 4be1266616 Fix xpack rest test with new xpack info output (timestamp instead of date)
Original commit: elastic/x-pack-elasticsearch@ccb89481cf
2016-04-28 11:46:31 -07:00
Ryan Ernst 09a0276a56 Merge pull request elastic/elasticsearch#2123 from rjernst/build_info
Build: use jar metadata instead of expecting a properties file for xpack build info

Original commit: elastic/x-pack-elasticsearch@a7238cf527
2016-04-28 08:58:52 -07:00
jaymode c39b3ba2fc security: add the proper behavior for the standard license
This change adds the proper behavior for the standard license which is:

* authentication is enabled but only the reserved, native and file realms are available
* authorization is enabled

Features that are disabled:

* auditing
* ip filtering
* custom realms
* LDAP, Active Directory, PKI realms

See elastic/elasticsearch#1263

Original commit: elastic/x-pack-elasticsearch@920c045bf1
2016-04-28 09:33:57 -04:00
markharwood 077599b63f X-plugin tests - added testing for Standard licence in graph plugin. See https://github.com/elastic/x-plugins/issues/1263
Original commit: elastic/x-pack-elasticsearch@6773ead0fc
2016-04-28 13:51:43 +01:00
jaymode 91943318bf security: cleanup authentication service
This commit removes duplicated code in the authentication service by combining
the authentication logic for rest and transport requests. As part of this we no longer
cache the authentication token since we put the user in the context and serialize the
user.

Additionally we now pass the thread context to the AuthenticationFailureHandler to
restore access to the headers and context.

Original commit: elastic/x-pack-elasticsearch@79e2375a13
2016-04-28 07:59:16 -04:00
jaymode 4f7dad8da2 security: handle null values for full name and email
This commit adds logic so that we properly handle null tokens for full name and
email.

Closes elastic/elasticsearch#1887

Original commit: elastic/x-pack-elasticsearch@e03188c29f
2016-04-28 07:41:27 -04:00
Alexander Reelsen 917101f7a3 Smoke Testing: Add smoke tester for licensing
In order to prevent shipping of RCs with the wrong license, this
smoke tester downloads the internal RC, installs x-pack and puts
a license in there.

if putting is successful, we can be sure, we got the right license.

Closes elastic/elasticsearch#2087

Original commit: elastic/x-pack-elasticsearch@021d228e29
2016-04-28 08:47:31 +02:00
Ryan Ernst 4d1f4a244a Build: use jar metadata instead of expecting a properties file for xpack
build info

There are many other things that should be cleaned up around this (eg
XpackInfoResponse.BuildInfo should not exist, it is the exact same as
what XPackBuild has), but this change gets the build info output working
again.

closes elastic/elasticsearch#2116

Original commit: elastic/x-pack-elasticsearch@0730daf031
2016-04-27 13:33:42 -07:00
jaymode f4f156b351 test: add awaits fix to FLS field stats tests
See elastic/elasticsearch#2120

Original commit: elastic/x-pack-elasticsearch@fc7950bf65
2016-04-27 13:55:59 -04:00
Alexander Reelsen 5d53080a1f Watcher: Remove build based property creation (elastic/elasticsearch#2107)
There we still left over files from the clean up PR to not use
build properties for the watcher templates.

Relates elastic/elasticsearch#2040

Original commit: elastic/x-pack-elasticsearch@b838d92124
2016-04-26 17:54:27 +02:00
Alexander Reelsen 3bbe5916d1 Fix compilation issue
Original commit: elastic/x-pack-elasticsearch@803275d634
2016-04-26 14:03:19 +02:00
Alexander Reelsen 91242f3a98 Tests: Increase logging for tests for randomly failing tests
Relates elastic/elasticsearch#2090

Original commit: elastic/x-pack-elasticsearch@4051354f45
2016-04-25 17:46:09 +02:00
jaymode c7ad6b9872 test: add a simple test for reserved realm authentication
See elastic/elasticsearch#2089

Original commit: elastic/x-pack-elasticsearch@1bede0a206
2016-04-25 07:34:14 -04:00
Alexander Reelsen b47d161b9e X-Pack: Porting watcher notifications to xpack notifications (elastic/elasticsearch#2056)
This mainly moves packages over to the x-pack directory and renames the settings
from `xpack.watcher.actions.` to `xpack.notification.`

Moved services include pagerduty, hipchat, slack and email.

Closes elastic/elasticsearch#1998

Original commit: elastic/x-pack-elasticsearch@40c16fe123
2016-04-22 15:57:34 +02:00
Martijn van Groningen 358fa38cf6 test: fix id, script_lang mix up
Original commit: elastic/x-pack-elasticsearch@7c4a3152ba
2016-04-22 15:12:35 +02:00
Martijn van Groningen 4650592150 Remove LazyInitializable from ScriptServiceProxy
Closes elastic/elasticsearch#2062

Original commit: elastic/x-pack-elasticsearch@4eaf323158
2016-04-22 14:31:02 +02:00
Martijn van Groningen b9515357fa Migrated from indexed scripts to store scripts
Original commit: elastic/x-pack-elasticsearch@a0218f1c9e
2016-04-22 13:43:55 +02:00
Alexander Reelsen 276d5fbbca Watcher: Updated dependencies (elastic/elasticsearch#2064)
Updated okhttp and moved the jsr305 dependency into testing.
This required a minor change in tests using SSL, as otherwise
the security manager barfs, when the okhttp webserver tries
to load sun internal SSL based classes.

Original commit: elastic/x-pack-elasticsearch@77131589e0
2016-04-22 09:45:46 +02:00
Alexander Reelsen 12ff8853f0 Monitoring/Watcher: Load version of templates in a static way (elastic/elasticsearch#2040)
The old implementation was to use properties at build-time. This however did not work,
as the tests could not be run in the IDE. This has been removed of monitoring for some
time already, but needs to be removed from watcher as well.

This commit uses static variables and refactors the code a bit. First, there is a generic
TemplateUtils class, to be used in monitoring and watcher. Also the watcher code has been changed
to copy the needed variables into the template registry class instead of keeping it in the
WatcherModule.

This commit also includes some refactoring to remove the version parameter in marvel, was static anyway

Closes elastic/elasticsearch#1372

Original commit: elastic/x-pack-elasticsearch@fbfc22ea09
2016-04-22 09:26:40 +02:00
uboness df3bbd42b9 Changed the default output of X-Pack Info API
- by default the response includes all info - build, license, features + human descriptions.
- you can still control the output using `categories` and `human` parameters
- Added docs to this API

Original commit: elastic/x-pack-elasticsearch@85115495ec
2016-04-21 18:43:17 -07:00
Chris Earle a84347f711 Monitoring: Ignore NodesStatsResposne if no stats are returned
This avoids exceptional cases where node stats are not returned due to some concurrent modification.

Original commit: elastic/x-pack-elasticsearch@6f6b8ec393
2016-04-21 16:16:00 -04:00
Nik Everett 629c585fba Handle core removing <T> from Writeable
Original commit: elastic/x-pack-elasticsearch@34632c8a67
2016-04-21 13:00:57 -04:00
Nik Everett c4dc28e7f7 Remove the last readFrom from xpack
This the last Writeable#readFrom in xpack!

Original commit: elastic/x-pack-elasticsearch@5412160bdd
2016-04-21 10:13:10 -04:00
jaymode 8c8e33889c build: remove test dependencies from published pom file
See elastic/elasticsearch#2063

Original commit: elastic/x-pack-elasticsearch@3653368363
2016-04-21 09:27:12 -04:00
Nik Everett de6d3e1a72 Remove readFrom from xpack
Writeable#readFrom has become a method you just implement because
the interface requires it but the prefered way to actually do the
reading is a ctor that takes a StreamReader. readFrom just delegates
to the ctor. This removes readFrom entirely because it is not needed
anymore and is going away in core.

Relates to https://github.com/elastic/elasticsearch/issues/17085

Original commit: elastic/x-pack-elasticsearch@dd74db5ded
2016-04-21 07:58:51 -04:00
uboness 5c9d96211f Extended X-Pack Info API with Features Info
- introduced the "Feature Set" notion - graph, security, monitoring, watcher, these are all feature sets
- each feature set can be:
 - `available` - indicates whether this feature set is available under the current license
 - `enabled` - indicates whether this feature set is enabled (note that the feature set can be enabled, yet unavailable under the current license)
- while at it, cleaned up the main modules of watcher, security, monitoring and graph.

Original commit: elastic/x-pack-elasticsearch@5b3e19fe8c
2016-04-20 14:30:48 -07:00
Jay Greenberg 8af3f91eb5 Merge pull request elastic/elasticsearch#2044 from PhaedrusTheGreek/group_search_noattrs
Change some LDAP searches to NOATTRS to avoid unnecessary results

Original commit: elastic/x-pack-elasticsearch@60c41af5a6
2016-04-20 09:29:38 -04:00
jaymode 659439841e test: adapt to removal of setting
Original commit: elastic/x-pack-elasticsearch@5f195001b9
2016-04-19 14:31:06 -04:00
Nik Everett 28bb39955c Replace (read|write)Query with (read|write)NamedWriteable
(read|write)Query is going away.

Original commit: elastic/x-pack-elasticsearch@5ac3ded68e
2016-04-19 11:06:39 -04:00
PhaedrusTheGreek 962729bd3b Changed LDAP searches to NOATTRS in order to avoid returning unecessary
data in searches where only getDn() is done on results

Original commit: elastic/x-pack-elasticsearch@5ce64235a1
2016-04-19 10:47:27 -04:00