Commit Graph

1562 Commits

Author SHA1 Message Date
Chris Earle d210213fc9 [Monitoring] Use Cgroup Data from _nodes/stats (elastic/elasticsearch#4484)
* [Monitoring] Use Cgroup Data from _nodes/stats

This adds Cgroup data from _nodes/stats.

- This also adds 5m and 15m from system load, which are sent from ES, but
  were not recorded. Kibana does record/report those values though.

Original commit: elastic/x-pack-elasticsearch@8e04452c60
2016-12-30 18:55:51 -05:00
Chris Earle 2bce702f62 [Monitoring] Add Logstash monitoring feature
This PR adds UI for visualizing Logstash internal metrics with contributions from @ph, @jsvd and @pickypg

Changes include:

* x-pack component for Logstash which installs a monitoring exporter plugin to Logstash core. This periodically ships
  monitoring data to the monitoring bulk API endpoint.
* Adds xpack.monitoring.* setting to logstash.yml when x-pack is installed.
* UI changes to graph Logstash monitoring data. Logstash processes are called nodes. They send separate monitoring info
  which gets aggregated and displayed at cluster level and also at individual node level.
* Adds gradle build support for Logstash x-pack which can be controled via `xpack.logstash.build=false`

Fixes elastic/elasticsearch#4169

Original commit: elastic/x-pack-elasticsearch@f58ef406c1
2016-12-30 10:36:11 -08:00
Luca Cavanna 37a29c0387 Remove some usages of ParseFieldMatcher in favour of using ParseField directly (elastic/elasticsearch#4492)
Relates to elastic/elasticsearch#19552
Relates to elastic/elasticsearch#22130

Original commit: elastic/x-pack-elasticsearch@28f8e3cca4
2016-12-30 18:58:03 +01:00
Jay Modi 39263d83d2 ldap and ad realms should obey the verification_mode setting (elastic/elasticsearch#4486)
When migrating realms to use the new settings for SSL, the removal of the
hostname_verification setting was missed in addition to actually using the
ssl.verification_mode setting. This change deprecates the
hostname_verification and enables the use of the ssl.verification_mode setting.

Original commit: elastic/x-pack-elasticsearch@ea24e83fd3
2016-12-30 10:04:56 -05:00
Jay Modi 18a2cf23d4 Build a single role that represents a user's permissions (elastic/elasticsearch#4449)
This PR changes how we use roles and how we look at the roles of a user. Previously we looked up each role individually, parsed each into their own `Role` object, and had a wrapper that essentially served as an iterator over the roles. The same pattern was also followed for the permissions that composed a role (ClusterPermission, IndicesPermission, and RunAsPermission). This resulted in a lot of code that was hard to follow and could be inefficient. 

Now, we look up the roles for a user in bulk and only get the RoleDescriptor for each role. Once all role descriptors have been retrieved, we build a single Role that represents the user's permissions and we also cache this combination for better performance as authorization can happen many times for a single top level request as we authorize the top level request and any sub requests, which could be a large number in the case of shard requests.

This change also enabled a large cleanup of our permission and privilege classes, which should reduce the footprint of what needs to be followed. Some of the notable changes are:

* Consolidation of GeneralPrivilege and AbstractAutomatonPrivilege into the Privilege class
* The DefaultRole class has been removed and the permissions it provided were moved into the AuthorizationService
* The GlobalPermission class was removed as there is a single role that represents a user's permissions
* The Global inner classes for the various permissions were removed
* The Core inner class was removed and ClusterPermission, IndexPermission, RunAsPermission became final classes instead of interfaces
* The Permission interface has been removed. The isEmpty() method defined by this interface is not needed as we can simply evaluate the permission to get the same effect
* The ClusterPermission#check method only takes the action name again
* The AutomatonPredicate class was removed and replaced by Automatons#predicate
* IndicesAccessControl objects no longer need to be merged when evaluating permissions
* MergedFieldPermissions has been removed
* The Name class that was used to hold an array of strings has been removed and replaced with the use of a Set
* Privilege resolution is more efficient by only combining automata once

Other items:
* NativeRolesStore no longer does caching, so the RoleAndVersion class could be removed
* FileRolesStore doesn't need to be an AbstractLifecycleComponent

Relates elastic/elasticsearch#4327

Original commit: elastic/x-pack-elasticsearch@c1901bc82e
2016-12-30 09:27:49 -05:00
jaymode 9c09b88e9f watcher: do not continue execution when get result doesn't exist
When attempting to a get a watch that does not exist, the listener is called once inside
an if statement and the execution of the method continues as if the watch existed. This
causes failures to happen including a NPE. This commit wraps the execution in a if-else
to prevent this.

Original commit: elastic/x-pack-elasticsearch@27f09852e7
2016-12-30 08:34:39 -05:00
Adrien Grand 2b92ee71bc Fix compilation.
Original commit: elastic/x-pack-elasticsearch@8e845fe3b3
2016-12-30 10:54:20 +01:00
Igor Motov 7597fa8ae9 Use java.nio.charset.StandardCharsets instead of com.google.common.base.Charsets
We are trying to get rid of guava, so it doesn't make much sense to use it as a dependency unless we really need to.

Original commit: elastic/x-pack-elasticsearch@e2e2d58fba
2016-12-29 10:49:40 -05:00
Tim Vernum 22a32c8748 Add logstash_system user to X-Pack security reserved realm
The `logstash_system` user/role are used by logstash internals (e.g. monitoring), and work similarly to the Kibana user and role

Included features:  
- New logstash_system user/role (elastic/elasticsearch#4350, elastic/elasticsearch#4021)
- We no longer hide the reserved roles in the API (applies to both kibana and logstash_system)
- Automatically Disable logstash_system user on upgrade. If upgrading from a pre-5.2.0 version, automatically set the `logstash_system` user to disabled so that the system stays locked down. (elastic/elasticsearch#4445, elastic/elasticsearch#4394)

Closes: elastic/elasticsearch#4021
Closes: elastic/elasticsearch#4394 

Original commit: elastic/x-pack-elasticsearch@a9e6d1f216
2016-12-29 12:36:47 +11:00
Igor Motov 576a512ef2 Remove PROTO-based custom cluster state components
Switches custom cluster state components from PROTO-based de-serialization to named objects based de-serialization

Original commit: elastic/x-pack-elasticsearch@e1be8597ee
2016-12-28 12:38:03 -05:00
Yannick Welsch 505ed14c97 Separate cluster update tasks that are published from those that are not
Companion commit to elastic/elasticsearchelastic/elasticsearch#21912

Original commit: elastic/x-pack-elasticsearch@4ba569f8da
2016-12-23 12:21:59 +01:00
Jason Tedor 935eec54c5 Enable assertions in integration tests
This commit fixes some bugs that were found after enabling assertions in
integration tests.

Relates elastic/elasticsearch#4443

Original commit: elastic/x-pack-elasticsearch@4fdd84166d
2016-12-22 20:14:02 -05:00
Nik Everett 4ff6279865 Remove exception from client suite
Original commit: elastic/x-pack-elasticsearch@51e94561d9
2016-12-22 09:10:42 -05:00
Areek Zillur 84db1b8731 x-pack changes for elasticsearchelastic/elasticsearch#21964
In https://github.com/elastic/elasticsearch/pull/21964, index
and delete operations are executed as single item bulk requests
internally. This means index and delete operations use the
bulk transport endpoints (indices:data/write/bulk[s][p] and
indices:data/write/bulk[s][r]).

This PR adds bulk transport endpoint to 'write' and 'delete'
index privilages and adds index and delete action as composite
actions to delay the authentication to the shard level.

Original commit: elastic/x-pack-elasticsearch@2305fc9ca0
2016-12-22 02:35:35 -05:00
Jason Tedor d5db68702c Adapt to method name change in core
This commit adapts to the change of the method name
ESTestCase#randomPostivieLong to ESTestCase#randomNonNegativeLong.

Original commit: elastic/x-pack-elasticsearch@689429cb54
2016-12-21 23:19:52 -05:00
Daniel Mitterdorfer b973d10685 Improve BWC
Original commit: elastic/x-pack-elasticsearch@f84cb5b1fa
2016-12-21 16:43:55 +01:00
Nik Everett 483711ed20 Handle removal of IndiciesQueriesRegistry
It has been replaced by `XContentParser#namedObject`.

Original commit: elastic/x-pack-elasticsearch@fd316791a2
2016-12-21 08:01:49 -05:00
Daniel Mitterdorfer fa1219fd13 Merge remote-tracking branch 'origin/master' into strict-booleans
Original commit: elastic/x-pack-elasticsearch@6fbbaf19dd
2016-12-21 08:47:45 +01:00
Nik Everett b2dc1bdca3 Introduce NamedXContentRegistry (elastic/elasticsearch#4399)
This "super registry" will eventually replace things like
`IndiciesQueriesRegistry` but for now it is just another thing
to plumb across requests.

Original commit: elastic/x-pack-elasticsearch@da26a42b36
2016-12-20 11:05:42 -05:00
Alexander Reelsen 6f7a065605 Watcher: Parse index action results to prevent exceptions (elastic/elasticsearch#4424)
When the index action is used to do some bulk indexing, the single
items of the response were not checked to have been indexed successful.
This could lead to NPEs due to an index response being null when the index
operation had failed. The action was still logged as a success though.

This commit only returns SUCCESS for the action, if all items were indexed
successfully. If all items failed, the result will be FAILED as well. Lastly
the result status PARTIAL_FAILURE is used if there were successful and unsuccessful
index operations.

Additionally some minor cleanups happened, like changing equals/hashcode.

Closes elastic/elasticsearch#4416

Original commit: elastic/x-pack-elasticsearch@692687e1af
2016-12-20 16:53:12 +01:00
Alexander Reelsen 0f8e6354f6 Watcher: Allow deletion of watches regardless of license (elastic/elasticsearch#4414)
The user might want to delete watches on expiry to prevent
further execution, without any actions being executed.

Closes elastic/elasticsearch#4413

Original commit: elastic/x-pack-elasticsearch@244a1b9cd8
2016-12-20 15:26:06 +01:00
Yannick Welsch 68bfc90cea Let ClusterStateObserver only hold onto state that's needed for change detection
Companion commit for elastic/elasticsearchelastic/elasticsearch#21631

Original commit: elastic/x-pack-elasticsearch@3333480707
2016-12-20 15:23:00 +01:00
Alexander Reelsen b994f16f4a Watcher: Ensure non duplicate Wid ids (elastic/elasticsearch#4423)
Removing the WatchLockService could result in duplication of wids, because of a wrong
call to replace underscores with dashes. As UUIDs.createBase64UUID() can contain underscores
but they are kind of reserved in the Wid class due to handling of watch ids, this just uses
the toString() representation of a random UUID.

Closes elastic/elasticsearch#4422

Original commit: elastic/x-pack-elasticsearch@dceb01ae5e
2016-12-20 14:02:00 +01:00
Daniel Mitterdorfer b9a38d9b97 Enable trace logging for LocalExporterTests#testSimpleExport()
This test fails due to a changing cluster state(?).  The
test checks that a local exporter is ready and then continues.
However, during the test, we see output similar to:

skipping exporter [_local] as it isn't ready yet

Which indicates that the cluster state has changed and the
exporter does not return a bulk anymore. Hence, the test is
failing although at one point in time it returned a bulk.

By enabling trace logging we should be able to find out
what's going on.

Original commit: elastic/x-pack-elasticsearch@d7e2200dd9
2016-12-20 08:54:26 +01:00
Ryan Ernst 789df7d2fa Internal: Update signatures for EnvironmentAwareCommand (elastic/elasticsearch#4367)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#22175

Original commit: elastic/x-pack-elasticsearch@4359cb1947
2016-12-19 15:23:53 -08:00
javanna adf22c3d24 Adapt to strict parsing mode removal
Original commit: elastic/x-pack-elasticsearch@2d6276a60f
2016-12-19 19:39:08 +01:00
Alexander Reelsen 79a8f27569 Watcher: Remove WatchLockService (elastic/elasticsearch#4365)
The watch lock service is not really needed, as there is already
a data structure that has information about the currently executing
watches, that can be consulted before executed.

This change will now check, if there is already a watch running with
the current id. If there is not, execution will happen as usual. If
there is however, than a watch record will be created, stating that
the watch is currently being executed - which means that it is either
being executed or in the list of planned executions.

This way users can check in the watch history, if a watch has been executed
more often than it should.

In order to easily search for this, a new execution state called
`NOT_EXECUTED_ALREADY_QUEUED` has been added.

Original commit: elastic/x-pack-elasticsearch@867acec3c3
2016-12-19 17:33:48 +01:00
Daniel Mitterdorfer 2cb41e690d Address initial review comments
Original commit: elastic/x-pack-elasticsearch@bcd6433afb
2016-12-19 13:27:40 +01:00
Daniel Mitterdorfer 24a4290379 Merge remote-tracking branch 'origin/master' into strict-booleans
Original commit: elastic/x-pack-elasticsearch@32a533f38f
2016-12-19 12:58:30 +01:00
Daniel Mitterdorfer 7a652fa090 Merge pull request elastic/elasticsearch#4398 from danielmitterdorfer/xcontent_strict_duplicate_checks
Enable strict duplicate checks for all XContent types

Original commit: elastic/x-pack-elasticsearch@ad249c2e65
2016-12-19 09:30:19 +01:00
Tim Vernum df7c528c66 Change classification of event_type in index audit trail (elastic/elasticsearch#4375)
In LoggingAuditTrail and the audit-event filtering, we distinguish between single-realm authentication failures ("realm_authentication_failure") and global failures ("authentication_failure").

Update the message output of IndexAuditTrail to reflect this distinction.

Original commit: elastic/x-pack-elasticsearch@56802ae2df
2016-12-19 14:39:21 +11:00
Simon Willnauer c9cfedc3cb [TEST] check if .watches index / alias exists before creating it
Original commit: elastic/x-pack-elasticsearch@74d85121eb
2016-12-18 10:17:43 +01:00
Areek Zillur 278496d5e5 x-pack changes for removing deprecated _suggest endpoint (elasticsearchelastic/elasticsearch#22203)
Original commit: elastic/x-pack-elasticsearch@b9bba20fae
2016-12-16 13:06:35 -05:00
Daniel Mitterdorfer d3c1fbe002 Enable strict duplicate checks for all XContent types
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default for all XContent types (not only JSON).

We have also changed the name of the system property to disable this feature
from `es.json.strict_duplicate_detection` to the now more appropriate name
`es.xcontent.strict_duplicate_detection`.

Relates elastic/elasticsearchelastic/elasticsearch#19614
Relates elastic/elasticsearchelastic/elasticsearch#22073

Original commit: elastic/x-pack-elasticsearch@c92e2a6c55
2016-12-16 13:57:55 +01:00
Alexander Reelsen 0d9cb5f7bd Docs: Updating execution phases of watch execution
Closes elastic/elasticsearch#4079

Original commit: elastic/x-pack-elasticsearch@658ba0a7f7
2016-12-16 11:30:27 +01:00
Daniel Mitterdorfer e3302da589 Make boolean conversion strict
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.

Original commit: elastic/x-pack-elasticsearch@6400f18911
2016-12-15 19:06:59 +01:00
Boaz Leskes b2972a142c Remove Cluster State Status (elastic/elasticsearch#4313)
A companion PR for https://github.com/elastic/elasticsearch/pull/21817

Original commit: elastic/x-pack-elasticsearch@392235877f
2016-12-15 17:07:02 +01:00
Alexander Reelsen e6ee905931 Watcher: Remove unused class
Original commit: elastic/x-pack-elasticsearch@ecd48b7914
2016-12-15 11:33:36 +01:00
Ryan Ernst 6d4d599f91 Build: Convert xplugins to use new extra projects setup (elastic/elasticsearch#4175)
* Build: Convert xplugins to use new extra projects setup

This change makes the gradle initialization for xplugins look in the
correct location for elasticsearch, which is now as a sibling of an
elasticsearch-extra directory, with x-plugins as a child of the extra
directory.

The elasticsearch side of this change is
elastic/elasticsearchelastic/elasticsearch#21773. This change will enable renaming x-plugins
to x-pack, see elastic/elasticsearch#3643.

Original commit: elastic/x-pack-elasticsearch@09398aea5a
2016-12-14 15:02:28 -08:00
Nik Everett 597ca9275e Finish centralizing XContentParser creation in tests
Now we're ready to add `NamedXContentRegistry`.

Original commit: elastic/x-pack-elasticsearch@c4f6c7c3d6
2016-12-14 11:19:09 -05:00
Daniel Mitterdorfer feabaaf1d6 Add missing import
Original commit: elastic/x-pack-elasticsearch@fbc8eb7ac6
2016-12-14 09:34:55 +01:00
Daniel Mitterdorfer 83240e25b4 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@af25e460d0
2016-12-14 08:53:45 +01:00
Nik Everett d4939e1972 Start centralizing XContentParser creation in tests
Original commit: elastic/x-pack-elasticsearch@65be7ecd27
2016-12-13 20:28:19 -05:00
Jason Tedor 3f5d24ae55 Add shutdown hook for closing CLI commands
This commit adapts some CLI command tests for some changes in core.

Relates elastic/elasticsearch#4342

Original commit: elastic/x-pack-elasticsearch@1e48c1404f
2016-12-13 19:10:35 -05:00
Alexander Reelsen 1e7985563a Tests: Make MockWebServer thread-safe (elastic/elasticsearch#4351)
The new MockWebServer assumed that responses are inserted at the beginning
and removed later on. This was not thread safe.

Also this fixes a bug in the HttpExporterIT where there was no wait time
for a bulk request, even though the request execution is asynchronous.

Closes elastic/elasticsearch#4335

Original commit: elastic/x-pack-elasticsearch@11f31f68bd
2016-12-13 19:34:33 +01:00
Daniel Mitterdorfer 8be3250601 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@a010b1b4e9
2016-12-13 16:22:47 +01:00
Daniel Mitterdorfer 7cb0771c46 Use assume* instead of custom skipping logic
Original commit: elastic/x-pack-elasticsearch@0f3a01735f
2016-12-13 16:21:46 +01:00
Luca Cavanna 1c846dd893 Stop execution once destructive operations check has failed (elastic/elasticsearch#4337)
Otherwise we do return an error, but we also go ahead and open/close/delete the indices anyways.

Original commit: elastic/x-pack-elasticsearch@176eca4cff
2016-12-13 11:22:48 +01:00
Alexander Reelsen b57c4f6ebe Watcher: Remove in memory watch store (elastic/elasticsearch#4201)
In order to prepare to the distributed watch execution, this commit
removes the in memory watch store.

Whenever a watch is needed now, a get request is executed and the parsing
is done. This happens when

* Put
* Get
* Ack
* Activate/Deactivate
* Execute

Note: This also means there are no usage stats currently regarding
the watch count, because we would need to execute a query. This would
require the usage stats to be async, see elastic/elasticsearch#3569

Another advantage is, that there is no dirty flag in the watch itself
needed anymore, because the watch is always the latest. Also write
operations store immediately and dont leave anything in memory.

Also ActionListener.wrap() was used a lot instead of more verbose anonmyous
inner classes.

Original commit: elastic/x-pack-elasticsearch@c47465b47c
2016-12-13 08:54:03 +01:00
Tim Vernum 7192c46307 Define explicit settings for security realms/ssl (elastic/elasticsearch#4311)
Modified the definition and loading of settings in Security to provide early detection and failure of invalid (unrecognised or syntactically invalid) settings.
Also consolidates the number of places where settings were defined.

Each realm now defines its supported settings.
This is facilitated for custom realms via a new "getRealmSettings" method on XPackExtension.
The realm group setting performs validation of the child settings with reference to the "type".
For backwards compatibility reasons, realm types that have no configuration defined, will be accepted during validation, but may fail at realm creation time.

All SSL settings have been centralised into SSLConfigurationSettings, which supports a variable "prefix" to accommodate the multiple places we define SSL config.

HTTP Proxy settings are explicitly defined rather than being a generic group.

Where possible all security settings now reference a Setting object, and there are less magic strings scattered in the code.

Closes: elastic/elasticsearch#3965

Original commit: elastic/x-pack-elasticsearch@2c76a137a9
2016-12-13 16:14:02 +11:00
javanna 85679fcf19 Adapt to removal of support for empty queries
Original commit: elastic/x-pack-elasticsearch@87915a9cde
2016-12-12 13:34:30 +01:00
Daniel Mitterdorfer 050a36bb28 Address review comments in ES core
Original commit: elastic/x-pack-elasticsearch@925c78d4fe
2016-12-12 13:30:39 +01:00
Daniel Mitterdorfer b7d2ef0160 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@601a663982
2016-12-12 12:36:38 +01:00
Alexander Reelsen 779eb44b66 Watcher: Cleanup - Replace list of integers with bitset (elastic/elasticsearch#4324)
In the triggered watch store a list of integers was returned to keep track
which watches had been successfully stored and thus should be executed.

This means, that an arraylist, plus autoboxing/unboxing needs to be done for
all the triggered watches. This data structure can easily be replaced with a BitSet,
resulting in much less objects being created or parsed - also it's a bit faster.

Original commit: elastic/x-pack-elasticsearch@e9fba67e34
2016-12-12 09:48:58 +01:00
Simon Willnauer 5716a20fd3 Fix compile issues after elastic/elasticsearchelastic/elasticsearch#22037
Original commit: elastic/x-pack-elasticsearch@c286839ba5
2016-12-10 11:05:58 +01:00
Nik Everett f6cb60204a Handle core moving some methods around
We're trying to move all the XContentParser creation into
`RestRequest`.

Original commit: elastic/x-pack-elasticsearch@87c05c4a9a
2016-12-09 20:01:53 -05:00
Jay Modi ac34c3c37f Remove deprecated methods from the Realm class
This change removes the deprecated methods from the realm class. These methods include
blocking authentication and lookup and the lookup supported method.

Original commit: elastic/x-pack-elasticsearch@cff21e21ee
2016-12-09 07:28:11 -05:00
Daniel Mitterdorfer 8153d2ca69 Enable strict duplicate checks for JSON content
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default. This ensures that JSON keys are always unique. While this has
a performance impact, benchmarking has indicated that the typical drop in
indexing throughput is around 1 - 2%.

As a last resort, we allow users to still disable strict duplicate checks
by setting `-Des.json.strict_duplicate_detection=false` which is
intentionally undocumented.

Relates elastic/elasticsearchelastic/elasticsearch#19614

Original commit: elastic/x-pack-elasticsearch@cced57b884
2016-12-09 12:48:28 +01:00
Alexander Reelsen 37b0d52882 Watcher: Remove mock web server from square (elastic/elasticsearch#4221)
The latest release of the mock web server requires more security permissions, and
we dont need all the functionality anyway.

This introduces a small MockWebServer using the JDK internal HttpServer, yet fullfilling
all our needs and supporting SSL as well for testing.

The MockWebServer allows to enqueue responses and also requires you to enqueue as many responses
as you requests will be executed - there is no fallback at the moment.

SSL is also supported by passing an SSL context - for which the TestsSSLService is needed, which
makes the required methods public.

Original commit: elastic/x-pack-elasticsearch@55f4a172a2
2016-12-09 09:07:09 +01:00
Ryan Ernst b1846190af Remove uses of 2.0 prerelease constants
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#22004

Original commit: elastic/x-pack-elasticsearch@55433d760c
2016-12-08 23:11:20 -08:00
Jason Tedor e6d758b92b Bump version to 5.1.2
This commit bumps the version to 5.1.2.

Relates elastic/elasticsearch#4318

Original commit: elastic/x-pack-elasticsearch@a922dc7288
2016-12-08 18:43:16 -05:00
Tanguy Leroux 1f9871c31e [TEST] Mute OldMonitoringIndicesBackwardsCompatibilityTests
This tests fails because of multiple issues in monitoring service and exporters lifecycle. These issues are tracked in https://github.com/elastic/x-plugins/issues/4314.

Original commit: elastic/x-pack-elasticsearch@f461d98a4c
2016-12-08 10:48:39 +01:00
Areek Zillur b274ddc0da Log license type on license update (elastic/elasticsearch#4308)
Now when a cluster gets updated with a new license,
the license uid and type will be logged as info. Making
it easier to verify the current license mode of the cluster.

Original commit: elastic/x-pack-elasticsearch@c205cb42ae
2016-12-07 18:01:58 -05:00
Yannick Welsch 754fada43b Use autoMinMasterNodes for most of the license tests
There were some test failures as LicenseServiceClusterTests was not properly setting minimum_master_nodes while having autoMinMasterNodes disabled.

Original commit: elastic/x-pack-elasticsearch@2030db2424
2016-12-07 17:59:25 +01:00
jaymode 714bf929af test: LocalExporterTests should wait for exporters to be started
Original commit: elastic/x-pack-elasticsearch@978933f9d6
2016-12-07 09:22:05 -05:00
Jim Ferenczi fba54d02d7 fix compilation error due to a missing parameter in SortedSetDVOrdinalsIndexFieldData.
Original commit: elastic/x-pack-elasticsearch@f9803968d9
2016-12-07 11:33:29 +01:00
Boaz Leskes affdf10274 Remove `InternalTestCluster.startNode(s)Async` (elastic/elasticsearch#4198)
A companion PR to https://github.com/elastic/elasticsearch/pull/21846 where the above methods were removed. See ES PR for details.

With the concurrent starting the issues with licenses and time freeze became more apparent and I had to apply my suggestion to only freeze time once the license has been applied (as opposed to freeze on node start up). Since this also means that a node that starts up after the cluster time has been frozen need to also immediately freeze, it felt natural to use a `ServiceDisruptionScheme`. Although the name doesn't really make sense here, it all has all the logic we need. 

Original commit: elastic/x-pack-elasticsearch@5641742f60
2016-12-06 12:07:28 +01:00
Ryan Ernst 13e427d83f Remove action filter response override (elastic/elasticsearch#4268)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21950

Original commit: elastic/x-pack-elasticsearch@0034a3384c
2016-12-05 16:14:10 -08:00
Jay Modi ffeaea0663 Migrate authentication calls to be asynchronous
This commit migrates the actual authentication calls that are performed by internal realms
to be asynchronous so that we do not execute blocking calls on network threads. The realms
that used LDAP were especially bad as they can issue many different requests and/or open
multiple connections per authentication attempt.

The LdapRealm now uses the ThreadPool to fork a thread for authentication. This is necessary
because a LDAP bind operation is not allowed to be performed asynchronously and must be done
in a blocking manner. After the bind request has completed, all other requests can be done
asynchrnously. The LdapUtils class now provides asynchronous methods for searching and also
includes the handling of referral following as automated following is not supported by
UnboundID when using the asynchronous calls.

Original commit: elastic/x-pack-elasticsearch@dfb259e498
2016-12-05 10:07:31 -05:00
Tanguy Leroux 549629b1aa [Watcher] HistoryStore should hide secrets (elastic/elasticsearch#4256)
* [Watcher] HistoryStore should hide secrets

Some watcher actions like Jira stores the whole HTTP request in case of a failure. This is very helpful when something goes wrong, but it has to hide the password used for Basic authentication otherwise it will appear in the watch record in the .watcher-history index. In general, I would consider better to always hide secrets when storing something in the history index.

 This commit removes the XContentParams "hide secrets" set at the JiraIssue so that it is now set at the HistoryStore level.

 It also fixes test failures that can be reproduced with:
 gradle :x-plugins:elasticsearch:test -Dtests.seed=BA730F93E726AECC -Dtests.class=org.elasticsearch.xpack.notification.jira.JiraIssueTests -Dtests.method="testToXContent" -Dtests.security.manager=true -Dtests.locale=es -Dtests.timezone=Asia/Choibalsan

Original commit: elastic/x-pack-elasticsearch@d686b3b3ad
2016-12-05 15:30:48 +01:00
Jay Modi 9afb6dd4f2 monitoring: local exporter waits for in flight requests before retrying
The local exporter previously fired off asynchronous requests every time a cluster state was
observed that did not contain all of the required items for monitoring. This change adds a
flag so that monitoring can wait for the pending requests to complete before retrying. This
will reduce the number of duplicated log messages as well.

Additionally, the log message for adding modern aliases now contains the name of the indices.

Closes elastic/elasticsearch#3756

Original commit: elastic/x-pack-elasticsearch@727a0adfbe
2016-12-05 09:01:24 -05:00
Ryan Ernst 97336ea946 Fix rest filter tests to expect passthrough when security is disabled
Original commit: elastic/x-pack-elasticsearch@ea6adabb4d
2016-12-02 16:28:18 -08:00
Ryan Ernst dbbf470734 Fix line length to appease checkstyle
Original commit: elastic/x-pack-elasticsearch@35df54d342
2016-12-02 15:06:33 -08:00
Ryan Ernst 923926ef28 Convert security rest filter to rest handler wrapper (elastic/elasticsearch#4234)
* Convert security rest filter to rest handler wrapper

This is the xpack side of elastic/elasticsearchelastic/elasticsearch#21905

Original commit: elastic/x-pack-elasticsearch@38bfa771b6
2016-12-02 14:55:10 -08:00
Tanguy Leroux f1a4a2fb73 [Monitoring] Remove unused code in Collectors (elastic/elasticsearch#4240)
This commit removes some unnecessary code: collectors do not need to extend AbstractLifecycleComponent and do not need to be started/stopped. The extra
interface Collector is removed and AbstractCollector renamed to Collector.

Original commit: elastic/x-pack-elasticsearch@75893d66e3
2016-12-02 13:29:09 +01:00
Simon Willnauer 7f17896927 Revert "Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)"
This reverts commit elastic/x-pack@c6c0ffd5d9.

Original commit: elastic/x-pack-elasticsearch@8b7386fb99
2016-12-02 10:55:46 +01:00
Alexander Reelsen 7c04897392 Watcher: Compile scripts on each invocation (elastic/elasticsearch#4239)
Transform and condition scripts were only compiled on its initial creation, so
when a new watch is created or when the master node loads all the watches. However
changing a script (like a stored one) did not lead to any changes in the in memory
watch store and thus the old script was executed again.

We do however have a mechanism in Elasticsearch's ScriptService that already does some
caching, and should reuse that one.

Closes elastic/elasticsearch#4237

Original commit: elastic/x-pack-elasticsearch@477548e237
2016-12-02 10:36:05 +01:00
Alexander Reelsen 946d943868 Watcher: Removing unused upgradeSource boolean in watch parsing (elastic/elasticsearch#4202)
This seems to be a leftover from elastic/elasticsearch#4162

However the boolean parameters is completely unused already and
can be removed.

Original commit: elastic/x-pack-elasticsearch@3371b089d6
2016-12-02 10:34:11 +01:00
Simon Willnauer ace1a7e6af Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)
Original commit: elastic/x-pack-elasticsearch@07cecdbf00
2016-12-02 08:06:46 +01:00
Jay Modi 139073e8f7 security: improve migrate tool output and remove trappy config option
This commit improves the output of the migrate tool in cases when there are errors parsing entries
in the roles or users files. This is done through the use of a logger that delegates its output to
the terminal.

Additionally, the `-c` option has been removed. This option was used to set the configuration directory
but this should be handled one way only and that is through the use of the `-Epath.conf` setting.

Closes elastic/elasticsearch#3757
Closes elastic/elasticsearch#3758

Original commit: elastic/x-pack-elasticsearch@811e367766
2016-12-01 10:17:28 -05:00
Boaz Leskes d881e4d9ad properly deal with overriding parent's tearDown method
Original commit: elastic/x-pack-elasticsearch@0edd4b1e20
2016-12-01 13:18:36 +01:00
Boaz Leskes 9e3ae67423 HttpClientTests don't override parent tearDown
Original commit: elastic/x-pack-elasticsearch@3cf9f6f352
2016-12-01 13:15:25 +01:00
Boaz Leskes a4cec2316b Add before and after logging for unit tests
Currently we have these logs for integration tests only.

This adds the following log at the start:
```
logger.info("[{}]: before test", getTestName());
```

and this is logged at the end, but before any clean up done in sub classes

```
 logger.info("[{}]: after test", getTestName());
```

Original commit: elastic/x-pack-elasticsearch@2ca7296665
2016-12-01 12:56:38 +01:00
Martijn van Groningen 0701f7bb18 test: verify whether templates exist in an assertBusy(...)
It may take a few clicks before the templates are added.
Waiting for started doesn't guarantee that the index templates have been added.

Original commit: elastic/x-pack-elasticsearch@ee94b740a8
2016-12-01 11:04:11 +01:00
javanna 792a821d1a Adapt to indices query removal
Original commit: elastic/x-pack-elasticsearch@8f2d4c23c5
2016-11-30 18:23:19 +01:00
Jay Modi 217bd8add0 Add tests for DNS only hostname verification with SSL
This commit adds tests for DNS only hostname verification. This is a followup of elastic/elasticsearchelastic/elasticsearch#21828, which fixes issues with this type of hostname verification
due to some addresses losing the host information.

Original commit: elastic/x-pack-elasticsearch@8a63bb113d
2016-11-30 12:20:11 -05:00
Jason Tedor 16e3bb4587 Add version 5.1.1
This commit adapts x-plugins for the removal of the version constant for
5.1.0 and the addition of the version constant for 5.1.1 in core.

Relates elastic/elasticsearch#4223

Original commit: elastic/x-pack-elasticsearch@2fa92f0056
2016-11-30 11:14:47 -05:00
javanna 69218af73f Remove subrequests method from CompositeIndicesRequest
Original commit: elastic/x-pack-elasticsearch@c644204598
2016-11-30 15:03:42 +01:00
Adrien Grand 981648774f Remove usage of `indices.ttl.interval`. (elastic/elasticsearch#4207)
Remove usage of `indices.ttl.interval`.

Original commit: elastic/x-pack-elasticsearch@54c987daec
2016-11-30 10:11:10 +01:00
Adrien Grand 84be89861a Remove 2.x bwc indices. (elastic/elasticsearch#4206)
Version 6.0 will only support reading 5.0+ indices.

Original commit: elastic/x-pack-elasticsearch@167f5f209d
2016-11-30 10:10:30 +01:00
Jason Tedor 3da81aa922 Add version 5.0.3
This commit adds version 5.0.3 and the BWC indices for version 5.0.2.

Relates elastic/elasticsearch#4211

Original commit: elastic/x-pack-elasticsearch@a0c83a0b92
2016-11-29 18:48:45 -05:00
Luca Cavanna 34d6dc1db1 Categorize search template action as a composite indices request (elastic/elasticsearch#4209)
When we encounter a composite request, we authorize at first without looking at the indices, to see whether the action can be executed at all. We then rely on the action to delegate to an inner action per sub-request, which will be authorized based on the indices it refers to. The first step works great for the simulate mode of search template, as it doesn't involve any index. The second step will make sure that when search template involves a search, it will be authorized as a normal search request would, based on the indices it reads from.

Note that the wildcard expansion happens now on the search side, it doesn't have to happen when executing the first authorization step, hence SearchTemplateRequest doesn't have to implement IndicesRequest, only SearchRequest has to (which it does already).

Closes elastic/elasticsearch#4171

Original commit: elastic/x-pack-elasticsearch@d586bd90cb
2016-11-29 20:53:01 +01:00
Tanguy Leroux a414e3a7d9 [TEST] Waits for security template to be created in XPackRestIT
This commit applies the same fix merged in  elastic/elasticsearch#4179 for XDocsClientYamlTestSuiteIT. It adds a waitForSecurityTemplate() method in order to wait for the security-index-template to be created by the SecurityTemplateService.

Original commit: elastic/x-pack-elasticsearch@1476f30e2d
2016-11-29 14:05:58 +01:00
Tanguy Leroux 0673d6b3d6 [Test] Add back ThreadLeakLingering in OldMonitoringIndicesBackwardsCompatibilityTests
Also changes a bit how collection is stopped.

Original commit: elastic/x-pack-elasticsearch@e28f8bc11d
2016-11-28 16:10:28 +01:00
Jay Modi 637154cc6e Iterate over realms asynchonously
This commit moves the iteration of realms for authentication and user lookup to
be done in an asynchronous fashion. The existing blocking methods have been deprecated
to allow custom realm implementors time to switch. All internal realms implement the
asynchronous methods.

This PR is another step toward the full migration to async authentication, but does not
complete the work. Additional work is needed for the LDAP realms, which make blocking
network calls. These blocking calls will be handled in a follow-up PR.

See elastic/elasticsearch#3790

Original commit: elastic/x-pack-elasticsearch@a65a9b2bb4
2016-11-28 09:28:51 -05:00
Alexander Reelsen f265ab7cae Watcher: Throw exception if HttpClient response is not a HTTP response (elastic/elasticsearch#4154)
If the HTTP response is an invalid one, it is still logged as success.
This commit changes the behaviour, that if the response status code is
set to -1 (which means it could not be interpreted), than an IOException
is thrown and thus the execution will be marked as a failure.

Closes elastic/elasticsearch#4152

Original commit: elastic/x-pack-elasticsearch@5736fbe3c0
2016-11-28 11:29:28 +01:00
Jason Tedor c4e890cba0 Adapt to unreleased versions change
Core has better support unreleased versions now, making maintenance of
these simpler. This commit adapts x-plugins to this change.

Relates elastic/elasticsearch#4168

Original commit: elastic/x-pack-elasticsearch@a5d8a2f7df
2016-11-23 15:49:53 -05:00
Alexander Reelsen d53dbe5283 Watcher: Clean up email server (elastic/elasticsearch#4163)
* Do not try bind to port range but use free ephemeral port
* Start a new email server in all tests, do not use static one
* Remove selection of username/password, as it was static anyway
* Remove Listener.Handle class, as it is not needed, when not running in static context

Original commit: elastic/x-pack-elasticsearch@8816cc25f6
2016-11-23 18:19:02 +01:00
Tanguy Leroux ddddee1e1f [Watcher] Increment watcher history template version (elastic/elasticsearch#4166)
This commit increments the version number of the watcher history template.

Original commit: elastic/x-pack-elasticsearch@1c86e781ca
2016-11-23 16:27:33 +01:00
Tanguy Leroux 52eb621309 [TEST] Fix OldMonitoringIndicesBackwardsCompatibilityTests
OldMonitoringIndicesBackwardsCompatibilityTests fails because it waits for more shards stats to be collected but that can only work if new indices are created in the meanwhile.

Original commit: elastic/x-pack-elasticsearch@003c28cf93
2016-11-23 15:16:22 +01:00
Tanguy Leroux a32f2096a6 Add mappings for Jira action (elastic/elasticsearch#4155)
This commit updates the watch_history.json file so that it includes mappings for the new Jira action. It also update the JiraIssue format so that it now includes the name of the account used to create the Jira issue. It also update the REST tests to check that Jira action result are searchable and hide the user's password.

Original commit: elastic/x-pack-elasticsearch@75888f7748
2016-11-23 11:53:06 +01:00
Martijn van Groningen 5d042fc1b8 updated testScriptConditionParserBadScript() test to face reality
Original commit: elastic/x-pack-elasticsearch@222537dbe2
2016-11-23 09:15:45 +01:00
Ryan Ernst 3f02111a92 Remove upgrade source test, no longer needed with groovy gone
Original commit: elastic/x-pack-elasticsearch@c4a6c87227
2016-11-22 23:11:28 -08:00
Ryan Ernst 1dc839bd98 Remove groovy scripting language (elastic/elasticsearch#4162)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21607

Original commit: elastic/x-pack-elasticsearch@125843e814
2016-11-22 22:45:15 -08:00
Jay Modi 7873bb73c4 add tests for concurrent user lookup
This commit adds tests to ensure that user lookup in caching realms works as expected. An unclear
contract in the Cache#computeIfAbsent method allowed for null values to be returned from this method
even if there should have been exception reported to the loader. This has been fixed in the cache
implementation and we add tests to verify that the caching of user lookups is done properly under
concurrent operations.

Closes elastic/elasticsearch#4054

Original commit: elastic/x-pack-elasticsearch@41567c6ed9
2016-11-22 15:16:49 -05:00
Simon Willnauer 7f28ca7fb7 Adopt changes to TransportClient constructor (elastic/elasticsearch#4141)
Follow-up for elastic/elasticsearchelastic/elasticsearch#21709

Original commit: elastic/x-pack-elasticsearch@40d3725ac2
2016-11-22 20:46:43 +01:00
jaymode 7821203422 test: add bwc index for 2.4.2
Original commit: elastic/x-pack-elasticsearch@f9fbd83b9c
2016-11-22 14:37:36 -05:00
Jay Modi 4239ba5415 allow reads of native users and roles when template version hasn't been updated
This change allows reads of our native users and roles when the template version has not been updated to
match the current version. This is useful for rolling upgrades where the nodes are also being actively
queried and/or indexed into. Without this, we can wreak havoc on a cluster by causing exceptions during
replication, which leads to shard failures. On nodes that match the version defined in the template,
write operations are allowed since we know that we are backwards compatible in terms of format but we
may have added new fields and shouldn't index them until the mappings and template have been updated.

As part of this, the rolling upgrade tests from core were used as the basis for a very basic set of tests
for doing a rolling upgrade with x-pack.

Closes elastic/elasticsearch#4126

Original commit: elastic/x-pack-elasticsearch@9be518ef00
2016-11-22 12:00:09 -05:00
Areek Zillur a9f3619b5a Enable merging license in tribe node (elastic/elasticsearch#4147)
Currently, a tribe node ignored underlying cluster licenses
due to inablity to select an appropriate license from  multiple
licenses. Now that tribe node supports merging custom metadata
(elasticsearch#elastic/elasticsearch#21552), we can enable license support in tribe
node.

Now, tribe node chooses license with the highest operation
mode from underlying cluster licenses. This commit also
adds integration tests for licensing to verify that:
 - autogenerated trial license propagates to tribe node
 - tribe node chooses the highest operation mode license
 - removing a license from underlying cluster license is
   removed from tribe

closes elastic/elasticsearch#3212

Original commit: elastic/x-pack-elasticsearch@b5c003decd
2016-11-22 11:42:51 -05:00
Tanguy Leroux c9b3de6f23 Make HttpRequest toXContent / parse method coherent (elastic/elasticsearch#4153)
This commit changes the toXContent() method so that it generates content that can be parsed again by the parse() method. Before this commit, the authentication of the HTTP request is rendered as:

{ "auth": {"username": "foo", "password": "bar" } }

but the parsing method expects the authentication type to be a root object:

{ "auth":  { "basic" :  {"username": "foo", "password": "bar" } } }

The toXContent method has been adapted to include the type of authentication in the generated content.

Original commit: elastic/x-pack-elasticsearch@b740466109
2016-11-22 11:24:06 +01:00
Tanguy Leroux efe6524253 Mute JiraIssueTests.toToXContent()
Until "https://github.com/elastic/x-plugins/pull/4153" is solved

Original commit: elastic/x-pack-elasticsearch@c7720faddf
2016-11-22 10:07:27 +01:00
Alexander Reelsen a709d78e5a Tests: Ensure mock webserver binds always on arbitrary port (elastic/elasticsearch#4139)
This removes the mock webserver trying to bind to a port range, just
try to bind to any port.

Also removed unneeded wrapper class in the process.

Original commit: elastic/x-pack-elasticsearch@19b6ac393a
2016-11-22 09:24:32 +01:00
Tanguy Leroux 4978d3a8e0 [TEST] Fix JiraIssueTests.testEquals()
This commit fixes a test bug were a random field is picked out of an empty collection.

Original commit: elastic/x-pack-elasticsearch@a42dff7257
2016-11-22 09:09:44 +01:00
Jason Tedor 158ab2e724 Die with dignity on the network layer
When a fatal error is thrown on the network layer, such an error never
makes its way to the uncaught exception handler. This prevents the node
from being torn down if an out of memory error or other fatal error is
thrown while handling HTTP or transport traffic. This commit adds logic
to ensure that such errors bubble their way up to the uncaught exception
handler, even though Netty tries really hard to swallow everything.

Original commit: elastic/x-pack-elasticsearch@f76757674f
2016-11-21 22:14:45 -05:00
Tanguy Leroux 7c1f4326fc [TEST] Fix OldMonitoringIndicesBackwardsCompatibilityTests
This commit aims to fix the regularly failing test OldMonitoringIndicesBackwardsCompatibilityTests. It enables/disables monitoring data collection using time interval "-1" as well as stopping AgentService instances directly and checks multiple times if indices are still present at the end of the test.

closes elastic/elasticsearch#3999

Original commit: elastic/x-pack-elasticsearch@8ac785061a
2016-11-21 16:15:42 +01:00
Tanguy Leroux 18478d63c2 Watcher: Add JIRA action (elastic/elasticsearch#4014)
closes elastic/elasticsearch#493

Original commit: elastic/x-pack-elasticsearch@6b7387d3e4
2016-11-21 10:52:55 +01:00
Tim Vernum 74b0a1e71a Record audit trail even if indicies-resolver throws exception (elastic/elasticsearch#4116)
* Record audit trail even if indicies-resolver throws exception

If the IndicesAndAliasesResolver throws an exception, treat is as "accessDenied" for the purpose of the audit-trail.
This can occur when an index request has a wildcard that doesn't match (and "allowNoIndices" is false)

Closes elastic/elasticsearch#3719

Original commit: elastic/x-pack-elasticsearch@ca6567e5ed
2016-11-21 09:18:49 +11:00
Tim Vernum ae6eb0d159 Enforce username validation on PUT security/user (elastic/elasticsearch#4118)
Validation was being applied within UsersTool but not via the REST API
This also causes some validation messages in PutUser to change.

Closes elastic/elasticsearch#4018

Original commit: elastic/x-pack-elasticsearch@8e02146dee
2016-11-18 12:33:45 +11:00
Simon Willnauer cc22888417 Stabelize RemoteIndexAuditTrailStartingTests and improve IndexAuditTrail (elastic/elasticsearch#4112)
RemoteIndexAuditTrailStartingTests has been failing quite often due to several
race conditions. This change simplifies the test (one node per cluster, local and
remote) and fixes some issues in IndexAuditTrail. The interrupt based shutdown has
been removed from the `QueueConsumer` and an additional health call has been added
to trigger another start attempt if the first one failed. This stabilize the test
that now run in 1 second rather than 1 minute.

Original commit: elastic/x-pack-elasticsearch@a1c8131cd2
2016-11-17 17:57:15 +01:00
Jason Tedor 7309f11f51 Respond to exposing of executor service
This commit responds to a chance in core which modified the interface
for ThreadPool#executor and ThreadPool#generic to return an executor
service rather than an executor.

Relates elastic/elasticsearch#4107

Original commit: elastic/x-pack-elasticsearch@7c7c6a3b90
2016-11-17 09:19:15 -05:00
Simon Willnauer 92040ef72e Remove netty_3 support from xpack (elastic/elasticsearch#4097)
This is a followup from elastic/elasticsearchelastic/elasticsearch#21590 and needs to be
committed first or at the same time since netty_3 is removed

Original commit: elastic/x-pack-elasticsearch@131d74dd6b
2016-11-17 12:44:24 +01:00
Adrien Grand a5e410ba59 Remove expectations about store throttling stats now that it has been removed.
Original commit: elastic/x-pack-elasticsearch@244afb234a
2016-11-17 10:13:29 +01:00
Adrien Grand 67cc424303 Remove call to the store throttling stats.
Store throttling has been removed in elastic/elasticsearchelastic/elasticsearch#21573.

Original commit: elastic/x-pack-elasticsearch@40bcb51101
2016-11-17 09:48:26 +01:00
Nik Everett 042072410c Rename some tests from IT to Tests (elastic/elasticsearch#4106)
If these tests were in `:core` then then `IT` would be an appropriate
suffix, but in xplugins they should have `Tests`. This moves them
from the `integTest` task to the `test` task which is a good idea
because it lets them run in parallel with the other tests. Naming
them `IT` means that they are run not in parallel and they are run
with a running instance of Elasticsearch which they don't need.

This cuts the build from 15.5 minutes to 13.5 minutes for me.

Original commit: elastic/x-pack-elasticsearch@7b4bcf8bc5
2016-11-16 17:15:34 -05:00
jaymode 782fb6ad41 test: fix testDefaultMetaFields after adding _seq_no
Original commit: elastic/x-pack-elasticsearch@97faea1dd1
2016-11-16 12:32:28 -05:00
jaymode 766b89b2fd test: move indices:admin/seq_no/global_checkpoint_sync to handlers
Original commit: elastic/x-pack-elasticsearch@89ae75de6e
2016-11-16 12:24:18 -05:00
jaymode 586a8d4a00 add _seq_no field to allowed metafields
Original commit: elastic/x-pack-elasticsearch@875cbea3e0
2016-11-16 12:21:15 -05:00
Jason Tedor 52eec4736d Add global checkpoint to known actions/handlers
This commit adds the global checkpoint action to the known
action/handlers.

Original commit: elastic/x-pack-elasticsearch@db083911d2
2016-11-16 11:58:51 -05:00
Jason Tedor 5426a86ba0 Fix compilation issue introduced by seq. no merge
This commit fixes a compilation issue introduced by the merge of
sequence numbers branch into master. This merge introduced a new
constructor parameter for ShardStats.

Original commit: elastic/x-pack-elasticsearch@b234b8ee50
2016-11-16 11:49:57 -05:00
Nik Everett 260a51fabe Disabled testDynamicIndexAction
It is failing consistently.

Original commit: elastic/x-pack-elasticsearch@21c96694bd
2016-11-15 21:24:29 -05:00
Nik Everett fedca4ae31 Add bwc index for 5.0.1 release
Original commit: elastic/x-pack-elasticsearch@92bc318313
2016-11-15 19:12:29 -05:00
Jay Modi 31c851f5c2 migrate authentication service to an async model
This commit migrates the authentication service to an asynchronous model where we use listeners
instead of blocking and waiting for the authentication to return. This is the first part of making
authentication asynchronous as we still have blocking I/O inside of realms.

See elastic/elasticsearch#3790

Original commit: elastic/x-pack-elasticsearch@9339af4af8
2016-11-15 13:01:49 -05:00
jaymode 68c026d273 retain all user information for a run as request
In the authentication service, we currently only copy the username and roles of the
user that was authenticated but we should instead preserve all of their information
in the newly created user object. This change does that through the user of a new
constructor in the user class that takes in both users.

Closes elastic/elasticsearch#3877

Original commit: elastic/x-pack-elasticsearch@7455078841
2016-11-15 10:57:46 -05:00
jaymode ddb032b71c test: disable auto minimum master nodes for NoMasterNodeTests
Original commit: elastic/x-pack-elasticsearch@59e0b3e97e
2016-11-15 10:33:59 -05:00
jaymode 27cb25581b unknown run as users should not be allowed to execute any APIs
If a authenticated user with run as permission attempts to run as an unknown user, the unknown
user will be assigned the default role and anonymous role if enabled. This change prevents this
from happening as we require the run as user to have been looked up by a realm.

Closes elastic/elasticsearch#3878

Original commit: elastic/x-pack-elasticsearch@034f44757d
2016-11-15 09:47:45 -05:00
Boaz Leskes 667bb340b4 Update tests for new auto management of min master nodes (elastic/elasticsearch#4068)
The internal test cluster now auto manages min_master_nodes  ( see https://github.com/elastic/elasticsearch/pull/21458 ). This requires some code changes but also changes the timings of forming a cluster. This has had a funny side effect where the master is no longer always the first node to be started in the cluster. This caused issues with watcher tests which freeze time.

Original commit: elastic/x-pack-elasticsearch@1e5ea8ae94
2016-11-15 13:42:59 +00:00
Ryan Ernst 207feb2fa2 Fix test lookup of Discovery to cast instead of lookup in injector directly
Original commit: elastic/x-pack-elasticsearch@2718dab928
2016-11-14 23:27:02 -08:00
Ryan Ernst 190c036932 Fix test to use TestZenDiscovery to get access to pings
Original commit: elastic/x-pack-elasticsearch@83ea58c2dd
2016-11-14 22:09:19 -08:00
Ryan Ernst 96ba09436e Tests: Use TestZenDiscovery instead of MockZenPing (elastic/elasticsearch#4052)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21488

Original commit: elastic/x-pack-elasticsearch@d3db54498a
2016-11-14 22:00:53 -08:00
Ryan Ernst f2c4d1f4da Remove generics from ActionRequest
x-plugins side of elastic/elasticsearchelastic/elasticsearch#21556

Original commit: elastic/x-pack-elasticsearch@a8e8873c84
2016-11-14 15:36:46 -08:00
Ryan Ernst 23e6cab7f1 Merge pull request elastic/elasticsearch#4044 from rjernst/realm_sig
Extensions: Make resource watcher available to custom realms

Original commit: elastic/x-pack-elasticsearch@3cb494e98d
2016-11-14 12:38:27 -08:00
jaymode a893f0d646 test: mock clusterservice to prevent NPE when remote indexing
This change mocks the ClusterService and sets it up so that when remote indexing, we see the
current node as the master.

Original commit: elastic/x-pack-elasticsearch@1f26b3df1f
2016-11-14 14:16:11 -05:00
jaymode 2ab7e52c79 improve comment
Original commit: elastic/x-pack-elasticsearch@a3662a81a2
2016-11-14 12:51:21 -05:00
jaymode 8285fbdd46 address feedback
Original commit: elastic/x-pack-elasticsearch@7f7f2c0e1f
2016-11-14 10:33:47 -05:00
jaymode 8e0f5f905e Merge branch 'master' into audit_index_csupdate
Original commit: elastic/x-pack-elasticsearch@5ca98373c7
2016-11-14 09:43:57 -05:00
Nik Everett 6acde61347 Reenable testDynamicIndexSearchInput
It works this morning....

Original commit: elastic/x-pack-elasticsearch@5d9bb37df8
2016-11-11 10:19:06 -05:00
Nik Everett ca3f13de69 Fix and disable some more tests
Template test is easy to fix. I have no idea what is up with the
dynamic index name test. It is finding results on the wrong time.

Original commit: elastic/x-pack-elasticsearch@535676164c
2016-11-10 22:20:08 -05:00
Nik Everett f15b8c94e4 Fix TemplateUtilsTests
The template has changed on disk so we have to change the assertion.

Original commit: elastic/x-pack-elasticsearch@cc9b76fcf6
2016-11-10 21:51:06 -05:00
Nik Everett 700467c3a7 Use index_patterns in templates
And skip a REST test that won't pass.

Original commit: elastic/x-pack-elasticsearch@e297add6c1
2016-11-10 21:42:59 -05:00
Jason Tedor 7da624739d Fix handler name on message not fully read
Today when a message is not fully read on a response, we log (among
other details) the handler name. Unfortunately, if the handler is a
wrapper, all that we see is

o.e.x.s.t.TransportService$ContextRestoreResponseHandler@7446ba18

completely losing the offending handler. This commit adds an override
for TransportService$ContextRestoreResponseHandler#toString so that the
underlying offender can be discovered.

Relates elastic/elasticsearch#4043

Original commit: elastic/x-pack-elasticsearch@3eb1003d5b
2016-11-10 16:13:23 -05:00