Commit Graph

1324 Commits

Author SHA1 Message Date
Jay Modi aa0e4d425f security: system user needs put mapping permissions to shrink indices
The system user gets used to put mappings for an index during recovery from local shards, which
is how the shrink index process works. The system user previously had this privilege in 2.x as
we did not have the ThreadContext and dynamic mapping updates would be done by the system user;
with the ThreadContext, these mapping updates are done by the actual user so this privilege
was removed from the SystemUser.

Closes elastic/elasticsearch#3766

Original commit: elastic/x-pack-elasticsearch@cd5d7bea53
2016-10-17 11:00:04 -04:00
javanna 0504f02026 inclusions and exclusions shouldn't be considered wildcard expressions
The security indices resolver checks through an assertion that shard level requests always have their wildcard expressions resolved. Index names that start with `-` or `+` though shouldn't be considered wild card expressions. Up to 6.x there can be indices with names starting with `-` or `+` and we have to take that into account.

Also moved from assertion to explicit exception so we can also test it better.

Original commit: elastic/x-pack-elasticsearch@a520bbf247
2016-10-17 16:45:47 +02:00
javanna 99d198c715 [TEST] remove leftover comment in createIndicesWithRandomAliases
Original commit: elastic/x-pack-elasticsearch@80546bae7f
2016-10-17 15:00:25 +02:00
javanna 667be843ce [TEST] modify aliases names in createIndicesWithRandomAliases
If we create index test1 and alias test1-alias, and tests configure access for test* for some users, this is going to cause problems when verifying exclusions like -test2, as the index itself gets excluded but the alias that points to it doesn't. That is expected behaviour, with this commit we modify the way aliases are named to use a prefix rather than a suffix (e.g. from test1-alias to alias-test1).

Changed also the way aliases creation is randomized.

Original commit: elastic/x-pack-elasticsearch@7f9877e858
2016-10-17 14:54:54 +02:00
javanna 3e5833e85c [TEST] random aliases were never created in createRandomIndicesWithAliases
missing `.get()` :)  the create index request was never sent. The indices were being automatically created when indexing a document into them.

Original commit: elastic/x-pack-elasticsearch@129d69c88e
2016-10-17 12:22:24 +02:00
Tanguy Leroux 21af0d5dc7 Fix OldMonitoringIndicesBackwardsCompatibilityIT (elastic/elasticsearch#3760)
The checkNodeStats method in this test checks for many fields in every documents of all bwc indices, but some fields like disk_threshold_enabled have been removed in 5.x. This commit changes the method so that it checks for the right fields in the right version.

closes elastic/elasticsearch#3672

Original commit: elastic/x-pack-elasticsearch@c95209cc3b
2016-10-14 18:30:05 +02:00
Jason Tedor 864cfb417a Remove artificial default processors limit
This commit responds to an API change in core migrating from
EsExecutors#boundedNumberOfProcessors to EsExecutors#numberOfProcessors.

Original commit: elastic/x-pack-elasticsearch@87d6fad971
2016-10-14 06:40:20 -04:00
javanna 71d2c25fcb Simplify AuthorizationService and extract loading of authorized indices to its own class
extracted loading of authorized indices and aliases to separate class (AuthorizedIndices) with reduced dependencies. Allows also to lazily load authorized indices the first time they are required, and reuse them if they are needed again later. Removes AuthzService dependency in indices resolver.

 Removed array of resolvers in authorization service as we support only one. Removed IndicesAndAliasesResolver interface and rename DefaultIndicesAndAliasesResolver to IndicesAndAliasesResolver.

Original commit: elastic/x-pack-elasticsearch@a267fefa07
2016-10-13 16:05:02 +02:00
javanna 06b5d42741 [TEST] consolidate different assertAuthorizationException methods in one place
Original commit: elastic/x-pack-elasticsearch@27de6db7e0
2016-10-13 16:05:02 +02:00
javanna 3dbea2f4c2 Simplify FieldAndDocumentLevelSecurityRequestInterceptor
FieldAndDocumentLevelSecurityRequestInterceptor really support intercepting only subclasses of IndicesRequests, we shouldn't have logic that is never used around intercepting CompositeIndicesRequest. Also we can guarantee at compile time, using generics, that only supported subclasses are intercepted through it, no need to verify that at runtime.

Original commit: elastic/x-pack-elasticsearch@6ab6e2d50e
2016-10-13 16:05:02 +02:00
javanna 4bb6e856f3 Authorize composite actions based on their action name only, subrequests and their indices will be later authorized individually
Eagerly authorizing CompositeIndicesRequests allowed the security plugin to fail fast up until now, but it makes it very hard to reason about each specific item in a multi items request. Either all items fail, or none do. We would rather want to adopt a similar behaviour to es core, where individual items fail without affecting other items that are part of the same request. We can rely on the fact that es core always authorizes both main action and every subaction too, and skip authorization for the main action. By subaction we mean either all sub search requests in msearch, as well as each shard level get in mget or shard level bulk request for bulk.

 BulkRequestInterceptor was converted to intercept BulkShardRequests rather than BulkRequest as that is where bulk is authorized after this change.

 Split IndicesAndAliasesResolverIntegrationTests into ReadActionsTests and WriteActionsTests as they require different set of permissions, lots of tests added.

Explicitly listing the composite actions makes sure that the actions that can bypass security are known, somebody adding a similar action must to add it to the list, so we know it doesn't happen by mistake. At this point the CompositeIndicesRequest can be used as a marker interface only (it is not really needed but can be used to verify that composite actions use a request that implements such interface).

Given that we don't authorize composite actions based on their indices anymore, but only their sub-requests which implement IndicesRequest, printing out the indices names in the audit log for requests like bulk and msearch is confusing. Removed support for that.

Authorize composite indices actions based on their name only, their indices will be authorized at the sub-request/shard level

Rather than simply granting bulk, mget, msearch etc. and relying on authorization at the sub-request/shard level, we check that the current user can at least execute the action. This justifies the grant line that gets written in the audit log, the action is potentially possible without looking at the indices. Each specific item will fail or succeed later and will yield its own specific audit log entry.

Original commit: elastic/x-pack-elasticsearch@4570caf019
2016-10-13 16:05:02 +02:00
javanna c6edec254a special case IndicesExistsRequest to make sure index not found is never thrown while resolving indices
Like es core does in TransportIndicesExistsAction, we should only consider expandWildcardsOpen and expandWildcardsClosed out of the indices options passed in with IndicesExistsRequest. ignore_unavailable and allow_no_indices should always be considered both true, to prevent the request from throwing exception as it is supposed to return true or false, no exceptions.

Original commit: elastic/x-pack-elasticsearch@daa274b3fd
2016-10-13 16:05:02 +02:00
javanna d27c4bee82 Support allowNoIndices option in security plugin
Supporting allowNoIndices means that the security plugin has a behaviour much more similar to vanilla es when dealing with wildcard expressions that match no indices, or empty clusters. The default for most request is to allow no indices, but security plugin could only disallow no indices all the time up until now.

The technical problem was that when anything gets resolved to an empty set of indices, we couldn't let that go through to es core, as that would become resolved to all indices by es core, which would be a security hole. We have now found a way though to replace an empty set of indices with something that es core will for sure resolve to no indices, so we can let the request through. We simply replace empty indices with '-*'.

Multi apis requests (e.g. _msearch) have yet to be fixed, as all their indices end up in the same bucket while they should each be authorized separately, so that every specific item can fail or be let through.

Original commit: elastic/x-pack-elasticsearch@0f67a0bfea
2016-10-13 16:05:02 +02:00
javanna 9b46b34bed Honour ignore_unavailable option when resolving indices
For all the requests that support multiple indices and wildcards, hence implementing IndicesRequest.Replaceable, we replace the wildcard expressions with the explicit names of the authorized indices they match. _all or empty indices is treated as a wildcard expression. We can also honour the ignore_unavailable option by going over all the explicit names and filter out the non authorized ones when ignore_unavailable is set to true. If ignore_unavailable is set to false, we leave everything as-is, which will cause an authorization exception to be thrown if only one of those explicit indices is not authorized for the current user.

This is the first step towards resolving elastic/elasticsearch#1250. The remaining issue is that in case we are left with no indices after stripping out the ones that the user is not authorized for, we throw an authorization exception rather than returning an empty response. That will require honouring the allow_no_indices option, which will also change the behaviour when a cluster is empty.

Relates to elastic/elasticsearch#1250

Original commit: elastic/x-pack-elasticsearch@e4ca940d05
2016-10-13 16:05:02 +02:00
Jay Modi 219c42d7ce update to use TimeValue in CacheBuilder
Original commit: elastic/x-pack-elasticsearch@7f5a59bd73
2016-10-13 09:27:51 -04:00
Areek Zillur 1a0802a157 Merge branch 'master' into cleanup/transport_bulk
Original commit: elastic/x-pack-elasticsearch@f0b88369f3
2016-10-12 13:12:16 -04:00
Jay Modi 68eb4d981e security: wildcards for superusers includes the security index
The superuser role is the only user assignable role that grants access to the .security index, but when
resolving wildcards the index was not getting resolved. The resolution of indices and aliases explicitly
excludes the .security index for users that are not the internal user without checking if the user has the
superuser role. This commit adds a check in for the superuser role.

Original commit: elastic/x-pack-elasticsearch@02ee0a8740
2016-10-12 11:42:02 -04:00
Jay Modi 6284db3a4d security: use correct time unit for role cache expire after write
The role cache was previously using the wrong time unit for its expire after write time; the
value passed to the cache was milliseconds instead of nanoseconds.

Original commit: elastic/x-pack-elasticsearch@65f7b08763
2016-10-12 08:04:49 -04:00
Tanguy Leroux 7ba55a4c99 Remove empty comments (elastic/elasticsearch#3731)
Original commit: elastic/x-pack-elasticsearch@a3e814bf34
2016-10-12 13:22:18 +02:00
Jay Modi 9a1d33d863 security: include anonymous roles when building the global permission
The anonymous role was being applied to other users for index access control but was not being applied
in terms of action level access control. This change makes the minimum required change to apply the
anonymous role for all users when anonymous is enabled. Additionally, some minor changes were made to the native roles store to not lookup roles before the service is started.

Closes elastic/elasticsearch#3711 

Original commit: elastic/x-pack-elasticsearch@a9398e178d
2016-10-12 06:52:24 -04:00
Yannick Welsch 4e00ab2f2b Remove test for transport handler that was removed in core (elastic/elasticsearch#3717)
Relates to elastic/elasticsearchelastic/elasticsearch#20836

Original commit: elastic/x-pack-elasticsearch@38f2d2e242
2016-10-12 09:08:02 +02:00
Alexander Reelsen 8b83cf067c Watcher: Ensure awesome painless exceptions are propagated to the user (elastic/elasticsearch#3707)
When adding a watch which has a painless component, the scriptexception
was wrapped into a deprecated exception which means, that the awesome
painless descriptions were lost. This wrapping has been removed.

Closes elastic/elasticsearch#3161

Original commit: elastic/x-pack-elasticsearch@1703fe4eb6
2016-10-12 08:14:06 +02:00
Areek Zillur 5d86c04441 Change bulk item requests from ActionRequest to DocumentWriteRequest
x-pack changes for elastisearchelastic/elasticsearch#20109

Original commit: elastic/x-pack-elasticsearch@8c12e1e102
2016-10-11 23:15:25 -04:00
Tanguy Leroux 2e7b7be25c Watcher: Re enable array compare test (elastic/elasticsearch#3708)
This test has been blacklisted and deactivated months ago. This commit reenables this test and moves it at the right place. It also change the test to use the Execute Watch API instead of being sleep based.

Original commit: elastic/x-pack-elasticsearch@e7a9689375
2016-10-11 10:25:40 +02:00
Alexander Reelsen fe00615965 Watcher: Moving test to new unified directory structure
Original commit: elastic/x-pack-elasticsearch@0cc22544a4
2016-10-10 11:25:30 +02:00
Nik Everett 769554460d Handle removing NodeServicesProvider
Original commit: elastic/x-pack-elasticsearch@b43637f2fb
2016-10-08 10:27:50 -04:00
jaymode f23e40b772 test: add bwc indices for 2.4.1
Original commit: elastic/x-pack-elasticsearch@19bec2111e
2016-10-07 14:21:48 -04:00
Nik Everett 1d2c6e5180 Handle new nullable ctor parmater
Original commit: elastic/x-pack-elasticsearch@d604dfe1d0
2016-10-07 10:40:35 -04:00
Simon Willnauer c226dfddc0 Filter out assertion transport interceptors in tests that expect an XPack request handler
in core we wrap request handlers with an asserting one to ensure we can serialize messages
with different versions. Yet, xpack uses the same functionality to add security aspects to
the network layer. These tests assert that the right handlers are in-place.

Original commit: elastic/x-pack-elasticsearch@e39c8995ae
2016-10-07 15:44:48 +02:00
Simon Willnauer 4c349a76fb just use hostname in tests since it's simplify forwarding
Original commit: elastic/x-pack-elasticsearch@b5cf3a4435
2016-10-07 11:58:59 +02:00
Simon Willnauer 2f70ae92b6 Cut over to MockTcpTransport since LocalTransport is remove in core (elastic/elasticsearch#3684)
This is a followup commit to elastic/elasticsearchelastic/elasticsearch#20695

Original commit: elastic/x-pack-elasticsearch@27cd454ba6
2016-10-07 11:28:05 +02:00
Colin Goodheart-Smithe f9aba3944e Changes to support the removal of the now callable in core (elastic/elasticsearch#3685)
Fixes to x-plugins code now that DateMathParser accepts a LongSupplier rather than a Callable to get the value of now

Relates to elastic/elasticsearchelastic/elasticsearch#20796

Original commit: elastic/x-pack-elasticsearch@99fc47a8a7
2016-10-07 10:26:42 +01:00
Simon Willnauer 31ed371ed0 Remove SearchContext#current and all it's threadlocals (elastic/elasticsearch#3677)
Followup PR for elastic/elasticsearchelastic/elasticsearch#20778

Original commit: elastic/x-pack-elasticsearch@1e3959545e
2016-10-06 19:52:34 +02:00
Igor Motov bb8c08f254 Explicitly specify analyzer scope
Related to elastic/elasticsearchelastic/elasticsearch#20197

Original commit: elastic/x-pack-elasticsearch@af9258a8a6
2016-10-06 09:11:28 -04:00
Jay Modi 6c587330fd security: use SSLParameters to set ciphers/protocols/client auth
This change moves to using SSLParameters as the configuration source for SSLEngine and SSLSocket
objects that are configured by the SSLService. Previously we used a mix of specific methods and
SSLParameters, which resulted in issues where ordering of calls is important. For example, if configuring
client authentication directly on the engine prior to setting the SSLParameters resulted in the client
authentication configuration being reset to the default.

Additionally, this change also sets use cipher suite order to true to ensure preferred ciphers will be used.

Original commit: elastic/x-pack-elasticsearch@8ddecdc20c
2016-10-06 07:19:28 -04:00
Colin Goodheart-Smithe 288f682fee elastic/elasticsearch#3667 Changes to DLS to support preventing requests that use scripts or now() from being cached
Changes to DLS to support preventing requests that use scripts or now() from being cached

Original commit: elastic/x-pack-elasticsearch@b69c2f5ca4
2016-10-06 10:24:59 +01:00
jaymode dd64ced206 test: wait for response before closing client
This change ensures we wait for a response before the async http client is closed. Otherwise we can
close the client during the connection to the remote endpoint or never even connect to the remote
endpoint.

Closes elastic/elasticsearch#3640

Original commit: elastic/x-pack-elasticsearch@54900b1b4a
2016-10-05 11:49:21 -04:00
Colin Goodheart-Smithe f2703f2d11 Changes to DLS to support elastic/elasticsearch#20750
This change fixes document level security to support the changes made in
elastic/elasticsearch#20750.

Original commit: elastic/x-pack-elasticsearch@d234be077d
2016-10-05 15:50:29 +01:00
Alexander Reelsen 5aacf3e205 Revert "Change Watcher thread pool to be scaling"
This reverts commit elastic/x-pack@943bd259f9.

See discussion in elastic/elasticsearch#3660

Original commit: elastic/x-pack-elasticsearch@35d236df59
2016-10-05 14:45:34 +02:00
jaymode a7e25cbaf9 test: ensure security index exists in tests expecting it to
This changes does two things in the tribe tests. The first is that when we split data up between
multiple clusters, we always force create the security index so that randomization does not cause
edge cases like the index not existing in the preferred cluster. The second is we look at the cluster
state of the nodes and ensure the tribe node sees the indices and has all primaries active.

Separate tests were also added to cover the scenario where the security index only exists in the non
preferred node.

Original commit: elastic/x-pack-elasticsearch@17b78ec837
2016-10-05 08:38:20 -04:00
Alexander Reelsen 53103e988f Watcher: Add proxy support to pagerduty action (elastic/elasticsearch#3542)
This is the last action that needs additional support for proxies.

You can set a proxy in the JSON like this:

```
"actions" : {
  "notify-pagerduty" : {
    "pagerduty" : {
      "description" : "Main system down, please check!",
      "proxy" : { "host" : "localhost", "port" : 8080 }
    }
  }
}
```

Closes elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@b99969fd6b
2016-10-05 10:10:02 +02:00
Alexander Reelsen 7ffebef2cd Watcher: Add proxy support to slack action (elastic/elasticsearch#3487)
You can set it like this in the JSON

"actions" : {
  "notify-slack" : {
    "slack" : {
      "account" : "integration-account",
      "proxy" : {
        "host" : "localhost",
        "port" : 8080
      },
      "message" : {
        ...
      }
    }
  }
}

Relates elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@de86233d4f
2016-10-05 09:07:09 +02:00
Jason Tedor 00cecac86e Change Watcher thread pool to be scaling
Watcher uses a custom thread pool. This is because executing watches can
be long-running tasks that often block on I/O and it is best to not
consume the core thread pools with these tasks. Today this thread pool
is fixed, and sized at five times the bounded number of cores (so 160 on
a 32-core box). It makes sense for there to possibly be so many threads,
again because these tasks can block on I/O and having excess capacity
lets unblocked watches execute. It's the fixed size that can cause
problem, all these threads are always consuming resources even when
there are no or not that many watches running. This commit changes this
thread pool to be a scaling thread pool.

Relates elastic/elasticsearch#3660

Original commit: elastic/x-pack-elasticsearch@3cafab6e83
2016-10-04 18:15:19 -04:00
Jason Tedor a0e1d44a44 Remove lenient URL parameter parsing
This commit adapts x-plugins for a change in core Elasticsearch that
removes lenient URL parameter parsing.

Relates elastic/elasticsearch#3641

Original commit: elastic/x-pack-elasticsearch@cc0687f32c
2016-10-04 12:46:54 -04:00
jaymode ddae0694c9 test: move SecurityTribeIT to right directory
Relates elastic/elasticsearch#3635

Original commit: elastic/x-pack-elasticsearch@b46ab0b63e
2016-10-04 09:22:21 -04:00
Luca Cavanna 91a68e9873 adapt to IndicesAliasesRequest not implementing CompositeIndicesRequest (elastic/elasticsearch#3645)
We need to special case IndicesAliasesRequest as it doesn't implement CompositeIndicesRequest anymore. Note that the similar loop for CompositeIndicesRequests's subrequests will soon go away

Relates to elastic/elasticsearch#3638

Original commit: elastic/x-pack-elasticsearch@50d119ff61
2016-10-04 10:39:31 +02:00
Ryan Ernst 36c7070217 Fix xpack api jar artifact naming
Original commit: elastic/x-pack-elasticsearch@bb7b0a6392
2016-10-03 19:25:44 -07:00
Ryan Ernst fa4d389c99 Fix artifact id for x-pack api jar
Original commit: elastic/x-pack-elasticsearch@26e47099a2
2016-10-03 16:38:25 -07:00
Ryan Ernst ee9dbac0c7 Update conditional version from alpha6 to beta1 in http exporter
Original commit: elastic/x-pack-elasticsearch@fdcef033b9
2016-10-03 09:37:12 -07:00
Ryan Ernst 947b3f26e6 Merge branch 'master' into reorg
Original commit: elastic/x-pack-elasticsearch@f279791e12
2016-10-03 08:36:39 -07:00
jaymode abc789c887 test: handle case where security index only exists in one tribe
Original commit: elastic/x-pack-elasticsearch@94f011387f
2016-10-03 11:27:15 -04:00
Jay Modi 52b7170121 security: native users and roles can be used on tribe nodes
This change allows native users and roles to be used on tribe nodes. The tribe node will actually
use the security index of one of the tribes, which must be specified with the `tribe.on_conflict`
setting. User and role modifications are not permitted when running on a tribe node.

Closes elastic/elasticsearch#3451

Original commit: elastic/x-pack-elasticsearch@2b762ca648
2016-10-03 10:12:30 -04:00
Ryan Ernst c309faec72 Merge branch 'master' into reorg
Original commit: elastic/x-pack-elasticsearch@24575f52a9
2016-10-01 09:49:14 +02:00
Ryan Ernst a51107769e Merge branch 'master' of github.com:elastic/x-plugins
Original commit: elastic/x-pack-elasticsearch@5ab54f94aa
2016-10-01 09:49:01 +02:00
Ryan Ernst 1fa0f835fe Build: Reorganize src roots
This change flattens the directory structure, both for the elasticsearch
specific directories, as well as within the elasticsearch x-pack plugin.

closes elastic/elasticsearch#2957

Original commit: elastic/x-pack-elasticsearch@45891a4632
2016-10-01 09:46:43 +02:00
Alexander Reelsen abe5f1dda9 Watcher: Ignore watch version when deleting a watch (elastic/elasticsearch#3630)
When deleting a watch the version was used as part of
the delete request. However a watch deletion means the
user really wants to get rid of it and not accidentally
run into a version exception because the watch was running in
between.

Original commit: elastic/x-pack-elasticsearch@e585f717f1
2016-09-30 15:45:05 +02:00
Ryan Ernst 1cca04924f Collapsed monitoring into xpack src
Original commit: elastic/x-pack-elasticsearch@9e523f2783
2016-09-29 12:25:57 +02:00
Ryan Ernst 98687d2884 Collapsed license code into xpack src
Original commit: elastic/x-pack-elasticsearch@42f7cb5a08
2016-09-29 12:22:19 +02:00
Ryan Ernst 1b55cc7c62 collapsed graph into xpack src
Original commit: elastic/x-pack-elasticsearch@de8085baed
2016-09-29 12:16:32 +02:00
Ryan Ernst 905237a56f Moved directories around
Original commit: elastic/x-pack-elasticsearch@2018bb5f9f
2016-09-29 12:03:14 +02:00
Ryan Ernst 09bcd8e5cd Merge pull request elastic/elasticsearch#3618 from rjernst/consolidate_dev_tools
Build: Conslidate dev-tools and x-dev-tools

Original commit: elastic/x-pack-elasticsearch@ff59116ea8
2016-09-29 11:29:10 +02:00
Ryan Ernst 0a3940103a Build: Conslidate dev-tools and x-dev-tools
In preparation for elastic/elasticsearch#2957, I found we have things both in the root level
dev-tools, as well as elasticsearch/x-dev-tools. Most of this stuff can
be removed as it had to do with the old manual release process. There
was also a signed license file checked in. I removed it here, we really
should not have licenses checked in IMO, and it is unclear what the
purpose of this license was for. The two remaining scripts were moved to
the root dev-tools.

Original commit: elastic/x-pack-elasticsearch@3e24ea2d56
2016-09-27 16:46:51 +02:00
Igor Motov aafc75ac51 Fix serialization issue in WatcherMetaData
Watcher meta data parser doesn't read the closing '}' which causes all following custom metadata to be ignored.

Similar to elastic/elasticsearch#1190

Original commit: elastic/x-pack-elasticsearch@d15b9ea466
2016-09-27 14:15:16 +02:00
Jay Modi d44ba28d27 security: always create the IPFilter in a node
When running as a node, we check the `xpack.security.transport.filter.enabled` setting to see
if we should create the IPFilter but this check is not really correct. The HTTP filter could be
enabled or a profile filter could be enabled so there are times when we may not be filtering connections
when we should. Additionally, since we do not bind the IPFilter to a null provider, Guice will try to create
one during startup to inject into the security transport. This results in an exception and startup fails.

This change always creates the IPFilter when running as a node. This IPFilter has its own settings and
logic to determine whether it should be filtering on a given network transport.

Closes elastic/elasticsearch#3592

Original commit: elastic/x-pack-elasticsearch@95c25651c4
2016-09-23 10:12:24 -04:00
Alexander Reelsen 7557168a0a Smoke tester: Adapt to new download structure (elastic/elasticsearch#3575)
This makes the smoke tester in x-pack work again after the work
on the unified release.

Original commit: elastic/x-pack-elasticsearch@4d4f1ec26c
2016-09-23 09:53:17 +02:00
Simon Willnauer 3c650e483b Followup from elastic/elasticsearchelastic/elasticsearch#20627 - Removal of AnalysisService
Original commit: elastic/x-pack-elasticsearch@75c14534a7
2016-09-23 08:54:44 +02:00
Tanguy Leroux 375bf95fb1 Remove duplicate methods in ByteSizeValue (elastic/elasticsearch#20560)
Some methods have been renamed in elastic/elasticsearchelastic/elasticsearch#20560. This commit change a .bytes() call to a .getBytes() call.

Original commit: elastic/x-pack-elasticsearch@4a0ff77361
2016-09-20 14:07:02 +02:00
Tanguy Leroux 1a7fbf9679 [Tests] Fix cat.templates to work when other templates exist
This commit fixes the cat.templates REST tests so that it works when other templates exist (like monitoring)

Original commit: elastic/x-pack-elasticsearch@2e27ad88b4
2016-09-20 12:05:21 +02:00
Ryan Ernst 803f20e267 Merge pull request elastic/elasticsearch#3534 from rjernst/license_prod_key
Build: Use licensing prod key when building release

Original commit: elastic/x-pack-elasticsearch@682c4eace5
2016-09-19 15:48:43 -07:00
Ryan Ernst 39160b5b22 Build: Use licensing prod key when building release
This change switches the build to use the licensing prod key when
building the xpack jar for release.

Original commit: elastic/x-pack-elasticsearch@54a21dae5b
2016-09-19 15:46:04 -07:00
Chris Earle a94c27d3de [Monitoring] Future-proof Monitoring Bulk API with "interval" param
This adds an "interval" placeholder parameter that is required to the Monitoring Bulk API, and adds it to the Kibana side of the plumbing.

Having this will allow us to add it to all incoming documents and start to report against it with the Insights, as well as to detect the _lack_ of incoming documents.

By adding it now, we can avoid having a non-BWC API change for Kibana in 5.1. We'll just pickup new data in our documents.

Original commit: elastic/x-pack-elasticsearch@5ba8aafe03
2016-09-19 18:21:09 -04:00
Simon Willnauer 2dde85ab33 Unguice Transport and friends (elastic/elasticsearch#3510)
This is a followup for elastic/elasticsearchelastic/elasticsearch#20526 removes the pluggability of
transport / http server transport via guice.

Original commit: elastic/x-pack-elasticsearch@5fb84949aa
2016-09-19 22:11:17 +02:00
Ryan Ernst 2a03af9849 Build: Remove x-pack client jar
The x-pack client jar and api jar are exactly the same: the entirety of
x-pack. Since we added the x-pack-transport jar, we no longer really
want the client jar as it is confusing. Additionally, it causes jar hell
when a test for an extension uses the transport client. This change
removes the client jar, and makes the x-pack transport client use the
api jar instead. This sounds odd at first, but since transport client is
going away eventually, it is a stopgap, and works.

closes elastic/elasticsearch#3309

Original commit: elastic/x-pack-elasticsearch@ee7a2c12c0
2016-09-19 11:42:27 -07:00
Alexander Reelsen 8c51b3b21a Docs/Release notes: Removed `force` parameter in delete watch API
Also updated the rest test descriptions

Original commit: elastic/x-pack-elasticsearch@890ba7703e
2016-09-19 11:49:10 +02:00
Alexander Reelsen 273a9fb46f Watcher: Fix possible chained input NPE for execution failures (elastic/elasticsearch#3490)
Due to untested code there was an NPE happening in production,
when a chained input execution failed, but the chained input
tried to access the resulting payload (which is never set on
failures). This payload now defaults to being empty.

This commit also drive-by fixes a broken logging statement, that
on the one side returned not the watch id, but a useless watch
toString() representation, and on the other hand only logs an error
message, but not a stack trace into the log, as this is what the
history is for.

Original commit: elastic/x-pack-elasticsearch@7dbe1afd90
2016-09-19 10:06:02 +02:00
Alexander Reelsen 5b265ea569 Watcher: Remove locking of watches for write operations elastic/elasticsearch#3481 (elastic/elasticsearch#3481)
Whenever a watch is updated (put, delete, set state), until now we
happened to reject those operations when a watch was executed at the
same time. However with long running reporting this might mean, that a
watch can never be changed, because it always gets executed.

* Removes the ability of write requests to obtain a lock at all (executing watches is still protected by a lock)
* Replaced the FairKeyedLock in watcher with the KeyedLock in Elasticsearch, which also takes a fair option, removed the FairKeyedLock
* Removed all the timeout parameters that are no longer needed, because there is no lock anymore
* Removed also the force parameter for watch deletion. Just do it[tm]
* Added a test that deleting a watch while it is being executed does not leave any leftovers

In case of a deletion of a watch during an execution, so that updating the status of the watch fails,
a warning is logged.

Closes elastic/elasticsearch#3417

Original commit: elastic/x-pack-elasticsearch@22fad1b797
2016-09-19 09:44:32 +02:00
jaymode e069c1f090 test: stop IndexAuditTrail before ESIntegTestCase#after runs
As part of the review of elastic/elasticsearch#3287, the stopping of the IndexAuditTrail was moved to the tearDown
method. This works sometimes but other times it fails because tearDown is run after
ESIntegTestCase#after, so the IndexAuditTrail is still running during the after checks which will
cause the test to fail since the shard lock cannot be obtained.

Closes elastic/elasticsearch#3520

Original commit: elastic/x-pack-elasticsearch@4cb52b15a2
2016-09-17 07:38:31 -04:00
Simon Willnauer 7be765d2a0 Ensure we have a consistent view on OperationMode in Watcher (elastic/elasticsearch#3507)
Today the operation mode can be set to default for a short amout of
time until it's reset to the actual mode this can cause weird sideeffects
for users if it's read concurrently. Also the test relies on a certain
happens before relationship that is not guaranteed since the operation
mode is set before the listerner is run. This change also rewrites the test
to not use busy waiting but wait for the actual listern to be executed.

Original commit: elastic/x-pack-elasticsearch@a2a42b89e5
2016-09-16 22:10:51 +02:00
Nik Everett c21a922778 Add backwards compatibility support to monitoring
1. We only support indexes created by Marvel 2.3+. All other indexes
are just ignored.
2. The tests don't assert a ton of interesting stuff because there
isn't a java API for Monitoring that we can just use. Instead we assert
that a few objects are there and look sane.
3. We don't migrate the contents of the data index. Instead we just
rely on Monitoring recreating it.

Original commit: elastic/x-pack-elasticsearch@86216c2d61
2016-09-16 13:20:02 -04:00
Jay Modi a6d55f26c6 security: simplify index audit trail stopping
The IndexAuditTrail had both a stop and close method that needed to be called in order
to stop the service. There was a race where we called either flush or close in a non
blocking fashion and then immediately closed the underlying client. This change makes
the stop method wait for up to 10 seconds when closing the bulk processor.

Closes elastic/elasticsearch#3279

Original commit: elastic/x-pack-elasticsearch@0d776bc91a
2016-09-16 10:31:27 -04:00
Simon Willnauer efeb9cefce Cut over SecurityServerTransportService to use the new Interceptor infrastructure (elastic/elasticsearch#3491)
TransportService is not pluggable anymore in core. Instead we now have a interceptor
infrastructure that allows to intercept send and receive calls on the transport layer.

Relates to elastic/elasticsearchelastic/elasticsearch#20505

Original commit: elastic/x-pack-elasticsearch@04194ecb09
2016-09-16 09:48:25 +02:00
Boaz Leskes 635b5a6800 fix import for ElectMasterService
it changed with https://github.com/elastic/elasticsearch/pull/20384

Original commit: elastic/x-pack-elasticsearch@c1e51de6a8
2016-09-15 23:45:24 +02:00
Alexander Reelsen efa2678691 Watcher: Add proxy support to hipchat action (elastic/elasticsearch#3475)
This adds proxy support to the hipchat action. Right now
neither hipchat nor slack nor pagerduty allow for this,
but if you dont need a proxy for internal http connections,
but you do for external, then this configuration cannot be done
without setting a proxy for those actions.

You can set it like this in the JSON

```
"actions" : {
  "notify-hipchat" : {
    "hipchat" : {
      "account" : "integration-account",
      "proxy" : {
        "host" : "localhost",
        "port" : 8080
      },
      "message" : {
        ...
      }
    }
  }
}

```

Relates elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@4e8447ce37
2016-09-15 11:24:04 +02:00
Yannick Welsch d35131ad52 Fix wrong logger usages (elastic/elasticsearch#3485)
Relates to elastic/elasticsearchelastic/elasticsearch#20490

Original commit: elastic/x-pack-elasticsearch@1ae7217b59
2016-09-15 10:48:06 +02:00
Jason Tedor b534bfec5a Complete Elasticsearch logger names
This commit modifies the loggers used within x-pack to adapt to the
change in core Elasticsearch where loggers are now named by the
fully-qualified class name.

Relates elastic/elasticsearch#3465

Original commit: elastic/x-pack-elasticsearch@4cac9ac62f
2016-09-13 22:47:10 -04:00
Chris Earle 6085c5aba9 [Monitoring UI] Cleanup Cluster Overview and Node Summary
This cleans up some of the reported stats to be a little clearer, including making the JVM Heap chart behave like Kibana's memory chart. This solves two problems: you can now determine the max heap size and you know what "x%" actually means relative to it.

Original commit: elastic/x-pack-elasticsearch@450f6fd546
2016-09-13 18:58:01 -04:00
Jason Tedor 6d4e4f5131 Fix failing logging audit tests
This commit fixes the logging audit tests which were broken due to an
upstream change in core Elasticsearch relating to the fact that prefixes
are no longer considered part of the log message, but are instead
implemented via markers.

Original commit: elastic/x-pack-elasticsearch@abd7ec23d8
2016-09-13 17:43:19 -04:00
Britta Weber 2c6d0b0cd2 Add option to deny access to fields (elastic/elasticsearch#2879)
To deny access to a fields users can name exceptions to field permissions with the following syntax:

"fields": {
     "grant": [list of field names patterns],
     "except": [list of patterns that are forbidden]
}

See doc for the rules for this.

This commit also reverts elastic/elasticsearch#2720

closes elastic/elasticsearch#2681



Original commit: elastic/x-pack-elasticsearch@d6537028ec
2016-09-13 16:38:58 +02:00
Martijn van Groningen 3b97936587 test: Remove WatcherBackwardsCompatibilityTests as it was specifically build for testing upgrade from 2.x to 5.x and to verify the scripts and template work/serialize as expected. On the master this is test is no longer relevant.
The OldWatcherIndicesBackwardsCompatibilityIT covers a major upgrade too.

Original commit: elastic/x-pack-elasticsearch@657881916b
2016-09-13 09:49:53 +00:00
Alexander Reelsen 67f7da18da Licensing: Parse start date in milliseconds as well
In order to behave like expiration date and and the issue date,
license parsing should be able to parse the start date in milliseconds
as well.

Relates elastic/elasticsearch#3385

Original commit: elastic/x-pack-elasticsearch@54c821192e
2016-09-13 09:57:29 +02:00
Simon Willnauer 3c619e8824 Follow up for elastic/elasticsearchelastic/elasticsearch#20423
Original commit: elastic/x-pack-elasticsearch@53c72d913a
2016-09-12 22:51:30 +02:00
jaymode 2358309f72 security: allow enabled and username fields in put user request body
The enabled and username fields are both now allowed in the request body for the put user
request. This makes it easier to perform a get and update a user without needing to edit more
of the request body than necessary.

Closes elastic/elasticsearch#3391

Original commit: elastic/x-pack-elasticsearch@ab763e843b
2016-09-12 16:14:57 -04:00
jaymode 5f4e6164e5 security: add a built-in role for reporting
This commit adds a built-in role that grants read and write privileges to the reporting
indices.

See elastic/elasticsearch#2374
Closes elastic/elasticsearch#3196

Original commit: elastic/x-pack-elasticsearch@c8c1b465f8
2016-09-12 16:04:08 -04:00
jaymode 7a321534ea security: allow the reserved realm to be disabled
This change allows the reserved realm to be disabled via a setting that is undocumented.

Closes elastic/elasticsearch#3399

Original commit: elastic/x-pack-elasticsearch@3c6c93d7eb
2016-09-12 14:13:47 -04:00
jaymode a119f7ccf1 update the HttpClient to allow for disabling hostname verification
The HttpClient used in xpack supports the new SSL configuration but did not properly obey
the hostname verification disabling that can be specified with these settings. This change
adds the functionality with a test.

Relates elastic/elasticsearch#3240
Relates elastic/elasticsearch#3164

Original commit: elastic/x-pack-elasticsearch@df8e12a5aa
2016-09-12 13:47:08 -04:00
jaymode 6d2fcbe688 security: fix typo is syskeygen script
Original commit: elastic/x-pack-elasticsearch@36abb4284a
2016-09-12 12:26:14 -04:00
Alexander Reelsen 0c3466180f Licensing: Add start date to licenses (elastic/elasticsearch#3385)
Start dates are a required feature for cloud. This functionality adds support
for specifying and enforcing a start date on licenses.

Behaviour: If the start date is > than now, the license will be rejected.

Due to another field in the license class, the version of the License class as well
as its serialization methods are adapted to this.

Closes elastic/elasticsearch#3370

Original commit: elastic/x-pack-elasticsearch@eb2a6f5be3
2016-09-12 17:53:33 +02:00
Nik Everett caf4bd2c82 Be careful when old index tests start nodes
We were starting nodes at weird times and then shutting them down again,
slowing down the tests and causing the watcher tests to fail because
watcher wasn't being shut down with its traditional kid gloves.

Original commit: elastic/x-pack-elasticsearch@2fd81b3eaf
2016-09-12 11:14:51 -04:00
Alexander Reelsen 2a6a9a10f7 Watcher: Improve http attachment history information (elastic/elasticsearch#3436)
When the HTTP attachment was not able to successfully retrieve the
data from and endpoint, there was no indication in the watch history
of what went wrong. Instead a logger was used, which is not useful
for the person running the watches.

This commit removes the logger statement and throws an exception,
so that the exception message can be stored in the watch history.

Source of this issue was a forum post:
https://discuss.elastic.co/t/sending-e-mail-with-generated-report-fails/60263/6

Original commit: elastic/x-pack-elasticsearch@acdaf7abef
2016-09-12 17:04:22 +02:00
Martijn van Groningen 5c8ece8583 test: mute test
Original commit: elastic/x-pack-elasticsearch@112f853bac
2016-09-10 15:10:41 +02:00
javanna 69c4c693f6 Merge branch 'enhancement/cleanup_parse_elements'
Original commit: elastic/x-pack-elasticsearch@0f86204352
2016-09-09 22:47:29 +02:00
Nik Everett 2aa0781971 Explicitly shut down watcher after tests
In our tests you have to explicitly shut down watcher rather than shut
down the node it is running on because of thread leak detection. Just
shutting down the node that it is running on will cause it to start up
on another node if there is another one running and then not properly
shut down. This is probably something that should be fixed in watcher
somehow but for now lets just be more careful with the tests.

Closes elastic/elasticsearch#2365
Closes elastic/elasticsearch#2588

Original commit: elastic/x-pack-elasticsearch@fb8a172972
2016-09-09 14:43:46 -04:00
javanna 5829d112d1 Move search ext section parsing to the coordinating node
Original commit: elastic/x-pack-elasticsearch@1ce38fcff9
2016-09-09 18:58:10 +02:00
Jason Tedor 592ca5ab7b Adjust to logger API changes in Elasticsearch
Previously core Elasticsearch had methods in a test class for removing
and adding appenders. However, these methods were moved to production
code. This commit adjusts x-plugins for this change.

Original commit: elastic/x-pack-elasticsearch@83e37ef65a
2016-09-09 09:15:53 -04:00
Martijn van Groningen b74f1e6cb2 watcher: Add limited capability to upgrade the source of a watcher upon startup.
This particular change focuses on upgrading the source of a watch when it comes to scripts that have no language specified explicitly.
The default language in version 5 changed to painless from whatever is specified in `script.default_lang` setting (this defaulted to groovy). In order to make sure that scripts in watcher remain to work we should rewrite the search source upon startup and set the legacy default language explicitly. The legacy script language is now controlled by `script.legacy.default_lang` setting and that defaults to groovy.

Changing the source upon startup should do the trick and only change the source of watches with scripts that don't have an explicit language set. For new watches the default language used in scripts is painless and because we now always serialize the language explicitly in scripts these watches won't be changed on startup.

 The upgrade logic added here tries to upgrade scripts in the following places in a watch:
 * script condition
 * script transform
 * any script defined inside of a search input

Original commit: elastic/x-pack-elasticsearch@4d578819eb
2016-09-09 14:26:24 +02:00
Britta Weber 4bf685cd31 [TEST] wait for pending tasks finished before testing cluster state collector
Original commit: elastic/x-pack-elasticsearch@d42c8c4bbe
2016-09-09 12:07:52 +02:00
Tanguy Leroux 6ca086b997 Fix line length in LoggingAuditTrailTests.java
Original commit: elastic/x-pack-elasticsearch@3f4185dce7
2016-09-09 10:50:43 +02:00
Colin Goodheart-Smithe 8b6d988cfb Security: Audit all HTTP requests
Adds a new audit event (authentication_success) which logs each request made to
the REST API along with the body of the request

Closes elastic/elasticsearch#912

Original commit: elastic/x-pack-elasticsearch@650b9d70c0
2016-09-09 09:28:27 +01:00
Simon Willnauer 923949ec67 Downgrade hard bwc break to a TODO for now
Original commit: elastic/x-pack-elasticsearch@73c2a6ee3e
2016-09-08 22:05:36 +02:00
Simon Willnauer 5c407cc526 Revert "Remove bwc layer for TimeUnit parsing"
This reverts commit elastic/x-pack@c6b2bd2c12.

Original commit: elastic/x-pack-elasticsearch@b4fbfb8afe
2016-09-08 22:03:46 +02:00
Jason Tedor 9baf88dfaa Handle node validation exception when starting
Node#start can now throw a checked NodeValidationException; this commit
adapts x-plugins for this new API.

Relates elastic/elasticsearch#3381

Original commit: elastic/x-pack-elasticsearch@660c5ab8c1
2016-09-08 10:56:54 -04:00
Simon Willnauer 20fce3280e Remove bwc layer for TimeUnit parsing
Original commit: elastic/x-pack-elasticsearch@79145f279b
2016-09-08 16:43:46 +02:00
Tanguy Leroux 97182fefb9 Clean up XContentBuilder in X-Pack
This commit cleans most of the methods of XContentBuilder so that:
    - Jackson's convenience methods are used instead of our custom ones (ie field(String,long) now uses Jackson's writeNumberField(String, long) instead of calling writeField(String) then writeNumber(long))
    - null checks are added for all field names and values
    - methods are grouped by type in the class source
    - methods have the same parameters names
    - duplicated methods like field(String, String...) and array(String, String...) are removed
    - varargs methods now have the "array" name to reflect that it builds arrays
    - unused methods like field(String,BigDecimal) are removed
    - all methods now follow the execution path: field(String,?) -> field(String) then value(?), and value(?) -> writeSomething() method. Methods to build arrays also follow the same execution path.

Original commit: elastic/x-pack-elasticsearch@d83f3aa6e2
2016-09-08 14:17:15 +02:00
jaymode 2257cc77f6 security: add reserved metadata to the ingest admin role
Closes elastic/elasticsearch#3376

Original commit: elastic/x-pack-elasticsearch@1909da2fb0
2016-09-07 19:19:02 -04:00
Alexander Reelsen 0f571685b9 Watcher: Fix proxy xcontent serialization (elastic/elasticsearch#3364)
Calling to xcontent in a HttpRequest, with a proxy enabled, lead to
serialization exceptions, resulting in failing to write the watch
history.

Closes elastic/elasticsearch#3334

Original commit: elastic/x-pack-elasticsearch@a04dff686c
2016-09-07 16:29:15 +02:00
Alexander Reelsen a296e31a7c Watcher: Ensure triggered watch is deleted on thread pool rejection (elastic/elasticsearch#3049)
This fixes a bug I found with a customer when he updated from 1.x to 2.x.
Due to an BWC incompatible change in the watch history mapping and a thread
pool rejection during execution a watch was not removed from the triggered
watches and tried to be executed again.

While trying to fix it it turned out that the execution of the failure
test case was still done in the transport thread and thus required some
offloading to another thread pool.

Original commit: elastic/x-pack-elasticsearch@df04ce31f2
2016-09-07 15:55:33 +02:00
Alexander Reelsen bb033f1e00 Watcher: Clear out WatchStore on watch index deletion (elastic/elasticsearch#2807)
If someone deletes the watch index (i.e. by deleting all indices), the watcher
in memory store still contains all the watches and tries to execute watches -
which results in exceptions as the watch itself cannot be updated anymore.

In order to minimize this problem (it cant be get rid of completely), we should
act accordingly if the watch index goes missing (either deleted or closed) and
clear out the memory representation of watches in the watchstore as well as trying
to finish all the current executions.

Closes elastic/elasticsearch#2794

Original commit: elastic/x-pack-elasticsearch@12d98cd566
2016-09-07 15:06:03 +02:00
javanna ad5606d1f7 [TEST] don't use null script lang in WatcherUtilsTests
same as elastic/x-pack@9e1e0988c1 but for testSerializeSearchRequest this time.

Original commit: elastic/x-pack-elasticsearch@aa7e3814b6
2016-09-06 23:00:56 +02:00
Nik Everett edb8c12f75 Always use mustache language
Using a null language just defaults to painless which is wrong for
watcher.

Original commit: elastic/x-pack-elasticsearch@48bdc31695
2016-09-06 15:45:23 -04:00
Nik Everett 05222b0d3f Deal with ConcreteShardRequest
It exists now and the intercepters have to deal with it.

Original commit: elastic/x-pack-elasticsearch@fdef0578ed
2016-09-06 15:42:06 -04:00
Martijn van Groningen 36486a7993 Changes for PR: https://github.com/elastic/elasticsearch/pull/20310
Original commit: elastic/x-pack-elasticsearch@9db3d05274
2016-09-06 18:44:41 +02:00
jaymode 7965608add security: make control of logfile audit output consistent with index output
This change moves the logfile audit output from determining what to log based on the
logger level to a enum based configuration that is used by the index output.

A few notable changes were made:

* We alway log all the information we have except for the request body
* The request body is no longer logged by default for REST events; the user needs to
explicitly opt in as there could be sensitive data in the body
* Added a `realm_authentication_failed` event that separates overall authentication
failure from that of an individual realm

Original commit: elastic/x-pack-elasticsearch@343a2bcdd9
2016-09-06 10:50:33 -04:00
Nik Everett 89ce4ebb08 Fix remaining fractional TimeValue issues with watcher
Removes the remaining spots where watcher makes fractional TimeValues.

Closes elastic/elasticsearch#3231

Original commit: elastic/x-pack-elasticsearch@22b0d37ed3
2016-09-06 09:53:22 -04:00
jaymode cd9add5350 test: remove unnecessary verify check
Original commit: elastic/x-pack-elasticsearch@f748c0a565
2016-09-06 08:36:04 -04:00
jaymode 1e6a924e74 security: add support for disabling users
This change adds support for disabling users. Users can be disabled by setting the enabled
property to false and the AuthenticationService will check to make sure that the user is enabled.
If the user is not enabled, this will be audited as an authentication failure.

Also as part of this work, the AnonymousUser was cleaned up to remove having a static instance
that caused issues with tests.

Finally, the poller of users was removed to simplify the code in the NativeUsersStore. In our other
realms we rely on the clear cache APIs and the timeout of the user cache. We should have the
same semantics for the native realm.

Closes elastic/elasticsearch#2172

Original commit: elastic/x-pack-elasticsearch@0820e40183
2016-09-06 07:41:48 -04:00
Chris Earle ec008ec4a9 Looong lines
Original commit: elastic/x-pack-elasticsearch@8c7931c179
2016-09-05 19:05:28 -04:00
Chris Earle 5d5a9afb57 [Monitoring] Use Low Level REST Client for HTTP Exporter
This rewrites the HTTP Exporter to use the REST client underneath. Functionality is improved in resource blocking (templates and pipelines existing) and the majority of the code fundamentall simplified by removing direct HTTP calls.

This is blocked by the SSLService pull request. After that is merged, the I will update this PR to reflect those changes and it could possibly allow us to remove the security privileges required for monitoring.

Original commit: elastic/x-pack-elasticsearch@1ad25f17f8
2016-09-05 15:16:03 -04:00
Alexander Reelsen 41334abda0 Watcher: Reenabling HipChatServiceTests
The authentication token stopped working, deleted it and created a new one
on the hipchat web site. Tests are working now again.

Closes elastic/elasticsearch#3162

Original commit: elastic/x-pack-elasticsearch@a0623af799
2016-09-05 12:27:14 +02:00
Martijn van Groningen 09c7f534b3 added jdocs and rename
Original commit: elastic/x-pack-elasticsearch@8a97b420c1
2016-09-03 11:01:47 +02:00
Nik Everett 3c33879018 Mark OldWatchIndices test AwaitsFix
Is freaking out in CI.

Original commit: elastic/x-pack-elasticsearch@3ab43467ed
2016-09-02 18:31:19 -04:00
Martijn van Groningen 565f50dbe5 Remove custom parsing and toXContent logic from TextTemplate and delegate to the Script as much as possible
Original commit: elastic/x-pack-elasticsearch@6d23549dc1
2016-09-02 23:36:07 +02:00
Nik Everett 203faaf4f4 2.x backwards compatibility support for watcher
Basic backwards compatibility support for watcher.

Closes elastic/elasticsearch#3230

Relates to elastic/elasticsearch#3231 - this actually should fix all the failures caused
by fractional time values but it does so by being able to parse them.
Being able to parse them is important for 2.x compatibility but 5.0
watches shouldn't produce fractional time values. This fixes the
particular way of making fractional time values mentioned in elastic/elasticsearch#3231
but I expect there are a half dozen more places to fix. The actual
watcher tests are fairly basic.

Original commit: elastic/x-pack-elasticsearch@328717455c
2016-09-02 09:24:46 -04:00
javanna 119bb67967 adapt to elastic/elasticsearch#20288: More info classes to implement Writeable rather than Streamable
Original commit: elastic/x-pack-elasticsearch@22cadf604d
2016-09-02 10:24:15 +02:00
Jack Conradson de4c776f0f Merge pull request elastic/elasticsearch#3272 from jdconrad/deprecate
Deprecate Groovy, Python, and Javascript

Original commit: elastic/x-pack-elasticsearch@e5c1d5a112
2016-09-01 16:30:25 -07:00
Chris Earle bb5ff7ab36 Fix test on Linux; setting name missing 'total.'
Original commit: elastic/x-pack-elasticsearch@de0183b3ef
2016-09-01 18:12:00 -04:00
Jack Conradson 670a57274a Merge branch 'master' into deprecate
Original commit: elastic/x-pack-elasticsearch@c9636bd3f1
2016-09-01 14:53:42 -07:00
Chris Earle 6d40cb63d0 [Monitoring] Collect IO Stats
This ensures that the data exists in 5.0.

Original commit: elastic/x-pack-elasticsearch@bc6e7931d1
2016-09-01 15:45:51 -04:00
Chris Earle b60e8aebd2 [Monitoring] Publish X-Pack Usage with Cluster Info
This publishes X-Pack usage data to the cluster info from the elected master node. This allows phone home to retrieve this data from the index, rather than fetching it live from the connected cluster (thereby not getting it from any n - 1 clusers that are not connceted).

Original commit: elastic/x-pack-elasticsearch@79bfaaaf0b
2016-09-01 15:38:54 -04:00
jaymode 2e010d52e9 security: do not allow built-in user/role names to be defined in the file realm
This change restricts built-in user/role names from passing validation when we are reading or modifying
the files used by this realm.

Closes elastic/elasticsearch#2078

Original commit: elastic/x-pack-elasticsearch@9f6b34f39d
2016-09-01 14:46:38 -04:00
jaymode 74f55bf46e security: add option to specify the certificate validity period
This commit adds the option to specify the number of days that a certificate is valid for.

Closes elastic/elasticsearch#2877

Original commit: elastic/x-pack-elasticsearch@fcf0a28321
2016-09-01 14:21:30 -04:00
jaymode 5d5bd015c0 add a PreBuiltXPackTransportClient
This change adds a transport client that comes preconfigured with the same plugins as the
PreBuiltTransportClient and also adds x-pack.

Closes elastic/elasticsearch#2970

Original commit: elastic/x-pack-elasticsearch@bb60534bd4
2016-09-01 13:49:10 -04:00
Jack Conradson 1d6d20679d Merge branch 'master' into deprecate
Original commit: elastic/x-pack-elasticsearch@afdfa26dc0
2016-09-01 08:52:34 -07:00
Chris Earle 6f115a532d [Monitoring] Remove "agent" package
This removes the "agent" package from org.elasticsearch.xpack.monitoring.agent.*, so that now everything is simply org.elasticsearch.xpack.monitoring.*.

Follow-on work will be refactoring some of the other code, but this is a first step now that it's always the agent (in effect).

Original commit: elastic/x-pack-elasticsearch@14025cb17c
2016-09-01 10:56:27 -04:00
jaymode ad9a7c9b96 Migrate xpack to use the common ssl configuration
This change migrates xpack (security, watcher, and monitoring) to use the common ssl
configuration for the elastic stack. As part of this work, several aspects of how we deal
with SSL has been modified.

From a functionality perspective, an xpack wide configuration for SSL was added and
all of the code that needs SSL uses the SSLService now. The following is a list of all
of the aspects of xpack that can have their own SSL configuration, which are separate
from the xpack wide configuration:

* Transport
* Transport profiles
* HTTP Transport
* Realms
* Monitoring Exporters
* HTTP Client

In terms of the code, some cleanups were made with these changes. SSLConfiguration is
now a concrete class and SSLConfiguration.Custom and SSLConfiguration.Global have been
removed. The validate method on key and trust configurations has been removed and these
classes will now throw exceptions when they are constructed with bad values. The
OptionalSettings helper class has been removed as it was just a file with one line functions
that made the code harder to understand. The SSL configuration and service classes have
been moved from the security source directories to the main xpack source set. The SSLService
now handles more of the configuration of the SSLEngine it returns to prevent callers from
having to handle those aspects. The settings that get registered for SSL have been moved to
XPackSettings.

Also included in this PR is a update to the docs around SSL. This includes a large simplification to
the documentation in that the certificate authority configuration section has been removed and the
process that is documented for generating certificates only includes the CLI tool that we bundle.

Closes elastic/elasticsearch#3104
Closes elastic/elasticsearch#2971
Closes elastic/elasticsearch#3164

Original commit: elastic/x-pack-elasticsearch@5bd9e5ef38
2016-09-01 10:51:41 -04:00
jaymode 54103127d0 fix line length
Original commit: elastic/x-pack-elasticsearch@caca0fc0b7
2016-09-01 09:37:05 -04:00
jaymode c5cde120b7 test: add 2.4.0 bwc index
Original commit: elastic/x-pack-elasticsearch@58bcf3abaf
2016-09-01 09:09:48 -04:00
jaymode 836e1d3a28 security: add a reserved role for the ingest feature in Kibana
Closes elastic/elasticsearch#1667

Original commit: elastic/x-pack-elasticsearch@da2f6d9c4e
2016-09-01 08:09:36 -04:00
Simon Willnauer 3638c8639f [TEST] Don't set create=true if no ID is specified
Original commit: elastic/x-pack-elasticsearch@874def17e8
2016-09-01 10:35:44 +02:00
Jason Tedor b29cdc0877 Fix additional exception logging call
This commit modifies an exception logging calls to use a parameterized
message from Log4j.

Original commit: elastic/x-pack-elasticsearch@7b0df7d056
2016-08-31 23:16:07 -04:00
Jack Conradson 0ca243dc83 Merge branch 'master' into deprecate
Original commit: elastic/x-pack-elasticsearch@25985e9144
2016-08-31 15:57:44 -07:00