Commit Graph

1195 Commits

Author SHA1 Message Date
Simon Willnauer 4c349a76fb just use hostname in tests since it's simplify forwarding
Original commit: elastic/x-pack-elasticsearch@b5cf3a4435
2016-10-07 11:58:59 +02:00
Simon Willnauer 2f70ae92b6 Cut over to MockTcpTransport since LocalTransport is remove in core (elastic/elasticsearch#3684)
This is a followup commit to elastic/elasticsearchelastic/elasticsearch#20695

Original commit: elastic/x-pack-elasticsearch@27cd454ba6
2016-10-07 11:28:05 +02:00
Colin Goodheart-Smithe f9aba3944e Changes to support the removal of the now callable in core (elastic/elasticsearch#3685)
Fixes to x-plugins code now that DateMathParser accepts a LongSupplier rather than a Callable to get the value of now

Relates to elastic/elasticsearchelastic/elasticsearch#20796

Original commit: elastic/x-pack-elasticsearch@99fc47a8a7
2016-10-07 10:26:42 +01:00
Simon Willnauer 31ed371ed0 Remove SearchContext#current and all it's threadlocals (elastic/elasticsearch#3677)
Followup PR for elastic/elasticsearchelastic/elasticsearch#20778

Original commit: elastic/x-pack-elasticsearch@1e3959545e
2016-10-06 19:52:34 +02:00
Igor Motov bb8c08f254 Explicitly specify analyzer scope
Related to elastic/elasticsearchelastic/elasticsearch#20197

Original commit: elastic/x-pack-elasticsearch@af9258a8a6
2016-10-06 09:11:28 -04:00
Jay Modi 6c587330fd security: use SSLParameters to set ciphers/protocols/client auth
This change moves to using SSLParameters as the configuration source for SSLEngine and SSLSocket
objects that are configured by the SSLService. Previously we used a mix of specific methods and
SSLParameters, which resulted in issues where ordering of calls is important. For example, if configuring
client authentication directly on the engine prior to setting the SSLParameters resulted in the client
authentication configuration being reset to the default.

Additionally, this change also sets use cipher suite order to true to ensure preferred ciphers will be used.

Original commit: elastic/x-pack-elasticsearch@8ddecdc20c
2016-10-06 07:19:28 -04:00
Colin Goodheart-Smithe 288f682fee elastic/elasticsearch#3667 Changes to DLS to support preventing requests that use scripts or now() from being cached
Changes to DLS to support preventing requests that use scripts or now() from being cached

Original commit: elastic/x-pack-elasticsearch@b69c2f5ca4
2016-10-06 10:24:59 +01:00
jaymode dd64ced206 test: wait for response before closing client
This change ensures we wait for a response before the async http client is closed. Otherwise we can
close the client during the connection to the remote endpoint or never even connect to the remote
endpoint.

Closes elastic/elasticsearch#3640

Original commit: elastic/x-pack-elasticsearch@54900b1b4a
2016-10-05 11:49:21 -04:00
Colin Goodheart-Smithe f2703f2d11 Changes to DLS to support elastic/elasticsearch#20750
This change fixes document level security to support the changes made in
elastic/elasticsearch#20750.

Original commit: elastic/x-pack-elasticsearch@d234be077d
2016-10-05 15:50:29 +01:00
Alexander Reelsen 5aacf3e205 Revert "Change Watcher thread pool to be scaling"
This reverts commit elastic/x-pack@943bd259f9.

See discussion in elastic/elasticsearch#3660

Original commit: elastic/x-pack-elasticsearch@35d236df59
2016-10-05 14:45:34 +02:00
jaymode a7e25cbaf9 test: ensure security index exists in tests expecting it to
This changes does two things in the tribe tests. The first is that when we split data up between
multiple clusters, we always force create the security index so that randomization does not cause
edge cases like the index not existing in the preferred cluster. The second is we look at the cluster
state of the nodes and ensure the tribe node sees the indices and has all primaries active.

Separate tests were also added to cover the scenario where the security index only exists in the non
preferred node.

Original commit: elastic/x-pack-elasticsearch@17b78ec837
2016-10-05 08:38:20 -04:00
Alexander Reelsen 53103e988f Watcher: Add proxy support to pagerduty action (elastic/elasticsearch#3542)
This is the last action that needs additional support for proxies.

You can set a proxy in the JSON like this:

```
"actions" : {
  "notify-pagerduty" : {
    "pagerduty" : {
      "description" : "Main system down, please check!",
      "proxy" : { "host" : "localhost", "port" : 8080 }
    }
  }
}
```

Closes elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@b99969fd6b
2016-10-05 10:10:02 +02:00
Alexander Reelsen 7ffebef2cd Watcher: Add proxy support to slack action (elastic/elasticsearch#3487)
You can set it like this in the JSON

"actions" : {
  "notify-slack" : {
    "slack" : {
      "account" : "integration-account",
      "proxy" : {
        "host" : "localhost",
        "port" : 8080
      },
      "message" : {
        ...
      }
    }
  }
}

Relates elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@de86233d4f
2016-10-05 09:07:09 +02:00
Jason Tedor 00cecac86e Change Watcher thread pool to be scaling
Watcher uses a custom thread pool. This is because executing watches can
be long-running tasks that often block on I/O and it is best to not
consume the core thread pools with these tasks. Today this thread pool
is fixed, and sized at five times the bounded number of cores (so 160 on
a 32-core box). It makes sense for there to possibly be so many threads,
again because these tasks can block on I/O and having excess capacity
lets unblocked watches execute. It's the fixed size that can cause
problem, all these threads are always consuming resources even when
there are no or not that many watches running. This commit changes this
thread pool to be a scaling thread pool.

Relates elastic/elasticsearch#3660

Original commit: elastic/x-pack-elasticsearch@3cafab6e83
2016-10-04 18:15:19 -04:00
Jason Tedor a0e1d44a44 Remove lenient URL parameter parsing
This commit adapts x-plugins for a change in core Elasticsearch that
removes lenient URL parameter parsing.

Relates elastic/elasticsearch#3641

Original commit: elastic/x-pack-elasticsearch@cc0687f32c
2016-10-04 12:46:54 -04:00
jaymode ddae0694c9 test: move SecurityTribeIT to right directory
Relates elastic/elasticsearch#3635

Original commit: elastic/x-pack-elasticsearch@b46ab0b63e
2016-10-04 09:22:21 -04:00
Luca Cavanna 91a68e9873 adapt to IndicesAliasesRequest not implementing CompositeIndicesRequest (elastic/elasticsearch#3645)
We need to special case IndicesAliasesRequest as it doesn't implement CompositeIndicesRequest anymore. Note that the similar loop for CompositeIndicesRequests's subrequests will soon go away

Relates to elastic/elasticsearch#3638

Original commit: elastic/x-pack-elasticsearch@50d119ff61
2016-10-04 10:39:31 +02:00
Ryan Ernst 36c7070217 Fix xpack api jar artifact naming
Original commit: elastic/x-pack-elasticsearch@bb7b0a6392
2016-10-03 19:25:44 -07:00
Ryan Ernst fa4d389c99 Fix artifact id for x-pack api jar
Original commit: elastic/x-pack-elasticsearch@26e47099a2
2016-10-03 16:38:25 -07:00
Ryan Ernst ee9dbac0c7 Update conditional version from alpha6 to beta1 in http exporter
Original commit: elastic/x-pack-elasticsearch@fdcef033b9
2016-10-03 09:37:12 -07:00
Ryan Ernst 947b3f26e6 Merge branch 'master' into reorg
Original commit: elastic/x-pack-elasticsearch@f279791e12
2016-10-03 08:36:39 -07:00
jaymode abc789c887 test: handle case where security index only exists in one tribe
Original commit: elastic/x-pack-elasticsearch@94f011387f
2016-10-03 11:27:15 -04:00
Jay Modi 52b7170121 security: native users and roles can be used on tribe nodes
This change allows native users and roles to be used on tribe nodes. The tribe node will actually
use the security index of one of the tribes, which must be specified with the `tribe.on_conflict`
setting. User and role modifications are not permitted when running on a tribe node.

Closes elastic/elasticsearch#3451

Original commit: elastic/x-pack-elasticsearch@2b762ca648
2016-10-03 10:12:30 -04:00
Ryan Ernst c309faec72 Merge branch 'master' into reorg
Original commit: elastic/x-pack-elasticsearch@24575f52a9
2016-10-01 09:49:14 +02:00
Ryan Ernst a51107769e Merge branch 'master' of github.com:elastic/x-plugins
Original commit: elastic/x-pack-elasticsearch@5ab54f94aa
2016-10-01 09:49:01 +02:00
Ryan Ernst 1fa0f835fe Build: Reorganize src roots
This change flattens the directory structure, both for the elasticsearch
specific directories, as well as within the elasticsearch x-pack plugin.

closes elastic/elasticsearch#2957

Original commit: elastic/x-pack-elasticsearch@45891a4632
2016-10-01 09:46:43 +02:00
Alexander Reelsen abe5f1dda9 Watcher: Ignore watch version when deleting a watch (elastic/elasticsearch#3630)
When deleting a watch the version was used as part of
the delete request. However a watch deletion means the
user really wants to get rid of it and not accidentally
run into a version exception because the watch was running in
between.

Original commit: elastic/x-pack-elasticsearch@e585f717f1
2016-09-30 15:45:05 +02:00
Ryan Ernst 1cca04924f Collapsed monitoring into xpack src
Original commit: elastic/x-pack-elasticsearch@9e523f2783
2016-09-29 12:25:57 +02:00
Ryan Ernst 98687d2884 Collapsed license code into xpack src
Original commit: elastic/x-pack-elasticsearch@42f7cb5a08
2016-09-29 12:22:19 +02:00
Ryan Ernst 1b55cc7c62 collapsed graph into xpack src
Original commit: elastic/x-pack-elasticsearch@de8085baed
2016-09-29 12:16:32 +02:00
Ryan Ernst 905237a56f Moved directories around
Original commit: elastic/x-pack-elasticsearch@2018bb5f9f
2016-09-29 12:03:14 +02:00
Ryan Ernst 09bcd8e5cd Merge pull request elastic/elasticsearch#3618 from rjernst/consolidate_dev_tools
Build: Conslidate dev-tools and x-dev-tools

Original commit: elastic/x-pack-elasticsearch@ff59116ea8
2016-09-29 11:29:10 +02:00
Ryan Ernst 0a3940103a Build: Conslidate dev-tools and x-dev-tools
In preparation for elastic/elasticsearch#2957, I found we have things both in the root level
dev-tools, as well as elasticsearch/x-dev-tools. Most of this stuff can
be removed as it had to do with the old manual release process. There
was also a signed license file checked in. I removed it here, we really
should not have licenses checked in IMO, and it is unclear what the
purpose of this license was for. The two remaining scripts were moved to
the root dev-tools.

Original commit: elastic/x-pack-elasticsearch@3e24ea2d56
2016-09-27 16:46:51 +02:00
Igor Motov aafc75ac51 Fix serialization issue in WatcherMetaData
Watcher meta data parser doesn't read the closing '}' which causes all following custom metadata to be ignored.

Similar to elastic/elasticsearch#1190

Original commit: elastic/x-pack-elasticsearch@d15b9ea466
2016-09-27 14:15:16 +02:00
Jay Modi d44ba28d27 security: always create the IPFilter in a node
When running as a node, we check the `xpack.security.transport.filter.enabled` setting to see
if we should create the IPFilter but this check is not really correct. The HTTP filter could be
enabled or a profile filter could be enabled so there are times when we may not be filtering connections
when we should. Additionally, since we do not bind the IPFilter to a null provider, Guice will try to create
one during startup to inject into the security transport. This results in an exception and startup fails.

This change always creates the IPFilter when running as a node. This IPFilter has its own settings and
logic to determine whether it should be filtering on a given network transport.

Closes elastic/elasticsearch#3592

Original commit: elastic/x-pack-elasticsearch@95c25651c4
2016-09-23 10:12:24 -04:00
Alexander Reelsen 7557168a0a Smoke tester: Adapt to new download structure (elastic/elasticsearch#3575)
This makes the smoke tester in x-pack work again after the work
on the unified release.

Original commit: elastic/x-pack-elasticsearch@4d4f1ec26c
2016-09-23 09:53:17 +02:00
Simon Willnauer 3c650e483b Followup from elastic/elasticsearchelastic/elasticsearch#20627 - Removal of AnalysisService
Original commit: elastic/x-pack-elasticsearch@75c14534a7
2016-09-23 08:54:44 +02:00
Tanguy Leroux 375bf95fb1 Remove duplicate methods in ByteSizeValue (elastic/elasticsearch#20560)
Some methods have been renamed in elastic/elasticsearchelastic/elasticsearch#20560. This commit change a .bytes() call to a .getBytes() call.

Original commit: elastic/x-pack-elasticsearch@4a0ff77361
2016-09-20 14:07:02 +02:00
Tanguy Leroux 1a7fbf9679 [Tests] Fix cat.templates to work when other templates exist
This commit fixes the cat.templates REST tests so that it works when other templates exist (like monitoring)

Original commit: elastic/x-pack-elasticsearch@2e27ad88b4
2016-09-20 12:05:21 +02:00
Ryan Ernst 803f20e267 Merge pull request elastic/elasticsearch#3534 from rjernst/license_prod_key
Build: Use licensing prod key when building release

Original commit: elastic/x-pack-elasticsearch@682c4eace5
2016-09-19 15:48:43 -07:00
Ryan Ernst 39160b5b22 Build: Use licensing prod key when building release
This change switches the build to use the licensing prod key when
building the xpack jar for release.

Original commit: elastic/x-pack-elasticsearch@54a21dae5b
2016-09-19 15:46:04 -07:00
Chris Earle a94c27d3de [Monitoring] Future-proof Monitoring Bulk API with "interval" param
This adds an "interval" placeholder parameter that is required to the Monitoring Bulk API, and adds it to the Kibana side of the plumbing.

Having this will allow us to add it to all incoming documents and start to report against it with the Insights, as well as to detect the _lack_ of incoming documents.

By adding it now, we can avoid having a non-BWC API change for Kibana in 5.1. We'll just pickup new data in our documents.

Original commit: elastic/x-pack-elasticsearch@5ba8aafe03
2016-09-19 18:21:09 -04:00
Simon Willnauer 2dde85ab33 Unguice Transport and friends (elastic/elasticsearch#3510)
This is a followup for elastic/elasticsearchelastic/elasticsearch#20526 removes the pluggability of
transport / http server transport via guice.

Original commit: elastic/x-pack-elasticsearch@5fb84949aa
2016-09-19 22:11:17 +02:00
Ryan Ernst 2a03af9849 Build: Remove x-pack client jar
The x-pack client jar and api jar are exactly the same: the entirety of
x-pack. Since we added the x-pack-transport jar, we no longer really
want the client jar as it is confusing. Additionally, it causes jar hell
when a test for an extension uses the transport client. This change
removes the client jar, and makes the x-pack transport client use the
api jar instead. This sounds odd at first, but since transport client is
going away eventually, it is a stopgap, and works.

closes elastic/elasticsearch#3309

Original commit: elastic/x-pack-elasticsearch@ee7a2c12c0
2016-09-19 11:42:27 -07:00
Alexander Reelsen 8c51b3b21a Docs/Release notes: Removed `force` parameter in delete watch API
Also updated the rest test descriptions

Original commit: elastic/x-pack-elasticsearch@890ba7703e
2016-09-19 11:49:10 +02:00
Alexander Reelsen 273a9fb46f Watcher: Fix possible chained input NPE for execution failures (elastic/elasticsearch#3490)
Due to untested code there was an NPE happening in production,
when a chained input execution failed, but the chained input
tried to access the resulting payload (which is never set on
failures). This payload now defaults to being empty.

This commit also drive-by fixes a broken logging statement, that
on the one side returned not the watch id, but a useless watch
toString() representation, and on the other hand only logs an error
message, but not a stack trace into the log, as this is what the
history is for.

Original commit: elastic/x-pack-elasticsearch@7dbe1afd90
2016-09-19 10:06:02 +02:00
Alexander Reelsen 5b265ea569 Watcher: Remove locking of watches for write operations elastic/elasticsearch#3481 (elastic/elasticsearch#3481)
Whenever a watch is updated (put, delete, set state), until now we
happened to reject those operations when a watch was executed at the
same time. However with long running reporting this might mean, that a
watch can never be changed, because it always gets executed.

* Removes the ability of write requests to obtain a lock at all (executing watches is still protected by a lock)
* Replaced the FairKeyedLock in watcher with the KeyedLock in Elasticsearch, which also takes a fair option, removed the FairKeyedLock
* Removed all the timeout parameters that are no longer needed, because there is no lock anymore
* Removed also the force parameter for watch deletion. Just do it[tm]
* Added a test that deleting a watch while it is being executed does not leave any leftovers

In case of a deletion of a watch during an execution, so that updating the status of the watch fails,
a warning is logged.

Closes elastic/elasticsearch#3417

Original commit: elastic/x-pack-elasticsearch@22fad1b797
2016-09-19 09:44:32 +02:00
jaymode e069c1f090 test: stop IndexAuditTrail before ESIntegTestCase#after runs
As part of the review of elastic/elasticsearch#3287, the stopping of the IndexAuditTrail was moved to the tearDown
method. This works sometimes but other times it fails because tearDown is run after
ESIntegTestCase#after, so the IndexAuditTrail is still running during the after checks which will
cause the test to fail since the shard lock cannot be obtained.

Closes elastic/elasticsearch#3520

Original commit: elastic/x-pack-elasticsearch@4cb52b15a2
2016-09-17 07:38:31 -04:00
Simon Willnauer 7be765d2a0 Ensure we have a consistent view on OperationMode in Watcher (elastic/elasticsearch#3507)
Today the operation mode can be set to default for a short amout of
time until it's reset to the actual mode this can cause weird sideeffects
for users if it's read concurrently. Also the test relies on a certain
happens before relationship that is not guaranteed since the operation
mode is set before the listerner is run. This change also rewrites the test
to not use busy waiting but wait for the actual listern to be executed.

Original commit: elastic/x-pack-elasticsearch@a2a42b89e5
2016-09-16 22:10:51 +02:00
Nik Everett c21a922778 Add backwards compatibility support to monitoring
1. We only support indexes created by Marvel 2.3+. All other indexes
are just ignored.
2. The tests don't assert a ton of interesting stuff because there
isn't a java API for Monitoring that we can just use. Instead we assert
that a few objects are there and look sane.
3. We don't migrate the contents of the data index. Instead we just
rely on Monitoring recreating it.

Original commit: elastic/x-pack-elasticsearch@86216c2d61
2016-09-16 13:20:02 -04:00