Commit Graph

3936 Commits

Author SHA1 Message Date
Alexander Reelsen 53103e988f Watcher: Add proxy support to pagerduty action (elastic/elasticsearch#3542)
This is the last action that needs additional support for proxies.

You can set a proxy in the JSON like this:

```
"actions" : {
  "notify-pagerduty" : {
    "pagerduty" : {
      "description" : "Main system down, please check!",
      "proxy" : { "host" : "localhost", "port" : 8080 }
    }
  }
}
```

Closes elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@b99969fd6b
2016-10-05 10:10:02 +02:00
Alexander Reelsen 7ffebef2cd Watcher: Add proxy support to slack action (elastic/elasticsearch#3487)
You can set it like this in the JSON

"actions" : {
  "notify-slack" : {
    "slack" : {
      "account" : "integration-account",
      "proxy" : {
        "host" : "localhost",
        "port" : 8080
      },
      "message" : {
        ...
      }
    }
  }
}

Relates elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@de86233d4f
2016-10-05 09:07:09 +02:00
Jason Tedor 00cecac86e Change Watcher thread pool to be scaling
Watcher uses a custom thread pool. This is because executing watches can
be long-running tasks that often block on I/O and it is best to not
consume the core thread pools with these tasks. Today this thread pool
is fixed, and sized at five times the bounded number of cores (so 160 on
a 32-core box). It makes sense for there to possibly be so many threads,
again because these tasks can block on I/O and having excess capacity
lets unblocked watches execute. It's the fixed size that can cause
problem, all these threads are always consuming resources even when
there are no or not that many watches running. This commit changes this
thread pool to be a scaling thread pool.

Relates elastic/elasticsearch#3660

Original commit: elastic/x-pack-elasticsearch@3cafab6e83
2016-10-04 18:15:19 -04:00
Jason Tedor a0e1d44a44 Remove lenient URL parameter parsing
This commit adapts x-plugins for a change in core Elasticsearch that
removes lenient URL parameter parsing.

Relates elastic/elasticsearch#3641

Original commit: elastic/x-pack-elasticsearch@cc0687f32c
2016-10-04 12:46:54 -04:00
jaymode ddae0694c9 test: move SecurityTribeIT to right directory
Relates elastic/elasticsearch#3635

Original commit: elastic/x-pack-elasticsearch@b46ab0b63e
2016-10-04 09:22:21 -04:00
Luca Cavanna 91a68e9873 adapt to IndicesAliasesRequest not implementing CompositeIndicesRequest (elastic/elasticsearch#3645)
We need to special case IndicesAliasesRequest as it doesn't implement CompositeIndicesRequest anymore. Note that the similar loop for CompositeIndicesRequests's subrequests will soon go away

Relates to elastic/elasticsearch#3638

Original commit: elastic/x-pack-elasticsearch@50d119ff61
2016-10-04 10:39:31 +02:00
Ryan Ernst 36c7070217 Fix xpack api jar artifact naming
Original commit: elastic/x-pack-elasticsearch@bb7b0a6392
2016-10-03 19:25:44 -07:00
Ryan Ernst fa4d389c99 Fix artifact id for x-pack api jar
Original commit: elastic/x-pack-elasticsearch@26e47099a2
2016-10-03 16:38:25 -07:00
Ryan Ernst 1911e5da02 Build: Fix artifact id for xpack transport client
Original commit: elastic/x-pack-elasticsearch@2658d3f087
2016-10-03 12:37:45 -07:00
Ryan Ernst ee9dbac0c7 Update conditional version from alpha6 to beta1 in http exporter
Original commit: elastic/x-pack-elasticsearch@fdcef033b9
2016-10-03 09:37:12 -07:00
Ryan Ernst 21522c8538 Merge pull request elastic/elasticsearch#3635 from rjernst/reorg
Reorganize src roots

Original commit: elastic/x-pack-elasticsearch@ac87e6c257
2016-10-03 08:39:25 -07:00
Ryan Ernst 947b3f26e6 Merge branch 'master' into reorg
Original commit: elastic/x-pack-elasticsearch@f279791e12
2016-10-03 08:36:39 -07:00
jaymode abc789c887 test: handle case where security index only exists in one tribe
Original commit: elastic/x-pack-elasticsearch@94f011387f
2016-10-03 11:27:15 -04:00
Jay Modi 52b7170121 security: native users and roles can be used on tribe nodes
This change allows native users and roles to be used on tribe nodes. The tribe node will actually
use the security index of one of the tribes, which must be specified with the `tribe.on_conflict`
setting. User and role modifications are not permitted when running on a tribe node.

Closes elastic/elasticsearch#3451

Original commit: elastic/x-pack-elasticsearch@2b762ca648
2016-10-03 10:12:30 -04:00
Ryan Ernst c309faec72 Merge branch 'master' into reorg
Original commit: elastic/x-pack-elasticsearch@24575f52a9
2016-10-01 09:49:14 +02:00
Ryan Ernst a51107769e Merge branch 'master' of github.com:elastic/x-plugins
Original commit: elastic/x-pack-elasticsearch@5ab54f94aa
2016-10-01 09:49:01 +02:00
Ryan Ernst 1fa0f835fe Build: Reorganize src roots
This change flattens the directory structure, both for the elasticsearch
specific directories, as well as within the elasticsearch x-pack plugin.

closes elastic/elasticsearch#2957

Original commit: elastic/x-pack-elasticsearch@45891a4632
2016-10-01 09:46:43 +02:00
Tyler Smalley c7b7a9f201 Merge pull request elastic/elasticsearch#3627 from tylersmalley/security-key
Use 32 character encryptionKeys

Original commit: elastic/x-pack-elasticsearch@ac21c8c461
2016-09-30 15:57:35 +02:00
Alexander Reelsen abe5f1dda9 Watcher: Ignore watch version when deleting a watch (elastic/elasticsearch#3630)
When deleting a watch the version was used as part of
the delete request. However a watch deletion means the
user really wants to get rid of it and not accidentally
run into a version exception because the watch was running in
between.

Original commit: elastic/x-pack-elasticsearch@e585f717f1
2016-09-30 15:45:05 +02:00
Ryan Ernst 1cca04924f Collapsed monitoring into xpack src
Original commit: elastic/x-pack-elasticsearch@9e523f2783
2016-09-29 12:25:57 +02:00
Ryan Ernst 98687d2884 Collapsed license code into xpack src
Original commit: elastic/x-pack-elasticsearch@42f7cb5a08
2016-09-29 12:22:19 +02:00
Ryan Ernst 1b55cc7c62 collapsed graph into xpack src
Original commit: elastic/x-pack-elasticsearch@de8085baed
2016-09-29 12:16:32 +02:00
Ryan Ernst 905237a56f Moved directories around
Original commit: elastic/x-pack-elasticsearch@2018bb5f9f
2016-09-29 12:03:14 +02:00
Ryan Ernst 09bcd8e5cd Merge pull request elastic/elasticsearch#3618 from rjernst/consolidate_dev_tools
Build: Conslidate dev-tools and x-dev-tools

Original commit: elastic/x-pack-elasticsearch@ff59116ea8
2016-09-29 11:29:10 +02:00
Tim Sullivan 91e5b07e79 Merge pull request elastic/elasticsearch#3615 from tsullivan/reporting-fix-hasownproperty
reporting: fix request.query.hasOwnProp => has()

Original commit: elastic/x-pack-elasticsearch@d757785bc3
2016-09-28 03:52:57 +02:00
Ryan Ernst 0a3940103a Build: Conslidate dev-tools and x-dev-tools
In preparation for elastic/elasticsearch#2957, I found we have things both in the root level
dev-tools, as well as elasticsearch/x-dev-tools. Most of this stuff can
be removed as it had to do with the old manual release process. There
was also a signed license file checked in. I removed it here, we really
should not have licenses checked in IMO, and it is unclear what the
purpose of this license was for. The two remaining scripts were moved to
the root dev-tools.

Original commit: elastic/x-pack-elasticsearch@3e24ea2d56
2016-09-27 16:46:51 +02:00
Igor Motov aafc75ac51 Fix serialization issue in WatcherMetaData
Watcher meta data parser doesn't read the closing '}' which causes all following custom metadata to be ignored.

Similar to elastic/elasticsearch#1190

Original commit: elastic/x-pack-elasticsearch@d15b9ea466
2016-09-27 14:15:16 +02:00
Jay Modi d44ba28d27 security: always create the IPFilter in a node
When running as a node, we check the `xpack.security.transport.filter.enabled` setting to see
if we should create the IPFilter but this check is not really correct. The HTTP filter could be
enabled or a profile filter could be enabled so there are times when we may not be filtering connections
when we should. Additionally, since we do not bind the IPFilter to a null provider, Guice will try to create
one during startup to inject into the security transport. This results in an exception and startup fails.

This change always creates the IPFilter when running as a node. This IPFilter has its own settings and
logic to determine whether it should be filtering on a given network transport.

Closes elastic/elasticsearch#3592

Original commit: elastic/x-pack-elasticsearch@95c25651c4
2016-09-23 10:12:24 -04:00
Alexander Reelsen 7557168a0a Smoke tester: Adapt to new download structure (elastic/elasticsearch#3575)
This makes the smoke tester in x-pack work again after the work
on the unified release.

Original commit: elastic/x-pack-elasticsearch@4d4f1ec26c
2016-09-23 09:53:17 +02:00
Simon Willnauer 3c650e483b Followup from elastic/elasticsearchelastic/elasticsearch#20627 - Removal of AnalysisService
Original commit: elastic/x-pack-elasticsearch@75c14534a7
2016-09-23 08:54:44 +02:00
Tanguy Leroux 375bf95fb1 Remove duplicate methods in ByteSizeValue (elastic/elasticsearch#20560)
Some methods have been renamed in elastic/elasticsearchelastic/elasticsearch#20560. This commit change a .bytes() call to a .getBytes() call.

Original commit: elastic/x-pack-elasticsearch@4a0ff77361
2016-09-20 14:07:02 +02:00
Tanguy Leroux 1a7fbf9679 [Tests] Fix cat.templates to work when other templates exist
This commit fixes the cat.templates REST tests so that it works when other templates exist (like monitoring)

Original commit: elastic/x-pack-elasticsearch@2e27ad88b4
2016-09-20 12:05:21 +02:00
Ryan Ernst 803f20e267 Merge pull request elastic/elasticsearch#3534 from rjernst/license_prod_key
Build: Use licensing prod key when building release

Original commit: elastic/x-pack-elasticsearch@682c4eace5
2016-09-19 15:48:43 -07:00
Ryan Ernst 39160b5b22 Build: Use licensing prod key when building release
This change switches the build to use the licensing prod key when
building the xpack jar for release.

Original commit: elastic/x-pack-elasticsearch@54a21dae5b
2016-09-19 15:46:04 -07:00
Chris Earle a94c27d3de [Monitoring] Future-proof Monitoring Bulk API with "interval" param
This adds an "interval" placeholder parameter that is required to the Monitoring Bulk API, and adds it to the Kibana side of the plumbing.

Having this will allow us to add it to all incoming documents and start to report against it with the Insights, as well as to detect the _lack_ of incoming documents.

By adding it now, we can avoid having a non-BWC API change for Kibana in 5.1. We'll just pickup new data in our documents.

Original commit: elastic/x-pack-elasticsearch@5ba8aafe03
2016-09-19 18:21:09 -04:00
Simon Willnauer 2dde85ab33 Unguice Transport and friends (elastic/elasticsearch#3510)
This is a followup for elastic/elasticsearchelastic/elasticsearch#20526 removes the pluggability of
transport / http server transport via guice.

Original commit: elastic/x-pack-elasticsearch@5fb84949aa
2016-09-19 22:11:17 +02:00
Ryan Ernst 3bfebc75c1 Merge pull request elastic/elasticsearch#3527 from rjernst/lose_one_xpack_jar
Build: Remove x-pack client jar

Original commit: elastic/x-pack-elasticsearch@fb6b2eccf5
2016-09-19 11:48:36 -07:00
Ryan Ernst 2a03af9849 Build: Remove x-pack client jar
The x-pack client jar and api jar are exactly the same: the entirety of
x-pack. Since we added the x-pack-transport jar, we no longer really
want the client jar as it is confusing. Additionally, it causes jar hell
when a test for an extension uses the transport client. This change
removes the client jar, and makes the x-pack transport client use the
api jar instead. This sounds odd at first, but since transport client is
going away eventually, it is a stopgap, and works.

closes elastic/elasticsearch#3309

Original commit: elastic/x-pack-elasticsearch@ee7a2c12c0
2016-09-19 11:42:27 -07:00
Alexander Reelsen 8c51b3b21a Docs/Release notes: Removed `force` parameter in delete watch API
Also updated the rest test descriptions

Original commit: elastic/x-pack-elasticsearch@890ba7703e
2016-09-19 11:49:10 +02:00
Alexander Reelsen 273a9fb46f Watcher: Fix possible chained input NPE for execution failures (elastic/elasticsearch#3490)
Due to untested code there was an NPE happening in production,
when a chained input execution failed, but the chained input
tried to access the resulting payload (which is never set on
failures). This payload now defaults to being empty.

This commit also drive-by fixes a broken logging statement, that
on the one side returned not the watch id, but a useless watch
toString() representation, and on the other hand only logs an error
message, but not a stack trace into the log, as this is what the
history is for.

Original commit: elastic/x-pack-elasticsearch@7dbe1afd90
2016-09-19 10:06:02 +02:00
Alexander Reelsen 5b265ea569 Watcher: Remove locking of watches for write operations elastic/elasticsearch#3481 (elastic/elasticsearch#3481)
Whenever a watch is updated (put, delete, set state), until now we
happened to reject those operations when a watch was executed at the
same time. However with long running reporting this might mean, that a
watch can never be changed, because it always gets executed.

* Removes the ability of write requests to obtain a lock at all (executing watches is still protected by a lock)
* Replaced the FairKeyedLock in watcher with the KeyedLock in Elasticsearch, which also takes a fair option, removed the FairKeyedLock
* Removed all the timeout parameters that are no longer needed, because there is no lock anymore
* Removed also the force parameter for watch deletion. Just do it[tm]
* Added a test that deleting a watch while it is being executed does not leave any leftovers

In case of a deletion of a watch during an execution, so that updating the status of the watch fails,
a warning is logged.

Closes elastic/elasticsearch#3417

Original commit: elastic/x-pack-elasticsearch@22fad1b797
2016-09-19 09:44:32 +02:00
jaymode e069c1f090 test: stop IndexAuditTrail before ESIntegTestCase#after runs
As part of the review of elastic/elasticsearch#3287, the stopping of the IndexAuditTrail was moved to the tearDown
method. This works sometimes but other times it fails because tearDown is run after
ESIntegTestCase#after, so the IndexAuditTrail is still running during the after checks which will
cause the test to fail since the shard lock cannot be obtained.

Closes elastic/elasticsearch#3520

Original commit: elastic/x-pack-elasticsearch@4cb52b15a2
2016-09-17 07:38:31 -04:00
Simon Willnauer 7be765d2a0 Ensure we have a consistent view on OperationMode in Watcher (elastic/elasticsearch#3507)
Today the operation mode can be set to default for a short amout of
time until it's reset to the actual mode this can cause weird sideeffects
for users if it's read concurrently. Also the test relies on a certain
happens before relationship that is not guaranteed since the operation
mode is set before the listerner is run. This change also rewrites the test
to not use busy waiting but wait for the actual listern to be executed.

Original commit: elastic/x-pack-elasticsearch@a2a42b89e5
2016-09-16 22:10:51 +02:00
Nik Everett c21a922778 Add backwards compatibility support to monitoring
1. We only support indexes created by Marvel 2.3+. All other indexes
are just ignored.
2. The tests don't assert a ton of interesting stuff because there
isn't a java API for Monitoring that we can just use. Instead we assert
that a few objects are there and look sane.
3. We don't migrate the contents of the data index. Instead we just
rely on Monitoring recreating it.

Original commit: elastic/x-pack-elasticsearch@86216c2d61
2016-09-16 13:20:02 -04:00
Jay Modi a6d55f26c6 security: simplify index audit trail stopping
The IndexAuditTrail had both a stop and close method that needed to be called in order
to stop the service. There was a race where we called either flush or close in a non
blocking fashion and then immediately closed the underlying client. This change makes
the stop method wait for up to 10 seconds when closing the bulk processor.

Closes elastic/elasticsearch#3279

Original commit: elastic/x-pack-elasticsearch@0d776bc91a
2016-09-16 10:31:27 -04:00
Simon Willnauer efeb9cefce Cut over SecurityServerTransportService to use the new Interceptor infrastructure (elastic/elasticsearch#3491)
TransportService is not pluggable anymore in core. Instead we now have a interceptor
infrastructure that allows to intercept send and receive calls on the transport layer.

Relates to elastic/elasticsearchelastic/elasticsearch#20505

Original commit: elastic/x-pack-elasticsearch@04194ecb09
2016-09-16 09:48:25 +02:00
Boaz Leskes 635b5a6800 fix import for ElectMasterService
it changed with https://github.com/elastic/elasticsearch/pull/20384

Original commit: elastic/x-pack-elasticsearch@c1e51de6a8
2016-09-15 23:45:24 +02:00
Alexander Reelsen efa2678691 Watcher: Add proxy support to hipchat action (elastic/elasticsearch#3475)
This adds proxy support to the hipchat action. Right now
neither hipchat nor slack nor pagerduty allow for this,
but if you dont need a proxy for internal http connections,
but you do for external, then this configuration cannot be done
without setting a proxy for those actions.

You can set it like this in the JSON

```
"actions" : {
  "notify-hipchat" : {
    "hipchat" : {
      "account" : "integration-account",
      "proxy" : {
        "host" : "localhost",
        "port" : 8080
      },
      "message" : {
        ...
      }
    }
  }
}

```

Relates elastic/elasticsearch#3372

Original commit: elastic/x-pack-elasticsearch@4e8447ce37
2016-09-15 11:24:04 +02:00
Yannick Welsch d35131ad52 Fix wrong logger usages (elastic/elasticsearch#3485)
Relates to elastic/elasticsearchelastic/elasticsearch#20490

Original commit: elastic/x-pack-elasticsearch@1ae7217b59
2016-09-15 10:48:06 +02:00
Jason Tedor b534bfec5a Complete Elasticsearch logger names
This commit modifies the loggers used within x-pack to adapt to the
change in core Elasticsearch where loggers are now named by the
fully-qualified class name.

Relates elastic/elasticsearch#3465

Original commit: elastic/x-pack-elasticsearch@4cac9ac62f
2016-09-13 22:47:10 -04:00