This mainly moves packages over to the x-pack directory and renames the settings
from `xpack.watcher.actions.` to `xpack.notification.`
Moved services include pagerduty, hipchat, slack and email.
Closeselastic/elasticsearch#1998
Original commit: elastic/x-pack-elasticsearch@40c16fe123
Updated okhttp and moved the jsr305 dependency into testing.
This required a minor change in tests using SSL, as otherwise
the security manager barfs, when the okhttp webserver tries
to load sun internal SSL based classes.
Original commit: elastic/x-pack-elasticsearch@77131589e0
The old implementation was to use properties at build-time. This however did not work,
as the tests could not be run in the IDE. This has been removed of monitoring for some
time already, but needs to be removed from watcher as well.
This commit uses static variables and refactors the code a bit. First, there is a generic
TemplateUtils class, to be used in monitoring and watcher. Also the watcher code has been changed
to copy the needed variables into the template registry class instead of keeping it in the
WatcherModule.
This commit also includes some refactoring to remove the version parameter in marvel, was static anyway
Closeselastic/elasticsearch#1372
Original commit: elastic/x-pack-elasticsearch@fbfc22ea09
- introduced the "Feature Set" notion - graph, security, monitoring, watcher, these are all feature sets
- each feature set can be:
- `available` - indicates whether this feature set is available under the current license
- `enabled` - indicates whether this feature set is enabled (note that the feature set can be enabled, yet unavailable under the current license)
- while at it, cleaned up the main modules of watcher, security, monitoring and graph.
Original commit: elastic/x-pack-elasticsearch@5b3e19fe8c
* The found-license project is removed since it is no longer needed
* The plugin-api classes have moved into the license-plugin since there is only one plugin
* The license/base project publishes the proper artifactId in the pom file
* The x-pack jar file is added as an artifact so that it can be installed
* The x-pack pom no longer declares the packaging as `zip`
* The x-pack pom uses the right artifactId for license-core
* Removed disabling of installing the x-plugins artifacts
* Cleaned up a use of guava in watcher (found when trying to remove guava as a compile
dependency but is needed by the HTML sanitizer)
* Removed the dependency on the mustache compiler since it is no longer necessary
Closeselastic/elasticsearch#1987
Original commit: elastic/x-pack-elasticsearch@9d3b50b054
The license check in Watcher was issued in the wrong way,
so that new licenses were not affected by the check. This
commit explicitely lists the license types that are allowed
to execute watcher actions as well as fixing the tests.
Relates elastic/elasticsearch#1263
Original commit: elastic/x-pack-elasticsearch@afd55965b0
Our documentation states that we have default attributes for
message.format and message.color, which in fact we do not have
as an NPE was triggered in that case.
This commit falls back to unset defaults and allows for hipchat messages
to be sent without having to configure color/format in the action
or the account.
Closeselastic/elasticsearch#1666
Original commit: elastic/x-pack-elasticsearch@bfb7e35112
The HTTP response toXContent() method contains the http response headers, which
are used as field names in Elasticsearch in the watch history.
These can contain dots, like `es.index` being returned when Elasticsearch
encounters an exception - which results in an index error.
This patch changes the dots to an underscore when calling toXContent()
Closeselastic/elasticsearch#1803
Original commit: elastic/x-pack-elasticsearch@e4070f8b70
This commit adds reserved or built-in user and role support to x-pack. The reserved roles
cannot be modified by users. The reserved users also cannot be modified with the exception
of changing the password for a user.
In order to change the password for a user, a new API has been added. This API only supports
changing passwords for native and reserved users.
To support allowing a user to change their own password, a default role has been added to grant
access. This default role only grants access to user operations that pertain to the user that is
being authorized. In other words, the default role grants `joe` the ability to change their own password
but does not allow them to change the password of a different user.
Additionally, the authenticate API was made a transport action and is granted by the default role.
Closeselastic/elasticsearch#1727Closeselastic/elasticsearch#1185Closeselastic/elasticsearch#1158
Original commit: elastic/x-pack-elasticsearch@1a6689d90f
This commit migrates all of the `shield.` settings to `xpack.security.` and makes changes to
use the new Settings infrastructure in core.
As a cleanup, this commit also renames Shield to Security since this class is only in master
and will not affect 2.x.
See elastic/elasticsearch#1441
Original commit: elastic/x-pack-elasticsearch@a5a9798b1b
A ParseField object is now required to register queries against the SearchModule rather than the QueryParser#names method. ParseField handles camel case automatically. Also this allows us to log deprecation warnings (or fail in strict mode) when deprecated names are used for queries (e.g. "in", "mlt", "fuzzy_match" etc.)
Original commit: elastic/x-pack-elasticsearch@b0146e6e3d
This adds default timeouts to the SMTP configuration to prevent infinite timeouts, that can lead to stuck watches.
This also requires to use time values instead of just milliseconds.
Closeselastic/elasticsearch#1830
Original commit: elastic/x-pack-elasticsearch@c886da7bff
Too many places are checking for enumerations when they're really more interested in a "higher" level of
information. This will help with the forthcoming addition of the STANDARD operation mode as well.
Original commit: elastic/x-pack-elasticsearch@2799c27e19
This commit is the forward port of renaming the type for esusers to file. There is no
backwards compatibility maintained here. Additionally, a few other renames and
cleanups have been made:
* `esusers` commands is now `users`
* org.elasticsearch.shield.authc.esusers -> org.elasticsearch.shield.authc.file
* Validation.ESUsers -> Validation.Users
* ESUsersTool -> UsersTool
* ESUsersToolTests -> UsersToolTests
* ESNativeUsersStore -> NativeUsersStore
* ESNativeRolesStore -> NativeRolesStore.
* org.elasticsearch.shield.authz.esnative collapsed to org.elasticsearch.shield.authz.store
* ESNativeTests -> NativeRealmIntegTests
Closeselastic/elasticsearch#1793
Original commit: elastic/x-pack-elasticsearch@d2a0c136f3
These privileges no longer need to be defined as a custom privilege since the
code is now consolidated into a single plugin. This also changes the manage
cluster privilege to be an alias to the all privilege.
Original commit: elastic/x-pack-elasticsearch@a7f444c898
This commit remove the pre-existing file parsing code and replaces it with the updated
code in the RoleDescriptor class. This unifies the parsing for the files and API for roles.
Closeselastic/elasticsearch#1596
Original commit: elastic/x-pack-elasticsearch@9e0b58fcf1
As part of the search refactoring effort, we need to pass a Suggester
to the methods that parse X-Content to a SuggestBuilder in every
instance where we are parsing search/suggest requests.
Original commit: elastic/x-pack-elasticsearch@7f815c617a
The reported time waiting for watches can be slightly different from the actual timeout (e.g. 2.1 seconds instead of 2 seconds) so checking the time waited in the message makes the test sometimes fail
Original commit: elastic/x-pack-elasticsearch@c2cd9da486
Adds a check to the settings at startup to ensure that the security and audit indices are
allowed to be auto created if a user has disabled auto create explicitly.
Additionally fixes a small issue with the error message for watcher passing the incorrect
value.
Closeselastic/elasticsearch#1453
Original commit: elastic/x-pack-elasticsearch@2b0698ff19
Today Watcher tries to load stuff from the bin's parent directory which
is not readable since the shared data directory has been moved out of
the nodes parent in elasticsearchelastic/elasticsearch#17072 which causes security exception
now. The test copies trust stores into the config dir and that's where
we should read it from by default or even better explicitly configure the path?!
Original commit: elastic/x-pack-elasticsearch@1d32a595cf
Also make logging message String constant to allow static checks
Relates to elastic/elasticsearchelastic/elasticsearch#16707
Original commit: elastic/x-pack-elasticsearch@b5bd423de4
This commit moves the InitializingModule and InitializingService classes in the common XPack package so that it can be used by any plugin. It also renames the module and service from Initializing* to LazyInitializing* and add a ClientProxy class.
Original commit: elastic/x-pack-elasticsearch@fbdf9d1614
Deep down buried there is an option to not only allow global
script execution, but also allow a single scripting language for
watcher only. It is time to document it as well.
Renamed this option to `script.engine.groovy.inline.xpack_watch`
to align with xpack renaming.
Closeselastic/elasticsearch#1422
Original commit: elastic/x-pack-elasticsearch@845eb5a0c0
Fix to ensure that the email attachment has a correctly set filename, which is
also now explained in the documentation.
In addition there is a check now for email attachments, that a filename can only
be specified once, otherwise an exception is thrown.
Closeselastic/elasticsearch#1503
Original commit: elastic/x-pack-elasticsearch@2a399058b3
In case that a single email attachment cannot be downloaded, this ensures
that the whole action fails with a correct Action.Failure.
This also fixes an NPE that would occur otherwise.
Original commit: elastic/x-pack-elasticsearch@7bb042a719
This is a small fix to use specified id when sending data attachments.
The current solution always used "data".
Also a minor refactoring was made to include get the different parser impls
from the EmailAttachmentsParser instead of specifying them twice in the
EmailAction.
Closeselastic/elasticsearch#1503
Original commit: elastic/x-pack-elasticsearch@9354e83c8b
The awaiting latch was not waiting as long as the sleep in the code
causing the latch to fail and the test to fail.
This code aligns the time to wait for the latch and the sleep code
in the mock http server.
Original commit: elastic/x-pack-elasticsearch@8a2cc61204
The request timeout and the real time the webserver slept was 5000ms.
In case of loaded systems, there might be cases, where the request was
still received in time.
This commit increases the server side sleep time to 10 seconds, to ensure
that the client aborts the request early
Original commit: elastic/x-pack-elasticsearch@718c05519f
The current HTTP timeout tests had two problems.
* Binding to port 9200-9300
* The first request to hit was having a delay, the other ones had not,
so if any other component hit the test inbetween (likely in a CI env),
the HTTP request from the test itself will not be delayed.
Both cases are fixed in this commit.
Original commit: elastic/x-pack-elasticsearch@d696e020cc
When an HTTP input returns an error body, right now we check if the
error code is below 400 and only then we include the body.
However using another method from URLConnection, the body can be
access always.
Closeselastic/elasticsearch#1550
Original commit: elastic/x-pack-elasticsearch@1743fd0a77
- `full_name` and `email` are optional user fields
- `metadata` is an optional arbitrary meta data that can be associated with the user
- cleaned up the user actions - consistent naming (e.g. `PutUserAction` vs. `AddUserAction`)
- moved source parsing from the `PutUserRequest` to the `PutUserRequestBuilder`
- renamed`WatcherXContentUtils` to `XContentUtils` and moved it to sit under `o.e.xpack.commons.xcontent`
Closeselastic/elasticsearch#412
Original commit: elastic/x-pack-elasticsearch@5460e3caf7
We shouldn't have marvel enabled for these tests because we get false test failures
due to marvel indices existing and failing to lock the shard.
Original commit: elastic/x-pack-elasticsearch@11123bb660
This removes the use of group setting for `shield.` and introduces some individual settings
and some group settings that should not overlap and cause issues when iteration order
changes.
See elastic/elasticsearch#1520
Original commit: elastic/x-pack-elasticsearch@193e937193
- Moved all settings in Marvel from `marvel.*` to `xpack.monitoring.*`
- Cleaned up marvel settings in general - they're all now under `MarvelSettings` class
- fixed some integration tests along the way (they were configured wrong and never actually tested anything)
- Updated the docs accordingly
- Added `migration-5_0.asciidoc` under the Marvel docs to explain how to migrate from Marvel 2.x to XPack 5.0.
- Replaced all `marvel` mentions in the logs to `monitoring`
- Removed the `xpack.monitoring.template.version` setting from the templates
- renamed the templates to `monitoring-es-data.json` and `monitoring-es.json`
- monitoring indices are now `.monitoring-es-<version>-data` and `.monitoring-es-<version>-<timestamp>`
Original commit: elastic/x-pack-elasticsearch@17f2abe17d
- renaming `ShieldPlugin` to `Shield` (it's no longer a plugin)
- renaming `WatcherPlugin` to `Watcher` (it's no longer a plugin)
- renaming `MarvelPlugin` to `Marvel` (it's no longer a plugin)
- renaming `LicensePlugin` to `Licensing` (it's no longer a plugin)
- renamed setting:`watcher.enabled` -> `xpack.watcher.enabled`
- renamed setting:`marvel.enabled` -> `xpack.marvel.enabled`
Original commit: elastic/x-pack-elasticsearch@35a6540b11
- Consolidated the `bin` and `config` directories of watcher, shield and marvel under a single `config/xpack` and `bin/xpack` directories.
- updated docs accordingly
Original commit: elastic/x-pack-elasticsearch@c2aa6132fa
- Started to move configuration under the `xpack` name
- Cleaned up `ShieldPlugin`
- renamed `ShieldClient` to `SecurityClient`
- Introduced `XPackClient` that wraps security and watcher clients
Original commit: elastic/x-pack-elasticsearch@f05be0c180
This loads an index template for the watch history to make sure,
that field changes are taken into account.
Also, the dynamic mapping for the watch history template has been
changed from strict to false.
This means that new fields can be included in a document, but they
will not indexed and are not searchable.
In addition the index names have been changed from .watch_history-$date to
.watcher-history-$template-$date - using dashes to be more consistent.
Closeselastic/elasticsearch#1299
Original commit: elastic/x-pack-elasticsearch@794f982234
This commit fixes the bad apple tests that failed when running them. The
IndexAuditTrailEnabledTest was removed and the test was folded into the
IndexAuditIT. Some watcher tests that relied on mustache were moved
into the QA tests with the mustache plugin.
Additionally, fixing these tests uncovered a issue with the privileges needed
for writing data into an index. If the mappings need to be updated because
of a write, then the update mapping action gets executed. In 2.x this was
handled by the system user, but now is executed under the user's context,
which is the correct thing to do. The update mapping action is now added to
the read, index, crud, and write privileges for an index.
Original commit: elastic/x-pack-elasticsearch@30711f9625
In elastic/elasticsearch#1442 checkstyle checks were added, but also some files were freed from this.
If we have support for checkstyle, we should check this for all files and not allow
exceptions. This commit removes the file list to ignore any files and fixes all the
java files.
Original commit: elastic/x-pack-elasticsearch@99e6cbc5be
According to RFC 2616 HTTP headers are case insensitive.
But `HttpResponse#contentType()` only looks up for Content-Type.
This stores all header responses lower cased in the HTTP response.
Closeselastic/elasticsearch#1357
Original commit: elastic/x-pack-elasticsearch@c009be8365
This change registers all filtered settings up-front and removes all
the unnecessary wrappers around SettingsFilter. This is a pretty big
change and needs some review but after all things are generally simplified and
settings are always filtered even if shield is not enabled which is the right thing
todo.
Relates to elastic/elasticsearchelastic/elasticsearch#16425
Original commit: elastic/x-pack-elasticsearch@c7df85492b
The lang-mustache module has been extended to meet Watcher's needs:
* The ability to refer the specific slots in arrays.
* An `content_type` option controls whether json string escaping is used. Otherwise there is no escaping.
Closeselastic/elasticsearch#1116
Other changes:
* I changed tests that were just using mustache just because it was around to not use mustache
* I moved tests to `test-xpack-with-mustache` module that were testing mustache with Watcher
* added smoke test for watcher and mustache
* moved some tests around
* instead of using DefaultTextTemplateEngine in watcher tests use MockTextTemplateEngine
* added a mock mustache script engine
* Cleanup some messy tests to not rely on mustache and move them back into xpack module
* moved array access test to smoke test watcher with mustache module
* test: simplified the condition search test to take the time component out of it, while still simulation a condition
* removed the mustache dependency in the messy-test-watcher-with-groovy module
Original commit: elastic/x-pack-elasticsearch@6a2a4e885f
- Consolidated `InternalMarvelUser`, `InternalWatcherUser` and `InternalShieldUser` into a single `XPackUser` - this is the single internal user for xpack that has all the permissions internally required by xpack (for marvel, watcher and shield)
- Renamed `InternalSystemUser` to `SystemUser`
- Removed the notion of "reserved roles". Now that we have a single internal user we know its role. The authz service now checks to see if the current user is the internal xpack user, and if so, it just uses its role (and not trying to resolve it from the role store). With this model, it's no longer possible for outside users to use the internal role (it's fully internal)
- Consolidated the notion of an `InternalClient` (in Marvel it was knows as the `SecuredClient`). This is an ES client that xpack is using to manage itself. If shield is enabled, it will execute all request on behalf of the internal xpack user.
- Removed the verification of the license plugin on plugin installation - no need to do it anymore as the license plugin is part of the distribution.
Original commit: elastic/x-pack-elasticsearch@c851410f93
When using a path like `"/<logstash-{now%2Fd}>/_search"` in the
http webhook. The already escaped slash (%2F) got escaped twice
and thus did not work any more.
The escaping happened when the code created an URI and was done
as part of that constructor. This is now switched to an URL (which
is used at the end anyway) which does not do the escaping, even though
this was required for the query string, which is now done when constructing.
Closeselastic/elasticsearch#1364
Original commit: elastic/x-pack-elasticsearch@861b6d2378
This change migrates all of the xpack code to use the new ThreadContext when
dealing with headers and context data. For the most part this is a simple
cutover, but there are some things that required special casing. The internal
actions that executed by a user's requests need to forcefully drop the context
and set the system user. The workaround for this will be improved in a followup.
Additionally, the RequestContext still lives on due to the OptOutQueryCache,
which requires some core changes to fix this issue.
Original commit: elastic/x-pack-elasticsearch@87d2966d93
This commit is the x-plugins side of the refactoring of script settings.
Relates elastic/elasticsearchelastic/elasticsearch#16197
Original commit: elastic/x-pack-elasticsearch@4c429933b9
As of elastic/elasticsearchelastic/elasticsearch#16054 all index level settings
must be registered and use the new settings infrastructure. This commit
prepares for the merge to provide a smooth transition.
Original commit: elastic/x-pack-elasticsearch@bc0a4fec07
In order to correctly check for equality in an EmailAction, all the email
attachments have to implement equals/hashCode methods.
This has already been added to the 2.x branch in elastic/x-pack@ebde22507f
Original commit: elastic/x-pack-elasticsearch@bb980ea934
This feature is mainly done for the integration with the commercial reporting, but can be used
for anything else as well.
This adds a `attachments` to the email configuration, which can be used like this
```
"attachments" : {
"some_id" : {
"http" : {
"request" : {
"url" : "http://example.org/foo.pdf"
}
}
},
"other_id" : {
"data" : {
"format" : "json"
}
}
}
```
The main reason to pick this format is extensibility. If we would like to support another
attachment type, like an file reader, we could do so easily from an API point of view.
Closeselastic/elasticsearch#870
Original commit: elastic/x-pack-elasticsearch@66d14be965
In order to have a shortcut for the execution of a watch and
specifying the record_execution and ignore_condition booleans,
so are now supported in the HTTP request parameters as well.
Closeselastic/elasticsearch#918
Original commit: elastic/x-pack-elasticsearch@bed5da40b7
* This action enables sending notifications to pager duty services.
* Utilizes pager duty's REST API
* Similar to the `email`, `hipchat` and `slack` actions, multiple `pagerduty` accounts can be configured, each with its own Service API key
* A `pagerduty` account is roughly mapped to a service in your pagerduty service.
* `pagerduty` actions are associated with an account, or if not, their events will be sent via the default account.
* An incident can be acknowledged, resolved or triggered
Closeselastic/elasticsearch#492
Original commit: elastic/x-pack-elasticsearch@72cc21d119
Instead of having to specify `scheme`, `host`, `port`, `path` and `params` each
one can now use the `url` field and specify something like `http://example.org/foo?bar=baz`
This works for HttpRequest and HttpRequestTemplate
Original commit: elastic/x-pack-elasticsearch@8c052cf419
Because of specifying the concrete implementations in WatcherPlugin.nodeServices(),
all of those services got created twice for each node (guice requires the interface here).
This resulted in NPEs as the first instantiation did not run doStart() and thus had empty
settings, that resulted in a NPE.
Closeselastic/elasticsearch#1179
Original commit: elastic/x-pack-elasticsearch@c6a1f5093f
Alexander Reelsen
Martijn van Groningen
uboness
Daniel Mitterdorfer
Ryan Ernst
Christoph Büscher
jaymode
Nik Everett
Adrien Grand
javanna
Tanguy Leroux
Chris Earle
Jim Ferenczi
Boaz Leskes
Simon Willnauer
Ali Beyad
Colin Goodheart-Smithe
Yannick Welsch
David Pilato
Robert Muir
Jason Tedor
markharwood
Lee Hinman
Igor Motov