This loads an index template for the watch history to make sure,
that field changes are taken into account.
Also, the dynamic mapping for the watch history template has been
changed from strict to false.
This means that new fields can be included in a document, but they
will not indexed and are not searchable.
In addition the index names have been changed from .watch_history-$date to
.watcher-history-$template-$date - using dashes to be more consistent.
Closeselastic/elasticsearch#1299
Original commit: elastic/x-pack-elasticsearch@794f982234
This commit fixes the bad apple tests that failed when running them. The
IndexAuditTrailEnabledTest was removed and the test was folded into the
IndexAuditIT. Some watcher tests that relied on mustache were moved
into the QA tests with the mustache plugin.
Additionally, fixing these tests uncovered a issue with the privileges needed
for writing data into an index. If the mappings need to be updated because
of a write, then the update mapping action gets executed. In 2.x this was
handled by the system user, but now is executed under the user's context,
which is the correct thing to do. The update mapping action is now added to
the read, index, crud, and write privileges for an index.
Original commit: elastic/x-pack-elasticsearch@30711f9625
In elastic/elasticsearch#1442 checkstyle checks were added, but also some files were freed from this.
If we have support for checkstyle, we should check this for all files and not allow
exceptions. This commit removes the file list to ignore any files and fixes all the
java files.
Original commit: elastic/x-pack-elasticsearch@99e6cbc5be
According to RFC 2616 HTTP headers are case insensitive.
But `HttpResponse#contentType()` only looks up for Content-Type.
This stores all header responses lower cased in the HTTP response.
Closeselastic/elasticsearch#1357
Original commit: elastic/x-pack-elasticsearch@c009be8365
This commit adds a new `source_node` field to all marvel documents that holds various information (node's name/ip/host/id/transport address) about the node that emitted the document.
(cherry picked from commit elastic/x-pack@29a411a931)
Original commit: elastic/x-pack-elasticsearch@66e057d334
If elasticsearch merges https://github.com/elastic/elasticsearch/pull/16413
then the build will fail catastrophically without this. The goal here is to
opt these files out of the line length checks while they don't pass and we
can get them passing as time permits and opt them back in. In the mean time
all files that pass the line length check will have the check enforced.
This also gives you a spot to add x-plugins opt outs for core's checkstyle
rules in case you have generated files or something like that.
Original commit: elastic/x-pack-elasticsearch@63a1ad2f79
This change registers all filtered settings up-front and removes all
the unnecessary wrappers around SettingsFilter. This is a pretty big
change and needs some review but after all things are generally simplified and
settings are always filtered even if shield is not enabled which is the right thing
todo.
Relates to elastic/elasticsearchelastic/elasticsearch#16425
Original commit: elastic/x-pack-elasticsearch@c7df85492b
The lang-mustache module has been extended to meet Watcher's needs:
* The ability to refer the specific slots in arrays.
* An `content_type` option controls whether json string escaping is used. Otherwise there is no escaping.
Closeselastic/elasticsearch#1116
Other changes:
* I changed tests that were just using mustache just because it was around to not use mustache
* I moved tests to `test-xpack-with-mustache` module that were testing mustache with Watcher
* added smoke test for watcher and mustache
* moved some tests around
* instead of using DefaultTextTemplateEngine in watcher tests use MockTextTemplateEngine
* added a mock mustache script engine
* Cleanup some messy tests to not rely on mustache and move them back into xpack module
* moved array access test to smoke test watcher with mustache module
* test: simplified the condition search test to take the time component out of it, while still simulation a condition
* removed the mustache dependency in the messy-test-watcher-with-groovy module
Original commit: elastic/x-pack-elasticsearch@6a2a4e885f
- Consolidated `InternalMarvelUser`, `InternalWatcherUser` and `InternalShieldUser` into a single `XPackUser` - this is the single internal user for xpack that has all the permissions internally required by xpack (for marvel, watcher and shield)
- Renamed `InternalSystemUser` to `SystemUser`
- Removed the notion of "reserved roles". Now that we have a single internal user we know its role. The authz service now checks to see if the current user is the internal xpack user, and if so, it just uses its role (and not trying to resolve it from the role store). With this model, it's no longer possible for outside users to use the internal role (it's fully internal)
- Consolidated the notion of an `InternalClient` (in Marvel it was knows as the `SecuredClient`). This is an ES client that xpack is using to manage itself. If shield is enabled, it will execute all request on behalf of the internal xpack user.
- Removed the verification of the license plugin on plugin installation - no need to do it anymore as the license plugin is part of the distribution.
Original commit: elastic/x-pack-elasticsearch@c851410f93
This commit cleans up the hack we had forcefully switching the request to execute under the system
user when a internal action gets triggered from a system request. The authorization service now tracks
the originating request in the context to allow us to validate if the request should be run as the system
user.
The system user should be used only when a user action causes an internal action, which needs to
be run by the system user.
Closeselastic/elasticsearch#1403
Original commit: elastic/x-pack-elasticsearch@4972df459f
- removed `/_shield/roles` and `/_shield/users` endpoints (only keeping the singular forms)
- fixed `ClearRealmsCacheTests` to use the correct endpoint for clearing the realms cache
- used action name constants where possible in `InternalShieldUser`
Original commit: elastic/x-pack-elasticsearch@d1481de389
- Moved all role action classes to live under `o.e.s.action.role`
- Moved all realm related action classes (for now just the clear cache) to live under `o.e.s.action.realm`
- Moved all user action classes to live under `o.e.s.action.user`
- Moved all the rest actions to live under `o.e.s.rest.action`
- Changed the `clear role cache` endpoint to `/_shield/role/{id}/_clear_cache` (aligned with all other role endpoints)
- Changed `InternalShieldUserHolder` to the `InternalShieldUser` singleton user... to be aligned with `InternalMarvelUser` and `InternalWatcherUser`.
- Removed the dedicated audit log user. The new `InternalShieldUser` is now the user that manages and writes to the audit log indices
- Extracted the `User.System` class to a top level `InternalSystemUser` class (to be aligned with the other internal user classes)
- Removed the `SystemRole` class (the `InternalSystemUser` class now holds all the needed info/logic)
Original commit: elastic/x-pack-elasticsearch@cf82b257d1
When using a path like `"/<logstash-{now%2Fd}>/_search"` in the
http webhook. The already escaped slash (%2F) got escaped twice
and thus did not work any more.
The escaping happened when the code created an URI and was done
as part of that constructor. This is now switched to an URL (which
is used at the end anyway) which does not do the escaping, even though
this was required for the query string, which is now done when constructing.
Closeselastic/elasticsearch#1364
Original commit: elastic/x-pack-elasticsearch@861b6d2378
Two regressions have been introduced in elastic/x-pack@156d9e4d5b: marvel index templates should not be deleted between tests and checking for marvel indices existence should not fail with IndexNotFoundException when the indices are not yet created and Shield enabled.
closeselastic/elasticsearch#1396elastic/elasticsearch#1394elastic/elasticsearch#1382
In MultiNodesStatsTests.java, multiple nodes are started in async: the first node may collect marvel data multiple times when the last one just started. So we should not check for exact 1 doc per node but at least 1 doc per node.
closeselastic/elasticsearch#1370
In HttpExporterTemplateTests.java, we must compare a long count with a long value.
Original commit: elastic/x-pack-elasticsearch@732fef995a
Similar to the lifecycle services, stopping the shield lifecycle should
also ensure that the poller threads are stopped, which is tricky, in case
they run through huge user/role lists.
Original commit: elastic/x-pack-elasticsearch@7a48f19853
Restoring empty contexts causes issues with searches, but failure to restore the
original context when executing index requests that auto-create results in a
the index operation being tried by the system user.
See elastic/elasticsearch#1380
Original commit: elastic/x-pack-elasticsearch@522f857de7
Load average is not available anymore on Windows, the tests should not check the presence of the field. Also, "node_stats.json" file is hard to maintain and quite useless so this commit removes it.
Original commit: elastic/x-pack-elasticsearch@74d2e0dce6
This removes a check in the ClearRolesCacheTests that is prone to failure due to the
possibility of the cache poller running while we modify documents and updating cached
values prior to the test issuing the get roles call.
See elastic/elasticsearch#1354
Original commit: elastic/x-pack-elasticsearch@ba0b803466
This change migrates all of the xpack code to use the new ThreadContext when
dealing with headers and context data. For the most part this is a simple
cutover, but there are some things that required special casing. The internal
actions that executed by a user's requests need to forcefully drop the context
and set the system user. The workaround for this will be improved in a followup.
Additionally, the RequestContext still lives on due to the OptOutQueryCache,
which requires some core changes to fix this issue.
Original commit: elastic/x-pack-elasticsearch@87d2966d93
This commit is the x-plugins side of the refactoring of script settings.
Relates elastic/elasticsearchelastic/elasticsearch#16197
Original commit: elastic/x-pack-elasticsearch@4c429933b9
As of elastic/elasticsearchelastic/elasticsearch#16054 all index level settings
must be registered and use the new settings infrastructure. This commit
prepares for the merge to provide a smooth transition.
Original commit: elastic/x-pack-elasticsearch@bc0a4fec07
This commit addresses the handling of load averages in Marvel due to
upstream changes in core Elasticsearch where the load average field was
changed from an array to an object.
Original commit: elastic/x-pack-elasticsearch@9ea57968bb
Shield expands wildcards to concrete names (aliases or indices) before each request gets executed in es core. It never resolves aliases to concrete indices though, as permissions may be set against aliases rather than indices. During this resolution, it also looks at the state of the indices and the current indices options (expand_wildcards) to expand only to indices with the relevant state. When it comes to aliases though, they may point to multiple indices each one having a different state, so it always expands ignoring expand_wildcards. At that point the request will contain the explicit name of the alias, no wildcards, thus the expand_wildcards option will have no effect in core. ignore_unavailable could be used instead when interacting with shield, which would affect how aliases are resolved to multiple indices. In this case we can only blacklist the test in shield, as it will return both the closed index and the open one.
Original commit: elastic/x-pack-elasticsearch@54c11dfc31
jaymode
Alexander Reelsen
Simon Willnauer
Jason Tedor
Tanguy Leroux
Jim Ferenczi
Yannick Welsch
Nik Everett
Ryan Ernst
Martijn van Groningen
uboness
Boaz Leskes
Ali Beyad
Jay Modi
Adrien Grand
Areek Zillur
markharwood
javanna
Robert Muir
Lee Hinman