The superuser role is the only user assignable role that grants access to the .security index, but when
resolving wildcards the index was not getting resolved. The resolution of indices and aliases explicitly
excludes the .security index for users that are not the internal user without checking if the user has the
superuser role. This commit adds a check in for the superuser role.
Original commit: elastic/x-pack-elasticsearch@02ee0a8740
The role cache was previously using the wrong time unit for its expire after write time; the
value passed to the cache was milliseconds instead of nanoseconds.
Original commit: elastic/x-pack-elasticsearch@65f7b08763
The anonymous role was being applied to other users for index access control but was not being applied
in terms of action level access control. This change makes the minimum required change to apply the
anonymous role for all users when anonymous is enabled. Additionally, some minor changes were made to the native roles store to not lookup roles before the service is started.
Closeselastic/elasticsearch#3711
Original commit: elastic/x-pack-elasticsearch@a9398e178d
When adding a watch which has a painless component, the scriptexception
was wrapped into a deprecated exception which means, that the awesome
painless descriptions were lost. This wrapping has been removed.
Closeselastic/elasticsearch#3161
Original commit: elastic/x-pack-elasticsearch@1703fe4eb6
This test has been blacklisted and deactivated months ago. This commit reenables this test and moves it at the right place. It also change the test to use the Execute Watch API instead of being sleep based.
Original commit: elastic/x-pack-elasticsearch@e7a9689375
in core we wrap request handlers with an asserting one to ensure we can serialize messages
with different versions. Yet, xpack uses the same functionality to add security aspects to
the network layer. These tests assert that the right handlers are in-place.
Original commit: elastic/x-pack-elasticsearch@e39c8995ae
Fixes to x-plugins code now that DateMathParser accepts a LongSupplier rather than a Callable to get the value of now
Relates to elastic/elasticsearchelastic/elasticsearch#20796
Original commit: elastic/x-pack-elasticsearch@99fc47a8a7
This change moves to using SSLParameters as the configuration source for SSLEngine and SSLSocket
objects that are configured by the SSLService. Previously we used a mix of specific methods and
SSLParameters, which resulted in issues where ordering of calls is important. For example, if configuring
client authentication directly on the engine prior to setting the SSLParameters resulted in the client
authentication configuration being reset to the default.
Additionally, this change also sets use cipher suite order to true to ensure preferred ciphers will be used.
Original commit: elastic/x-pack-elasticsearch@8ddecdc20c
This change ensures we wait for a response before the async http client is closed. Otherwise we can
close the client during the connection to the remote endpoint or never even connect to the remote
endpoint.
Closeselastic/elasticsearch#3640
Original commit: elastic/x-pack-elasticsearch@54900b1b4a
This changes does two things in the tribe tests. The first is that when we split data up between
multiple clusters, we always force create the security index so that randomization does not cause
edge cases like the index not existing in the preferred cluster. The second is we look at the cluster
state of the nodes and ensure the tribe node sees the indices and has all primaries active.
Separate tests were also added to cover the scenario where the security index only exists in the non
preferred node.
Original commit: elastic/x-pack-elasticsearch@17b78ec837
This is the last action that needs additional support for proxies.
You can set a proxy in the JSON like this:
```
"actions" : {
"notify-pagerduty" : {
"pagerduty" : {
"description" : "Main system down, please check!",
"proxy" : { "host" : "localhost", "port" : 8080 }
}
}
}
```
Closeselastic/elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@b99969fd6b
You can set it like this in the JSON
"actions" : {
"notify-slack" : {
"slack" : {
"account" : "integration-account",
"proxy" : {
"host" : "localhost",
"port" : 8080
},
"message" : {
...
}
}
}
}
Relates elastic/elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@de86233d4f
Watcher uses a custom thread pool. This is because executing watches can
be long-running tasks that often block on I/O and it is best to not
consume the core thread pools with these tasks. Today this thread pool
is fixed, and sized at five times the bounded number of cores (so 160 on
a 32-core box). It makes sense for there to possibly be so many threads,
again because these tasks can block on I/O and having excess capacity
lets unblocked watches execute. It's the fixed size that can cause
problem, all these threads are always consuming resources even when
there are no or not that many watches running. This commit changes this
thread pool to be a scaling thread pool.
Relates elastic/elasticsearch#3660
Original commit: elastic/x-pack-elasticsearch@3cafab6e83
We need to special case IndicesAliasesRequest as it doesn't implement CompositeIndicesRequest anymore. Note that the similar loop for CompositeIndicesRequests's subrequests will soon go away
Relates to elastic/elasticsearch#3638
Original commit: elastic/x-pack-elasticsearch@50d119ff61
This change allows native users and roles to be used on tribe nodes. The tribe node will actually
use the security index of one of the tribes, which must be specified with the `tribe.on_conflict`
setting. User and role modifications are not permitted when running on a tribe node.
Closeselastic/elasticsearch#3451
Original commit: elastic/x-pack-elasticsearch@2b762ca648
When deleting a watch the version was used as part of
the delete request. However a watch deletion means the
user really wants to get rid of it and not accidentally
run into a version exception because the watch was running in
between.
Original commit: elastic/x-pack-elasticsearch@e585f717f1
In preparation for elastic/elasticsearch#2957, I found we have things both in the root level
dev-tools, as well as elasticsearch/x-dev-tools. Most of this stuff can
be removed as it had to do with the old manual release process. There
was also a signed license file checked in. I removed it here, we really
should not have licenses checked in IMO, and it is unclear what the
purpose of this license was for. The two remaining scripts were moved to
the root dev-tools.
Original commit: elastic/x-pack-elasticsearch@3e24ea2d56
When running as a node, we check the `xpack.security.transport.filter.enabled` setting to see
if we should create the IPFilter but this check is not really correct. The HTTP filter could be
enabled or a profile filter could be enabled so there are times when we may not be filtering connections
when we should. Additionally, since we do not bind the IPFilter to a null provider, Guice will try to create
one during startup to inject into the security transport. This results in an exception and startup fails.
This change always creates the IPFilter when running as a node. This IPFilter has its own settings and
logic to determine whether it should be filtering on a given network transport.
Closeselastic/elasticsearch#3592
Original commit: elastic/x-pack-elasticsearch@95c25651c4
Some methods have been renamed in elastic/elasticsearchelastic/elasticsearch#20560. This commit change a .bytes() call to a .getBytes() call.
Original commit: elastic/x-pack-elasticsearch@4a0ff77361
This commit fixes the cat.templates REST tests so that it works when other templates exist (like monitoring)
Original commit: elastic/x-pack-elasticsearch@2e27ad88b4
This change switches the build to use the licensing prod key when
building the xpack jar for release.
Original commit: elastic/x-pack-elasticsearch@54a21dae5b
This adds an "interval" placeholder parameter that is required to the Monitoring Bulk API, and adds it to the Kibana side of the plumbing.
Having this will allow us to add it to all incoming documents and start to report against it with the Insights, as well as to detect the _lack_ of incoming documents.
By adding it now, we can avoid having a non-BWC API change for Kibana in 5.1. We'll just pickup new data in our documents.
Original commit: elastic/x-pack-elasticsearch@5ba8aafe03
This is a followup for elastic/elasticsearchelastic/elasticsearch#20526 removes the pluggability of
transport / http server transport via guice.
Original commit: elastic/x-pack-elasticsearch@5fb84949aa
The x-pack client jar and api jar are exactly the same: the entirety of
x-pack. Since we added the x-pack-transport jar, we no longer really
want the client jar as it is confusing. Additionally, it causes jar hell
when a test for an extension uses the transport client. This change
removes the client jar, and makes the x-pack transport client use the
api jar instead. This sounds odd at first, but since transport client is
going away eventually, it is a stopgap, and works.
closeselastic/elasticsearch#3309
Original commit: elastic/x-pack-elasticsearch@ee7a2c12c0
Due to untested code there was an NPE happening in production,
when a chained input execution failed, but the chained input
tried to access the resulting payload (which is never set on
failures). This payload now defaults to being empty.
This commit also drive-by fixes a broken logging statement, that
on the one side returned not the watch id, but a useless watch
toString() representation, and on the other hand only logs an error
message, but not a stack trace into the log, as this is what the
history is for.
Original commit: elastic/x-pack-elasticsearch@7dbe1afd90
Whenever a watch is updated (put, delete, set state), until now we
happened to reject those operations when a watch was executed at the
same time. However with long running reporting this might mean, that a
watch can never be changed, because it always gets executed.
* Removes the ability of write requests to obtain a lock at all (executing watches is still protected by a lock)
* Replaced the FairKeyedLock in watcher with the KeyedLock in Elasticsearch, which also takes a fair option, removed the FairKeyedLock
* Removed all the timeout parameters that are no longer needed, because there is no lock anymore
* Removed also the force parameter for watch deletion. Just do it[tm]
* Added a test that deleting a watch while it is being executed does not leave any leftovers
In case of a deletion of a watch during an execution, so that updating the status of the watch fails,
a warning is logged.
Closeselastic/elasticsearch#3417
Original commit: elastic/x-pack-elasticsearch@22fad1b797
As part of the review of elastic/elasticsearch#3287, the stopping of the IndexAuditTrail was moved to the tearDown
method. This works sometimes but other times it fails because tearDown is run after
ESIntegTestCase#after, so the IndexAuditTrail is still running during the after checks which will
cause the test to fail since the shard lock cannot be obtained.
Closeselastic/elasticsearch#3520
Original commit: elastic/x-pack-elasticsearch@4cb52b15a2
Today the operation mode can be set to default for a short amout of
time until it's reset to the actual mode this can cause weird sideeffects
for users if it's read concurrently. Also the test relies on a certain
happens before relationship that is not guaranteed since the operation
mode is set before the listerner is run. This change also rewrites the test
to not use busy waiting but wait for the actual listern to be executed.
Original commit: elastic/x-pack-elasticsearch@a2a42b89e5
1. We only support indexes created by Marvel 2.3+. All other indexes
are just ignored.
2. The tests don't assert a ton of interesting stuff because there
isn't a java API for Monitoring that we can just use. Instead we assert
that a few objects are there and look sane.
3. We don't migrate the contents of the data index. Instead we just
rely on Monitoring recreating it.
Original commit: elastic/x-pack-elasticsearch@86216c2d61
The IndexAuditTrail had both a stop and close method that needed to be called in order
to stop the service. There was a race where we called either flush or close in a non
blocking fashion and then immediately closed the underlying client. This change makes
the stop method wait for up to 10 seconds when closing the bulk processor.
Closeselastic/elasticsearch#3279
Original commit: elastic/x-pack-elasticsearch@0d776bc91a
TransportService is not pluggable anymore in core. Instead we now have a interceptor
infrastructure that allows to intercept send and receive calls on the transport layer.
Relates to elastic/elasticsearchelastic/elasticsearch#20505
Original commit: elastic/x-pack-elasticsearch@04194ecb09
This adds proxy support to the hipchat action. Right now
neither hipchat nor slack nor pagerduty allow for this,
but if you dont need a proxy for internal http connections,
but you do for external, then this configuration cannot be done
without setting a proxy for those actions.
You can set it like this in the JSON
```
"actions" : {
"notify-hipchat" : {
"hipchat" : {
"account" : "integration-account",
"proxy" : {
"host" : "localhost",
"port" : 8080
},
"message" : {
...
}
}
}
}
```
Relates elastic/elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@4e8447ce37
This cleans up some of the reported stats to be a little clearer, including making the JVM Heap chart behave like Kibana's memory chart. This solves two problems: you can now determine the max heap size and you know what "x%" actually means relative to it.
Original commit: elastic/x-pack-elasticsearch@450f6fd546
This commit fixes the logging audit tests which were broken due to an
upstream change in core Elasticsearch relating to the fact that prefixes
are no longer considered part of the log message, but are instead
implemented via markers.
Original commit: elastic/x-pack-elasticsearch@abd7ec23d8
To deny access to a fields users can name exceptions to field permissions with the following syntax:
"fields": {
"grant": [list of field names patterns],
"except": [list of patterns that are forbidden]
}
See doc for the rules for this.
This commit also reverts elastic/elasticsearch#2720closeselastic/elasticsearch#2681
Original commit: elastic/x-pack-elasticsearch@d6537028ec
The enabled and username fields are both now allowed in the request body for the put user
request. This makes it easier to perform a get and update a user without needing to edit more
of the request body than necessary.
Closeselastic/elasticsearch#3391
Original commit: elastic/x-pack-elasticsearch@ab763e843b
Start dates are a required feature for cloud. This functionality adds support
for specifying and enforcing a start date on licenses.
Behaviour: If the start date is > than now, the license will be rejected.
Due to another field in the license class, the version of the License class as well
as its serialization methods are adapted to this.
Closeselastic/elasticsearch#3370
Original commit: elastic/x-pack-elasticsearch@eb2a6f5be3
We were starting nodes at weird times and then shutting them down again,
slowing down the tests and causing the watcher tests to fail because
watcher wasn't being shut down with its traditional kid gloves.
Original commit: elastic/x-pack-elasticsearch@2fd81b3eaf
When the HTTP attachment was not able to successfully retrieve the
data from and endpoint, there was no indication in the watch history
of what went wrong. Instead a logger was used, which is not useful
for the person running the watches.
This commit removes the logger statement and throws an exception,
so that the exception message can be stored in the watch history.
Source of this issue was a forum post:
https://discuss.elastic.co/t/sending-e-mail-with-generated-report-fails/60263/6
Original commit: elastic/x-pack-elasticsearch@acdaf7abef
In our tests you have to explicitly shut down watcher rather than shut
down the node it is running on because of thread leak detection. Just
shutting down the node that it is running on will cause it to start up
on another node if there is another one running and then not properly
shut down. This is probably something that should be fixed in watcher
somehow but for now lets just be more careful with the tests.
Closeselastic/elasticsearch#2365Closeselastic/elasticsearch#2588
Original commit: elastic/x-pack-elasticsearch@fb8a172972
Previously core Elasticsearch had methods in a test class for removing
and adding appenders. However, these methods were moved to production
code. This commit adjusts x-plugins for this change.
Original commit: elastic/x-pack-elasticsearch@83e37ef65a
This particular change focuses on upgrading the source of a watch when it comes to scripts that have no language specified explicitly.
The default language in version 5 changed to painless from whatever is specified in `script.default_lang` setting (this defaulted to groovy). In order to make sure that scripts in watcher remain to work we should rewrite the search source upon startup and set the legacy default language explicitly. The legacy script language is now controlled by `script.legacy.default_lang` setting and that defaults to groovy.
Changing the source upon startup should do the trick and only change the source of watches with scripts that don't have an explicit language set. For new watches the default language used in scripts is painless and because we now always serialize the language explicitly in scripts these watches won't be changed on startup.
The upgrade logic added here tries to upgrade scripts in the following places in a watch:
* script condition
* script transform
* any script defined inside of a search input
Original commit: elastic/x-pack-elasticsearch@4d578819eb
This commit cleans most of the methods of XContentBuilder so that:
- Jackson's convenience methods are used instead of our custom ones (ie field(String,long) now uses Jackson's writeNumberField(String, long) instead of calling writeField(String) then writeNumber(long))
- null checks are added for all field names and values
- methods are grouped by type in the class source
- methods have the same parameters names
- duplicated methods like field(String, String...) and array(String, String...) are removed
- varargs methods now have the "array" name to reflect that it builds arrays
- unused methods like field(String,BigDecimal) are removed
- all methods now follow the execution path: field(String,?) -> field(String) then value(?), and value(?) -> writeSomething() method. Methods to build arrays also follow the same execution path.
Original commit: elastic/x-pack-elasticsearch@d83f3aa6e2
Jay Modi
Areek Zillur
Tanguy Leroux
Yannick Welsch
Alexander Reelsen
Nik Everett
jaymode
Simon Willnauer
Simon Willnauer
Colin Goodheart-Smithe
Igor Motov
jaymode
Jason Tedor
Luca Cavanna
Ryan Ernst
Chris Earle
Boaz Leskes
Britta Weber
Martijn van Groningen
javanna
Britta Weber