The chained input in watcher is a useful feature to
call several endpoints before execution a condition.
However it was pretty hard to modify data from a previous
input in order to be able to execute it in another input.
This commit adds a another input, called a `transform` input,
which allows you to do a transform as another input in a chained
input.
See this example
```
"input" : {
"chain" : {
"inputs" : [ <1>
{
"first" : {
"simple" : { "path" : "/_search" }
}
},
{
"second" : {
"transform" : {
"script" : "return [ 'path' : 'ctx.payload.first.path' + '/' ]"
}
}
},
{
"third" : {
"http" : {
"request" : {
"host" : "localhost",
"port" : 9200,
"path" : "{{ctx.payload.second.path}}" <2>
}
}
}
}
]
}
}
```
This allows for far more flexibility before executing the next input in a chained
one.
Original commit: elastic/x-pack-elasticsearch@3af9ba6e9b
We have to ensure logging is configured for any CLI command that depends
on core Elasticsearch (since it might directly or indirectly touch
logging). This commit does this for all commands in X-Pack.
Relates elastic/x-pack-elasticsearch#3112
Original commit: elastic/x-pack-elasticsearch@f77f9b5052
In order to be able to execute a watch as the user, who stored the
watch, this commit stores certain headers of the thread context, that
was used when the watch was stored.
Upon loading the watch the headers are loaded and applied for the
following watcher execution features
* search transform
* search input
* index action
A special case is the execute watch API, which overrides the headers loaded
from the watch with the one of the current request, so that a user
cannot execute this watch with other privileges of the user who stored it.
Only the headers "es-security-runas-user", "_xpack_security_authentication" are
copied for now, as those are needed for our security features.
The headers are stored in watch status in the watch and are not returned by default,
when the GET Watch API is used. A search reveals those of course.
relates elastic/x-pack-elasticsearch#2201
Original commit: elastic/x-pack-elasticsearch@9803bd51c2
Fixes to the build system, particularly around BWC testing, and to make future
version bumps less painful.
Original commit: elastic/x-pack-elasticsearch@a1d456f30a
* [DOCS] Added ML forecast API
* [DOCS] Added forecast API to build.gradle
* [DOCS] Added forecast API example
* [DOCS] Fixed forecast API intro
* [DOCS] Addressed feedback on forecast API
* [DOCS] Added duration to forecast API
* [DOCS] Removed end time from forecast API
* [DOCS] Fixed gradle errors for forecast API
Original commit: elastic/x-pack-elasticsearch@db79e3d5bb
The 6.x and 6.0 versions of the documentation show
```yml
transport.profiles.client.xpack.security.ssl.client_authentication: no
```
Which results in
```
2017-11-22T11:13:33,225][ERROR][org.elasticsearch.bootstrap.Bootstrap] Exception
java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.XPackPlugin]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:452) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:392) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:142) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.node.Node.<init>(Node.java:302) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.node.Node.<init>(Node.java:245) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:322) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:130) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:121) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:69) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.0.0.jar:6.0.0]
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:443) ~[elasticsearch-6.0.0.jar:6.0.0]
... 14 more
Caused by: java.lang.IllegalArgumentException: could not resolve ssl client auth. unknown value [no]
at org.elasticsearch.xpack.ssl.SSLClientAuth.parse(SSLClientAuth.java:78) ~[?:?]
at org.elasticsearch.xpack.ssl.SSLConfigurationSettings.lambda$null$27(SSLConfigurationSettings.java:183) ~[?:?]
at org.elasticsearch.common.settings.Setting.get(Setting.java:352) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.common.settings.Setting.get(Setting.java:346) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.xpack.ssl.SSLConfiguration.<init>(SSLConfiguration.java:80) ~[?:?]
at org.elasticsearch.xpack.ssl.SSLService.lambda$loadSSLConfigurations$1(SSLService.java:462) ~[?:?]
at java.util.ArrayList.forEach(ArrayList.java:1249) ~[?:1.8.0_144]
at org.elasticsearch.xpack.ssl.SSLService.loadSSLConfigurations(SSLService.java:461) ~[?:?]
at org.elasticsearch.xpack.ssl.SSLService.<init>(SSLService.java:87) ~[?:?]
at org.elasticsearch.xpack.XPackPlugin.<init>(XPackPlugin.java:237) ~[?:?]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:443) ~[elasticsearch-6.0.0.jar:6.0.0]
... 14 more
```
I think the valid value there is `none`, so
```yml
transport.profiles.client.xpack.security.ssl.client_authentication: none
```
The tests seem to confirm that https://github.com/elastic/x-pack-elasticsearch/blob/elastic/x-pack-elasticsearch@4860e92d906e046a23aa07b39ee6ef637f011dc1/plugin/src/test/java/org/elasticsearch/xpack/ssl/SSLServiceTests.java#L269
Original commit: elastic/x-pack-elasticsearch@a35b3ac8c9
Instead of returning "error response" objects and then translating them
into SQL exceptions this just throws the SQL exceptions directly. This
means the CLI catches exceptions and prints out the messages which isn't
ideal if this were hot code but it isn't and this is a much simpler way
of doing things.
Original commit: elastic/x-pack-elasticsearch@08431d3941
This change removes the InternalClient and the InternalSecurityClient. These are replaced with
usage of the ThreadContext and a transient value, `action.origin`, to indicate which component the
request came from. The security code has been updated to look for this value and ensure the
request is executed as the proper user. This work comes from elastic/x-pack-elasticsearch#2808 where @s1monw suggested
that we do this.
While working on this, I came across index template registries and rather than updating them to use
the new method, I replaced the ML one with the template upgrade framework so that we could
remove this template registry. The watcher template registry is still needed as the template must be
updated for rolling upgrades to work (see elastic/x-pack-elasticsearch#2950).
Original commit: elastic/x-pack-elasticsearch@7dbf2f263e
Room names in hipchat were not properly URL encoded, thus room names
with spaces would not work as expected. This fixes all the hipchat
accounts by properly using spaces.
Also the hipchat tests are reenabled, as the IT team gave me new access to hipchat,
allowing to create a fresh set of oauth tokens for the integration account type.
The HipchatServiceTests have also been converted to XPackSingleNodeTestCase
relates elastic/x-pack-elasticsearch#2371
relates elastic/x-pack-elasticsearch#2429
Original commit: elastic/x-pack-elasticsearch@9f8872f686
This commit changes the IndexLifecycleManager's handling of variables about an index to only update
all of the values at a single time. Previously, all of the index state variables were volatile
members of the IndexLifecycleManager, which meant we could get an inconsistent view of the index
state. Although rare, this is still incorrect so this change adds a single volatile variable that
holds the state as of the last processed cluster state update.
Additionally, the IndexLifecycleManagerIntegTests were updated to have more concurrency and further
stress this portion of the code and its checks.
relates elastic/x-pack-elasticsearch#2973
Original commit: elastic/x-pack-elasticsearch@5f1552b298
Adds the option to specify an elasticsearch filter in addition to the SQL query by introducing a filter parameter in the REST query which would create a boolean filter if the SQL query generates an elasticsearch query or a constant score query if SQL if the SQL query doesn't generates an elasticsearch query. Usage:
{
"query": "SELECT * FROM index",
"filter" : { "term" : { "tag" : "tech" } }
}
relates elastic/x-pack-elasticsearch#2895
Original commit: elastic/x-pack-elasticsearch@9a73813c7f
This commits also simplifies the serialization mechanism by remove 2 ways to serialize the cursor. Adding the version there was complicating things too much otherwise.
Original commit: elastic/x-pack-elasticsearch@4f2c541e0a
This commit adds the manage_index_templates permission to the kibana_system role that is used by
the kibana system user. This is needed due to an upcoming feature in kibana where a index template
will be used to create the saved objects index.
relates elastic/x-pack-elasticsearch#2937
Original commit: elastic/x-pack-elasticsearch@85a67c73aa
Some of our REST tests open many jobs, and assuming each will use 1GB of
RAM on a single node cluster could fail the test. The solution is to
explicitly say the test jobs will use very little RAM.
Original commit: elastic/x-pack-elasticsearch@a3fcfc4589
Currently, any errors that occur during the DeleteExpiredDataAction are logged and the deletion proceeds to the next job. The user will get no indication in the response that something went wrong although nothing should really go wrong unless the cluster is messed up.
This commit changes this so that errors are reported back to the action.
Original commit: elastic/x-pack-elasticsearch@489cf03c3e
Elasticsearch plugins are traditionally using org.elasticsearch.plugin group, clients are using org.elasticsearch.client, so it makes sense to move all sql pieces and deliverables into org.elasticsearch.sql group. Otherwise, users might get really confused about an artifact named `server` in the group `org.elasticsearch`.
Original commit: elastic/x-pack-elasticsearch@c3a4042040
This change modifies the way ML jobs are assigned to nodes to primarily
base the decision on the estimated memory footprint of the jobs. The
memory footprint comes from the model size stats if the job has been
running long enough, otherwise from the model memory limit. In addition,
an allowance for the program code and stack is added.
If insufficient information is available to base the allocation decision on
memory requirements then the decision falls back to using simple job
counts per node.
relates elastic/x-pack-elasticsearch#546
Original commit: elastic/x-pack-elasticsearch@b276aedf2f
Nik Everett
Igor Motov
Alexander Reelsen
David Kyle
David Roberts
Jason Tedor
Dimitris Athanasiou
Russ Cam
David Turner
lcawley
jaymode
Igor Kupczyński
Jay Modi
Hendrik Muhs
Simon Willnauer
javanna
Luca Cavanna
Michael Basnight