1489 Commits

Author SHA1 Message Date
Nik Everett
d2df25072a Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@f7d99dcbb7
2017-12-20 12:03:20 -05:00
Boaz Leskes
5e51422f4d PkiAuthenticationTests & SslIntegrationTests should instead use NetworkAddress's format magic
Original commit: elastic/x-pack-elasticsearch@601e8e774d
2017-12-20 16:05:41 +01:00
Boaz Leskes
133d70bc6a PkiAuthenticationTests & SslIntegrationTests should properly handle ipv6 addresses
Original commit: elastic/x-pack-elasticsearch@b11d90e584
2017-12-20 15:53:22 +01:00
Boaz Leskes
b89b1d5cc5 PkiAuthenticationTests & SslIntegrationTests shouldn't hard code local host
The test uses the bound address to determine how to speak to the node via http.
It currently takes the port but hard codes the host to `localhost`. This can lead
to mismatches where a port for ipv6 is used but localhost resolves to ipv4

relates elastic/x-pack-elasticsearch#3382

Original commit: elastic/x-pack-elasticsearch@e97363a521
2017-12-20 14:50:51 +01:00
Jason Tedor
1ac31fe626 Mute SSL HTTP connection test
This test is failing for days, possibly due to a change in core
Elasticsearch. This commit marks this test as awaits fix.

Original commit: elastic/x-pack-elasticsearch@8ed3965795
2017-12-20 06:42:20 -05:00
Jason Tedor
fe7c8fb4da Mute PKI REST authentication test
This test is failing for days, possibly due to a change in core
Elasticsearch. This commit marks this test as awaits fix.

Original commit: elastic/x-pack-elasticsearch@8d7db1c423
2017-12-20 06:41:23 -05:00
Jason Tedor
1bf63350f8 Mute PKI REST authentication failure test
This test is failing for days, possibly due to a change in core
Elasticsearch. This commit marks this test as awaits fix.

Original commit: elastic/x-pack-elasticsearch@f519c2a7ee
2017-12-20 06:04:58 -05:00
Simon Willnauer
3dd9445f4b [TEST] Catch ISE instead of IAE for illegal array size
relates elastic/x-pack-elasticsearch#2493

Original commit: elastic/x-pack-elasticsearch@605dcebf0e
2017-12-20 09:49:48 +01:00
Nik Everett
7e11a1b388 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@82985d6481
2017-12-19 13:43:49 -05:00
Dimitris Athanasiou
08a35d44c6 [ML] Support multiple rule actions and renames (elastic/x-pack-elasticsearch#3356)
Relates elastic/x-pack-elasticsearch#3325

Original commit: elastic/x-pack-elasticsearch@a7f400aeeb
2017-12-19 16:28:36 +00:00
jaymode
f1f1be3927 Test: tests that use security index should not delete template
Tests that rely on the security index and security index template being present should not remove
the template between tests as this can cause test failures. The template upgrade service relies
on cluster state updates to trigger the template being added after a delete, but there is a
scenario where the test will just wait for template that never shows up as there is no cluster
state update in that time. Instead of fighting ourselves, we should just leave the template in
place.

Relates elastic/x-pack-elasticsearch#2915
Relates elastic/x-pack-elasticsearch#2911

Original commit: elastic/x-pack-elasticsearch@3ca4aef0be
2017-12-19 08:24:23 -07:00
David Kyle
a8997387b7 [ML] Calendar jobs endpoints (elastic/x-pack-elasticsearch#3320)
* Calendar jobs endpoints

* Refactor put and delete calendar job to use the same action

* Check jobs exist when creating the calendar

* Address review comments

* Add isGroupOrJobMethod

* Increase default page size for calendar query


Original commit: elastic/x-pack-elasticsearch@7484799fe9
2017-12-19 13:57:32 +00:00
Tanguy Leroux
3efd35cadf [Monitoring] Add missing mapping for interval_ms (elastic/x-pack-elasticsearch#3339)
# 2650 added the mapping for the interval_ms field in the Elasticsearch 
template but not for Kibana,Logstash and Beats templates.

Original commit: elastic/x-pack-elasticsearch@44fb501bb3
2017-12-19 09:14:10 +01:00
Nik Everett
4680e1e166 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@2067b14cf8
2017-12-18 12:15:04 -05:00
Chris Earle
01f5318642 [Watcher] Use index.auto_expand_replicas: 0-1 (elastic/x-pack-elasticsearch#3284)
This changes the default behavior of .watch* indices to be green on one-node clusters, instead of constantly yellow.

Original commit: elastic/x-pack-elasticsearch@cdaee7cd72
2017-12-15 18:32:34 -05:00
Alexander Reelsen
5f8a0711f5 Watcher: Set index and type dynamically in index action (elastic/x-pack-elasticsearch#3264)
The index action allowed to set the id of a document dynamically,
however this was not allowed for the index or the type.

If a user wants to execute a search, modify the found documents and
index them back, then this would only work across a single index and a
single type. This change allows the watch writer to just take a search
result, read index and type out of that and configure this as part of
the index action.

On top of that the integration tests have been changed to become fast
running unit tests.

Original commit: elastic/x-pack-elasticsearch@640b085dd4
2017-12-15 16:59:29 +01:00
Alexander Reelsen
f518501df4 Tests: Ensure that watcher is started in HipchatServiceTests
One of those tests requires watcher to be started, so a proper
assertBusy() block has been added to this tests.

relates elastic/x-pack-elasticsearch#3324

Original commit: elastic/x-pack-elasticsearch@324830316f
2017-12-15 16:05:43 +01:00
Alexander Reelsen
758433a0fa Monitoring: Ensure all monitoring watches filter by timestamp (elastic/x-pack-elasticsearch#3238)
Only the Logstash and Kibana version mismatch watches contain a time
filter, the others are only sorting by timestamp. In combination with
searching in all `.monitoring-es-*` indices, this is IMO pretty resource
intensive, as we cannot exit early on any search request.

This commit adds time based filters to remaining three watches, using
the same range than the other two.

Original commit: elastic/x-pack-elasticsearch@3eb6bf0de2
2017-12-15 15:23:57 +01:00
Shaunak Kashyap
78f7c0e27a Fix license messaging for Logstash functionality (elastic/x-pack-elasticsearch#3268)
* Fix license messaging for Logstash functionality

With a Basic license, users are still able to perform CRUD operations on the `.logstash` index, therefore manage their Logstash pipelines. However, Logstash itself will not pick up any changes from this index and act on them. With an expired license Logstash functionality continues to operate as normal.

* Fixing messages after feedback

* Removing extraneous tabs at end of line

* Fixing typo

Original commit: elastic/x-pack-elasticsearch@bc069cf00f
2017-12-15 05:56:01 -08:00
Nik Everett
2268e592c6 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@ded26a3834
2017-12-14 10:31:51 -05:00
Colin Goodheart-Smithe
51b80aa998 Fixes test to support BytesSizeValue changes (elastic/x-pack-elasticsearch#3321)
Original commit: elastic/x-pack-elasticsearch@0bc8b0e847
2017-12-14 12:17:25 +00:00
Ioannis Kakavas
d24921ea60 Check for existing x-pack directory when running the users CLI tool (elastic/x-pack-elasticsearch#3271)
Verify that the configuration directory `$ES_PATH_CONF/x-pack`
exists before attempting to run any of the `users` commands, and 
return a helpful error message to the user if it doesn't.

Original commit: elastic/x-pack-elasticsearch@6d663b6654
2017-12-14 13:45:53 +02:00
Chris Earle
876fc5612a [Monitoring] Use index.auto_expand_replicas: 0-1 (elastic/x-pack-elasticsearch#3282)
This changes the default behavior of .monitoring indices to be green on one-node clusters, instead of constantly yellow.

Note: This only impacts .monitoring* indices. Watcher indices currently still require a replica.

Original commit: elastic/x-pack-elasticsearch@6eb8a48a9f
2017-12-13 13:46:31 -05:00
Costin Leau
cdfe0f1f5a Revert "Revert "Merge branch 'feature/sql'""
This reverts commit elastic/x-pack-elasticsearch@cc79e19911.

We'll merge this when we're good and ready.

Original commit: elastic/x-pack-elasticsearch@b3ef4f2836
2017-12-13 10:19:31 -05:00
Costin Leau
2e60e831c0 Revert "Merge branch 'feature/sql'"
This reverts commit elastic/x-pack-elasticsearch@2b3f7697a5, reversing
changes made to elastic/x-pack-elasticsearch@b79f16673c.

We're backing out all of SQL.

Original commit: elastic/x-pack-elasticsearch@cc79e19911
2017-12-13 09:33:13 -05:00
Nik Everett
aef5ea6527 Merge branch 'feature/sql'
It is time.

Original commit: elastic/x-pack-elasticsearch@2b3f7697a5
2017-12-12 14:49:27 -05:00
Jay Modi
d48ad6dbd6 Enhance security for alias and resize operations (elastic/x-pack-elasticsearch#3302)
This commit adds additional checks around resize operations and alias creation operations to
add an extra layer of security around these APIs.

Original commit: elastic/x-pack-elasticsearch@b79f16673c
2017-12-12 12:01:27 -07:00
Nik Everett
eaa75cfdbb Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@b5fc2b484d
2017-12-12 11:29:23 -05:00
Dimitris Athanasiou
92c40061c5 [ML] Skip index health checks for remote indices on datafeed node sel (elastic/x-pack-elasticsearch#3301)
Upon selecting a node to run a datafeed we normally check that the
data indices exists and their primaries are active. However, these
checks cannot be applied for CCS to a remote cluster. This commit
skips these checks for remote indices.

This removes the last obstacle for running CCS datafeeds.

Relates elastic/x-pack-elasticsearch#1071

Original commit: elastic/x-pack-elasticsearch@092f44feee
2017-12-12 15:12:33 +00:00
Tanguy Leroux
711254fd24 [Monitoring] Use the same Cluster State for all Collectors (elastic/x-pack-elasticsearch#3216)
This commit changes the Collectors so that they all use the 
same instance of ClusterState.

relates elastic/x-pack-elasticsearch#3156

Original commit: elastic/x-pack-elasticsearch@4f537b026c
2017-12-12 12:29:40 +01:00
David Kyle
0d46e9035c [ML] Get Filters should use executeAsyncWithOrigin (elastic/x-pack-elasticsearch#3295)
* Get Filters should use executeAsyncWithOrigin

Original commit: elastic/x-pack-elasticsearch@786c7bfd06
2017-12-12 11:15:54 +00:00
David Kyle
6113b86bdb [ML] Special events calendar CRUD endpoints (elastic/x-pack-elasticsearch#3267)
* Calendar CRUD endpoints

* Get calendars requires monitor permission

* Address review comments

* Add page params to get calendars

Original commit: elastic/x-pack-elasticsearch@badd1e6add
2017-12-12 09:21:44 +00:00
Nik Everett
15f84840f2 SQL: Revert some security changes so NOOP merge
SQL used to have some changes to security. We've since reverted them but
we have some leftover stuff like import reordering and spacing changes.
We may as well remove them so merging SQL to master is smaller.

Original commit: elastic/x-pack-elasticsearch@c632256ddd
2017-12-11 16:06:48 -05:00
David Roberts
249d06b256 [ML] Fix permissions for field caps in scroll data extractor
Follow up to elastic/x-pack-elasticsearch#3254

Original commit: elastic/x-pack-elasticsearch@d4df9446c0
2017-12-11 17:45:32 +00:00
Igor Motov
4bebc307c3 SQL: Add ability to close cursors (elastic/x-pack-elasticsearch#3249)
This commits adds a new end point for closing in-flight cursors, it also ensures that all cursors are properly closed by adding after test checks that ensures that we don't leave any search context open.

relates elastic/x-pack-elasticsearch#2878

Original commit: elastic/x-pack-elasticsearch@1052ea28dc
2017-12-11 11:36:02 -05:00
David Kyle
04c07688a2 [ML] Align special events with buckets (elastic/x-pack-elasticsearch#3258)
Original commit: elastic/x-pack-elasticsearch@71f9d0fb13
2017-12-11 15:42:06 +00:00
David Roberts
5fd68959a0 [ML] Make datafeeds run-as the user who created/updated them (elastic/x-pack-elasticsearch#3254)
This is the ML equivalent of what was done for Watcher in elastic/x-pack-elasticsearch#2808.

For security reasons, ML datafeeds should not run as the _xpack
user.  Instead, they record the security headers from the request
to create/update them, and reuse these when performing the search
to retrieve data for analysis.

Relates elastic/x-pack-elasticsearch#1071

Original commit: elastic/x-pack-elasticsearch@29f85de404
2017-12-11 13:01:16 +00:00
Alexander Reelsen
6bae4681e2 Tests: Replace http input integration test with REST test (elastic/x-pack-elasticsearch#3215)
Remove HttpInputIntegrationTests, which only tested existing
functionality, the remaining part was moved over into a REST tests.

relates elastic/x-pack-elasticsearch#3210

Original commit: elastic/x-pack-elasticsearch@474e5337b6
2017-12-11 13:08:58 +01:00
Igor Motov
018d4d7722 Merge remote-tracking branch 'elastic/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@a113c54b94
2017-12-09 12:21:07 -05:00
Tim Brooks
9dbb887a58 Update x-pack for change to channel tracking (elastic/x-pack-elasticsearch#3261)
This commit updates x-pack to be compatible with
elastic/elasticsearch#27711. That commit removed the need for channels
to be internally tracked inside transport implementations. This commit
removes a test mocking class that is not necessary after that change.

Original commit: elastic/x-pack-elasticsearch@75d99ba1d1
2017-12-08 16:57:08 -07:00
Igor Motov
d3cd462700 Merge remote-tracking branch 'elastic/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@496f9cc46b
2017-12-08 13:22:20 -05:00
Tim Brooks
a1d72fc004 Modify security transport to take PageCacheRecycler (elastic/x-pack-elasticsearch#3251)
This is related to elastic/elasticsearhc#27696. That commit introduced
signature changes to `NetworkPlugin.getTransports`. This commit updates
x-pack to be compatible with that change.

Original commit: elastic/x-pack-elasticsearch@2e331ef6cd
2017-12-08 10:39:41 -07:00
Igor Motov
4330195cdf Merge remote-tracking branch 'elastic/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@f4ef2d9cdf
2017-12-08 12:35:59 -05:00
Jason Tedor
27a3a80c07 Adapt to core Strings method rename
This commit adapts a single usage of a method that was renamed in core.

Relates elastic/x-pack-elasticsearch#3265

Original commit: elastic/x-pack-elasticsearch@107c495d63
2017-12-08 12:17:29 -05:00
Igor Motov
b72a9b5674 SQL: switch from _sql/ endpoint to _xpack/sql endpoint (elastic/x-pack-elasticsearch#3270)
As a part of xpack, SQL should use _xpack/sql endpoint instead of _sql endpoint.

relates elastic/x-pack-elasticsearch#3114

Original commit: elastic/x-pack-elasticsearch@f561b57f16
2017-12-08 12:17:26 -05:00
Ioannis Kakavas
aff5cc840f Make generated passwords shell safe (elastic/x-pack-elasticsearch#3253)
Generate passwords from  [A-Za-z0-9] so that they are safe to be
used in shell scripts.
Entropy deterioration is not significant (124.9 -> 119), generated
passwords still meet guidelines and best practices regarding length
and complexity.

Resolves elastic/x-pack-elasticsearch#3087

Original commit: elastic/x-pack-elasticsearch@078639e7c2
2017-12-08 18:03:13 +02:00
Dimitrios Athanasiou
4082d6fa48 [Watcher] Remove line that accidentally sneaked in
Original commit: elastic/x-pack-elasticsearch@e63b1942ca
2017-12-08 13:47:53 +00:00
Dimitris Athanasiou
434dc94eb2 [ML] Avoid all overhead when renormalization window is zero (elastic/x-pack-elasticsearch#3255)
relates elastic/x-pack-elasticsearch#3244

Original commit: elastic/x-pack-elasticsearch@fe41c23ad7
2017-12-08 12:22:52 +00:00
Jason Tedor
0c6ec82317 Grant Netty necessary permissions
When using the security networking implementations, the Netty jars that
are in play are those that are loaded in the X-Pack classloader. This
means that permissions granted to the Netty jars loaded in the
transport-netty4 module classloader do nothing. Instead, we have to
grant the same permissions to the Netty jars in the X-Pack
classloader. This commit does this.

Relates elastic/x-pack-elasticsearch#3247

Original commit: elastic/x-pack-elasticsearch@91780597b9
2017-12-07 12:57:48 -05:00
Jay Modi
c6799de2a4 Do not enforce TLS if discovery type is single-node (elastic/x-pack-elasticsearch#3245)
This commit adds a check for the discovery type so that the TLS join validator does not fail join
requests when the discovery type is single-node.

relates elastic/x-pack-elasticsearch#2828

Original commit: elastic/x-pack-elasticsearch@fdfdb76b0b
2017-12-07 09:50:25 -07:00