Commit Graph

7300 Commits

Author SHA1 Message Date
jaymode e775e84a7e Test: add additional logging to SslMultiPortTests
This commit adds some additional logging to the SslMultiPortTests to help with debugging an
intermittent failure.

Relates elastic/x-pack-elasticsearch#1809

Original commit: elastic/x-pack-elasticsearch@ae9c967249
2018-01-18 12:46:01 -07:00
jaymode 40d315b244 Test: only use a single shard in RemoteIndexAuditTrailStartingTests
Relates elastic/x-pack-elasticsearch#435

Original commit: elastic/x-pack-elasticsearch@05db61297b
2018-01-18 12:07:30 -07:00
Tim Brooks fb12a0e383 Support TLS/SSL renegotiation (elastic/x-pack-elasticsearch#3600)
This commit is related to elastic/x-pack-elasticsearch#3246. It adds support for receiving TLS/SSL
renegotiation requests for peers.

Original commit: elastic/x-pack-elasticsearch@c22c16b3bc
2018-01-18 10:59:44 -07:00
Lisa Cawley 0ea43c1aa1 [DOCS] Move auditing settings to Elasticsearch Reference (elastic/x-pack-elasticsearch#3608)
Original commit: elastic/x-pack-elasticsearch@a108afd26b
2018-01-18 09:18:24 -08:00
Dimitris Athanasiou eefd8e7940 [ML] Allow categorical conditions to use all analysis fields (elastic/x-pack-elasticsearch#3615)
Categorical conditions should be allowed to use any of the analysis
fields as a field_name.

Original commit: elastic/x-pack-elasticsearch@a0d25e7445
2018-01-18 15:48:56 +00:00
Ryan Ernst b785f9c61b Build: Use md5 to determine whether ml snapshot needs downloading (elastic/x-pack-elasticsearch#3612)
This commit makes ML snapshot downloading happen less often. It does
that by first moving the download location to a directory outside the
destructive power of gradle clean, and then also uses the md5 of the zip
to compare to that found in s3. This allows us to do a cheap HEAD
request to find if the file has changed.

Original commit: elastic/x-pack-elasticsearch@cd8b00fd31
2018-01-17 22:12:23 -08:00
Lisa Cawley 9f6064f9ac [DOCS] Edited documentation for ML categorization_analyzer (elastic/x-pack-elasticsearch#3587)
Original commit: elastic/x-pack-elasticsearch@6dd179107a
2018-01-17 13:11:36 -08:00
Jay Modi 60d4b7e53e Add the ability to refresh tokens obtained via the API (elastic/x-pack-elasticsearch#3468)
This commit adds the ability to refresh tokens that have been obtained by the API using a refresh
token. Refresh tokens are one time use tokens that are valid for 24 hours. The tokens may be used
to get a new access and refresh token if the refresh token has not been invalidated or
already refreshed.

relates elastic/x-pack-elasticsearch#2595

Original commit: elastic/x-pack-elasticsearch@23435eb815
2018-01-17 12:18:44 -07:00
Dimitris Athanasiou f91631664a [ML] Rename Condition member valueFilter -> value (elastic/x-pack-elasticsearch#3599)
Original commit: elastic/x-pack-elasticsearch@5ef8415ca5
2018-01-17 18:39:47 +00:00
Tim Brooks dda3a8dee0 Add TLS/SSL enabled SecurityNioTransport (elastic/x-pack-elasticsearch#3519)
This is related to elastic/x-pack-elasticsearch#3246. This commit adds a SSL/TLS layer to the nio
work implemented in the SSLChannelContext and SSLDriver classes.
This work is used to build up a SecurityNioTransport implementation.
This transport does yet offer feature parity with our normal security
transport. It mainly offers SSL/TLS security.

Original commit: elastic/x-pack-elasticsearch@d0e0484418
2018-01-17 09:44:31 -07:00
Lisa Cawley a4fad02d9a [DOCS] Added SSL certificates API (elastic/x-pack-elasticsearch#3136)
Original commit: elastic/x-pack-elasticsearch@62cb574fcf
2018-01-17 08:14:02 -08:00
Alexander Reelsen ef2d2764a5 Watcher: Improve cluster state listener behaviour (elastic/x-pack-elasticsearch#3538)
The cluster state listener used by watch now have two additional checks.
First, when no master node exists in the cluster state, watcher will
stop and the indexing listener will not try to trigger any new watch.
Second, when there is a global cluster write level block, it would not
be possible to update the watches index or write into the watcher
history, so the listener can bail at that case as well.

In addition this also changes the log level from debug to info when
watcher is stopped. It turned out that there are zero insights when or
if watcher is stopped when normal logging is activated. This makes it
super hard for support to know when watcher is stopped or started at all
due to shards being moved around.

Original commit: elastic/x-pack-elasticsearch@5e9ce24380
2018-01-17 14:18:17 +01:00
Jason Tedor 7ca8b72f97 Push runtime Java home through to Gradle
We need to start Gradle on the same JVM as runtime Java home to ensure
that certain tasks (e.g., third party audit) are run on all the flavors
of the JDK that we support (without this change it would only run on the
same JVM as Gradle which would usually be compiler Java home).

Relates elastic/x-pack-elasticsearch#3588

Original commit: elastic/x-pack-elasticsearch@d5f6a54fbe
2018-01-16 22:55:26 -05:00
Tim Vernum b0552e1c6e [Security] Handle cache expiry in token service (elastic/x-pack-elasticsearch#3565)
* [Security] Handle cache expiry in token service

The keyCache on TokenService.KeyAndCache has a 60 minute expiry.
If the token service was idle for more than 60 minutes, the current
key would be expired and it would then fail to generate user tokens.

Original commit: elastic/x-pack-elasticsearch@fd98130a27
2018-01-17 13:04:59 +10:00
Yogesh Gaikwad 29663c1f38 Fix for Issue elastic/x-pack-elasticsearch#3403 - Predictable ordering of security realms (elastic/x-pack-elasticsearch#3533)
* Security Realms: Predictable ordering for realms

To have predictable ordering of realms, by having secondary
sorting on realm name resulting in stable and consistent documentation.
Documentation update describing how ordering of realms is determined.
Testing done by adding unit test for the change, ran gradle clean check locally.

relates elastic/x-pack-elasticsearch#3403

Original commit: elastic/x-pack-elasticsearch@98c42a8c51
2018-01-17 10:29:00 +11:00
lcawley d6f0b59e7a Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@7f3f3ce66a
2018-01-16 13:24:55 -08:00
lcawley 067f55b805 [DOCS] Added elasticsearch_nodes to monitoring blacklist
Original commit: elastic/x-pack-elasticsearch@53012a15a8
2018-01-16 13:21:42 -08:00
Lisa Cawley 117802042a [DOCS] Added monitoring PRs to 6.1.2 release notes (elastic/x-pack-elasticsearch#3548)
Original commit: elastic/x-pack-elasticsearch@e36e558563
2018-01-16 13:21:22 -08:00
Jason Tedor 8a0a14b9a3 Adapt to compile and runtime Java home distinction
We now separate the compiler Java home from the time runtime Java home
(the one that is used to compile class files versus the one that is used
to run tests). This commit adapts x-pack-elasticsearch to this change.

Relates elastic/x-pack-elasticsearch#3477

Original commit: elastic/x-pack-elasticsearch@bdb096e21c
2018-01-16 13:45:42 -05:00
Dimitris Athanasiou 368c4fff56 [ML] Expand job groups on scheduled events update (elastic/x-pack-elasticsearch#3577)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@73334d8e01
2018-01-16 11:53:37 +00:00
Dimitrios Athanasiou 0b8723015f [ML] Capitalize first letter in error message
Original commit: elastic/x-pack-elasticsearch@7062851357
2018-01-16 11:50:39 +00:00
Dimitris Athanasiou 2350b85507 [ML] Improve error on get calendar when ml-meta index is missing (elastic/x-pack-elasticsearch#3576)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@bc60a0cb72
2018-01-16 11:00:07 +00:00
Alexander Reelsen ac2884f034 Watcher: Fix NPE in watcher index template registry (elastic/x-pack-elasticsearch#3571)
The current code throws an NPE, when there is no master node available
in the cluster state. This commit adds an additional check for an
existing master node.

relates elastic/x-pack-elasticsearch#3569

Original commit: elastic/x-pack-elasticsearch@e0024b5170
2018-01-16 10:11:12 +01:00
Dimitris Athanasiou 6c6dcafd0e [ML] Improve error message when post-calendar-events-request is empty (elastic/x-pack-elasticsearch#3572)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@3d25ed4723
2018-01-15 18:13:50 +00:00
Dimitris Athanasiou aff9a4a2ba [ML] Autodetect should receive events from the earliest valid timestamp (elastic/x-pack-elasticsearch#3570)
When events are searched to be passed to the autodetect process, they
are currently calculated based on the latest record timestamp, when
a job opens, and `now` when the process is updated.

This commit changes both to be consistent and based on the earliest
valid timestamp for the job. The earliest valid timestamp is the
latest record timestamp minus the job latency.

Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@7f882ea053
2018-01-15 18:07:48 +00:00
Alexander Reelsen 57d887c9df Tests: Add noop test to prevent test failures
Follow up for elastic/x-pack-elasticsearch@a15338415a

Original commit: elastic/x-pack-elasticsearch@c0a46a5977
2018-01-15 18:06:54 +01:00
Tim Brooks 5eef187b19 Modify x-pack to support elasticsearch-core jar (elastic/x-pack-elasticsearch#3544)
This is related to elastic/elasticsearch#27933. It modifies x-pack to be
compatible with the new elasticsearch-core jar.

Original commit: elastic/x-pack-elasticsearch@42227b6034
2018-01-15 09:59:03 -07:00
lcawley e44a09f5df Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@29bc3c3b44
2018-01-15 08:44:30 -08:00
lcawley c243d6eb21 [DOCS] Fixed short descriptions for ML APIs
Original commit: elastic/x-pack-elasticsearch@75937c0da1
2018-01-15 08:44:08 -08:00
Albert Zaharovits 93b8ef49e9 Fix CI failure elastic/x-pack-elasticsearch#3568
Bad test design & excessive randomness

Original commit: elastic/x-pack-elasticsearch@076d0b246a
2018-01-15 18:39:13 +02:00
lcawley 86f612ae3a [DOCS] Fixed link to Analyze API
Original commit: elastic/x-pack-elasticsearch@e203d839c2
2018-01-15 08:29:08 -08:00
David Roberts e9dafbd78d [DOCS] Add documentation for ML categorization_analyzer (elastic/x-pack-elasticsearch#3554)
This is the documentation for the changes made in elastic/x-pack-elasticsearch#3372.

Relates elastic/machine-learning-cpp#491

Original commit: elastic/x-pack-elasticsearch@7d67e9d894
2018-01-15 15:47:19 +00:00
Alexander Reelsen d4cddc12d0 Tests: Add @AwaitsFix annotation to flaky tests
These contribute to unstable builds, and should be reenabled on a
case by case base and worked on.

Original commit: elastic/x-pack-elasticsearch@a15338415a
2018-01-15 10:28:05 +01:00
lcawley 56b0f28aa3 [DOCS] More broken link fixes
Original commit: elastic/x-pack-elasticsearch@dd52976660
2018-01-12 13:52:19 -08:00
Tim Sullivan c9aa4e3d59 [Monitoring] Add additional metric fields to Beats template (elastic/x-pack-elasticsearch#3337)
* [Monitoring] Add uptime.ms field to Beats template

* Add cpu mapping [WIP]

* sort fields

* hierarchical - no dots in fields

* add rss, more event metrics, system cpu and os load

* fix some mapping errors

Original commit: elastic/x-pack-elasticsearch@59cbcb4ee5
2018-01-12 14:26:04 -07:00
Igor Motov aba3f15d81 Add adding ability to associate an ID with tasks (elastic/x-pack-elasticsearch#3500)
X-Pack portion of elastic/elasticsearch#23250

Original commit: elastic/x-pack-elasticsearch@3c9a5d2d08
2018-01-12 15:34:28 -05:00
lcawley adde96f54a Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@24d927e455
2018-01-12 11:59:50 -08:00
lcawley d7f81fd95e [DOCS] Fixed broken TLS links
Original commit: elastic/x-pack-elasticsearch@d99a0be781
2018-01-12 11:59:15 -08:00
Jay Modi cec90f452a IndexLifecycleManager checks for index existense and up to date mappings (elastic/x-pack-elasticsearch#3515)
This change makes the IndexLifecycleManager check for both index existence and up to date mappings
on the index prior to executing the provided runnable. Doing this provides a mechanism to make
non-breaking mapping updates to the security index in minor versions.

relates elastic/x-pack-elasticsearch#3462

Original commit: elastic/x-pack-elasticsearch@80f05d83b4
2018-01-12 12:45:52 -07:00
Lisa Cawley 1369a49b9f [DOCS] Move appropriate TLS content to Elasticsearch Ref (elastic/x-pack-elasticsearch#3416)
Original commit: elastic/x-pack-elasticsearch@a5f96bd7a2
2018-01-12 11:35:16 -08:00
Lisa Cawley 860014ff5b [DOCS] Drafted overview of calendars and special events (elastic/x-pack-elasticsearch#3395)
Original commit: elastic/x-pack-elasticsearch@970cfea67e
2018-01-12 10:13:57 -08:00
Dimitris Athanasiou 73f8559fca [ML] Update process when filters or calendars are updated (elastic/x-pack-elasticsearch#3385)
Relates elastic/x-pack-elasticsearch#3325

Original commit: elastic/x-pack-elasticsearch@9da4973cda
2018-01-12 17:48:07 +00:00
David Kyle 86e9f63b19 [ML] Expect an array of events in request (elastic/x-pack-elasticsearch#3537)
* Expect an array of events in request

Original commit: elastic/x-pack-elasticsearch@f60bc0b544
2018-01-11 22:16:09 +00:00
jaymode 79a9576596 Test: do not wipe templates from clusters in audit tests
This commit adds both the security index template and the audit index template to the excluded
templates for the RemoteIndexAuditTrailStartingTests and IndexAuditTrailTests. By allowing the
templates to remain in place, we make it easier for subsequent tests in the suite to have their
audit trails started.

Relates elastic/x-pack-elasticsearch#435

Original commit: elastic/x-pack-elasticsearch@6e54de19f9
2018-01-11 14:32:23 -07:00
jaymode 858178602e Test: update TokenAuthIntegTests to modify document to test expiration
This change modifies how we test the deletion of expired documents to be in line with elastic/x-pack-elasticsearch#3468 and
also adds debugging output in case the failures in CI continue.

Relates elastic/x-pack-elasticsearch#2253

Original commit: elastic/x-pack-elasticsearch@979b5357f5
2018-01-11 13:25:07 -07:00
Tim Brooks 0a8e4da3bd Support changing name of core module to server (elastic/x-pack-elasticsearch#3532)
This is related to elastic/elasticsearch#28178. It modifies x-pack to be
compatible with the elasticsearch core module to server name change.

Original commit: elastic/x-pack-elasticsearch@c8d313f882
2018-01-11 11:30:45 -07:00
Lisa Cawley 0998c875d6 [DOCS] Added forecast limitations (elastic/x-pack-elasticsearch#3503)
Original commit: elastic/x-pack-elasticsearch@b4aae8f4db
2018-01-11 10:08:15 -08:00
David Roberts c647d7f38b [ML] Adjust BWC for categorization analyzer config after backporting elastic/x-pack-elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@27fcd5bdcb
2018-01-11 17:01:37 +00:00
David Kyle 999c1dab48 [ML] Remove extra quotes in field config
Original commit: elastic/x-pack-elasticsearch@d9a7ca5ebf
2018-01-11 15:57:17 +00:00
David Roberts a386b5727e [ML] Increase tokenization flexibility for categorization (elastic/x-pack-elasticsearch#3372)
By moving tokenization for categorization to Java we give users access to considerably more options for tokenizing their log messages prior to using ML to categorize them. Now all Elasticsearch analyzer functionality is available, which opens up the possibility to sensibly categorize non-English log messages.

Relates elastic/machine-learning-cpp#491

Original commit: elastic/x-pack-elasticsearch@5d61b67614
2018-01-11 15:40:13 +00:00