Commit Graph

7726 Commits

Author SHA1 Message Date
Dimitrios Athanasiou 0b8723015f [ML] Capitalize first letter in error message
Original commit: elastic/x-pack-elasticsearch@7062851357
2018-01-16 11:50:39 +00:00
Dimitris Athanasiou 2350b85507 [ML] Improve error on get calendar when ml-meta index is missing (elastic/x-pack-elasticsearch#3576)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@bc60a0cb72
2018-01-16 11:00:07 +00:00
Alexander Reelsen ac2884f034 Watcher: Fix NPE in watcher index template registry (elastic/x-pack-elasticsearch#3571)
The current code throws an NPE, when there is no master node available
in the cluster state. This commit adds an additional check for an
existing master node.

relates elastic/x-pack-elasticsearch#3569

Original commit: elastic/x-pack-elasticsearch@e0024b5170
2018-01-16 10:11:12 +01:00
Dimitris Athanasiou 6c6dcafd0e [ML] Improve error message when post-calendar-events-request is empty (elastic/x-pack-elasticsearch#3572)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@3d25ed4723
2018-01-15 18:13:50 +00:00
Dimitris Athanasiou aff9a4a2ba [ML] Autodetect should receive events from the earliest valid timestamp (elastic/x-pack-elasticsearch#3570)
When events are searched to be passed to the autodetect process, they
are currently calculated based on the latest record timestamp, when
a job opens, and `now` when the process is updated.

This commit changes both to be consistent and based on the earliest
valid timestamp for the job. The earliest valid timestamp is the
latest record timestamp minus the job latency.

Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@7f882ea053
2018-01-15 18:07:48 +00:00
Alexander Reelsen 57d887c9df Tests: Add noop test to prevent test failures
Follow up for elastic/x-pack-elasticsearch@a15338415a

Original commit: elastic/x-pack-elasticsearch@c0a46a5977
2018-01-15 18:06:54 +01:00
Tim Brooks 5eef187b19 Modify x-pack to support elasticsearch-core jar (elastic/x-pack-elasticsearch#3544)
This is related to elastic/elasticsearch#27933. It modifies x-pack to be
compatible with the new elasticsearch-core jar.

Original commit: elastic/x-pack-elasticsearch@42227b6034
2018-01-15 09:59:03 -07:00
lcawley e44a09f5df Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@29bc3c3b44
2018-01-15 08:44:30 -08:00
lcawley c243d6eb21 [DOCS] Fixed short descriptions for ML APIs
Original commit: elastic/x-pack-elasticsearch@75937c0da1
2018-01-15 08:44:08 -08:00
Albert Zaharovits 93b8ef49e9 Fix CI failure elastic/x-pack-elasticsearch#3568
Bad test design & excessive randomness

Original commit: elastic/x-pack-elasticsearch@076d0b246a
2018-01-15 18:39:13 +02:00
lcawley 86f612ae3a [DOCS] Fixed link to Analyze API
Original commit: elastic/x-pack-elasticsearch@e203d839c2
2018-01-15 08:29:08 -08:00
David Roberts e9dafbd78d [DOCS] Add documentation for ML categorization_analyzer (elastic/x-pack-elasticsearch#3554)
This is the documentation for the changes made in elastic/x-pack-elasticsearch#3372.

Relates elastic/machine-learning-cpp#491

Original commit: elastic/x-pack-elasticsearch@7d67e9d894
2018-01-15 15:47:19 +00:00
Alexander Reelsen d4cddc12d0 Tests: Add @AwaitsFix annotation to flaky tests
These contribute to unstable builds, and should be reenabled on a
case by case base and worked on.

Original commit: elastic/x-pack-elasticsearch@a15338415a
2018-01-15 10:28:05 +01:00
Nik Everett c4474f8574 SQL: Remove empty interface (elastic/x-pack-elasticsearch#3564)
After some recent changes this interface is now empty.

Original commit: elastic/x-pack-elasticsearch@4a44812f78
2018-01-14 10:32:31 -05:00
lcawley 56b0f28aa3 [DOCS] More broken link fixes
Original commit: elastic/x-pack-elasticsearch@dd52976660
2018-01-12 13:52:19 -08:00
Tim Sullivan c9aa4e3d59 [Monitoring] Add additional metric fields to Beats template (elastic/x-pack-elasticsearch#3337)
* [Monitoring] Add uptime.ms field to Beats template

* Add cpu mapping [WIP]

* sort fields

* hierarchical - no dots in fields

* add rss, more event metrics, system cpu and os load

* fix some mapping errors

Original commit: elastic/x-pack-elasticsearch@59cbcb4ee5
2018-01-12 14:26:04 -07:00
Igor Motov 70cea58262 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@9ac41d008e
2018-01-12 15:35:56 -05:00
Igor Motov aba3f15d81 Add adding ability to associate an ID with tasks (elastic/x-pack-elasticsearch#3500)
X-Pack portion of elastic/elasticsearch#23250

Original commit: elastic/x-pack-elasticsearch@3c9a5d2d08
2018-01-12 15:34:28 -05:00
lcawley adde96f54a Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@24d927e455
2018-01-12 11:59:50 -08:00
lcawley d7f81fd95e [DOCS] Fixed broken TLS links
Original commit: elastic/x-pack-elasticsearch@d99a0be781
2018-01-12 11:59:15 -08:00
Jay Modi cec90f452a IndexLifecycleManager checks for index existense and up to date mappings (elastic/x-pack-elasticsearch#3515)
This change makes the IndexLifecycleManager check for both index existence and up to date mappings
on the index prior to executing the provided runnable. Doing this provides a mechanism to make
non-breaking mapping updates to the security index in minor versions.

relates elastic/x-pack-elasticsearch#3462

Original commit: elastic/x-pack-elasticsearch@80f05d83b4
2018-01-12 12:45:52 -07:00
Lisa Cawley 1369a49b9f [DOCS] Move appropriate TLS content to Elasticsearch Ref (elastic/x-pack-elasticsearch#3416)
Original commit: elastic/x-pack-elasticsearch@a5f96bd7a2
2018-01-12 11:35:16 -08:00
Lisa Cawley 860014ff5b [DOCS] Drafted overview of calendars and special events (elastic/x-pack-elasticsearch#3395)
Original commit: elastic/x-pack-elasticsearch@970cfea67e
2018-01-12 10:13:57 -08:00
Dimitris Athanasiou 73f8559fca [ML] Update process when filters or calendars are updated (elastic/x-pack-elasticsearch#3385)
Relates elastic/x-pack-elasticsearch#3325

Original commit: elastic/x-pack-elasticsearch@9da4973cda
2018-01-12 17:48:07 +00:00
David Kyle 86e9f63b19 [ML] Expect an array of events in request (elastic/x-pack-elasticsearch#3537)
* Expect an array of events in request

Original commit: elastic/x-pack-elasticsearch@f60bc0b544
2018-01-11 22:16:09 +00:00
jaymode 79a9576596 Test: do not wipe templates from clusters in audit tests
This commit adds both the security index template and the audit index template to the excluded
templates for the RemoteIndexAuditTrailStartingTests and IndexAuditTrailTests. By allowing the
templates to remain in place, we make it easier for subsequent tests in the suite to have their
audit trails started.

Relates elastic/x-pack-elasticsearch#435

Original commit: elastic/x-pack-elasticsearch@6e54de19f9
2018-01-11 14:32:23 -07:00
jaymode 858178602e Test: update TokenAuthIntegTests to modify document to test expiration
This change modifies how we test the deletion of expired documents to be in line with elastic/x-pack-elasticsearch#3468 and
also adds debugging output in case the failures in CI continue.

Relates elastic/x-pack-elasticsearch#2253

Original commit: elastic/x-pack-elasticsearch@979b5357f5
2018-01-11 13:25:07 -07:00
Igor Motov 9170183e70 SQL: Rename :core to :server in sql gradle files
Related to elastic/elasticsearch#28180

Original commit: elastic/x-pack-elasticsearch@375f437580
2018-01-11 14:29:35 -05:00
Igor Motov 915ed35e27 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@02fddccd84
2018-01-11 13:41:40 -05:00
Tim Brooks 0a8e4da3bd Support changing name of core module to server (elastic/x-pack-elasticsearch#3532)
This is related to elastic/elasticsearch#28178. It modifies x-pack to be
compatible with the elasticsearch core module to server name change.

Original commit: elastic/x-pack-elasticsearch@c8d313f882
2018-01-11 11:30:45 -07:00
Lisa Cawley 0998c875d6 [DOCS] Added forecast limitations (elastic/x-pack-elasticsearch#3503)
Original commit: elastic/x-pack-elasticsearch@b4aae8f4db
2018-01-11 10:08:15 -08:00
Igor Motov c6f216a3f7 SQL: Use core's jna version
Relates to elastic/elasticsearch#28172

Original commit: elastic/x-pack-elasticsearch@f58fc39c14
2018-01-11 12:09:57 -05:00
David Roberts c647d7f38b [ML] Adjust BWC for categorization analyzer config after backporting elastic/x-pack-elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@27fcd5bdcb
2018-01-11 17:01:37 +00:00
David Kyle 999c1dab48 [ML] Remove extra quotes in field config
Original commit: elastic/x-pack-elasticsearch@d9a7ca5ebf
2018-01-11 15:57:17 +00:00
David Roberts a386b5727e [ML] Increase tokenization flexibility for categorization (elastic/x-pack-elasticsearch#3372)
By moving tokenization for categorization to Java we give users access to considerably more options for tokenizing their log messages prior to using ML to categorize them. Now all Elasticsearch analyzer functionality is available, which opens up the possibility to sensibly categorize non-English log messages.

Relates elastic/machine-learning-cpp#491

Original commit: elastic/x-pack-elasticsearch@5d61b67614
2018-01-11 15:40:13 +00:00
Alexander Reelsen 992a7af126 Watcher: Add support for actions in slack attachments (elastic/x-pack-elasticsearch#3355)
In order to support buttons that can be clicked on within a slack
message, this commits adds support for so called actions within
attachments. This allows to create buttons, that are clicked and execute
a GET request, so actions must be idempotent according to the official
slack documentation.

Official slack documentation is available at
https://api.slack.com/docs/message-attachments#action_fields

Original commit: elastic/x-pack-elasticsearch@29ddc90b01
2018-01-11 11:43:24 +01:00
Albert Zaharovits 3fc17ab918 Audit Event ignore policy (elastic/x-pack-elasticsearch#3005)
This commit introduces audit event filtering policies for the logging audit.
Until now the only way to temper the gush of audit events was to specifically
pick some event types that were reported.
This superposes a way (named policies) to filter events using lucene regexp filters on 
the following event fields: users, realms, roles and indices. The policies are ignore
policies, ie when an event matches it is ignored (not reported).

Original commit: elastic/x-pack-elasticsearch@233f685121
2018-01-11 11:07:46 +02:00
Chris Earle e74f90eba0 [Monitoring] Add Cluster Alert for Nodes Changed (elastic/x-pack-elasticsearch#3504)
This creates a cluster alert that triggers whenever a node is:

- Added
- Removed
- Restarted (aka Removed and Added between collections)

Unlike all previous cluster alerts, this cluster is always instantly resolved because there is no way to otherwise resolve it later (especially if nodes are replaced).

This will require a small change to the UI so that it does not simply ignore resolved alerts.

Original commit: elastic/x-pack-elasticsearch@6340bf7960
2018-01-10 21:29:00 -05:00
Costin Leau e5e5583bd8 SQL: equality on analyzed through TermQuery on not-analyzed (elastic/x-pack-elasticsearch#3524)
Always use term equality for analyzed terms by falling back (to possible
not present) to the not-analyzed field.
Add clarifying comment in test

Original commit: elastic/x-pack-elasticsearch@9e735d3fef
2018-01-11 00:01:30 +02:00
Costin Leau 2870312320 SQL: Optimizer tests (elastic/x-pack-elasticsearch#3518)
Unit tests for the Optimizer
Disabled for now RemoveDuplicateFunctions until elastic/x-pack-elasticsearch#3527 gets resolved

Original commit: elastic/x-pack-elasticsearch@3224e3937f
2018-01-10 19:42:15 +02:00
Costin Leau c204cc0aa3 SQL: removed fallback to exact prediction in the parser (elastic/x-pack-elasticsearch#3522)
The grammar definition should not require the exact prediction on the
listener. Falling back to it hides potential ambiguities.
Moved it to a separate method so it can be enabled for debugging in
development similar to Painless picky mode.

Original commit: elastic/x-pack-elasticsearch@969cb0b5cb
2018-01-10 17:52:35 +02:00
Igor Motov ce81a34467 SQL: Add jdbc mode to REST API (elastic/x-pack-elasticsearch#3499)
Adds a mode parameter to all SQL-related requests. The mode parameter is used for license checks as well as to define the response content. For now only two modes are supported plain (default) and jdbc. We will add other modes in the future as we add more clients.

Relates elastic/x-pack-elasticsearch#3419 

Original commit: elastic/x-pack-elasticsearch@b49ca38d4b
2018-01-10 09:41:36 -05:00
David Kyle 4c973c16c2 [ML] Rename event filter params (elastic/x-pack-elasticsearch#3525)
Original commit: elastic/x-pack-elasticsearch@3089193d45
2018-01-10 14:40:32 +00:00
Colin Goodheart-Smithe 45a31c953c Fixes docs test build
Original commit: elastic/x-pack-elasticsearch@23272a29ca
2018-01-10 12:05:45 +00:00
David Kyle d3d965c8ee [ML] Configure autodetect with scheduled events (elastic/x-pack-elasticsearch#3521)
Original commit: elastic/x-pack-elasticsearch@e51b70ec69
2018-01-10 12:02:14 +00:00
David Kyle 66b187fe17 [ML] Account for job groups membership when getting events (elastic/x-pack-elasticsearch#3473)
* Account for job groups membership when getting events

Original commit: elastic/x-pack-elasticsearch@a4185cc460
2018-01-10 11:14:09 +00:00
Dimitris Athanasiou aa25704170 [ML] Add scheduled_events field to bucket (elastic/x-pack-elasticsearch#3513)
Adds `scheduled_events` to the bucket object so
it can read the corresponding field written from the
autodetect process.

Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@3f35f867f0
2018-01-10 10:57:09 +00:00
Lee Hinman 0865063740 SQL: Fix multi-line CLI commands to join correctly (elastic/x-pack-elasticsearch#3516)
Previously multi-line CLI SQL statements were joined, but the space command was
missing, so a command like:

```
sql> SHOW
   | functions;
```

Would incorrectly parse as "showfunctions" and throw an error.

This fixes the behavior and adds a test for multi-line commands.

Resolves elastic/x-pack-elasticsearch#3410

Original commit: elastic/x-pack-elasticsearch@3870924ccd
2018-01-09 16:55:30 -07:00
Lisa Cawley 923428e19f [DOCS] Add links to Beats security pages (elastic/x-pack-elasticsearch#3514)
* [DOCS] Added link to new content location

* [DOCS] Add links to Beats security pages

Original commit: elastic/x-pack-elasticsearch@f54f0ef076
2018-01-09 13:33:53 -08:00
Jay Modi cbf1427ff4 Allow license installation with TLS disabled and single-node discovery (elastic/x-pack-elasticsearch#3427)
This change allows production licenses to be installed when TLS is not enabled and the discovery
type is set to single-node.

Relates elastic/x-pack-elasticsearch#3245

Original commit: elastic/x-pack-elasticsearch@73815a3976
2018-01-09 12:32:37 -07:00