Commit Graph

6069 Commits

Author SHA1 Message Date
Dimitris Athanasiou bd06a7b9b4 [ML] Do not write JSON records when metric value is not finite (elastic/x-pack-elasticsearch#1849)
relates elastic/x-pack-elasticsearch#1847

Original commit: elastic/x-pack-elasticsearch@26a69b840f
2017-06-26 14:45:00 +01:00
Alexander Reelsen 3db3cd0f0b Watcher: Reduce logging noise if no local watcher shards exist (elastic/x-pack-elasticsearch#1846)
Every cluster state update resulted in a log message, that watcher
pauses execution. This has been fixed to only log, if there was an
actual state switch from executing to pausing, but do nothing if
there are no local shards anyway.

This will reduce the logging noise in tests a lot.

Original commit: elastic/x-pack-elasticsearch@32ab86610c
2017-06-26 12:49:08 +02:00
David Kyle 232d59b855 [ML] Give kill a chance to stop autodetect before closing input (elastic/x-pack-elasticsearch#1824)
* Give kill a chance to kill the process before closing input

* Remove variable that can be refactored out

Original commit: elastic/x-pack-elasticsearch@42f7a3cece
2017-06-26 10:17:44 +01:00
Colin Goodheart-Smithe 27aa3094f6 Adds GraphExploreRequest as a remote index request (elastic/x-pack-elasticsearch#1836)
The graph API needs to be able to search in remote indices. Although it uses the Search API to perform the search and so doesn’t need to deal with remote indexes directly, the security feature needs to know it can be used with remote indexes so it knows to include remote indices in the list of indices accessible from the API for index level security

Original commit: elastic/x-pack-elasticsearch@e3cd84963e
2017-06-26 09:59:00 +01:00
Tim Vernum a73be456ec [Security] Don't apply local restrictions on remote cluster search (elastic/x-pack-elasticsearch#1609)
This change removes all local security checks against remote cluster names.
Any user is allowed to attempt a cross-cluster search, and it is the responsibility of the remote cluster to authorise the search (or not).

This includes support for remote searches even if you have _no_ local search privileges.

Original commit: elastic/x-pack-elasticsearch@1620c3a8fa
2017-06-26 12:30:31 +10:00
Nik Everett 872badccb4 Fix docs build
We committed broken examples and I don't know how to fix them
so I'll disable them for now.

Original commit: elastic/x-pack-elasticsearch@5f127e3e8e
2017-06-23 17:48:13 -04:00
lcawley 29873a37e9 [DOCS] Add transforms to build.gradle
Original commit: elastic/x-pack-elasticsearch@f8fa32f444
2017-06-23 13:50:41 -07:00
lcawley 1ee5c32620 [DOCS] Remove datafeedresource from build.gradle
Original commit: elastic/x-pack-elasticsearch@4bb411f693
2017-06-23 12:07:30 -07:00
Lisa Cawley 2b1d5f2cf2 [DOCS] Add ML configuration details and examples for script fields (elastic/x-pack-elasticsearch#1593)
* [DOCS] Add ML info about script fields

* [DOCS] Add links to ML script fields page

* [DOCS] Add ML API examples to transforms.asciidoc

* [DOCS] Addressed feedback in ML script field examples

* [DOCS] Add preview to ML script fields example

* [DOCS] Expanded code snippets in ML transform examples

* [DOCS] Add output for ML scripted fields example

* [DOCS] Add output for more ML scripted field examples

* [DOCS] Add output for final ML scripted field examples

* [DOC] Add Kibana details for ML script fields

* [DOCS] Remove example from ML transforms

Original commit: elastic/x-pack-elasticsearch@51057b029f
2017-06-23 11:42:37 -07:00
lcawley 431118b123 [DOCS] Add links to population analysis
Original commit: elastic/x-pack-elasticsearch@52e1cd36e2
2017-06-23 11:24:47 -07:00
Lisa Cawley 527b789e6f [DOCS] Add configuration information for population analysis (elastic/x-pack-elasticsearch#1653)
* [DOCS] Add configuration information for population analysis

* [DOCS] Add ML population analysis examples

* [DOCS] Address feedback for population analysis

* [DOCS] More feedback on population analysis

Original commit: elastic/x-pack-elasticsearch@ffa2bfeed9
2017-06-23 10:53:16 -07:00
Lisa Cawley 00c40c8299 [DOCS] Remove ML validate APIs (elastic/x-pack-elasticsearch#1801)
Original commit: elastic/x-pack-elasticsearch@fb11c1ad49
2017-06-23 08:55:31 -07:00
Dimitris Athanasiou 701dc53c2a [ML] Refactor aggregation response handling to make more flexible (elastic/x-pack-elasticsearch#1795)
Currently, aggregated datafeeds construct JSON from the aggregation
response by traversing all nested aggregations. In order to
achieve this, multiple leaf aggregations are not supported. Also,
scenarios it makes it impossible to effectively use pipeline
aggregations as it will not ignore the intermediate bucket
aggregations.

This commit refactors AggregationToJsonProcessor in order to
support the above scenarios. This is achieved by only converting
the fields of interest, that is the job analysis fields.

Original commit: elastic/x-pack-elasticsearch@8b575956ca
2017-06-23 12:36:32 +01:00
Alexander Reelsen efc93c7246 Tests: Increase loglevel, deactivate ML/monitoring to unclutter logs
Relates elastic/x-pack-elasticsearch#1807

Original commit: elastic/x-pack-elasticsearch@a958f32a3c
2017-06-23 10:40:31 +02:00
Deb Adair 7e1d8d5750 [DOCS] Added Kibana CCS section from x-pack-kibana.
Original commit: elastic/x-pack-elasticsearch@1504efc452
2017-06-22 19:48:29 -07:00
Tim Vernum 5c157fcd7b [TESTS] Fix potential for negative sleep (elastic/x-pack-elasticsearch#1818)
Original commit: elastic/x-pack-elasticsearch@cb19908a61
2017-06-23 11:45:09 +10:00
Andy Bristol 855c63dbc7 User/role names can be longer with more characters (elastic/x-pack-elasticsearch#1745)
This changes the validation criteria we use for user and role
names in the file realm, native realm, and the
realm-agnostic code in x-pack security. The new criteria is:

A valid username's length must be at least 1 and no more than 1024
characters. It may not contain leading or trailing whitespace. All
characters in the name must be be alphanumeric (`a-z`, `A-Z`, `0-9`),
printable punctuation or symbols in the https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)[Basic Latin (ASCII) block],
or the space character.

Original commit: elastic/x-pack-elasticsearch@f77640f269
2017-06-22 13:05:56 -07:00
Simon Willnauer 2fef2c72eb Simplify SSL server configuratin validation (elastic/x-pack-elasticsearch#1826)
Today we have some hidden complexity related to default configurations
might specify NO_KEY which is in some cases valid for server configuration.
This change removes the leniencey paramenters on the validation methods and removes
obsolet asserts.

Original commit: elastic/x-pack-elasticsearch@17ed4b1d20
2017-06-22 21:48:51 +02:00
David Kyle 12eec0e911 [ML] Wait for job to close in test (elastic/x-pack-elasticsearch#1825)
Original commit: elastic/x-pack-elasticsearch@3dbeed58f3
2017-06-22 17:21:16 +01:00
lcawley fc2f68609d [DOCS] Update links to X-Pack settings
Original commit: elastic/x-pack-elasticsearch@365c911cb1
2017-06-22 09:12:02 -07:00
lcawley 3844be33e7 [DOCS] Add link to Graph Explore API
Original commit: elastic/x-pack-elasticsearch@eaf68a6e86
2017-06-22 08:50:56 -07:00
Jay Modi 940722d5fe Document settings for the token service (elastic/x-pack-elasticsearch#1381)
relates elastic/x-pack-elasticsearch#1345

Original commit: elastic/x-pack-elasticsearch@87234ad866
2017-06-22 08:45:14 -06:00
Alexander Reelsen 404cac7b6b Tests: Disabling another failing OpenLDAP test
Relates elastic/x-pack-elasticsearch#1823

Original commit: elastic/x-pack-elasticsearch@88ee3d3eac
2017-06-22 16:16:50 +02:00
Alexander Reelsen f105cc0755 Tests: Added missing handlers for KnownActionsTest
Added in core in https://github.com/elastic/elasticsearch/pull/24841

Original commit: elastic/x-pack-elasticsearch@c70eb073ac
2017-06-22 15:41:07 +02:00
Alexander Reelsen 2147e2aa4f Tests: Another OpenLDAP tests that needs to be muted for now
Relates elastic/x-pack-elasticsearch#1823

Original commit: elastic/x-pack-elasticsearch@406eee05fe
2017-06-22 15:26:07 +02:00
Alexander Reelsen 217ac5bac5 Tests: Further muting of OpenLDAP dependent tests
Relates elastic/x-pack-elasticsearch#182

Original commit: elastic/x-pack-elasticsearch@10826c4c20
2017-06-22 14:41:40 +02:00
Alexander Reelsen 8666f96b62 Tests: Muting OpenLDAP tests due to network issues
Original commit: elastic/x-pack-elasticsearch@c32d397a20
2017-06-22 14:18:01 +02:00
Dimitris Athanasiou 0399be6406 [ML] Respect requested start time when datafeed is resumed (elastic/x-pack-elasticsearch#1822)
This commit fixes a bug where if a datafeed is resumed with `now`
as the `start` time, the datafeed still pulls all data from last
seen record.

relates elastic/x-pack-elasticsearch#1821

Original commit: elastic/x-pack-elasticsearch@ff066c0e04
2017-06-22 12:13:42 +01:00
Adrien Grand bd973aaadb Upgrade to lucene-7.0.0-snapshot-ad2cb77. (elastic/x-pack-elasticsearch#1820)
Sibling of elastic/elasticsearch#25349.

Original commit: elastic/x-pack-elasticsearch@2ab35a91e5
2017-06-22 12:35:51 +02:00
David Kyle 2eb939842d [ML] More informative logging for condition that prevents job creation (elastic/x-pack-elasticsearch#1811)
Original commit: elastic/x-pack-elasticsearch@fb223d3406
2017-06-22 09:48:48 +01:00
Lisa Cawley 5dfccb0d64 [DOCS] Drafted graph explore API page (elastic/x-pack-elasticsearch#1802)
* [DOCS] Drafted graph explore API page

* [DOCS] Remove Graph from X-Pack settings

Original commit: elastic/x-pack-elasticsearch@be59018bc7
2017-06-21 15:51:01 -07:00
Deb Adair 4367adffe6 [DOCS] Fixed typo.
Original commit: elastic/x-pack-elasticsearch@e09feb4863
2017-06-21 14:29:13 -07:00
Lisa Cawley 4b03566726 [DOCS] Fix edit_url for Elasticsearch Ref (elastic/x-pack-elasticsearch#1814)
Original commit: elastic/x-pack-elasticsearch@170a9c91c2
2017-06-21 10:33:11 -07:00
Chris Earle 0c7e802704 [Monitoring] Allow Cluster Alerts to be disabled in LocalExporter (elastic/x-pack-elasticsearch#1808)
You can now disable cluster alerts in the local exporter, which you can do in the HTTP exporter already.

This helps users that mess up their watcher configuration (e.g., disabling scripts) can turn off the feature to avoid log spam.

Original commit: elastic/x-pack-elasticsearch@f2096b553d
2017-06-21 12:46:28 -04:00
Chris Earle 1f59efa6a7 [Monitoring] Cluster Alert should reference cluster_stats in 5.5+ (elastic/x-pack-elasticsearch#1812)
The cluster alert did not get updated when we dropped the logical 'type' for cluster_state in favor of merging it with cluster_stats in 5.5+.

Original commit: elastic/x-pack-elasticsearch@c7105be36f
2017-06-21 12:02:51 -04:00
Simon Willnauer 9e5a8f3df5 Ensure transport impls read SSLConfiguration early (elastic/x-pack-elasticsearch#1797)
Today we parse / construct SSLConfiguration late when client / server
channels are created. This is problematic if we try to read from secure settings
in the future since we need to read all secure settings as soon as the node is
constructed. If we keep on reading late, we will access a closed keystore
since channel creation happens during node startup.

Original commit: elastic/x-pack-elasticsearch@78d6061990
2017-06-21 16:58:20 +02:00
Alexander Reelsen 889ee11c83 Watcher: Do not start, if index meta data version is wrong (elastic/x-pack-elasticsearch#1770)
If the internal index version of an index is not the right one, do
not start watcher.

Also, add the internal index version of 6 to all our index templates.

Original commit: elastic/x-pack-elasticsearch@20b50aa82b
2017-06-21 13:51:36 +02:00
Dimitris Athanasiou c661ee0934 [ML] Flush job even when datafeed retrieves no data (elastic/x-pack-elasticsearch#1786)
In order to detect anomalies due to the absence of data we
need to flush the job even if the datafeed retrieves no data.

relates elastic/x-pack-elasticsearch#1794

Original commit: elastic/x-pack-elasticsearch@645e393976
2017-06-21 10:37:29 +01:00
David Roberts a8e394c3b5 [ML] Fix race condition between multiple job deletion and immediate recreation (elastic/x-pack-elasticsearch#1793)
If multiple job deletion requests were sent in quick succession, there was a
race condition that meant they could both get through the check to enforce
one active deletion request at a time.  Then, if the job was immediately
recreated after the first deletion request returned, the second, still running,
deletion request could interfere with it and delete the aliases that the put
job request created.

This problem can be avoided by using the "ask forgiveness, not permission"
idiom when checking if the job is already being deleted at the beginning of
each deletion request.

Additionally, now even force delete requests will wait for a certain amount
of time for a prior delete request to complete.  This is to avoid the same
race conditions.  However, force delete requests will eventually start an
(unsafe) parallel delete to provide a get-out in case a delete request
completely dies.

relates elastic/x-pack-elasticsearch#1765

Original commit: elastic/x-pack-elasticsearch@b5c8f26a0e
2017-06-21 09:14:51 +01:00
David Kyle 410b210736 [ML] Retry after SearchPhaseExecutionException in ScrollDataExtractor (elastic/x-pack-elasticsearch#1788)
Original commit: elastic/x-pack-elasticsearch@bbe287b9c3
2017-06-21 08:55:09 +01:00
Lisa Cawley 8103a1bf8b [DOCS] Fix broken links (elastic/x-pack-elasticsearch#1799)
Original commit: elastic/x-pack-elasticsearch@7ff7ff1dbc
2017-06-20 17:41:16 -07:00
Lisa Cawley 1b3b7f2944 [DOCS] Update links in X-Pack Reference for Reporting info (elastic/x-pack-elasticsearch#1584)
* [DOCS] Update links to Reporting docs

* [DOCS] Fix incomplete link

Original commit: elastic/x-pack-elasticsearch@c3ccafb191
2017-06-20 15:38:27 -07:00
Lisa Cawley fe0c5cd3dc [DOCS] Update doc build info in README
Original commit: elastic/x-pack-elasticsearch@c93986436c
2017-06-20 11:49:15 -07:00
Alexander Reelsen 1e7f61b4c8 Tests: Ensure watcher index templates are installed in REST tests (elastic/x-pack-elasticsearch#1784)
The current testing setup only checked if watcher was started, but it
also needs to check for the index template in order to be sure that
everything is set up correctly, before trying to put a watch.

relates elastic/x-pack-elasticsearch#1762

Original commit: elastic/x-pack-elasticsearch@3ed78b15a1
2017-06-20 14:17:36 +02:00
David Roberts 96927cc1b6 [ML] Handle failures in the idiomatic way (elastic/x-pack-elasticsearch#1785)
This commit changes a couple of places where our ExceptionsHelper
class was throwing exceptions to instead return the exceptions.
Then they can be passed to onFailure() methods or thrown depending
on what's appropriate for the caller.  This is the standard Elastic
way of handling failures.

Original commit: elastic/x-pack-elasticsearch@fce07eb075
2017-06-20 13:06:28 +01:00
Alexander Reelsen d6254c9fd3 Watcher: Add proxy support for reporting attachment action (elastic/x-pack-elasticsearch#1781)
This allows to configure a proxy for the reporting attachment
action. The proxy is used by the HTTP client.

Original commit: elastic/x-pack-elasticsearch@87b6ab1b68
2017-06-20 13:49:32 +02:00
Alexander Reelsen 6e6629bd18 Tests: Log HTTP requests/response for external hipchat tests
Relates elastic/x-pack-elasticsearch#1774

Original commit: elastic/x-pack-elasticsearch@bdd86236e2
2017-06-20 10:47:42 +02:00
Boaz Leskes 27731eb523 remove NOTCONSOLE markers from installing-xes.asciidoc
It can only be used in shells that actually use curl

Original commit: elastic/x-pack-elasticsearch@b6bf9a98e0
2017-06-20 09:59:38 +02:00
lcawley 28bebe48bf [DOCS] Fix links to ML APIs
Original commit: elastic/x-pack-elasticsearch@406c9ace57
2017-06-19 20:28:56 -07:00
Lisa Cawley 0e133ada9e [DOCS]Fix links to ML APIs (elastic/x-pack-elasticsearch#1779)
Original commit: elastic/x-pack-elasticsearch@81be1e49b6
2017-06-19 19:31:39 -07:00