Commit Graph

4979 Commits

Author SHA1 Message Date
Yannick Welsch 68bfc90cea Let ClusterStateObserver only hold onto state that's needed for change detection
Companion commit for elastic/elasticsearchelastic/elasticsearch#21631

Original commit: elastic/x-pack-elasticsearch@3333480707
2016-12-20 15:23:00 +01:00
Martijn van Groningen 999a1eb234 fix JobTests#testEquals_noId by fixing regex expression and including `-`
Original commit: elastic/x-pack-elasticsearch@b295d5f621
2016-12-20 15:15:47 +01:00
David Roberts 8b135226f6 Fix long line
Original commit: elastic/x-pack-elasticsearch@8d030d8cf8
2016-12-20 14:09:39 +00:00
David Kyle 537fea313f Add job config option to set the index name (elastic/elasticsearch#560)
* Add job config option to set the index name

* Check index does not already exist if ’index_name’ is set

* Don’t create alias if ‘index_name’ is the same as ‘job_id’

* Default index_name value

Set it to job_id if null and only create the index alias if job_id != index_name

* Fix compile errors after rebasing

* Address review comments

* Test if the index exists by checking the cluster state

* Update comment

Original commit: elastic/x-pack-elasticsearch@a3e7f1a5bb
2016-12-20 14:03:24 +00:00
David Roberts 3383c67cf2 Ensure automatically generated job IDs meet the validation criteria (elastic/elasticsearch#586)
Original commit: elastic/x-pack-elasticsearch@34b67cb509
2016-12-20 13:53:29 +00:00
Alexander Reelsen b994f16f4a Watcher: Ensure non duplicate Wid ids (elastic/elasticsearch#4423)
Removing the WatchLockService could result in duplication of wids, because of a wrong
call to replace underscores with dashes. As UUIDs.createBase64UUID() can contain underscores
but they are kind of reserved in the Wid class due to handling of watch ids, this just uses
the toString() representation of a random UUID.

Closes elastic/elasticsearch#4422

Original commit: elastic/x-pack-elasticsearch@dceb01ae5e
2016-12-20 14:02:00 +01:00
David Roberts 5b029ee595 Correct test condition and add a message in case of future failures
Original commit: elastic/x-pack-elasticsearch@55729b39fa
2016-12-20 11:22:03 +00:00
David Roberts 1615be4ff3 Moved the C++ code under the cpp sub-directory (elastic/elasticsearch#584)
The $PRELERT_SRC_HOME environment variable is replaced with $CPP_SRC_HOME,
which points to the cpp sub-directory off the repository root

Original commit: elastic/x-pack-elasticsearch@02ef6d6be6
2016-12-20 11:03:00 +00:00
David Kyle 78fdca65d5 Add job Id to state document IDs (elastic/elasticsearch#579)
* Persist quantile documents with the jobId in the document Id

* Add job Id to snapshot Id

* Add job Id to categoriser state document Id

* Rename quantiles doc to start with job id as the other state docs do

* Fix restoring categoriser state

Original commit: elastic/x-pack-elasticsearch@3e5d3368b5
2016-12-20 10:47:37 +00:00
Dimitrios Athanasiou 316f6bf6cb Remove unused methods and fix test
Original commit: elastic/x-pack-elasticsearch@12290517d0
2016-12-20 10:15:53 +00:00
Dimitris Athanasiou bee58f8b81 Add get API for schedulers (elastic/elasticsearch#582)
Original commit: elastic/x-pack-elasticsearch@59c555d60a
2016-12-20 10:09:45 +00:00
Daniel Mitterdorfer b9a38d9b97 Enable trace logging for LocalExporterTests#testSimpleExport()
This test fails due to a changing cluster state(?).  The
test checks that a local exporter is ready and then continues.
However, during the test, we see output similar to:

skipping exporter [_local] as it isn't ready yet

Which indicates that the cluster state has changed and the
exporter does not return a bulk anymore. Hence, the test is
failing although at one point in time it returned a bulk.

By enabling trace logging we should be able to find out
what's going on.

Original commit: elastic/x-pack-elasticsearch@d7e2200dd9
2016-12-20 08:54:26 +01:00
Ryan Ernst 789df7d2fa Internal: Update signatures for EnvironmentAwareCommand (elastic/elasticsearch#4367)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#22175

Original commit: elastic/x-pack-elasticsearch@4359cb1947
2016-12-19 15:23:53 -08:00
javanna adf22c3d24 Adapt to strict parsing mode removal
Original commit: elastic/x-pack-elasticsearch@2d6276a60f
2016-12-19 19:39:08 +01:00
Dimitrios Liappis 8a3181ceed [ci] Add jdk9 target to the dev-tools/ci script
To add support for jdk9 tests for x-pack we need a number of parameters
for the jvm, passed in the gradle command. To resolve issues with arg
passing through bash variables and improve readability, switch to an
array construct for storing the arguments.

Add new jdk9 target to allow execution of jdk-9-ea based tests in the CI
infra.

Relates: elastic/elasticsearch#4415 
Relates: https://github.com/elastic/infra/issues/1573#issuecomment-267832307

Original commit: elastic/x-pack-elasticsearch@b41e5f4b98
2016-12-19 19:53:00 +02:00
David Roberts 9b3764b7fa Max jobs per node cannot be dynamic due to threadpool (elastic/elasticsearch#578)
The threadpool that supplies the threads used for job IO cannot be
resized, so the number of jobs cannot be dynamic either

Original commit: elastic/x-pack-elasticsearch@c584bf7147
2016-12-19 17:48:22 +00:00
Alexander Reelsen 79a8f27569 Watcher: Remove WatchLockService (elastic/elasticsearch#4365)
The watch lock service is not really needed, as there is already
a data structure that has information about the currently executing
watches, that can be consulted before executed.

This change will now check, if there is already a watch running with
the current id. If there is not, execution will happen as usual. If
there is however, than a watch record will be created, stating that
the watch is currently being executed - which means that it is either
being executed or in the list of planned executions.

This way users can check in the watch history, if a watch has been executed
more often than it should.

In order to easily search for this, a new execution state called
`NOT_EXECUTED_ALREADY_QUEUED` has been added.

Original commit: elastic/x-pack-elasticsearch@867acec3c3
2016-12-19 17:33:48 +01:00
David Roberts f65125df55 No point rethrowing exception in thread pool task
It will just get logged twice, but with a less useful message the
second time

Original commit: elastic/x-pack-elasticsearch@7d32f65cc1
2016-12-19 16:24:45 +00:00
Dimitrios Athanasiou 081bebbbcd Remove unused constants
Original commit: elastic/x-pack-elasticsearch@0f72f1fc6e
2016-12-19 14:22:05 +00:00
Dimitris Athanasiou 8750e5f926 Redesign the get anomaly_detectors APIs (elastic/elasticsearch#570)
* Redesign the get anomaly_detectors APIs

This commit redesigns the APIs to get anomaly_detectors.

The new design has 2 GET APIs:

- An API to get the configurations: /anomaly_detectors/{job_id}
- An API to get the stats: /anomaly_detectors/{job_id}/_stats

For both APIs entering "_all" as the job_id returns results for
all jobs.

Note that page params have been removed. They were useful
when the configs were stored in an index. Now that they are part
of cluster state there is no need. Additionally, future support
for wildcard job_id expressions will give users a tool to narrow
down the GET actions to a certain subset of jobs which will be
more useful than the from/size approach.

Follow up:

- Implement similar GET APIs for schedulers
- Remove scheduler_stats from the anomaly_detectors _stats API
as it will be part of the schedulers _stats API

Closes elastic/elasticsearch#548

Original commit: elastic/x-pack-elasticsearch@046a0db8f5
2016-12-19 13:09:54 +00:00
Daniel Mitterdorfer 2cb41e690d Address initial review comments
Original commit: elastic/x-pack-elasticsearch@bcd6433afb
2016-12-19 13:27:40 +01:00
Daniel Mitterdorfer 24a4290379 Merge remote-tracking branch 'origin/master' into strict-booleans
Original commit: elastic/x-pack-elasticsearch@32a533f38f
2016-12-19 12:58:30 +01:00
David Kyle 4eda09d24a Don’t write job description if not set
Original commit: elastic/x-pack-elasticsearch@6e7ff939d3
2016-12-19 11:27:32 +00:00
David Roberts 854dc88cbd Corrections to previous commit
Original commit: elastic/x-pack-elasticsearch@a9e94119e1
2016-12-19 10:28:23 +00:00
David Roberts 028a505fe5 Update copyright headers (elastic/elasticsearch#572)
* Added the Elastic copyright header to C++ files

* Added the name of the copyright holder to Java files

Original commit: elastic/x-pack-elasticsearch@aea1b5a656
2016-12-19 10:19:21 +00:00
Dimitris Athanasiou 6a14f52d2e Remove aggs and retrieve_whole_source from SchedulerConfig (elastic/elasticsearch#569)
Aggs does not need to be a separate member field. There can simply
be an aggs parse field which also then stored onto the aggregations
parse field.

Finally, retrieve_whole_source is unnecessary as we move towards
a node client based data extraction.

Original commit: elastic/x-pack-elasticsearch@14024c2ee5
2016-12-19 09:54:59 +00:00
David Roberts 89e506f050 Merge branch 'analytics-dev' into master
Original commit: elastic/x-pack-elasticsearch@dc9d10a502
2016-12-19 09:07:12 +00:00
Daniel Mitterdorfer 7a652fa090 Merge pull request elastic/elasticsearch#4398 from danielmitterdorfer/xcontent_strict_duplicate_checks
Enable strict duplicate checks for all XContent types

Original commit: elastic/x-pack-elasticsearch@ad249c2e65
2016-12-19 09:30:19 +01:00
Tim Vernum df7c528c66 Change classification of event_type in index audit trail (elastic/elasticsearch#4375)
In LoggingAuditTrail and the audit-event filtering, we distinguish between single-realm authentication failures ("realm_authentication_failure") and global failures ("authentication_failure").

Update the message output of IndexAuditTrail to reflect this distinction.

Original commit: elastic/x-pack-elasticsearch@56802ae2df
2016-12-19 14:39:21 +11:00
Simon Willnauer c9cfedc3cb [TEST] check if .watches index / alias exists before creating it
Original commit: elastic/x-pack-elasticsearch@74d85121eb
2016-12-18 10:17:43 +01:00
Areek Zillur 278496d5e5 x-pack changes for removing deprecated _suggest endpoint (elasticsearchelastic/elasticsearch#22203)
Original commit: elastic/x-pack-elasticsearch@b9bba20fae
2016-12-16 13:06:35 -05:00
David Kyle aacb6f3949 Move JobResultsPersister.getJobIndexName to new class AnomalyDetectorsIndex (elastic/elasticsearch#567)
Original commit: elastic/x-pack-elasticsearch@bf54dd56a9
2016-12-16 17:47:52 +00:00
Martijn van Groningen e3ec908828 run with native process by default
Original commit: elastic/x-pack-elasticsearch@c73beec522
2016-12-16 15:21:11 +01:00
Martijn van Groningen eaca8fb06a Stop local allocated jobs before closing node.
Original commit: elastic/x-pack-elasticsearch@ad3ed7c86a
2016-12-16 14:59:35 +01:00
David Roberts c43a9ba1dd Fix intermittent failure of renormalization (elastic/elasticsearch#558)
The synchronization was flawed

Original commit: elastic/x-pack-elasticsearch@a968c68c7d
2016-12-16 13:45:47 +00:00
Daniel Mitterdorfer d3c1fbe002 Enable strict duplicate checks for all XContent types
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default for all XContent types (not only JSON).

We have also changed the name of the system property to disable this feature
from `es.json.strict_duplicate_detection` to the now more appropriate name
`es.xcontent.strict_duplicate_detection`.

Relates elastic/elasticsearchelastic/elasticsearch#19614
Relates elastic/elasticsearchelastic/elasticsearch#22073

Original commit: elastic/x-pack-elasticsearch@c92e2a6c55
2016-12-16 13:57:55 +01:00
Colin Goodheart-Smithe 5e6c63bc51 Removes warnings
Original commit: elastic/x-pack-elasticsearch@59626a3a3a
2016-12-16 12:35:45 +00:00
Alexander Reelsen 0d9cb5f7bd Docs: Updating execution phases of watch execution
Closes elastic/elasticsearch#4079

Original commit: elastic/x-pack-elasticsearch@658ba0a7f7
2016-12-16 11:30:27 +01:00
Dimitris Athanasiou c932588c68 Repackage scheduler classes (elastic/elasticsearch#559)
* Repackage scheduler classes

* Rename and move scheduler status tests

Original commit: elastic/x-pack-elasticsearch@a9d7cc4e48
2016-12-16 09:50:06 +00:00
Daniel Mitterdorfer e3302da589 Make boolean conversion strict
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.

Original commit: elastic/x-pack-elasticsearch@6400f18911
2016-12-15 19:06:59 +01:00
Boaz Leskes b2972a142c Remove Cluster State Status (elastic/elasticsearch#4313)
A companion PR for https://github.com/elastic/elasticsearch/pull/21817

Original commit: elastic/x-pack-elasticsearch@392235877f
2016-12-15 17:07:02 +01:00
David Kyle 8d5c8b24d6 Make ModelDebugOutput fields snake case (elastic/elasticsearch#555)
* snake_case model debug output fields

* Rename CategorizerState and BucketInfluencer types to snake_case

Original commit: elastic/x-pack-elasticsearch@da94dc7ec1
2016-12-15 15:42:38 +00:00
David Kyle 8e14aee978 Audit message were being persisted with the type ‘audit_activity’ (elastic/elasticsearch#554)
Original commit: elastic/x-pack-elasticsearch@7c00495c98
2016-12-15 15:31:11 +00:00
Dimitris Athanasiou 8841a97659 Disallow non alphanumeric chars from start or end in IDs (elastic/elasticsearch#552)
Closes elastic/elasticsearch#550

Original commit: elastic/x-pack-elasticsearch@ca0ca9f4da
2016-12-15 13:49:07 +00:00
Dimitrios Athanasiou bc8d966690 Fix integration tests
Original commit: elastic/x-pack-elasticsearch@a5955cfde3
2016-12-15 12:03:51 +00:00
David Kyle b97610f245 Streamed boolean values can on be 0 or 1
Original commit: elastic/x-pack-elasticsearch@16ffa0727b
2016-12-15 12:01:09 +00:00
Alexander Reelsen e6ee905931 Watcher: Remove unused class
Original commit: elastic/x-pack-elasticsearch@ecd48b7914
2016-12-15 11:33:36 +01:00
Ryan Ernst 6d4d599f91 Build: Convert xplugins to use new extra projects setup (elastic/elasticsearch#4175)
* Build: Convert xplugins to use new extra projects setup

This change makes the gradle initialization for xplugins look in the
correct location for elasticsearch, which is now as a sibling of an
elasticsearch-extra directory, with x-plugins as a child of the extra
directory.

The elasticsearch side of this change is
elastic/elasticsearchelastic/elasticsearch#21773. This change will enable renaming x-plugins
to x-pack, see elastic/elasticsearch#3643.

Original commit: elastic/x-pack-elasticsearch@09398aea5a
2016-12-14 15:02:28 -08:00
Martijn van Groningen faf54d4952 renamed test classes
Original commit: elastic/x-pack-elasticsearch@5296de69d7
2016-12-14 19:52:10 +01:00
Martijn van Groningen 985bdf6cb1 Changed rest start scheduler api to wait with returning a response until the the scheduler status has been set to STARTED and
changed stop scheduler api to wait with returning a response until the scheduler status has been set to STOPPED.

Original commit: elastic/x-pack-elasticsearch@e20fcd1ae9
2016-12-14 19:50:02 +01:00