* Redesign the get anomaly_detectors APIs
This commit redesigns the APIs to get anomaly_detectors.
The new design has 2 GET APIs:
- An API to get the configurations: /anomaly_detectors/{job_id}
- An API to get the stats: /anomaly_detectors/{job_id}/_stats
For both APIs entering "_all" as the job_id returns results for
all jobs.
Note that page params have been removed. They were useful
when the configs were stored in an index. Now that they are part
of cluster state there is no need. Additionally, future support
for wildcard job_id expressions will give users a tool to narrow
down the GET actions to a certain subset of jobs which will be
more useful than the from/size approach.
Follow up:
- Implement similar GET APIs for schedulers
- Remove scheduler_stats from the anomaly_detectors _stats API
as it will be part of the schedulers _stats API
Closeselastic/elasticsearch#548
Original commit: elastic/x-pack-elasticsearch@046a0db8f5
* Added the Elastic copyright header to C++ files
* Added the name of the copyright holder to Java files
Original commit: elastic/x-pack-elasticsearch@aea1b5a656
Aggs does not need to be a separate member field. There can simply
be an aggs parse field which also then stored onto the aggregations
parse field.
Finally, retrieve_whole_source is unnecessary as we move towards
a node client based data extraction.
Original commit: elastic/x-pack-elasticsearch@14024c2ee5
* snake_case model debug output fields
* Rename CategorizerState and BucketInfluencer types to snake_case
Original commit: elastic/x-pack-elasticsearch@da94dc7ec1
changed stop scheduler api to wait with returning a response until the scheduler status has been set to STOPPED.
Original commit: elastic/x-pack-elasticsearch@e20fcd1ae9
* Check use of mappings
* Add unit tests for JobProvider.createJobRelatedIndices
* Remove ‘index: no’ from mappings as no longer required
The entire type mapping has ‘enabled: false’
* Restore “index.analysis.analyzer.default.type” setting
* Remove include_in_all from nested mappings
* Add audit and usage mappings to the job index
* Revert ‘Restore “index.analysis.analyzer.default.type” setting’
Original commit: elastic/x-pack-elasticsearch@c7d62e0c7e
This builds on PR elastic/elasticsearch#526 to get normalization working end-to-end using the
native normalizer process.
The ShortCircuitingRenormalizer class is basically doing what the old
BlockingQueueRenormaliser class did but using the ES threadpool instead
of its own thread.
Also fixed a bug where the C++ was calling the score field of partition_score
documents normalized_probability but the Java was calling it anomaly_score.
Original commit: elastic/x-pack-elasticsearch@d4cecae150
These test are just checking response / status code. It is cleaner to have these tests as yaml tests.
Original commit: elastic/x-pack-elasticsearch@61c323059e
* Test for job existence before updating its state
* Add unit tests covering expected missing job exceptions
Original commit: elastic/x-pack-elasticsearch@bcd270dafd
* Added back Normalizable classes
* Added back normalization process management classes
* Added back the scores updater
Original commit: elastic/x-pack-elasticsearch@ac8edf6ed6
This also shuffles results under /anomaly_detectors/. Note: the cluster state still refers to
"jobs" which should probably be fixed in a separate PR
Original commit: elastic/x-pack-elasticsearch@c9e634621c
Removed SchedulerStats as scheduler status is all we need and start and end times are only needed in start scheduler api.
Original commit: elastic/x-pack-elasticsearch@80c563cb69
* Use well-defined IDs for records and influencers
Removes the reliance on ES autogenerated UUIDs for all types that will
be renormalized
* Address some review comments
Original commit: elastic/x-pack-elasticsearch@85fde8b957
* Make ModelDebugOutput a result type
* Delete unused ElasticsearchBatchedModelDebugOutputIterator
* Add result_type field to ModelDebugOutput
* Address review comments
Original commit: elastic/x-pack-elasticsearch@a48e4cd946
The start scheduler api call will run until the scheduler has completed. Either when lookback only scheduler completes or the scheduler has been stopped.
The start scheduler api will first update the scheduler status from STOPPED to STARTED on master node and then start running the scheduler.
Once a scheduled job completes it updates the scheduler status from STARTED to STOPPED and then the start schedule api returns.
The STARTING and STOPPING statuses are no longer used, so have been removed.
The stop scheduler api is a sugar api that uses the task list and cancel apis stop the scheduler.
Renamed ScheduledJobService to ScheduledJobRunner
Original commit: elastic/x-pack-elasticsearch@ab504fe3d9
Make jobId and from/size mutually exclusive options.
This approach has the main properties of not allowing an invalid Request to be built, and alerting the user if they set an incorrect configuration. It has the downside that PageParams can be null so the consumer will have to check for it. Since jobId could be null before, this seemed acceptable.
Original commit: elastic/x-pack-elasticsearch@106dcdf61a
Adds a delete_list endpoint. If a list is currently in use by a job, it is not allowed to be deleted
Original commit: elastic/x-pack-elasticsearch@7d9a984b3a
This avoids the confusing situation that a there is no allocation when a job hasn't been opened yet. Now it complains about the fact that the job status is closed.
Original commit: elastic/x-pack-elasticsearch@3159dc6954
* Collapse ElasticsearchBulkDeleter into JobDataDeleter
* Add blocking delete to JobDataDeleter
* Delete interim results only after all the results are parsed.
* Remove unused deleteModelSizeStats and deleteModelDebugOutput methods.
Document missing javadoc tags
Original commit: elastic/x-pack-elasticsearch@1997541673
* A job now has the following statuses: OPENING, OPENED, CLOSING, CLOSED and FAILED.
* The open job and close job APIs wait until the job gets into a OPENED or CLOSED state.
* The post data api no longer lazily opens a job and fails if the job has not been opened.
* When a job gets into a failed state also the reason is recorded in the allocation.
* Removed pause and resume APIs.
* Made `max_running_jobs` setting dynamically updatedable.
Original commit: elastic/x-pack-elasticsearch@3485ec5317
* Add test to read model size stats
* Most recent model_size_stats document should have the name ‘model_size_stats’
Original commit: elastic/x-pack-elasticsearch@e192d4c34d
* Allow POST with body to get records
* Allow records endpoint to accept POST requests with body
* CategoryDefinition can accept POST requests with body parameters
Original commit: elastic/x-pack-elasticsearch@2edb7a9c47
* Make persist DataCounts a non-blocking operation
* Add trace and debug level logging to the persist data counts action listener.
Remove dead code from test
Original commit: elastic/x-pack-elasticsearch@84bbfa880a
The changes the way the Elasticsearch plugin module gets the cpp code when running the bundlePlugin task. Instead of manually going into the cpp builds output and grabbing the folders it depends on the buildZip task of the :cpp project and grabs the resource that task builds. This means that it does not matter if the cpp buildZip task is changed as long as it results in a zip being generated.
Original commit: elastic/x-pack-elasticsearch@257e5df2e7
David Roberts
Dimitrios Athanasiou
David Kyle
Martijn van Groningen
Colin Goodheart-Smithe
Zachary Tong