Commit Graph

364 Commits

Author SHA1 Message Date
Martijn van Groningen 372cb7c964 Let the scheduler use the data transport action instead of directly using autodetect process manager.
Original commit: elastic/x-pack-elasticsearch@2442e222fd
2016-12-09 09:41:04 +01:00
David Roberts 65f03a8888 Use well-defined IDs for records and influencers (elastic/elasticsearch#510)
* Use well-defined IDs for records and influencers

Removes the reliance on ES autogenerated UUIDs for all types that will
be renormalized

* Address some review comments

Original commit: elastic/x-pack-elasticsearch@85fde8b957
2016-12-08 18:39:03 +00:00
David Roberts 256ab7f3e2 Fix another misuse of ParameterizedMessage
Original commit: elastic/x-pack-elasticsearch@9e36900432
2016-12-08 17:32:59 +00:00
David Roberts 3f460b030e Fix a couple of ParameterizedMessage mistakes
Original commit: elastic/x-pack-elasticsearch@0006755c8b
2016-12-08 16:56:49 +00:00
Zachary Tong 8fa17f7ea8 Add mutually-exclusive param validation to GetBuckets (elastic/elasticsearch#497)
Related elastic/elasticsearch#430

Original commit: elastic/x-pack-elasticsearch@2579efca46
2016-12-08 10:28:30 -05:00
Zachary Tong b594e93a5d Remove SingleDocument as it is now unused (elastic/elasticsearch#498)
Closes elastic/elasticsearch#314

Original commit: elastic/x-pack-elasticsearch@88d2d20bdc
2016-12-08 10:28:12 -05:00
David Kyle a55944b284 Make ModelDebugOutput a result type (elastic/elasticsearch#484)
* Make ModelDebugOutput a result type

* Delete unused ElasticsearchBatchedModelDebugOutputIterator

* Add result_type field to ModelDebugOutput

* Address review comments

Original commit: elastic/x-pack-elasticsearch@a48e4cd946
2016-12-08 15:05:06 +00:00
Colin Goodheart-Smithe 581b1be217 Updates aws credential variables in upload task
Original commit: elastic/x-pack-elasticsearch@4fc9f02392
2016-12-08 14:58:15 +00:00
David Kyle d18daf0b3d Make ModelSizeStats a result type (elastic/elasticsearch#500)
* Make ModelSizeStats a result type

* Address review comments

Original commit: elastic/x-pack-elasticsearch@74de7b36d8
2016-12-08 14:44:59 +00:00
Martijn van Groningen 9c2c831996 Hooked scheduler into task api infrastructure.
The start scheduler api call will run until the scheduler has completed. Either when lookback only scheduler completes or the scheduler has been stopped.
 The start scheduler api will first update the scheduler status from STOPPED to STARTED on master node and then start running the scheduler.
 Once a scheduled job completes it updates the scheduler status from STARTED to STOPPED and then the start schedule api returns.
 The STARTING and STOPPING statuses are no longer used, so have been removed.
 The stop scheduler api is a sugar api that uses the task list and cancel apis stop the scheduler.
 Renamed ScheduledJobService to ScheduledJobRunner

Original commit: elastic/x-pack-elasticsearch@ab504fe3d9
2016-12-08 14:37:55 +01:00
Colin Goodheart-Smithe 037392ddd4 remove debug lines from build
Original commit: elastic/x-pack-elasticsearch@7766e171b9
2016-12-08 13:35:03 +00:00
Colin Goodheart-Smithe 3eb3b7f38d change aws cred properties
Changing properties to `PRELERT_AWS_ACCESS_KEY_ID` and `PRELERT_AWS_SECRET_ACCESS_KEY` to avoid conflicts on the ci servers

Original commit: elastic/x-pack-elasticsearch@2b90e752f1
2016-12-08 13:27:03 +00:00
Colin Goodheart-Smithe 2f3e5af046 more debug lines
Original commit: elastic/x-pack-elasticsearch@f653ce3822
2016-12-08 13:05:29 +00:00
Colin Goodheart-Smithe 9218cd4531 temporary debug lines in build to debug CI failures
Original commit: elastic/x-pack-elasticsearch@524fa8b5b8
2016-12-08 12:51:34 +00:00
Martijn van Groningen c8bda6b1d9 Collapsed JobProvider and JobResultsProvider and renamed ElasticsearchJobProvider to JobProvider
Original commit: elastic/x-pack-elasticsearch@05d5969d73
2016-12-08 10:53:31 +01:00
Colin Goodheart-Smithe 631ff9386a Changes AWS property names to be in line with the Elastic standard names (elastic/elasticsearch#499)
Properties are now `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`

Original commit: elastic/x-pack-elasticsearch@5947dc5ff1
2016-12-08 09:34:05 +00:00
Dimitris Athanasiou 4990195c90 Convert field references in error messages to snake case (elastic/elasticsearch#495)
* Convert field references in error messages to snake case

Closes elastic/elasticsearch#493

Original commit: elastic/x-pack-elasticsearch@38f4cd91bc
2016-12-08 09:32:10 +00:00
Zachary Tong 4d86670772 Make jobId and from/size mutually exclusive options (elastic/elasticsearch#477)
Make jobId and from/size mutually exclusive options.  

This approach has the main properties of not allowing an invalid Request to be built, and alerting the user if they set an incorrect configuration. It has the downside that PageParams can be null so the consumer will have to check for it. Since jobId could be null before, this seemed acceptable.

Original commit: elastic/x-pack-elasticsearch@106dcdf61a
2016-12-07 12:48:33 -05:00
Zachary Tong 3c711e6dff Add delete_list endpoint (elastic/elasticsearch#409)
Adds a delete_list endpoint.  If a list is currently in use by a job, it is not allowed to be deleted

Original commit: elastic/x-pack-elasticsearch@7d9a984b3a
2016-12-07 12:40:12 -05:00
David Kyle af61a51e22 Fix results not being persisted (elastic/elasticsearch#489)
Original commit: elastic/x-pack-elasticsearch@d0ee02ccf6
2016-12-07 16:52:58 +00:00
Martijn van Groningen 7cc2b8c5ce create allocation when the job has been created instead of creating it when opening the job.
This avoids the confusing situation that a there is no allocation when a job hasn't been opened yet. Now it complains about the fact that the job status is closed.

Original commit: elastic/x-pack-elasticsearch@3159dc6954
2016-12-07 17:15:18 +01:00
David Kyle ccf8cb7e0d Refactor delete interim results (elastic/elasticsearch#470)
* Collapse ElasticsearchBulkDeleter into JobDataDeleter

* Add blocking delete to JobDataDeleter

* Delete interim results only after all the results are parsed.

* Remove unused deleteModelSizeStats and deleteModelDebugOutput methods.

Document missing javadoc tags

Original commit: elastic/x-pack-elasticsearch@1997541673
2016-12-07 11:23:27 +00:00
Martijn van Groningen 14f43af818 moved metric part to be the suffix of the path
Original commit: elastic/x-pack-elasticsearch@6fc1b861ed
2016-12-07 12:09:16 +01:00
Martijn van Groningen e396e8aa68 Changes rest actions with data namespace into the job namespace.
Also added `_status` suffic to get job api as it would otherwise clash with open/close/flush APIs.

Original commit: elastic/x-pack-elasticsearch@6e8ef0ef7d
2016-12-07 12:09:16 +01:00
Martijn van Groningen d807eda9ed replaced custom ClosableIterator with Stream
Stream is closable, which is the reason ClosableIterator was introduced.

Original commit: elastic/x-pack-elasticsearch@b5a4a37e9e
2016-12-07 12:07:59 +01:00
Martijn van Groningen 5812ef4a86 use ActionListener.wrap(...) for delegate wrappers
Original commit: elastic/x-pack-elasticsearch@f1ccdb3f40
2016-12-07 10:23:47 +01:00
David Kyle 36d6141885 Move opening braces on new line to the end of the previous line (elastic/elasticsearch#473)
Original commit: elastic/x-pack-elasticsearch@57aedab104
2016-12-06 13:05:15 +00:00
Martijn van Groningen 570cde7a6a Added open and close job APIs.
* A job now has the following statuses: OPENING, OPENED, CLOSING, CLOSED and FAILED.
* The open job and close job APIs wait until the job gets into a OPENED or CLOSED state.
* The post data api no longer lazily opens a job and fails if the job has not been opened.
* When a job gets into a failed state also the reason is recorded in the allocation.
* Removed pause and resume APIs.
* Made `max_running_jobs` setting dynamically updatedable.

Original commit: elastic/x-pack-elasticsearch@3485ec5317
2016-12-06 13:51:26 +01:00
Dimitrios Athanasiou f960eea4b1 Remove obsolete comments in SchedulerConfig
Original commit: elastic/x-pack-elasticsearch@e7fc30bb2a
2016-12-06 12:45:10 +00:00
Dimitris Athanasiou 50df0d4326 Remove unnecessary parts of scheduler config (elastic/elasticsearch#468)
* Remove credentials from SchedulerConfig

* Remove dataSource and baseUrl from SchedulerConfig

Original commit: elastic/x-pack-elasticsearch@f5b92be252
2016-12-06 11:38:39 +00:00
Colin Goodheart-Smithe 3f35eac183 Adds ability to specify the aws credentials as environment variables
Original commit: elastic/x-pack-elasticsearch@c22428069e
2016-12-05 13:58:27 +00:00
David Kyle d3b4261759 Fix querying for model size stats (elastic/elasticsearch#465)
* Add test to read model size stats

* Most recent model_size_stats document should have the name ‘model_size_stats’

Original commit: elastic/x-pack-elasticsearch@e192d4c34d
2016-12-05 13:35:29 +00:00
David Kyle 2785cc727d Get records & categoryDefinitions with post body (elastic/elasticsearch#438)
* Allow POST with body to get records

* Allow records endpoint to accept POST requests with body

* CategoryDefinition can accept POST requests with body parameters

Original commit: elastic/x-pack-elasticsearch@2edb7a9c47
2016-12-05 11:46:12 +00:00
Colin Goodheart-Smithe a8d2cf16b9 Removes compile warnings (elastic/elasticsearch#463)
These warnings include:
* Resource not used
* Resource leak due to not being closed

Original commit: elastic/x-pack-elasticsearch@e0fb068a0c
2016-12-05 11:24:56 +00:00
David Roberts 98eb5534ee Camel to snake case (elastic/elasticsearch#461)
Original commit: elastic/x-pack-elasticsearch@222065cc4c
2016-12-05 10:34:41 +00:00
Martijn van Groningen b5c6970209 test: removed jackson databind leftover
Original commit: elastic/x-pack-elasticsearch@0cbd5595c8
2016-12-05 08:38:33 +01:00
David Kyle 850e43028b Make persist DataCounts a non-blocking operation (elastic/elasticsearch#447)
* Make persist DataCounts a non-blocking operation

* Add trace and debug level logging to the persist data counts action listener.

Remove dead code from test

Original commit: elastic/x-pack-elasticsearch@84bbfa880a
2016-12-02 17:44:53 +00:00
Dimitris Athanasiou f10c4818e7 Move count to the top of QueryPage (elastic/elasticsearch#452)
Original commit: elastic/x-pack-elasticsearch@8973bde039
2016-12-02 17:22:08 +00:00
David Kyle 385ec37bc3 Make the bulk results deleter non-blocking (elastic/elasticsearch#456)
* Make deleting old results a non blocking operation

* Add test for ElasticsearchBulkDeleter

Original commit: elastic/x-pack-elasticsearch@9fd9fb0b02
2016-12-02 17:16:16 +00:00
Colin Goodheart-Smithe d530edc263 Centralises where the version is defined
Original commit: elastic/x-pack-elasticsearch@e822136d97
2016-12-02 15:17:49 +00:00
David Roberts 3173f7a727 Remove redundant imports
Original commit: elastic/x-pack-elasticsearch@01123ea9fd
2016-12-02 14:27:24 +00:00
Zachary Tong 04445ce95f Throw 400 on unknown metrics in Job Stats (elastic/elasticsearch#446)
Closes elastic/elasticsearch#426

Original commit: elastic/x-pack-elasticsearch@86f136f5c0
2016-12-02 08:26:32 -05:00
Zachary Tong b6bdef474d Throw 403 instead of 429 when max job capacity full (elastic/elasticsearch#445)
Closes elastic/elasticsearch#444

Original commit: elastic/x-pack-elasticsearch@dfaf8dbec4
2016-12-02 08:26:19 -05:00
Colin Goodheart-Smithe 41bd367071 cenrealise group across all projects
Original commit: elastic/x-pack-elasticsearch@8a62d12149
2016-12-02 13:12:57 +00:00
Colin Goodheart-Smithe ea1c8b5ddd centralise detection of operating system
Original commit: elastic/x-pack-elasticsearch@0ba41dde0d
2016-12-02 12:18:51 +00:00
Colin Goodheart-Smithe 00dc347217 Build now downloads c++ dist for all platforms
Original commit: elastic/x-pack-elasticsearch@8fd9850377
2016-12-02 09:46:05 +00:00
Martijn van Groningen 1e5a12fc6a Move post data api over to task api.
Also made post data api cancellable.

Original commit: elastic/x-pack-elasticsearch@55360609de
2016-12-01 18:49:06 +01:00
Colin Goodheart-Smithe a82dc82439 remove debug code in build
Original commit: elastic/x-pack-elasticsearch@590a605c8a
2016-12-01 16:35:16 +00:00
Colin Goodheart-Smithe 8dc12d0f88 Gets ES plugin to download the cpp code if not built locally
Original commit: elastic/x-pack-elasticsearch@64cc4aeb38
2016-12-01 16:33:28 +00:00
Colin Goodheart-Smithe 19c1424984 fix build to not require AWS credentials
Original commit: elastic/x-pack-elasticsearch@74c7fb5c0f
2016-12-01 16:18:29 +00:00