Commit Graph

50 Commits

Author SHA1 Message Date
Benjamin Trent 1ae2923632
[7.x] [ML] adding docs + hlrc for data frame analysis feature_processors (#61149) (#61493)
* [ML] adding docs + hlrc for data frame analysis feature_processors (#61149)

Adds HLRC and some docs for the new feature_processors field in Data frame analytics.

Co-authored-by: Przemysław Witek <przemyslaw.witek@elastic.co>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-08-24 12:56:21 -04:00
Lisa Cawley 2665bfffce
[DOCS] Fix security links in machine learning APIs (#60098) (#60152) 2020-07-23 16:43:10 -07:00
James Rodewig b302b09b85
[DOCS] Reformat snippets to use two-space indents (#59973) (#59994) 2020-07-21 15:49:58 -04:00
Lisa Cawley 8f8d24b3c1
[DOCS] Changes level offset in data frame analytics APIs (#59919) (#59923) 2020-07-20 13:06:29 -07:00
Dimitris Athanasiou b2243337d8
[7.x][ML] Data frame analytics max_num_threads setting (#59254) (#59308)
This adds a setting to data frame analytics jobs called
`max_number_threads`. The setting expects a positive integer.
When used the user specifies the max number of threads that may
be used by the analysis. Note that the actual number of threads
used is limited by the number of processors on the node where
the job is assigned. Also, the process may use a couple more threads
for operational functionality that is not the analysis itself.

This setting may also be updated for a stopped job.

More threads may reduce the time it takes to complete the job at the cost
of using more CPU.

Backport of #59254 and #57274
2020-07-09 19:15:46 +03:00
Lisa Cawley a1514c9ffe
[DOCS] Replaces docdir attributes in ML APIs (#57390) (#57467) 2020-06-01 13:46:15 -07:00
István Zoltán Szabó e1cab4feb4 [DOCS] Puts a link into the loss_function variable description (#56678) 2020-05-28 09:46:11 +02:00
Dimitris Athanasiou 75dadb7a6d
[7.x][ML] Add loss_function to regression (#56118) (#56187)
Adds parameters `loss_function` and `loss_function_parameter`
to regression.

Backport of #56118
2020-05-05 14:59:51 +03:00
István Zoltán Szabó 9bcc975bd1 [DOCS] Simplifies footnote text in DFA APIs (#56105)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-05-05 09:05:08 +02:00
Lisa Cawley b816ab0c18
[DOCS] Synchs and links hyperparameter descriptions (#56131) 2020-05-04 10:37:26 -07:00
Lisa Cawley 006e00ed0a
[DOCS] Adds documentation for secondary authorization headers (#55365) (#55986) 2020-04-29 16:29:38 -07:00
István Zoltán Szabó a5cf4712e5 [DOCS] Changes feature importance links to point to the new page (#55531)
* [DOCS] Changes feature importance links to point to the new page.

* [DOCS] Fixes line breaks.
2020-04-28 09:03:43 +02:00
Lisa Cawley c7cf6e621d [DOCS] Remove text fields from classification dependent variables (#54849) 2020-04-16 13:40:28 -07:00
Lisa Cawley 2910d01179
[DOCS] Removes unshared sections from ml-shared.asciidoc (#55192) 2020-04-14 18:47:09 -07:00
lcawl fcd96db006 [DOCS] Edits create data frame analytics job API (#54751) 2020-04-13 10:43:52 -07:00
István Zoltán Szabó 374f633b6e [DOCS] Adds link points to the data frame analytics supported fields (#55004)
Co-authored-by: lcawl <lcawley@elastic.co>
2020-04-09 11:27:57 -07:00
István Zoltán Szabó 4cba1e6368 [DOCS] Changes kibana_user to kibana_admin in DFA API prerequisites. (#54806) 2020-04-06 15:46:18 +02:00
Benjamin Trent 65233383f6
[7.x] [ML] prefer secondary authorization header for data[feed|frame] authz (#54121) (#54645)
* [ML] prefer secondary authorization header for data[feed|frame] authz (#54121)

Secondary authorization headers are to be used to facilitate Kibana spaces support + ML jobs/datafeeds.

Now on PUT/Update/Preview datafeed, and PUT data frame analytics the secondary authorization is preferred over the primary (if provided).

closes https://github.com/elastic/elasticsearch/issues/53801

* fixing for backport
2020-04-02 11:20:25 -04:00
Lisa Cawley 922ec8e961
[DOCS] Collapses nested objects in data frame analytics APIs (#54472) (#54526) 2020-03-31 12:51:04 -07:00
Tom Veasey 690099553c
[7.x][ML] Adds the class_assignment_objective parameter to classification (#53552)
Adds a new parameter for classification that enables choosing whether to assign labels to
maximise accuracy or to maximise the minimum class recall.

Fixes #52427.
2020-03-13 17:35:51 +00:00
Dimitris Athanasiou 0fd0516d0d
[7.x][ML] Rename data frame analytics maximum_number_trees to max_trees (#53300) (#53390)
Deprecates `maximum_number_trees` parameter of classification and
regression and replaces it with `max_trees`.

Backport of #53300
2020-03-11 12:45:27 +02:00
Lisa Cawley 123b3c6f55 [DOCS] Clarifies description of num_top_feature_importance_values (#52246)
Co-Authored-By: Valeriy Khakhutskyy <1292899+valeriy42@users.noreply.github.com>
2020-02-18 08:50:21 -08:00
Christoph Büscher d291f189a8
Fix hardcoded version replacement in put-dfanalytics.asciidoc #51053
The version replacement for the code snippet should replace 7.6 with the current version,
but doesn't match because of a missing whitespace.

Closes #51052
2020-01-15 18:09:37 +01:00
Dimitris Athanasiou 1d8cb3c741
[7.x][ML] Add num_top_feature_importance_values param to regression and classi… (#50914) (#50976)
Adds a new parameter to regression and classification that enables computation
of importance for the top most important features. The computation of the importance
is based on SHAP (SHapley Additive exPlanations) method.

Backport of #50914
2020-01-14 16:46:09 +02:00
István Zoltán Szabó 4f150e4961
[7.x][DOCS] Moves analysis resources to PUT DFA API docs (#50793) 2020-01-09 16:21:35 +01:00
István Zoltán Szabó 71afeec7d0 Revert "[DOCS] Moves analysis resources to PUT DFA API docs (#50704)"
This reverts commit 4e1107d5d7.
2020-01-09 14:31:35 +01:00
István Zoltán Szabó 4e1107d5d7 [DOCS] Moves analysis resources to PUT DFA API docs (#50704)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-01-09 14:13:37 +01:00
István Zoltán Szabó 0ac6786f41 [DOCS] Forms role and privilege requirements as bulleted lists in DFA API docs (#50732)
Co-Authored-By: Lisa Cawley <lcawley@elastic.co>
2020-01-09 10:45:18 +01:00
István Zoltán Szabó 7611b3c9be
[7.x][DOCS] Moves data frame analytics job resource definitions into APIs (#50165)
* [7.x][DOCS] Moves data frame analytics job resource definitions into APIs.
2019-12-13 11:48:21 +01:00
Dimitris Athanasiou 8891f4db88
[7.x][ML] Introduce randomize_seed setting for regression and classification (#49990) (#50023)
This adds a new `randomize_seed` for regression and classification.
When not explicitly set, the seed is randomly generated. One can
reuse the seed in a similar job in order to ensure the same docs
are picked for training.

Backport of #49990
2019-12-10 15:29:19 +02:00
István Zoltán Szabó f4b3bb7d6b [DOCS] Adds an example of preprocessing actions to the PUT DFA API docs (#49831) 2019-12-05 14:16:38 +01:00
Dimitris Athanasiou 4edb2e7bb6
[7.x][ML] Add optional source filtering during data frame reindexing (#49690) (#49718)
This adds a `_source` setting under the `source` setting of a data
frame analytics config. The new `_source` is reusing the structure
of a `FetchSourceContext` like `analyzed_fields` does. Specifying
includes and excludes for source allows selecting which fields
will get reindexed and will be available in the destination index.

Closes #49531

Backport of #49690
2019-11-29 16:10:44 +02:00
lcawl a42003b95b [DOCS] Fixes data type formatting 2019-11-26 08:22:50 -08:00
István Zoltán Szabó c2f52015d3 [DOCS] Removes best practice about fields that are highly correlated to the dependent variable. (#48935) 2019-11-11 16:01:21 +01:00
István Zoltán Szabó 91888959e8 [DOCS] Extends analyzed_fields description in PUT DFA API docs. (#48307) 2019-11-11 15:55:12 +01:00
István Zoltán Szabó 3c9bd13dca [DOCS] Adds classification type DFA API docs and ml-shared.asciidoc (#48241) 2019-11-06 07:41:38 -05:00
David Roberts 984323783e
[ML][7.x] Add lazy assignment job config option (#47993)
This change adds:

- A new option, allow_lazy_open, to anomaly detection jobs
- A new option, allow_lazy_start, to data frame analytics jobs

Both work in the same way: they allow a job to be
opened/started even if no ML node exists that can
accommodate the job immediately. In this situation
the job waits in the opening/starting state until ML
node capacity is available. (The starting state for data
frame analytics jobs is new in this change.)

Additionally, the ML nightly maintenance tasks now
creates audit warnings for ML jobs that are unassigned.
This means that jobs that cannot be assigned to an ML
node for a very long time will show a yellow warning
triangle in the UI.

A final change is that it is now possible to close a job
that is not assigned to a node without using force.
This is because previously jobs that were open but
not assigned to a node were an aberration, whereas
after this change they'll be relatively common.
2019-10-15 06:55:11 +01:00
István Zoltán Szabó 9eac8bf2a8 [DOCS] Adds supported fields section to the PUT DFA API description (#47842) 2019-10-10 12:42:54 +02:00
István Zoltán Szabó 6f4b7e9a7f [DOCS] Extends the analyzed_fields description in the PUT DFA API docs (#47791) 2019-10-09 18:14:58 +02:00
Lisa Cawley 39ef795085
[DOCS] Cleans up links to security content (#47610) (#47703) 2019-10-07 15:23:19 -07:00
Dimitris Athanasiou 7667ea5f6f
[7.x][ML] Additional outlier detection parameters (#47600) (#47669)
Adds the following parameters to `outlier_detection`:

- `compute_feature_influence` (boolean): whether to compute or not
   feature influence scores
- `outlier_fraction` (double): the proportion of the data set assumed
   to be outlying prior to running outlier detection
- `standardization_enabled` (boolean): whether to apply standardization
   to the feature values

Backport of #47600
2019-10-07 18:21:33 +03:00
István Zoltán Szabó 033aa9cf9b [DOCS] Adds examples to the PUT dfa and the evaluate dfa APIs (#46966)
* [DOCS] Adds examples to the PUT dfa and the evaluate dfa APIs.

* [DOCS] Removes extra lines from examples.

* Update docs/reference/ml/df-analytics/apis/evaluate-dfanalytics.asciidoc

Co-Authored-By: Lisa Cawley <lcawley@elastic.co>

* Update docs/reference/ml/df-analytics/apis/put-dfanalytics.asciidoc

Co-Authored-By: Lisa Cawley <lcawley@elastic.co>

* [DOCS] Explains examples.
2019-10-02 10:33:45 +02:00
István Zoltán Szabó 6a9f04ee76 [DOCS] Fixes typos in the PUT dfa and the evaluate dfa documentation. (#47348) 2019-10-02 09:52:29 +02:00
István Zoltán Szabó 3be51fbdf7 [DOCS] Adds regression analytics resources and examples to the data frame analytics APIs and the evaluation API (#46176)
* [DOCS] Adds regression analytics resources and examples to the data frame analytics APIs.
Co-Authored-By: Benjamin Trent <ben.w.trent@gmail.com>
Co-Authored-By: Tom Veasey <tveasey@users.noreply.github.com>
2019-09-19 09:23:18 +02:00
James Rodewig e253ee6ba6
[DOCS] Change // CONSOLE comments to [source,console] (#46440) (#46494) 2019-09-09 12:35:50 -04:00
James Rodewig f04573f8e8
[DOCS] [5 of 5] Change // TESTRESPONSE comments to [source,console-results] (#46449) (#46459) 2019-09-06 16:09:09 -04:00
István Zoltán Szabó a75348d1fb [DOCS] [PUT DFA] Documents inline the child params of source and dest (#45649)
* [DOCS] [PUT DFA] Documents inline the child params of source and dest.

* [DOCS] Fixes indentation issues and amends dfa definitions.
2019-08-29 15:09:02 +02:00
Dimitris Athanasiou dd6c13fdf9
[ML] Add description to DF analytics (#45774) (#46019) 2019-08-27 15:48:59 +03:00
István Zoltán Szabó cd7ba9f302 [DOCS] Amends data frame analytics resources, GET, and PUT API docs (#44806)
This PR addresses the feedback in  https://github.com/elastic/ml-team/issues/175#issuecomment-512215731.

* Adds an example to `analyzed_fields`
* Includes `source` and `dest` objects inline in the resource page
* Lists `model_memory_limit` in the PUT API page
* Amends the `analysis` section in the resource page
* Removes Properties headings in subsections
2019-07-26 11:52:43 +02:00
Lisa Cawley 53514b0477 [DOCS] Separates data frame analytics APIs (#44451) 2019-07-16 13:33:23 -07:00