2019-12-19 06:19:44 -05:00
[role="xpack"]
[testenv="basic"]
[[inference-processor]]
=== {infer-cap} Processor
Uses a pre-trained {dfanalytics} model to infer against the data that is being
ingested in the pipeline.
[[inference-options]]
.{infer-cap} Options
[options="header"]
|======
| Name | Required | Default | Description
2020-01-21 10:21:17 -05:00
| `model_id` | yes | - | (String) The ID of the model to load and infer against.
2019-12-19 06:19:44 -05:00
| `target_field` | no | `ml.inference.<processor_tag>` | (String) Field added to incoming documents to contain results objects.
2020-03-13 15:40:57 -04:00
| `field_map` | yes | - | (Object) Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
2019-12-19 06:19:44 -05:00
| `inference_config` | yes | - | (Object) Contains the inference type and its options. There are two types: <<inference-processor-regression-opt,`regression`>> and <<inference-processor-classification-opt,`classification`>>.
include::common-options.asciidoc[]
|======
[source,js]
--------------------------------------------------
{
2020-01-21 10:21:17 -05:00
"inference": {
"model_id": "flight_delay_regression-1571767128603",
"target_field": "FlightDelayMin_prediction_infer",
2020-03-13 15:40:57 -04:00
"field_map": {},
2020-02-05 05:49:36 -05:00
"inference_config": { "regression": {} }
2020-01-21 10:21:17 -05:00
}
2019-12-19 06:19:44 -05:00
}
--------------------------------------------------
// NOTCONSOLE
[discrete]
[[inference-processor-regression-opt]]
==== {regression-cap} configuration options
2020-04-02 12:25:10 -04:00
Regression configuration for inference.
2019-12-19 06:19:44 -05:00
`results_field`::
(Optional, string)
2020-04-03 03:45:42 -04:00
include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-results-field]
2019-12-19 06:19:44 -05:00
2020-04-02 12:25:10 -04:00
`num_top_feature_importance_values`::
2020-02-21 18:42:31 -05:00
(Optional, integer)
2020-04-02 12:25:10 -04:00
include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-regression-num-top-feature-importance-values]
2019-12-19 06:19:44 -05:00
2020-03-26 04:22:12 -04:00
2019-12-19 06:19:44 -05:00
[discrete]
[[inference-processor-classification-opt]]
==== {classification-cap} configuration options
2020-04-02 12:25:10 -04:00
Classification configuration for inference.
2019-12-19 06:19:44 -05:00
`num_top_classes`::
2020-04-02 12:25:10 -04:00
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-num-top-classes]
2019-12-19 06:19:44 -05:00
2020-04-02 12:25:10 -04:00
`num_top_feature_importance_values`::
2020-02-21 18:42:31 -05:00
(Optional, integer)
2020-04-02 12:25:10 -04:00
include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-num-top-feature-importance-values]
2020-03-26 04:22:12 -04:00
2020-04-02 12:25:10 -04:00
`results_field`::
(Optional, string)
2020-04-03 03:45:42 -04:00
include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-results-field]
2020-04-02 12:25:10 -04:00
`top_classes_results_field`::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-top-classes-results-field]
2019-12-19 06:19:44 -05:00
2020-04-15 09:45:22 -04:00
`prediction_field_type`::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-prediction-field-type]
2019-12-19 06:19:44 -05:00
[discrete]
[[inference-processor-config-example]]
==== `inference_config` examples
[source,js]
--------------------------------------------------
{
"inference_config": {
2020-01-21 10:21:17 -05:00
"regression": {
"results_field": "my_regression"
2019-12-19 06:19:44 -05:00
}
2020-01-21 10:21:17 -05:00
}
2019-12-19 06:19:44 -05:00
}
--------------------------------------------------
// NOTCONSOLE
This configuration specifies a `regression` inference and the results are
written to the `my_regression` field contained in the `target_field` results
object.
[source,js]
--------------------------------------------------
{
"inference_config": {
2020-01-21 10:21:17 -05:00
"classification": {
"num_top_classes": 2,
"results_field": "prediction",
"top_classes_results_field": "probabilities"
2019-12-19 06:19:44 -05:00
}
2020-01-21 10:21:17 -05:00
}
2019-12-19 06:19:44 -05:00
}
--------------------------------------------------
// NOTCONSOLE
This configuration specifies a `classification` inference. The number of
categories for which the predicted probabilities are reported is 2
(`num_top_classes`). The result is written to the `prediction` field and the top
classes to the `probabilities` field. Both fields are contained in the
`target_field` results object.
2020-03-26 04:22:12 -04:00
[discrete]
[[inference-processor-feature-importance]]
==== {feat-imp-cap} object mapping
Update your index mapping of the {feat-imp} result field as you can see below to
get the full benefit of aggregating and searching for
2020-04-28 03:02:14 -04:00
{ml-docs}/ml-feature-importance.html[{feat-imp}].
2020-03-26 04:22:12 -04:00
[source,js]
--------------------------------------------------
"ml.inference.feature_importance": {
"type": "nested",
"dynamic": true,
"properties": {
"feature_name": {
"type": "keyword"
},
"importance": {
"type": "double"
}
}
}
--------------------------------------------------
// NOTCONSOLE
The mapping field name for {feat-imp} is compounded as follows:
`<ml.inference.target_field>`.`<inference.tag>`.`feature_importance`
If `inference.tag` is not provided in the processor definition, it is not part
of the field path. The `<ml.inference.target_field>` defaults to `ml.inference`.
For example, you provide a tag `foo` in the definition as you can see below:
[source,js]
--------------------------------------------------
{
"tag": "foo",
...
}
--------------------------------------------------
// NOTCONSOLE
2020-04-01 17:46:38 -04:00
The {feat-imp} value is written to the `ml.inference.foo.feature_importance`
2020-03-26 04:22:12 -04:00
field.
You can also specify a target field as follows:
[source,js]
--------------------------------------------------
{
"tag": "foo",
"target_field": "my_field"
}
--------------------------------------------------
// NOTCONSOLE
2020-04-01 17:46:38 -04:00
In this case, {feat-imp} is exposed in the
2020-04-02 12:25:10 -04:00
`my_field.foo.feature_importance` field.