OpenSearch/x-pack/plugin
Benjamin Trent 19af869243
[ML] adds multi-class feature importance support (#53803) (#54024)
Adds multi-class feature importance calculation. 

Feature importance objects are now mapped as follows
(logistic) Regression:
```
{
   "feature_name": "feature_0",
   "importance": -1.3
}
```
Multi-class [class names are `foo`, `bar`, `baz`]
```
{ 
   “feature_name”: “feature_0”, 
   “importance”: 2.0, // sum(abs()) of class importances
   “foo”: 1.0, 
   “bar”: 0.5, 
   “baz”: -0.5 
},
```

For users to get the full benefit of aggregating and searching for feature importance, they should update their index mapping as follows (before turning this option on in their pipelines)
```
 "ml.inference.feature_importance": {
          "type": "nested",
          "dynamic": true,
          "properties": {
            "feature_name": {
              "type": "keyword"
            },
            "importance": {
              "type": "double"
            }
          }
        }
```
The mapping field name is as follows
`ml.<inference.target_field>.<inference.tag>.feature_importance`
if `inference.tag` is not provided in the processor definition, it is not part of the field path.
`inference.target_field` is defaulted to `ml.inference`.
//cc @lcawl ^ Where should we document this?

If this makes it in for 7.7, there shouldn't be any feature_importance at inference BWC worries as 7.7 is the first version to have it.
2020-03-23 18:49:07 -04:00
..
analytics Move pipeline agg validation to coordinating node (backport of #53669) (#54019) 2020-03-23 17:22:56 -04:00
async-search Add async_search get and delete APIs to HLRC (#53828) (#53980) 2020-03-23 21:21:36 +01:00
autoscaling [7.x] Optimize which Rest resources are used by the Rest tests… (#53766) 2020-03-19 12:28:59 -05:00
ccr Better Incrementality for Snapshots of Unchanged Shards (#52182) (#53984) 2020-03-23 16:43:41 +01:00
core [ML] adds multi-class feature importance support (#53803) (#54024) 2020-03-23 18:49:07 -04:00
deprecation Add deprecation check for listener thread pool (#53438) 2020-03-12 14:32:41 -04:00
enrich Revert "Introduce system index APIs for Kibana (#53035)" (#53992) 2020-03-23 10:29:35 -07:00
eql EQL: EqlActionIT improvements (#53780) (#53888) 2020-03-20 17:28:15 -04:00
frozen-indices Fix doc_stats and segment_stats of ReadOnlyEngine (#53345) 2020-03-10 21:51:33 -04:00
graph [7.x] Optimize which Rest resources are used by the Rest tests… (#53766) 2020-03-19 12:28:59 -05:00
ilm [7.x] Optimize which Rest resources are used by the Rest tests… (#53766) 2020-03-19 12:28:59 -05:00
logstash Revert "Introduce system index APIs for Kibana (#53035)" (#53992) 2020-03-23 10:29:35 -07:00
mapper-constant-keyword Introduce a `constant_keyword` field. (#49713) (#53024) 2020-03-03 16:01:47 +01:00
mapper-flattened Rename AtomicFieldData to LeafFieldData (#53554) 2020-03-17 12:30:12 +00:00
ml [ML] adds multi-class feature importance support (#53803) (#54024) 2020-03-23 18:49:07 -04:00
monitoring Validate SSL settings at parse time (#49196) (#53473) 2020-03-12 10:14:51 -05:00
ql SQL: Extend DATE_TRUNC to also operate on intervals(elastic - #46632 ) (#47720) (#53972) 2020-03-23 15:05:16 +01:00
rollup Stop using round-tripped PipelineAggregators (backport of #53423) (#53629) 2020-03-16 16:15:23 -04:00
search-business-rules Fix highlighter support in PinnedQuery and added test (#53716) (#53729) 2020-03-18 15:39:17 +00:00
security Refactor global build info plugin to leverage JavaInstallationRegistry (#54026) 2020-03-23 15:30:10 -07:00
spatial Add support for distance queries on shape queries (#53468) (#53796) 2020-03-19 15:32:09 +01:00
sql SQL: Extend DATE_TRUNC to also operate on intervals(elastic - #46632 ) (#47720) (#53972) 2020-03-23 15:05:16 +01:00
src/test Backport of async search changes (#53976) 2020-03-23 19:13:31 +01:00
transform Revert "Introduce system index APIs for Kibana (#53035)" (#53992) 2020-03-23 10:29:35 -07:00
vectors Rename AtomicFieldData to LeafFieldData (#53554) 2020-03-17 12:30:12 +00:00
voting-only-node Single instance of the IndexNameExpressionResolver (#52604) 2020-02-21 07:50:02 -07:00
watcher Refactor global build info plugin to leverage JavaInstallationRegistry (#54026) 2020-03-23 15:30:10 -07:00
wildcard Backport of new wildcard field type (#53590) 2020-03-16 15:07:13 +00:00
build.gradle [7.x] Add Watcher to available rest resources (#53620) (#53764) 2020-03-19 12:29:36 -05:00