mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-03-04 01:49:15 +00:00
c09b62d5bf
Backport of: #48254 This changes tests to create a valid source index prior to creating the enrich policy.
356 lines
7.9 KiB
Plaintext
356 lines
7.9 KiB
Plaintext
[role="xpack"]
|
|
[testenv="basic"]
|
|
[[put-enrich-policy-api]]
|
|
=== Put enrich policy API
|
|
++++
|
|
<titleabbrev>Put enrich policy</titleabbrev>
|
|
++++
|
|
|
|
Creates an enrich policy.
|
|
|
|
////
|
|
[source,console]
|
|
----
|
|
PUT /users
|
|
{
|
|
"mappings" : {
|
|
"properties" : {
|
|
"email" : { "type" : "keyword" }
|
|
}
|
|
}
|
|
}
|
|
----
|
|
////
|
|
|
|
[source,console]
|
|
----
|
|
PUT /_enrich/policy/my-policy
|
|
{
|
|
"match": {
|
|
"indices": "users",
|
|
"match_field": "email",
|
|
"enrich_fields": ["first_name", "last_name", "city", "zip", "state"]
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
////
|
|
[source,console]
|
|
--------------------------------------------------
|
|
DELETE /_enrich/policy/my-policy
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
////
|
|
|
|
|
|
[[put-enrich-policy-api-request]]
|
|
==== {api-request-title}
|
|
|
|
`PUT /_enrich/policy/<enrich-policy>`
|
|
|
|
|
|
[[put-enrich-policy-api-prereqs]]
|
|
==== {api-prereq-title}
|
|
|
|
// tag::enrich-policy-api-prereqs[]
|
|
If you use {es} {security-features}, you must have:
|
|
|
|
* `read` index privileges for any indices used
|
|
* The `enrich_user` {stack-ov}/built-in-roles.html[built-in role]
|
|
// end::enrich-policy-api-prereqs[]
|
|
|
|
|
|
[[put-enrich-policy-api-desc]]
|
|
==== {api-description-title}
|
|
|
|
Use the put enrich policy API
|
|
to create a new enrich policy.
|
|
|
|
// tag::enrich-policy-def[]
|
|
An *enrich policy* is a set of rules the enrich processor uses
|
|
to append the appropriate data to incoming documents.
|
|
An enrich policy contains:
|
|
|
|
* The *policy type*,
|
|
which determines how the processor enriches incoming documents
|
|
* A list of source indices
|
|
* The *match field* used to match incoming documents
|
|
* *Enrich fields* appended to incoming documents
|
|
from matching documents
|
|
// end::enrich-policy-def[]
|
|
|
|
|
|
===== Update an enrich policy
|
|
|
|
// tag::update-enrich-policy[]
|
|
You cannot update an existing enrich policy.
|
|
Instead, you can:
|
|
|
|
. Create and execute a new enrich policy.
|
|
|
|
. Replace the previous enrich policy
|
|
with the new enrich policy
|
|
in any in-use enrich processors.
|
|
|
|
. Use the <<delete-enrich-policy-api, delete enrich policy>> API
|
|
to delete the previous enrich policy.
|
|
// end::update-enrich-policy[]
|
|
|
|
|
|
[[put-enrich-policy-api-path-params]]
|
|
==== {api-path-parms-title}
|
|
|
|
`<enrich-policy>`::
|
|
(Required, string)
|
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=enrich-policy]
|
|
|
|
|
|
[[put-enrich-policy-api-request-body]]
|
|
==== {api-request-body-title}
|
|
|
|
`<policy-type>`::
|
|
+
|
|
--
|
|
(Required, enrich policy object)
|
|
The parameter key is the enrich policy type.
|
|
The enrich policy type indicates
|
|
how the enrich processor matches incoming documents
|
|
to documents in the enrich index.
|
|
|
|
Valid key values are:
|
|
|
|
`match`::
|
|
Match documents in the enrich index
|
|
using a <<query-dsl-term-query,term query>> for the `match_field`.
|
|
See <<enrich-setup>> for an example.
|
|
|
|
`geo_match`::
|
|
Match documents in the enrich index
|
|
using a <<query-dsl-geo-shape-query,`geo_shape` query>> for the `match_field`.
|
|
See <<put-enrich-policy-geo-match-ex>> for an example.
|
|
|
|
The parameter value is the enrich policy.
|
|
The enrich policy is a set of rules
|
|
used to create an <<execute-enrich-policy,enrich index>>.
|
|
The enrich processor also uses these rules
|
|
to append field data to incoming documents.
|
|
|
|
Parameters include:
|
|
|
|
`indices`::
|
|
(Required, array of strings)
|
|
Source indices used to create the enrich index.
|
|
|
|
`query`::
|
|
(Optional, string)
|
|
Query type used to find and select documents in the enrich index.
|
|
Valid value is <<query-dsl-match-all-query,`match_all`>> (default).
|
|
|
|
`match_field`::
|
|
(Required, string)
|
|
Field used to match incoming documents
|
|
to documents in the enrich index.
|
|
|
|
`enrich_fields`::
|
|
(Required, Array of string)
|
|
Fields appended to incoming documents
|
|
from matching documents in the enrich index.
|
|
--
|
|
|
|
[[put-enrich-policy-api-example]]
|
|
==== {api-examples-title}
|
|
|
|
[[put-enrich-policy-geo-match-ex]]
|
|
===== `geo_match` policy type
|
|
|
|
You can use the `geo_match` enrich policy type
|
|
to enrich incoming documents
|
|
based on matching geo_shapes.
|
|
For example,
|
|
you can add postal codes
|
|
to incoming documents
|
|
based on a set of coordinates.
|
|
|
|
To see how the `geo_match` policy type works,
|
|
try the following example.
|
|
|
|
Use the <<indices-create-index, create index API>>
|
|
to create a source index.
|
|
The field mappings for the source index
|
|
must contain:
|
|
|
|
* A <<geo-shape,`geo_shape`>> field
|
|
which the enrich processor can use to match incoming documents
|
|
* One or more enrich fields
|
|
you'd like to append to incoming documents
|
|
|
|
[source,console]
|
|
----
|
|
PUT /postal_codes
|
|
{
|
|
"mappings": {
|
|
"properties": {
|
|
"location": {
|
|
"type": "geo_shape"
|
|
},
|
|
"postal_code": {
|
|
"type": "keyword"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
----
|
|
|
|
Use the <<docs-index_,index API>>
|
|
to index data to this source index.
|
|
|
|
[source,console]
|
|
----
|
|
PUT /postal_codes/_doc/1?refresh=wait_for
|
|
{
|
|
"location": {
|
|
"type": "envelope",
|
|
"coordinates": [[13.0, 53.0], [14.0, 52.0]]
|
|
},
|
|
"postal_code": "96598"
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
Use the put enrich policy API
|
|
to create an enrich policy
|
|
with the `geo_match` policy type.
|
|
This policy must include:
|
|
|
|
* One or more source indices
|
|
* A `match_field`,
|
|
the `geo_shape` field from the source indices
|
|
used to match incoming documents
|
|
* Enrich fields from the source indices
|
|
you'd like to append to incoming documents
|
|
|
|
[source,console]
|
|
----
|
|
PUT /_enrich/policy/postal_policy
|
|
{
|
|
"geo_match": {
|
|
"indices": "postal_codes",
|
|
"match_field": "location",
|
|
"enrich_fields": ["location","postal_code"]
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
Use the <<execute-enrich-policy-api,execute enrich policy API>>
|
|
to create an enrich index for the policy.
|
|
|
|
include::execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def]
|
|
|
|
[source,console]
|
|
----
|
|
POST /_enrich/policy/postal_policy/_execute
|
|
----
|
|
// TEST[continued]
|
|
|
|
Use the <<put-pipeline-api,put pipeline API>>
|
|
to create an ingest pipeline.
|
|
In the pipeline,
|
|
add an <<enrich-processor,enrich processor>>
|
|
that includes:
|
|
|
|
* Your enrich policy
|
|
* The `field` of incoming documents used
|
|
to match the geo_shape of documents from the enrich index.
|
|
* The `target_field` used
|
|
to store appended enrich data for incoming documents.
|
|
* The `shape_relation`,
|
|
which indicates how the processor matches geo_shapes in incoming documents
|
|
to geo_shapes in documents from the enrich index.
|
|
See <<_spatial_relations>> for valid options and more information.
|
|
|
|
[source,console]
|
|
----
|
|
PUT /_ingest/pipeline/postal_lookup
|
|
{
|
|
"description": "Enrich postal codes",
|
|
"processors": [
|
|
{
|
|
"enrich": {
|
|
"policy_name": "postal_policy",
|
|
"field": "geo_location",
|
|
"target_field": "geo_data",
|
|
"shape_relation": "INTERSECTS"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
Use the ingest pipeline
|
|
to index a document.
|
|
The incoming document
|
|
should include the `field`
|
|
specified in your enrich processor.
|
|
|
|
[source,console]
|
|
----
|
|
PUT /users/_doc/0?pipeline=postal_lookup
|
|
{
|
|
"first_name": "Mardy",
|
|
"last_name": "Brown",
|
|
"geo_location": "POINT (13.5 52.5)"
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
To verify the enrich processor matched
|
|
and appended the appropriate field data,
|
|
use the <<docs-get,get API>>
|
|
to view the indexed document.
|
|
|
|
[source,console]
|
|
----
|
|
GET /users/_doc/0
|
|
----
|
|
// TEST[continued]
|
|
|
|
The API returns the following response:
|
|
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"found": true,
|
|
"_index": "users",
|
|
"_type": "_doc",
|
|
"_id": "0",
|
|
"_version": 1,
|
|
"_seq_no": 55,
|
|
"_primary_term": 1,
|
|
"_source": {
|
|
"geo_data": {
|
|
"location": {
|
|
"type": "envelope",
|
|
"coordinates": [[13.0, 53.0], [14.0, 52.0]]
|
|
},
|
|
"postal_code": "96598"
|
|
},
|
|
"first_name": "Mardy",
|
|
"last_name": "Brown",
|
|
"geo_location": "POINT (13.5 52.5)"
|
|
}
|
|
}
|
|
----
|
|
// TESTRESPONSE[s/"_seq_no": \d+/"_seq_no" : $body._seq_no/ s/"_primary_term":1/"_primary_term" : $body._primary_term/]
|
|
|
|
////
|
|
[source,console]
|
|
--------------------------------------------------
|
|
DELETE /_ingest/pipeline/postal_lookup
|
|
|
|
DELETE /_enrich/policy/postal_policy
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
//// |