[DOCS] Add docs for `geo_match` enrich policy type (#47745)
This commit is contained in:
parent
be0e17770c
commit
65f8294378
|
@ -71,7 +71,7 @@ Use the execute enrich policy API
|
||||||
to create the enrich index for an existing enrich policy.
|
to create the enrich index for an existing enrich policy.
|
||||||
|
|
||||||
// tag::execute-enrich-policy-def[]
|
// tag::execute-enrich-policy-def[]
|
||||||
The *enrich index* contains documents from the policy's source indices.
|
The _enrich index_ contains documents from the policy's source indices.
|
||||||
Enrich indices always begin with `.enrich-*`,
|
Enrich indices always begin with `.enrich-*`,
|
||||||
are read-only,
|
are read-only,
|
||||||
and are <<indices-forcemerge,force merged>>.
|
and are <<indices-forcemerge,force merged>>.
|
||||||
|
|
|
@ -115,7 +115,13 @@ Valid key values are:
|
||||||
|
|
||||||
`match`::
|
`match`::
|
||||||
Match documents in the enrich index
|
Match documents in the enrich index
|
||||||
using a <<query-dsl-term-query, term query>> for the `match_field`.
|
using a <<query-dsl-term-query,term query>> for the `match_field`.
|
||||||
|
See <<enrich-setup>> for an example.
|
||||||
|
|
||||||
|
`geo_match`::
|
||||||
|
Match documents in the enrich index
|
||||||
|
using a <<query-dsl-geo-shape-query,`geo_shape` query>> for the `match_field`.
|
||||||
|
See <<put-enrich-policy-geo-match-ex>> for an example.
|
||||||
|
|
||||||
The parameter value is the enrich policy.
|
The parameter value is the enrich policy.
|
||||||
The enrich policy is a set of rules
|
The enrich policy is a set of rules
|
||||||
|
@ -144,3 +150,202 @@ to documents in the enrich index.
|
||||||
Fields appended to incoming documents
|
Fields appended to incoming documents
|
||||||
from matching documents in the enrich index.
|
from matching documents in the enrich index.
|
||||||
--
|
--
|
||||||
|
|
||||||
|
[[put-enrich-policy-api-example]]
|
||||||
|
==== {api-examples-title}
|
||||||
|
|
||||||
|
[[put-enrich-policy-geo-match-ex]]
|
||||||
|
===== `geo_match` policy type
|
||||||
|
|
||||||
|
You can use the `geo_match` enrich policy type
|
||||||
|
to enrich incoming documents
|
||||||
|
based on matching geo_shapes.
|
||||||
|
For example,
|
||||||
|
you can add postal codes
|
||||||
|
to incoming documents
|
||||||
|
based on a set of coordinates.
|
||||||
|
|
||||||
|
To see how the `geo_match` policy type works,
|
||||||
|
try the following example.
|
||||||
|
|
||||||
|
Use the <<indices-create-index, create index API>>
|
||||||
|
to create a source index.
|
||||||
|
The field mappings for the source index
|
||||||
|
must contain:
|
||||||
|
|
||||||
|
* A <<geo-shape,`geo_shape`>> field
|
||||||
|
which the enrich processor can use to match incoming documents
|
||||||
|
* One or more enrich fields
|
||||||
|
you'd like to append to incoming documents
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
PUT /postal_codes
|
||||||
|
{
|
||||||
|
"mappings": {
|
||||||
|
"properties": {
|
||||||
|
"location": {
|
||||||
|
"type": "geo_shape"
|
||||||
|
},
|
||||||
|
"postal_code": {
|
||||||
|
"type": "keyword"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
----
|
||||||
|
|
||||||
|
Use the <<docs-index_,index API>>
|
||||||
|
to index data to this source index.
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
PUT /postal_codes/_doc/1?refresh=wait_for
|
||||||
|
{
|
||||||
|
"location": {
|
||||||
|
"type": "envelope",
|
||||||
|
"coordinates": [[13.0, 53.0], [14.0, 52.0]]
|
||||||
|
},
|
||||||
|
"postal_code": "96598"
|
||||||
|
}
|
||||||
|
----
|
||||||
|
// TEST[continued]
|
||||||
|
|
||||||
|
Use the put enrich policy API
|
||||||
|
to create an enrich policy
|
||||||
|
with the `geo_match` policy type.
|
||||||
|
This policy must include:
|
||||||
|
|
||||||
|
* One or more source indices
|
||||||
|
* A `match_field`,
|
||||||
|
the `geo_shape` field from the source indices
|
||||||
|
used to match incoming documents
|
||||||
|
* Enrich fields from the source indices
|
||||||
|
you'd like to append to incoming documents
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
PUT /_enrich/policy/postal_policy
|
||||||
|
{
|
||||||
|
"geo_match": {
|
||||||
|
"indices": "postal_codes",
|
||||||
|
"match_field": "location",
|
||||||
|
"enrich_fields": ["location","postal_code"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
----
|
||||||
|
// TEST[continued]
|
||||||
|
|
||||||
|
Use the <<execute-enrich-policy-api,execute enrich policy API>>
|
||||||
|
to create an enrich index for the policy.
|
||||||
|
|
||||||
|
include::execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def]
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
POST /_enrich/policy/postal_policy/_execute
|
||||||
|
----
|
||||||
|
// TEST[continued]
|
||||||
|
|
||||||
|
Use the <<put-pipeline-api,put pipeline API>>
|
||||||
|
to create an ingest pipeline.
|
||||||
|
In the pipeline,
|
||||||
|
add an <<enrich-processor,enrich processor>>
|
||||||
|
that includes:
|
||||||
|
|
||||||
|
* Your enrich policy
|
||||||
|
* The `field` of incoming documents used
|
||||||
|
to match the geo_shape of documents from the enrich index.
|
||||||
|
* The `target_field` used
|
||||||
|
to store appended enrich data for incoming documents.
|
||||||
|
* The `shape_relation`,
|
||||||
|
which indicates how the processor matches geo_shapes in incoming documents
|
||||||
|
to geo_shapes in documents from the enrich index.
|
||||||
|
See <<_spatial_relations>> for valid options and more information.
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
PUT /_ingest/pipeline/postal_lookup
|
||||||
|
{
|
||||||
|
"description": "Enrich postal codes",
|
||||||
|
"processors": [
|
||||||
|
{
|
||||||
|
"enrich": {
|
||||||
|
"policy_name": "postal_policy",
|
||||||
|
"field": "geo_location",
|
||||||
|
"target_field": "geo_data",
|
||||||
|
"shape_relation": "INTERSECTS"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
----
|
||||||
|
// TEST[continued]
|
||||||
|
|
||||||
|
Use the ingest pipeline
|
||||||
|
to index a document.
|
||||||
|
The incoming document
|
||||||
|
should include the `field`
|
||||||
|
specified in your enrich processor.
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
PUT /users/_doc/0?pipeline=postal_lookup
|
||||||
|
{
|
||||||
|
"first_name": "Mardy",
|
||||||
|
"last_name": "Brown",
|
||||||
|
"geo_location": "POINT (13.5 52.5)"
|
||||||
|
}
|
||||||
|
----
|
||||||
|
// TEST[continued]
|
||||||
|
|
||||||
|
To verify the enrich processor matched
|
||||||
|
and appended the appropriate field data,
|
||||||
|
use the <<docs-get,get API>>
|
||||||
|
to view the indexed document.
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
GET /users/_doc/0
|
||||||
|
----
|
||||||
|
// TEST[continued]
|
||||||
|
|
||||||
|
The API returns the following response:
|
||||||
|
|
||||||
|
[source,console-result]
|
||||||
|
----
|
||||||
|
{
|
||||||
|
"found": true,
|
||||||
|
"_index": "users",
|
||||||
|
"_type": "_doc",
|
||||||
|
"_id": "0",
|
||||||
|
"_version": 1,
|
||||||
|
"_seq_no": 55,
|
||||||
|
"_primary_term": 1,
|
||||||
|
"_source": {
|
||||||
|
"geo_data": [
|
||||||
|
{
|
||||||
|
"location": {
|
||||||
|
"type": "envelope",
|
||||||
|
"coordinates": [[13.0, 53.0], [14.0, 52.0]]
|
||||||
|
},
|
||||||
|
"postal_code": "96598"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"first_name": "Mardy",
|
||||||
|
"last_name": "Brown",
|
||||||
|
"geo_location": "POINT (13.5 52.5)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
----
|
||||||
|
// TESTRESPONSE[s/"_seq_no": \d+/"_seq_no" : $body._seq_no/ s/"_primary_term":1/"_primary_term" : $body._primary_term/]
|
||||||
|
|
||||||
|
////
|
||||||
|
[source,console]
|
||||||
|
--------------------------------------------------
|
||||||
|
DELETE /_ingest/pipeline/postal_lookup
|
||||||
|
|
||||||
|
DELETE /_enrich/policy/postal_policy
|
||||||
|
--------------------------------------------------
|
||||||
|
// TEST[continued]
|
||||||
|
////
|
|
@ -12,6 +12,7 @@ For example, you can use the enrich processor to:
|
||||||
* Identify web services or vendors based on known IP addresses
|
* Identify web services or vendors based on known IP addresses
|
||||||
* Add product information to retail orders based on product IDs
|
* Add product information to retail orders based on product IDs
|
||||||
* Supplement contact information based on an email address
|
* Supplement contact information based on an email address
|
||||||
|
* Add postal codes based on user coordinates
|
||||||
|
|
||||||
|
|
||||||
[float]
|
[float]
|
||||||
|
@ -31,7 +32,7 @@ follow these steps:
|
||||||
Once you have an enrich processor set up,
|
Once you have an enrich processor set up,
|
||||||
you can <<update-enrich-data,update your enrich data>>
|
you can <<update-enrich-data,update your enrich data>>
|
||||||
and <<update-enrich-policies, update your enrich policies>>
|
and <<update-enrich-policies, update your enrich policies>>
|
||||||
using the <<enrich-apis,enrich>> APIs.
|
using the <<enrich-apis,enrich APIs>>.
|
||||||
|
|
||||||
[IMPORTANT]
|
[IMPORTANT]
|
||||||
====
|
====
|
||||||
|
@ -59,17 +60,17 @@ include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-polic
|
||||||
To begin,
|
To begin,
|
||||||
create one or more source indices.
|
create one or more source indices.
|
||||||
|
|
||||||
A *source index* contains data you want to append to incoming documents.
|
A _source index_ contains data you want to append to incoming documents.
|
||||||
You can index and manage documents in a source index
|
You can index and manage documents in a source index
|
||||||
like a regular index.
|
like a regular index.
|
||||||
|
|
||||||
The following <<docs-index_,index>> API request creates the `users` source index
|
The following <<docs-index_,index API>> request creates the `users` source index
|
||||||
containing user data.
|
containing user data.
|
||||||
This request also indexes a new document to the `users` source index.
|
This request also indexes a new document to the `users` source index.
|
||||||
|
|
||||||
[source,console]
|
[source,console]
|
||||||
----
|
----
|
||||||
PUT /users/_doc/1?refresh
|
PUT /users/_doc/1?refresh=wait_for
|
||||||
{
|
{
|
||||||
"email": "mardy.brown@asciidocsmith.com",
|
"email": "mardy.brown@asciidocsmith.com",
|
||||||
"first_name": "Mardy",
|
"first_name": "Mardy",
|
||||||
|
@ -93,7 +94,7 @@ See {beats-ref}/getting-started.html[Getting started with {beats}].
|
||||||
[[create-enrich-policy]]
|
[[create-enrich-policy]]
|
||||||
==== Create an enrich policy
|
==== Create an enrich policy
|
||||||
|
|
||||||
Use the <<put-enrich-policy-api, put enrich policy>> API
|
Use the <<put-enrich-policy-api,put enrich policy API>>
|
||||||
to create an enrich policy.
|
to create an enrich policy.
|
||||||
|
|
||||||
include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-policy-def]
|
include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-policy-def]
|
||||||
|
@ -116,7 +117,7 @@ PUT /_enrich/policy/users-policy
|
||||||
[[execute-enrich-policy]]
|
[[execute-enrich-policy]]
|
||||||
==== Execute an enrich policy
|
==== Execute an enrich policy
|
||||||
|
|
||||||
Use the <<execute-enrich-policy-api, execute enrich policy>> API
|
Use the <<execute-enrich-policy-api,execute enrich policy API>>
|
||||||
to create an enrich index for the policy.
|
to create an enrich index for the policy.
|
||||||
|
|
||||||
include::apis/enrich/execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def]
|
include::apis/enrich/execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def]
|
||||||
|
@ -136,7 +137,7 @@ POST /_enrich/policy/users-policy/_execute
|
||||||
[[add-enrich-processor]]
|
[[add-enrich-processor]]
|
||||||
==== Add the enrich processor to an ingest pipeline
|
==== Add the enrich processor to an ingest pipeline
|
||||||
|
|
||||||
Use the <<put-pipeline-api,put pipeline>> API
|
Use the <<put-pipeline-api,put pipeline API>>
|
||||||
to create an ingest pipeline.
|
to create an ingest pipeline.
|
||||||
Include an <<enrich-processor,enrich processor>>
|
Include an <<enrich-processor,enrich processor>>
|
||||||
that uses your enrich policy.
|
that uses your enrich policy.
|
||||||
|
@ -144,14 +145,12 @@ that uses your enrich policy.
|
||||||
When defining an enrich processor,
|
When defining an enrich processor,
|
||||||
you must include the following:
|
you must include the following:
|
||||||
|
|
||||||
* The *field* used to match incoming documents
|
* The field used to match incoming documents
|
||||||
to documents in the enrich index.
|
to documents in the enrich index.
|
||||||
+
|
+
|
||||||
This field should be included in incoming documents.
|
This field should be included in incoming documents.
|
||||||
To match, this field must contain the exact
|
|
||||||
value of the match field of a document in the enrich index.
|
|
||||||
|
|
||||||
* The *target field* added to incoming documents.
|
* The target field added to incoming documents.
|
||||||
This field contains all appended enrich data.
|
This field contains all appended enrich data.
|
||||||
|
|
||||||
The following request adds a new pipeline, `user_lookup`.
|
The following request adds a new pipeline, `user_lookup`.
|
||||||
|
@ -204,10 +203,10 @@ the processor appends data from those documents to the array.
|
||||||
If the incoming document matches no documents in the enrich index,
|
If the incoming document matches no documents in the enrich index,
|
||||||
the processor appends no data.
|
the processor appends no data.
|
||||||
|
|
||||||
The following <<docs-index_,index>> API request uses the ingest pipeline
|
The following <<docs-index_,index API>> request uses the ingest pipeline
|
||||||
to index a document
|
to index a document
|
||||||
containing the `email` field,
|
containing the `email` field
|
||||||
the `match_field` specified in the `users-policy` enrich policy.
|
specified in the enrich processor.
|
||||||
|
|
||||||
[source,console]
|
[source,console]
|
||||||
----
|
----
|
||||||
|
@ -220,7 +219,7 @@ PUT /my_index/_doc/my_id?pipeline=user_lookup
|
||||||
|
|
||||||
To verify the enrich processor matched
|
To verify the enrich processor matched
|
||||||
and appended the appropriate field data,
|
and appended the appropriate field data,
|
||||||
use the <<docs-get,get>> API to view the indexed document.
|
use the <<docs-get,get API>> to view the indexed document.
|
||||||
|
|
||||||
[source,console]
|
[source,console]
|
||||||
----
|
----
|
||||||
|
|
|
@ -16,7 +16,18 @@ check out the <<ingest-enriching-data,tutorial>> to get familiar with enrich pol
|
||||||
| `field` | yes | - | The field in the input document that matches the policies match_field used to retrieve the enrichment data.
|
| `field` | yes | - | The field in the input document that matches the policies match_field used to retrieve the enrichment data.
|
||||||
| `target_field` | yes | - | The field that will be used for the enrichment data.
|
| `target_field` | yes | - | The field that will be used for the enrichment data.
|
||||||
| `ignore_missing` | no | false | If `true` and `field` does not exist, the processor quietly exits without modifying the document
|
| `ignore_missing` | no | false | If `true` and `field` does not exist, the processor quietly exits without modifying the document
|
||||||
| `override` | no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched.
|
| `override` | no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched.
|
||||||
| `max_matches` | no | 1 | The maximum number of matched documents to include under the configured target field. In order to avoid documents getting too large, the maximum allowed value is 128.
|
| `max_matches` | no | 1 | The maximum number of matched documents to include under the configured target field. In order to avoid documents getting too large, the maximum allowed value is 128.
|
||||||
|
| `shape_relation` | no | `INTERSECTS` a| Spatial relation operator
|
||||||
|
used to match the <<geo-shape,geo_shape>> of incoming documents
|
||||||
|
to documents in the enrich index.
|
||||||
|
+
|
||||||
|
This option is only used for `geo_match` enrich policy types.
|
||||||
|
+
|
||||||
|
The <<spatial-strategy, geo_shape strategy>> mapping parameter determines
|
||||||
|
which spatial relation operators are availlble.
|
||||||
|
See <<_spatial_relations>>
|
||||||
|
for operators and more information.
|
||||||
|
|
||||||
include::common-options.asciidoc[]
|
include::common-options.asciidoc[]
|
||||||
|======
|
|======
|
||||||
|
|
Loading…
Reference in New Issue