[DOCS] Add docs for `geo_match` enrich policy type (#47745)

This commit is contained in:
James Rodewig 2019-10-09 08:39:11 -04:00
parent be0e17770c
commit 65f8294378
4 changed files with 234 additions and 19 deletions

View File

@ -71,7 +71,7 @@ Use the execute enrich policy API
to create the enrich index for an existing enrich policy. to create the enrich index for an existing enrich policy.
// tag::execute-enrich-policy-def[] // tag::execute-enrich-policy-def[]
The *enrich index* contains documents from the policy's source indices. The _enrich index_ contains documents from the policy's source indices.
Enrich indices always begin with `.enrich-*`, Enrich indices always begin with `.enrich-*`,
are read-only, are read-only,
and are <<indices-forcemerge,force merged>>. and are <<indices-forcemerge,force merged>>.

View File

@ -115,7 +115,13 @@ Valid key values are:
`match`:: `match`::
Match documents in the enrich index Match documents in the enrich index
using a <<query-dsl-term-query, term query>> for the `match_field`. using a <<query-dsl-term-query,term query>> for the `match_field`.
See <<enrich-setup>> for an example.
`geo_match`::
Match documents in the enrich index
using a <<query-dsl-geo-shape-query,`geo_shape` query>> for the `match_field`.
See <<put-enrich-policy-geo-match-ex>> for an example.
The parameter value is the enrich policy. The parameter value is the enrich policy.
The enrich policy is a set of rules The enrich policy is a set of rules
@ -144,3 +150,202 @@ to documents in the enrich index.
Fields appended to incoming documents Fields appended to incoming documents
from matching documents in the enrich index. from matching documents in the enrich index.
-- --
[[put-enrich-policy-api-example]]
==== {api-examples-title}
[[put-enrich-policy-geo-match-ex]]
===== `geo_match` policy type
You can use the `geo_match` enrich policy type
to enrich incoming documents
based on matching geo_shapes.
For example,
you can add postal codes
to incoming documents
based on a set of coordinates.
To see how the `geo_match` policy type works,
try the following example.
Use the <<indices-create-index, create index API>>
to create a source index.
The field mappings for the source index
must contain:
* A <<geo-shape,`geo_shape`>> field
which the enrich processor can use to match incoming documents
* One or more enrich fields
you'd like to append to incoming documents
[source,console]
----
PUT /postal_codes
{
"mappings": {
"properties": {
"location": {
"type": "geo_shape"
},
"postal_code": {
"type": "keyword"
}
}
}
}
----
Use the <<docs-index_,index API>>
to index data to this source index.
[source,console]
----
PUT /postal_codes/_doc/1?refresh=wait_for
{
"location": {
"type": "envelope",
"coordinates": [[13.0, 53.0], [14.0, 52.0]]
},
"postal_code": "96598"
}
----
// TEST[continued]
Use the put enrich policy API
to create an enrich policy
with the `geo_match` policy type.
This policy must include:
* One or more source indices
* A `match_field`,
the `geo_shape` field from the source indices
used to match incoming documents
* Enrich fields from the source indices
you'd like to append to incoming documents
[source,console]
----
PUT /_enrich/policy/postal_policy
{
"geo_match": {
"indices": "postal_codes",
"match_field": "location",
"enrich_fields": ["location","postal_code"]
}
}
----
// TEST[continued]
Use the <<execute-enrich-policy-api,execute enrich policy API>>
to create an enrich index for the policy.
include::execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def]
[source,console]
----
POST /_enrich/policy/postal_policy/_execute
----
// TEST[continued]
Use the <<put-pipeline-api,put pipeline API>>
to create an ingest pipeline.
In the pipeline,
add an <<enrich-processor,enrich processor>>
that includes:
* Your enrich policy
* The `field` of incoming documents used
to match the geo_shape of documents from the enrich index.
* The `target_field` used
to store appended enrich data for incoming documents.
* The `shape_relation`,
which indicates how the processor matches geo_shapes in incoming documents
to geo_shapes in documents from the enrich index.
See <<_spatial_relations>> for valid options and more information.
[source,console]
----
PUT /_ingest/pipeline/postal_lookup
{
"description": "Enrich postal codes",
"processors": [
{
"enrich": {
"policy_name": "postal_policy",
"field": "geo_location",
"target_field": "geo_data",
"shape_relation": "INTERSECTS"
}
}
]
}
----
// TEST[continued]
Use the ingest pipeline
to index a document.
The incoming document
should include the `field`
specified in your enrich processor.
[source,console]
----
PUT /users/_doc/0?pipeline=postal_lookup
{
"first_name": "Mardy",
"last_name": "Brown",
"geo_location": "POINT (13.5 52.5)"
}
----
// TEST[continued]
To verify the enrich processor matched
and appended the appropriate field data,
use the <<docs-get,get API>>
to view the indexed document.
[source,console]
----
GET /users/_doc/0
----
// TEST[continued]
The API returns the following response:
[source,console-result]
----
{
"found": true,
"_index": "users",
"_type": "_doc",
"_id": "0",
"_version": 1,
"_seq_no": 55,
"_primary_term": 1,
"_source": {
"geo_data": [
{
"location": {
"type": "envelope",
"coordinates": [[13.0, 53.0], [14.0, 52.0]]
},
"postal_code": "96598"
}
],
"first_name": "Mardy",
"last_name": "Brown",
"geo_location": "POINT (13.5 52.5)"
}
}
----
// TESTRESPONSE[s/"_seq_no": \d+/"_seq_no" : $body._seq_no/ s/"_primary_term":1/"_primary_term" : $body._primary_term/]
////
[source,console]
--------------------------------------------------
DELETE /_ingest/pipeline/postal_lookup
DELETE /_enrich/policy/postal_policy
--------------------------------------------------
// TEST[continued]
////

View File

@ -12,6 +12,7 @@ For example, you can use the enrich processor to:
* Identify web services or vendors based on known IP addresses * Identify web services or vendors based on known IP addresses
* Add product information to retail orders based on product IDs * Add product information to retail orders based on product IDs
* Supplement contact information based on an email address * Supplement contact information based on an email address
* Add postal codes based on user coordinates
[float] [float]
@ -31,7 +32,7 @@ follow these steps:
Once you have an enrich processor set up, Once you have an enrich processor set up,
you can <<update-enrich-data,update your enrich data>> you can <<update-enrich-data,update your enrich data>>
and <<update-enrich-policies, update your enrich policies>> and <<update-enrich-policies, update your enrich policies>>
using the <<enrich-apis,enrich>> APIs. using the <<enrich-apis,enrich APIs>>.
[IMPORTANT] [IMPORTANT]
==== ====
@ -59,17 +60,17 @@ include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-polic
To begin, To begin,
create one or more source indices. create one or more source indices.
A *source index* contains data you want to append to incoming documents. A _source index_ contains data you want to append to incoming documents.
You can index and manage documents in a source index You can index and manage documents in a source index
like a regular index. like a regular index.
The following <<docs-index_,index>> API request creates the `users` source index The following <<docs-index_,index API>> request creates the `users` source index
containing user data. containing user data.
This request also indexes a new document to the `users` source index. This request also indexes a new document to the `users` source index.
[source,console] [source,console]
---- ----
PUT /users/_doc/1?refresh PUT /users/_doc/1?refresh=wait_for
{ {
"email": "mardy.brown@asciidocsmith.com", "email": "mardy.brown@asciidocsmith.com",
"first_name": "Mardy", "first_name": "Mardy",
@ -93,7 +94,7 @@ See {beats-ref}/getting-started.html[Getting started with {beats}].
[[create-enrich-policy]] [[create-enrich-policy]]
==== Create an enrich policy ==== Create an enrich policy
Use the <<put-enrich-policy-api, put enrich policy>> API Use the <<put-enrich-policy-api,put enrich policy API>>
to create an enrich policy. to create an enrich policy.
include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-policy-def] include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-policy-def]
@ -116,7 +117,7 @@ PUT /_enrich/policy/users-policy
[[execute-enrich-policy]] [[execute-enrich-policy]]
==== Execute an enrich policy ==== Execute an enrich policy
Use the <<execute-enrich-policy-api, execute enrich policy>> API Use the <<execute-enrich-policy-api,execute enrich policy API>>
to create an enrich index for the policy. to create an enrich index for the policy.
include::apis/enrich/execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def] include::apis/enrich/execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def]
@ -136,7 +137,7 @@ POST /_enrich/policy/users-policy/_execute
[[add-enrich-processor]] [[add-enrich-processor]]
==== Add the enrich processor to an ingest pipeline ==== Add the enrich processor to an ingest pipeline
Use the <<put-pipeline-api,put pipeline>> API Use the <<put-pipeline-api,put pipeline API>>
to create an ingest pipeline. to create an ingest pipeline.
Include an <<enrich-processor,enrich processor>> Include an <<enrich-processor,enrich processor>>
that uses your enrich policy. that uses your enrich policy.
@ -144,14 +145,12 @@ that uses your enrich policy.
When defining an enrich processor, When defining an enrich processor,
you must include the following: you must include the following:
* The *field* used to match incoming documents * The field used to match incoming documents
to documents in the enrich index. to documents in the enrich index.
+ +
This field should be included in incoming documents. This field should be included in incoming documents.
To match, this field must contain the exact
value of the match field of a document in the enrich index.
* The *target field* added to incoming documents. * The target field added to incoming documents.
This field contains all appended enrich data. This field contains all appended enrich data.
The following request adds a new pipeline, `user_lookup`. The following request adds a new pipeline, `user_lookup`.
@ -204,10 +203,10 @@ the processor appends data from those documents to the array.
If the incoming document matches no documents in the enrich index, If the incoming document matches no documents in the enrich index,
the processor appends no data. the processor appends no data.
The following <<docs-index_,index>> API request uses the ingest pipeline The following <<docs-index_,index API>> request uses the ingest pipeline
to index a document to index a document
containing the `email` field, containing the `email` field
the `match_field` specified in the `users-policy` enrich policy. specified in the enrich processor.
[source,console] [source,console]
---- ----
@ -220,7 +219,7 @@ PUT /my_index/_doc/my_id?pipeline=user_lookup
To verify the enrich processor matched To verify the enrich processor matched
and appended the appropriate field data, and appended the appropriate field data,
use the <<docs-get,get>> API to view the indexed document. use the <<docs-get,get API>> to view the indexed document.
[source,console] [source,console]
---- ----

View File

@ -16,7 +16,18 @@ check out the <<ingest-enriching-data,tutorial>> to get familiar with enrich pol
| `field` | yes | - | The field in the input document that matches the policies match_field used to retrieve the enrichment data. | `field` | yes | - | The field in the input document that matches the policies match_field used to retrieve the enrichment data.
| `target_field` | yes | - | The field that will be used for the enrichment data. | `target_field` | yes | - | The field that will be used for the enrichment data.
| `ignore_missing` | no | false | If `true` and `field` does not exist, the processor quietly exits without modifying the document | `ignore_missing` | no | false | If `true` and `field` does not exist, the processor quietly exits without modifying the document
| `override` | no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched. | `override` | no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched.
| `max_matches` | no | 1 | The maximum number of matched documents to include under the configured target field. In order to avoid documents getting too large, the maximum allowed value is 128. | `max_matches` | no | 1 | The maximum number of matched documents to include under the configured target field. In order to avoid documents getting too large, the maximum allowed value is 128.
| `shape_relation` | no | `INTERSECTS` a| Spatial relation operator
used to match the <<geo-shape,geo_shape>> of incoming documents
to documents in the enrich index.
+
This option is only used for `geo_match` enrich policy types.
+
The <<spatial-strategy, geo_shape strategy>> mapping parameter determines
which spatial relation operators are availlble.
See <<_spatial_relations>>
for operators and more information.
include::common-options.asciidoc[] include::common-options.asciidoc[]
|====== |======