diff --git a/docs/reference/ingest/apis/enrich/execute-enrich-policy.asciidoc b/docs/reference/ingest/apis/enrich/execute-enrich-policy.asciidoc index a569f40d5c1..f859c1201d9 100644 --- a/docs/reference/ingest/apis/enrich/execute-enrich-policy.asciidoc +++ b/docs/reference/ingest/apis/enrich/execute-enrich-policy.asciidoc @@ -71,7 +71,7 @@ Use the execute enrich policy API to create the enrich index for an existing enrich policy. // tag::execute-enrich-policy-def[] -The *enrich index* contains documents from the policy's source indices. +The _enrich index_ contains documents from the policy's source indices. Enrich indices always begin with `.enrich-*`, are read-only, and are <>. diff --git a/docs/reference/ingest/apis/enrich/put-enrich-policy.asciidoc b/docs/reference/ingest/apis/enrich/put-enrich-policy.asciidoc index 8a546d8529e..e6be87b7556 100644 --- a/docs/reference/ingest/apis/enrich/put-enrich-policy.asciidoc +++ b/docs/reference/ingest/apis/enrich/put-enrich-policy.asciidoc @@ -115,7 +115,13 @@ Valid key values are: `match`:: Match documents in the enrich index -using a <> for the `match_field`. +using a <> for the `match_field`. +See <> for an example. + +`geo_match`:: +Match documents in the enrich index +using a <> for the `match_field`. +See <> for an example. The parameter value is the enrich policy. The enrich policy is a set of rules @@ -144,3 +150,202 @@ to documents in the enrich index. Fields appended to incoming documents from matching documents in the enrich index. -- + +[[put-enrich-policy-api-example]] +==== {api-examples-title} + +[[put-enrich-policy-geo-match-ex]] +===== `geo_match` policy type + +You can use the `geo_match` enrich policy type +to enrich incoming documents +based on matching geo_shapes. +For example, +you can add postal codes +to incoming documents +based on a set of coordinates. + +To see how the `geo_match` policy type works, +try the following example. + +Use the <> +to create a source index. +The field mappings for the source index +must contain: + +* A <> field + which the enrich processor can use to match incoming documents +* One or more enrich fields + you'd like to append to incoming documents + +[source,console] +---- +PUT /postal_codes +{ + "mappings": { + "properties": { + "location": { + "type": "geo_shape" + }, + "postal_code": { + "type": "keyword" + } + } + } +} +---- + +Use the <> +to index data to this source index. + +[source,console] +---- +PUT /postal_codes/_doc/1?refresh=wait_for +{ + "location": { + "type": "envelope", + "coordinates": [[13.0, 53.0], [14.0, 52.0]] + }, + "postal_code": "96598" +} +---- +// TEST[continued] + +Use the put enrich policy API +to create an enrich policy +with the `geo_match` policy type. +This policy must include: + +* One or more source indices +* A `match_field`, + the `geo_shape` field from the source indices + used to match incoming documents +* Enrich fields from the source indices + you'd like to append to incoming documents + +[source,console] +---- +PUT /_enrich/policy/postal_policy +{ + "geo_match": { + "indices": "postal_codes", + "match_field": "location", + "enrich_fields": ["location","postal_code"] + } +} +---- +// TEST[continued] + +Use the <> +to create an enrich index for the policy. + +include::execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def] + +[source,console] +---- +POST /_enrich/policy/postal_policy/_execute +---- +// TEST[continued] + +Use the <> +to create an ingest pipeline. +In the pipeline, +add an <> +that includes: + +* Your enrich policy +* The `field` of incoming documents used + to match the geo_shape of documents from the enrich index. +* The `target_field` used + to store appended enrich data for incoming documents. +* The `shape_relation`, + which indicates how the processor matches geo_shapes in incoming documents + to geo_shapes in documents from the enrich index. + See <<_spatial_relations>> for valid options and more information. + +[source,console] +---- +PUT /_ingest/pipeline/postal_lookup +{ + "description": "Enrich postal codes", + "processors": [ + { + "enrich": { + "policy_name": "postal_policy", + "field": "geo_location", + "target_field": "geo_data", + "shape_relation": "INTERSECTS" + } + } + ] +} +---- +// TEST[continued] + +Use the ingest pipeline +to index a document. +The incoming document +should include the `field` +specified in your enrich processor. + +[source,console] +---- +PUT /users/_doc/0?pipeline=postal_lookup +{ + "first_name": "Mardy", + "last_name": "Brown", + "geo_location": "POINT (13.5 52.5)" +} +---- +// TEST[continued] + +To verify the enrich processor matched +and appended the appropriate field data, +use the <> +to view the indexed document. + +[source,console] +---- +GET /users/_doc/0 +---- +// TEST[continued] + +The API returns the following response: + +[source,console-result] +---- +{ + "found": true, + "_index": "users", + "_type": "_doc", + "_id": "0", + "_version": 1, + "_seq_no": 55, + "_primary_term": 1, + "_source": { + "geo_data": [ + { + "location": { + "type": "envelope", + "coordinates": [[13.0, 53.0], [14.0, 52.0]] + }, + "postal_code": "96598" + } + ], + "first_name": "Mardy", + "last_name": "Brown", + "geo_location": "POINT (13.5 52.5)" + } +} +---- +// TESTRESPONSE[s/"_seq_no": \d+/"_seq_no" : $body._seq_no/ s/"_primary_term":1/"_primary_term" : $body._primary_term/] + +//// +[source,console] +-------------------------------------------------- +DELETE /_ingest/pipeline/postal_lookup + +DELETE /_enrich/policy/postal_policy +-------------------------------------------------- +// TEST[continued] +//// \ No newline at end of file diff --git a/docs/reference/ingest/enrich.asciidoc b/docs/reference/ingest/enrich.asciidoc index a9d4555fd39..c161d928e87 100644 --- a/docs/reference/ingest/enrich.asciidoc +++ b/docs/reference/ingest/enrich.asciidoc @@ -12,6 +12,7 @@ For example, you can use the enrich processor to: * Identify web services or vendors based on known IP addresses * Add product information to retail orders based on product IDs * Supplement contact information based on an email address +* Add postal codes based on user coordinates [float] @@ -31,7 +32,7 @@ follow these steps: Once you have an enrich processor set up, you can <> and <> -using the <> APIs. +using the <>. [IMPORTANT] ==== @@ -59,17 +60,17 @@ include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-polic To begin, create one or more source indices. -A *source index* contains data you want to append to incoming documents. +A _source index_ contains data you want to append to incoming documents. You can index and manage documents in a source index like a regular index. -The following <> API request creates the `users` source index +The following <> request creates the `users` source index containing user data. This request also indexes a new document to the `users` source index. [source,console] ---- -PUT /users/_doc/1?refresh +PUT /users/_doc/1?refresh=wait_for { "email": "mardy.brown@asciidocsmith.com", "first_name": "Mardy", @@ -93,7 +94,7 @@ See {beats-ref}/getting-started.html[Getting started with {beats}]. [[create-enrich-policy]] ==== Create an enrich policy -Use the <> API +Use the <> to create an enrich policy. include::{docdir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-policy-def] @@ -116,7 +117,7 @@ PUT /_enrich/policy/users-policy [[execute-enrich-policy]] ==== Execute an enrich policy -Use the <> API +Use the <> to create an enrich index for the policy. include::apis/enrich/execute-enrich-policy.asciidoc[tag=execute-enrich-policy-def] @@ -136,7 +137,7 @@ POST /_enrich/policy/users-policy/_execute [[add-enrich-processor]] ==== Add the enrich processor to an ingest pipeline -Use the <> API +Use the <> to create an ingest pipeline. Include an <> that uses your enrich policy. @@ -144,14 +145,12 @@ that uses your enrich policy. When defining an enrich processor, you must include the following: -* The *field* used to match incoming documents +* The field used to match incoming documents to documents in the enrich index. + This field should be included in incoming documents. -To match, this field must contain the exact -value of the match field of a document in the enrich index. -* The *target field* added to incoming documents. +* The target field added to incoming documents. This field contains all appended enrich data. The following request adds a new pipeline, `user_lookup`. @@ -204,10 +203,10 @@ the processor appends data from those documents to the array. If the incoming document matches no documents in the enrich index, the processor appends no data. -The following <> API request uses the ingest pipeline +The following <> request uses the ingest pipeline to index a document -containing the `email` field, -the `match_field` specified in the `users-policy` enrich policy. +containing the `email` field +specified in the enrich processor. [source,console] ---- @@ -220,7 +219,7 @@ PUT /my_index/_doc/my_id?pipeline=user_lookup To verify the enrich processor matched and appended the appropriate field data, -use the <> API to view the indexed document. +use the <> to view the indexed document. [source,console] ---- diff --git a/docs/reference/ingest/processors/enrich.asciidoc b/docs/reference/ingest/processors/enrich.asciidoc index 17c8e15aa9e..d5af9d46991 100644 --- a/docs/reference/ingest/processors/enrich.asciidoc +++ b/docs/reference/ingest/processors/enrich.asciidoc @@ -16,7 +16,18 @@ check out the <> to get familiar with enrich pol | `field` | yes | - | The field in the input document that matches the policies match_field used to retrieve the enrichment data. | `target_field` | yes | - | The field that will be used for the enrichment data. | `ignore_missing` | no | false | If `true` and `field` does not exist, the processor quietly exits without modifying the document -| `override` | no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched. -| `max_matches` | no | 1 | The maximum number of matched documents to include under the configured target field. In order to avoid documents getting too large, the maximum allowed value is 128. +| `override` | no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched. +| `max_matches` | no | 1 | The maximum number of matched documents to include under the configured target field. In order to avoid documents getting too large, the maximum allowed value is 128. +| `shape_relation` | no | `INTERSECTS` a| Spatial relation operator +used to match the <> of incoming documents +to documents in the enrich index. ++ +This option is only used for `geo_match` enrich policy types. ++ +The <> mapping parameter determines +which spatial relation operators are availlble. +See <<_spatial_relations>> +for operators and more information. + include::common-options.asciidoc[] |======