OpenSearch/docs/reference/ingest.asciidoc

42 lines
1.6 KiB
Plaintext
Raw Normal View History

2016-02-11 17:16:56 -05:00
[[ingest]]
= Ingest Node
[partintro]
--
2016-03-04 01:00:07 -05:00
You can use ingest node to pre-process documents before the actual indexing takes place.
2016-02-11 17:16:56 -05:00
This pre-processing happens by an ingest node that intercepts bulk and index requests, applies the
2016-03-04 01:00:07 -05:00
transformations, and then passes the documents back to the index or bulk APIs.
2016-02-11 17:16:56 -05:00
2016-03-04 01:00:07 -05:00
You can enable ingest on any node or even have dedicated ingest nodes. Ingest is enabled by default
on all nodes. To disable ingest on a node, configure the following setting in the `elasticsearch.yml` file:
2016-02-11 17:16:56 -05:00
[source,yaml]
--------------------------------------------------
node.ingest: false
--------------------------------------------------
To pre-process documents before indexing, you <<pipeline,define a pipeline>> that specifies
2016-03-04 01:00:07 -05:00
a series of <<ingest-processors,processors>>. Each processor transforms the document in some way.
For example, you may have a pipeline that consists of one processor that removes a field from
the document followed by another processor that renames a field. Configured pipelines are then stored
in the <<cluster-state,cluster state>>.
2016-02-11 17:16:56 -05:00
2016-03-04 01:00:07 -05:00
To use a pipeline, you simply specify the `pipeline` parameter on an index or bulk request to
tell the ingest node which pipeline to use. For example:
2016-02-11 17:16:56 -05:00
[source,js]
--------------------------------------------------
PUT my-index/_doc/my-id?pipeline=my_pipeline_id
2016-02-11 17:16:56 -05:00
{
"foo": "bar"
2016-02-11 17:16:56 -05:00
}
--------------------------------------------------
// CONSOLE
// TEST[catch:bad_request]
2016-02-11 17:16:56 -05:00
See <<ingest-apis,Ingest APIs>> for more information about creating, adding, and deleting pipelines.
2016-03-04 01:00:07 -05:00
2016-02-11 17:16:56 -05:00
--
include::ingest/ingest-node.asciidoc[]