2022-02-04 10:37:42 -05:00
---
layout: default
2023-08-29 16:07:34 -04:00
title: Simulate pipeline
nav_order: 11
2023-05-04 13:44:02 -04:00
redirect_from:
- /opensearch/rest-api/ingest-apis/simulate-ingest/
2023-10-17 17:22:28 -04:00
- /api-reference/ingest-apis/simulate-ingest/
2022-02-04 10:37:42 -05:00
---
2023-08-29 16:07:34 -04:00
# Simulate pipeline
2023-10-23 10:53:52 -04:00
**Introduced 1.0**
{: .label .label-purple }
2022-02-04 10:37:42 -05:00
2023-08-29 16:07:34 -04:00
Use the simulate ingest pipeline API operation to run or test the pipeline.
2022-02-04 10:37:42 -05:00
## Path and HTTP methods
2023-08-29 16:07:34 -04:00
The following requests **simulate the latest ingest pipeline created** :
2022-02-04 10:37:42 -05:00
```
GET _ingest/pipeline/_simulate
POST _ingest/pipeline/_simulate
```
2023-08-29 16:07:34 -04:00
{% include copy-curl.html %}
2022-02-04 10:37:42 -05:00
2023-08-29 16:07:34 -04:00
The following requests **simulate a single pipeline based on the pipeline ID** :
2022-02-04 10:37:42 -05:00
```
2023-08-29 16:07:34 -04:00
GET _ingest/pipeline/< pipeline-id > /_simulate
POST _ingest/pipeline/< pipeline-id > /_simulate
2022-02-04 10:37:42 -05:00
```
2023-08-29 16:07:34 -04:00
{% include copy-curl.html %}
2022-02-04 10:37:42 -05:00
## Request body fields
2023-08-29 16:07:34 -04:00
The following table lists the request body fields used to run a pipeline.
2022-02-07 13:49:43 -05:00
Field | Required | Type | Description
:--- | :--- | :--- | :---
2023-08-29 16:07:34 -04:00
`docs` | Required | Array | The documents to be used to test the pipeline.
`pipeline` | Optional | Object | The pipeline to be simulated. If the pipeline identifier is not included, then the response simulates the latest pipeline created.
2022-02-04 10:37:42 -05:00
2023-08-29 16:07:34 -04:00
The `docs` field can include subfields listed in the following table.
2022-02-04 10:37:42 -05:00
2022-02-07 13:49:43 -05:00
Field | Required | Type | Description
2023-08-29 16:07:34 -04:00
:--- | :--- | :--- | :---
`source` | Required | Object | The document's JSON body.
`id` | Optional | String | A unique document identifier. The identifier cannot be used elsewhere in the index.
`index` | Optional | String | The index where the document's transformed data appears.
## Query parameters
2022-02-04 10:37:42 -05:00
2023-08-29 16:07:34 -04:00
The following table lists the query parameters for running a pipeline.
2022-02-04 10:37:42 -05:00
2023-08-29 16:07:34 -04:00
Parameter | Type | Description
:--- | :--- | :---
`verbose` | Boolean | Verbose mode. Display data output for each processor in the executed pipeline.
2022-02-07 13:49:43 -05:00
2023-08-29 16:07:34 -04:00
#### Example: Specify a pipeline in the path
2022-02-04 10:37:42 -05:00
```json
2023-08-29 16:07:34 -04:00
POST /_ingest/pipeline/my-pipeline/_simulate
2022-02-04 10:37:42 -05:00
{
2023-08-29 16:07:34 -04:00
"docs": [
2022-02-04 10:37:42 -05:00
{
2023-08-29 16:07:34 -04:00
"_index": "my-index",
"_id": "1",
"_source": {
"grad_year": 2024,
"graduated": false,
"name": "John Doe"
2022-02-04 10:37:42 -05:00
}
},
{
2023-08-29 16:07:34 -04:00
"_index": "my-index",
"_id": "2",
"_source": {
"grad_year": 2025,
"graduated": false,
"name": "Jane Doe"
2022-02-04 10:37:42 -05:00
}
}
]
}
2022-02-07 14:22:55 -05:00
```
2023-08-29 16:07:34 -04:00
{% include copy-curl.html %}
2022-02-04 10:37:42 -05:00
2023-08-29 16:07:34 -04:00
The request returns the following response:
2022-02-04 10:37:42 -05:00
```json
{
2023-08-29 16:07:34 -04:00
"docs": [
2022-02-04 10:37:42 -05:00
{
2023-08-29 16:07:34 -04:00
"doc": {
"_index": "my-index",
"_id": "1",
"_source": {
"name": "JOHN DOE",
"grad_year": 2023,
"graduated": true
2022-02-04 10:37:42 -05:00
},
2023-08-29 16:07:34 -04:00
"_ingest": {
"timestamp": "2023-06-20T23:19:54.635306588Z"
2022-02-04 10:37:42 -05:00
}
}
},
{
2023-08-29 16:07:34 -04:00
"doc": {
"_index": "my-index",
"_id": "2",
"_source": {
"name": "JANE DOE",
"grad_year": 2023,
"graduated": true
2022-02-04 10:37:42 -05:00
},
2023-08-29 16:07:34 -04:00
"_ingest": {
"timestamp": "2023-06-20T23:19:54.635746046Z"
2022-02-04 10:37:42 -05:00
}
}
}
]
}
```
2023-10-23 10:53:52 -04:00
#### Example: Verbose mode
2022-02-04 10:37:42 -05:00
2023-08-29 16:07:34 -04:00
When the previous request is run with the `verbose` parameter set to `true` , the response shows the sequence of transformations for each document. For example, for the document with the ID `1` , the response contains the results of applying each processor in the pipeline in sequence:
2022-02-04 10:37:42 -05:00
```json
{
2023-08-29 16:07:34 -04:00
"docs": [
2022-02-04 10:37:42 -05:00
{
2023-08-29 16:07:34 -04:00
"processor_results": [
2022-02-04 10:37:42 -05:00
{
2023-08-29 16:07:34 -04:00
"processor_type": "set",
"status": "success",
"description": "Sets the graduation year to 2023",
"doc": {
"_index": "my-index",
"_id": "1",
"_source": {
"name": "John Doe",
"grad_year": 2023,
"graduated": false
2022-02-04 10:37:42 -05:00
},
2023-08-29 16:07:34 -04:00
"_ingest": {
"pipeline": "my-pipeline",
"timestamp": "2023-06-20T23:23:26.656564631Z"
2022-02-04 10:37:42 -05:00
}
}
2023-08-29 16:07:34 -04:00
},
2022-02-04 10:37:42 -05:00
{
2023-08-29 16:07:34 -04:00
"processor_type": "set",
"status": "success",
"description": "Sets 'graduated' to true",
"doc": {
"_index": "my-index",
"_id": "1",
"_source": {
"name": "John Doe",
"grad_year": 2023,
"graduated": true
2022-02-04 10:37:42 -05:00
},
2023-08-29 16:07:34 -04:00
"_ingest": {
"pipeline": "my-pipeline",
"timestamp": "2023-06-20T23:23:26.656564631Z"
}
}
},
{
"processor_type": "uppercase",
"status": "success",
"doc": {
"_index": "my-index",
"_id": "1",
"_source": {
"name": "JOHN DOE",
"grad_year": 2023,
"graduated": true
},
"_ingest": {
"pipeline": "my-pipeline",
"timestamp": "2023-06-20T23:23:26.656564631Z"
2022-02-04 10:37:42 -05:00
}
}
}
]
}
]
}
2023-08-29 16:07:34 -04:00
```
2023-10-23 10:53:52 -04:00
#### Example: Specify a pipeline in the request body
2023-08-29 16:07:34 -04:00
Alternatively, you can specify a pipeline directly in the request body without first creating a pipeline:
```json
POST /_ingest/pipeline/_simulate
{
"pipeline" :
{
2024-01-02 14:27:35 -05:00
"description": "Splits text on white space characters",
2023-08-29 16:07:34 -04:00
"processors": [
{
"csv" : {
"field" : "name",
"separator": ",",
"target_fields": ["last_name", "first_name"],
"trim": true
}
},
{
"uppercase": {
"field": "last_name"
}
}
]
},
"docs": [
{
"_index": "second-index",
"_id": "1",
"_source": {
"name": "Doe,John"
}
},
{
"_index": "second-index",
"_id": "2",
"_source": {
"name": "Doe, Jane"
}
}
]
}
```
{% include copy-curl.html %}
#### Response
The request returns the following response:
```json
{
"docs": [
{
"doc": {
"_index": "second-index",
"_id": "1",
"_source": {
"name": "Doe,John",
"last_name": "DOE",
"first_name": "John"
},
"_ingest": {
"timestamp": "2023-08-24T19:20:44.816219673Z"
}
}
},
{
"doc": {
"_index": "second-index",
"_id": "2",
"_source": {
"name": "Doe, Jane",
"last_name": "DOE",
"first_name": "Jane"
},
"_ingest": {
"timestamp": "2023-08-24T19:20:44.816492381Z"
}
}
}
]
}
```