Refactors JS client helper documentation (#1358)
* Fix Circuit Breaker section in JS client docs Fix #823 Signed-off-by: Robert Da Silva <mail@robdasilva.com> * Add documentation for JS client bulk helper Signed-off-by: Robert Da Silva <mail@robdasilva.com> * Refactors js client helper documentation * Incorporated tech review comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Incorporated doc review comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Update _clients/javascript/helpers.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _clients/javascript/helpers.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update helpers.md * Incorporated tech review feedback Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Implemented editorial comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: Robert Da Silva <mail@robdasilva.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Robert Da Silva <mail@robdasilva.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com>
This commit is contained in:
kolchfa-aws
GitHub
parent
6583bfeeff
commit
89f966a0f1
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: OpenSearch CLI
|
||||
nav_order: 52
|
||||
nav_order: 80
|
||||
has_children: false
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Go client
|
||||
nav_order: 80
|
||||
nav_order: 50
|
||||
---
|
||||
|
||||
# Go client
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Grafana
|
||||
nav_order: 150
|
||||
nav_order: 200
|
||||
has_children: false
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,15 +1,19 @@
|
|||
---
|
||||
layout: default
|
||||
title: Compatibility
|
||||
title: Language clients
|
||||
nav_order: 1
|
||||
has_children: false
|
||||
redirect_from:
|
||||
- /clients/
|
||||
---
|
||||
|
||||
# OpenSearch client compatibility
|
||||
# OpenSearch language clients
|
||||
|
||||
OpenSearch provides clients for several popular programming languages, with more coming. In general, clients are compatible with clusters running the same major version of OpenSearch (`major.minor.patch`).
|
||||
OpenSearch provides clients for several popular programming languages, with more to come.
|
||||
|
||||
## OpenSearch language client compatibility
|
||||
|
||||
In general, clients are compatible with clusters running the same major version of OpenSearch (`major.minor.patch`).
|
||||
|
||||
For example, a 1.0.0 client works with an OpenSearch 1.1.0 cluster, but might not support any non-breaking API changes in OpenSearch 1.1.0. A 1.2.0 client works with the same cluster, but might allow you to pass unsupported options in certain functions. We recommend using the same version for both, but if your tests pass after a cluster upgrade, you don't necessarily need to upgrade your clients immediately.
|
||||
|
||||
|
|
@ -17,7 +21,7 @@ For example, a 1.0.0 client works with an OpenSearch 1.1.0 cluster, but might no
|
|||
* [OpenSearch Java client]({{site.url}}{{site.baseurl}}/clients/java/)
|
||||
{% endcomment %}
|
||||
* [OpenSearch Python client]({{site.url}}{{site.baseurl}}/clients/python/)
|
||||
* [OpenSearch JavaScript (Node.js) client]({{site.url}}{{site.baseurl}}/clients/javascript/)
|
||||
* [OpenSearch JavaScript (Node.js) client]({{site.url}}{{site.baseurl}}/clients/javascript/index)
|
||||
* [OpenSearch .NET clients]({{site.url}}{{site.baseurl}}/clients/dot-net/)
|
||||
* [OpenSearch Go client]({{site.url}}{{site.baseurl}}/clients/go/)
|
||||
* [OpenSearch PHP client]({{site.url}}{{site.baseurl}}/clients/php/)
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Java high-level REST client
|
||||
nav_order: 60
|
||||
nav_order: 20
|
||||
---
|
||||
|
||||
# Java high-level REST client
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Java client
|
||||
nav_order: 65
|
||||
nav_order: 30
|
||||
---
|
||||
|
||||
# Java client
|
||||
|
|
|
|||
|
|
@ -0,0 +1,194 @@
|
|||
---
|
||||
layout: default
|
||||
title: Helper methods
|
||||
parent: JavaScript client
|
||||
nav_order: 2
|
||||
---
|
||||
|
||||
# Helper methods
|
||||
|
||||
Helper methods simplify the use of complicated API tasks.
|
||||
|
||||
## Bulk helper
|
||||
|
||||
The bulk helper simplifies making complex bulk API requests.
|
||||
|
||||
### Usage
|
||||
|
||||
The following code creates a bulk helper instance:
|
||||
|
||||
```javascript
|
||||
const { Client } = require('@opensearch-project/opensearch')
|
||||
const documents = require('./docs.json')
|
||||
|
||||
const client = new Client({ ... })
|
||||
|
||||
const result = await client.helpers.bulk({
|
||||
datasource: documents,
|
||||
onDocument (doc) {
|
||||
return {
|
||||
index: { _index: 'example-index' }
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
console.log(result)
|
||||
```
|
||||
|
||||
Bulk helper operations return an object with the following fields:
|
||||
|
||||
```json
|
||||
{
|
||||
total: number,
|
||||
failed: number,
|
||||
retry: number,
|
||||
successful: number,
|
||||
time: number,
|
||||
bytes: number,
|
||||
aborted: boolean
|
||||
}
|
||||
```
|
||||
|
||||
#### Bulk helper configuration options
|
||||
|
||||
When creating a new bulk helper instance, you can use the following configuration options.
|
||||
|
||||
| Option | Data Type | Required/Default | Description
|
||||
| :--- | :--- | :--- | :---
|
||||
| `datasource` | An array, async generator or a readable stream of strings or objects | Required | Represents the documents you need to create, delete, index, or update.
|
||||
| `onDocument` | Function | Required | A function to be invoked with each document in the given `datasource`. It returns the operation to be executed for this document. Optionally, the document can be manipulated for `create` and `index` operations by returning a new document as part of the function's result.
|
||||
| `concurrency` | Integer | Optional. Default is 5. | The number of requests to be executed in parallel.
|
||||
| `flushBytes` | Integer | Optional. Default is 5,000,000. | Maximum bulk body size to send in bytes.
|
||||
| `flushInterval` | Integer | Optional. Default is 30,000. | Time in milliseconds to wait before flushing the body after the last document has been read.
|
||||
| `onDrop` | Function | Optional. Default is `noop`. | A function to be invoked for every document that can’t be indexed after reaching the maximum number of retries.
|
||||
| `refreshOnCompletion` | Boolean | Optional. Default is false. | Whether or not a refresh should be run on all affected indexes at the end of the bulk operation.
|
||||
| `retries` | Integer | Optional. Defaults to the client's `maxRetries` value. | The number of times an operation is retried before `onDrop` is called for that document.
|
||||
| `wait` | Integer | Optional. Default is 5,000. | Time in milliseconds to wait before retrying an operation.
|
||||
|
||||
### Examples
|
||||
|
||||
The following examples illustrate the index, create, update, and delete bulk helper operations.
|
||||
|
||||
#### Index
|
||||
|
||||
The index operation creates a new document if it doesn’t exist and recreates the document if it already exists.
|
||||
|
||||
The following bulk operation indexes documents into `example-index`:
|
||||
|
||||
```javascript
|
||||
client.helpers.bulk({
|
||||
datasource: arrayOfDocuments,
|
||||
onDocument (doc) {
|
||||
return {
|
||||
index: { _index: 'example-index' }
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
The following bulk operation indexes documents into `example-index` with document overwrite:
|
||||
|
||||
```javascript
|
||||
client.helpers.bulk({
|
||||
datasource: arrayOfDocuments,
|
||||
onDocument (doc) {
|
||||
return [
|
||||
{
|
||||
index: { _index: 'example-index' }
|
||||
},
|
||||
{ ...doc, createdAt: new Date().toISOString() }
|
||||
]
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
#### Create
|
||||
|
||||
The create operation creates a new document only if the document does not already exist.
|
||||
|
||||
The following bulk operation creates documents in the `example-index`:
|
||||
|
||||
```javascript
|
||||
client.helpers.bulk({
|
||||
datasource: arrayOfDocuments,
|
||||
onDocument (doc) {
|
||||
return {
|
||||
create: { _index: 'example-index', _id: doc.id }
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
The following bulk operation creates documents in the `example-index` with document overwrite:
|
||||
|
||||
```javascript
|
||||
client.helpers.bulk({
|
||||
datasource: arrayOfDocuments,
|
||||
onDocument (doc) {
|
||||
return [
|
||||
{
|
||||
create: { _index: 'example-index', _id: doc.id }
|
||||
},
|
||||
{ ...doc, createdAt: new Date().toISOString() }
|
||||
]
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
#### Update
|
||||
|
||||
The update operation updates the document with the fields being sent. The document must already exist in the index.
|
||||
|
||||
The following bulk operation updates documents in the `arrayOfDocuments`:
|
||||
|
||||
```javascript
|
||||
client.helpers.bulk({
|
||||
datasource: arrayOfDocuments,
|
||||
onDocument (doc) {
|
||||
// The update operation always requires a tuple to be returned, with the
|
||||
// first element being the action and the second being the update options.
|
||||
return [
|
||||
{
|
||||
update: { _index: 'example-index', _id: doc.id }
|
||||
},
|
||||
{ doc_as_upsert: true }
|
||||
]
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
The following bulk operation updates documents in the `arrayOfDocuments` with document overwrite:
|
||||
|
||||
```javascript
|
||||
client.helpers.bulk({
|
||||
datasource: arrayOfDocuments,
|
||||
onDocument (doc) {
|
||||
return [
|
||||
{
|
||||
update: { _index: 'example-index', _id: doc.id }
|
||||
},
|
||||
{
|
||||
doc: { ...doc, createdAt: new Date().toISOString() },
|
||||
doc_as_upsert: true
|
||||
}
|
||||
]
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
#### Delete
|
||||
|
||||
The delete operation deletes a document.
|
||||
|
||||
The following bulk operation deletes documents from the `example-index`:
|
||||
|
||||
```javascript
|
||||
client.helpers.bulk({
|
||||
datasource: arrayOfDocuments,
|
||||
onDocument (doc) {
|
||||
return {
|
||||
delete: { _index: 'example-index', _id: doc.id }
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
layout: default
|
||||
title: JavaScript client
|
||||
nav_order: 100
|
||||
has_children: true
|
||||
nav_order: 40
|
||||
---
|
||||
|
||||
# JavaScript client
|
||||
# Getting started
|
||||
|
||||
The OpenSearch JavaScript (JS) client provides a safer and easier way to interact with your OpenSearch cluster. Rather than using OpenSearch from the browser and potentially exposing your data to the public, you can build an OpenSearch client that takes care of sending requests to your cluster.
|
||||
|
||||
|
|
@ -140,20 +141,22 @@ async function search() {
|
|||
search().catch(console.log);
|
||||
```
|
||||
|
||||
## Circuit Breaker
|
||||
## Circuit breaker
|
||||
|
||||
The `memoryCircuitBreaker` parameter in the [Cluster Settings API]({{site.url}}{{site.baseurl}}/opensearch/rest-api/cluster-settings/) gives you the ability to reject large query responses where the size of the response could crash OpenSearch Dashboards. To set the Circuit Breaker setting, use the `POST _cluster/settings` API operation on your active JS OpenSearch cluster.
|
||||
The `memoryCircuitBreaker` option can be used to prevent errors caused by a response payload being too large to fit into the heap memory available to the client.
|
||||
|
||||
`memoryCircuitBreaker` contains two fields:
|
||||
The `memoryCircuitBreaker` object contains two fields:
|
||||
|
||||
- `enabled`: A Boolean used to turn the Circuit Breaker on or off. Defaults to `false`.
|
||||
- `maxPercentage`: The threshold that determines whether the Circuit Breaker engages. The input range must be between `[0 ,1]`. Any number that exceeds that range will correct to `1.0`.
|
||||
- `enabled`: A Boolean used to turn the circuit breaker on or off. Defaults to `false`.
|
||||
- `maxPercentage`: The threshold that determines whether the circuit breaker engages. Valid values are floats in the [0, 1] range that represent percentages in decimal form. Any value that exceeds that range will correct to `1.0`.
|
||||
|
||||
The following example turns on the Circuit Breaker and sets the maximum percentage of a query response to 80% of the cluster's storage. You can customize this example for use in the `POST _cluster/settings` request body.
|
||||
The following example instantiates a client with the circuit breaker enabled and its threshold set to 80% of the available heap size limit:
|
||||
|
||||
```json
|
||||
memoryCircuitBreaker: {
|
||||
enabled: true,
|
||||
maxPercentage: 0.8
|
||||
}
|
||||
```javascript
|
||||
var client = new Client({
|
||||
memoryCircuitBreaker: {
|
||||
enabled: true,
|
||||
maxPercentage: 0.8,
|
||||
},
|
||||
});
|
||||
```
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: OpenSearch Kubernetes Operator
|
||||
nav_order: 210
|
||||
nav_order: 180
|
||||
---
|
||||
|
||||
The OpenSearch Kubernetes Operator is an open-source kubernetes operator that helps automate the deployment and provisioning of OpenSearch and OpenSearch Dashboards in a containerized environment. The operator can manage multiple OpenSearch clusters that can be scaled up and down depending on your needs.
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Logstash
|
||||
nav_order: 200
|
||||
nav_order: 150
|
||||
has_children: true
|
||||
has_toc: true
|
||||
redirect_from:
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: PHP client
|
||||
nav_order: 90
|
||||
nav_order: 70
|
||||
---
|
||||
|
||||
# PHP client
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Python client
|
||||
nav_order: 70
|
||||
nav_order: 10
|
||||
---
|
||||
|
||||
# Python client
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Ruby client
|
||||
nav_order: 77
|
||||
nav_order: 60
|
||||
has_children: false
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue