Merge branch 'main' into improve_cluster_health

This commit is contained in:
Naarcha-AWS 2022-05-18 14:53:22 -05:00 committed by GitHub
commit 46d7a45ba4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 64 additions and 208 deletions

View File

@ -12,23 +12,26 @@ OpenSearch can operate as a single-node or multi-node cluster. The steps to conf
To create and deploy an OpenSearch cluster according to your requirements, its important to understand how node discovery and cluster formation work and what settings govern them.
There are many ways to design a cluster. The following illustration shows a basic architecture:
There are many ways to design a cluster. The following illustration shows a basic architecture that includes a four-node cluster that has one dedicated cluster manager node, one dedicated coordinating node, and two data nodes that are cluster manager eligible and also used for ingesting data.
The nomenclature recently changed for the master node; it is now called the cluster manager node.
{: .note }
![multi-node cluster architecture diagram]({{site.url}}{{site.baseurl}}/images/cluster.png)
This is a four-node cluster that has one dedicated master node, one dedicated coordinating node, and two data nodes that are master-eligible and also used for ingesting data.
### Nodes
The following table provides brief descriptions of the node types:
Node type | Description | Best practices for production
:--- | :--- | :-- |
`Master` | Manages the overall operation of a cluster and keeps track of the cluster state. This includes creating and deleting indices, keeping track of the nodes that join and leave the cluster, checking the health of each node in the cluster (by running ping requests), and allocating shards to nodes. | Three dedicated master nodes in three different zones is the right approach for almost all production use cases. This configuration ensures your cluster never loses quorum. Two nodes will be idle for most of the time except when one node goes down or needs some maintenance.
`Master-eligible` | Elects one node among them as the master node through a voting process. | For production clusters, make sure you have dedicated master nodes. The way to achieve a dedicated node type is to mark all other node types as false. In this case, you have to mark all the other nodes as not master-eligible.
`Data` | Stores and searches data. Performs all data-related operations (indexing, searching, aggregating) on local shards. These are the worker nodes of your cluster and need more disk space than any other node type. | As you add data nodes, keep them balanced between zones. For example, if you have three zones, add data nodes in multiples of three, one for each zone. We recommend using storage and RAM-heavy nodes.
`Ingest` | Preprocesses data before storing it in the cluster. Runs an ingest pipeline that transforms your data before adding it to an index. | If you plan to ingest a lot of data and run complex ingest pipelines, we recommend you use dedicated ingest nodes. You can also optionally offload your indexing from the data nodes so that your data nodes are used exclusively for searching and aggregating.
`Coordinating` | Delegates client requests to the shards on the data nodes, collects and aggregates the results into one final result, and sends this result back to the client. | A couple of dedicated coordinating-only nodes is appropriate to prevent bottlenecks for search-heavy workloads. We recommend using CPUs with as many cores as you can.
Cluster manager | Manages the overall operation of a cluster and keeps track of the cluster state. This includes creating and deleting indexes, keeping track of the nodes that join and leave the cluster, checking the health of each node in the cluster (by running ping requests), and allocating shards to nodes. | Three dedicated cluster manager nodes in three different zones is the right approach for almost all production use cases. This configuration ensures your cluster never loses quorum. Two nodes will be idle for most of the time except when one node goes down or needs some maintenance.
Cluster manager eligible | Elects one node among them as the cluster manager node through a voting process. | For production clusters, make sure you have dedicated cluster manager nodes. The way to achieve a dedicated node type is to mark all other node types as false. In this case, you have to mark all the other nodes as not cluster manager eligible.
Data | Stores and searches data. Performs all data-related operations (indexing, searching, aggregating) on local shards. These are the worker nodes of your cluster and need more disk space than any other node type. | As you add data nodes, keep them balanced between zones. For example, if you have three zones, add data nodes in multiples of three, one for each zone. We recommend using storage and RAM-heavy nodes.
Ingest | Pre-processes data before storing it in the cluster. Runs an ingest pipeline that transforms your data before adding it to an index. | If you plan to ingest a lot of data and run complex ingest pipelines, we recommend you use dedicated ingest nodes. You can also optionally offload your indexing from the data nodes so that your data nodes are used exclusively for searching and aggregating.
Coordinating | Delegates client requests to the shards on the data nodes, collects and aggregates the results into one final result, and sends this result back to the client. | A couple of dedicated coordinating-only nodes is appropriate to prevent bottlenecks for search-heavy workloads. We recommend using CPUs with as many cores as you can.
By default, each node is a master-eligible, data, ingest, and coordinating node. Deciding on the number of nodes, assigning node types, and choosing the hardware for each node type depends on your use case. You must take into account factors like the amount of time you want to hold on to your data, the average size of your documents, your typical workload (indexing, searches, aggregations), your expected price-performance ratio, your risk tolerance, and so on.
By default, each node is a cluster-manager-eligible, data, ingest, and coordinating node. Deciding on the number of nodes, assigning node types, and choosing the hardware for each node type depends on your use case. You must take into account factors like the amount of time you want to hold on to your data, the average size of your documents, your typical workload (indexing, searches, aggregations), your expected price-performance ratio, your risk tolerance, and so on.
After you assess all these requirements, we recommend you use a benchmark testing tool like Rally to provision a small sample cluster and run tests with varying workloads and configurations. Compare and analyze the system and query metrics for these tests to design an optimum architecture. To get started with Rally, see the [Rally documentation](https://esrally.readthedocs.io/en/stable/).
@ -62,18 +65,18 @@ Make the same change on all the nodes to make sure that they'll join to form a c
After you name the cluster, set node attributes for each node in your cluster.
#### Master node
#### Cluster manager node
Give your master node a name. If you don't specify a name, OpenSearch assigns a machine-generated name that makes the node difficult to monitor and troubleshoot.
Give your cluster manager node a name. If you don't specify a name, OpenSearch assigns a machine-generated name that makes the node difficult to monitor and troubleshoot.
```yml
node.name: opensearch-master
node.name: opensearch-cluster_manager
```
You can also explicitly specify that this node is a master node. This is already true by default, but adding it makes it easier to identify the master node.
You can also explicitly specify that this node is a cluster manager node, even though it is already set to true by default. Set the node role to `cluster_manager` to make it easier to identify the cluster manager node.
```yml
node.roles: [ master ]
node.roles: [ cluster_manager ]
```
#### Data nodes
@ -88,7 +91,7 @@ node.name: opensearch-d1
node.name: opensearch-d2
```
You can make them master-eligible data nodes that will also be used for ingesting data:
You can make them cluster-manager-eligible data nodes that will also be used for ingesting data:
```yml
node.roles: [ data, ingest ]
@ -132,9 +135,9 @@ Now that you've configured the network hosts, you need to configure the discover
Zen Discovery is the built-in, default mechanism that uses [unicast](https://en.wikipedia.org/wiki/Unicast) to find other nodes in the cluster.
You can generally just add all your master-eligible nodes to the `discovery.seed_hosts` array. When a node starts up, it finds the other master-eligible nodes, determines which one is the master, and asks to join the cluster.
You can generally just add all of your cluster-manager-eligible nodes to the `discovery.seed_hosts` array. When a node starts up, it finds the other cluster-manager-eligible nodes, determines which one is the cluster manager, and asks to join the cluster.
For example, for `opensearch-master` the line looks something like this:
For example, for `opensearch-cluster_manager` the line looks something like this:
```yml
discovery.seed_hosts: ["<private IP of opensearch-d1>", "<private IP of opensearch-d2>", "<private IP of opensearch-c1>"]
@ -161,8 +164,8 @@ curl -XGET https://<private-ip>:9200/_cat/nodes?v -u 'admin:admin' --insecure
```
```
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
x.x.x.x 13 61 0 0.02 0.04 0.05 mi * opensearch-master
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role cluster_manager name
x.x.x.x 13 61 0 0.02 0.04 0.05 mi * opensearch-cluster_manager
x.x.x.x 16 60 0 0.06 0.05 0.05 md - opensearch-d1
x.x.x.x 34 38 0 0.12 0.07 0.06 md - opensearch-d2
x.x.x.x 23 38 0 0.12 0.07 0.06 md - opensearch-c1

View File

@ -24,7 +24,7 @@ services:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_master_nodes=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
- network.host=0.0.0.0 # required if not using the demo security configuration
@ -60,7 +60,7 @@ services:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_master_nodes=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
- network.host=0.0.0.0

View File

@ -1,157 +0,0 @@
---
layout: default
title: RPM
parent: Install OpenSearch
nav_order: 51
---
# RPM
The RPM Package Manager (RPM) installation provides everything you need to run OpenSearch inside Red Hat or Red Hat-based Linux Distributions.
RPM supports CentOS 7 and 8, and Amazon Linux 2. If you have your own Java installation and set `JAVA_HOME` in your terminal application, macOS works, as well.
There are two methods for installing OpenSearch on RPM:
## Manual method
1. Download the RPM package directly from the [OpenSearch downloads page](https://opensearch.org/downloads.html){:target='\_blank'}. The RPM package can be download both as `x64` and `arm64`.
2. Import the public GPG key. This key verifies that the your OpenSearch instance is signed.
```bash
sudo rpm --import https://artifacts.opensearch.org/publickeys/opensearch.pgp
```
3. On your host, use `sudo yum install` or `sudo rpm -ivh` to install the package.
**x64**
```bash
sudo yum install opensearch-{{site.opensearch_version}}-linux-x64.rpm
sudo yum install opensearch-dashboards-{{site.opensearch_version}}-linux-x64.rpm
```
```bash
sudo rpm -ivh opensearch-{{site.opensearch_version}}-linux-x64.rpm
sudo rpm -ivh opensearch-dashboards-{{site.opensearch_version}}-linux-x64.rpm
```
**arm64**
```bash
sudo yum install opensearch-{{site.opensearch_version}}-linux-x64.rpm
sudo yum install opensearch-dashboards-{{site.opensearch_version}}-linux-arm64.rpm
```
```bash
sudo rpm -ivh opensearch-{{site.opensearch_version}}-linux-x64.rpm
sudo rpm -ivh opensearch-dashboards-{{site.opensearch_version}}-linux-arm64.rpm
```
Once complete, you can run OpenSearch inside your distribution.
## YUM method
YUM, an RPM package management tool, allows you to pull the RPM package from the YUM repository library.
1. Create a repository file for both OpenSearch and OpenSearch Dashboards:
```bash
sudo curl -SL https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/opensearch-{{site.opensearch_version}}.repo -o /etc/yum.repos.d/{{site.opensearch_version}}.repo
```
```bash
sudo curl -SL https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/{{site.opensearch_version}}/opensearch-dashboards-{{site.opensearch_version}}.repo -o /etc/yum.repos.d/{{site.opensearch_version}}.repo
```
To verify that the repos appear in your repo list, use `sudo yum repolist`.
2. Clean your YUM cache, to ensure a smooth installation:
```bash
sudo yum clean all
```
3. With the repository file downloaded, list all available versions of OpenSearch:
```bash
sudo yum list | grep opensearch
```
4. Choose the version of OpenSearch you want to install:
```bash
sudo yum install opensearch
sudo yum install opensearch-dashboards
```
Unless otherwise indicated, the highest minor version of OpenSearch installs.
To install a specific version of OpenSearch:
```bash
sudo yum install 'opensearch-{{site.opensearch_version}}'
```
5. During installation, the installer stops to see if the GPG key matches the OpenSearch project. Verify that the `Fingerprint` matches the following:
```bash
Fingerprint: c5b7 4989 65ef d1c2 924b a9d5 39d3 1987 9310 d3fc
```
If correct, enter `yes` or `y`. The OpenSearch installation continues.
Once complete, you can run OpenSearch inside your distribution.
## Run OpenSearch
1. Run OpenSearch and OpenSearch Dashboards using `systemctl`.
```bash
sudo systemctl start opensearch.service
sudo systemctl start opensearch-dashboards.service
```
2. Send requests to the server to verify that OpenSearch is running:
```bash
curl -XGET https://localhost:9200 -u 'admin:admin' --insecure
curl -XGET https://localhost:9200/_cat/config?v -u 'admin:admin' --insecure
```
3. To stop running OpenSearch, enter:
```bash
sudo systemctl stop opensearch.service
sudo systemctl stop opensearch-dashboards.service
```
## *(Optional)* Set up Performance Analyzer
When enabled, the Performance Analyzer plugin collects data related to the performance of your OpenSearch instance. To start the Performance Analyzer plugin, enter:
```bash
sudo systemctl start opensearch-performance-analyzer.service
```
To stop the Performance Analyzer, enter:
```bash
sudo systemctl stop opensearch-performance-analyzer.service
```
## Upgrade RPM
You can upgrade your RPM OpenSearch instance both manually and through YUM.
### Manual
Download the new version of OpenSearch you want to use, and then use `rmp -Uvh` to upgrade.
### YUM
To upgrade to the latest version of OpenSearch with YUM, use `sudo yum update`. You can also upgrade to a specific OpenSearch version by using `sudo yum update opensearch-<version-number>`.

View File

@ -1,44 +1,44 @@
---
layout: default
title: cat master
title: CAT cluster manager
parent: CAT
grand_parent: REST API reference
nav_order: 30
has_children: false
---
# cat master
# CAT cluster_manager
Introduced 1.0
{: .label .label-purple }
The cat master operation lists information that helps identify the elected master node.
The cat cluster manager operation lists information that helps identify the elected cluster manager node.
## Example
```
GET _cat/master?v
GET _cat/cluster_manager?v
```
## Path and HTTP methods
```
GET _cat/master
GET _cat/cluster_manager
```
## URL parameters
All cat master URL parameters are optional.
All cat cluster manager URL parameters are optional.
In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/cat/index#common-url-parameters), you can specify the following parameters:
Parameter | Type | Description
:--- | :--- | :---
master_timeout | Time | The amount of time to wait for a connection to the master node. Default is 30 seconds.
cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds.
## Response
```json
id | host | ip | node
ZaIkkUd4TEiAihqJGkp5CA | 172.18.0.3 | 172.18.0.3 | odfe-node2
ZaIkkUd4TEiAihqJGkp5CA | 172.18.0.3 | 172.18.0.3 | opensearch-node2
```

View File

@ -13,7 +13,7 @@ Introduced 1.0
The cat nodes operation lists node-level information, including node roles and load metrics.
A few important node metrics are `pid`, `name`, `master`, `ip`, `port`, `version`, `build`, `jdk`, along with `disk`, `heap`, `ram`, and `file_desc`.
A few important node metrics are `pid`, `name`, `cluster_manager`, `ip`, `port`, `version`, `build`, `jdk`, along with `disk`, `heap`, `ram`, and `file_desc`.
## Example
@ -37,8 +37,8 @@ Parameter | Type | Description
:--- | :--- | :---
bytes | Byte size | Specify the units for byte size. For example, `7kb` or `6gb`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).
full_id | Boolean | If true, return the full node ID. If false, return the shortened node ID. Defaults to false.
local | Boolean | Whether to return information from the local node only instead of from the master node. Default is false.
master_timeout | Time | The amount of time to wait for a connection to the master node. Default is 30 seconds.
local | Boolean | Whether to return information from the local node only instead of from the cluster_manager node. Default is false.
cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds.
time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).
include_unloaded_segments | Boolean | Whether to include information from segments not loaded into memory. Default is false.
@ -46,7 +46,9 @@ include_unloaded_segments | Boolean | Whether to include information from segmen
## Response
```json
ip | heap.percent | ram.percent | cpu load_1m | load_5m | load_15m | node.role | master | name
172.18.0.3 | 31 | 97 | 3 | 0.03 | 0.10 | 0.14 dimr | * | odfe-node2
172.18.0.4 | 45 | 97 | 3 | 0.19 | 0.14 | 0.15 dimr | - | odfe-node1
ip | heap.percent | ram.percent | cpu load_1m | load_5m | load_15m | node.role | cluster_manager | name
172.18.0.3 | 31 | 97 | 3 | 0.03 | 0.10 | 0.14 dimr | * | opensearch-node2
172.18.0.4 | 45 | 97 | 3 | 0.19 | 0.14 | 0.15 dimr | - | opensearch-node1
```

View File

@ -33,8 +33,8 @@ In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/opensear
Parameter | Type | Description
:--- | :--- | :---
local | Boolean | Whether to return information from the local node only instead of from the master node. Default is false.
master_timeout | Time | The amount of time to wait for a connection to the master node. Default is 30 seconds.
local | Boolean | Whether to return information from the local node only instead of from the cluster_manager node. Default is false.
cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds.
time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).

View File

@ -1,6 +1,6 @@
---
layout: default
title: CAT
title: CAT API
parent: REST API reference
nav_order: 100
has_children: true
@ -8,15 +8,15 @@ redirect_from:
- /opensearch/catapis/
---
# cat API
# Compact and aligned text (CAT) API
You can get essential statistics about your cluster in an easy-to-understand, tabular format using the compact and aligned text (CAT) API. The cat API is a human-readable interface that returns plain text instead of traditional JSON.
You can get essential statistics about your cluster in an easy-to-understand, tabular format using the compact and aligned text (CAT) API. The CAT API is a human-readable interface that returns plain text instead of traditional JSON.
Using the cat API, you can answer questions like which node is the elected master, what state is the cluster in, how many documents are in each index, and so on.
Using the CAT API, you can answer questions like which node is the elected master, what state is the cluster in, how many documents are in each index, and so on.
## Example
To see the available operations in the cat API, use the following command:
To see the available operations in the CAT API, use the following command:
```
GET _cat

View File

@ -36,10 +36,15 @@ All cluster health parameters are optional.
Parameter | Type | Description
:--- | :--- | :---
expand_wildcards | Enum | Expands wildcard expressions to concrete indices. Combine multiple values with commas. Supported values are `all`, `open`, `closed`, `hidden`, and `none`. Default is `open`.
expand_wildcards | Enum | Expands wildcard expressions to concrete indexes. Combine multiple values with commas. Supported values are `all`, `open`, `closed`, `hidden`, and `none`. Default is `open`.
level | Enum | The level of detail for returned health information. Supported values are `cluster`, `indices`, and `shards`. Default is `cluster`.
<<<<<<< improve_cluster_health
local | Boolean | Whether to return information from the local node only instead of from the master node. Default is false.
cluster_manager_timeout<br>(deprecated `master_timeout`) | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds.
=======
local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is false.
cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds.
>>>>>>> main
timeout | Time | The amount of time to wait for a response. If the timeout expires, the request fails. Default is 30 seconds.
wait_for_active_shards | String | Wait until the specified number of shards is active before returning a response. `all` for all shards. Default is `0`.
wait_for_nodes | String | Wait for N number of nodes. Use `12` for exact match, `>12` and `<12` for range.

View File

@ -44,7 +44,7 @@ Parameter | Type | Description
:--- | :--- | :---
flat_settings | Boolean | Whether to return settings in the flat form, which can improve readability, especially for heavily nested settings. For example, the flat form of `"cluster": { "max_shards_per_node": 500 }` is `"cluster.max_shards_per_node": "500"`.
include_defaults (GET only) | Boolean | Whether to include default settings as part of the response. This parameter is useful for identifying the names and current values of settings you want to update.
master_timeout | Time | The amount of time to wait for a response from the master node. Default is 30 seconds.
cluster_manager_timeout | Time | The amount of time to wait for a response from the cluster manager node. Default is 30 seconds.
timeout (PUT only) | Time | The amount of time to wait for a response from the cluster. Default is 30 seconds.

View File

@ -14,7 +14,7 @@ The cluster stats API operation returns statistics about your cluster.
## Examples
```json
GET _cluster/stats/nodes/_master
GET _cluster/stats/nodes/_cluster_manager
```
## Path and HTTP methods
@ -31,8 +31,10 @@ All cluster stats parameters are optional.
Parameter | Type | Description
:--- | :--- | :---
&lt;node-filters&gt; | List | A comma-separated list of node-filters that OpenSearch uses to filter results. Available options are `all`, `_local`, `_master`, a node name or ID, `master:true`, `master:false`, `data:true`, `data:false`, `ingest:true`, `ingest:false`, `voting_only:true`, `voting_only:false`, `ml:true`, `ml:false`, `coordinating_only:true`, `coordinating_only:false`, and &lt;custom node attributes&gt; : &lt;attribute values&gt; pairs.
&lt;node-filters&gt; | List | A comma-separated list of node-filters that OpenSearch uses to filter results. Available options are `all`, `_local`, `_cluster_manager`, a node name or ID, `cluster_manager:true`, `cluster_manager:false`, `data:true`, `data:false`, `ingest:true`, `ingest:false`, `voting_only:true`, `voting_only:false`, `ml:true`, `ml:false`, `coordinating_only:true`, `coordinating_only:false`, and &lt;custom node attributes&gt; : &lt;attribute values&gt; pairs.
Although the `master` node is now called `cluster_manager` for version 2.0, we retained the `master` field for backwards compatibility. If you have a node that has either a `master` role or a `cluster_manager` role, the `count` increases for both fields by 1. To see an example node count increase, see the Response sample.
{: .note }
## Response
@ -218,6 +220,7 @@ Parameter | Type | Description
"data": 1,
"ingest": 1,
"master": 1,
"cluster_manager": 1,
"remote_cluster_client": 1
},
"versions": [

View File

@ -44,7 +44,7 @@ If `enforced` is `true`:
"roles": [
"data",
"ingest",
"master",
"cluster_manager",
"remote_cluster_client"
],
"attributes": {
@ -151,7 +151,7 @@ If `enforced` is `false`:
"roles": [
"data",
"ingest",
"master",
"cluster_manager",
"remote_cluster_client"
],
"attributes": {
@ -264,7 +264,7 @@ GET _nodes/_local/stats/shard_indexing_pressure?include_all
"roles": [
"data",
"ingest",
"master",
"cluster_manager",
"remote_cluster_client"
],
"attributes": {
@ -379,7 +379,7 @@ If `enforced` is `true`:
"roles": [
"data",
"ingest",
"master",
"cluster_manager",
"remote_cluster_client"
],
"attributes": {
@ -422,7 +422,7 @@ If `enforced` is `false`:
"roles": [
"data",
"ingest",
"master",
"cluster_manager",
"remote_cluster_client"
],
"attributes": {
@ -471,7 +471,7 @@ GET _nodes/stats/shard_indexing_pressure
"roles": [
"data",
"ingest",
"master",
"cluster_manager",
"remote_cluster_client"
],
"attributes": {

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 26 KiB