Merge pull request #40 from opensearch-project/migration

Initial migration content

_config.yml

@@ -27,6 +27,9 @@ color_scheme: opensearch
 # Define Jekyll collections
 collections:
   # Define a collection named "tests", its documents reside in the "_tests" directory
+  migrate:
+    permalink: /:collection/:path/
+    output: true
   opensearch:
     permalink: /:collection/:path/
     output: true
@@ -58,11 +61,14 @@ collections:
 just_the_docs:
   # Define the collections used in the theme
   collections:
-    opensearch:
+    migrate:
-      name: OpenSearch
+      name: Migrate to OpenSearch
       # nav_exclude: true
       nav_fold: true
       # search_exclude: true
+    opensearch:
+      name: OpenSearch
+      nav_fold: true
     dashboards:
       name: OpenSearch Dashboards
       nav_fold: true

_migrate/docker-migrate.md

@@ -0,0 +1,13 @@
+---
+layout: default
+title: Docker migration
+nav_order: 25
+---
+
+# Docker migration
+
+If you use a container orchestration system like Kubernetes (or manage your containers manually) and want to avoid downtime, think of the process not as an upgrade of each node, but as a decommissioning and replacement of each node. One by one, add OpenSearch nodes to the cluster and remove Elasticsearch OSS nodes, allowing time for all indices to return to a green status prior to proceeding.
+
+If you use Docker Compose, we highly recommend that you perform what amounts to a [cluster restart upgrade]({{site.url}}{{site.baseurl}}/migrate/upgrade-migrate/). Update your cluster configuration with new images, new settings, and new environment variables, and test it. Then stop and start the cluster. This process requires downtime, but takes very few steps and lets you continue to treat the cluster as a single entity that you can reliably deploy and redeploy.
+
+The most important step is to leave your data volumes intact. **Don't** run `docker-compose down -v`.

_migrate/index.md

@@ -0,0 +1,13 @@
+---
+layout: default
+title: About migrating
+nav_order: 1
+redirect_from: /migrate/
+---
+
+# About migrating
+
+The process of migrating from Elasticsearch OSS (including Open Distro for Elasticsearch) to OpenSearch varies depending on your current version of Elasticsearch OSS, install type, tolerance for downtime, and cost-sensitivity. Rather than concrete steps to cover every situation, we have general guidance for the process.
+
+To safeguard against data loss, we recommend that you take a [snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/) of all indices prior to any migration.
+{: .tip }

_migrate/snapshot-migrate.md

@@ -0,0 +1,11 @@
+---
+layout: default
+title: Snapshot migration
+nav_order: 5
+---
+
+# Snapshot migration
+
+One popular approach is to take a [snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/) of your Elasticsearch OSS 6.x or 7.x indices, [create an OpenSearch cluster]({{site.url}}{{site.baseurl}}/opensearch/install/), restore the snapshot on the new cluster, and point your clients to the new host.
+
+The snapshot approach can mean running two clusters in parallel, but lets you validate that the OpenSearch cluster is working in a way that meets your needs prior to modifying the Elasticsearch OSS cluster.

_migrate/upgrade-migrate.md

@@ -0,0 +1,172 @@
+---
+layout: default
+title: Upgrade migration
+nav_order: 15
+---
+
+# Upgrade to OpenSearch (Linux)
+
+If you want to migrate an existing Elasticsearch OSS cluster to OpenSearch and find the [snapshot approach]({{site.url}}{{site.baseurl}}/migrate/snapshot-migrate/) unappealing, you can upgrade the cluster instead. The first step is to upgrade your Elasticsearch OSS cluster to version 6.x or 7.x.
+
+Elasticsearch OSS supports two types of upgrades: rolling and cluster restart.
+
+- Rolling upgrades let you shut down one node at a time for minimal disruption of service.
+
+  Rolling upgrades work between minor versions (e.g. 6.5 to 6.8) and also support a single path to the next major version (e.g. 6.8 to 7.10.2). Performing these upgrades might require intermediate upgrades to arrive at your desired version and can affect cluster performance as nodes leave and rejoin, but the cluster remains available throughout the process.
+
+- Cluster restart upgrades require you to shut down all nodes, perform the upgrade, and restart the cluster.
+
+  Cluster restart upgrades work between minor versions (e.g. 6.5 to 6.8) and the next major version (for example, 6.x to 7.10.2). Cluster restart upgrades are faster to perform and require fewer intermediate upgrades, but require downtime.
+
+
+## Upgrade paths
+
+Elasticsearch OSS version | Rolling upgrade path | Cluster restart upgrade path
+:--- | :--- | :---
+5.x | Upgrade to 5.6, upgrade to 6.8, reindex all 5.x indices, upgrade to 7.10.2, and upgrade to OpenSearch. | Upgrade to 6.8, reindex all 5.x indices, and upgrade to OpenSearch.
+6.x | Upgrade to 6.8, upgrade to 7.10.2, and upgrade to OpenSearch. | Upgrade to OpenSearch.
+7.x | Upgrade to OpenSearch. | Upgrade to OpenSearch.
+
+
+## Upgrade Elasticsearch OSS
+
+1. Disable shard allocation to prevent Elasticsearch OSS from replicating shards as you shut down nodes:
+
+   ```json
+   PUT _cluster/settings
+   {
+     "persistent": {
+       "cluster.routing.allocation.enable": "primaries"
+     }
+   }
+   ```
+
+1. Stop Elasticsearch OSS on one node (rolling upgrade) or all nodes (cluster restart upgrade).
+
+   On Linux distributions that use systemd, use this command:
+
+   ```bash
+   sudo systemctl stop elasticsearch.service
+   ```
+
+   For tarball installations, find the process ID (`ps aux`) and kill it (`kill <pid>`).
+
+1. Upgrade the node (rolling) or all nodes (cluster restart).
+
+   The exact command varies by package manager, but likely looks something like this:
+
+   ```bash
+   sudo yum install elasticsearch-oss-7.10.2 --enablerepo=elasticsearch
+   ```
+
+   For tarball installations, extract to a new directory to ensure you **do not overwrite** your `config`, `data`, and `logs` directories. Ideally, these directories should have their own, independent paths and *not* be colocated with the Elasticsearch application directory. Then set the `ES_PATH_CONF` environment variable to the directory that contains `elasticsearch.yml` (e.g. `/etc/elasticesarch/`). In `elasticsearch.yml`, set `path.data` and `path.logs` to your `data` and `logs` directories (e.g. `/var/lib/elasticsearch` and `/var/log/opensearch`).
+
+1. Restart Elasticsearch OSS on the node (rolling) or all nodes (cluster restart).
+
+   On Linux distributions that use systemd, use this command:
+
+   ```bash
+   sudo systemctl start elasticsearch.service
+   ```
+
+   For tarball installations, run `./bin/elasticsearch -d`.
+
+1. Wait for the node to rejoin the cluster (rolling) or for the cluster to start (cluster restart). Check the `_nodes` summary to verify that all nodes are available and running the expected version:
+
+   ```bash
+   # Elasticsearch OSS
+   curl -XGET 'localhost:9200/_nodes/_all?pretty=true'
+   # Open Distro for Elasticsearch with security plugin enabled
+   curl -XGET 'https://localhost:9200/_nodes/_all?pretty=true' -u 'admin:admin' -k
+   ```
+
+   Specifically, check the `nodes.<node-id>.version` portion of the response. Also check `_cat/indices?v` for a green status on all indices.
+
+1. (Rolling) Repeat steps 2--5 until all nodes are using the new version.
+
+1. After all nodes are using the new version, re-enable shard allocation:
+
+   ```json
+   PUT _cluster/settings
+   {
+     "persistent": {
+       "cluster.routing.allocation.enable": "all"
+     }
+   }
+   ```
+
+1. If you upgraded from 5.x to 6.x, [reindex]({{site.url}}{{site.baseurl}}/opensearch/reindex-data/) all indices.
+
+1. Repeat all steps as necessary until you arrive at your desired Elasticsearch OSS version.
+
+
+## Upgrade to OpenSearch
+
+1. Disable shard allocation to prevent Elasticsearch OSS from replicating shards as you shut down nodes:
+
+   ```json
+   PUT _cluster/settings
+   {
+     "persistent": {
+       "cluster.routing.allocation.enable": "primaries"
+     }
+   }
+   ```
+
+1. Stop Elasticsearch OSS on one node (rolling upgrade) or all nodes (cluster restart upgrade).
+
+   On Linux distributions that use systemd, use this command:
+
+   ```bash
+   sudo systemctl stop elasticsearch.service
+   ```
+
+   For tarball installations, find the process ID and kill it.
+
+1. Upgrade the node (rolling) or all nodes (cluster restart).
+
+   1. Extract the OpenSearch tarball to a new directory to ensure you **do not overwrite** your Elasticsearch OSS `config`, `data`, and `logs` directories.
+
+   1. (Optional) Copy or move your Elasticsearch OSS `data` and `logs` directories to new paths. For example, you might move `/var/lib/elasticsearch` to `/var/lib/opensearch`.
+
+   1. Set the `OPENSEARCH_PATH_CONF` environment variable to the directory that contains `opensearch.yml` (e.g. `/etc/opensearch`).
+
+   1. In `opensearch.yml`, set `path.data` and `path.logs`. You might also want to disable the security plugin for now. `opensearch.yml` might look something like this:
+
+      ```yml
+      path.data: /var/lib/opensearch
+      path.logs: /var/log/opensearch
+      plugins.security.disabled: true
+      ```
+
+   1. Port your settings from `elasticsearch.yml` to `opensearch.yml`. Most settings use the same names. At a minimum, specify `cluster.name`, `node.name`, `discovery.seed_hosts`, and `cluster.initial_master_nodes`.
+
+   1. (Optional) Add your certificates to your `config` directory, add them to `opensearch.yml`, and initialize the security plugin.
+
+1. Start OpenSearch on the node (rolling) or all nodes (cluster restart).
+
+   For the tarball, run `./bin/opensearch -d`.
+
+1. Wait for the OpenSearch node to rejoin the cluster (rolling) or for the cluster to start (cluster restart). Check the `_nodes` summary to verify that all nodes are available and running the expected version:
+
+   ```bash
+   # Security plugin disabled
+   curl -XGET 'localhost:9200/_nodes/_all?pretty=true'
+   # Security plugin enabled
+   curl -XGET -k -u 'admin:admin' 'https://localhost:9200/_nodes/_all?pretty=true'
+   ```
+
+   Specifically, check the `nodes.<node-id>.version` portion of the response. Also check `_cat/indices?v` for a green status on all indices.
+
+1. (Rolling) Repeat steps 2--5 until all nodes are using OpenSearch.
+
+1. After all nodes are using the new version, re-enable shard allocation:
+
+   ```json
+   PUT _cluster/settings
+   {
+     "persistent": {
+       "cluster.routing.allocation.enable": "all"
+     }
+   }
+   ```