OpenSearch/docs/plugins/repository-azure.asciidoc

[[repository-azure]]
=== Azure Repository Plugin

The Azure Repository plugin adds support for using Azure as a repository for
{ref}/modules-snapshots.html[Snapshot/Restore].

:plugin_name: repository-azure
include::install_remove.asciidoc[]

[[repository-azure-usage]]
==== Azure Repository

To enable Azure repositories, you have first to define your azure storage settings as
{ref}/secure-settings.html[secure settings], before starting up the node:

[source,sh]
----------------------------------------------------------------
bin/elasticsearch-keystore add azure.client.default.account
bin/elasticsearch-keystore add azure.client.default.key
----------------------------------------------------------------

Where `account` is the azure account name and `key` the azure secret key. Instead of an azure secret key under `key`, you can alternatively
define a shared access signatures (SAS) token under `sas_token` to use for authentication instead. When using an SAS token instead of an
account key, the SAS token must have read (r), write (w), list (l), and delete (d) permissions for the repository base path and
all its contents. These permissions need to be granted for the blob service (b) and apply to resource types service (s), container (c), and
object (o).
These settings are used by the repository's internal azure client.

Note that you can also define more than one account:

[source,sh]
----------------------------------------------------------------
bin/elasticsearch-keystore add azure.client.default.account
bin/elasticsearch-keystore add azure.client.default.key
bin/elasticsearch-keystore add azure.client.secondary.account
bin/elasticsearch-keystore add azure.client.secondary.sas_token
----------------------------------------------------------------

`default` is the default account name which will be used by a repository,
unless you set an explicit one in the
<<repository-azure-repository-settings, repository settings>>.

The `account`, `key`, and `sas_token` storage settings are
{ref}/secure-settings.html#reloadable-secure-settings[reloadable]. After you
reload the settings, the internal azure clients, which are used to transfer the
snapshot, will utilize the latest settings from the keystore.

NOTE: In progress snapshot/restore jobs will not be preempted by a *reload*
of the storage secure settings. They will complete using the client as it was built
when the operation started.

You can set the client side timeout to use when making any single request. It can be defined globally, per account or both.
It's not set by default which means that Elasticsearch is using the
http://azure.github.io/azure-storage-java/com/microsoft/azure/storage/RequestOptions.html#setTimeoutIntervalInMs(java.lang.Integer)[default value]
set by the azure client (known as 5 minutes).

`max_retries` can help to control the exponential backoff policy. It will fix the number of retries
in case of failures before considering the snapshot is failing. Defaults to `3` retries.
The initial backoff period is defined by Azure SDK as `30s`. Which means `30s` of wait time
before retrying after a first timeout or failure. The maximum backoff period is defined by Azure SDK as
`90s`.

`endpoint_suffix` can be used to specify Azure endpoint suffix explicitly. Defaults to `core.windows.net`.

[source,yaml]
----
azure.client.default.timeout: 10s
azure.client.default.max_retries: 7
azure.client.default.endpoint_suffix: core.chinacloudapi.cn
azure.client.secondary.timeout: 30s
----

In this example, timeout will be `10s` per try for `default` with `7` retries before failing
and endpoint suffix will be `core.chinacloudapi.cn` and `30s` per try for `secondary` with `3` retries.

[IMPORTANT]
.Supported Azure Storage Account types
===============================================
The Azure Repository plugin works with all Standard storage accounts

* Standard Locally Redundant Storage - `Standard_LRS`
* Standard Zone-Redundant Storage - `Standard_ZRS`
* Standard Geo-Redundant Storage - `Standard_GRS`
* Standard Read Access Geo-Redundant Storage - `Standard_RAGRS`

https://azure.microsoft.com/en-gb/documentation/articles/storage-premium-storage[Premium Locally Redundant Storage] (`Premium_LRS`) is **not supported** as it is only usable as VM disk storage, not as general storage.
===============================================

You can register a proxy per client using the following settings:

[source,yaml]
----
azure.client.default.proxy.host: proxy.host
azure.client.default.proxy.port: 8888
azure.client.default.proxy.type: http
----

Supported values for `proxy.type` are `direct` (default), `http` or `socks`.
When `proxy.type` is set to `http` or `socks`, `proxy.host` and `proxy.port` must be provided.


[[repository-azure-repository-settings]]
==== Repository settings

The Azure repository supports following settings:

`client`::

    Azure named client to use. Defaults to `default`.

`container`::

    Container name. You must create the azure container before creating the repository.
    Defaults to `elasticsearch-snapshots`.

`base_path`::

    Specifies the path within container to repository data. Defaults to empty
    (root directory).

`chunk_size`::

    Big files can be broken down into chunks during snapshotting if needed.
    The chunk size can be specified in bytes or by using size value notation,
    i.e. `1g`, `10m`, `5k`. Defaults to `64m` (64m max)

`compress`::

    When set to `true` metadata files are stored in compressed format. This
    setting doesn't affect index files that are already compressed by default.
    Defaults to `false`.

include::repository-shared-settings.asciidoc[]

`location_mode`::

    `primary_only` or `secondary_only`. Defaults to `primary_only`. Note that if you set it
    to `secondary_only`, it will force `readonly` to true.

Some examples, using scripts:

[source,js]
----
# The simplest one
PUT _snapshot/my_backup1
{
    "type": "azure"
}

# With some settings
PUT _snapshot/my_backup2
{
    "type": "azure",
    "settings": {
        "container": "backup-container",
        "base_path": "backups",
        "chunk_size": "32m",
        "compress": true
    }
}


# With two accounts defined in elasticsearch.yml (my_account1 and my_account2)
PUT _snapshot/my_backup3
{
    "type": "azure",
    "settings": {
        "client": "secondary"
    }
}
PUT _snapshot/my_backup4
{
    "type": "azure",
    "settings": {
        "client": "secondary",
        "location_mode": "primary_only"
    }
}
----
// CONSOLE
// TEST[skip:we don't have azure setup while testing this]

Example using Java:

[source,java]
----
client.admin().cluster().preparePutRepository("my_backup_java1")
    .setType("azure").setSettings(Settings.builder()
        .put(Storage.CONTAINER, "backup-container")
        .put(Storage.CHUNK_SIZE, new ByteSizeValue(32, ByteSizeUnit.MB))
    ).get();
----

[[repository-azure-validation]]
==== Repository validation rules

According to the http://msdn.microsoft.com/en-us/library/dd135715.aspx[containers naming guide], a container name must
be a valid DNS name, conforming to the following naming rules:

* Container names must start with a letter or number, and can contain only letters, numbers, and the dash (-) character.
* Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not
permitted in container names.
* All letters in a container name must be lowercase.
* Container names must be from 3 through 63 characters long.