Commit Graph

86 Commits

Author SHA1 Message Date
David Pilato 39776944d5 Add EC2/S3 Signer API setting
If you are using a compatible EC2 or S3 service, they might be using an older API to sign the requests.

You can set your compatible signer API using `cloud.aws.signer` (or `cloud.aws.ec2.signer` and `cloud.aws.s3.signer`) with the right signer to use.

Defaults to `AWS4SignerType`.

Supported today (time when this commit is done):

* `QueryStringSignerType`
* `AWS3SignerType`
* `AWS4SignerType`
* `NoOpSignerType`

Closes #155.
(cherry picked from commit 33b18b4)
(cherry picked from commit 9809af5)
2015-04-24 12:47:11 +02:00
David Pilato a723875148 update documentation with release 2.5.0 2015-03-25 10:35:48 +01:00
David Pilato 6f1efd24e4 Create `es-1.5` branch 2015-03-16 12:42:08 -07:00
paul-e-cooley cb7beec5b2 Separate proxy traffic for ec2 and s3
Based on PR #178 by @paul-e-cooley. Thanks Paul!

In addition to:

```yaml
cloud:
    aws:
        protocol: https
        proxy_host: proxy1.company.com
        proxy_port: 8083
```

You can also set different proxies for `ec2` and `s3`:

```yaml
cloud:
    aws:
        s3:
            proxy_host: proxy1.company.com
            proxy_port: 8083
        ec2:
            proxy_host: proxy2.company.com
            proxy_port: 8083
```

PR rebased on master and lastest changes about component settings removal.
Documentation added.
Changes in tests. If a proxy is provided we run the tests, otherwise we ignore the test.

Closes #177.
2015-03-09 14:19:11 -07:00
Jerry 3a4e472752 [doc] secret_key: <access key> should be <secret key>
Closes #168.
2015-02-19 21:34:03 +01:00
David Pilato ea9609c642 Add China region for s3 and ec2
China region name is `cn-north-1`.

Closes #156.
2015-01-02 22:28:50 +01:00
tlrx d0c712299b Merge #137
Closes #136
2014-11-26 16:38:37 +01:00
tlrx d680b94eb7 Fix unit tests and some bugs introduced by #128
Closes #119, #147
2014-11-26 11:04:29 +01:00
tlrx 20576820a6 Merge branch 'feature/per-repo-endpoint' of https://github.com/brutasse/elasticsearch-cloud-aws into brutasse-feature/per-repo-endpoint 2014-11-26 10:57:20 +01:00
David Pilato fca3addbaf Fix latest plugin version 2014-11-24 18:16:35 +01:00
David Pilato 9f359bcd62 update documentation with release 2.4.1 2014-11-24 17:30:33 +01:00
Bruno Renié 30d80cc27d Allow per-repo protocols
Also add an integration test for custom endpoints/protocols.
2014-11-21 18:05:48 +01:00
Bruno Renié 2203f439e2 Merge branch 'master' into feature/per-repo-endpoint
Conflicts:
	src/main/java/org/elasticsearch/cloud/aws/AwsS3Service.java
	src/main/java/org/elasticsearch/cloud/aws/InternalAwsS3Service.java
	src/main/java/org/elasticsearch/repositories/s3/S3Repository.java
2014-11-21 11:00:01 +01:00
Andreas Kohn c8839ee68c Use the new 'install' form when invoking bin/plugin
'-install' was deprecated in https://github.com/elasticsearch/elasticsearch/issues/3112,
and 'install' is the proper variant from https://github.com/elasticsearch/elasticsearch/pull/7339.
2014-11-20 17:25:46 +01:00
Bruno Renié 06629409a3 Allow per-repository endpoints 2014-10-28 11:43:48 +01:00
tlrx 37a920152c update documentation with release 2.4.0 2014-10-23 11:41:38 +02:00
tlrx bfaa608f3a [DOCS] Add documentation about proxy settings
Closes #110
2014-10-20 10:10:00 +02:00
Ayumi Yu a373ae1156 Update README IAM policy fix version number 2014-10-20 09:34:38 +02:00
tlrx 4bc4ea6fca BlobStore: BlobContainer interface changed in elasticsearch 1.4.0
Adding a S3OutputStream that upload blobs to the S3 Storage service with two modes (single/multipart).
When the length of the chunk is lower than buffer_size (default to 5mb), the chunk is uploaded with a single request.
Otherwise multiple requests are made, each of buffer_size (except the last one which can be lower than buffer_size).
For example, when uploading a blob (say, 1Gb) with chunk_size set for accepting large chunks (chunk_size = 5Gb) and buffer_size set to 100Mb, the blob will be sent into 10 multiple parts, each of ~100Mb. Each part upload may failed independently and will be retried 3 times.

Closes #117
2014-10-16 10:24:13 +02:00
tlrx 57ce2cc083 Create branch es-1.4 for elasticsearch 1.4.0 2014-10-09 16:12:55 +02:00
Jon Dokulil 13f4be5491 [Docs] Include EC2 IAM policy example
Closes #112.
Closes #113.
2014-09-04 06:41:57 +02:00
David Pilato bcc4f6a67a Update to elasticsearch 1.4.0
Closes #114.

(cherry picked from commit 6c83341)
2014-09-04 06:40:27 +02:00
David Pilato aac156cef9 update documentation with release 2.3.0 2014-08-05 13:56:02 +02:00
David Pilato 610d9a70a1 Switch to https communication for Amazon APIs by default
We should use `https` by default instead of `http` for communication between elasticsearch and AWS API.

Note that it can be modified in case of trouble and fallback to the older setting using `cloud.aws.protocol: http`

Closes #109.
2014-08-05 12:28:46 +02:00
bitsofinfo 0474a1bfea Allow https communication per ec2 or s3 service
By default all communication w/ AWS services done by this plugin is sent the clear over `http`, overriding amazons own default of https: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/ClientConfiguration.html#getProtocol()

One has to set `cloud.aws.protocol` in `elasticsearch.yml` to force SSL.

    cloud.aws.protocol: https

This is not entirely clear to the average user, and should be added to the documentation on both this project's README.

Closes #101.
2014-08-05 12:21:39 +02:00
David Pilato d9638e6a94 Simplify documentation
Closes #106
2014-08-05 11:35:24 +02:00
David Pilato 776677f7e2 Tests: update to Lucene 4.9.0
Closes #107.

(cherry picked from commit c3c9a44)
2014-07-30 13:41:31 +02:00
Igor Motov 11f4e9c063 Added retry mechanism for S3 connection errors
Closes #95
2014-06-21 16:35:31 +02:00
David Pilato ba185e026b Allow AWS compatible cloud services
Users can use their own endpoints for any ec2/s3 compatible API using: `cloud.aws.ec2.endpoint` or `cloud.aws.s3.endpoint`

Closes #91.
(cherry picked from commit f0fbea5)
2014-06-20 15:32:53 +02:00
Peter Burkholder a0fa97d9c9 Update README about SNAPSHOT vs released versions
Closes #77.
2014-06-19 19:35:59 +02:00
Phil Wills 7af13165bf Elaborate on how to specify AWS credentials
Closes #85.
2014-06-19 19:27:28 +02:00
Igor Motov f470e34143 Cleaning documentation
(cherry picked from commit 67c5e39)
2014-05-28 23:04:57 +02:00
Wilkes Joiner c743f06702 Add Server Side Encryption support for S3 repository
For legal reasons, we need to store our ES backups in S3 using server side encryption.

Closes #78.

(cherry picked from commit b3f9e12)
2014-05-28 22:49:04 +02:00
Igor Motov 6fd4945860 Update to elasticsearch 1.3.0
Closes #89.
(cherry picked from commit d65a970)
2014-05-28 21:42:57 +02:00
Igor Motov d322b10488 Switch to shared thread pool for snapshot operations
Closes #87

(cherry picked from commit 36d4da6)
2014-05-28 21:30:54 +02:00
David Pilato d59756583f S3 Repo documentation
Closes #83.

(cherry picked from commit 67ac149)
2014-05-27 16:44:14 +02:00
Britta Weber 77143f06d9 Add comment about snapshot/restore if bucket does not exist
closes #84
2014-05-13 18:02:30 +02:00
Kurt Hurtado 34e6d73293 Adding example S3 bucket permissions, with js syntax 2014-05-07 10:15:46 -07:00
David Pilato 1d7ca8fa9e create branch es-1.2 2014-04-29 21:16:25 +02:00
David Pilato d9326af8b9 Remove S3 gateway
In elasticsearch core code, we removed Gateways (see https://github.com/elasticsearch/elasticsearch/issues/5422).

We need now to remove S3 gateway from aws plugin.

Closes #75.
2014-04-18 11:15:54 +02:00
Igor Motov 68a8e78b59 [DOCS] Change bucket names to use "-" instead of "_" in examples
While it's possible to create a bucket with "_" in US Standard region, AWS Java SDK cannot work with such buckets. Changing all examples to use "-" instead of "_" in bucket names.

Closes #72
2014-04-15 19:30:30 -04:00
Konrad Beiske 7f271fd37a Add per repository credentials
Changed AwsS3Service to use one client per region and credentials combination.
Made S3Repository specify credentials if such exists in the repository settings.

Updated readme with repository specific credentials settings.

Closes #54.
Closes #55.
Closes #56.
(cherry picked from commit d4ea2dd)
2014-03-26 18:52:48 +01:00
David Pilato 254fb81708 Create branches according to elasticsearch versions
We create branches:

* es-0.90 for elasticsearch 0.90
* es-1.0 for elasticsearch 1.0
* es-1.1 for elasticsearch 1.1
* master for elasticsearch master

We also check that before releasing we don't have a dependency to an elasticsearch SNAPSHOT version.

Add links to each version in documentation
(cherry picked from commit e0f06c6)
2014-03-26 18:35:21 +01:00
David Pilato 68326676da Replace gateway by S3 repository 2014-03-19 20:58:45 +01:00
Kenta Suzuki c117c119eb Add plugin release semi-automatic script
Closes #58.
2014-03-17 12:59:18 +01:00
Kenta Suzuki 4c7e384c96 Support ap-northeast region for EC2 and S3
Closes #52.
2014-03-17 11:50:28 +01:00
David Pilato bbb064468c Support group id in addition to group names
Based on issue #43, we want to support both notations:

* group name first
* group id if it does not work with group name

Closes #49.
2014-03-17 09:53:56 +01:00
David Pilato 8f87ed8620 Fix date for 2.0.0.RC1 2014-02-06 09:08:58 +01:00
Chris Zelenak 0fef547659 Add note regarding groups value
Instances are filtered on group name, not group id
2014-01-27 15:42:56 +01:00
Kendall Gifford 87731b913b Correct version number in README.md
Correct the version number when automatic instance-level security support became available.
2014-01-27 15:17:22 +01:00