honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
190 Commits 7 Branches 6 Tags 546 MiB
Commit Graph

190 Commits

Author SHA1 Message Date
David Pilato 6f1efd24e4 Create `es-1.5` branch 2015-03-16 12:42:08 -07:00
paul-e-cooley cb7beec5b2 Separate proxy traffic for ec2 and s3 
Based on PR #178 by @paul-e-cooley. Thanks Paul!

In addition to:

```yaml
cloud:
    aws:
        protocol: https
        proxy_host: proxy1.company.com
        proxy_port: 8083
```

You can also set different proxies for `ec2` and `s3`:

```yaml
cloud:
    aws:
        s3:
            proxy_host: proxy1.company.com
            proxy_port: 8083
        ec2:
            proxy_host: proxy2.company.com
            proxy_port: 8083
```

PR rebased on master and lastest changes about component settings removal.
Documentation added.
Changes in tests. If a proxy is provided we run the tests, otherwise we ignore the test.

Closes #177.
 2015-03-09 14:19:11 -07:00
David Pilato 7a7de12538 Remove component settings from AbstractComponent 
Related to elasticsearch/elasticsearch#9919

Closes #182.
 2015-02-27 16:32:50 +01:00
Igor Motov 20a99919bf Update settings filter 
Update settings filter to match elasticsearch/elasticsearch#9748
 2015-02-24 13:24:02 -05:00
Jerry 3a4e472752 [doc] secret_key: <access key> should be <secret key> 
Closes #168.
 2015-02-19 21:34:03 +01:00
David Pilato de2ab73c8a Simplify region setting code 2015-02-19 21:26:45 +01:00
Marcin Matlaszek 68fea581a2 Fix region settings for s3 snapshot repository in eu-west-1 
Closes #169.
 2015-02-19 21:26:38 +01:00
David Pilato 4c02a017b4 Wrong endpoint for `cn-north-1` 
Commit ea9609c642 does not work as expected.

China is not part of the [S3 regions](http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) but part of [Isolated Regions](http://docs.aws.amazon.com/general/latest/gr/isolated_regions.html).

The patch uses for S3 `s3-cn-north-1.amazonaws.com` which is wrong because in that case the end point is `s3.cn-north-1.amazonaws.com.cn`.
Same goes for EC2 which was `ec2.cn-north-1.amazonaws.com` but should be `ec2.cn-north-1.amazonaws.com.cn`.

Related documentation:

* http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
* http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region
* http://docs.aws.amazon.com/general/latest/gr/isolated_regions.html

Closing #156
(cherry picked from commit 491a59b)
(cherry picked from commit f12cacc)
 2015-02-19 16:50:48 +01:00
Andreas Kohn c7187f2db5 Remove an excess quote character in an exception message 2015-02-19 16:15:14 +01:00
Serhiy Suprun e05036f257 Fixed address resolution 2015-02-18 16:50:58 +02:00
David Pilato e6080e9017 Move AwsEc2UnicastHostsProvider to Ec2DiscoveryModule 
Related to https://github.com/elasticsearch/elasticsearch/pull/9099
 2015-01-30 11:14:40 +01:00
David Pilato 377e7e0c78 Fix typo (azure -> s3) 2015-01-02 23:09:36 +01:00
David Pilato ea9609c642 Add China region for s3 and ec2 
China region name is `cn-north-1`.

Closes #156.
 2015-01-02 22:28:50 +01:00
David Pilato a76e93502f Implement BlobContainer.move() 
In elasticsearch master (2.0) a new method `move(String sourceBlobName, String targetBlobName)` has been added to `BlobContainer`.

Closes #152.
 2015-01-02 22:16:59 +01:00
David Pilato d36e24a626 Add sonatype snapshot repository 2015-01-02 21:10:01 +01:00
David Pilato 2df835008e Depend on elasticsearch-parent 
To simplify plugins maintenance and provide more value in the future, we are starting to build an `elasticsearch-parent` project.
This commit is the first step for this plugin to depend on this new `pom` maven project.
 2014-12-17 10:18:59 +01:00
Adrien Grand 50e5149485 Upgrade to Lucene 5.0.0-snapshot-1642891 2014-12-02 18:19:35 +01:00
tlrx 849e697385 Update AWS SDK to 1.9.8 
Closes #148
 2014-11-26 16:47:15 +01:00
tlrx 77e9d7f16f Merge branch 'split-aws-java-sdk' of https://github.com/Collaborne/elasticsearch-cloud-aws into Collaborne-split-aws-java-sdk 2014-11-26 16:44:44 +01:00
tlrx d0c712299b Merge #137 
Closes #136
 2014-11-26 16:38:37 +01:00
tlrx f4057f2a83 Remove unnecessary settings in test method 2014-11-26 12:48:45 +01:00
tlrx d680b94eb7 Fix unit tests and some bugs introduced by #128 
Closes #119, #147
 2014-11-26 11:04:29 +01:00
tlrx 61a307f28d Merge branch 'brutasse-feature/per-repo-endpoint' 2014-11-26 10:58:01 +01:00
tlrx 20576820a6 Merge branch 'feature/per-repo-endpoint' of https://github.com/brutasse/elasticsearch-cloud-aws into brutasse-feature/per-repo-endpoint 2014-11-26 10:57:20 +01:00
David Pilato fca3addbaf Fix latest plugin version 2014-11-24 18:16:35 +01:00
David Pilato 9f359bcd62 update documentation with release 2.4.1 2014-11-24 17:30:33 +01:00
David Pilato 66cedb350e AwsEc2UnicastHostsProvider should use version.minimumCompatibilityVersion() 
The AwsEc2UnicastHostsProvider creates DiscoveryNodes that are used as an initial seed for unicast based discovery. At the moment it uses Version.CURRENT (see [1]) for those DiscoveryNode object, which confuses the backwards compatibility layer to think this nodes are of the latest version. This causes new nodes to fail to join old nodes as the ping serialization goes wrong. Instead we should use version.minimumCompatibilityVersion(). See [2]

[1] https://github.com/elasticsearch/elasticsearch-cloud-aws/blob/es-1.x/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2UnicastHostsProvider.java#L165
[2] https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/discovery/zen/ping/unicast/UnicastZenPing.java#L130

Ps. this was reported on the mailing list. See: https://groups.google.com/forum/#!msg/elasticsearch/8pUwFld88tI/7jRuG6hqtbAJ

Closes #143.
 2014-11-24 16:06:11 +01:00
Michael McCandless 3f8a053da8 Upgrade to Lucene 5.0.0-snapshot-1641343 2014-11-24 05:52:10 -05:00
Bruno Renié 30d80cc27d Allow per-repo protocols 
Also add an integration test for custom endpoints/protocols.
 2014-11-21 18:05:48 +01:00
Bruno Renié 2203f439e2 Merge branch 'master' into feature/per-repo-endpoint 
Conflicts:
	src/main/java/org/elasticsearch/cloud/aws/AwsS3Service.java
	src/main/java/org/elasticsearch/cloud/aws/InternalAwsS3Service.java
	src/main/java/org/elasticsearch/repositories/s3/S3Repository.java
 2014-11-21 11:00:01 +01:00
Andreas Kohn c8839ee68c Use the new 'install' form when invoking bin/plugin 
'-install' was deprecated in https://github.com/elasticsearch/elasticsearch/issues/3112,
and 'install' is the proper variant from https://github.com/elasticsearch/elasticsearch/pull/7339.
 2014-11-20 17:25:46 +01:00
tlrx e757722291 Add random read failures in unit tests to test the retry logic added to S3BlobContainer.openInput() 2014-11-20 17:10:59 +01:00
tlrx ea91adf6a1 Add retry logic for S3 connection errors when restoring snapshots 
This commit adds a retry logic when reading blobs from S3. It also adds a retry logic when initializing a multipart upload and sets the internal "max retries" parameter of the Amazon S3 client with the same value as the "max_retry" parameter set for the snapshot repository (so in worst cases with the default value set to 3, 3x3=9 attempts will be made). The internal S3 client uses an exponential back off strategy between each connection exception (mainly IOException).

Closes elasticsearch/elasticsearch#8280
 2014-11-20 17:10:59 +01:00
Andreas Kohn dce47eb481 Use only the S3 and EC2 parts of the AWS Java SDK 
Note that jackson is not marked as optional in the SDK POM, and the SDK uses Jackson 2 now.
 2014-11-12 01:05:46 +01:00
Michael McCandless 65bda62388 Upgrade to Lucene 5.0.0-snapshot-1637347 2014-11-10 16:46:24 -05:00
tlrx 4be25988d1 Update S3BlobContainer because BlobContainer changed 
See elasticsearch/elasticsearch#8366
 2014-11-07 09:52:38 +01:00
Robert Muir 3cfb2df686 upgrade to lucene 5 snapshot 2014-11-05 17:49:08 -05:00
tlrx 338a94dbe6 Fix constructors of ZenDiscovery's sub classes 
Closes #133
 2014-11-04 10:02:52 +01:00
Jun Ohtani 4658f0e2df Tests: Fix randomizedtest fail 
Closes #131
 2014-10-31 14:32:33 +09:00
Jun Ohtani c35093b2a4 Update to Lucene 4.10.2 
Closes #130
 2014-10-31 14:17:45 +09:00
Bruno Renié 06629409a3 Allow per-repository endpoints 2014-10-28 11:43:48 +01:00
tlrx 37a920152c update documentation with release 2.4.0 2014-10-23 11:41:38 +02:00
tlrx bfaa608f3a [DOCS] Add documentation about proxy settings 
Closes #110
 2014-10-20 10:10:00 +02:00
Ayumi Yu a373ae1156 Update README IAM policy fix version number 2014-10-20 09:34:38 +02:00
tlrx 4bc4ea6fca BlobStore: BlobContainer interface changed in elasticsearch 1.4.0 
Adding a S3OutputStream that upload blobs to the S3 Storage service with two modes (single/multipart).
When the length of the chunk is lower than buffer_size (default to 5mb), the chunk is uploaded with a single request.
Otherwise multiple requests are made, each of buffer_size (except the last one which can be lower than buffer_size).
For example, when uploading a blob (say, 1Gb) with chunk_size set for accepting large chunks (chunk_size = 5Gb) and buffer_size set to 100Mb, the blob will be sent into 10 multiple parts, each of ~100Mb. Each part upload may failed independently and will be retried 3 times.

Closes #117
 2014-10-16 10:24:13 +02:00
tlrx 039a7937ee Update to Lucene 4.10.1 2014-10-09 16:28:51 +02:00
tlrx 57ce2cc083 Create branch es-1.4 for elasticsearch 1.4.0 2014-10-09 16:12:55 +02:00
David Pilato d8a6f21e71 ZenDiscovery constructor needs ElectMasterService instance 
Introduced in https://github.com/elasticsearch/elasticsearch/pull/7336 (elasticsearch 1.4 and 2.0), we need to change Ec2Discovery constructor.

Closes #115.
 2014-09-05 16:23:38 +02:00
Jon Dokulil 13f4be5491 [Docs] Include EC2 IAM policy example 
Closes #112.
Closes #113.
 2014-09-04 06:41:57 +02:00
David Pilato bcc4f6a67a Update to elasticsearch 1.4.0 
Closes #114.

(cherry picked from commit 6c83341)
 2014-09-04 06:40:27 +02:00