honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
222 Commits 7 Branches 6 Tags 546 MiB
Commit Graph

222 Commits

Author SHA1 Message Date
tlrx 77e9d7f16f Merge branch 'split-aws-java-sdk' of https://github.com/Collaborne/elasticsearch-cloud-aws into Collaborne-split-aws-java-sdk 2014-11-26 16:44:44 +01:00
tlrx d0c712299b Merge #137 
Closes #136
 2014-11-26 16:38:37 +01:00
tlrx f4057f2a83 Remove unnecessary settings in test method 2014-11-26 12:48:45 +01:00
tlrx d680b94eb7 Fix unit tests and some bugs introduced by #128 
Closes #119, #147
 2014-11-26 11:04:29 +01:00
tlrx 61a307f28d Merge branch 'brutasse-feature/per-repo-endpoint' 2014-11-26 10:58:01 +01:00
tlrx 20576820a6 Merge branch 'feature/per-repo-endpoint' of https://github.com/brutasse/elasticsearch-cloud-aws into brutasse-feature/per-repo-endpoint 2014-11-26 10:57:20 +01:00
David Pilato fca3addbaf Fix latest plugin version 2014-11-24 18:16:35 +01:00
David Pilato 9f359bcd62 update documentation with release 2.4.1 2014-11-24 17:30:33 +01:00
David Pilato 66cedb350e AwsEc2UnicastHostsProvider should use version.minimumCompatibilityVersion() 
The AwsEc2UnicastHostsProvider creates DiscoveryNodes that are used as an initial seed for unicast based discovery. At the moment it uses Version.CURRENT (see [1]) for those DiscoveryNode object, which confuses the backwards compatibility layer to think this nodes are of the latest version. This causes new nodes to fail to join old nodes as the ping serialization goes wrong. Instead we should use version.minimumCompatibilityVersion(). See [2]

[1] https://github.com/elasticsearch/elasticsearch-cloud-aws/blob/es-1.x/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2UnicastHostsProvider.java#L165
[2] https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/discovery/zen/ping/unicast/UnicastZenPing.java#L130

Ps. this was reported on the mailing list. See: https://groups.google.com/forum/#!msg/elasticsearch/8pUwFld88tI/7jRuG6hqtbAJ

Closes #143.
 2014-11-24 16:06:11 +01:00
Michael McCandless 3f8a053da8 Upgrade to Lucene 5.0.0-snapshot-1641343 2014-11-24 05:52:10 -05:00
Bruno Renié 30d80cc27d Allow per-repo protocols 
Also add an integration test for custom endpoints/protocols.
 2014-11-21 18:05:48 +01:00
Bruno Renié 2203f439e2 Merge branch 'master' into feature/per-repo-endpoint 
Conflicts:
	src/main/java/org/elasticsearch/cloud/aws/AwsS3Service.java
	src/main/java/org/elasticsearch/cloud/aws/InternalAwsS3Service.java
	src/main/java/org/elasticsearch/repositories/s3/S3Repository.java
 2014-11-21 11:00:01 +01:00
Andreas Kohn c8839ee68c Use the new 'install' form when invoking bin/plugin 
'-install' was deprecated in https://github.com/elasticsearch/elasticsearch/issues/3112,
and 'install' is the proper variant from https://github.com/elasticsearch/elasticsearch/pull/7339.
 2014-11-20 17:25:46 +01:00
tlrx e757722291 Add random read failures in unit tests to test the retry logic added to S3BlobContainer.openInput() 2014-11-20 17:10:59 +01:00
tlrx ea91adf6a1 Add retry logic for S3 connection errors when restoring snapshots 
This commit adds a retry logic when reading blobs from S3. It also adds a retry logic when initializing a multipart upload and sets the internal "max retries" parameter of the Amazon S3 client with the same value as the "max_retry" parameter set for the snapshot repository (so in worst cases with the default value set to 3, 3x3=9 attempts will be made). The internal S3 client uses an exponential back off strategy between each connection exception (mainly IOException).

Closes elasticsearch/elasticsearch#8280
 2014-11-20 17:10:59 +01:00
Andreas Kohn dce47eb481 Use only the S3 and EC2 parts of the AWS Java SDK 
Note that jackson is not marked as optional in the SDK POM, and the SDK uses Jackson 2 now.
 2014-11-12 01:05:46 +01:00
Michael McCandless 65bda62388 Upgrade to Lucene 5.0.0-snapshot-1637347 2014-11-10 16:46:24 -05:00
tlrx 4be25988d1 Update S3BlobContainer because BlobContainer changed 
See elasticsearch/elasticsearch#8366
 2014-11-07 09:52:38 +01:00
Robert Muir 3cfb2df686 upgrade to lucene 5 snapshot 2014-11-05 17:49:08 -05:00
tlrx 338a94dbe6 Fix constructors of ZenDiscovery's sub classes 
Closes #133
 2014-11-04 10:02:52 +01:00
Jun Ohtani 4658f0e2df Tests: Fix randomizedtest fail 
Closes #131
 2014-10-31 14:32:33 +09:00
Jun Ohtani c35093b2a4 Update to Lucene 4.10.2 
Closes #130
 2014-10-31 14:17:45 +09:00
Bruno Renié 06629409a3 Allow per-repository endpoints 2014-10-28 11:43:48 +01:00
tlrx 37a920152c update documentation with release 2.4.0 2014-10-23 11:41:38 +02:00
tlrx bfaa608f3a [DOCS] Add documentation about proxy settings 
Closes #110
 2014-10-20 10:10:00 +02:00
Ayumi Yu a373ae1156 Update README IAM policy fix version number 2014-10-20 09:34:38 +02:00
tlrx 4bc4ea6fca BlobStore: BlobContainer interface changed in elasticsearch 1.4.0 
Adding a S3OutputStream that upload blobs to the S3 Storage service with two modes (single/multipart).
When the length of the chunk is lower than buffer_size (default to 5mb), the chunk is uploaded with a single request.
Otherwise multiple requests are made, each of buffer_size (except the last one which can be lower than buffer_size).
For example, when uploading a blob (say, 1Gb) with chunk_size set for accepting large chunks (chunk_size = 5Gb) and buffer_size set to 100Mb, the blob will be sent into 10 multiple parts, each of ~100Mb. Each part upload may failed independently and will be retried 3 times.

Closes #117
 2014-10-16 10:24:13 +02:00
tlrx 039a7937ee Update to Lucene 4.10.1 2014-10-09 16:28:51 +02:00
tlrx 57ce2cc083 Create branch es-1.4 for elasticsearch 1.4.0 2014-10-09 16:12:55 +02:00
David Pilato d8a6f21e71 ZenDiscovery constructor needs ElectMasterService instance 
Introduced in https://github.com/elasticsearch/elasticsearch/pull/7336 (elasticsearch 1.4 and 2.0), we need to change Ec2Discovery constructor.

Closes #115.
 2014-09-05 16:23:38 +02:00
Jon Dokulil 13f4be5491 [Docs] Include EC2 IAM policy example 
Closes #112.
Closes #113.
 2014-09-04 06:41:57 +02:00
David Pilato bcc4f6a67a Update to elasticsearch 1.4.0 
Closes #114.

(cherry picked from commit 6c83341)
 2014-09-04 06:40:27 +02:00
David Pilato aac156cef9 update documentation with release 2.3.0 2014-08-05 13:56:02 +02:00
David Pilato f1f173e249 Use new release tool for elasticsearch 2014-08-05 13:40:22 +02:00
David Pilato 0a5eb7c7df Fix tests 
(cherry picked from commit 521d946)
 2014-08-05 13:40:22 +02:00
David Pilato b4e7cc062d Move log4j properties to xml and remove unused elasticsearch.yml file 
(cherry picked from commit b07fc7c)
 2014-08-05 13:01:43 +02:00
David Pilato 610d9a70a1 Switch to https communication for Amazon APIs by default 
We should use `https` by default instead of `http` for communication between elasticsearch and AWS API.

Note that it can be modified in case of trouble and fallback to the older setting using `cloud.aws.protocol: http`

Closes #109.
 2014-08-05 12:28:46 +02:00
bitsofinfo 0474a1bfea Allow https communication per ec2 or s3 service 
By default all communication w/ AWS services done by this plugin is sent the clear over `http`, overriding amazons own default of https: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/ClientConfiguration.html#getProtocol()

One has to set `cloud.aws.protocol` in `elasticsearch.yml` to force SSL.

    cloud.aws.protocol: https

This is not entirely clear to the average user, and should be added to the documentation on both this project's README.

Closes #101.
 2014-08-05 12:21:39 +02:00
David Pilato f3a3262edf Wrong exception thrown when snapshot doesn't exist 
When a Snapshot does not exist, we should raise a `SnapshotMissingException`.
Add also tests for GET/DELETE on non existing repo

Closes #86.

(cherry picked from commit 77ab672)
 2014-08-05 11:44:53 +02:00
David Pilato d9638e6a94 Simplify documentation 
Closes #106
 2014-08-05 11:35:24 +02:00
David Pilato 8a83143e17 Update maven plugins versions and change compile to 1.7 
(cherry picked from commit 4a3929e)
 2014-08-05 11:27:41 +02:00
David Pilato 776677f7e2 Tests: update to Lucene 4.9.0 
Closes #107.

(cherry picked from commit c3c9a44)
 2014-07-30 13:41:31 +02:00
Igor Motov 11f4e9c063 Added retry mechanism for S3 connection errors 
Closes #95
 2014-06-21 16:35:31 +02:00
David Pilato ba185e026b Allow AWS compatible cloud services 
Users can use their own endpoints for any ec2/s3 compatible API using: `cloud.aws.ec2.endpoint` or `cloud.aws.s3.endpoint`

Closes #91.
(cherry picked from commit f0fbea5)
 2014-06-20 15:32:53 +02:00
David Pilato 6a04ae1b28 Update to AWS SDK 1.7.13 
Closes #97.

(cherry picked from commit 4380b84)
 2014-06-20 15:23:12 +02:00
Peter Burkholder a0fa97d9c9 Update README about SNAPSHOT vs released versions 
Closes #77.
 2014-06-19 19:35:59 +02:00
Phil Wills 7af13165bf Elaborate on how to specify AWS credentials 
Closes #85.
 2014-06-19 19:27:28 +02:00
David Pilato f2571343fc Update to elasticsearch 1.3.0 
Related to #89.
(cherry picked from commit 11e1d8a)
 2014-06-19 19:23:51 +02:00
Igor Motov f470e34143 Cleaning documentation 
(cherry picked from commit 67c5e39)
 2014-05-28 23:04:57 +02:00
Wilkes Joiner c743f06702 Add Server Side Encryption support for S3 repository 
For legal reasons, we need to store our ES backups in S3 using server side encryption.

Closes #78.

(cherry picked from commit b3f9e12)
 2014-05-28 22:49:04 +02:00