Commit Graph

37 Commits

Author SHA1 Message Date
David Turner 4108722052
Add support for AWS session tokens (#30414)
AWS supports the creation and use of credentials that are only valid for a
fixed period of time. These credentials comprise three parts: the usual access
key and secret key, together with a session token. This commit adds support for
these three-part credentials to the EC2 discovery plugin and the S3 repository
plugin.

Note that session tokens are only valid for a limited period of time and yet
there is no mechanism for refreshing or rotating them when they expire without
restarting Elasticsearch.  Nonetheless, this feature is already useful for
nodes that need only run for a few days, such as for training, testing or
evaluation. #29135 tracks the work towards allowing these credentials to be
refreshed at runtime.

Resolves #16428
2018-07-03 14:12:07 +01:00
Christoph Büscher 0d11b9fe34
[Docs] Unify spelling of Elasticsearch (#27567)
Removes occurences of "elasticsearch" or "ElasticSearch" in favour of
"Elasticsearch" where appropriate.
2017-11-29 09:44:25 +01:00
Jay Greenberg df5c8bb3bf
Update discovery-ec2.asciidoc
Changed the recommendation to use Tribe Node to Cross Cluster Search.
2017-11-07 10:18:38 -05:00
Ryan Ernst a51faea79f Docs: Cleanup docs for ec2 discovery (#26065)
This commit clears up which settings are allowed for ec2 discovery, and
clearly marks those that require setting in the keystore.

closes #25619
2017-08-15 10:14:51 -07:00
Ryan Ernst e23919856e Docs: Remove docs for aws region and signer type (#26006)
These settings are removed in 6.0. This commit removes the corresponding
documentation.

relates #22872
relates #23984
2017-08-01 19:16:15 -04:00
Clinton Gormley ba552a0736 Added "release-state" support to plugin docs 2017-04-20 15:04:20 +02:00
Clinton Gormley e181a020a9 Replaced absolute URLs in docs with attributes 2017-02-04 12:05:03 +01:00
David Pilato 17930930a7 Update after review 2017-01-26 17:10:37 +01:00
David Pilato 3804bfcc60 Read ec2 discovery address from aws instance tags
This PR adds a new option for `host_type`: `tag:TAGNAME` where `TAGNAME` is the tag field you defined for your ec2 instance.

For example if you defined a tag `my-elasticsearch-host` in ec2 and set it to `myhostname1.mydomain.com`, then
setting `host_type: tag:my-elasticsearch-host` will tell Discovery Ec2 plugin to read the host name from the
`my-elasticsearch-host` tag. In this case, it will be resolved to `myhostname1.mydomain.com`.

Closes #22566.
2017-01-26 17:10:37 +01:00
animageofmine e3546d59c4 Add support for ca-central-1 region to EC2 and S3 plugins
Closes #22458 #22454
2017-01-06 16:27:08 -06:00
Nic Palmer 3894ec9bae Fixed eu-west-2 entries for discovery-ec2 and repository-s3 also updated the asciidocs 2016-12-21 15:48:07 +00:00
David Pilato 8923b36780 Merge pull request #21956 from alexshadow007/aws_read_timeout
Add setting to set read timeout for EC2 discovery and S3 repository plugins
2016-12-07 16:00:48 +01:00
Alexander Kazakov 0a03a62ab6 Using ClientConfiguration.DEFAULT_SOCKET_TIMEOUT as default value for read timeout 2016-12-06 21:13:28 +03:00
Alexander Kazakov 1491e2dec9 Remove default value for read_timeout setting
Fix tests and docs
2016-12-05 21:29:17 +03:00
Alexander Kazakov 23550f277b Add us-east-2 AWS region 2016-12-04 20:02:05 +03:00
Alexander Kazakov 5695eaf19e Add setting to set read timeout for EC2 discovery and S3 repository plugins 2016-12-04 01:58:53 +03:00
Chris Earle 3f2d22bc61 [DOCS] Show EC2's auto attribute (#21474)
* [DOCS] Show EC2's auto attribute

This documents the `aws_availability_zone` node attribute as part of the `discovery-ec2` plugin. Also fixes outdated usage of "cloud aws".
2016-11-21 11:04:53 -05:00
Ryan Ernst edb9d8fc03 Docs: Update discovery plugins example configuration with discovery.zen.hosts_provider (#21390)
This new setting was added with deguicing unicast hosts providers, but
the docs were not updated.
2016-11-07 13:33:08 -08:00
David Pilato 2e1539b879 Fix URL for plugins
Reported at https://discuss.elastic.co/t/ingest-geoip-for-beta1/62505
2016-10-07 23:33:13 +02:00
David Pilato 4c9a77e6c0 Add link to "offline install" and show the full URL 2016-09-19 15:04:29 +02:00
David Pilato 472c679981 Add offline install instructions for plugins
Follow up of https://github.com/elastic/elasticsearch/issues/15925#issuecomment-171250150

This commit adds offline install instructions for plugins.
2016-09-12 15:36:16 +02:00
Nik Everett 5cff2a046d Remove most of the need for `// NOTCONSOLE`
and be much more stingy about what we consider a console candidate.

* Add `// CONSOLE` to check-running
* Fix version in some snippets
* Mark groovy snippets as groovy
* Fix versions in plugins
* Fix language marker errors
* Fix language parsing in snippets

  This adds support for snippets who's language is written like
  `[source, txt]` and `["source","js",subs="attributes,callouts"]`.

  This also makes language required for snippets which is nice because
  then we can be sure we can grep for snippets in a particular language.
2016-09-06 10:32:54 -04:00
Clinton Gormley 9cce45d4af Add cloud to S3/EC2 plugin docs 2016-08-24 15:38:43 +02:00
Nik Everett ffd226efa0 Add `// NOTCONSOLE` to docs
We have 1074 snippets that look like they should be converted to
`// CONSOLE`. At least that is what `gradle docs:listConsoleCandidates`
says. This adds `// NOTCONSOLE` to explicitly mark snippets that
*shouldn't* be converted to `// CONSOLE`. After marking the blindingly
obvious ones this cuts the remaining snippet count to 1032.
2016-08-12 16:49:57 -04:00
David Pilato 3bcf2653bb ping_timeout is documented in discovery-ec2 but does not exist in code
Also mentioned in discovery-gce

Actually ping timeout can be set using `discovery.zen.ping_timeout`.

Closes #16600.
2016-07-21 15:03:56 +02:00
David Pilato af989c0780 Support new Asia Pacific (Mumbai) ap-south-1 AWS region
AWS [announced](http://www.allthingsdistributed.com/2016/06/introducing-aws-asia-pacific-mumbai-region.html) a new region: Asia Pacific (Mumbai)	`ap-south-1`.

We need to support it for:

* repository-s3: s3.ap-south-1.amazonaws.com or s3-ap-south-1.amazonaws.com
* discovery-ec2: ec2.ap-south-1.amazonaws.com

For reference: http://docs.aws.amazon.com/general/latest/gr/rande.html

Closes #19110.
2016-06-28 08:50:50 +02:00
Itamar Syn-Hershko 8ea6264f55 Format settings in discovery-ec2 docs
Closes #16846
2016-02-28 11:15:22 -05:00
Jim Ferenczi 7d0181b5d4 Rename bin/plugin in bin/elasticsearch-plugin 2016-02-05 10:09:14 +01:00
David Pilato aff3c564b3 Add seoul endpoints for EC2 discovery and S3 snapshots
Add documentation for #16167
2016-01-25 08:46:51 +01:00
Ryan Schneider c42149ebe2 merge PR 14212 - add "Best Practices" section to the cloud-aws docs 2016-01-13 11:01:24 -08:00
David Pilato 7dcb40bcac Add support for proxy authentication for s3 and ec2
When using S3 or EC2, it was possible to use a proxy to access EC2 or S3 API but username and password were not possible to be set.

This commit adds support for this. Also, to make all that consistent, proxy settings for both plugins have been renamed:

* from `cloud.aws.proxy_host` to `cloud.aws.proxy.host`
* from `cloud.aws.ec2.proxy_host` to `cloud.aws.ec2.proxy.host`
* from `cloud.aws.s3.proxy_host` to `cloud.aws.s3.proxy.host`
* from `cloud.aws.proxy_port` to `cloud.aws.proxy.port`
* from `cloud.aws.ec2.proxy_port` to `cloud.aws.ec2.proxy.port`
* from `cloud.aws.s3.proxy_port` to `cloud.aws.s3.proxy.port`

New settings are `proxy.username` and `proxy.password`.

```yml
cloud:
    aws:
        protocol: https
        proxy:
            host: proxy1.company.com
            port: 8083
            username: myself
            password: theBestPasswordEver!
```

You can also set different proxies for `ec2` and `s3`:

```yml
cloud:
    aws:
        s3:
            proxy:
                host: proxy1.company.com
                port: 8083
                username: myself1
                password: theBestPasswordEver1!
        ec2:
            proxy:
                host: proxy2.company.com
                port: 8083
                username: myself2
                password: theBestPasswordEver2!
```

Note that `password` is filtered with `SettingsFilter`.

We also fix a potential issue in S3 repository. We were supposed to accept key/secret either set under `cloud.aws` or `cloud.aws.s3` but the actual code never implemented that.

It was:

```java
account = settings.get("cloud.aws.access_key");
key = settings.get("cloud.aws.secret_key");
```

We replaced that by:

```java
String account = settings.get(CLOUD_S3.KEY, settings.get(CLOUD_AWS.KEY));
String key = settings.get(CLOUD_S3.SECRET, settings.get(CLOUD_AWS.SECRET));
```

Also, we extract all settings for S3 in `AwsS3Service` as it's already the case for `AwsEc2Service` class.

Closes #15268.
2015-12-07 23:10:54 +01:00
Christian Haudum 3334160003 improved building of disco nodes
* improved retry policy of ec2 client
* cache results for 10s
2015-10-29 18:07:52 +01:00
David Pilato 7b431ecc16 Fix cross links
Related to #13602
(cherry picked from commit ff4cdab)
(cherry picked from commit 8b46a4b)
2015-10-06 12:18:32 +02:00
David Pilato 685c1f4a54 [discovery-ec2] network.host must be set
With 2.0, we now bind to `localhost` by default instead of binding to the network card and use its IP address.

 When the discovery plugin gets from AWS API the list of nodes that should form the cluster, this list is pinged then. But as each node is bound to `localhost`, ping does not get an answer and the node elects itself as the master node.

`network.host` must be set.

 Closes #13589.
2015-10-06 11:18:47 +02:00
Mikko Nylén a379400228 Add docs note about cloud.aws.region (#13668) 2015-09-22 14:53:54 +03:00
xuzha e69551d8df This commit remove the block for user to set invalid Signer Type.
Instead of throw an exception, this change log a warning message
and accept any signer type.
2015-09-05 22:57:05 -07:00
David Pilato 30aa231f8e [plugin] split cloud-aws in repository-s3 and discovery-ec2
Until now we had a cloud-aws plugin which is providing 2 disctinct features:

* discovery on EC2
* snapshot/restore on S3

This commit splits the plugin by feature so people can use either one or the other or both features.

Doc is updated accordingly.
2015-09-03 11:12:20 +02:00