honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix network binding for ipv4/ipv6 

When elasticsearch is configured by interface (or default: loopback interfaces),
bind to all addresses on the interface rather than an arbitrary one.

If the publish address is not specified, default it from the bound addresses
based on the following sort ordering:

* ipv4/ipv6 (java.net.preferIPv4Stack, defaults to true)
* ordinary addresses
* site-local addresses
* link local addresses
* loopback addresses

One one address is published, and multicast is still always over ipv4: these
need to be future improvements.

Closes #12906
Closes #12915

Squashed commit of the following:

commit 7e60833312f329a5749f9a256b9c1331a956d98f
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 14:45:33 2015 -0400

    fix java 7 compilation oops

commit c7b9f3a42058beb061b05c6dd67fd91477fd258a
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 14:24:16 2015 -0400

    Cleanup/fix logic around custom resolvers

commit bd7065f1936e14a29c9eb8fe4ecab0ce512ac08e
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 13:29:42 2015 -0400

    Add some unit tests for utility methods

commit 0faf71cb0ee9a45462d58af3d1bf214e8a79347c
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 12:11:48 2015 -0400

    localhost all the way down

commit e198bb2bc0d1673288b96e07e6e6ad842179978c
Merge: b55d092 b93a75f
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 12:05:02 2015 -0400

    Merge branch 'master' into network_cleanup

commit b55d092811d7832bae579c5586e171e9cc1ebe9d
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 12:03:03 2015 -0400

    fix docs, fix another bug in multicast (publish host = bad here!)

commit 88c462eb302b30a82585f95413927a5cbb7d54c4
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 11:50:49 2015 -0400

    remove nocommit

commit 89547d7b10d68b23d7f24362e1f4782f5e1ca03c
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 11:49:35 2015 -0400

    fix http too

commit 9b9413aca8a3f6397b5031831f910791b685e5be
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Aug 17 11:06:02 2015 -0400

    Fix transport / interface code

    Next up: multicast and then http
This commit is contained in:
Robert Muir 2015-08-17 15:37:07 -04:00
parent 75ced057ed
commit 68307aa9f3
17 changed files with 475 additions and 487 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
README.textile
View File

@ -42,7 +42,7 @@ h3. Installation
* "Download":https://www.elastic.co/downloads/elasticsearch and unzip the Elasticsearch official distribution. * "Download":https://www.elastic.co/downloads/elasticsearch and unzip the Elasticsearch official distribution.
* Run @bin/elasticsearch@ on unix, or @bin\elasticsearch.bat@ on windows. * Run @bin/elasticsearch@ on unix, or @bin\elasticsearch.bat@ on windows.
* Run @curl -X GET http://127.0.0.1:9200/@. * Run @curl -X GET http://localhost:9200/@.
* Start more servers ... * Start more servers ...
h3. Indexing h3. Indexing
@ -50,16 +50,16 @@ h3. Indexing
Let's try and index some twitter like information. First, let's create a twitter user, and add some tweets (the @twitter@ index will be created automatically): Let's try and index some twitter like information. First, let's create a twitter user, and add some tweets (the @twitter@ index will be created automatically):
<pre> <pre>
curl -XPUT 'http://127.0.0.1:9200/twitter/user/kimchy' -d '{ "name" : "Shay Banon" }' curl -XPUT 'http://localhost:9200/twitter/user/kimchy' -d '{ "name" : "Shay Banon" }'
curl -XPUT 'http://127.0.0.1:9200/twitter/tweet/1' -d ' curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T13:12:00", "postDate": "2009-11-15T13:12:00",
"message": "Trying out Elasticsearch, so far so good?" "message": "Trying out Elasticsearch, so far so good?"
}' }'
curl -XPUT 'http://127.0.0.1:9200/twitter/tweet/2' -d ' curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T14:12:12", "postDate": "2009-11-15T14:12:12",
@ -70,9 +70,9 @@ curl -XPUT 'http://127.0.0.1:9200/twitter/tweet/2' -d '
Now, let's see if the information was added by GETting it: Now, let's see if the information was added by GETting it:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/user/kimchy?pretty=true' curl -XGET 'http://localhost:9200/twitter/user/kimchy?pretty=true'
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/1?pretty=true' curl -XGET 'http://localhost:9200/twitter/tweet/1?pretty=true'
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/2?pretty=true' curl -XGET 'http://localhost:9200/twitter/tweet/2?pretty=true'
</pre> </pre>
h3. Searching h3. Searching
@ -81,13 +81,13 @@ Mmm search..., shouldn't it be elastic?
Let's find all the tweets that @kimchy@ posted: Let's find all the tweets that @kimchy@ posted:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/_search?q=user:kimchy&pretty=true' curl -XGET 'http://localhost:9200/twitter/tweet/_search?q=user:kimchy&pretty=true'
</pre> </pre>
We can also use the JSON query language Elasticsearch provides instead of a query string: We can also use the JSON query language Elasticsearch provides instead of a query string:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/twitter/tweet/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"match" : { "user": "kimchy" } "match" : { "user": "kimchy" }
@ -98,7 +98,7 @@ curl -XGET 'http://127.0.0.1:9200/twitter/tweet/_search?pretty=true' -d '
Just for kicks, let's get all the documents stored (we should see the user as well): Just for kicks, let's get all the documents stored (we should see the user as well):
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/twitter/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"matchAll" : {} "matchAll" : {}
@ -109,7 +109,7 @@ curl -XGET 'http://127.0.0.1:9200/twitter/_search?pretty=true' -d '
We can also do range search (the @postDate@ was automatically identified as date) We can also do range search (the @postDate@ was automatically identified as date)
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/twitter/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"range" : { "range" : {
@ -130,16 +130,16 @@ Elasticsearch supports multiple indices, as well as multiple types per index. In
Another way to define our simple twitter system is to have a different index per user (note, though that each index has an overhead). Here is the indexing curl's in this case: Another way to define our simple twitter system is to have a different index per user (note, though that each index has an overhead). Here is the indexing curl's in this case:
<pre> <pre>
curl -XPUT 'http://127.0.0.1:9200/kimchy/info/1' -d '{ "name" : "Shay Banon" }' curl -XPUT 'http://localhost:9200/kimchy/info/1' -d '{ "name" : "Shay Banon" }'
curl -XPUT 'http://127.0.0.1:9200/kimchy/tweet/1' -d ' curl -XPUT 'http://localhost:9200/kimchy/tweet/1' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T13:12:00", "postDate": "2009-11-15T13:12:00",
"message": "Trying out Elasticsearch, so far so good?" "message": "Trying out Elasticsearch, so far so good?"
}' }'
curl -XPUT 'http://127.0.0.1:9200/kimchy/tweet/2' -d ' curl -XPUT 'http://localhost:9200/kimchy/tweet/2' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T14:12:12", "postDate": "2009-11-15T14:12:12",
@ -152,7 +152,7 @@ The above will index information into the @kimchy@ index, with two types, @info@
Complete control on the index level is allowed. As an example, in the above case, we would want to change from the default 5 shards with 1 replica per index, to only 1 shard with 1 replica per index (== per twitter user). Here is how this can be done (the configuration can be in yaml as well): Complete control on the index level is allowed. As an example, in the above case, we would want to change from the default 5 shards with 1 replica per index, to only 1 shard with 1 replica per index (== per twitter user). Here is how this can be done (the configuration can be in yaml as well):
<pre> <pre>
curl -XPUT http://127.0.0.1:9200/another_user/ -d ' curl -XPUT http://localhost:9200/another_user/ -d '
{ {
"index" : { "index" : {
"numberOfShards" : 1, "numberOfShards" : 1,
@ -165,7 +165,7 @@ Search (and similar operations) are multi index aware. This means that we can ea
index (twitter user), for example: index (twitter user), for example:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/kimchy,another_user/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/kimchy,another_user/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"matchAll" : {} "matchAll" : {}
@ -176,7 +176,7 @@ curl -XGET 'http://127.0.0.1:9200/kimchy,another_user/_search?pretty=true' -d '
Or on all the indices: Or on all the indices:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"matchAll" : {} "matchAll" : {}

34
core/README.textile
View File

@ -42,7 +42,7 @@ h3. Installation
* "Download":https://www.elastic.co/downloads/elasticsearch and unzip the Elasticsearch official distribution. * "Download":https://www.elastic.co/downloads/elasticsearch and unzip the Elasticsearch official distribution.
* Run @bin/elasticsearch@ on unix, or @bin\elasticsearch.bat@ on windows. * Run @bin/elasticsearch@ on unix, or @bin\elasticsearch.bat@ on windows.
* Run @curl -X GET http://127.0.0.1:9200/@. * Run @curl -X GET http://localhost:9200/@.
* Start more servers ... * Start more servers ...
h3. Indexing h3. Indexing
@ -50,16 +50,16 @@ h3. Indexing
Let's try and index some twitter like information. First, let's create a twitter user, and add some tweets (the @twitter@ index will be created automatically): Let's try and index some twitter like information. First, let's create a twitter user, and add some tweets (the @twitter@ index will be created automatically):
<pre> <pre>
curl -XPUT 'http://127.0.0.1:9200/twitter/user/kimchy' -d '{ "name" : "Shay Banon" }' curl -XPUT 'http://localhost:9200/twitter/user/kimchy' -d '{ "name" : "Shay Banon" }'
curl -XPUT 'http://127.0.0.1:9200/twitter/tweet/1' -d ' curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T13:12:00", "postDate": "2009-11-15T13:12:00",
"message": "Trying out Elasticsearch, so far so good?" "message": "Trying out Elasticsearch, so far so good?"
}' }'
curl -XPUT 'http://127.0.0.1:9200/twitter/tweet/2' -d ' curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T14:12:12", "postDate": "2009-11-15T14:12:12",
@ -70,9 +70,9 @@ curl -XPUT 'http://127.0.0.1:9200/twitter/tweet/2' -d '
Now, let's see if the information was added by GETting it: Now, let's see if the information was added by GETting it:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/user/kimchy?pretty=true' curl -XGET 'http://localhost:9200/twitter/user/kimchy?pretty=true'
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/1?pretty=true' curl -XGET 'http://localhost:9200/twitter/tweet/1?pretty=true'
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/2?pretty=true' curl -XGET 'http://localhost:9200/twitter/tweet/2?pretty=true'
</pre> </pre>
h3. Searching h3. Searching
@ -81,13 +81,13 @@ Mmm search..., shouldn't it be elastic?
Let's find all the tweets that @kimchy@ posted: Let's find all the tweets that @kimchy@ posted:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/_search?q=user:kimchy&pretty=true' curl -XGET 'http://localhost:9200/twitter/tweet/_search?q=user:kimchy&pretty=true'
</pre> </pre>
We can also use the JSON query language Elasticsearch provides instead of a query string: We can also use the JSON query language Elasticsearch provides instead of a query string:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/tweet/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/twitter/tweet/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"match" : { "user": "kimchy" } "match" : { "user": "kimchy" }
@ -98,7 +98,7 @@ curl -XGET 'http://127.0.0.1:9200/twitter/tweet/_search?pretty=true' -d '
Just for kicks, let's get all the documents stored (we should see the user as well): Just for kicks, let's get all the documents stored (we should see the user as well):
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/twitter/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"matchAll" : {} "matchAll" : {}
@ -109,7 +109,7 @@ curl -XGET 'http://127.0.0.1:9200/twitter/_search?pretty=true' -d '
We can also do range search (the @postDate@ was automatically identified as date) We can also do range search (the @postDate@ was automatically identified as date)
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/twitter/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/twitter/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"range" : { "range" : {
@ -130,16 +130,16 @@ Elasticsearch supports multiple indices, as well as multiple types per index. In
Another way to define our simple twitter system is to have a different index per user (note, though that each index has an overhead). Here is the indexing curl's in this case: Another way to define our simple twitter system is to have a different index per user (note, though that each index has an overhead). Here is the indexing curl's in this case:
<pre> <pre>
curl -XPUT 'http://127.0.0.1:9200/kimchy/info/1' -d '{ "name" : "Shay Banon" }' curl -XPUT 'http://localhost:9200/kimchy/info/1' -d '{ "name" : "Shay Banon" }'
curl -XPUT 'http://127.0.0.1:9200/kimchy/tweet/1' -d ' curl -XPUT 'http://localhost:9200/kimchy/tweet/1' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T13:12:00", "postDate": "2009-11-15T13:12:00",
"message": "Trying out Elasticsearch, so far so good?" "message": "Trying out Elasticsearch, so far so good?"
}' }'
curl -XPUT 'http://127.0.0.1:9200/kimchy/tweet/2' -d ' curl -XPUT 'http://localhost:9200/kimchy/tweet/2' -d '
{ {
"user": "kimchy", "user": "kimchy",
"postDate": "2009-11-15T14:12:12", "postDate": "2009-11-15T14:12:12",
@ -152,7 +152,7 @@ The above will index information into the @kimchy@ index, with two types, @info@
Complete control on the index level is allowed. As an example, in the above case, we would want to change from the default 5 shards with 1 replica per index, to only 1 shard with 1 replica per index (== per twitter user). Here is how this can be done (the configuration can be in yaml as well): Complete control on the index level is allowed. As an example, in the above case, we would want to change from the default 5 shards with 1 replica per index, to only 1 shard with 1 replica per index (== per twitter user). Here is how this can be done (the configuration can be in yaml as well):
<pre> <pre>
curl -XPUT http://127.0.0.1:9200/another_user/ -d ' curl -XPUT http://localhost:9200/another_user/ -d '
{ {
"index" : { "index" : {
"numberOfShards" : 1, "numberOfShards" : 1,
@ -165,7 +165,7 @@ Search (and similar operations) are multi index aware. This means that we can ea
index (twitter user), for example: index (twitter user), for example:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/kimchy,another_user/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/kimchy,another_user/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"matchAll" : {} "matchAll" : {}
@ -176,7 +176,7 @@ curl -XGET 'http://127.0.0.1:9200/kimchy,another_user/_search?pretty=true' -d '
Or on all the indices: Or on all the indices:
<pre> <pre>
curl -XGET 'http://127.0.0.1:9200/_search?pretty=true' -d ' curl -XGET 'http://localhost:9200/_search?pretty=true' -d '
{ {
"query" : { "query" : {
"matchAll" : {} "matchAll" : {}

4
core/src/main/java/org/elasticsearch/cluster/node/DiscoveryNode.java
View File

@ -21,6 +21,7 @@ package org.elasticsearch.cluster.node;
import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap; import com.google.common.collect.ImmutableMap;
import org.elasticsearch.Version; import org.elasticsearch.Version;
import org.elasticsearch.common.Booleans; import org.elasticsearch.common.Booleans;
import org.elasticsearch.common.Strings; import org.elasticsearch.common.Strings;
@ -33,6 +34,7 @@ import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentBuilder;
import java.io.IOException; import java.io.IOException;
import java.net.InetAddress;
import java.util.Map; import java.util.Map;
import static org.elasticsearch.common.transport.TransportAddressSerializers.addressToStream; import static org.elasticsearch.common.transport.TransportAddressSerializers.addressToStream;
@ -136,7 +138,7 @@ public class DiscoveryNode implements Streamable, ToXContent {
* @param version the version of the node. * @param version the version of the node.
*/ */
public DiscoveryNode(String nodeName, String nodeId, TransportAddress address, Map<String, String> attributes, Version version) { public DiscoveryNode(String nodeName, String nodeId, TransportAddress address, Map<String, String> attributes, Version version) {
this(nodeName, nodeId, NetworkUtils.getLocalHostName(""), NetworkUtils.getLocalHostAddress(""), address, attributes, version); this(nodeName, nodeId, NetworkUtils.getLocalHost().getHostName(), NetworkUtils.getLocalHost().getHostAddress(), address, attributes, version);
} }
/** /**

160
core/src/main/java/org/elasticsearch/common/network/NetworkService.java
View File

@ -28,11 +28,8 @@ import org.elasticsearch.common.unit.TimeValue;
import java.io.IOException; import java.io.IOException;
import java.net.InetAddress; import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.UnknownHostException; import java.net.UnknownHostException;
import java.util.Collection;
import java.util.List; import java.util.List;
import java.util.Locale;
import java.util.concurrent.CopyOnWriteArrayList; import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeUnit;
@ -41,7 +38,8 @@ import java.util.concurrent.TimeUnit;
*/ */
public class NetworkService extends AbstractComponent { public class NetworkService extends AbstractComponent {
public static final String LOCAL = "#local#"; /** By default, we bind to loopback interfaces */
public static final String DEFAULT_NETWORK_HOST = "_local_";
private static final String GLOBAL_NETWORK_HOST_SETTING = "network.host"; private static final String GLOBAL_NETWORK_HOST_SETTING = "network.host";
private static final String GLOBAL_NETWORK_BINDHOST_SETTING = "network.bind_host"; private static final String GLOBAL_NETWORK_BINDHOST_SETTING = "network.bind_host";
@ -71,12 +69,12 @@ public class NetworkService extends AbstractComponent {
/** /**
* Resolves the default value if possible. If not, return <tt>null</tt>. * Resolves the default value if possible. If not, return <tt>null</tt>.
*/ */
InetAddress resolveDefault(); InetAddress[] resolveDefault();
/** /**
* Resolves a custom value handling, return <tt>null</tt> if can't handle it. * Resolves a custom value handling, return <tt>null</tt> if can't handle it.
*/ */
InetAddress resolveIfPossible(String value); InetAddress[] resolveIfPossible(String value);
} }
private final List<CustomNameResolver> customNameResolvers = new CopyOnWriteArrayList<>(); private final List<CustomNameResolver> customNameResolvers = new CopyOnWriteArrayList<>();
@ -94,100 +92,86 @@ public class NetworkService extends AbstractComponent {
customNameResolvers.add(customNameResolver); customNameResolvers.add(customNameResolver);
} }
public InetAddress[] resolveBindHostAddress(String bindHost) throws IOException {
public InetAddress resolveBindHostAddress(String bindHost) throws IOException { // first check settings
return resolveBindHostAddress(bindHost, InetAddress.getLoopbackAddress().getHostAddress()); if (bindHost == null) {
} bindHost = settings.get(GLOBAL_NETWORK_BINDHOST_SETTING, settings.get(GLOBAL_NETWORK_HOST_SETTING));
public InetAddress resolveBindHostAddress(String bindHost, String defaultValue2) throws IOException {
return resolveInetAddress(bindHost, settings.get(GLOBAL_NETWORK_BINDHOST_SETTING, settings.get(GLOBAL_NETWORK_HOST_SETTING)), defaultValue2);
}
public InetAddress resolvePublishHostAddress(String publishHost) throws IOException {
InetAddress address = resolvePublishHostAddress(publishHost,
InetAddress.getLoopbackAddress().getHostAddress());
// verify that its not a local address
if (address == null || address.isAnyLocalAddress()) {
address = NetworkUtils.getFirstNonLoopbackAddress(NetworkUtils.StackType.IPv4);
if (address == null) {
address = NetworkUtils.getFirstNonLoopbackAddress(NetworkUtils.getIpStackType());
if (address == null) {
address = NetworkUtils.getLocalAddress();
if (address == null) {
return NetworkUtils.getLocalhost(NetworkUtils.StackType.IPv4);
}
}
}
} }
return address; // next check any registered custom resolvers
} if (bindHost == null) {
public InetAddress resolvePublishHostAddress(String publishHost, String defaultValue2) throws IOException {
return resolveInetAddress(publishHost, settings.get(GLOBAL_NETWORK_PUBLISHHOST_SETTING, settings.get(GLOBAL_NETWORK_HOST_SETTING)), defaultValue2);
}
public InetAddress resolveInetAddress(String host, String defaultValue1, String defaultValue2) throws UnknownHostException, IOException {
if (host == null) {
host = defaultValue1;
}
if (host == null) {
host = defaultValue2;
}
if (host == null) {
for (CustomNameResolver customNameResolver : customNameResolvers) { for (CustomNameResolver customNameResolver : customNameResolvers) {
InetAddress inetAddress = customNameResolver.resolveDefault(); InetAddress addresses[] = customNameResolver.resolveDefault();
if (inetAddress != null) { if (addresses != null) {
return inetAddress; return addresses;
} }
} }
return null;
} }
String origHost = host; // finally, fill with our default
if (bindHost == null) {
bindHost = DEFAULT_NETWORK_HOST;
}
return resolveInetAddress(bindHost);
}
// TODO: needs to be InetAddress[]
public InetAddress resolvePublishHostAddress(String publishHost) throws IOException {
// first check settings
if (publishHost == null) {
publishHost = settings.get(GLOBAL_NETWORK_PUBLISHHOST_SETTING, settings.get(GLOBAL_NETWORK_HOST_SETTING));
}
// next check any registered custom resolvers
if (publishHost == null) {
for (CustomNameResolver customNameResolver : customNameResolvers) {
InetAddress addresses[] = customNameResolver.resolveDefault();
if (addresses != null) {
return addresses[0];
}
}
}
// finally, fill with our default
if (publishHost == null) {
publishHost = DEFAULT_NETWORK_HOST;
}
// TODO: allow publishing multiple addresses
return resolveInetAddress(publishHost)[0];
}
private InetAddress[] resolveInetAddress(String host) throws UnknownHostException, IOException {
if ((host.startsWith("#") && host.endsWith("#")) || (host.startsWith("_") && host.endsWith("_"))) { if ((host.startsWith("#") && host.endsWith("#")) || (host.startsWith("_") && host.endsWith("_"))) {
host = host.substring(1, host.length() - 1); host = host.substring(1, host.length() - 1);
// allow custom resolvers to have special names
for (CustomNameResolver customNameResolver : customNameResolvers) { for (CustomNameResolver customNameResolver : customNameResolvers) {
InetAddress inetAddress = customNameResolver.resolveIfPossible(host); InetAddress addresses[] = customNameResolver.resolveIfPossible(host);
if (inetAddress != null) { if (addresses != null) {
return inetAddress; return addresses;
} }
} }
switch (host) {
if (host.equals("local")) { case "local":
return NetworkUtils.getLocalAddress(); return NetworkUtils.getLoopbackAddresses();
} else if (host.startsWith("non_loopback")) { case "local:ipv4":
if (host.toLowerCase(Locale.ROOT).endsWith(":ipv4")) { return NetworkUtils.filterIPV4(NetworkUtils.getLoopbackAddresses());
return NetworkUtils.getFirstNonLoopbackAddress(NetworkUtils.StackType.IPv4); case "local:ipv6":
} else if (host.toLowerCase(Locale.ROOT).endsWith(":ipv6")) { return NetworkUtils.filterIPV6(NetworkUtils.getLoopbackAddresses());
return NetworkUtils.getFirstNonLoopbackAddress(NetworkUtils.StackType.IPv6); case "non_loopback":
} else { return NetworkUtils.getFirstNonLoopbackAddresses();
return NetworkUtils.getFirstNonLoopbackAddress(NetworkUtils.getIpStackType()); case "non_loopback:ipv4":
} return NetworkUtils.filterIPV4(NetworkUtils.getFirstNonLoopbackAddresses());
} else { case "non_loopback:ipv6":
NetworkUtils.StackType stackType = NetworkUtils.getIpStackType(); return NetworkUtils.filterIPV6(NetworkUtils.getFirstNonLoopbackAddresses());
if (host.toLowerCase(Locale.ROOT).endsWith(":ipv4")) { default:
stackType = NetworkUtils.StackType.IPv4; /* an interface specification */
host = host.substring(0, host.length() - 5); if (host.endsWith(":ipv4")) {
} else if (host.toLowerCase(Locale.ROOT).endsWith(":ipv6")) { host = host.substring(0, host.length() - 5);
stackType = NetworkUtils.StackType.IPv6; return NetworkUtils.filterIPV4(NetworkUtils.getAddressesForInterface(host));
host = host.substring(0, host.length() - 5); } else if (host.endsWith(":ipv6")) {
} host = host.substring(0, host.length() - 5);
Collection<NetworkInterface> allInterfs = NetworkUtils.getAllAvailableInterfaces(); return NetworkUtils.filterIPV6(NetworkUtils.getAddressesForInterface(host));
for (NetworkInterface ni : allInterfs) { } else {
if (!ni.isUp()) { return NetworkUtils.getAddressesForInterface(host);
continue;
} }
if (host.equals(ni.getName()) || host.equals(ni.getDisplayName())) {
if (ni.isLoopback()) {
return NetworkUtils.getFirstAddress(ni, stackType);
} else {
return NetworkUtils.getFirstNonLoopbackAddress(ni, stackType);
}
}
}
} }
throw new IOException("Failed to find network interface for [" + origHost + "]");
} }
return InetAddress.getByName(host); return NetworkUtils.getAllByName(host);
} }
} }

444
core/src/main/java/org/elasticsearch/common/network/NetworkUtils.java
View File

@ -19,303 +19,205 @@
package org.elasticsearch.common.network; package org.elasticsearch.common.network;
import com.google.common.collect.Lists;
import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.BytesRef;
import org.apache.lucene.util.CollectionUtil;
import org.apache.lucene.util.Constants; import org.apache.lucene.util.Constants;
import org.elasticsearch.common.logging.ESLogger; import org.elasticsearch.common.logging.ESLogger;
import org.elasticsearch.common.logging.Loggers; import org.elasticsearch.common.logging.Loggers;
import java.net.*; import java.net.Inet4Address;
import java.util.*; import java.net.Inet6Address;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.SocketException;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
/** /**
* * Utilities for network interfaces / addresses
*/ */
public abstract class NetworkUtils { public abstract class NetworkUtils {
/** no instantation */
private NetworkUtils() {}
/**
* By default we bind to any addresses on an interface/name, unless restricted by :ipv4 etc.
* This property is unrelated to that, this is about what we *publish*. Today the code pretty much
* expects one address so this is used for the sort order.
* @deprecated transition mechanism only
*/
@Deprecated
static final boolean PREFER_V4 = Boolean.parseBoolean(System.getProperty("java.net.preferIPv4Stack", "true"));
/** Sorts an address by preference. This way code like publishing can just pick the first one */
static int sortKey(InetAddress address, boolean prefer_v4) {
int key = address.getAddress().length;
if (prefer_v4 == false) {
key = -key;
}
if (address.isAnyLocalAddress()) {
key += 5;
}
if (address.isMulticastAddress()) {
key += 4;
}
if (address.isLoopbackAddress()) {
key += 3;
}
if (address.isLinkLocalAddress()) {
key += 2;
}
if (address.isSiteLocalAddress()) {
key += 1;
}
return key;
}
/**
* Sorts addresses by order of preference. This is used to pick the first one for publishing
* @deprecated remove this when multihoming is really correct
*/
@Deprecated
private static void sortAddresses(List<InetAddress> list) {
Collections.sort(list, new Comparator<InetAddress>() {
@Override
public int compare(InetAddress left, InetAddress right) {
int cmp = Integer.compare(sortKey(left, PREFER_V4), sortKey(right, PREFER_V4));
if (cmp == 0) {
cmp = new BytesRef(left.getAddress()).compareTo(new BytesRef(right.getAddress()));
}
return cmp;
}
});
}
private final static ESLogger logger = Loggers.getLogger(NetworkUtils.class); private final static ESLogger logger = Loggers.getLogger(NetworkUtils.class);
public static enum StackType { /** Return all interfaces (and subinterfaces) on the system */
IPv4, IPv6, Unknown static List<NetworkInterface> getInterfaces() throws SocketException {
List<NetworkInterface> all = new ArrayList<>();
addAllInterfaces(all, Collections.list(NetworkInterface.getNetworkInterfaces()));
Collections.sort(all, new Comparator<NetworkInterface>() {
@Override
public int compare(NetworkInterface left, NetworkInterface right) {
return Integer.compare(left.getIndex(), right.getIndex());
}
});
return all;
} }
public static final String IPv4_SETTING = "java.net.preferIPv4Stack"; /** Helper for getInterfaces, recursively adds subinterfaces to {@code target} */
public static final String IPv6_SETTING = "java.net.preferIPv6Addresses"; private static void addAllInterfaces(List<NetworkInterface> target, List<NetworkInterface> level) {
if (!level.isEmpty()) {
public static final String NON_LOOPBACK_ADDRESS = "non_loopback_address"; target.addAll(level);
for (NetworkInterface intf : level) {
private final static InetAddress localAddress; addAllInterfaces(target, Collections.list(intf.getSubInterfaces()));
}
static {
InetAddress localAddressX;
try {
localAddressX = InetAddress.getLocalHost();
} catch (Throwable e) {
logger.warn("failed to resolve local host, fallback to loopback", e);
localAddressX = InetAddress.getLoopbackAddress();
} }
localAddress = localAddressX;
} }
/** Returns system default for SO_REUSEADDR */
public static boolean defaultReuseAddress() { public static boolean defaultReuseAddress() {
return Constants.WINDOWS ? false : true; return Constants.WINDOWS ? false : true;
} }
public static boolean isIPv4() { /** Returns localhost, or if its misconfigured, falls back to loopback. Use with caution!!!! */
return System.getProperty("java.net.preferIPv4Stack") != null && System.getProperty("java.net.preferIPv4Stack").equals("true"); // TODO: can we remove this?
} public static InetAddress getLocalHost() {
public static InetAddress getIPv4Localhost() throws UnknownHostException {
return getLocalhost(StackType.IPv4);
}
public static InetAddress getIPv6Localhost() throws UnknownHostException {
return getLocalhost(StackType.IPv6);
}
public static InetAddress getLocalAddress() {
return localAddress;
}
public static String getLocalHostName(String defaultHostName) {
if (localAddress == null) {
return defaultHostName;
}
String hostName = localAddress.getHostName();
if (hostName == null) {
return defaultHostName;
}
return hostName;
}
public static String getLocalHostAddress(String defaultHostAddress) {
if (localAddress == null) {
return defaultHostAddress;
}
String hostAddress = localAddress.getHostAddress();
if (hostAddress == null) {
return defaultHostAddress;
}
return hostAddress;
}
public static InetAddress getLocalhost(StackType ip_version) throws UnknownHostException {
if (ip_version == StackType.IPv4)
return InetAddress.getByName("127.0.0.1");
else
return InetAddress.getByName("::1");
}
/**
* Returns the first non-loopback address on any interface on the current host.
*
* @param ip_version Constraint on IP version of address to be returned, 4 or 6
*/
public static InetAddress getFirstNonLoopbackAddress(StackType ip_version) throws SocketException {
InetAddress address;
for (NetworkInterface intf : getInterfaces()) {
try {
if (!intf.isUp() || intf.isLoopback())
continue;
} catch (Exception e) {
// might happen when calling on a network interface that does not exists
continue;
}
address = getFirstNonLoopbackAddress(intf, ip_version);
if (address != null) {
return address;
}
}
return null;
}
private static List<NetworkInterface> getInterfaces() throws SocketException {
Enumeration intfs = NetworkInterface.getNetworkInterfaces();
List<NetworkInterface> intfsList = Lists.newArrayList();
while (intfs.hasMoreElements()) {
intfsList.add((NetworkInterface) intfs.nextElement());
}
sortInterfaces(intfsList);
return intfsList;
}
private static void sortInterfaces(List<NetworkInterface> intfsList) {
// order by index, assuming first ones are more interesting
CollectionUtil.timSort(intfsList, new Comparator<NetworkInterface>() {
@Override
public int compare(NetworkInterface o1, NetworkInterface o2) {
return Integer.compare (o1.getIndex(), o2.getIndex());
}
});
}
/**
* Returns the first non-loopback address on the given interface on the current host.
*
* @param intf the interface to be checked
* @param ipVersion Constraint on IP version of address to be returned, 4 or 6
*/
public static InetAddress getFirstNonLoopbackAddress(NetworkInterface intf, StackType ipVersion) throws SocketException {
if (intf == null)
throw new IllegalArgumentException("Network interface pointer is null");
for (Enumeration addresses = intf.getInetAddresses(); addresses.hasMoreElements(); ) {
InetAddress address = (InetAddress) addresses.nextElement();
if (!address.isLoopbackAddress()) {
if ((address instanceof Inet4Address && ipVersion == StackType.IPv4) ||
(address instanceof Inet6Address && ipVersion == StackType.IPv6))
return address;
}
}
return null;
}
/**
* Returns the first address with the proper ipVersion on the given interface on the current host.
*
* @param intf the interface to be checked
* @param ipVersion Constraint on IP version of address to be returned, 4 or 6
*/
public static InetAddress getFirstAddress(NetworkInterface intf, StackType ipVersion) throws SocketException {
if (intf == null)
throw new IllegalArgumentException("Network interface pointer is null");
for (Enumeration addresses = intf.getInetAddresses(); addresses.hasMoreElements(); ) {
InetAddress address = (InetAddress) addresses.nextElement();
if ((address instanceof Inet4Address && ipVersion == StackType.IPv4) ||
(address instanceof Inet6Address && ipVersion == StackType.IPv6))
return address;
}
return null;
}
/**
* A function to check if an interface supports an IP version (i.e has addresses
* defined for that IP version).
*
* @param intf
* @return
*/
public static boolean interfaceHasIPAddresses(NetworkInterface intf, StackType ipVersion) throws SocketException, UnknownHostException {
boolean supportsVersion = false;
if (intf != null) {
// get all the InetAddresses defined on the interface
Enumeration addresses = intf.getInetAddresses();
while (addresses != null && addresses.hasMoreElements()) {
// get the next InetAddress for the current interface
InetAddress address = (InetAddress) addresses.nextElement();
// check if we find an address of correct version
if ((address instanceof Inet4Address && (ipVersion == StackType.IPv4)) ||
(address instanceof Inet6Address && (ipVersion == StackType.IPv6))) {
supportsVersion = true;
break;
}
}
} else {
throw new UnknownHostException("network interface not found");
}
return supportsVersion;
}
/**
* Tries to determine the type of IP stack from the available interfaces and their addresses and from the
* system properties (java.net.preferIPv4Stack and java.net.preferIPv6Addresses)
*
* @return StackType.IPv4 for an IPv4 only stack, StackYTypeIPv6 for an IPv6 only stack, and StackType.Unknown
* if the type cannot be detected
*/
public static StackType getIpStackType() {
boolean isIPv4StackAvailable = isStackAvailable(true);
boolean isIPv6StackAvailable = isStackAvailable(false);
// if only IPv4 stack available
if (isIPv4StackAvailable && !isIPv6StackAvailable) {
return StackType.IPv4;
}
// if only IPv6 stack available
else if (isIPv6StackAvailable && !isIPv4StackAvailable) {
return StackType.IPv6;
}
// if dual stack
else if (isIPv4StackAvailable && isIPv6StackAvailable) {
// get the System property which records user preference for a stack on a dual stack machine
if (Boolean.getBoolean(IPv4_SETTING)) // has preference over java.net.preferIPv6Addresses
return StackType.IPv4;
if (Boolean.getBoolean(IPv6_SETTING))
return StackType.IPv6;
return StackType.IPv6;
}
return StackType.Unknown;
}
public static boolean isStackAvailable(boolean ipv4) {
Collection<InetAddress> allAddrs = getAllAvailableAddresses();
for (InetAddress addr : allAddrs)
if (ipv4 && addr instanceof Inet4Address || (!ipv4 && addr instanceof Inet6Address))
return true;
return false;
}
/**
* Returns all the available interfaces, including first level sub interfaces.
*/
public static List<NetworkInterface> getAllAvailableInterfaces() throws SocketException {
List<NetworkInterface> allInterfaces = new ArrayList<>();
for (Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces(); interfaces.hasMoreElements(); ) {
NetworkInterface intf = interfaces.nextElement();
allInterfaces.add(intf);
Enumeration<NetworkInterface> subInterfaces = intf.getSubInterfaces();
if (subInterfaces != null && subInterfaces.hasMoreElements()) {
while (subInterfaces.hasMoreElements()) {
allInterfaces.add(subInterfaces.nextElement());
}
}
}
sortInterfaces(allInterfaces);
return allInterfaces;
}
public static Collection<InetAddress> getAllAvailableAddresses() {
// we want consistent order here.
final Set<InetAddress> retval = new TreeSet<>(new Comparator<InetAddress>() {
BytesRef left = new BytesRef();
BytesRef right = new BytesRef();
@Override
public int compare(InetAddress o1, InetAddress o2) {
return set(left, o1).compareTo(set(right, o1));
}
private BytesRef set(BytesRef ref, InetAddress addr) {
ref.bytes = addr.getAddress();
ref.offset = 0;
ref.length = ref.bytes.length;
return ref;
}
});
try { try {
for (NetworkInterface intf : getInterfaces()) { return InetAddress.getLocalHost();
Enumeration<InetAddress> addrs = intf.getInetAddresses(); } catch (UnknownHostException e) {
while (addrs.hasMoreElements()) logger.warn("failed to resolve local host, fallback to loopback", e);
retval.add(addrs.nextElement()); return InetAddress.getLoopbackAddress();
}
} catch (SocketException e) {
logger.warn("Failed to derive all available interfaces", e);
} }
return retval;
} }
/** Returns addresses for all loopback interfaces that are up. */
public static InetAddress[] getLoopbackAddresses() throws SocketException {
List<InetAddress> list = new ArrayList<>();
for (NetworkInterface intf : getInterfaces()) {
if (intf.isLoopback() && intf.isUp()) {
list.addAll(Collections.list(intf.getInetAddresses()));
}
}
if (list.isEmpty()) {
throw new IllegalArgumentException("No up-and-running loopback interfaces found, got " + getInterfaces());
}
sortAddresses(list);
return list.toArray(new InetAddress[list.size()]);
}
private NetworkUtils() { /** Returns addresses for the first non-loopback interface that is up. */
public static InetAddress[] getFirstNonLoopbackAddresses() throws SocketException {
List<InetAddress> list = new ArrayList<>();
for (NetworkInterface intf : getInterfaces()) {
if (intf.isLoopback() == false && intf.isUp()) {
list.addAll(Collections.list(intf.getInetAddresses()));
break;
}
}
if (list.isEmpty()) {
throw new IllegalArgumentException("No up-and-running non-loopback interfaces found, got " + getInterfaces());
}
sortAddresses(list);
return list.toArray(new InetAddress[list.size()]);
}
/** Returns addresses for the given interface (it must be marked up) */
public static InetAddress[] getAddressesForInterface(String name) throws SocketException {
NetworkInterface intf = NetworkInterface.getByName(name);
if (intf == null) {
throw new IllegalArgumentException("No interface named '" + name + "' found, got " + getInterfaces());
}
if (!intf.isUp()) {
throw new IllegalArgumentException("Interface '" + name + "' is not up and running");
}
List<InetAddress> list = Collections.list(intf.getInetAddresses());
if (list.isEmpty()) {
throw new IllegalArgumentException("Interface '" + name + "' has no internet addresses");
}
sortAddresses(list);
return list.toArray(new InetAddress[list.size()]);
}
/** Returns addresses for the given host, sorted by order of preference */
public static InetAddress[] getAllByName(String host) throws UnknownHostException {
InetAddress addresses[] = InetAddress.getAllByName(host);
sortAddresses(Arrays.asList(addresses));
return addresses;
}
/** Returns only the IPV4 addresses in {@code addresses} */
public static InetAddress[] filterIPV4(InetAddress addresses[]) {
List<InetAddress> list = new ArrayList<>();
for (InetAddress address : addresses) {
if (address instanceof Inet4Address) {
list.add(address);
}
}
if (list.isEmpty()) {
throw new IllegalArgumentException("No ipv4 addresses found in " + Arrays.toString(addresses));
}
return list.toArray(new InetAddress[list.size()]);
}
/** Returns only the IPV6 addresses in {@code addresses} */
public static InetAddress[] filterIPV6(InetAddress addresses[]) {
List<InetAddress> list = new ArrayList<>();
for (InetAddress address : addresses) {
if (address instanceof Inet6Address) {
list.add(address);
}
}
if (list.isEmpty()) {
throw new IllegalArgumentException("No ipv6 addresses found in " + Arrays.toString(addresses));
}
return list.toArray(new InetAddress[list.size()]);
} }
} }

4
core/src/main/java/org/elasticsearch/discovery/zen/ping/multicast/MulticastZenPing.java
View File

@ -131,7 +131,9 @@ public class MulticastZenPing extends AbstractLifecycleComponent<ZenPing> implem
boolean deferToInterface = settings.getAsBoolean("discovery.zen.ping.multicast.defer_group_to_set_interface", Constants.MAC_OS_X); boolean deferToInterface = settings.getAsBoolean("discovery.zen.ping.multicast.defer_group_to_set_interface", Constants.MAC_OS_X);
multicastChannel = MulticastChannel.getChannel(nodeName(), shared, multicastChannel = MulticastChannel.getChannel(nodeName(), shared,
new MulticastChannel.Config(port, group, bufferSize, ttl, new MulticastChannel.Config(port, group, bufferSize, ttl,
networkService.resolvePublishHostAddress(address), // don't use publish address, the use case for that is e.g. a firewall or proxy and
// may not even be bound to an interface on this machine! use the first bound address.
networkService.resolveBindHostAddress(address)[0],
deferToInterface), deferToInterface),
new Receiver()); new Receiver());
} catch (Throwable t) { } catch (Throwable t) {

67
core/src/main/java/org/elasticsearch/http/netty/NettyHttpServerTransport.java
View File

@ -51,6 +51,10 @@ import org.jboss.netty.handler.timeout.ReadTimeoutException;
import java.io.IOException; import java.io.IOException;
import java.net.InetAddress; import java.net.InetAddress;
import java.net.InetSocketAddress; import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.Executors; import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicReference; import java.util.concurrent.atomic.AtomicReference;
@ -128,7 +132,7 @@ public class NettyHttpServerTransport extends AbstractLifecycleComponent<HttpSer
protected volatile BoundTransportAddress boundAddress; protected volatile BoundTransportAddress boundAddress;
protected volatile Channel serverChannel; protected volatile List<Channel> serverChannels = new ArrayList<>();
protected OpenChannelsHandler serverOpenChannels; protected OpenChannelsHandler serverOpenChannels;
@ -243,33 +247,18 @@ public class NettyHttpServerTransport extends AbstractLifecycleComponent<HttpSer
serverBootstrap.setOption("child.reuseAddress", reuseAddress); serverBootstrap.setOption("child.reuseAddress", reuseAddress);
// Bind and start to accept incoming connections. // Bind and start to accept incoming connections.
InetAddress hostAddressX; InetAddress hostAddresses[];
try { try {
hostAddressX = networkService.resolveBindHostAddress(bindHost); hostAddresses = networkService.resolveBindHostAddress(bindHost);
} catch (IOException e) { } catch (IOException e) {
throw new BindHttpException("Failed to resolve host [" + bindHost + "]", e); throw new BindHttpException("Failed to resolve host [" + bindHost + "]", e);
} }
final InetAddress hostAddress = hostAddressX;
PortsRange portsRange = new PortsRange(port); for (InetAddress address : hostAddresses) {
final AtomicReference<Exception> lastException = new AtomicReference<>(); bindAddress(address);
boolean success = portsRange.iterate(new PortsRange.PortCallback() {
@Override
public boolean onPortNumber(int portNumber) {
try {
serverChannel = serverBootstrap.bind(new InetSocketAddress(hostAddress, portNumber));
} catch (Exception e) {
lastException.set(e);
return false;
}
return true;
}
});
if (!success) {
throw new BindHttpException("Failed to bind to [" + port + "]", lastException.get());
} }
InetSocketAddress boundAddress = (InetSocketAddress) serverChannel.getLocalAddress(); InetSocketAddress boundAddress = (InetSocketAddress) serverChannels.get(0).getLocalAddress();
InetSocketAddress publishAddress; InetSocketAddress publishAddress;
if (0 == publishPort) { if (0 == publishPort) {
publishPort = boundAddress.getPort(); publishPort = boundAddress.getPort();
@ -282,11 +271,41 @@ public class NettyHttpServerTransport extends AbstractLifecycleComponent<HttpSer
this.boundAddress = new BoundTransportAddress(new InetSocketTransportAddress(boundAddress), new InetSocketTransportAddress(publishAddress)); this.boundAddress = new BoundTransportAddress(new InetSocketTransportAddress(boundAddress), new InetSocketTransportAddress(publishAddress));
} }
private void bindAddress(final InetAddress hostAddress) {
PortsRange portsRange = new PortsRange(port);
final AtomicReference<Exception> lastException = new AtomicReference<>();
final AtomicReference<SocketAddress> boundSocket = new AtomicReference<>();
boolean success = portsRange.iterate(new PortsRange.PortCallback() {
@Override
public boolean onPortNumber(int portNumber) {
try {
synchronized (serverChannels) {
Channel channel = serverBootstrap.bind(new InetSocketAddress(hostAddress, portNumber));
serverChannels.add(channel);
boundSocket.set(channel.getLocalAddress());
}
} catch (Exception e) {
lastException.set(e);
return false;
}
return true;
}
});
if (!success) {
throw new BindHttpException("Failed to bind to [" + port + "]", lastException.get());
}
logger.info("Bound http to address [{}]", boundSocket.get());
}
@Override @Override
protected void doStop() { protected void doStop() {
if (serverChannel != null) { synchronized (serverChannels) {
serverChannel.close().awaitUninterruptibly(); if (serverChannels != null) {
serverChannel = null; for (Channel channel : serverChannels) {
channel.close().awaitUninterruptibly();
}
serverChannels = null;
}
} }
if (serverOpenChannels != null) { if (serverOpenChannels != null) {

52
core/src/main/java/org/elasticsearch/transport/netty/NettyTransport.java
View File

@ -146,8 +146,8 @@ public class NettyTransport extends AbstractLifecycleComponent<Transport> implem
// node id to actual channel // node id to actual channel
protected final ConcurrentMap<DiscoveryNode, NodeChannels> connectedNodes = newConcurrentMap(); protected final ConcurrentMap<DiscoveryNode, NodeChannels> connectedNodes = newConcurrentMap();
protected final Map<String, ServerBootstrap> serverBootstraps = newConcurrentMap(); protected final Map<String, ServerBootstrap> serverBootstraps = newConcurrentMap();
protected final Map<String, Channel> serverChannels = newConcurrentMap(); protected final Map<String, List<Channel>> serverChannels = newConcurrentMap();
protected final Map<String, BoundTransportAddress> profileBoundAddresses = newConcurrentMap(); protected final ConcurrentMap<String, BoundTransportAddress> profileBoundAddresses = newConcurrentMap();
protected volatile TransportServiceAdapter transportServiceAdapter; protected volatile TransportServiceAdapter transportServiceAdapter;
protected volatile BoundTransportAddress boundAddress; protected volatile BoundTransportAddress boundAddress;
protected final KeyedLock<String> connectionLock = new KeyedLock<>(); protected final KeyedLock<String> connectionLock = new KeyedLock<>();
@ -286,7 +286,7 @@ public class NettyTransport extends AbstractLifecycleComponent<Transport> implem
bindServerBootstrap(name, mergedSettings); bindServerBootstrap(name, mergedSettings);
} }
InetSocketAddress boundAddress = (InetSocketAddress) serverChannels.get(DEFAULT_PROFILE).getLocalAddress(); InetSocketAddress boundAddress = (InetSocketAddress) serverChannels.get(DEFAULT_PROFILE).get(0).getLocalAddress();
int publishPort = settings.getAsInt("transport.netty.publish_port", settings.getAsInt("transport.publish_port", boundAddress.getPort())); int publishPort = settings.getAsInt("transport.netty.publish_port", settings.getAsInt("transport.publish_port", boundAddress.getPort()));
String publishHost = settings.get("transport.netty.publish_host", settings.get("transport.publish_host", settings.get("transport.host"))); String publishHost = settings.get("transport.netty.publish_host", settings.get("transport.publish_host", settings.get("transport.host")));
InetSocketAddress publishAddress = createPublishAddress(publishHost, publishPort); InetSocketAddress publishAddress = createPublishAddress(publishHost, publishPort);
@ -397,23 +397,38 @@ public class NettyTransport extends AbstractLifecycleComponent<Transport> implem
private void bindServerBootstrap(final String name, final Settings settings) { private void bindServerBootstrap(final String name, final Settings settings) {
// Bind and start to accept incoming connections. // Bind and start to accept incoming connections.
InetAddress hostAddressX; InetAddress hostAddresses[];
String bindHost = settings.get("bind_host"); String bindHost = settings.get("bind_host");
try { try {
hostAddressX = networkService.resolveBindHostAddress(bindHost); hostAddresses = networkService.resolveBindHostAddress(bindHost);
} catch (IOException e) { } catch (IOException e) {
throw new BindTransportException("Failed to resolve host [" + bindHost + "]", e); throw new BindTransportException("Failed to resolve host [" + bindHost + "]", e);
} }
final InetAddress hostAddress = hostAddressX; for (InetAddress hostAddress : hostAddresses) {
bindServerBootstrap(name, hostAddress, settings);
}
}
private void bindServerBootstrap(final String name, final InetAddress hostAddress, Settings settings) {
String port = settings.get("port"); String port = settings.get("port");
PortsRange portsRange = new PortsRange(port); PortsRange portsRange = new PortsRange(port);
final AtomicReference<Exception> lastException = new AtomicReference<>(); final AtomicReference<Exception> lastException = new AtomicReference<>();
final AtomicReference<SocketAddress> boundSocket = new AtomicReference<>();
boolean success = portsRange.iterate(new PortsRange.PortCallback() { boolean success = portsRange.iterate(new PortsRange.PortCallback() {
@Override @Override
public boolean onPortNumber(int portNumber) { public boolean onPortNumber(int portNumber) {
try { try {
serverChannels.put(name, serverBootstraps.get(name).bind(new InetSocketAddress(hostAddress, portNumber))); Channel channel = serverBootstraps.get(name).bind(new InetSocketAddress(hostAddress, portNumber));
synchronized (serverChannels) {
List<Channel> list = serverChannels.get(name);
if (list == null) {
list = new ArrayList<>();
serverChannels.put(name, list);
}
list.add(channel);
boundSocket.set(channel.getLocalAddress());
}
} catch (Exception e) { } catch (Exception e) {
lastException.set(e); lastException.set(e);
return false; return false;
@ -426,14 +441,15 @@ public class NettyTransport extends AbstractLifecycleComponent<Transport> implem
} }
if (!DEFAULT_PROFILE.equals(name)) { if (!DEFAULT_PROFILE.equals(name)) {
InetSocketAddress boundAddress = (InetSocketAddress) serverChannels.get(name).getLocalAddress(); InetSocketAddress boundAddress = (InetSocketAddress) boundSocket.get();
int publishPort = settings.getAsInt("publish_port", boundAddress.getPort()); int publishPort = settings.getAsInt("publish_port", boundAddress.getPort());
String publishHost = settings.get("publish_host", boundAddress.getHostString()); String publishHost = settings.get("publish_host", boundAddress.getHostString());
InetSocketAddress publishAddress = createPublishAddress(publishHost, publishPort); InetSocketAddress publishAddress = createPublishAddress(publishHost, publishPort);
profileBoundAddresses.put(name, new BoundTransportAddress(new InetSocketTransportAddress(boundAddress), new InetSocketTransportAddress(publishAddress))); // TODO: support real multihoming with publishing. Today we use putIfAbsent so only the prioritized address is published
profileBoundAddresses.putIfAbsent(name, new BoundTransportAddress(new InetSocketTransportAddress(boundAddress), new InetSocketTransportAddress(publishAddress)));
} }
logger.debug("Bound profile [{}] to address [{}]", name, serverChannels.get(name).getLocalAddress()); logger.info("Bound profile [{}] to address [{}]", name, boundSocket.get());
} }
private void createServerBootstrap(String name, Settings settings) { private void createServerBootstrap(String name, Settings settings) {
@ -500,15 +516,17 @@ public class NettyTransport extends AbstractLifecycleComponent<Transport> implem
nodeChannels.close(); nodeChannels.close();
} }
Iterator<Map.Entry<String, Channel>> serverChannelIterator = serverChannels.entrySet().iterator(); Iterator<Map.Entry<String, List<Channel>>> serverChannelIterator = serverChannels.entrySet().iterator();
while (serverChannelIterator.hasNext()) { while (serverChannelIterator.hasNext()) {
Map.Entry<String, Channel> serverChannelEntry = serverChannelIterator.next(); Map.Entry<String, List<Channel>> serverChannelEntry = serverChannelIterator.next();
String name = serverChannelEntry.getKey(); String name = serverChannelEntry.getKey();
Channel serverChannel = serverChannelEntry.getValue(); List<Channel> serverChannels = serverChannelEntry.getValue();
try { for (Channel serverChannel : serverChannels) {
serverChannel.close().awaitUninterruptibly(); try {
} catch (Throwable t) { serverChannel.close().awaitUninterruptibly();
logger.debug("Error closing serverChannel for profile [{}]", t, name); } catch (Throwable t) {
logger.debug("Error closing serverChannel for profile [{}]", t, name);
}
} }
serverChannelIterator.remove(); serverChannelIterator.remove();
} }

77
core/src/test/java/org/elasticsearch/common/network/NetworkUtilsTests.java Normal file
View File

@ -0,0 +1,77 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.common.network;
import org.elasticsearch.test.ESTestCase;
import java.net.InetAddress;
/**
* Tests for network utils. Please avoid using any methods that cause DNS lookups!
*/
public class NetworkUtilsTests extends ESTestCase {
/**
* test sort key order respects PREFER_IPV4
*/
public void testSortKey() throws Exception {
InetAddress localhostv4 = InetAddress.getByName("127.0.0.1");
InetAddress localhostv6 = InetAddress.getByName("::1");
assertTrue(NetworkUtils.sortKey(localhostv4, true) < NetworkUtils.sortKey(localhostv6, true));
assertTrue(NetworkUtils.sortKey(localhostv6, false) < NetworkUtils.sortKey(localhostv4, false));
}
/**
* test ordinary addresses sort before private addresses
*/
public void testSortKeySiteLocal() throws Exception {
InetAddress siteLocal = InetAddress.getByName("172.16.0.1");
assert siteLocal.isSiteLocalAddress();
InetAddress ordinary = InetAddress.getByName("192.192.192.192");
assertTrue(NetworkUtils.sortKey(ordinary, true) < NetworkUtils.sortKey(siteLocal, true));
assertTrue(NetworkUtils.sortKey(ordinary, false) < NetworkUtils.sortKey(siteLocal, false));
InetAddress siteLocal6 = InetAddress.getByName("fec0::1");
assert siteLocal6.isSiteLocalAddress();
InetAddress ordinary6 = InetAddress.getByName("fddd::1");
assertTrue(NetworkUtils.sortKey(ordinary6, true) < NetworkUtils.sortKey(siteLocal6, true));
assertTrue(NetworkUtils.sortKey(ordinary6, false) < NetworkUtils.sortKey(siteLocal6, false));
}
/**
* test private addresses sort before link local addresses
*/
public void testSortKeyLinkLocal() throws Exception {
InetAddress linkLocal = InetAddress.getByName("fe80::1");
assert linkLocal.isLinkLocalAddress();
InetAddress ordinary = InetAddress.getByName("fddd::1");
assertTrue(NetworkUtils.sortKey(ordinary, true) < NetworkUtils.sortKey(linkLocal, true));
assertTrue(NetworkUtils.sortKey(ordinary, false) < NetworkUtils.sortKey(linkLocal, false));
}
/**
* Test filtering out ipv4/ipv6 addresses
*/
public void testFilter() throws Exception {
InetAddress addresses[] = { InetAddress.getByName("::1"), InetAddress.getByName("127.0.0.1") };
assertArrayEquals(new InetAddress[] { InetAddress.getByName("127.0.0.1") }, NetworkUtils.filterIPV4(addresses));
assertArrayEquals(new InetAddress[] { InetAddress.getByName("::1") }, NetworkUtils.filterIPV6(addresses));
}
}

2
core/src/test/java/org/elasticsearch/test/InternalTestCluster.java
View File

@ -504,7 +504,7 @@ public final class InternalTestCluster extends TestCluster {
public static String clusterName(String prefix, long clusterSeed) { public static String clusterName(String prefix, long clusterSeed) {
StringBuilder builder = new StringBuilder(prefix); StringBuilder builder = new StringBuilder(prefix);
final int childVM = RandomizedTest.systemPropertyAsInt(SysGlobals.CHILDVM_SYSPROP_JVM_ID, 0); final int childVM = RandomizedTest.systemPropertyAsInt(SysGlobals.CHILDVM_SYSPROP_JVM_ID, 0);
builder.append('-').append(NetworkUtils.getLocalHostName("__default_host__")); builder.append('-').append(NetworkUtils.getLocalHost().getHostName());
builder.append("-CHILD_VM=[").append(childVM).append(']'); builder.append("-CHILD_VM=[").append(childVM).append(']');
builder.append("-CLUSTER_SEED=[").append(clusterSeed).append(']'); builder.append("-CLUSTER_SEED=[").append(clusterSeed).append(']');
// if multiple maven task run on a single host we better have an identifier that doesn't rely on input params // if multiple maven task run on a single host we better have an identifier that doesn't rely on input params

23
core/src/test/java/org/elasticsearch/transport/netty/NettyTransportMultiPortTests.java
View File

@ -135,29 +135,6 @@ public class NettyTransportMultiPortTests extends ESTestCase {
} }
} }
@Test
public void testThatBindingOnDifferentHostsWorks() throws Exception {
int[] ports = getRandomPorts(2);
InetAddress firstNonLoopbackAddress = NetworkUtils.getFirstNonLoopbackAddress(NetworkUtils.StackType.IPv4);
assumeTrue("No IP-v4 non-loopback address available - are you on a plane?", firstNonLoopbackAddress != null);
Settings settings = settingsBuilder()
.put("network.host", "127.0.0.1")
.put("transport.tcp.port", ports[0])
.put("transport.profiles.default.bind_host", "127.0.0.1")
.put("transport.profiles.client1.bind_host", firstNonLoopbackAddress.getHostAddress())
.put("transport.profiles.client1.port", ports[1])
.build();
ThreadPool threadPool = new ThreadPool("tst");
try (NettyTransport ignored = startNettyTransport(settings, threadPool)) {
assertPortIsBound("127.0.0.1", ports[0]);
assertPortIsBound(firstNonLoopbackAddress.getHostAddress(), ports[1]);
assertConnectionRefused(ports[1]);
} finally {
terminate(threadPool);
}
}
@Test @Test
public void testThatProfileWithoutValidNameIsIgnored() throws Exception { public void testThatProfileWithoutValidNameIsIgnored() throws Exception {
int[] ports = getRandomPorts(3); int[] ports = getRandomPorts(3);

4
dev-tools/src/main/resources/ant/integration-tests.xml
View File

@ -124,7 +124,7 @@
<waitfor maxwait="30" maxwaitunit="second" <waitfor maxwait="30" maxwaitunit="second"
checkevery="500" checkeveryunit="millisecond" checkevery="500" checkeveryunit="millisecond"
timeoutproperty="@{timeoutproperty}"> timeoutproperty="@{timeoutproperty}">
<http url="http://127.0.0.1:@{port}"/> <http url="http://localhost:@{port}"/>
</waitfor> </waitfor>
</sequential> </sequential>
</macrodef> </macrodef>
@ -138,7 +138,7 @@
<waitfor maxwait="30" maxwaitunit="second" <waitfor maxwait="30" maxwaitunit="second"
checkevery="500" checkeveryunit="millisecond" checkevery="500" checkeveryunit="millisecond"
timeoutproperty="@{timeoutproperty}"> timeoutproperty="@{timeoutproperty}">
<http url="http://127.0.0.1:@{port}/_cluster/health?wait_for_nodes=2"/> <http url="http://localhost:@{port}/_cluster/health?wait_for_nodes=2"/>
</waitfor> </waitfor>
</sequential> </sequential>
</macrodef> </macrodef>

2
distribution/pom.xml
View File

@ -153,7 +153,7 @@
<parallelism>1</parallelism> <parallelism>1</parallelism>
<systemProperties> <systemProperties>
<!-- use external cluster --> <!-- use external cluster -->
<tests.cluster>127.0.0.1:${integ.transport.port}</tests.cluster> <tests.cluster>localhost:${integ.transport.port}</tests.cluster>
</systemProperties> </systemProperties>
</configuration> </configuration>
</execution> </execution>

2
docs/reference/modules/discovery/zen.asciidoc
View File

@ -38,7 +38,7 @@ respond to. It provides the following settings with the
|`ttl` |The ttl of the multicast message. Defaults to `3`. |`ttl` |The ttl of the multicast message. Defaults to `3`.
|`address` |The address to bind to, defaults to `null` which means it |`address` |The address to bind to, defaults to `null` which means it
will bind to all available network interfaces. will bind `network.bind_host`
|`enabled` |Whether multicast ping discovery is enabled. Defaults to `true`. |`enabled` |Whether multicast ping discovery is enabled. Defaults to `true`.
|======================================================================= |=======================================================================

28
docs/reference/modules/network.asciidoc
View File

@ -9,13 +9,15 @@ network settings allows to set common settings that will be shared among
all network based modules (unless explicitly overridden in each module). all network based modules (unless explicitly overridden in each module).
The `network.bind_host` setting allows to control the host different network The `network.bind_host` setting allows to control the host different network
components will bind on. By default, the bind host will be `anyLoopbackAddress` components will bind on. By default, the bind host will be `_local_`
(typically `127.0.0.1` or `::1`). (loopback addresses such as `127.0.0.1`, `::1`).
The `network.publish_host` setting allows to control the host the node will The `network.publish_host` setting allows to control the host the node will
publish itself within the cluster so other nodes will be able to connect to it. publish itself within the cluster so other nodes will be able to connect to it.
Of course, this can't be the `anyLocalAddress`, and by default, it will be the Currently an elasticsearch node may be bound to multiple addresses, but only
first loopback address (if possible), or the local address. publishes one. If not specified, this defaults to the "best" address from
`network.bind_host`. By default, IPv4 addresses are preferred to IPv6, and
ordinary addresses are preferred to site-local or link-local addresses.
The `network.host` setting is a simple setting to automatically set both The `network.host` setting is a simple setting to automatically set both
`network.bind_host` and `network.publish_host` to the same host value. `network.bind_host` and `network.publish_host` to the same host value.
@ -27,21 +29,25 @@ in the following table:
[cols="<,<",options="header",] [cols="<,<",options="header",]
|======================================================================= |=======================================================================
|Logical Host Setting Value |Description |Logical Host Setting Value |Description
|`_local_` |Will be resolved to the local ip address. |`_local_` |Will be resolved to loopback addresses
|`_non_loopback_` |The first non loopback address. |`_local:ipv4_` |Will be resolved to loopback IPv4 addresses
|`_non_loopback:ipv4_` |The first non loopback IPv4 address. |`_local:ipv6_` |Will be resolved to loopback IPv6 addresses
|`_non_loopback:ipv6_` |The first non loopback IPv6 address. |`_non_loopback_` |Addresses of the first non loopback interface
|`_[networkInterface]_` |Resolves to the ip address of the provided |`_non_loopback:ipv4_` |IPv4 addresses of the first non loopback interface
|`_non_loopback:ipv6_` |IPv6 addresses of the first non loopback interface
|`_[networkInterface]_` |Resolves to the addresses of the provided
network interface. For example `_en0_`. network interface. For example `_en0_`.
|`_[networkInterface]:ipv4_` |Resolves to the ipv4 address of the |`_[networkInterface]:ipv4_` |Resolves to the ipv4 addresses of the
provided network interface. For example `_en0:ipv4_`. provided network interface. For example `_en0:ipv4_`.
|`_[networkInterface]:ipv6_` |Resolves to the ipv6 address of the |`_[networkInterface]:ipv6_` |Resolves to the ipv6 addresses of the
provided network interface. For example `_en0:ipv6_`. provided network interface. For example `_en0:ipv6_`.
|======================================================================= |=======================================================================

9
plugins/cloud-aws/src/main/java/org/elasticsearch/cloud/aws/network/Ec2NameResolver.java
View File

@ -93,7 +93,7 @@ public class Ec2NameResolver extends AbstractComponent implements CustomNameReso
* @throws IOException if ec2 meta-data cannot be obtained. * @throws IOException if ec2 meta-data cannot be obtained.
* @see CustomNameResolver#resolveIfPossible(String) * @see CustomNameResolver#resolveIfPossible(String)
*/ */
public InetAddress resolve(Ec2HostnameType type, boolean warnOnFailure) { public InetAddress[] resolve(Ec2HostnameType type, boolean warnOnFailure) {
URLConnection urlConnection = null; URLConnection urlConnection = null;
InputStream in = null; InputStream in = null;
try { try {
@ -109,7 +109,8 @@ public class Ec2NameResolver extends AbstractComponent implements CustomNameReso
logger.error("no ec2 metadata returned from {}", url); logger.error("no ec2 metadata returned from {}", url);
return null; return null;
} }
return InetAddress.getByName(metadataResult); // only one address: because we explicitly ask for only one via the Ec2HostnameType
return new InetAddress[] { InetAddress.getByName(metadataResult) };
} catch (IOException e) { } catch (IOException e) {
if (warnOnFailure) { if (warnOnFailure) {
logger.warn("failed to get metadata for [" + type.configName + "]: " + ExceptionsHelper.detailedMessage(e)); logger.warn("failed to get metadata for [" + type.configName + "]: " + ExceptionsHelper.detailedMessage(e));
@ -123,13 +124,13 @@ public class Ec2NameResolver extends AbstractComponent implements CustomNameReso
} }
@Override @Override
public InetAddress resolveDefault() { public InetAddress[] resolveDefault() {
return null; // using this, one has to explicitly specify _ec2_ in network setting return null; // using this, one has to explicitly specify _ec2_ in network setting
// return resolve(Ec2HostnameType.DEFAULT, false); // return resolve(Ec2HostnameType.DEFAULT, false);
} }
@Override @Override
public InetAddress resolveIfPossible(String value) { public InetAddress[] resolveIfPossible(String value) {
for (Ec2HostnameType type : Ec2HostnameType.values()) { for (Ec2HostnameType type : Ec2HostnameType.values()) {
if (type.configName.equals(value)) { if (type.configName.equals(value)) {
return resolve(type, true); return resolve(type, true);

2
plugins/pom.xml
View File

@ -414,7 +414,7 @@
<parallelism>1</parallelism> <parallelism>1</parallelism>
<systemProperties> <systemProperties>
<!-- use external cluster --> <!-- use external cluster -->
<tests.cluster>127.0.0.1:${integ.transport.port}</tests.cluster> <tests.cluster>localhost:${integ.transport.port}</tests.cluster>
</systemProperties> </systemProperties>
</configuration> </configuration>
</execution> </execution>