honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Properly document keepalive and other tcp options (#60216) 

Keepalive options are not well-documented (only in transport section, although also available at http and network level).

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
This commit is contained in:
Yannick Welsch 2020-07-28 10:14:56 +02:00
parent ffe114b890
commit a55c869aab
7 changed files with 148 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
docs/reference/migration/migrate_7_10.asciidoc
View File

@ -29,4 +29,15 @@ checked this limit when `docvalue_fields` appeared in the top-level search
request body. Now we also enforce the limit when loading doc value fields in
an `inner_hits` section or in a `top_hits` aggregation.
[discrete]
[[breaking_710_networking_changes]]
=== Networking Changes
[discrete]
==== *.tcp.keep_idle and *.tcp.keep_interval now bounded
The settings `{network,transport,http}.tcp.keep_idle` and
`{network,transport,http}.tcp.keep_interval` are now limited to a maximum
value of 300, which amounts to 5 minutes. This helps ensure that, if
explicitly configured, only reasonable values are set.
//end::notable-breaking-changes[]

38
docs/reference/modules/http.asciidoc
View File

@ -127,6 +127,44 @@ The maximum number of warning headers in client HTTP responses. Defaults to `unb
`http.max_warning_header_size`::
The maximum total size of warning headers in client HTTP responses. Defaults to `unbounded`.
`http.tcp.no_delay`::
Enable or disable the https://en.wikipedia.org/wiki/Nagle%27s_algorithm[TCP no delay]
setting. Defaults to `network.tcp.no_delay`.
`http.tcp.keep_alive`::
Configures the `SO_KEEPALIVE` option for this socket, which
determines whether it sends TCP keepalive probes.
Defaults to `network.tcp.keep_alive`.
`http.tcp.keep_idle`:: Configures the `TCP_KEEPIDLE` option for this socket, which
determines the time in seconds that a connection must be idle before
starting to send TCP keepalive probes. Defaults to `network.tcp.keep_idle`, which
uses the system default. This value cannot exceed `300` seconds. Only applicable on
Linux and macOS, and requires Java 11 or newer.
`http.tcp.keep_interval`:: Configures the `TCP_KEEPINTVL` option for this socket,
which determines the time in seconds between sending TCP keepalive probes.
Defaults to `network.tcp.keep_interval`, which uses the system default.
This value cannot exceed `300` seconds. Only applicable on Linux and macOS, and requires
Java 11 or newer.
`http.tcp.keep_count`:: Configures the `TCP_KEEPCNT` option for this socket, which
determines the number of unacknowledged TCP keepalive probes that may be
sent on a connection before it is dropped. Defaults to `network.tcp.keep_count`,
which uses the system default. Only applicable on Linux and macOS, and
requires Java 11 or newer.
`http.tcp.reuse_address`::
Should an address be reused or not. Defaults to `network.tcp.reuse_address`.
`http.tcp.send_buffer_size`::
The size of the TCP send buffer (specified with <<size-units,size units>>).
Defaults to `network.tcp.send_buffer_size`.
`http.tcp.receive_buffer_size`::
The size of the TCP receive buffer (specified with <<size-units,size units>>).
Defaults to `network.tcp.receive_buffer_size`.
[http-rest-request-tracer]
==== REST request tracer

21
docs/reference/modules/network.asciidoc
View File

@ -130,8 +130,25 @@ Enable or disable the https://en.wikipedia.org/wiki/Nagle%27s_algorithm[TCP no d
setting. Defaults to `true`.
`network.tcp.keep_alive`::
Enable or disable https://en.wikipedia.org/wiki/Keepalive[TCP keep alive].
Defaults to `true`.
Configures the `SO_KEEPALIVE` option for this socket, which
determines whether it sends TCP keepalive probes.
`network.tcp.keep_idle`:: Configures the `TCP_KEEPIDLE` option for this socket, which
determines the time in seconds that a connection must be idle before
starting to send TCP keepalive probes. Defaults to `-1`, which uses
the system default. This value cannot exceed `300` seconds. Only applicable on Linux and macOS,
and requires Java 11 or newer.
`network.tcp.keep_interval`:: Configures the `TCP_KEEPINTVL` option for this socket,
which determines the time in seconds between sending TCP keepalive probes.
Defaults to `-1`, which uses the system default. This value cannot exceed `300` seconds.
Only applicable on Linux and macOS, and requires Java 11 or newer.
`network.tcp.keep_count`:: Configures the `TCP_KEEPCNT` option for this socket, which
determines the number of unacknowledged TCP keepalive probes that may be
sent on a connection before it is dropped. Defaults to `-1`,
which uses the system default. Only applicable on Linux and macOS, and requires
Java 11 or newer.
`network.tcp.reuse_address`::
Should an address be reused or not. Defaults to `true` on non-windows

106
docs/reference/modules/transport.asciidoc
View File

@ -18,44 +18,90 @@ as search in Elasticsearch.
[[transport-settings]]
==== Transport settings
The internal transport communicates over TCP. You can configure it with the
following settings:
The following settings can be configured for the internal transport that
communicates over TCP. These settings also use the common
<<modules-network,network settings>>.
[cols="<,<",options="header",]
|=======================================================================
|Setting |Description
|`transport.port` |A bind port range. Defaults to `9300-9400`.
`transport.port`::
A bind port range. Defaults to `9300-9400`.
|`transport.publish_port` |The port that other nodes in the cluster
`transport.publish_port`::
The port that other nodes in the cluster
should use when communicating with this node. Useful when a cluster node
is behind a proxy or firewall and the `transport.port` is not directly
addressable from the outside. Defaults to the actual port assigned via
`transport.port`.
|`transport.bind_host` |The host address to bind the transport service to. Defaults to `transport.host` (if set) or `network.bind_host`.
`transport.bind_host`::
The host address to bind the transport service to. Defaults to
`transport.host` (if set) or `network.bind_host`.
|`transport.publish_host` |The host address to publish for nodes in the cluster to connect to. Defaults to `transport.host` (if set) or `network.publish_host`.
`transport.publish_host`::
The host address to publish for nodes in the cluster to connect to.
Defaults to `transport.host` (if set) or `network.publish_host`.
|`transport.host` |Used to set the `transport.bind_host` and the `transport.publish_host`.
`transport.host`::
Used to set the `transport.bind_host` and the `transport.publish_host`.
|`transport.connect_timeout` |The connect timeout for initiating a new connection (in
`transport.connect_timeout`::
The connect timeout for initiating a new connection (in
time setting format). Defaults to `30s`.
|`transport.compress` |Set to `true` to enable compression (`DEFLATE`) between
`transport.compress`::
Set to `true` to enable compression (`DEFLATE`) between
all nodes. Defaults to `false`.
|`transport.ping_schedule` | Schedule a regular application-level ping message
`transport.ping_schedule`::
Schedule a regular application-level ping message
to ensure that transport connections between nodes are kept alive. Defaults to
`5s` in the transport client and `-1` (disabled) elsewhere. It is preferable
to correctly configure TCP keep-alives instead of using this feature, because
TCP keep-alives apply to all kinds of long-lived connections and not just to
transport connections.
|=======================================================================
`transport.tcp.no_delay`::
Enable or disable the https://en.wikipedia.org/wiki/Nagle%27s_algorithm[TCP no delay]
setting. Defaults to `network.tcp.no_delay`.
It also uses the common
<<modules-network,network settings>>.
`transport.tcp.keep_alive`::
Configures the `SO_KEEPALIVE` option for this socket, which
determines whether it sends TCP keepalive probes.
Defaults to `network.tcp.keep_alive`.
`transport.tcp.keep_idle`::
Configures the `TCP_KEEPIDLE` option for this socket, which
determines the time in seconds that a connection must be idle before
starting to send TCP keepalive probes. Defaults to `network.tcp.keep_idle` if set,
or the system default otherwise.
This value cannot exceed `300` seconds. In cases where the system default
is higher than `300`, the value is automatically lowered to `300`. Only applicable on
Linux and macOS, and requires Java 11 or newer.
`transport.tcp.keep_interval`::
Configures the `TCP_KEEPINTVL` option for this socket,
which determines the time in seconds between sending TCP keepalive probes.
Defaults to `network.tcp.keep_interval` if set, or the system default otherwise.
This value cannot exceed `300` seconds. In cases where the system default is higher than `300`,
the value is automatically lowered to `300`. Only applicable on Linux and macOS,
and requires Java 11 or newer.
`transport.tcp.keep_count`::
Configures the `TCP_KEEPCNT` option for this socket, which
determines the number of unacknowledged TCP keepalive probes that may be
sent on a connection before it is dropped. Defaults to `network.tcp.keep_count`
if set, or the system default otherwise. Only applicable on Linux and macOS, and
requires Java 11 or newer.
transport.tcp.reuse_address::
Should an address be reused or not. Defaults to `network.tcp.reuse_address`.
`transport.tcp.send_buffer_size`::
The size of the TCP send buffer (specified with <<size-units,size units>>).
Defaults to `network.tcp.send_buffer_size`.
`transport.tcp.receive_buffer_size`::
The size of the TCP receive buffer (specified with <<size-units,size units>>).
Defaults to `network.tcp.receive_buffer_size`.
[[transport-profiles]]
===== Transport profiles
@ -83,27 +129,11 @@ example above:
* `port`: The port to which to bind.
* `bind_host`: The host to which to bind.
* `publish_host`: The host which is published in informational APIs.
* `tcp.no_delay`: Configures the `TCP_NO_DELAY` option for this socket.
* `tcp.keep_alive`: Configures the `SO_KEEPALIVE` option for this socket, which
determines whether it sends TCP keepalive probes.
* `tcp.keep_idle`: Configures the `TCP_KEEPIDLE` option for this socket, which
determines the time in seconds that a connection must be idle before
starting to send TCP keepalive probes. Defaults to `-1` which means to use
the smaller of 300 or the system default. May not be greater than 300. Only
applicable on Linux and macOS, and requires Java 11 or newer.
* `tcp.keep_interval`: Configures the `TCP_KEEPINTVL` option for this socket,
which determines the time in seconds between sending TCP keepalive probes.
Defaults to `-1` which means to use the smaller of 300 or the system
default. May not be greater than 300. Only applicable on Linux and macOS,
and requires Java 11 or newer.
* `tcp.keep_count`: Configures the `TCP_KEEPCNT` option for this socket, which
determines the number of unacknowledged TCP keepalive probes that may be
sent on a connection before it is dropped. Defaults to `-1` which means to
use the system default. Only applicable on Linux and macOS, and requires
Java 11 or newer.
* `tcp.reuse_address`: Configures the `SO_REUSEADDR` option for this socket.
* `tcp.send_buffer_size`: Configures the send buffer size of the socket.
* `tcp.receive_buffer_size`: Configures the receive buffer size of the socket.
Profiles also support all the other transport settings specified in the
<<transport-settings,transport settings>> section, and use these as defaults.
For example, `transport.profiles.client.tcp.reuse_address` can be explicitly
configured, and defaults otherwise to `transport.tcp.reuse_address`.
[[long-lived-connections]]
===== Long-lived idle connections

7
server/src/main/java/org/elasticsearch/common/settings/Setting.java
View File

@ -1182,6 +1182,11 @@ public class Setting<T> implements ToXContentObject {
return new Setting<>(key, fallbackSetting, (s) -> parseInt(s, minValue, key, isFiltered(properties)), properties);
}
public static Setting<Integer> intSetting(String key, Setting<Integer> fallbackSetting, int minValue, int maxValue,
Property... properties) {
return new Setting<>(key, fallbackSetting, (s) -> parseInt(s, minValue, maxValue, key, isFiltered(properties)), properties);
}
public static Setting<Integer> intSetting(String key, Setting<Integer> fallbackSetting, int minValue, Validator<Integer> validator,
Property... properties) {
return new Setting<>(new SimpleKey(key), fallbackSetting, fallbackSetting::getRaw,
@ -1246,7 +1251,7 @@ public class Setting<T> implements ToXContentObject {
return parseInt(s, minValue, maxValue, key, false);
}
static int parseInt(String s, int minValue, int maxValue, String key, boolean isFiltered) {
public static int parseInt(String s, int minValue, int maxValue, String key, boolean isFiltered) {
int value = Integer.parseInt(s);
if (value < minValue) {
String err = "Failed to parse value" + (isFiltered ? "" : " [" + s + "]") + " for setting [" + key + "] must be >= " + minValue;

4
server/src/main/java/org/elasticsearch/http/HttpTransportSettings.java
View File

@ -116,9 +116,9 @@ public final class HttpTransportSettings {
public static final Setting<Boolean> SETTING_HTTP_TCP_KEEP_ALIVE =
boolSetting("http.tcp.keep_alive", NetworkService.TCP_KEEP_ALIVE, Setting.Property.NodeScope);
public static final Setting<Integer> SETTING_HTTP_TCP_KEEP_IDLE =
intSetting("http.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, Setting.Property.NodeScope);
intSetting("http.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, 300, Setting.Property.NodeScope);
public static final Setting<Integer> SETTING_HTTP_TCP_KEEP_INTERVAL =
intSetting("http.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, Setting.Property.NodeScope);
intSetting("http.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, 300, Setting.Property.NodeScope);
public static final Setting<Integer> SETTING_HTTP_TCP_KEEP_COUNT =
intSetting("http.tcp.keep_count", NetworkService.TCP_KEEP_COUNT, -1, Setting.Property.NodeScope);
public static final Setting<Boolean> SETTING_HTTP_TCP_REUSE_ADDRESS =

8
server/src/main/java/org/elasticsearch/transport/TransportSettings.java
View File

@ -101,15 +101,15 @@ public final class TransportSettings {
fallback(key, OLD_TCP_KEEP_ALIVE_PROFILE, "tcp\\.keep_alive$", "tcp_keep_alive"),
Setting.Property.NodeScope));
public static final Setting<Integer> TCP_KEEP_IDLE =
intSetting("transport.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, Setting.Property.NodeScope);
intSetting("transport.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, 300, Setting.Property.NodeScope);
public static final Setting.AffixSetting<Integer> TCP_KEEP_IDLE_PROFILE =
affixKeySetting("transport.profiles.", "tcp.keep_idle",
key -> intSetting(key, TCP_KEEP_IDLE, -1, Setting.Property.NodeScope));
key -> intSetting(key, TCP_KEEP_IDLE, -1, 300, Setting.Property.NodeScope));
public static final Setting<Integer> TCP_KEEP_INTERVAL =
intSetting("transport.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, Setting.Property.NodeScope);
intSetting("transport.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, 300, Setting.Property.NodeScope);
public static final Setting.AffixSetting<Integer> TCP_KEEP_INTERVAL_PROFILE =
affixKeySetting("transport.profiles.", "tcp.keep_interval",
key -> intSetting(key, TCP_KEEP_INTERVAL, -1, Setting.Property.NodeScope));
key -> intSetting(key, TCP_KEEP_INTERVAL, -1, 300, Setting.Property.NodeScope));
public static final Setting<Integer> TCP_KEEP_COUNT =
intSetting("transport.tcp.keep_count", NetworkService.TCP_KEEP_COUNT, -1, Setting.Property.NodeScope);
public static final Setting.AffixSetting<Integer> TCP_KEEP_COUNT_PROFILE =