2014-12-11 07:17:29 -05:00
|
|
|
# Detecting Dead Connections
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
In this section we will discuss connection time-to-live (TTL) and
|
2015-04-27 17:32:30 -04:00
|
|
|
explain how Apache ActiveMQ Artemis deals with crashed clients and clients which have
|
2014-12-04 10:25:29 -05:00
|
|
|
exited without cleanly closing their resources.
|
|
|
|
|
2018-03-08 15:46:38 -05:00
|
|
|
## Cleaning up Resources on the Server
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2015-04-27 17:32:30 -04:00
|
|
|
Before an Apache ActiveMQ Artemis client application exits it is considered good
|
2014-12-04 10:25:29 -05:00
|
|
|
practice that it should close its resources in a controlled manner,
|
|
|
|
using a `finally` block.
|
|
|
|
|
|
|
|
Here's an example of a well behaved core client application closing its
|
|
|
|
session and session factory in a finally block:
|
|
|
|
|
2018-03-08 15:46:38 -05:00
|
|
|
```java
|
2014-12-11 07:17:29 -05:00
|
|
|
ServerLocator locator = null;
|
|
|
|
ClientSessionFactory sf = null;
|
|
|
|
ClientSession session = null;
|
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
try {
|
2014-12-11 07:17:29 -05:00
|
|
|
locator = ActiveMQClient.createServerLocatorWithoutHA(..);
|
|
|
|
|
|
|
|
sf = locator.createClientSessionFactory();;
|
|
|
|
|
|
|
|
session = sf.createSession(...);
|
2015-02-25 08:37:19 -05:00
|
|
|
|
2014-12-11 07:17:29 -05:00
|
|
|
... do some stuff with the session...
|
2018-03-09 10:07:38 -05:00
|
|
|
} finally {
|
|
|
|
if (session != null) {
|
2014-12-11 07:17:29 -05:00
|
|
|
session.close();
|
|
|
|
}
|
2015-02-25 08:37:19 -05:00
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
if (sf != null) {
|
2014-12-11 07:17:29 -05:00
|
|
|
sf.close();
|
|
|
|
}
|
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
if(locator != null) {
|
2014-12-11 07:17:29 -05:00
|
|
|
locator.close();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
```
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
And here's an example of a well behaved JMS client application:
|
|
|
|
|
2018-03-08 15:46:38 -05:00
|
|
|
```java
|
2014-12-11 07:17:29 -05:00
|
|
|
Connection jmsConnection = null;
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
try {
|
2017-08-31 11:43:56 -04:00
|
|
|
ConnectionFactory jmsConnectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61616");
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2014-12-11 07:17:29 -05:00
|
|
|
jmsConnection = jmsConnectionFactory.createConnection();
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2014-12-11 07:17:29 -05:00
|
|
|
... do some stuff with the connection...
|
2018-03-09 10:07:38 -05:00
|
|
|
} finally {
|
|
|
|
if (connection != null) {
|
2014-12-11 07:17:29 -05:00
|
|
|
connection.close();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
```
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2016-08-10 18:37:40 -04:00
|
|
|
|
|
|
|
Or with using auto-closeable feature from Java, which can save a few lines of code:
|
|
|
|
|
2018-03-08 15:46:38 -05:00
|
|
|
```java
|
2016-08-10 18:37:40 -04:00
|
|
|
try (
|
2017-08-31 11:43:56 -04:00
|
|
|
ActiveMQConnectionFactory jmsConnectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61616");
|
2018-03-09 10:07:38 -05:00
|
|
|
Connection jmsConnection = jmsConnectionFactory.createConnection()) {
|
2016-08-10 18:37:40 -04:00
|
|
|
... do some stuff with the connection...
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
2014-12-04 10:25:29 -05:00
|
|
|
Unfortunately users don't always write well behaved applications, and
|
|
|
|
sometimes clients just crash so they don't have a chance to clean up
|
|
|
|
their resources!
|
|
|
|
|
|
|
|
If this occurs then it can leave server side resources, like sessions,
|
|
|
|
hanging on the server. If these were not removed they would cause a
|
|
|
|
resource leak on the server and over time this result in the server
|
|
|
|
running out of memory or other resources.
|
|
|
|
|
|
|
|
We have to balance the requirement for cleaning up dead client resources
|
|
|
|
with the fact that sometimes the network between the client and the
|
|
|
|
server can fail and then come back, allowing the client to reconnect.
|
2015-04-27 17:32:30 -04:00
|
|
|
Apache ActiveMQ Artemis supports client reconnection, so we don't want to clean up
|
2014-12-04 10:25:29 -05:00
|
|
|
"dead" server side resources too soon or this will prevent any client
|
|
|
|
from reconnecting, as it won't be able to find its old sessions on the
|
|
|
|
server.
|
|
|
|
|
2017-08-31 11:43:56 -04:00
|
|
|
Apache ActiveMQ Artemis makes all of this configurable via a *connection TTL*.
|
|
|
|
Basically, the TTL determines how long the server will keep a connection
|
|
|
|
alive in the absence of any data arriving from the client. The client will
|
|
|
|
automatically send "ping" packets periodically to prevent the server from
|
|
|
|
closing it down. If the server doesn't receive any packets on a connection
|
|
|
|
for the connection TTL time, then it will automatically close all the
|
|
|
|
sessions on the server that relate to that connection.
|
|
|
|
|
|
|
|
The connection TTL is configured on the URI using the `connectionTtl`
|
|
|
|
parameter.
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
The default value for connection ttl on an "unreliable" connection (e.g.
|
2017-08-31 11:43:56 -04:00
|
|
|
a Netty connection using the `tcp` URL scheme) is `60000`ms, i.e. 1 minute.
|
|
|
|
The default value for connection ttl on a "reliable" connection (e.g. an
|
|
|
|
in-vm connection using the `vm` URL scheme) is `-1`. A value of `-1` for
|
|
|
|
`connectionTTL` means the server will never time out the connection on
|
|
|
|
the server side.
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
If you do not wish clients to be able to specify their own connection
|
|
|
|
TTL, you can override all values used by a global value set on the
|
|
|
|
server side. This can be done by specifying the
|
|
|
|
`connection-ttl-override` attribute in the server side configuration.
|
|
|
|
The default value for `connection-ttl-override` is `-1` which means "do
|
|
|
|
not override" (i.e. let clients use their own values).
|
|
|
|
|
2016-07-18 17:34:22 -04:00
|
|
|
The logic to check connections for TTL violations runs periodically on
|
|
|
|
the broker. By default, the checks are done every 2,000 milliseconds.
|
|
|
|
However, this can be changed if necessary by using the
|
|
|
|
`connection-ttl-check-interval` attribute.
|
|
|
|
|
2018-03-08 15:46:38 -05:00
|
|
|
## Closing Forgotten Resources
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
As previously discussed, it's important that all core client sessions
|
|
|
|
and JMS connections are always closed explicitly in a `finally` block
|
|
|
|
when you are finished using them.
|
|
|
|
|
2015-04-27 17:32:30 -04:00
|
|
|
If you fail to do so, Apache ActiveMQ Artemis will detect this at garbage collection
|
2017-08-31 11:43:56 -04:00
|
|
|
time, and log a warning (If you are using JMS the warning will involve a JMS connection).
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2015-04-27 17:32:30 -04:00
|
|
|
Apache ActiveMQ Artemis will then close the connection / client session for you.
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
Note that the log will also tell you the exact line of your user code
|
|
|
|
where you created the JMS connection / client session that you later did
|
|
|
|
not close. This will enable you to pinpoint the error in your code and
|
|
|
|
correct it appropriately.
|
|
|
|
|
2018-03-08 15:46:38 -05:00
|
|
|
## Detecting Failure from the Client
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
In the previous section we discussed how the client sends pings to the
|
|
|
|
server and how "dead" connection resources are cleaned up by the server.
|
|
|
|
There's also another reason for pinging, and that's for the *client* to
|
|
|
|
be able to detect that the server or network has failed.
|
|
|
|
|
|
|
|
As long as the client is receiving data from the server it will consider
|
|
|
|
the connection to be still alive.
|
|
|
|
|
2016-07-05 12:33:16 -04:00
|
|
|
If the client does not receive any packets for a configurable number
|
|
|
|
of milliseconds then it will consider the connection failed and will
|
|
|
|
either initiate failover, or call any `FailureListener` instances (or
|
|
|
|
`ExceptionListener` instances if you are using JMS) depending on how
|
|
|
|
it has been configured.
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2017-08-31 11:43:56 -04:00
|
|
|
This is controlled by setting the `clientFailureCheckPeriod` parameter
|
|
|
|
on the URI your client is using to connect, e.g.
|
2016-07-05 12:33:16 -04:00
|
|
|
`tcp://localhost:61616?clientFailureCheckPeriod=30000`.
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
The default value for client failure check period on an "unreliable"
|
2016-07-05 12:33:16 -04:00
|
|
|
connection (e.g. a Netty connection) is `30000` ms, i.e. 30 seconds. The
|
2014-12-04 10:25:29 -05:00
|
|
|
default value for client failure check period on a "reliable" connection
|
|
|
|
(e.g. an in-vm connection) is `-1`. A value of `-1` means the client
|
|
|
|
will never fail the connection on the client side if no data is received
|
|
|
|
from the server. Typically this is much lower than connection TTL to
|
|
|
|
allow clients to reconnect in case of transitory failure.
|
|
|
|
|
2014-12-11 07:17:29 -05:00
|
|
|
## Configuring Asynchronous Connection Execution
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
Most packets received on the server side are executed on the remoting
|
|
|
|
thread. These packets represent short-running operations and are always
|
|
|
|
executed on the remoting thread for performance reasons.
|
|
|
|
|
|
|
|
However, by default some kinds of packets are executed using a thread
|
|
|
|
from a thread pool so that the remoting thread is not tied up for too
|
|
|
|
long. Please note that processing operations asynchronously on another
|
|
|
|
thread adds a little more latency. These packets are:
|
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
- `org.apache.activemq.artemis.core.protocol.core.impl.wireformat.RollbackMessage`
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
- `org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionCloseMessage`
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
- `org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionCommitMessage`
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
- `org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionXACommitMessage`
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
- `org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionXAPrepareMessage`
|
2014-12-04 10:25:29 -05:00
|
|
|
|
2018-03-09 10:07:38 -05:00
|
|
|
- `org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionXARollbackMessage`
|
2014-12-04 10:25:29 -05:00
|
|
|
|
|
|
|
To disable asynchronous connection execution, set the parameter
|
2015-04-29 05:30:31 -04:00
|
|
|
`async-connection-execution-enabled` in `broker.xml` to
|
2016-08-10 18:37:40 -04:00
|
|
|
`false` (default value is `true`).
|