1018 lines
68 KiB
XML
1018 lines
68 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!-- ============================================================================= -->
|
|
<!-- Licensed to the Apache Software Foundation (ASF) under one or more -->
|
|
<!-- contributor license agreements. See the NOTICE file distributed with -->
|
|
<!-- this work for additional information regarding copyright ownership. -->
|
|
<!-- The ASF licenses this file to You under the Apache License, Version 2.0 -->
|
|
<!-- (the "License"); you may not use this file except in compliance with -->
|
|
<!-- the License. You may obtain a copy of the License at -->
|
|
<!-- -->
|
|
<!-- http://www.apache.org/licenses/LICENSE-2.0 -->
|
|
<!-- -->
|
|
<!-- Unless required by applicable law or agreed to in writing, software -->
|
|
<!-- distributed under the License is distributed on an "AS IS" BASIS, -->
|
|
<!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
|
|
<!-- See the License for the specific language governing permissions and -->
|
|
<!-- limitations under the License. -->
|
|
<!-- ============================================================================= -->
|
|
|
|
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
|
|
<!ENTITY % BOOK_ENTITIES SYSTEM "ActiveMQ_User_Manual.ent">
|
|
%BOOK_ENTITIES;
|
|
]>
|
|
<chapter id="clusters">
|
|
<title>Clusters</title>
|
|
<section>
|
|
<title>Clusters Overview</title>
|
|
<para>ActiveMQ clusters allow groups of ActiveMQ servers to be grouped together in order to
|
|
share message processing load. Each active node in the cluster is an active ActiveMQ
|
|
server which manages its own messages and handles its own connections. </para>
|
|
<note id="clustered-deprecation">
|
|
<para>The <emphasis>clustered</emphasis> parameter is deprecated and no longer needed for
|
|
setting up a cluster. If your configuration contains this parameter it will be ignored and
|
|
a message with the ID <literal>HQ221038</literal> will be logged.</para>
|
|
</note>
|
|
<para>The cluster is formed by each node declaring <emphasis>cluster connections</emphasis>
|
|
to other nodes in the core configuration file <literal
|
|
>activemq-configuration.xml</literal>. When a node forms a cluster connection to
|
|
another node, internally it creates a <emphasis>core bridge</emphasis> (as described in
|
|
<xref linkend="core-bridges"/>) connection between it and the other node, this is
|
|
done transparently behind the scenes - you don't have to declare an explicit bridge for
|
|
each node. These cluster connections allow messages to flow between the nodes of the
|
|
cluster to balance load.</para>
|
|
<para>Nodes can be connected together to form a cluster in many different topologies, we
|
|
will discuss a couple of the more common topologies later in this chapter.</para>
|
|
<para>We'll also discuss client side load balancing, where we can balance client connections
|
|
across the nodes of the cluster, and we'll consider message redistribution where ActiveMQ
|
|
will redistribute messages between nodes to avoid starvation.</para>
|
|
<para>Another important part of clustering is <emphasis>server discovery</emphasis> where
|
|
servers can broadcast their connection details so clients or other servers can connect
|
|
to them with the minimum of configuration.</para>
|
|
<warning id="copy-warning">
|
|
<para>Once a cluster node has been configured it is common to simply copy that configuration
|
|
to other nodes to produce a symmetric cluster. However, care must be taken when copying the
|
|
ActiveMQ files. Do not copy the ActiveMQ <emphasis>data</emphasis> (i.e. the
|
|
<literal>bindings</literal>, <literal>journal</literal>, and <literal>large-messages</literal>
|
|
directories) from one node to another. When a node is started for the first time and initializes
|
|
its journal files it also persists a special identifier to the <literal>journal</literal>
|
|
directory. This id <emphasis>must</emphasis> be unique among nodes in the cluster or the
|
|
cluster will not form properly.</para>
|
|
</warning>
|
|
</section>
|
|
<section id="clusters.server-discovery">
|
|
<title>Server discovery</title>
|
|
<para>Server discovery is a mechanism by which servers can propagate their connection details to:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Messaging clients. A messaging client wants to be able to connect
|
|
to the servers of the cluster without having specific knowledge of which servers
|
|
in the cluster are up at any one time.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Other servers. Servers in a cluster want to be able to create
|
|
cluster connections to each other without having prior knowledge of all the
|
|
other servers in the cluster.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
This information, let's call it the Cluster Topology, is actually sent around normal ActiveMQ
|
|
connections to clients and to other servers over cluster connections. This being the case we need a
|
|
way of establishing the initial first connection. This can be done using
|
|
dynamic discovery techniques like <ulink url="http://en.wikipedia.org/wiki/User_Datagram_Protocol" >UDP</ulink>
|
|
and <ulink url="http://www.jgroups.org/">JGroups</ulink>, or by
|
|
providing a list of initial connectors.
|
|
</para>
|
|
<section>
|
|
<title>Dynamic Discovery</title>
|
|
<para>
|
|
Server discovery uses <ulink url="http://en.wikipedia.org/wiki/User_Datagram_Protocol" >UDP</ulink>
|
|
multicast or <ulink url="http://www.jgroups.org/">JGroups</ulink> to broadcast server connection settings.
|
|
</para>
|
|
<section id="clusters.broadcast-groups">
|
|
<title>Broadcast Groups</title>
|
|
<para>A broadcast group is the means by which a server broadcasts connectors over the
|
|
network. A connector defines a way in which a client (or other server) can make
|
|
connections to the server. For more information on what a connector is, please see
|
|
<xref linkend="configuring-transports"/>.</para>
|
|
<para>The broadcast group takes a set of connector pairs, each connector pair contains
|
|
connection settings for a live and backup server (if one exists) and broadcasts them on
|
|
the network. Depending on which broadcasting technique you configure the cluster, it
|
|
uses either UDP or JGroups to broadcast connector pairs information.</para>
|
|
<para>Broadcast groups are defined in the server configuration file <literal
|
|
>activemq-configuration.xml</literal>. There can be many broadcast groups per
|
|
ActiveMQ server. All broadcast groups must be defined in a <literal
|
|
>broadcast-groups</literal> element.</para>
|
|
<para>Let's take a look at an example broadcast group from <literal
|
|
>activemq-configuration.xml</literal> that defines a UDP broadcast group:</para>
|
|
<programlisting>
|
|
<broadcast-groups>
|
|
<broadcast-group name="my-broadcast-group">
|
|
<local-bind-address>172.16.9.3</local-bind-address>
|
|
<local-bind-port>5432</local-bind-port>
|
|
<group-address>231.7.7.7</group-address>
|
|
<group-port>9876</group-port>
|
|
<broadcast-period>2000</broadcast-period>
|
|
<connector-ref connector-name="netty-connector"/>
|
|
</broadcast-group>
|
|
</broadcast-groups></programlisting>
|
|
<para>Some of the broadcast group parameters are optional and you'll normally use the
|
|
defaults, but we specify them all in the above example for clarity. Let's discuss
|
|
each one in turn:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><literal>name</literal> attribute. Each broadcast group in the server must
|
|
have a unique name. </para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>local-bind-address</literal>. This is the local bind address that
|
|
the datagram socket is bound to. If you have multiple network interfaces on
|
|
your server, you would specify which one you wish to use for broadcasts by
|
|
setting this property. If this property is not specified then the socket
|
|
will be bound to the wildcard address, an IP address chosen by the
|
|
kernel. This is a UDP specific attribute.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>local-bind-port</literal>. If you want to specify a local port to
|
|
which the datagram socket is bound you can specify it here. Normally you
|
|
would just use the default value of <literal>-1</literal> which signifies
|
|
that an anonymous port should be used. This parameter is always specified in conjunction with
|
|
<literal>local-bind-address</literal>. This is a UDP specific attribute.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>group-address</literal>. This is the multicast address to which
|
|
the data will be broadcast. It is a class D IP address in the range <literal
|
|
>224.0.0.0</literal> to <literal>239.255.255.255</literal>, inclusive.
|
|
The address <literal>224.0.0.0</literal> is reserved and is not available
|
|
for use. This parameter is mandatory. This is a UDP specific attribute.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>group-port</literal>. This is the UDP port number used for
|
|
broadcasting. This parameter is mandatory. This is a UDP specific attribute.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>broadcast-period</literal>. This is the period in milliseconds
|
|
between consecutive broadcasts. This parameter is optional, the default
|
|
value is <literal>2000</literal> milliseconds.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>connector-ref</literal>. This specifies the connector and
|
|
optional backup connector that will be broadcasted (see <xref
|
|
linkend="configuring-transports"/> for more information on connectors).
|
|
The connector to be broadcasted is specified by the <literal
|
|
>connector-name</literal> attribute.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para id="clusters.jgroups-example">Here is another example broadcast group that defines a JGroups broadcast group:</para>
|
|
<programlisting>
|
|
<broadcast-groups>
|
|
<broadcast-group name="my-broadcast-group">
|
|
<jgroups-file>test-jgroups-file_ping.xml</jgroups-file>
|
|
<jgroups-channel>activemq_broadcast_channel</jgroups-channel>
|
|
<broadcast-period>2000</broadcast-period>
|
|
<connector-ref connector-name="netty-connector"/>
|
|
</broadcast-group>
|
|
</broadcast-groups></programlisting>
|
|
<para>To be able to use JGroups to broadcast, one must specify two attributes, i.e.
|
|
<literal>jgroups-file</literal> and <literal>jgroups-channel</literal>, as discussed
|
|
in details as following:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><literal>jgroups-file</literal> attribute. This is the name of JGroups configuration
|
|
file. It will be used to initialize JGroups channels. Make sure the file is in the
|
|
java resource path so that ActiveMQ can load it. </para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>jgroups-channel</literal> attribute. The name that JGroups channels connect
|
|
to for broadcasting.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<note>
|
|
<para>The JGroups attributes (<literal>jgroups-file</literal> and <literal>jgroups-channel</literal>)
|
|
and UDP specific attributes described above are exclusive of each other. Only one set can be
|
|
specified in a broadcast group configuration. Don't mix them!</para>
|
|
</note>
|
|
<para id="clusters.jgroups-file">
|
|
The following is an example of a JGroups file
|
|
<programlisting>
|
|
<config xmlns="urn:org:jgroups"
|
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
|
xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.0.xsd">
|
|
<TCP loopback="true"
|
|
recv_buf_size="20000000"
|
|
send_buf_size="640000"
|
|
discard_incompatible_packets="true"
|
|
max_bundle_size="64000"
|
|
max_bundle_timeout="30"
|
|
enable_bundling="true"
|
|
use_send_queues="false"
|
|
sock_conn_timeout="300"
|
|
|
|
thread_pool.enabled="true"
|
|
thread_pool.min_threads="1"
|
|
thread_pool.max_threads="10"
|
|
thread_pool.keep_alive_time="5000"
|
|
thread_pool.queue_enabled="false"
|
|
thread_pool.queue_max_size="100"
|
|
thread_pool.rejection_policy="run"
|
|
|
|
oob_thread_pool.enabled="true"
|
|
oob_thread_pool.min_threads="1"
|
|
oob_thread_pool.max_threads="8"
|
|
oob_thread_pool.keep_alive_time="5000"
|
|
oob_thread_pool.queue_enabled="false"
|
|
oob_thread_pool.queue_max_size="100"
|
|
oob_thread_pool.rejection_policy="run"/>
|
|
|
|
<FILE_PING location="../file.ping.dir"/>
|
|
<MERGE2 max_interval="30000"
|
|
min_interval="10000"/>
|
|
<FD_SOCK/>
|
|
<FD timeout="10000" max_tries="5" />
|
|
<VERIFY_SUSPECT timeout="1500" />
|
|
<BARRIER />
|
|
<pbcast.NAKACK
|
|
use_mcast_xmit="false"
|
|
retransmit_timeout="300,600,1200,2400,4800"
|
|
discard_delivered_msgs="true"/>
|
|
<UNICAST timeout="300,600,1200" />
|
|
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
|
|
max_bytes="400000"/>
|
|
<pbcast.GMS print_local_addr="true" join_timeout="3000"
|
|
view_bundling="true"/>
|
|
<FC max_credits="2000000"
|
|
min_threshold="0.10"/>
|
|
<FRAG2 frag_size="60000" />
|
|
<pbcast.STATE_TRANSFER/>
|
|
<pbcast.FLUSH timeout="0"/>
|
|
</config></programlisting>
|
|
</para>
|
|
<para>
|
|
As it shows, the file content defines a jgroups protocol stacks. If you want activemq
|
|
to use this stacks for channel creation, you have to make sure the value of
|
|
<literal>jgroups-file</literal> in your broadcast-group/discovery-group configuration
|
|
to be the name of this jgroups configuration file. For example if the above stacks
|
|
configuration is stored in a file named "jgroups-stacks.xml" then your
|
|
<literal>jgroups-file</literal> should be like
|
|
<programlisting>
|
|
<jgroups-file>jgroups-stacks.xml</jgroups-file></programlisting>
|
|
</para>
|
|
</section>
|
|
<section id="clusters.discovery-groups">
|
|
<title>Discovery Groups</title>
|
|
<para>While the broadcast group defines how connector information is broadcasted from a
|
|
server, a discovery group defines how connector information is received from a
|
|
broadcast endpoint (a UDP multicast address or JGroup channel).</para>
|
|
<para>A discovery group maintains a list of connector pairs - one for each broadcast by
|
|
a different server. As it receives broadcasts on the broadcast endpoint from a
|
|
particular server it updates its entry in the list for that server.</para>
|
|
<para>If it has not received a broadcast from a particular server for a length of time
|
|
it will remove that server's entry from its list.</para>
|
|
<para>Discovery groups are used in two places in ActiveMQ:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>By cluster connections so they know how to obtain an initial connection to download the topology</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>By messaging clients so they know how to obtain an initial connection to download the topology</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
Although a discovery group will always accept broadcasts, its current list of available live and
|
|
backup servers is only ever used when an initial connection is made, from then server discovery is
|
|
done over the normal ActiveMQ connections.
|
|
</para>
|
|
<note>
|
|
<para>
|
|
Each discovery group must be configured with broadcast endpoint (UDP or JGroups) that matches its broadcast
|
|
group counterpart. For example, if broadcast is configured using UDP, the discovery group must also use UDP, and the same
|
|
multicast address.
|
|
</para>
|
|
</note>
|
|
</section>
|
|
<section>
|
|
<title>Defining Discovery Groups on the Server</title>
|
|
<para>For cluster connections, discovery groups are defined in the server side
|
|
configuration file <literal>activemq-configuration.xml</literal>. All discovery
|
|
groups must be defined inside a <literal>discovery-groups</literal> element. There
|
|
can be many discovery groups defined by ActiveMQ server. Let's look at an
|
|
example:</para>
|
|
<programlisting>
|
|
<discovery-groups>
|
|
<discovery-group name="my-discovery-group">
|
|
<local-bind-address>172.16.9.7</local-bind-address>
|
|
<group-address>231.7.7.7</group-address>
|
|
<group-port>9876</group-port>
|
|
<refresh-timeout>10000</refresh-timeout>
|
|
</discovery-group>
|
|
</discovery-groups></programlisting>
|
|
<para>We'll consider each parameter of the discovery group:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><literal>name</literal> attribute. Each discovery group must have a unique
|
|
name per server.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>local-bind-address</literal>. If you are running with multiple network interfaces on the same machine, you
|
|
may want to specify that the discovery group listens only only a specific interface. To do this you can specify the interface
|
|
address with this parameter. This parameter is optional. This is a UDP specific attribute.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>group-address</literal>. This is the multicast IP address of the
|
|
group to listen on. It should match the <literal>group-address</literal> in
|
|
the broadcast group that you wish to listen from. This parameter is
|
|
mandatory. This is a UDP specific attribute.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>group-port</literal>. This is the UDP port of the multicast
|
|
group. It should match the <literal>group-port</literal> in the broadcast
|
|
group that you wish to listen from. This parameter is mandatory. This is a UDP specific attribute.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>refresh-timeout</literal>. This is the period the discovery group
|
|
waits after receiving the last broadcast from a particular server before
|
|
removing that servers connector pair entry from its list. You would normally
|
|
set this to a value significantly higher than the <literal
|
|
>broadcast-period</literal> on the broadcast group otherwise servers
|
|
might intermittently disappear from the list even though they are still
|
|
broadcasting due to slight differences in timing. This parameter is
|
|
optional, the default value is <literal>10000</literal> milliseconds (10
|
|
seconds).</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>Here is another example that defines a JGroups discovery group:</para>
|
|
<programlisting>
|
|
<discovery-groups>
|
|
<discovery-group name="my-broadcast-group">
|
|
<jgroups-file>test-jgroups-file_ping.xml</jgroups-file>
|
|
<jgroups-channel>activemq_broadcast_channel</jgroups-channel>
|
|
<refresh-timeout>10000</refresh-timeout>
|
|
</discovery-group>
|
|
</discovery-groups></programlisting>
|
|
<para>To receive broadcast from JGroups channels, one must specify two attributes,
|
|
<literal>jgroups-file</literal> and <literal>jgroups-channel</literal>, as discussed
|
|
in details as following:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><literal>jgroups-file</literal> attribute. This is the name of JGroups configuration
|
|
file. It will be used to initialize JGroups channels. Make sure the file is in the
|
|
java resource path so that ActiveMQ can load it. </para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>jgroups-channel</literal> attribute. The name that JGroups channels connect
|
|
to for receiving broadcasts.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<note>
|
|
<para>The JGroups attributes (<literal>jgroups-file</literal> and <literal>jgroups-channel</literal>)
|
|
and UDP specific attributes described above are exclusive of each other. Only one set can be
|
|
specified in a discovery group configuration. Don't mix them!</para>
|
|
</note>
|
|
</section>
|
|
<section id="clusters-discovery.groups.clientside">
|
|
<title>Discovery Groups on the Client Side</title>
|
|
<para>Let's discuss how to configure a ActiveMQ client to use discovery to discover a
|
|
list of servers to which it can connect. The way to do this differs depending on
|
|
whether you're using JMS or the core API.</para>
|
|
<section>
|
|
<title>Configuring client discovery using JMS</title>
|
|
<para>If you're using JMS and you're also using the JMS Service on the server to
|
|
load your JMS connection factory instances into JNDI, then you can specify which
|
|
discovery group to use for your JMS connection factory in the server side xml
|
|
configuration <literal>activemq-jms.xml</literal>. Let's take a look at an
|
|
example:</para>
|
|
<programlisting>
|
|
<connection-factory name="ConnectionFactory">
|
|
<discovery-group-ref discovery-group-name="my-discovery-group"/>
|
|
<entries>
|
|
<entry name="ConnectionFactory"/>
|
|
</entries>
|
|
</connection-factory></programlisting>
|
|
<para>The element <literal>discovery-group-ref</literal> specifies the name of a
|
|
discovery group defined in <literal>activemq-configuration.xml</literal>.</para>
|
|
<para>When this connection factory is downloaded from JNDI by a client application
|
|
and JMS connections are created from it, those connections will be load-balanced
|
|
across the list of servers that the discovery group maintains by listening on
|
|
the multicast address specified in the discovery group configuration.</para>
|
|
<para>If you're using JMS, but you're not using JNDI to lookup a connection factory
|
|
- you're instantiating the JMS connection factory directly then you can specify
|
|
the discovery group parameters directly when creating the JMS connection
|
|
factory. Here's an
|
|
example:</para>
|
|
<programlisting>
|
|
final String groupAddress = "231.7.7.7";
|
|
|
|
final int groupPort = 9876;
|
|
|
|
ConnectionFactory jmsConnectionFactory =
|
|
ActiveMQJMSClient.createConnectionFactory(new DiscoveryGroupConfiguration(groupAddress, groupPort,
|
|
new UDPBroadcastGroupConfiguration(groupAddress, groupPort, null, -1)), JMSFactoryType.CF);
|
|
|
|
Connection jmsConnection1 = jmsConnectionFactory.createConnection();
|
|
|
|
Connection jmsConnection2 = jmsConnectionFactory.createConnection();</programlisting>
|
|
<para>The <literal>refresh-timeout</literal> can be set directly on the DiscoveryGroupConfiguration
|
|
by using the setter method <literal>setDiscoveryRefreshTimeout()</literal> if you
|
|
want to change the default value.</para>
|
|
<para>There is also a further parameter settable on the DiscoveryGroupConfiguration using the
|
|
setter method <literal>setDiscoveryInitialWaitTimeout()</literal>. If the connection
|
|
factory is used immediately after creation then it may not have had enough time
|
|
to received broadcasts from all the nodes in the cluster. On first usage, the
|
|
connection factory will make sure it waits this long since creation before
|
|
creating the first connection. The default value for this parameter is <literal
|
|
>10000</literal> milliseconds.</para>
|
|
</section>
|
|
<section>
|
|
<title>Configuring client discovery using Core</title>
|
|
<para>If you're using the core API to directly instantiate
|
|
<literal>ClientSessionFactory</literal> instances, then you can specify the
|
|
discovery group parameters directly when creating the session factory. Here's an
|
|
example:</para>
|
|
<programlisting>
|
|
final String groupAddress = "231.7.7.7";
|
|
final int groupPort = 9876;
|
|
ServerLocator factory = ActiveMQClient.createServerLocatorWithHA(new DiscoveryGroupConfiguration(groupAddress, groupPort,
|
|
new UDPBroadcastGroupConfiguration(groupAddress, groupPort, null, -1))));
|
|
ClientSessionFactory factory = locator.createSessionFactory();
|
|
ClientSession session1 = factory.createSession();
|
|
ClientSession session2 = factory.createSession();</programlisting>
|
|
<para>The <literal>refresh-timeout</literal> can be set directly on the DiscoveryGroupConfiguration
|
|
by using the setter method <literal>setDiscoveryRefreshTimeout()</literal> if you
|
|
want to change the default value.</para>
|
|
<para>There is also a further parameter settable on the DiscoveryGroupConfiguration using the
|
|
setter method <literal>setDiscoveryInitialWaitTimeout()</literal>. If the session factory
|
|
is used immediately after creation then it may not have had enough time to
|
|
received broadcasts from all the nodes in the cluster. On first usage, the
|
|
session factory will make sure it waits this long since creation before creating
|
|
the first session. The default value for this parameter is <literal
|
|
>10000</literal> milliseconds.</para>
|
|
</section>
|
|
</section>
|
|
</section>
|
|
<section>
|
|
<title>Discovery using static Connectors</title>
|
|
<para>Sometimes it may be impossible to use UDP on the network you are using. In this case its
|
|
possible to configure a connection with an initial list if possible servers. This could be just
|
|
one server that you know will always be available or a list of servers where at least one will
|
|
be available.</para>
|
|
<para>This doesn't mean that you have to know where all your servers are going to be hosted, you
|
|
can configure these servers to use the reliable servers to connect to. Once they are connected
|
|
there connection details will be propagated via the server it connects to</para>
|
|
<section>
|
|
<title>Configuring a Cluster Connection</title>
|
|
<para>For cluster connections there is no extra configuration needed, you just need to make sure that any
|
|
connectors are defined in the usual manner, (see <xref linkend="configuring-transports"/> for more
|
|
information on connectors). These are then referenced by the cluster connection configuration.</para>
|
|
</section>
|
|
<section>
|
|
<title>Configuring a Client Connection</title>
|
|
<para>A static list of possible servers can also be used by a normal client.</para>
|
|
<section>
|
|
<title>Configuring client discovery using JMS</title>
|
|
<para>If you're using JMS and you're also using the JMS Service on the server to
|
|
load your JMS connection factory instances into JNDI, then you can specify which
|
|
connectors to use for your JMS connection factory in the server side xml
|
|
configuration <literal>activemq-jms.xml</literal>. Let's take a look at an
|
|
example:</para>
|
|
<programlisting>
|
|
<connection-factory name="ConnectionFactory">
|
|
<connectors>
|
|
<connector-ref connector-name="netty-connector"/>
|
|
<connector-ref connector-name="netty-connector2"/>
|
|
<connector-ref connector-name="netty-connector3"/>
|
|
</connectors>
|
|
<entries>
|
|
<entry name="ConnectionFactory"/>
|
|
</entries>
|
|
</connection-factory></programlisting>
|
|
<para>
|
|
The element <literal>connectors</literal> contains a list of pre defined connectors in the
|
|
<literal>activemq-configuration.xml</literal> file. When this connection factory is downloaded
|
|
from JNDI by a client application and JMS connections are created from it, those connections will
|
|
be load-balanced across the list of servers defined by these connectors.
|
|
</para>
|
|
<para>
|
|
If you're using JMS, but you're not using JNDI to lookup a connection factory - you're instantiating
|
|
the JMS connection factory directly then you can specify the connector list directly when creating
|
|
the JMS connection factory. Here's an example:
|
|
</para>
|
|
<programlisting>
|
|
HashMap<String, Object> map = new HashMap<String, Object>();
|
|
map.put("host", "myhost");
|
|
map.put("port", "5445");
|
|
TransportConfiguration server1 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map);
|
|
HashMap<String, Object> map2 = new HashMap<String, Object>();
|
|
map2.put("host", "myhost2");
|
|
map2.put("port", "5446");
|
|
TransportConfiguration server2 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map2);
|
|
|
|
ActiveMQConnectionFactory cf = ActiveMQJMSClient.createConnectionFactoryWithHA(JMSFactoryType.CF, server1, server2);</programlisting>
|
|
</section>
|
|
<section>
|
|
<title>Configuring client discovery using Core</title>
|
|
<para>If you are using the core API then the same can be done as follows:</para>
|
|
<programlisting>
|
|
HashMap<String, Object> map = new HashMap<String, Object>();
|
|
map.put("host", "myhost");
|
|
map.put("port", "5445");
|
|
TransportConfiguration server1 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map);
|
|
HashMap<String, Object> map2 = new HashMap<String, Object>();
|
|
map2.put("host", "myhost2");
|
|
map2.put("port", "5446");
|
|
TransportConfiguration server2 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map2);
|
|
|
|
ServerLocator locator = ActiveMQClient.createServerLocatorWithHA(server1, server2);
|
|
ClientSessionFactory factory = locator.createSessionFactory();
|
|
ClientSession session = factory.createSession();</programlisting>
|
|
</section>
|
|
</section>
|
|
</section>
|
|
</section>
|
|
<section>
|
|
<title>Server-Side Message Load Balancing</title>
|
|
<para>If cluster connections are defined between nodes of a cluster, then ActiveMQ will load
|
|
balance messages arriving at a particular node from a client.</para>
|
|
<para>Let's take a simple example of a cluster of four nodes A, B, C, and D arranged in a
|
|
<emphasis>symmetric cluster</emphasis> (described in
|
|
<xref linkend="symmetric-cluster"/>). We have a queue called <literal>OrderQueue</literal>
|
|
deployed on each node of the cluster.</para>
|
|
<para>We have client Ca connected to node A, sending orders to the server. We have also have
|
|
order processor clients Pa, Pb, Pc, and Pd connected to each of the nodes A, B, C, D. If
|
|
no cluster connection was defined on node A, then as order messages arrive on node A
|
|
they will all end up in the <literal>OrderQueue</literal> on node A, so will only get
|
|
consumed by the order processor client attached to node A, Pa.</para>
|
|
<para>If we define a cluster connection on node A, then as ordered messages arrive on node A
|
|
instead of all of them going into the local <literal>OrderQueue</literal> instance, they
|
|
are distributed in a round-robin fashion between all the nodes of the cluster. The
|
|
messages are forwarded from the receiving node to other nodes of the cluster. This is
|
|
all done on the server side, the client maintains a single connection to node A.</para>
|
|
<para>For example, messages arriving on node A might be distributed in the following order
|
|
between the nodes: B, D, C, A, B, D, C, A, B, D. The exact order depends on the order
|
|
the nodes started up, but the algorithm used is round robin.</para>
|
|
<para>ActiveMQ cluster connections can be configured to always blindly load balance messages
|
|
in a round robin fashion irrespective of whether there are any matching consumers on
|
|
other nodes, but they can be a bit cleverer than that and also be configured to only
|
|
distribute to other nodes if they have matching consumers. We'll look at both these
|
|
cases in turn with some examples, but first we'll discuss configuring cluster
|
|
connections in general.</para>
|
|
<section id="clusters.cluster-connections">
|
|
<title>Configuring Cluster Connections</title>
|
|
<para>Cluster connections group servers into clusters so that messages can be load
|
|
balanced between the nodes of the cluster. Let's take a look at a typical cluster
|
|
connection. Cluster connections are always defined in <literal
|
|
>activemq-configuration.xml</literal> inside a <literal
|
|
>cluster-connection</literal> element. There can be zero or more cluster
|
|
connections defined per ActiveMQ server.</para>
|
|
<programlisting>
|
|
<cluster-connections>
|
|
<cluster-connection name="my-cluster">
|
|
<address>jms</address>
|
|
<connector-ref>netty-connector</connector-ref>
|
|
<check-period>1000</check-period>
|
|
<connection-ttl>5000</connection-ttl>
|
|
<min-large-message-size>50000</min-large-message-size>
|
|
<call-timeout>5000</call-timeout>
|
|
<retry-interval>500</retry-interval>
|
|
<retry-interval-multiplier>1.0</retry-interval-multiplier>
|
|
<max-retry-interval>5000</max-retry-interval>
|
|
<initial-connect-attempts>-1</initial-connect-attempts>
|
|
<reconnect-attempts>-1</reconnect-attempts>
|
|
<use-duplicate-detection>true</use-duplicate-detection>
|
|
<forward-when-no-consumers>false</forward-when-no-consumers>
|
|
<max-hops>1</max-hops>
|
|
<confirmation-window-size>32000</confirmation-window-size>
|
|
<call-failover-timeout>30000</call-failover-timeout>
|
|
<notification-interval>1000</notification-interval>
|
|
<notification-attempts>2</notification-attempts>
|
|
<discovery-group-ref discovery-group-name="my-discovery-group"/>
|
|
</cluster-connection>
|
|
</cluster-connections></programlisting>
|
|
<para>In the above cluster connection all parameters have been explicitly specified. The following
|
|
shows all the available configuration options</para>
|
|
<itemizedlist>
|
|
<listitem id="clusters.address">
|
|
<para><literal>address</literal> Each cluster connection only applies to addresses that match the
|
|
specified address field. An address is matched on the cluster connection when it begins with the
|
|
string specified in this field. The address field on a cluster connection also supports comma
|
|
separated lists and an exclude syntax '!'. To prevent an address from being matched on this
|
|
cluster connection, prepend a cluster connection address string with '!'.</para>
|
|
<para>In the case shown above the cluster connection will load balance messages sent to
|
|
addresses that start with <literal>jms</literal>. This cluster connection,
|
|
will, in effect apply to all JMS queues and topics since they map to core
|
|
queues that start with the substring "jms".</para>
|
|
<para>The address can be any value and you can have many cluster connections
|
|
with different values of <literal>address</literal>, simultaneously
|
|
balancing messages for those addresses, potentially to different clusters of
|
|
servers. By having multiple cluster connections on different addresses a
|
|
single ActiveMQ Server can effectively take part in multiple clusters
|
|
simultaneously.</para>
|
|
<para>Be careful not to have multiple cluster connections with overlapping
|
|
values of <literal>address</literal>, e.g. "europe" and "europe.news" since
|
|
this could result in the same messages being distributed between more than
|
|
one cluster connection, possibly resulting in duplicate deliveries.</para>
|
|
<para>
|
|
Examples:
|
|
<itemizedlist>
|
|
<listitem><literal>'jms.eu'</literal> matches all addresses starting with 'jms.eu'</listitem>
|
|
<listitem><literal>'!jms.eu'</literal> matches all address except for those starting with
|
|
'jms.eu'</listitem>
|
|
<listitem><literal>'jms.eu.uk,jms.eu.de'</literal> matches all addresses starting with either
|
|
'jms.eu.uk' or 'jms.eu.de'</listitem>
|
|
<listitem><literal>'jms.eu,!jms.eu.uk'</literal> matches all addresses starting with 'jms.eu'
|
|
but not those starting with 'jms.eu.uk'</listitem>
|
|
</itemizedlist>
|
|
Notes:
|
|
<itemizedlist>
|
|
<listitem>Address exclusion will always takes precedence over address inclusion.</listitem>
|
|
<listitem>Address matching on cluster connections does not support wild-card matching.
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>This parameter is mandatory.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>connector-ref</literal>. This is the connector which will be sent to other nodes in
|
|
the cluster so they have the correct cluster topology.</para>
|
|
<para>This parameter is mandatory.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>check-period</literal>. The period (in milliseconds) used to check if the cluster connection
|
|
has failed to receive pings from another server. Default is 30000.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>connection-ttl</literal>. This is how long a cluster connection should stay alive if it
|
|
stops receiving messages from a specific node in the cluster. Default is 60000.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>min-large-message-size</literal>. If the message size (in bytes) is larger than this
|
|
value then it will be split into multiple segments when sent over the network to other cluster
|
|
members. Default is 102400.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>call-timeout</literal>. When a packet is sent via a cluster connection and is a blocking
|
|
call, i.e. for acknowledgements, this is how long it will wait (in milliseconds) for the reply before
|
|
throwing an exception. Default is 30000.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>retry-interval</literal>. We mentioned before that, internally,
|
|
cluster connections cause bridges to be created between the nodes of the
|
|
cluster. If the cluster connection is created and the target node has not
|
|
been started, or say, is being rebooted, then the cluster connections from
|
|
other nodes will retry connecting to the target until it comes back up, in
|
|
the same way as a bridge does.</para>
|
|
<para>This parameter determines the interval in milliseconds between retry
|
|
attempts. It has the same meaning as the <literal>retry-interval</literal>
|
|
on a bridge (as described in <xref linkend="core-bridges"/>).</para>
|
|
<para>This parameter is optional and its default value is <literal>500</literal>
|
|
milliseconds.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>retry-interval-multiplier</literal>. This is a multiplier used to increase the
|
|
<literal>retry-interval</literal> after each reconnect attempt, default is 1.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>max-retry-interval</literal>. The maximum delay (in milliseconds) for retries.
|
|
Default is 2000.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>initial-connect-attempts</literal>. The number of times the system will
|
|
try to connect a node in the cluster initially. If the max-retry is achieved this
|
|
node will be considered permanently down and the system will not route messages
|
|
to this node. Default is -1 (infinite retries).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>reconnect-attempts</literal>. The number of times the system will
|
|
try to reconnect to a node in the cluster. If the max-retry is achieved this node will
|
|
be considered permanently down and the system will stop routing messages to this
|
|
node. Default is -1 (infinite retries).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>use-duplicate-detection</literal>. Internally cluster connections
|
|
use bridges to link the nodes, and bridges can be configured to add a
|
|
duplicate id property in each message that is forwarded. If the target node
|
|
of the bridge crashes and then recovers, messages might be resent from the
|
|
source node. By enabling duplicate detection any duplicate messages will be
|
|
filtered out and ignored on receipt at the target node.</para>
|
|
<para>This parameter has the same meaning as <literal>use-duplicate-detection</literal>
|
|
on a bridge. For more information on duplicate detection, please see
|
|
<xref linkend="duplicate-detection"/>. Default is true.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>forward-when-no-consumers</literal>. This parameter determines
|
|
whether messages will be distributed round robin between other nodes of the
|
|
cluster <emphasis>regardless</emphasis> of whether or not there are matching or
|
|
indeed any consumers on other nodes. </para>
|
|
<para>If this is set to <literal>true</literal> then each incoming message will
|
|
be round robin'd even though the same queues on the other nodes of the
|
|
cluster may have no consumers at all, or they may have consumers that have
|
|
non matching message filters (selectors). Note that ActiveMQ will
|
|
<emphasis>not</emphasis> forward messages to other nodes if there are no
|
|
<emphasis>queues</emphasis> of the same name on the other nodes, even if
|
|
this parameter is set to <literal>true</literal>.</para>
|
|
<para>If this is set to <literal>false</literal> then ActiveMQ will only forward
|
|
messages to other nodes of the cluster if the address to which they are
|
|
being forwarded has queues which have consumers, and if those consumers have
|
|
message filters (selectors) at least one of those selectors must match the
|
|
message.</para>
|
|
<para>Default is false.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>max-hops</literal>. When a cluster connection decides the set of
|
|
nodes to which it might load balance a message, those nodes do not have to
|
|
be directly connected to it via a cluster connection. ActiveMQ can be
|
|
configured to also load balance messages to nodes which might be connected
|
|
to it only indirectly with other ActiveMQ servers as intermediates in a
|
|
chain.</para>
|
|
<para>This allows ActiveMQ to be configured in more complex topologies and still
|
|
provide message load balancing. We'll discuss this more later in this
|
|
chapter.</para>
|
|
<para>The default value for this parameter is <literal>1</literal>, which means
|
|
messages are only load balanced to other ActiveMQ serves which are directly
|
|
connected to this server. This parameter is optional.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>confirmation-window-size</literal>. The size (in bytes) of the window
|
|
used for sending confirmations from the server connected to. So once the server has
|
|
received <literal>confirmation-window-size</literal> bytes it notifies its client,
|
|
default is 1048576. A value of -1 means no window.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>call-failover-timeout</literal>. Similar to <literal>call-timeout</literal> but used
|
|
when a call is made during a failover attempt. Default is -1 (no timeout).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>notification-interval</literal>. How often (in milliseconds) the cluster connection
|
|
should broadcast itself when attaching to the cluster. Default is 1000.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>notification-attempts</literal>. How many times the cluster connection should
|
|
broadcast itself when connecting to the cluster. Default is 2.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>discovery-group-ref</literal>. This parameter determines which
|
|
discovery group is used to obtain the list of other servers in the cluster
|
|
that this cluster connection will make connections to.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
Alternatively if you would like your cluster connections to use a static list of
|
|
servers for discovery then you can do it like this.
|
|
</para>
|
|
<programlisting>
|
|
<cluster-connection name="my-cluster">
|
|
...
|
|
<static-connectors>
|
|
<connector-ref>server0-connector</connector-ref>
|
|
<connector-ref>server1-connector</connector-ref>
|
|
</static-connectors>
|
|
</cluster-connection></programlisting>
|
|
<para>
|
|
Here we have defined 2 servers that we know for sure will that at least one will be available. There may
|
|
be many more servers in the cluster but these will; be discovered via one of these connectors once an
|
|
initial connection has been made.</para>
|
|
</section>
|
|
<section id="clusters.clusteruser">
|
|
<title>Cluster User Credentials</title>
|
|
<para>When creating connections between nodes of a cluster to form a cluster connection,
|
|
ActiveMQ uses a cluster user and cluster password which is defined in <literal
|
|
>activemq-configuration.xml</literal>:</para>
|
|
<programlisting>
|
|
<cluster-user>ACTIVEMQ.CLUSTER.ADMIN.USER</cluster-user>
|
|
<cluster-password>CHANGE ME!!</cluster-password></programlisting>
|
|
<warning>
|
|
<para>It is imperative that these values are changed from their default, or remote
|
|
clients will be able to make connections to the server using the default values.
|
|
If they are not changed from the default, ActiveMQ will detect this and pester
|
|
you with a warning on every start-up.</para>
|
|
</warning>
|
|
</section>
|
|
</section>
|
|
<section id="clusters.client.loadbalancing">
|
|
<title>Client-Side Load balancing</title>
|
|
<para>With ActiveMQ client-side load balancing, subsequent sessions created using a single
|
|
session factory can be connected to different nodes of the cluster. This allows sessions
|
|
to spread smoothly across the nodes of a cluster and not be "clumped" on any particular
|
|
node.</para>
|
|
<para>The load balancing policy to be used by the client factory is configurable. ActiveMQ
|
|
provides four out-of-the-box load balancing policies, and you can also implement your own
|
|
and use that.</para>
|
|
<para>The out-of-the-box policies are</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Round Robin. With this policy the first node is chosen randomly then each
|
|
subsequent node is chosen sequentially in the same order.</para>
|
|
<para>For example nodes might be chosen in the order B, C, D, A, B, C, D, A, B or D,
|
|
A, B, C, D, A, B, C, D or C, D, A, B, C, D, A, B, C.</para>
|
|
<para>Use <literal>org.apache.activemq.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy</literal>
|
|
as the <literal><connection-load-balancing-policy-class-name></literal>.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Random. With this policy each node is chosen randomly.</para>
|
|
<para>Use <literal>org.apache.activemq.api.core.client.loadbalance.RandomConnectionLoadBalancingPolicy</literal>
|
|
as the <literal><connection-load-balancing-policy-class-name></literal>.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Random Sticky. With this policy the first node is chosen randomly and then re-used for subsequent
|
|
connections.</para>
|
|
<para>Use <literal>org.apache.activemq.api.core.client.loadbalance.RandomStickyConnectionLoadBalancingPolicy</literal>
|
|
as the <literal><connection-load-balancing-policy-class-name></literal>.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>First Element. With this policy the "first" (i.e. 0th) node is always returned.</para>
|
|
<para>Use <literal>org.apache.activemq.api.core.client.loadbalance.FirstElementConnectionLoadBalancingPolicy</literal>
|
|
as the <literal><connection-load-balancing-policy-class-name></literal>.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>You can also implement your own policy by implementing the interface <literal
|
|
>org.apache.activemq.api.core.client.loadbalance.ConnectionLoadBalancingPolicy</literal></para>
|
|
<para>Specifying which load balancing policy to use differs whether you are using JMS or the
|
|
core API. If you don't specify a policy then the default will be used which is <literal
|
|
>org.apache.activemq.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy</literal>.</para>
|
|
<para>If you're using JMS, and you're using JNDI on the server to put your JMS connection
|
|
factories into JNDI, then you can specify the load balancing policy directly in the
|
|
<literal>activemq-jms.xml</literal> configuration file on the server as follows:</para>
|
|
<programlisting>
|
|
<connection-factory name="ConnectionFactory">
|
|
<discovery-group-ref discovery-group-name="my-discovery-group"/>
|
|
<entries>
|
|
<entry name="ConnectionFactory"/>
|
|
</entries>
|
|
<connection-load-balancing-policy-class-name>
|
|
org.apache.activemq.api.core.client.loadbalance.RandomConnectionLoadBalancingPolicy
|
|
</connection-load-balancing-policy-class-name>
|
|
</connection-factory></programlisting>
|
|
<para>The above example would deploy a JMS connection factory that uses the random connection load
|
|
balancing policy. </para>
|
|
<para>If you're using JMS but you're instantiating your connection factory directly on the
|
|
client side then you can set the load balancing policy using the setter on the
|
|
<literal>ActiveMQConnectionFactory</literal> before using it:</para>
|
|
<programlisting>
|
|
ConnectionFactory jmsConnectionFactory = ActiveMQJMSClient.createConnectionFactory(...);
|
|
jmsConnectionFactory.setLoadBalancingPolicyClassName("com.acme.MyLoadBalancingPolicy");</programlisting>
|
|
<para>If you're using the core API, you can set the load balancing policy directly on the
|
|
<literal>ServerLocator</literal> instance you are using:</para>
|
|
<programlisting>
|
|
ServerLocator locator = ActiveMQClient.createServerLocatorWithHA(server1, server2);
|
|
locator.setLoadBalancingPolicyClassName("com.acme.MyLoadBalancingPolicy");</programlisting>
|
|
<para>The set of servers over which the factory load balances can be determined in one of
|
|
two ways:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Specifying servers explicitly</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Using discovery.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</section>
|
|
<section>
|
|
<title>Specifying Members of a Cluster Explicitly</title>
|
|
<para>
|
|
Sometimes you want to explicitly define a cluster more explicitly, that is control which
|
|
server connect to each other in the cluster. This is typically used to form non symmetrical clusters
|
|
such as chain cluster or ring clusters. This can only be done using a static list of connectors and is
|
|
configured as follows:
|
|
</para>
|
|
<programlisting>
|
|
<cluster-connection name="my-cluster">
|
|
<address>jms</address>
|
|
<connector-ref>netty-connector</connector-ref>
|
|
<retry-interval>500</retry-interval>
|
|
<use-duplicate-detection>true</use-duplicate-detection>
|
|
<forward-when-no-consumers>true</forward-when-no-consumers>
|
|
<max-hops>1</max-hops>
|
|
<static-connectors allow-direct-connections-only="true">
|
|
<connector-ref>server1-connector</connector-ref>
|
|
</static-connectors>
|
|
</cluster-connection></programlisting>
|
|
<para>
|
|
In this example we have set the attribute <literal>allow-direct-connections-only</literal> which means that
|
|
the only server that this server can create a cluster connection to is server1-connector. This means you can
|
|
explicitly create any cluster topology you want.
|
|
</para>
|
|
</section>
|
|
<section id="clusters.message-redistribution">
|
|
<title>Message Redistribution</title>
|
|
<para>Another important part of clustering is message redistribution. Earlier we learned how
|
|
server side message load balancing round robins messages across the cluster. If <literal
|
|
>forward-when-no-consumers</literal> is false, then messages won't be forwarded to
|
|
nodes which don't have matching consumers, this is great and ensures that messages don't
|
|
arrive on a queue which has no consumers to consume them, however there is a situation
|
|
it doesn't solve: What happens if the consumers on a queue close after the messages have
|
|
been sent to the node? If there are no consumers on the queue the message won't get
|
|
consumed and we have a <emphasis>starvation</emphasis> situation.</para>
|
|
<para>This is where message redistribution comes in. With message redistribution ActiveMQ can
|
|
be configured to automatically <emphasis>redistribute</emphasis> messages from queues
|
|
which have no consumers back to other nodes in the cluster which do have matching
|
|
consumers.</para>
|
|
<para>Message redistribution can be configured to kick in immediately after the last
|
|
consumer on a queue is closed, or to wait a configurable delay after the last consumer
|
|
on a queue is closed before redistributing. By default message redistribution is
|
|
disabled.</para>
|
|
<para>Message redistribution can be configured on a per address basis, by specifying the
|
|
redistribution delay in the address settings, for more information on configuring
|
|
address settings, please see <xref linkend="queue-attributes"/>.</para>
|
|
<para>Here's an address settings snippet from <literal>activemq-configuration.xml</literal>
|
|
showing how message redistribution is enabled for a set of queues:</para>
|
|
<programlisting>
|
|
<address-settings>
|
|
<address-setting match="jms.#">
|
|
<redistribution-delay>0</redistribution-delay>
|
|
</address-setting>
|
|
</address-settings></programlisting>
|
|
<para>The above <literal>address-settings</literal> block would set a <literal
|
|
>redistribution-delay</literal> of <literal>0</literal> for any queue which is bound
|
|
to an address that starts with "jms.". All JMS queues and topic subscriptions are bound
|
|
to addresses that start with "jms.", so the above would enable instant (no delay)
|
|
redistribution for all JMS queues and topic subscriptions.</para>
|
|
<para>The attribute <literal>match</literal> can be an exact match or it can be a string
|
|
that conforms to the ActiveMQ wildcard syntax (described in <xref
|
|
linkend="wildcard-syntax"/>).</para>
|
|
<para>The element <literal>redistribution-delay</literal> defines the delay in milliseconds
|
|
after the last consumer is closed on a queue before redistributing messages from that
|
|
queue to other nodes of the cluster which do have matching consumers. A delay of zero
|
|
means the messages will be immediately redistributed. A value of <literal>-1</literal>
|
|
signifies that messages will never be redistributed. The default value is <literal
|
|
>-1</literal>.</para>
|
|
<para>It often makes sense to introduce a delay before redistributing as it's a common case
|
|
that a consumer closes but another one quickly is created on the same queue, in such a
|
|
case you probably don't want to redistribute immediately since the new consumer will
|
|
arrive shortly.</para>
|
|
</section>
|
|
<section>
|
|
<title>Cluster topologies</title>
|
|
<para>ActiveMQ clusters can be connected together in many different topologies, let's
|
|
consider the two most common ones here</para>
|
|
<section id="symmetric-cluster">
|
|
<title>Symmetric cluster</title>
|
|
<para>A symmetric cluster is probably the most common cluster topology, and you'll be
|
|
familiar with if you've had experience of JBoss Application Server
|
|
clustering.</para>
|
|
<para>With a symmetric cluster every node in the cluster is connected to every other
|
|
node in the cluster. In other words every node in the cluster is no more than one
|
|
hop away from every other node.</para>
|
|
<para>To form a symmetric cluster every node in the cluster defines a cluster connection
|
|
with the attribute <literal>max-hops</literal> set to <literal>1</literal>.
|
|
Typically the cluster connection will use server discovery in order to know what
|
|
other servers in the cluster it should connect to, although it is possible to
|
|
explicitly define each target server too in the cluster connection if, for example,
|
|
UDP is not available on your network.</para>
|
|
<para>With a symmetric cluster each node knows about all the queues that exist on all
|
|
the other nodes and what consumers they have. With this knowledge it can determine
|
|
how to load balance and redistribute messages around the nodes.</para>
|
|
<para>Don't forget <link linkend="copy-warning">this warning</link> when creating a
|
|
symmetric cluster.</para>
|
|
</section>
|
|
<section>
|
|
<title>Chain cluster</title>
|
|
<para>With a chain cluster, each node in the cluster is not connected to every node in
|
|
the cluster directly, instead the nodes form a chain with a node on each end of the
|
|
chain and all other nodes just connecting to the previous and next nodes in the
|
|
chain.</para>
|
|
<para>An example of this would be a three node chain consisting of nodes A, B and C.
|
|
Node A is hosted in one network and has many producer clients connected to it
|
|
sending order messages. Due to corporate policy, the order consumer clients need to
|
|
be hosted in a different network, and that network is only accessible via a third
|
|
network. In this setup node B acts as a mediator with no producers or consumers on
|
|
it. Any messages arriving on node A will be forwarded to node B, which will in turn
|
|
forward them to node C where they can get consumed. Node A does not need to directly
|
|
connect to C, but all the nodes can still act as a part of the cluster.</para>
|
|
<para>To set up a cluster in this way, node A would define a cluster connection that
|
|
connects to node B, and node B would define a cluster connection that connects to
|
|
node C. In this case we only want cluster connections in one direction since we're
|
|
only moving messages from node A->B->C and never from C->B->A.</para>
|
|
<para>For this topology we would set <literal>max-hops</literal> to <literal
|
|
>2</literal>. With a value of <literal>2</literal> the knowledge of what queues and
|
|
consumers that exist on node C would be propagated from node C to node B to node A.
|
|
Node A would then know to distribute messages to node B when they arrive, even
|
|
though node B has no consumers itself, it would know that a further hop away is node
|
|
C which does have consumers.</para>
|
|
</section>
|
|
</section>
|
|
<section>
|
|
<title>Scaling Down</title>
|
|
<para>ActiveMQ supports scaling down a cluster with no message loss (even for non-durable messages). This is especially
|
|
useful in certain environments (e.g. the cloud) where the size of a cluster may change relatively frequently.
|
|
When scaling up a cluster (i.e. adding nodes) there is no risk of message loss, but when scaling down a cluster
|
|
(i.e. removing nodes) the messages on those nodes would be lost unless the broker sent them to another node in
|
|
the cluster. ActiveMQ can be configured to do just that.</para>
|
|
<para>The simplest way to enable this behavior is to set <literal>scale-down</literal> to
|
|
<literal>true</literal>. If the server is clustered and <literal>scale-down</literal> is
|
|
<literal>true</literal> then when the server is shutdown gracefully (i.e. stopped without crashing) it will find
|
|
another node in the cluster and send <emphasis>all</emphasis> of its messages (both durable and non-durable)
|
|
to that node. The messages are processed in order and go to the <emphasis>back</emphasis> of the respective
|
|
queues on the other node (just as if the messages were sent from an external client for the first time).</para>
|
|
<para>If more control over where the messages go is required then specify <literal>scale-down-group-name</literal>.
|
|
Messages will only be sent to another node in the cluster that uses the same <literal>scale-down-group-name</literal>
|
|
as the server being shutdown.</para>
|
|
<warning>
|
|
<para>If cluster nodes are grouped together with different <literal>scale-down-group-name</literal> values beware.
|
|
If all the nodes in a single group are shut down then the messages from that node/group will be lost.</para>
|
|
</warning>
|
|
<para>If the server is using multiple <literal>cluster-connection</literal> then use <literal>scale-down-clustername</literal>
|
|
to identify the name of the <literal>cluster-connection</literal> which should be used for scaling down.</para>
|
|
</section>
|
|
</chapter>
|