208 lines
13 KiB
XML
208 lines
13 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
||
<!-- ============================================================================= -->
|
||
<!-- Copyright © 2009 Red Hat, Inc. and others. -->
|
||
<!-- -->
|
||
<!-- The text of and illustrations in this document are licensed by Red Hat under -->
|
||
<!-- a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). -->
|
||
<!-- -->
|
||
<!-- An explanation of CC-BY-SA is available at -->
|
||
<!-- -->
|
||
<!-- http://creativecommons.org/licenses/by-sa/3.0/. -->
|
||
<!-- -->
|
||
<!-- In accordance with CC-BY-SA, if you distribute this document or an adaptation -->
|
||
<!-- of it, you must provide the URL for the original version. -->
|
||
<!-- -->
|
||
<!-- Red Hat, as the licensor of this document, waives the right to enforce, -->
|
||
<!-- and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent -->
|
||
<!-- permitted by applicable law. -->
|
||
<!-- ============================================================================= -->
|
||
|
||
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
|
||
<!ENTITY % BOOK_ENTITIES SYSTEM "ActiveMQ_User_Manual.ent">
|
||
%BOOK_ENTITIES;
|
||
]>
|
||
<chapter id="message-grouping">
|
||
<title>Message Grouping</title>
|
||
<para>Message groups are sets of messages that have the following characteristics:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Messages in a message group share the same group id, i.e. they have same group
|
||
identifier property (<literal>JMSXGroupID</literal> for JMS, <literal
|
||
>_HQ_GROUP_ID</literal> for ActiveMQ Core API).</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Messages in a message group are always consumed by the same consumer, even if there
|
||
are many consumers on a queue. They pin all messages with the same group id to the same
|
||
consumer. If that consumer closes another consumer is chosen and will receive all
|
||
messages with the same group id.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>Message groups are useful when you want all messages for a certain value of the property to
|
||
be processed serially by the same consumer.</para>
|
||
<para>An example might be orders for a certain stock. You may want orders for any particular
|
||
stock to be processed serially by the same consumer. To do this you can create a pool of
|
||
consumers (perhaps one for each stock, but less will work too), then set the stock name as the
|
||
value of the _HQ_GROUP_ID property.</para>
|
||
<para>This will ensure that all messages for a particular stock will always be processed by the
|
||
same consumer.</para>
|
||
<note>
|
||
<para>Grouped messages can impact the concurrent processing of non-grouped messages due to the
|
||
underlying FIFO semantics of a queue. For example, if there is a chunk of 100 grouped messages at
|
||
the head of a queue followed by 1,000 non-grouped messages then all the grouped messages will need
|
||
to be sent to the appropriate client (which is consuming those grouped messages serially) before
|
||
any of the non-grouped messages can be consumed. The functional impact in this scenario is a
|
||
temporary suspension of concurrent message processing while all the grouped messages are processed.
|
||
This can be a performance bottleneck so keep it in mind when determining the size of your message
|
||
groups, and consider whether or not you should isolate your grouped messages from your non-grouped
|
||
messages.</para>
|
||
</note>
|
||
<section>
|
||
<title>Using Core API</title>
|
||
<para>The property name used to identify the message group is <literal
|
||
>"_HQ_GROUP_ID"</literal> (or the constant <literal
|
||
>MessageImpl.HDR_GROUP_ID</literal>). Alternatively, you can set <literal
|
||
>autogroup</literal> to true on the <literal>SessionFactory</literal> which will pick a
|
||
random unique id. </para>
|
||
</section>
|
||
<section id="message-grouping.jmsconfigure">
|
||
<title>Using JMS</title>
|
||
<para>The property name used to identify the message group is <literal
|
||
>JMSXGroupID</literal>.</para>
|
||
<programlisting>
|
||
// send 2 messages in the same group to ensure the same
|
||
// consumer will receive both
|
||
Message message = ...
|
||
message.setStringProperty("JMSXGroupID", "Group-0");
|
||
producer.send(message);
|
||
|
||
message = ...
|
||
message.setStringProperty("JMSXGroupID", "Group-0");
|
||
producer.send(message);</programlisting>
|
||
<para>Alternatively, you can set <literal>autogroup</literal> to true on the <literal
|
||
>ActiveMQConnectonFactory</literal> which will pick a random unique id. This can also be
|
||
set in the <literal>activemq-jms.xml</literal> file like this:</para>
|
||
<programlisting>
|
||
<connection-factory name="ConnectionFactory">
|
||
<connectors>
|
||
<connector-ref connector-name="netty-connector"/>
|
||
</connectors>
|
||
<entries>
|
||
<entry name="ConnectionFactory"/>
|
||
</entries>
|
||
<autogroup>true</autogroup>
|
||
</connection-factory></programlisting>
|
||
<para>Alternatively you can set the group id via the connection factory. All messages sent
|
||
with producers created via this connection factory will set the <literal
|
||
>JMSXGroupID</literal> to the specified value on all messages sent. To configure the
|
||
group id set it on the connection factory in the <literal>activemq-jms.xml</literal> config
|
||
file as follows
|
||
<programlisting>
|
||
<connection-factory name="ConnectionFactory">
|
||
<connectors>
|
||
<connector-ref connector-name="netty-connector"/>
|
||
</connectors>
|
||
<entries>
|
||
<entry name="ConnectionFactory"/>
|
||
</entries>
|
||
<group-id>Group-0</group-id>
|
||
</connection-factory></programlisting></para>
|
||
</section>
|
||
<section>
|
||
<title>Example</title>
|
||
<para>See <xref linkend="examples.message-group"/> for an example which shows how message
|
||
groups are configured and used with JMS.</para>
|
||
</section>
|
||
<section>
|
||
<title>Example</title>
|
||
<para>See <xref linkend="examples.message-group2"/> for an example which shows how message
|
||
groups are configured via a connection factory.</para>
|
||
</section>
|
||
<section>
|
||
<title> Clustered Grouping</title>
|
||
<para>Using message groups in a cluster is a bit more complex. This is because messages with a
|
||
particular group id can arrive on any node so each node needs to know about which group
|
||
id's are bound to which consumer on which node. The consumer handling messages for a
|
||
particular group id may be on a different node of the cluster, so each node needs to know
|
||
this information so it can route the message correctly to the node which has that consumer. </para>
|
||
<para>To solve this there is the notion of a grouping handler. Each node will have its own
|
||
grouping handler and when a messages is sent with a group id assigned, the handlers will
|
||
decide between them which route the message should take.</para>
|
||
<para id="message-grouping.type">There are 2 types of handlers; Local and Remote. Each cluster should choose 1 node to
|
||
have a local grouping handler and all the other nodes should have remote handlers- it's the
|
||
local handler that actually makes the decision as to what route should be used, all the
|
||
other remote handlers converse with this. Here is a sample config for both types of
|
||
handler, this should be configured in the <emphasis role="italic"
|
||
>activemq-configuration.xml</emphasis>
|
||
file.<programlisting>
|
||
<grouping-handler name="my-grouping-handler">
|
||
<type>LOCAL</type>
|
||
<address>jms</address>
|
||
<timeout>5000</timeout>
|
||
</grouping-handler>
|
||
|
||
<grouping-handler name="my-grouping-handler">
|
||
<type>REMOTE</type>
|
||
<address>jms</address>
|
||
<timeout>5000</timeout>
|
||
</grouping-handler></programlisting></para>
|
||
<para id="message-grouping.address">The <emphasis role="italic">address</emphasis> attribute refers to a
|
||
<link linkend="clusters.address">cluster connection and the address it uses</link>,
|
||
refer to the clustering section on how to configure clusters. The
|
||
<emphasis role="italic">timeout</emphasis> attribute referees to how long to wait for a
|
||
decision to be made, an exception will be thrown during the send if this timeout is
|
||
reached, this ensures that strict ordering is kept.</para>
|
||
<para>The decision as to where a message should be routed to is initially proposed by the node
|
||
that receives the message. The node will pick a suitable route as per the normal clustered
|
||
routing conditions, i.e. round robin available queues, use a local queue first and choose a
|
||
queue that has a consumer. If the proposal is accepted by the grouping handlers the node
|
||
will route messages to this queue from that point on, if rejected an alternative route will
|
||
be offered and the node will again route to that queue indefinitely. All other nodes will
|
||
also route to the queue chosen at proposal time. Once the message arrives at the queue then
|
||
normal single server message group semantics take over and the message is pinned to a
|
||
consumer on that queue.</para>
|
||
<para>You may have noticed that there is a single point of failure with the single local
|
||
handler. If this node crashes then no decisions will be able to be made. Any messages sent
|
||
will be not be delivered and an exception thrown. To avoid this happening Local Handlers
|
||
can be replicated on another backup node. Simple create your back up node and configure it
|
||
with the same Local handler.</para>
|
||
<para/>
|
||
<section>
|
||
<title>Clustered Grouping Best Practices</title>
|
||
<para>Some best practices should be followed when using clustered grouping:<orderedlist>
|
||
<listitem>
|
||
<para>Make sure your consumers are distributed evenly across the different nodes
|
||
if possible. This is only an issue if you are creating and closing consumers
|
||
regularly. Since messages are always routed to the same queue once pinned,
|
||
removing a consumer from this queue may leave it with no consumers meaning the
|
||
queue will just keep receiving the messages. Avoid closing consumers or make
|
||
sure that you always have plenty of consumers, i.e., if you have 3 nodes have 3
|
||
consumers.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Use durable queues if possible. If queues are removed once a group is bound
|
||
to it, then it is possible that other nodes may still try to route messages to
|
||
it. This can be avoided by making sure that the queue is deleted by the session
|
||
that is sending the messages. This means that when the next message is sent it
|
||
is sent to the node where the queue was deleted meaning a new proposal can
|
||
successfully take place. Alternatively you could just start using a different
|
||
group id.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Always make sure that the node that has the Local Grouping Handler is
|
||
replicated. These means that on failover grouping will still occur.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>In case you are using group-timeouts, the remote node should have a smaller group-timeout with at least half
|
||
of the value on the main coordinator. This is because this will determine how often the last-time-use
|
||
value should be updated with a round trip for a request to the group between the nodes.</para>
|
||
</listitem>
|
||
</orderedlist></para>
|
||
</section>
|
||
<section>
|
||
<title>Clustered Grouping Example</title>
|
||
<para>See <xref linkend="examples.clustered.grouping"/> for an example of how to configure
|
||
message groups with a ActiveMQ cluster</para>
|
||
</section>
|
||
</section>
|
||
</chapter>
|