207 lines
12 KiB
XML
207 lines
12 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!-- ============================================================================= -->
|
|
<!-- Licensed to the Apache Software Foundation (ASF) under one or more -->
|
|
<!-- contributor license agreements. See the NOTICE file distributed with -->
|
|
<!-- this work for additional information regarding copyright ownership. -->
|
|
<!-- The ASF licenses this file to You under the Apache License, Version 2.0 -->
|
|
<!-- (the "License"); you may not use this file except in compliance with -->
|
|
<!-- the License. You may obtain a copy of the License at -->
|
|
<!-- -->
|
|
<!-- http://www.apache.org/licenses/LICENSE-2.0 -->
|
|
<!-- -->
|
|
<!-- Unless required by applicable law or agreed to in writing, software -->
|
|
<!-- distributed under the License is distributed on an "AS IS" BASIS, -->
|
|
<!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
|
|
<!-- See the License for the specific language governing permissions and -->
|
|
<!-- limitations under the License. -->
|
|
<!-- ============================================================================= -->
|
|
|
|
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
|
|
<!ENTITY % BOOK_ENTITIES SYSTEM "ActiveMQ_User_Manual.ent">
|
|
%BOOK_ENTITIES;
|
|
]>
|
|
<chapter id="message-grouping">
|
|
<title>Message Grouping</title>
|
|
<para>Message groups are sets of messages that have the following characteristics:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Messages in a message group share the same group id, i.e. they have same group
|
|
identifier property (<literal>JMSXGroupID</literal> for JMS, <literal
|
|
>_HQ_GROUP_ID</literal> for ActiveMQ Core API).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Messages in a message group are always consumed by the same consumer, even if there
|
|
are many consumers on a queue. They pin all messages with the same group id to the same
|
|
consumer. If that consumer closes another consumer is chosen and will receive all
|
|
messages with the same group id.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>Message groups are useful when you want all messages for a certain value of the property to
|
|
be processed serially by the same consumer.</para>
|
|
<para>An example might be orders for a certain stock. You may want orders for any particular
|
|
stock to be processed serially by the same consumer. To do this you can create a pool of
|
|
consumers (perhaps one for each stock, but less will work too), then set the stock name as the
|
|
value of the _HQ_GROUP_ID property.</para>
|
|
<para>This will ensure that all messages for a particular stock will always be processed by the
|
|
same consumer.</para>
|
|
<note>
|
|
<para>Grouped messages can impact the concurrent processing of non-grouped messages due to the
|
|
underlying FIFO semantics of a queue. For example, if there is a chunk of 100 grouped messages at
|
|
the head of a queue followed by 1,000 non-grouped messages then all the grouped messages will need
|
|
to be sent to the appropriate client (which is consuming those grouped messages serially) before
|
|
any of the non-grouped messages can be consumed. The functional impact in this scenario is a
|
|
temporary suspension of concurrent message processing while all the grouped messages are processed.
|
|
This can be a performance bottleneck so keep it in mind when determining the size of your message
|
|
groups, and consider whether or not you should isolate your grouped messages from your non-grouped
|
|
messages.</para>
|
|
</note>
|
|
<section>
|
|
<title>Using Core API</title>
|
|
<para>The property name used to identify the message group is <literal
|
|
>"_HQ_GROUP_ID"</literal> (or the constant <literal
|
|
>MessageImpl.HDR_GROUP_ID</literal>). Alternatively, you can set <literal
|
|
>autogroup</literal> to true on the <literal>SessionFactory</literal> which will pick a
|
|
random unique id. </para>
|
|
</section>
|
|
<section id="message-grouping.jmsconfigure">
|
|
<title>Using JMS</title>
|
|
<para>The property name used to identify the message group is <literal
|
|
>JMSXGroupID</literal>.</para>
|
|
<programlisting>
|
|
// send 2 messages in the same group to ensure the same
|
|
// consumer will receive both
|
|
Message message = ...
|
|
message.setStringProperty("JMSXGroupID", "Group-0");
|
|
producer.send(message);
|
|
|
|
message = ...
|
|
message.setStringProperty("JMSXGroupID", "Group-0");
|
|
producer.send(message);</programlisting>
|
|
<para>Alternatively, you can set <literal>autogroup</literal> to true on the <literal
|
|
>ActiveMQConnectonFactory</literal> which will pick a random unique id. This can also be
|
|
set in the <literal>activemq-jms.xml</literal> file like this:</para>
|
|
<programlisting>
|
|
<connection-factory name="ConnectionFactory">
|
|
<connectors>
|
|
<connector-ref connector-name="netty-connector"/>
|
|
</connectors>
|
|
<entries>
|
|
<entry name="ConnectionFactory"/>
|
|
</entries>
|
|
<autogroup>true</autogroup>
|
|
</connection-factory></programlisting>
|
|
<para>Alternatively you can set the group id via the connection factory. All messages sent
|
|
with producers created via this connection factory will set the <literal
|
|
>JMSXGroupID</literal> to the specified value on all messages sent. To configure the
|
|
group id set it on the connection factory in the <literal>activemq-jms.xml</literal> config
|
|
file as follows
|
|
<programlisting>
|
|
<connection-factory name="ConnectionFactory">
|
|
<connectors>
|
|
<connector-ref connector-name="netty-connector"/>
|
|
</connectors>
|
|
<entries>
|
|
<entry name="ConnectionFactory"/>
|
|
</entries>
|
|
<group-id>Group-0</group-id>
|
|
</connection-factory></programlisting></para>
|
|
</section>
|
|
<section>
|
|
<title>Example</title>
|
|
<para>See <xref linkend="examples.message-group"/> for an example which shows how message
|
|
groups are configured and used with JMS.</para>
|
|
</section>
|
|
<section>
|
|
<title>Example</title>
|
|
<para>See <xref linkend="examples.message-group2"/> for an example which shows how message
|
|
groups are configured via a connection factory.</para>
|
|
</section>
|
|
<section>
|
|
<title> Clustered Grouping</title>
|
|
<para>Using message groups in a cluster is a bit more complex. This is because messages with a
|
|
particular group id can arrive on any node so each node needs to know about which group
|
|
id's are bound to which consumer on which node. The consumer handling messages for a
|
|
particular group id may be on a different node of the cluster, so each node needs to know
|
|
this information so it can route the message correctly to the node which has that consumer. </para>
|
|
<para>To solve this there is the notion of a grouping handler. Each node will have its own
|
|
grouping handler and when a messages is sent with a group id assigned, the handlers will
|
|
decide between them which route the message should take.</para>
|
|
<para id="message-grouping.type">There are 2 types of handlers; Local and Remote. Each cluster should choose 1 node to
|
|
have a local grouping handler and all the other nodes should have remote handlers- it's the
|
|
local handler that actually makes the decision as to what route should be used, all the
|
|
other remote handlers converse with this. Here is a sample config for both types of
|
|
handler, this should be configured in the <emphasis role="italic"
|
|
>activemq-configuration.xml</emphasis>
|
|
file.<programlisting>
|
|
<grouping-handler name="my-grouping-handler">
|
|
<type>LOCAL</type>
|
|
<address>jms</address>
|
|
<timeout>5000</timeout>
|
|
</grouping-handler>
|
|
|
|
<grouping-handler name="my-grouping-handler">
|
|
<type>REMOTE</type>
|
|
<address>jms</address>
|
|
<timeout>5000</timeout>
|
|
</grouping-handler></programlisting></para>
|
|
<para id="message-grouping.address">The <emphasis role="italic">address</emphasis> attribute refers to a
|
|
<link linkend="clusters.address">cluster connection and the address it uses</link>,
|
|
refer to the clustering section on how to configure clusters. The
|
|
<emphasis role="italic">timeout</emphasis> attribute referees to how long to wait for a
|
|
decision to be made, an exception will be thrown during the send if this timeout is
|
|
reached, this ensures that strict ordering is kept.</para>
|
|
<para>The decision as to where a message should be routed to is initially proposed by the node
|
|
that receives the message. The node will pick a suitable route as per the normal clustered
|
|
routing conditions, i.e. round robin available queues, use a local queue first and choose a
|
|
queue that has a consumer. If the proposal is accepted by the grouping handlers the node
|
|
will route messages to this queue from that point on, if rejected an alternative route will
|
|
be offered and the node will again route to that queue indefinitely. All other nodes will
|
|
also route to the queue chosen at proposal time. Once the message arrives at the queue then
|
|
normal single server message group semantics take over and the message is pinned to a
|
|
consumer on that queue.</para>
|
|
<para>You may have noticed that there is a single point of failure with the single local
|
|
handler. If this node crashes then no decisions will be able to be made. Any messages sent
|
|
will be not be delivered and an exception thrown. To avoid this happening Local Handlers
|
|
can be replicated on another backup node. Simple create your back up node and configure it
|
|
with the same Local handler.</para>
|
|
<para/>
|
|
<section>
|
|
<title>Clustered Grouping Best Practices</title>
|
|
<para>Some best practices should be followed when using clustered grouping:<orderedlist>
|
|
<listitem>
|
|
<para>Make sure your consumers are distributed evenly across the different nodes
|
|
if possible. This is only an issue if you are creating and closing consumers
|
|
regularly. Since messages are always routed to the same queue once pinned,
|
|
removing a consumer from this queue may leave it with no consumers meaning the
|
|
queue will just keep receiving the messages. Avoid closing consumers or make
|
|
sure that you always have plenty of consumers, i.e., if you have 3 nodes have 3
|
|
consumers.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Use durable queues if possible. If queues are removed once a group is bound
|
|
to it, then it is possible that other nodes may still try to route messages to
|
|
it. This can be avoided by making sure that the queue is deleted by the session
|
|
that is sending the messages. This means that when the next message is sent it
|
|
is sent to the node where the queue was deleted meaning a new proposal can
|
|
successfully take place. Alternatively you could just start using a different
|
|
group id.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Always make sure that the node that has the Local Grouping Handler is
|
|
replicated. These means that on failover grouping will still occur.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>In case you are using group-timeouts, the remote node should have a smaller group-timeout with at least half
|
|
of the value on the main coordinator. This is because this will determine how often the last-time-use
|
|
value should be updated with a round trip for a request to the group between the nodes.</para>
|
|
</listitem>
|
|
</orderedlist></para>
|
|
</section>
|
|
<section>
|
|
<title>Clustered Grouping Example</title>
|
|
<para>See <xref linkend="examples.clustered.grouping"/> for an example of how to configure
|
|
message groups with a ActiveMQ cluster</para>
|
|
</section>
|
|
</section>
|
|
</chapter>
|