150 lines
10 KiB
XML
150 lines
10 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
||
|
||
<!-- ============================================================================= -->
|
||
<!-- Copyright © 2009 Red Hat, Inc. and others. -->
|
||
<!-- -->
|
||
<!-- The text of and illustrations in this document are licensed by Red Hat under -->
|
||
<!-- a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). -->
|
||
<!-- -->
|
||
<!-- An explanation of CC-BY-SA is available at -->
|
||
<!-- -->
|
||
<!-- http://creativecommons.org/licenses/by-sa/3.0/. -->
|
||
<!-- -->
|
||
<!-- In accordance with CC-BY-SA, if you distribute this document or an adaptation -->
|
||
<!-- of it, you must provide the URL for the original version. -->
|
||
<!-- -->
|
||
<!-- Red Hat, as the licensor of this document, waives the right to enforce, -->
|
||
<!-- and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent -->
|
||
<!-- permitted by applicable law. -->
|
||
<!-- ============================================================================= -->
|
||
|
||
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
|
||
<!ENTITY % BOOK_ENTITIES SYSTEM "ActiveMQ_User_Manual.ent">
|
||
%BOOK_ENTITIES;
|
||
]>
|
||
|
||
<chapter id="duplicate-detection">
|
||
<title>Duplicate Message Detection</title>
|
||
<para>ActiveMQ includes powerful automatic duplicate message detection, filtering out
|
||
duplicate messages without you having to code your own fiddly duplicate detection logic at
|
||
the application level. This chapter will explain what duplicate detection is, how ActiveMQ
|
||
uses it and how and where to configure it.</para>
|
||
<para>When sending messages from a client to a server, or indeed from a server to another
|
||
server, if the target server or connection fails sometime after sending the message, but
|
||
before the sender receives a response that the send (or commit) was processed successfully
|
||
then the sender cannot know for sure if the message was sent successfully to the
|
||
address.</para>
|
||
<para>If the target server or connection failed after the send was received and processed but
|
||
before the response was sent back then the message will have been sent to the address
|
||
successfully, but if the target server or connection failed before the send was received and
|
||
finished processing then it will not have been sent to the address successfully. From the
|
||
senders point of view it's not possible to distinguish these two cases.</para>
|
||
<para>When the server recovers this leaves the client in a difficult situation. It knows the
|
||
target server failed, but it does not know if the last message reached its destination ok.
|
||
If it decides to resend the last message, then that could result in a duplicate message
|
||
being sent to the address. If each message was an order or a trade then this could result in
|
||
the order being fulfilled twice or the trade being double booked. This is clearly not a
|
||
desirable situation.</para>
|
||
<para>Sending the message(s) in a transaction does not help out either. If the server or
|
||
connection fails while the transaction commit is being processed it is also indeterminate
|
||
whether the transaction was successfully committed or not!</para>
|
||
<para>To solve these issues ActiveMQ provides automatic duplicate messages detection for
|
||
messages sent to addresses.</para>
|
||
<section>
|
||
<title>Using Duplicate Detection for Message Sending</title>
|
||
<para>Enabling duplicate message detection for sent messages is simple: you just need to set
|
||
a special property on the message to a unique value. You can create the value however
|
||
you like, as long as it is unique. When the target server receives the message it will
|
||
check if that property is set, if it is, then it will check in its in memory cache if it
|
||
has already received a message with that value of the header. If it has received a
|
||
message with the same value before then it will ignore the message.</para>
|
||
<note>
|
||
<para>Using duplicate detection to move messages between nodes can give you the same
|
||
<emphasis>once and only once</emphasis> delivery guarantees as if you were using
|
||
an XA transaction to consume messages from source and send them to the target, but
|
||
with less overhead and much easier configuration than using XA.</para>
|
||
</note>
|
||
<para>If you're sending messages in a transaction then you don't have to set the property
|
||
for <emphasis>every</emphasis> message you send in that transaction, you only need to
|
||
set it once in the transaction. If the server detects a duplicate message for any
|
||
message in the transaction, then it will ignore the entire transaction.</para>
|
||
<para>The name of the property that you set is given by the value of <literal
|
||
>org.apache.activemq.api.core.Message.HDR_DUPLICATE_DETECTION_ID</literal>, which
|
||
is <literal>_HQ_DUPL_ID</literal></para>
|
||
<para>The value of the property can be of type <literal>byte[]</literal> or <literal
|
||
>SimpleString</literal> if you're using the core API. If you're using JMS it must be
|
||
a <literal>String</literal>, and its value should be unique. An easy way of generating
|
||
a unique id is by generating a UUID.</para>
|
||
<para>Here's an example of setting the property using the core API:</para>
|
||
<programlisting>
|
||
...
|
||
|
||
ClientMessage message = session.createMessage(true);
|
||
|
||
SimpleString myUniqueID = "This is my unique id"; // Could use a UUID for this
|
||
|
||
message.setStringProperty(HDR_DUPLICATE_DETECTION_ID, myUniqueID);
|
||
|
||
...</programlisting>
|
||
<para>And here's an example using the JMS API:</para>
|
||
<programlisting>
|
||
...
|
||
|
||
Message jmsMessage = session.createMessage();
|
||
|
||
String myUniqueID = "This is my unique id"; // Could use a UUID for this
|
||
|
||
message.setStringProperty(HDR_DUPLICATE_DETECTION_ID.toString(), myUniqueID);
|
||
|
||
...</programlisting>
|
||
</section>
|
||
<section id="duplicate.id.cache">
|
||
<title>Configuring the Duplicate ID Cache</title>
|
||
<para>The server maintains caches of received values of the <literal
|
||
>org.apache.activemq.core.message.impl.HDR_DUPLICATE_DETECTION_ID</literal> property
|
||
sent to each address. Each address has its own distinct cache.</para>
|
||
<para>The cache is a circular fixed size cache. If the cache has a maximum size of <literal
|
||
>n</literal> elements, then the <literal>n + 1</literal>th id stored will overwrite
|
||
the <literal>0</literal>th element in the cache.</para>
|
||
<para>The maximum size of the cache is configured by the parameter <literal
|
||
>id-cache-size</literal> in <literal>activemq-configuration.xml</literal>, the default
|
||
value is <literal>2000</literal> elements.</para>
|
||
<para>The caches can also be configured to persist to disk or not. This is configured by the
|
||
parameter <literal>persist-id-cache</literal>, also in <literal
|
||
>activemq-configuration.xml</literal>. If this is set to <literal>true</literal> then
|
||
each id will be persisted to permanent storage as they are received. The default value
|
||
for this parameter is <literal>true</literal>.</para>
|
||
<note>
|
||
<para>When choosing a size of the duplicate id cache be sure to set it to a larger
|
||
enough size so if you resend messages all the previously sent ones are in the cache
|
||
not having been overwritten.</para>
|
||
</note>
|
||
</section>
|
||
<section>
|
||
<title>Duplicate Detection and Bridges</title>
|
||
<para>Core bridges can be configured to automatically add a unique duplicate id value (if there
|
||
isn't already one in the message) before forwarding the message to it's target. This
|
||
ensures that if the target server crashes or the connection is interrupted and the
|
||
bridge resends the message, then if it has already been received by the target server,
|
||
it will be ignored.</para>
|
||
<para>To configure a core bridge to add the duplicate id header, simply set the <parameter
|
||
>use-duplicate-detection</parameter> to <literal>true</literal> when configuring a
|
||
bridge in <literal>activemq-configuration.xml</literal>.</para>
|
||
<para>The default value for this parameter is <literal>true</literal>.</para>
|
||
<para>For more information on core bridges and how to configure them, please see
|
||
<xref linkend="core-bridges" />.</para>
|
||
</section>
|
||
<section>
|
||
<title>Duplicate Detection and Cluster Connections</title>
|
||
<para>Cluster connections internally use core bridges to move messages reliable between
|
||
nodes of the cluster. Consequently they can also be configured to insert the duplicate
|
||
id header for each message they move using their internal bridges.</para>
|
||
<para>To configure a cluster connection to add the duplicate id header, simply set the
|
||
<parameter>use-duplicate-detection</parameter> to <literal>true</literal> when
|
||
configuring a cluster connection in <literal>activemq-configuration.xml</literal>.</para>
|
||
<para>The default value for this parameter is <literal>true</literal>.</para>
|
||
<para>For more information on cluster connections and how to configure them, please see <xref
|
||
linkend="clusters"/>.</para>
|
||
</section>
|
||
</chapter>
|