217 lines
12 KiB
XML
217 lines
12 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
||
<!-- ============================================================================= -->
|
||
<!-- Copyright © 2009 Red Hat, Inc. and others. -->
|
||
<!-- -->
|
||
<!-- The text of and illustrations in this document are licensed by Red Hat under -->
|
||
<!-- a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). -->
|
||
<!-- -->
|
||
<!-- An explanation of CC-BY-SA is available at -->
|
||
<!-- -->
|
||
<!-- http://creativecommons.org/licenses/by-sa/3.0/. -->
|
||
<!-- -->
|
||
<!-- In accordance with CC-BY-SA, if you distribute this document or an adaptation -->
|
||
<!-- of it, you must provide the URL for the original version. -->
|
||
<!-- -->
|
||
<!-- Red Hat, as the licensor of this document, waives the right to enforce, -->
|
||
<!-- and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent -->
|
||
<!-- permitted by applicable law. -->
|
||
<!-- ============================================================================= -->
|
||
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
|
||
<!ENTITY % BOOK_ENTITIES SYSTEM "ActiveMQ_User_Manual.ent">
|
||
%BOOK_ENTITIES;
|
||
]>
|
||
<chapter id="paging">
|
||
<title>Paging</title>
|
||
<para>ActiveMQ transparently supports huge queues containing millions of messages while the
|
||
server is running with limited memory.</para>
|
||
<para>In such a situation it's not possible to store all of the queues in memory at any one
|
||
time, so ActiveMQ transparently <emphasis>pages</emphasis> messages into and out of memory as
|
||
they are needed, thus allowing massive queues with a low memory footprint.</para>
|
||
<para>ActiveMQ will start paging messages to disk, when the size of all messages in memory for an
|
||
address exceeds a configured maximum size.</para>
|
||
<para>By default, ActiveMQ does not page messages - this must be explicitly configured to
|
||
activate it.</para>
|
||
<section>
|
||
<title>Page Files</title>
|
||
<para>Messages are stored per address on the file system. Each address has an individual
|
||
folder where messages are stored in multiple files (page files). Each file will contain
|
||
messages up to a max configured size (<literal>page-size-bytes</literal>). The system
|
||
will navigate on the files as needed, and it will remove the page file as soon as all
|
||
the messages are acknowledged up to that point.</para>
|
||
<para>Browsers will read through the page-cursor system.</para>
|
||
<para>Consumers with selectors will also navigate through the page-files and it will ignore
|
||
messages that don't match the criteria.</para>
|
||
</section>
|
||
<section id="paging.main.config">
|
||
<title>Configuration</title>
|
||
<para>You can configure the location of the paging folder</para>
|
||
<para>Global paging parameters are specified on the main configuration file (<literal
|
||
>activemq-configuration.xml</literal>).</para>
|
||
<programlisting>
|
||
<configuration xmlns="urn:activemq"
|
||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||
xsi:schemaLocation="urn:activemq /schema/activemq-configuration.xsd">
|
||
...
|
||
<paging-directory>/somewhere/paging-directory</paging-directory>
|
||
...</programlisting>
|
||
<para>
|
||
<table frame="topbot">
|
||
<title>Paging Configuration Parameters</title>
|
||
<tgroup cols="3">
|
||
<colspec colname="c1" colnum="1"/>
|
||
<colspec colname="c2" colnum="2"/>
|
||
<colspec colname="c3" colnum="3"/>
|
||
<thead>
|
||
<row>
|
||
<entry>Property Name</entry>
|
||
<entry>Description</entry>
|
||
<entry>Default</entry>
|
||
</row>
|
||
</thead>
|
||
<tbody>
|
||
<row>
|
||
<entry><literal>paging-directory</literal></entry>
|
||
<entry>Where page files are stored. ActiveMQ will create one folder for
|
||
each address being paged under this configured location.</entry>
|
||
<entry>data/paging</entry>
|
||
</row>
|
||
</tbody>
|
||
</tgroup>
|
||
</table>
|
||
</para>
|
||
</section>
|
||
<section id="paging.mode">
|
||
<title>Paging Mode</title>
|
||
<para>As soon as messages delivered to an address exceed the configured size, that address
|
||
alone goes into page mode.</para>
|
||
<note>
|
||
<para>Paging is done individually per address. If you configure a max-size-bytes for an
|
||
address, that means each matching address will have a maximum size that you
|
||
specified. It DOES NOT mean that the total overall size of all matching addresses is
|
||
limited to max-size-bytes.</para>
|
||
</note>
|
||
<section>
|
||
<title>Configuration</title>
|
||
<para>Configuration is done at the address settings, done at the main configuration file
|
||
(<literal>activemq-configuration.xml</literal>).</para>
|
||
<programlisting>
|
||
<address-settings>
|
||
<address-setting match="jms.someaddress">
|
||
<max-size-bytes>104857600</max-size-bytes>
|
||
<page-size-bytes>10485760</page-size-bytes>
|
||
<address-full-policy>PAGE</address-full-policy>
|
||
</address-setting>
|
||
</address-settings></programlisting>
|
||
<para>This is the list of available parameters on the address settings.</para>
|
||
<para>
|
||
<table frame="topbot">
|
||
<title>Paging Address Settings</title>
|
||
<tgroup cols="3">
|
||
<colspec colname="c1" colnum="1"/>
|
||
<colspec colname="c2" colnum="2"/>
|
||
<colspec colname="c3" colnum="3"/>
|
||
<thead>
|
||
<row>
|
||
<entry>Property Name</entry>
|
||
<entry>Description</entry>
|
||
<entry>Default</entry>
|
||
</row>
|
||
</thead>
|
||
<tbody>
|
||
<row>
|
||
<entry><literal>max-size-bytes</literal></entry>
|
||
<entry>What's the max memory the address could have before entering
|
||
on page mode.</entry>
|
||
<entry>-1 (disabled)</entry>
|
||
</row>
|
||
<row>
|
||
<entry><literal>page-size-bytes</literal></entry>
|
||
<entry>The size of each page file used on the paging system</entry>
|
||
<entry>10MiB (10 * 1024 * 1024 bytes)</entry>
|
||
</row>
|
||
<row>
|
||
<entry><literal>address-full-policy</literal></entry>
|
||
<entry>This must be set to PAGE for paging to enable. If the value
|
||
is PAGE then further messages will be paged to disk. If the
|
||
value is DROP then further messages will be silently dropped. If
|
||
the value is FAIL then the messages will be dropped and the client
|
||
message producers will receive an exception. If the value is
|
||
BLOCK then client message producers will block when they try and
|
||
send further messages.</entry>
|
||
<entry>PAGE</entry>
|
||
</row>
|
||
<row>
|
||
<entry><literal>page-max-cache-size</literal></entry>
|
||
<entry>The system will keep up to <<literal
|
||
>page-max-cache-size</literal> page files in memory to
|
||
optimize IO during paging navigation.</entry>
|
||
<entry>5</entry>
|
||
</row>
|
||
</tbody>
|
||
</tgroup>
|
||
</table>
|
||
</para>
|
||
</section>
|
||
</section>
|
||
<section>
|
||
<title>Dropping messages</title>
|
||
<para>Instead of paging messages when the max size is reached, an address can also be
|
||
configured to just drop messages when the address is full.</para>
|
||
<para>To do this just set the <literal>address-full-policy</literal> to <literal
|
||
>DROP</literal> in the address settings</para>
|
||
</section>
|
||
<section>
|
||
<title>Dropping messages and throwing an exception to producers</title>
|
||
<para>Instead of paging messages when the max size is reached, an address can also be
|
||
configured to drop messages and also throw an exception on the client-side
|
||
when the address is full.</para>
|
||
<para>To do this just set the <literal>address-full-policy</literal> to <literal
|
||
>FAIL</literal> in the address settings</para>
|
||
</section>
|
||
<section>
|
||
<title>Blocking producers</title>
|
||
<para>Instead of paging messages when the max size is reached, an address can also be
|
||
configured to block producers from sending further messages when the address is full,
|
||
thus preventing the memory being exhausted on the server.</para>
|
||
<para>When memory is freed up on the server, producers will automatically unblock and be
|
||
able to continue sending.</para>
|
||
<para>To do this just set the <literal>address-full-policy</literal> to <literal
|
||
>BLOCK</literal> in the address settings</para>
|
||
<para>In the default configuration, all addresses are configured to block producers after 10
|
||
MiB of data are in the address.</para>
|
||
</section>
|
||
<section>
|
||
<title>Caution with Addresses with Multiple Queues</title>
|
||
<para>When a message is routed to an address that has multiple queues bound to it, e.g. a
|
||
JMS subscription in a Topic, there is only 1 copy of the message in memory. Each queue
|
||
only deals with a reference to this. Because of this the memory is only freed up once
|
||
all queues referencing the message have delivered it.</para>
|
||
<para>If you have a single lazy subscription, the entire address will suffer IO performance
|
||
hit as all the queues will have messages being sent through an extra storage on the
|
||
paging system.</para>
|
||
<para>For example:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>An address has 10 queues </para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>One of the queues does not deliver its messages (maybe because of a slow
|
||
consumer).</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Messages continually arrive at the address and paging is started.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>The other 9 queues are empty even though messages have been sent.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>In this example all the other 9 queues will be consuming messages from the page
|
||
system. This may cause performance issues if this is an undesirable state.</para>
|
||
</section>
|
||
<section>
|
||
<title>Example</title>
|
||
<para>See <xref linkend="examples.paging"/> for an example which shows how to use paging
|
||
with ActiveMQ.</para>
|
||
</section>
|
||
</chapter>
|