187 lines
6.6 KiB
Markdown
187 lines
6.6 KiB
Markdown
# Ring Queue
|
||
|
||
Queues operate with first-in, first-out (FIFO) semantics which means that
|
||
messages, in general, are added to the "tail" of the queue and removed from the
|
||
"head." A "ring" queue is a special type of queue with a *fixed* size. The
|
||
fixed size is maintained by removing the message at the head of the queue when
|
||
the number of messages on the queue reaches the configured size.
|
||
|
||
For example, consider a queue configured with a ring size of 3 and a producer
|
||
which sends the messages `A`, `B`, `C`, & `D` in that order. Once `C` is sent
|
||
the number of messages in the queue will be 3 which is the same as the
|
||
configured ring size. We can visualize the queue growth like this...
|
||
|
||
After `A` is sent:
|
||
```
|
||
|---|
|
||
head/tail -> | A |
|
||
|---|
|
||
```
|
||
|
||
After `B` is sent:
|
||
|
||
```
|
||
|---|
|
||
head -> | A |
|
||
|---|
|
||
tail -> | B |
|
||
|---|
|
||
```
|
||
|
||
After `C` is sent:
|
||
|
||
```
|
||
|---|
|
||
head -> | A |
|
||
|---|
|
||
| B |
|
||
|---|
|
||
tail -> | C |
|
||
|---|
|
||
```
|
||
|
||
When `D` is sent it will be added to the tail of the queue and the message at
|
||
the head of the queue (i.e. `A`) will be removed so the queue will look like
|
||
this:
|
||
|
||
```
|
||
|---|
|
||
head -> | B |
|
||
|---|
|
||
| C |
|
||
|---|
|
||
tail -> | D |
|
||
|---|
|
||
```
|
||
|
||
This example covers the most basic use case with messages being added to the
|
||
tail of the queue. However, there are a few other important use cases
|
||
involving:
|
||
|
||
- Messages in delivery & rollbacks
|
||
- Scheduled messages
|
||
- Paging
|
||
|
||
However, before we get to those use cases let's look at the basic configuration
|
||
of a ring queue.
|
||
|
||
## Configuration
|
||
|
||
There are 2 parameters related to ring queue configuration.
|
||
|
||
The `ring-size` parameter can be set directly on the `queue` element. The
|
||
default value comes from the `default-ring-size` `address-setting` (see below).
|
||
|
||
```xml
|
||
<addresses>
|
||
<address name="myRing">
|
||
<anycast>
|
||
<queue name="myRing" ring-size="3" />
|
||
</anycast>
|
||
</address>
|
||
</addresses>
|
||
```
|
||
|
||
The `default-ring-size` is an `address-setting` which applies to queues on
|
||
matching addresses which don't have an explicit `ring-size` set. This is
|
||
especially useful for auto-created queues. The default value is `-1` (i.e.
|
||
no limit).
|
||
|
||
```xml
|
||
<address-settings>
|
||
<address-setting match="ring.#">
|
||
<default-ring-size>3</default-ring-size>
|
||
</address-setting>
|
||
</address-settings>
|
||
```
|
||
|
||
The `ring-size` may be updated at runtime. If the new `ring-size` is set
|
||
*lower* than the previous `ring-size` the broker will not immediately delete
|
||
enough messages from the head of the queue to enforce the new size. New
|
||
messages sent to the queue will force the deletion of old messages (i.e. the
|
||
queue won't grow any larger), but the queue will not reach its new size until
|
||
it does so *naturally* through the normal consumption of messages by
|
||
clients.
|
||
|
||
## Messages in Delivery & Rollbacks
|
||
|
||
When messages are "in delivery" they are in an in-between state where they are
|
||
not technically on the queue but they are also not yet acknowledged. The
|
||
broker is at the consumer’s mercy to either acknowledge such messages or not.
|
||
In the context of a ring queue, messages which are in-delivery cannot be
|
||
removed from the queue.
|
||
|
||
This presents a few dilemmas.
|
||
|
||
Due to the nature of messages in delivery a client can actually send more
|
||
messages to a ring queue than it would otherwise permit. This can make it
|
||
appear that the ring-size is not being enforced properly. Consider this
|
||
simple scenario:
|
||
|
||
- Queue `foo` with `ring-size="3"`
|
||
- 1 Consumer on queue `foo`
|
||
- Message `A` sent to `foo` & dispatched to consumer
|
||
- `messageCount`=1, `deliveringCount`=1
|
||
- Message `B` sent to `foo` & dispatched to consumer
|
||
- `messageCount`=2, `deliveringCount`=2
|
||
- Message `C` sent to `foo` & dispatched to consumer
|
||
- `messageCount`=3, `deliveringCount`=3
|
||
- Message `D` sent to `foo` & dispatched to consumer
|
||
- `messageCount`=4, `deliveringCount`=4
|
||
|
||
The `messageCount` for `foo` is now 4, one *greater* than the `ring-size`
|
||
of 3! However, the broker has no choice but to allow this because it cannot
|
||
remove messages from the queue which are in delivery.
|
||
|
||
Now consider that the consumer is closed without actually acknowledging any
|
||
of these 4 messages. These 4 in-delivery, unacknowledged messages will be
|
||
cancelled back to the broker and added to the *head* of the queue in the
|
||
reverse order from which they were consumed. This, of course, will put the
|
||
queue over its configured `ring-size`. Therefore, since a ring queue
|
||
prefers messages at the tail of the queue over messages at the head it will
|
||
keep `B`, `C`, & `D` and delete `A` (since `A` was the last message added
|
||
to the head of the queue).
|
||
|
||
Transaction or core session rollbacks are treated the same way.
|
||
|
||
If you wish to avoid these kinds of situations and you're using the core
|
||
client directly or the core JMS client you can minimize messages in delivery
|
||
by reducing the size of `consumerWindowSize` (1024 * 1024 bytes by default).
|
||
|
||
## Scheduled Messages
|
||
|
||
When a scheduled message is sent to a queue it isn't immediately added to the
|
||
tail of the queue like normal messages. It is held in an intermediate buffer
|
||
and scheduled for delivery onto the *head* of the queue according to the
|
||
details of the message. However, scheduled messages are nevertheless reflected
|
||
in the message count of the queue. As with messages which are in delivery this
|
||
can make it appear that the ring queue's size is not being enforced. Consider
|
||
this simple scenario:
|
||
|
||
- Queue `foo` with `ring-size="3"`
|
||
- At 12:00 message `A` sent to `foo` scheduled for 12:05
|
||
- `messageCount`=1, `scheduledCount`=1
|
||
- At 12:01 message `B` sent to `foo`
|
||
- `messageCount`=2, `scheduledCount`=1
|
||
- At 12:02 message `C` sent to `foo`
|
||
- `messageCount`=3, `scheduledCount`=1
|
||
- At 12:03 message `D` sent to `foo`
|
||
- `messageCount`=4, `scheduledCount`=1
|
||
|
||
The `messageCount` for `foo` is now 4, one *greater* than the `ring-size` of 3!
|
||
However, the scheduled message is not technically on the queue yet (i.e. it is
|
||
on the broker and scheduled to be put on the queue). When the scheduled
|
||
delivery time for 12:05 comes the message will put on the head of the queue,
|
||
but since the ring queue's size has already been reach the scheduled message
|
||
`A` will be removed.
|
||
|
||
## Paging
|
||
|
||
Similar to scheduled messages and messages in delivery, paged messages don't
|
||
count against a ring queue's size because messages are actually paged at the
|
||
*address* level, not the queue level. A paged message is not technically on a
|
||
queue although it is reflected in a queue's `messageCount`.
|
||
|
||
It is recommended that paging is not used for addresses with ring queues. In
|
||
other words, ensure that the entire address will be able to fit into memory or
|
||
use the `DROP`, `BLOCK` or `FAIL` `address-full-policy`. |