2014-12-11 12:17:29 +00:00
|
|
|
# Last-Value Queues
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
Last-Value queues are special queues which discard any messages when a newer
|
|
|
|
message with the same value for a well-defined Last-Value property is put in
|
|
|
|
the queue. In other words, a Last-Value queue only retains the last value.
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
A typical example for Last-Value queue is for stock prices, where you are only
|
|
|
|
interested by the latest value for a particular stock.
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
Messages sent to an Last-Value queue without the specified property will be
|
|
|
|
delivered as normal and will never be "replaced".
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2018-03-08 14:46:38 -06:00
|
|
|
## Configuration
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2018-10-14 22:02:13 +01:00
|
|
|
#### Last Value Key Configuration
|
2022-02-21 14:59:56 -06:00
|
|
|
Last-Value queues can be statically configured in broker.xml via the
|
|
|
|
`last-value-key`
|
2018-10-14 22:02:13 +01:00
|
|
|
|
|
|
|
```xml
|
|
|
|
<address name="foo.bar">
|
|
|
|
<multicast>
|
|
|
|
<queue name="orders1" last-value-key="reuters_code" />
|
|
|
|
</multicast>
|
|
|
|
</address>
|
|
|
|
```
|
|
|
|
|
|
|
|
Specified on creating a queue by using the CORE api specifying the parameter
|
|
|
|
`lastValue` to `true`.
|
|
|
|
|
|
|
|
Or on auto-create when using the JMS Client by using address parameters when
|
|
|
|
creating the destination used by the consumer.
|
|
|
|
|
|
|
|
```java
|
|
|
|
Queue queue = session.createQueue("my.destination.name?last-value-key=reuters_code");
|
|
|
|
Topic topic = session.createTopic("my.destination.name?last-value-key=reuters_code");
|
|
|
|
```
|
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
Address wildcards can be used to configure Last-Value queues for a set of
|
|
|
|
addresses (see [here](wildcard-syntax.md)).
|
2018-10-14 22:02:13 +01:00
|
|
|
|
|
|
|
```xml
|
|
|
|
<address-setting match="lastValueQueue">
|
|
|
|
<default-last-value-key>reuters_code</default-last-value-key>
|
|
|
|
</address-setting>
|
|
|
|
```
|
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
By default, `default-last-value-key` is `null`.
|
2018-10-14 22:02:13 +01:00
|
|
|
|
|
|
|
#### Legacy Last Value Configuration
|
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
Last-Value queues can also just be configured via the `last-value` boolean
|
|
|
|
property, doing so it will default the last-value-key to `_AMQ_LVQ_NAME`.
|
2018-02-20 08:39:37 +00:00
|
|
|
|
|
|
|
```xml
|
2018-03-09 09:07:38 -06:00
|
|
|
<address name="foo.bar">
|
|
|
|
<multicast>
|
2018-10-14 22:02:13 +01:00
|
|
|
<queue name="orders1" last-value="true" />
|
2018-03-09 09:07:38 -06:00
|
|
|
</multicast>
|
|
|
|
</address>
|
2018-02-20 08:39:37 +00:00
|
|
|
```
|
|
|
|
|
2018-03-09 09:07:38 -06:00
|
|
|
Specified on creating a queue by using the CORE api specifying the parameter
|
|
|
|
`lastValue` to `true`.
|
2018-02-20 08:39:37 +00:00
|
|
|
|
2018-03-09 09:07:38 -06:00
|
|
|
Or on auto-create when using the JMS Client by using address parameters when
|
|
|
|
creating the destination used by the consumer.
|
2018-02-20 08:39:37 +00:00
|
|
|
|
2018-03-09 09:07:38 -06:00
|
|
|
```java
|
|
|
|
Queue queue = session.createQueue("my.destination.name?last-value=true");
|
|
|
|
Topic topic = session.createTopic("my.destination.name?last-value=true");
|
|
|
|
```
|
2018-02-20 08:39:37 +00:00
|
|
|
|
2018-03-09 09:07:38 -06:00
|
|
|
Also the default for all queues under and address can be defaulted using the
|
|
|
|
`address-setting` configuration:
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2018-03-08 14:46:38 -06:00
|
|
|
```xml
|
|
|
|
<address-setting match="lastValueQueue">
|
|
|
|
<default-last-value-queue>true</default-last-value-queue>
|
|
|
|
</address-setting>
|
|
|
|
```
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2018-02-20 08:39:37 +00:00
|
|
|
By default, `default-last-value-queue` is false.
|
|
|
|
|
2018-03-09 09:07:38 -06:00
|
|
|
Note that `address-setting` `last-value-queue` config is deprecated, please use
|
|
|
|
`default-last-value-queue` instead.
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2018-03-08 14:46:38 -06:00
|
|
|
## Last-Value Property
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2018-10-14 22:02:13 +01:00
|
|
|
The property name used to identify the last value is configurable
|
|
|
|
at the queue level mentioned above.
|
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
If using the legacy setting to configure an LVQ then the default property
|
|
|
|
`"_AMQ_LVQ_NAME"` is used (or the constant `Message.HDR_LAST_VALUE_NAME` from
|
|
|
|
the Core API).
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2018-10-14 22:02:13 +01:00
|
|
|
For example, using the sample configuration
|
|
|
|
|
|
|
|
```xml
|
|
|
|
<address name="foo.bar">
|
|
|
|
<multicast>
|
|
|
|
<queue name="orders1" last-value-key="reuters_code" />
|
|
|
|
</multicast>
|
|
|
|
</address>
|
|
|
|
```
|
|
|
|
|
|
|
|
if two messages with the same value for the Last-Value
|
2014-12-04 15:25:29 +00:00
|
|
|
property are sent to a Last-Value queue, only the latest message will be
|
|
|
|
kept in the queue:
|
|
|
|
|
2018-03-08 14:46:38 -06:00
|
|
|
```java
|
2018-10-14 22:02:13 +01:00
|
|
|
// send 1st message with Last-Value property `reuters_code` set to `VOD`
|
2014-12-11 12:17:29 +00:00
|
|
|
TextMessage message = session.createTextMessage("1st message with Last-Value property set");
|
2018-10-14 22:02:13 +01:00
|
|
|
message.setStringProperty("reuters_code", "VOD");
|
2014-12-11 12:17:29 +00:00
|
|
|
producer.send(message);
|
|
|
|
|
2018-10-14 22:02:13 +01:00
|
|
|
// send 2nd message with Last-Value property `reuters_code` set to `VOD`
|
2014-12-11 12:17:29 +00:00
|
|
|
message = session.createTextMessage("2nd message with Last-Value property set");
|
2018-10-14 22:02:13 +01:00
|
|
|
message.setStringProperty("reuters_code", "VOD");
|
2014-12-11 12:17:29 +00:00
|
|
|
producer.send(message);
|
2015-01-23 14:28:07 +00:00
|
|
|
|
2014-12-11 12:17:29 +00:00
|
|
|
...
|
2015-01-23 14:28:07 +00:00
|
|
|
|
|
|
|
// only the 2nd message will be received: it is the latest with
|
2014-12-11 12:17:29 +00:00
|
|
|
// the Last-Value property set
|
|
|
|
TextMessage messageReceived = (TextMessage)messageConsumer.receive(5000);
|
|
|
|
System.out.format("Received message: %s\n", messageReceived.getText());
|
|
|
|
```
|
|
|
|
|
2018-10-14 22:02:13 +01:00
|
|
|
## Forcing all consumers to be non-destructive
|
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
When a consumer attaches to a queue, the normal behaviour is that messages are
|
|
|
|
sent to that consumer are acquired exclusively by that consumer, and when the
|
|
|
|
consumer acknowledges them, the messages are removed from the queue.
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
Another common pattern is to have queue "browsers" which send all messages to
|
|
|
|
the browser, but do not prevent other consumers from receiving the messages,
|
|
|
|
and do not remove them from the queue when the browser is done with them. Such
|
|
|
|
a browser is an instance of a "non-destructive" consumer.
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
If every consumer on a queue is non destructive then we can obtain some
|
|
|
|
interesting behaviours. In the case of a LVQ then the queue will always contain
|
|
|
|
the most up to date value for every key.
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
A queue can be created to enforce all consumers are non-destructive for last
|
|
|
|
value queue. This can be achieved using the following queue configuration:
|
2018-10-14 22:02:13 +01:00
|
|
|
|
|
|
|
```xml
|
|
|
|
<address name="foo.bar">
|
|
|
|
<multicast>
|
|
|
|
<queue name="orders1" last-value-key="reuters_code" non-destructive="true" />
|
|
|
|
</multicast>
|
|
|
|
</address>
|
|
|
|
```
|
|
|
|
|
|
|
|
Or on auto-create when using the JMS Client by using address parameters when
|
|
|
|
creating the destination used by the consumer.
|
|
|
|
|
|
|
|
```java
|
|
|
|
Queue queue = session.createQueue("my.destination.name?last-value-key=reuters_code&non-destructive=true");
|
|
|
|
Topic topic = session.createTopic("my.destination.name?last-value-key=reuters_code&non-destructive=true");
|
|
|
|
```
|
|
|
|
|
|
|
|
Also the default for all queues under and address can be defaulted using the
|
|
|
|
`address-setting` configuration:
|
|
|
|
|
|
|
|
```xml
|
|
|
|
<address-setting match="lastValueQueue">
|
|
|
|
<default-last-value-key>reuters_code</default-last-value-key>
|
|
|
|
<default-non-destructive>true</default-non-destructive>
|
|
|
|
</address-setting>
|
|
|
|
```
|
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
By default, `default-non-destructive` is `false`.
|
|
|
|
|
|
|
|
#### Bounding size using `expiry-delay`
|
|
|
|
|
|
|
|
For queues other than LVQs, having only non-destructive consumers could mean
|
|
|
|
that messages would never get deleted, leaving the queue to grow unconstrained.
|
|
|
|
To prevent this you can use the ability to set a default `expiry-delay`.
|
|
|
|
|
|
|
|
See [expiry-delay](message-expiry.md#configuring-expiry-delay) for more details
|
|
|
|
on this.
|
|
|
|
|
|
|
|
## Clustering
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
The fundamental ideas behind last-value queues and clustering are at odds with
|
|
|
|
each other.
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
Clustering was designed as a way to increase message throughput through
|
|
|
|
horizontal scaling. The messages in a clustered queue can be spread across
|
|
|
|
_all_ nodes in the cluster. This allows clients to be distributed across the
|
|
|
|
cluster to leverage the computing resources all the nodes rather than being
|
|
|
|
bottlenecked on a single node.
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
However, if you wanted to use a last-value queue in a cluster then in order
|
|
|
|
to enforce last-value semantics all messages would be required to go to a
|
|
|
|
queue on a _single_ node. This would effectively _nullify_ the benefits of
|
|
|
|
clustering. Also, the arrival of messages on and and redistribution of
|
|
|
|
those messages from nodes other than the node where the last-value semantics
|
|
|
|
would be enforced would almost certainly impact which message is considered
|
|
|
|
"last."
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2022-02-21 14:59:56 -06:00
|
|
|
For these reasons last-value queues are not supported in a traditional cluster.
|
|
|
|
However, it would be possible to use a [broker balancer](broker-balancers.md)
|
|
|
|
in front of a cluster (or even a set of non-clustered brokers) to ensure all
|
|
|
|
clients which need to use the same last-value queue are directed to the same
|
|
|
|
node. See the [broker balancer chapter](broker-balancers.md) for more details
|
|
|
|
on configuration, etc.
|
2018-10-14 22:02:13 +01:00
|
|
|
|
2014-12-11 12:17:29 +00:00
|
|
|
## Example
|
2014-12-04 15:25:29 +00:00
|
|
|
|
2018-03-09 09:07:38 -06:00
|
|
|
See the [last-value queue example](examples.md#last-value-queue) which shows
|
|
|
|
how last value queues are configured and used with JMS.
|