activemq-artemis/docs/user-manual/address-model.adoc

468 lines
18 KiB
Plaintext
Raw Normal View History

ARTEMIS-4383 migrate user docs to AsciiDoc Markdown, which is currently used for user-facing documentation, is good for a lot of things. However, it's not great for the kind of complex documentation we have and our need to produce both multi-page HTML and single-page PDF output via Maven. Markdown lacks features which would make the documentation easier to read, easier to navigate, and just look better overall. The current tool-chain uses honkit and a tool called Calibre. Honkit is written in TypeScript and is installed via NPM. Calibre is a native tool so it must be installed via an OS-specific package manager. All this complexity makes building, releasing, uploading, etc. a pain. AsciiDoc is relatively simple like Markdown, but it has more features for presentation and navigation not to mention Java-based Maven tooling to generate both HTML and PDF. Migrating will improve both the appearance of the documentation as well as the processes to generate and upload it. This commit contains the following changes: - Convert all the Markdown for the User Manual, Migration Guide, and Hacking guide to AsciiDoc via kramdown [1]. - Update the `artemis-website` build to use AsciiDoctor Maven tooling. - Update `RELEASING.md` with simplified instructions. - Update Hacking Guide with simplified instructions. - Use AsciiDoc link syntax in Artemis Maven doc plugin. - Drop EPUB & MOBI docs for User Manual as well as PDF for the Hacking Guide. All docs will be HTML only except for the User Manual which will have PDF. - Move all docs up out of their respective "en" directory. This was a hold-over from when we had docs in different languages. - Migration & Hacking Guides are now single-page HTML since they are relatively short. - Refactor README.md to simplify and remove redundant content. Benefits of the change: - Much simplified tooling. No more NPM packages or native tools. - Auto-generated table of contents for every chapter. - Auto-generated anchor links for every sub-section. - Overall more appealing presentation. - All docs will use the ActiveMQ favicon. - No more manual line-wrapping! AsciiDoc recommends one sentence per line and paragraphs are separated by a blank line. - AsciiDoctor plugins for IDEA are quite good. - Resulting HTML is less than *half* of the previous size. All previous links/bookmarks should continue to work. [1] https://github.com/asciidoctor/kramdown-asciidoc
2023-07-27 23:45:17 -04:00
= Address Model
:idprefix:
:idseparator: -
Every messaging protocol and API that Apache ActiveMQ Artemis supports defines a different set of messaging resources.
* JMS uses _queues_ and _topics_
* STOMP uses generic _destinations_
* MQTT uses _topics_
* AMQP uses generic _nodes_
In order to deal the the unique semantics and use-cases for each of these the broker has a flexible and powerful address model based on the following _core_ set of resources:
* *address*
* *queue*
* *routing type*
== Address
Messages are _sent_ to an address.
An address is given a unique name, a routing type, and zero or more queues.
== Queue
Messages are _consumed_ from a queue.
A queue is bound to an address.
It is given a unique name and a routing type.
There can be zero or more queues bound to one address.
When a message is sent to an address it is routed to one or more of its queues based on the configured routing type.
The name of the queue must be _globally_ unique.
For example, you can't have a queue named `q1` on address `a1` and also a queue named `q1` address `a2`.
== Routing Type
A routing type determines how messages are routed from an address to the queue(s) bound to that address.
Two different routing types are supported, *anycast* and *multicast*.
|===
| If you want your messages routed to... | Use this routing type...
| a single queue on the address | anycast
| every queue on the address | multicast
|===
WARNING: It is possible to define queues with a different routing type for the same address, but this typically results in an anti-pattern and is therefore not recommended.
== Automatic Configuration
By default Apache ActiveMQ Artemis will automatically create addresses and queues to support the semantics of whatever protocol you're using.
The broker understands how to support each protocol's functionality with the core resources so that in most cases no manual configuration is required.
This saves you from having to preconfigure each address and queue before a client can connect to it.
The broker can optionally be configured to automatically delete addresses and queues when they are no longer in use.
Automatic creation and deletion is configured on a per address basis and is controlled by the following xref:address-settings.adoc#address-settings[`address-setting`] elements:
* `auto-create-addresses`
* `auto-delete-addresses`
* `default-address-routing-type`
* `auto-create-queues`
* `auto-delete-queues`
* `default-queue-routing-type`
include::_auto-queue-creation-note.adoc[]
ARTEMIS-4383 migrate user docs to AsciiDoc Markdown, which is currently used for user-facing documentation, is good for a lot of things. However, it's not great for the kind of complex documentation we have and our need to produce both multi-page HTML and single-page PDF output via Maven. Markdown lacks features which would make the documentation easier to read, easier to navigate, and just look better overall. The current tool-chain uses honkit and a tool called Calibre. Honkit is written in TypeScript and is installed via NPM. Calibre is a native tool so it must be installed via an OS-specific package manager. All this complexity makes building, releasing, uploading, etc. a pain. AsciiDoc is relatively simple like Markdown, but it has more features for presentation and navigation not to mention Java-based Maven tooling to generate both HTML and PDF. Migrating will improve both the appearance of the documentation as well as the processes to generate and upload it. This commit contains the following changes: - Convert all the Markdown for the User Manual, Migration Guide, and Hacking guide to AsciiDoc via kramdown [1]. - Update the `artemis-website` build to use AsciiDoctor Maven tooling. - Update `RELEASING.md` with simplified instructions. - Update Hacking Guide with simplified instructions. - Use AsciiDoc link syntax in Artemis Maven doc plugin. - Drop EPUB & MOBI docs for User Manual as well as PDF for the Hacking Guide. All docs will be HTML only except for the User Manual which will have PDF. - Move all docs up out of their respective "en" directory. This was a hold-over from when we had docs in different languages. - Migration & Hacking Guides are now single-page HTML since they are relatively short. - Refactor README.md to simplify and remove redundant content. Benefits of the change: - Much simplified tooling. No more NPM packages or native tools. - Auto-generated table of contents for every chapter. - Auto-generated anchor links for every sub-section. - Overall more appealing presentation. - All docs will use the ActiveMQ favicon. - No more manual line-wrapping! AsciiDoc recommends one sentence per line and paragraphs are separated by a blank line. - AsciiDoctor plugins for IDEA are quite good. - Resulting HTML is less than *half* of the previous size. All previous links/bookmarks should continue to work. [1] https://github.com/asciidoctor/kramdown-asciidoc
2023-07-27 23:45:17 -04:00
See xref:address-settings.adoc#address-settings[the documentation on address settings] for more details on these elements.
Of course, automatic configuration can be disabled and everything can be configured manually.
Read on for more details about manual configuration.
== Basic Manual Configuration
The following examples show how to configure resources for basic anycast and multicast use-cases.
NOTE: Many of the details of these use-cases are protocol agnostic.
The goal here is to demonstrate and explain the basic configuration elements and how the address model works fundamentally.
=== Anycast
The most common use-case for anycast semantics, sometimes referred to as xref:messaging-concepts.adoc#point-to-point[point-to-point], involves applications following a "competing consumer" pattern to receive messages from a shared queue.
The more consumers receiving messages the greater the overall message throughput.
Multiple Java applications sharing a JMS queue is a classic example of this use-case.
In this use-case the broker is configured, for example, with an address, `address.foo` using the `anycast` routing type with just one queue, `q1`.
When a producer sends a message to `address.foo` it is then routed to `q1` and finally dispatched to one of the consumers.
image:images/anycast.png[Anycast] Figure 1.
Anycast
This is what the configuration for this use-case would look like in `etc/broker.xml`:
[,xml]
----
<addresses>
<address name="address.foo">
<anycast>
<queue name="q1"/>
</anycast>
</address>
</addresses>
----
For most protocols and APIs which support this kind of use-case (e.g. JMS, AMQP, etc.) it is customary to use the _same name_ when sending and consuming messages.
In that case you'd use a configuration like this:
[,xml]
----
<addresses>
<address name="orderQueue">
<anycast>
<queue name="orderQueue"/>
</anycast>
</address>
</addresses>
----
=== Multicast
The most common use-case for multicast semantics, sometimes referred to as xref:messaging-concepts.adoc#publish-subscribe[publish/subscribe] or "pub/sub", involves each application receiving every message sent to an address.
Multiple applications consuming from a JMS topic is a classic example of this use-case.
MQTT subscriptions is another supported example of multicast semantics.
In this use-case the broker is configured with an address, `address.foo` using the `multicast` routing type with two queues, `q1` & `q2`.
When a producer sends a message to `address.foo` it is then routed to _both_ `q1` & `q2` so that ultimately both consumers receive the same messages.
image:images/multicast.png[Multicast] Figure 2.
Multicast
This is what the configuration for this use-case would look like in `etc/broker.xml`:
[,xml]
----
<addresses>
<address name="address.foo">
<multicast>
<queue name="q1"/>
<queue name="q2"/>
</multicast>
</address>
</addresses>
----
This basic configuration is simple and straight-forward, but there's a problem.
In a normal pub/sub use-case like with a JMS topic or with MQTT the number of subscribers _isn't known ahead of time_.
In that case, this is the recommended configuration:
[,xml]
----
<addresses>
<address name="address.foo">
<multicast/>
</address>
</addresses>
----
Define `<multicast/>` with no queues and the broker will automatically create queues for each subscription when the consumers connect to `address.foo`.
Then when a message is sent to `address.foo` it will be routed to each queue for each subscriber and therefore each subscriber will get every message.
These queues are often referred to as _subscription queues_ for obvious reasons.
These subscription queues are typically named based on the semantics of the protocol used to create them.
For example, JMS supports durable and non-durable subscriptions.
The queue for a non-durable subscription is named with a UUID, but the queue used for a durable subscription is named according to the JMS "client ID" and "subscription name." Similar conventions are used for AMQP, MQTT, STOMP, etc.
== Advanced Manual Configuration
=== Fully Qualified Queue Names
In most cases it's not necessary or desirable to statically configure the aforementioned subscription queues.
However, there are scenarios where a user may want to statically configure a subscription queue and later connect to that queue directly using a *Fully Qualified Queue Name* (FQQN).
An FQQN uses a special syntax to specify _both_ the address and the queue so that applications using protocols and APIs which don't natively understand the address/queue separation (e.g. AMQP, JMS, etc.) can send messages or subscribe _directly_ to a queue rather than being limited to the address.
Applications simply need to use the address name and the queue name separated by `::` (e.g. `address::queue`).
In this example, the address `a1` is configured with two queues: `q1`, `q2` as shown in the configuration below.
[,xml]
----
<addresses>
<address name="a1">
<multicast>
<queue name="q1" />
<queue name="q2" />
</multicast>
</address>
</addresses>
----
Here's a snippet of Java code using JMS which demonstrates the FQQN syntax:
[,java]
----
Queue q1 session.createQueue("a1::q1");
MessageConsumer consumer = session.createConsumer(q1);
----
NOTE: The string `::` should only be used for FQQN and not in any other context in address or queue names.
The examples below show how to use broker side configuration to statically configure a queue with publish subscribe behavior for shared, non-shared, durable and non-durable subscription behavior.
==== Shared, Durable Subscription Queue using `max-consumers`
The default behavior for queues is to not limit the number connected queue consumers.
The `max-consumers` parameter of the queue element can be used to limit the number of connected consumers allowed at any one time.
Open the file `etc/broker.xml` for editing.
[,xml]
----
<addresses>
<address name="durable.foo">
<multicast>
<!-- pre-configured shared durable subscription queue -->
<queue name="q1" max-consumers="10">
<durable>true</durable>
</queue>
</multicast>
</address>
</addresses>
----
==== Non-shared, Durable Subscription Queue
The broker can be configured to prevent more than one consumer from connecting to a queue at any one time.
The subscriptions to queues configured this way are therefore "non-shared".
To do this simply set the `max-consumers` parameter to `1`:
[,xml]
----
<addresses>
<address name="durable.foo">
<multicast>
<!-- pre-configured non shared durable subscription queue -->
<queue name="q1" max-consumers="1">
<durable>true</durable>
</queue>
</multicast>
</address>
</addresses>
----
==== Non-durable Subscription Queue
Non-durable subscriptions are again usually managed by the relevant protocol manager, by creating and deleting temporary queues.
If a user requires to pre-create a queue that behaves like a non-durable subscription queue the `purge-on-no-consumers` flag can be enabled on the queue.
When `purge-on-no-consumers` is set to `true`.
The queue will not start receiving messages until a consumer is attached.
When the last consumer is detached from the queue.
The queue is purged (its messages are removed) and will not receive any more messages until a new consumer is attached.
Open the file `etc/broker.xml` for editing.
[,xml]
----
<addresses>
<address name="non.shared.durable.foo">
<multicast>
<queue name="orders1" purge-on-no-consumers="true"/>
</multicast>
</address>
</addresses>
----
=== Disabled Queue
If a user requires to statically configure a queue and disable routing to it, for example where a queue needs to be defined so a consumer can bind, but you want to disable message routing to it for the time being.
Or you need to stop message flow to the queue to allow investigation keeping the consumer bound, but don't wish to have further messages routed to the queue to avoid message build up.
When `enabled` is set to `true` the queue will have messages routed to it.
(default)
When `enabled` is set to `false` the queue will NOT have messages routed to it.
Open the file `etc/broker.xml` for editing.
[,xml]
----
<addresses>
<address name="foo.bar">
<multicast>
<queue name="orders1" enabled="false"/>
</multicast>
</address>
</addresses>
----
[WARNING]
====
Disabling all the queues on an address means that any message sent to that address will be silently dropped.
====
=== Temporary Queues
For some protocols and APIs which only support monolithic "destinations" without the address/queue separation (e.g. AMQP, JMS, etc.) temporary queues are created by the broker using a UUID (i.e universally unique identifier) as the name for both the address and the queue.
Because the name is a UUID it is impossible to create an `address-setting` for it whose `match` is anything but `#`.
To solve this problem one can specify the `temporary-queue-namespace` in `broker.xml` and then create an `address-setting` whose `match` value corresponds to the configured `temporary-queue-namespace`.
When the `temporary-queue-namespace` is set and a temporary queue is created then the broker will prepend the `temporary-queue-namespace` value along with the `delimiter` value configured in `wildcard-addresses` (defaults to `.`) to the address name and use that to lookup the associated `address-setting` values.
Here's a simple example configuration:
[,xml]
----
<temporary-queue-namespace>temp</temporary-queue-namespace>
<address-settings>
<address-setting match="temp.#">
<enable-metrics>false</enable-metrics>
</address-setting>
</address-settings>
----
Using this configuration any temporary queue will have metrics disabled.
NOTE: This setting does _not_ change the actual name of the temporary queue.
It only changes the name used to _lookup_ the address-settings.
=== Other Advanced Configurations
Each of the following advanced configurations have their own chapter so their details are not repeated here:
* xref:exclusive-queues.adoc#exclusive-queues[Exclusive queues]
* xref:last-value-queues.adoc#last-value-queues[Last Value queues]
* xref:non-destructive-queues.adoc#non-destructive-queues[Non-Destructive queues]
* xref:ring-queues.adoc#ring-queue[Ring queues]
* xref:retroactive-addresses.adoc#retroactive-addresses[Retroactive addresses]
== How to filter messages
Apache ActiveMQ Artemis supports the ability to filter messages using xref:filter-expressions.adoc#filter-expressions[Filter Expressions].
Filters can be applied in two places - on a queue and on a consumer.
Filtering messages on a queue increases performance vs.
filtering on the consumer because the messages don't need to be scanned.
However, a queue filter is often not as flexible.
=== Queue Filter
When a filter is applied to a queue, messages are filtered _before_ they are routed to the queue.
To add a filter use the `filter` element when configuring a queue, e.g.:
[,xml]
----
<addresses>
<address name="filter">
<anycast>
<queue name="filter">
<filter string="color='red'"/>
</queue>
</anycast>
</address>
</addresses>
----
The filter defined above ensures that only messages with an attribute `"color='red'"` is sent to this queue.
=== Consumer Filters
Consumer filters are applied _after_ messages have routed to the queue and are defined using the appropriate client APIs.
The following JMS example shows how consumer filters work.
Define an address with a single queue, with no filter applied in `etc/broker.xml`.
[,xml]
----
<addresses>
<address name="filter">
<anycast>
<queue name="filter"/>
</anycast>
</address>
</addresses>
----
Then send some messages to the queue.
[,java]
----
...
// Send some messages
for (int i = 0; i < 3; i ++) {
TextMessage redMessage = senderSession.createTextMessage("Red");
redMessage.setStringProperty("color", "red");
producer.send(redMessage)
TextMessage greenMessage = senderSession.createTextMessage("Green");
greenMessage.setStringProperty("color", "green");
producer.send(greenMessage)
}
----
At this point the queue would have 6 messages: red, green, red, green, red, green.
Create a consumer with the filter `color='red'`.
[,java]
----
MessageConsumer redConsumer = redSession.createConsumer(queue, "color='red'");
----
The `redConsumer` has a filter that only matches "red" messages.
The `redConsumer` will receive 3 messages.
----
red, red, red
----
The resulting queue would now be
----
green, green, green
----
== Alternate Ways to Determine Routing Type
Typically the routing type is determined either by the static XML configuration or by the `default-address-routing-type` and `default-queue-routing-type` `address-setting` elements used for <<automatic-configuration,automatic address and queue creation>>.
However, there are two other ways to specify routing type:
* a configurable prefix which client applications can use when sending messages or creating consumers
* a property client applications can set on the messages they send
=== Using a Prefix to Determine Routing Type
These prefixes are configured using the `anycastPrefix` and `multicastPrefix` parameters within the URL of the `acceptor` which the client is using.
When multiple values are needed, these can be separated by a comma.
==== Configuring an Anycast Prefix
In `etc/broker.xml`, add the `anycastPrefix` to the URL of the desired `acceptor`.
In the example below, the acceptor is configured to use `queue/` for the `anycastPrefix`.
Client code can specify `queue/foo/` if the client wants anycast routing.
[,xml]
----
<acceptor name="artemis">tcp://0.0.0.0:61616?protocols=AMQP;anycastPrefix=queue/</acceptor>
----
Consider, for example, a STOMP client that wants to send a message using anycast semantics to a queue that doesn't exist.
Consider also that the broker is configured to auto-create addresses and queues, but the `default-address-routing-type` and `default-queue-routing-type` are both `MULTICAST`.
Since the `anycastPrefix` is `queue/` it can just send a message to `queue/foo` and the broker will automatically create an address named `foo` with an anycast queue also named `foo`.
==== Configuring a Multicast Prefix
In `etc/broker.xml`, add the `multicastPrefix` to the URL of the desired `acceptor`.
In the example below, the acceptor is configured to use `topic/` for the `multicastPrefix`.
Client code can specify `topic/foo/` if the client wants multicast routing.
[,xml]
----
<acceptor name="artemis">tcp://0.0.0.0:61616?protocols=AMQP;multicastPrefix=topic/</acceptor>
----
Consider, for example, a STOMP client that wants to create a subscription with multicast semantics on an address that doesn't exist.
Consider also that the broker is configured to auto-create addresses and queues, but the `default-address-routing-type` and `default-queue-routing-type` are both `ANYCAST`.
Since the `multicastPrefix` is `topic/` it can just subscribe to `topic/foo` and the broker will automatically create an address named `foo` with a multicast queue for the subscription.
Any messages sent to `foo` will then be routed to the subscription queue.
=== Using a Message Property to Determine Routing Type
The `_AMQ_ROUTING_TYPE` property represents a `byte` value which will be used by the broker to determine the routing type when a message is _sent_.
Use `0` for anycast routing or `1` for multicast routing.
NOTE: A message will *only* be routed to queues which match its `_AMQ_ROUTING_TYPE` property value (if any).
For example, if a message with an `_AMQ_ROUTING_TYPE` value of `1` (i.e. multicast) is sent to an address that only has anycast queues then the message won't actually be routed to any of the queues since the routing types don't match.
If no `_AMQ_ROUTING_TYPE` is set then the message will be routed to all the queues on the address according to the queues' routing semantics.