This closes #4938
This commit is contained in:
Justin Bertram
commit
7d6ab48763
|
|
@ -21,7 +21,7 @@ Broker connections are configured by the `<broker-connections>` XML element in t
|
|||
An ActiveMQ Artemis broker can initiate connections using the AMQP protocol.
|
||||
This means that the broker can connect to another AMQP server (not necessarily ActiveMQ Artemis) and create elements on that connection.
|
||||
|
||||
To define an AMQP broker connection, add an `<amqp-connection>` element within the `<broker-connections` element in the `broker.xml` configuration file.
|
||||
To define an AMQP broker connection, add an `<amqp-connection>` element within the `<broker-connections>` element in the `broker.xml` configuration file.
|
||||
For example:
|
||||
|
||||
[,xml]
|
||||
|
|
@ -34,44 +34,43 @@ For example:
|
|||
----
|
||||
|
||||
uri::
|
||||
: tcp://host:myport[?options] (this is a required argument)
|
||||
tcp://host:myport[?options] (this is a required argument)
|
||||
name::
|
||||
: Name of the connection used for management purposes
|
||||
Name of the connection used for management purposes
|
||||
user::
|
||||
: User name with which to connect to the endpoint (this is an optional argument)
|
||||
Username with which to connect to the endpoint (this is an optional argument)
|
||||
password::
|
||||
: Password with which to connect to the endpoint (this is an optional argument)
|
||||
Password with which to connect to the endpoint (this is an optional argument)
|
||||
retry-interval::
|
||||
: Time, in milliseconds to wait before retrying a connection after an error.
|
||||
Time, in milliseconds to wait before retrying a connection after an error.
|
||||
The default value is `5000`.
|
||||
reconnect-attempts::
|
||||
: default is -1 meaning infinite
|
||||
default is -1 meaning infinite
|
||||
auto-start::
|
||||
: Should the broker connection start automatically with the broker.
|
||||
Default is `true`.
|
||||
If false it is necessary to call a management operation to start it.
|
||||
Should the broker connection start automatically with the broker. Default is `true`.
|
||||
If `false` it is necessary to call a management operation to start it.
|
||||
|
||||
NOTE: The connection URI options for transport settings, such as enabling and configuring TLS, are common with other Artemis connector URIs.
|
||||
See xref:configuring-transports.adoc#configuring-netty-ssl[the transport doc] for more.
|
||||
An example configuration for a TLS AMQP broker-connection can be found in the broker xref:examples.adoc[examples] at ./examples/features/broker-connection/amqp-sending-overssl.
|
||||
An example configuration for a TLS AMQP broker-connection can be found in the broker xref:examples.adoc[examples] at _./examples/features/broker-connection/amqp-sending-overssl_.
|
||||
|
||||
NOTE: If auto-start is disabled on the broker connection, the start of the broker connection will only happen after the management method `startBrokerConnection(connectionName)` is called on the ServerController.
|
||||
|
||||
IMPORTANT: The target endpoint needs permission for all operations that are configured.
|
||||
Therefore, If a security manager is being used, ensure that you perform the configured operations with a user with sufficient permissions.
|
||||
Therefore, if a security manager is being used, ensure that you perform the configured operations with a user with sufficient permissions.
|
||||
|
||||
== AMQP Server Connection Operations
|
||||
|
||||
The following types of operations are supported on an AMQP server connection:
|
||||
|
||||
* Mirrors
|
||||
** The broker uses an AMQP connection to another broker and duplicates messages and sends acknowledgements over the wire.
|
||||
** The broker uses an AMQP connection to another broker and duplicates messages and sends acknowledgements over the wire.
|
||||
* Senders
|
||||
** Messages received on specific queues are transferred to another endpoint.
|
||||
** Messages received on specific queues are transferred to another endpoint.
|
||||
* Receivers
|
||||
** The broker pulls messages from another endpoint.
|
||||
** The broker pulls messages from another endpoint.
|
||||
* Peers
|
||||
** The broker creates both senders and receivers on another endpoint that knows how to handle them.
|
||||
** The broker creates both senders and receivers on another endpoint that knows how to handle them.
|
||||
This is currently implemented by Apache Qpid Dispatch.
|
||||
|
||||
== Reconnecting and Failover
|
||||
|
|
@ -80,11 +79,11 @@ It is possible to determine how reconnection will happen on a broker connection.
|
|||
|
||||
These are the attributes that are available on the amqp-connection XML element:
|
||||
|
||||
* reconnect-attempts: default is -1 (infinite).
|
||||
reconnect-attempts:: default is -1 (infinite).
|
||||
How many attempts will be done after a failed connection
|
||||
* retry-interval: default is 5000, in milliseconds, the wait between each retry of a connection.
|
||||
retry-interval:: default is 5000, in milliseconds, the wait between each retry of a connection
|
||||
|
||||
It is also possible to specify alternate hosts on a broker connection by appending a comma separated list after a # at the end of the URI.
|
||||
It is also possible to specify alternate hosts on a broker connection by appending a comma separated list after a `#` at the end of the URI.
|
||||
The broker connection would keep trying on the alternate list until one of the targets is available to connect.
|
||||
Example:
|
||||
|
||||
|
|
@ -110,15 +109,15 @@ Mirroring will reproduce any operation that happened on the source brokers towar
|
|||
The following events are sent through mirroring:
|
||||
|
||||
* Message sending
|
||||
** Messages sent to one broker will be "replicated" to the target broker.
|
||||
** Messages sent to one broker will be "replicated" to the target broker.
|
||||
* Message acknowledgement
|
||||
** Acknowledgements removing messages at one broker will be sent to the target broker.
|
||||
** Note that if the message is pending for a consumer on the target mirror, the ack will not succeed and the message might be delivered by both brokers.
|
||||
** Acknowledgements removing messages at one broker will be sent to the target broker.
|
||||
** Note that if the message is pending for a consumer on the target mirror, the ack will not succeed and the message might be delivered by both brokers.
|
||||
* Queue and address creation.
|
||||
* Queue and address deletion.
|
||||
|
||||
By default every operation is sent asynchronously without blocking any clients.
|
||||
However if you set sync="true" on the mirror configuration, the clients will always wait for the mirror on every blocking operation.
|
||||
By default, every operation is sent asynchronously without blocking any clients.
|
||||
However, if you set `sync=true` on the mirror configuration the clients will always wait for the mirror on every blocking operation.
|
||||
|
||||
=== Mirror configuration
|
||||
|
||||
|
|
@ -126,41 +125,42 @@ Add a `<mirror>` element within the `<amqp-connection>` element to configure mir
|
|||
|
||||
The following optional arguments can be utilized:
|
||||
|
||||
queue-removal::
|
||||
: Specifies whether a queue- or address-removal event is sent.
|
||||
The default value is `true`.
|
||||
message-acknowledgements::
|
||||
: Specifies whether message acknowledgements are sent.
|
||||
Specifies whether message acknowledgements are sent.
|
||||
The default value is `true`.
|
||||
queue-creation::
|
||||
: Specifies whether a queue- or address-creation event is sent.
|
||||
Specifies whether a queue- or address-creation event is sent.
|
||||
The default value is `true`.
|
||||
queue-removal::
|
||||
Specifies whether a queue- or address-removal event is sent.
|
||||
The default value is `true`.
|
||||
address-filter::
|
||||
: An optional comma-separated list of inclusion and/or exclusion filter entries used to govern which addresses (and related queues) mirroring events will be created for on this broker-connection.
|
||||
An optional comma-separated list of inclusion and/or exclusion filter entries used to govern which addresses (and related queues) mirroring events will be created for on this broker-connection.
|
||||
That is, events will only be mirrored to the target broker for addresses that match the filter.
|
||||
An address is matched when it begins with an inclusion entry specified in this field, unless the address is also explicitly excluded by another entry.
|
||||
+
|
||||
An exclusion entry is prefixed with `!` to denote any address beginning with that value does not match.
|
||||
If no inclusion entry is specified in the list, all addresses not explicitly excluded will match.
|
||||
If the address-filter attribute is not specified, then all addresses (and related queues) will match and be mirrored.
|
||||
+
|
||||
Examples:
|
||||
|
||||
** 'eu' matches all addresses starting with 'eu'
|
||||
** '!eu' matches all address except for those starting with 'eu'
|
||||
** 'eu.uk,eu.de' matches all addresses starting with either 'eu.uk' or 'eu.de'
|
||||
** 'eu,!eu.uk' matches all addresses starting with 'eu' but not those starting with 'eu.uk'
|
||||
* `eu` matches all addresses starting with `eu`
|
||||
* `!eu` matches all address except for those starting with `eu`
|
||||
* `eu.uk,eu.de` matches all addresses starting with either `eu.uk` or `eu.de`
|
||||
* `eu,!eu.uk` matches all addresses starting with `eu` but not those starting with `eu.uk`
|
||||
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
** Address exclusion will always take precedence over address inclusion.
|
||||
** Address matching on mirror elements is prefix-based and does not support wild-card matching.
|
||||
* Address exclusion will always take precedence over address inclusion.
|
||||
* Address matching on mirror elements is prefix-based and does not support wild-card matching.
|
||||
====
|
||||
sync::
|
||||
By default is `false`.
|
||||
If set it to true any client blocking operation will be held until the mirror has confirmed receiving the operation.
|
||||
If set to `true` any client blocking operation will be held until the mirror has confirmed receiving the operation.
|
||||
Default is `false`.
|
||||
* Notice that a disconnected node would hold all operations from the client.
|
||||
If you set sync=true you must reconnect a mirror before performing any operations.
|
||||
If you set `sync=true` you must reconnect a mirror before performing any operations.
|
||||
|
||||
An example of a mirror configuration is shown below:
|
||||
|
||||
|
|
@ -175,9 +175,9 @@ An example of a mirror configuration is shown below:
|
|||
|
||||
=== Store and Forward Queue
|
||||
|
||||
Mirror events are always stored on a local queue prefixed as "$ACTIVEMQ_ARTEMIS_MIRROR_" and then concatenated with the broker connection's configured name.
|
||||
Mirror events are always stored on a local queue prefixed as `$ACTIVEMQ_ARTEMIS_MIRROR_` and then concatenated with the broker connection's configured name.
|
||||
|
||||
So, in the following configuration mirror events will be stored on a queue named "$ACTIVEMQ_ARTEMIS_MIRROR_brokerB".
|
||||
So, in the following configuration, mirror events will be stored on a queue named `$ACTIVEMQ_ARTEMIS_MIRROR_brokerB`.
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -188,11 +188,11 @@ So, in the following configuration mirror events will be stored on a queue named
|
|||
</broker-connection>
|
||||
----
|
||||
|
||||
These messages are then transferred to brokerB:5672.
|
||||
A producer to the address $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will be created towards brokerB.
|
||||
These messages are then transferred to _brokerB:5672_.
|
||||
A producer to the address `$ACTIVEMQ_ARTEMIS_MIRROR_brokerB` will be created towards _brokerB_.
|
||||
If there is a security manager configured, security roles must be provided to the user on the broker connection.
|
||||
|
||||
Notice the queue $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will not actually exist on brokerB and so it wont be visible on the administration console.
|
||||
Notice the queue `$ACTIVEMQ_ARTEMIS_MIRROR_brokerB` will not actually exist on _brokerB_ and so it will not be visible on the administration console.
|
||||
The target broker will treat these messages accordingly as mirror events and perform the appropriate operations at the target broker.
|
||||
|
||||
=== Pre Existing Messages
|
||||
|
|
@ -205,13 +205,13 @@ Previously existing messages will not be forwarded to other brokers.
|
|||
ActiveMQ Artemis supports automatic fallback mirroring.
|
||||
Every sent message and every acknowledgement is asynchronously replicated to the mirrored broker.
|
||||
|
||||
On the following diagram, there will be two servers called DataCenter1, and DataCenter2.
|
||||
In order to have a dual mirror configuration, it is necessary is to add the mirror broker connection on each broker.xml:
|
||||
On the following diagram there will be two servers called _DataCenter1_, and _DataCenter2_.
|
||||
In order to have a dual mirror configuration it is necessary to add the mirror broker connection on each `broker.xml`:
|
||||
|
||||
image:images/broker-connection-DR.jpg[Broker Connection DR] Figure 2.
|
||||
Broker Connection - Disaster Recovery.
|
||||
|
||||
on DataCenter1, the following code should be added on broker.xml:
|
||||
On _DataCenter1_ the following code should be added on `broker.xml`:
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -222,7 +222,7 @@ on DataCenter1, the following code should be added on broker.xml:
|
|||
</broker-connections>
|
||||
----
|
||||
|
||||
The following xml should be added on DataCenter2's broker.xml:
|
||||
The following xml snippet should be added on _DataCenter2_'s `broker.xml`:
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -234,16 +234,20 @@ The following xml should be added on DataCenter2's broker.xml:
|
|||
----
|
||||
|
||||
The broker connections will replicate sends and acknowledgements to the other broker, no matter where they originated.
|
||||
If messages are sent on DC1 (DataCenter1) these will be automatically transferred to DC2 (DataCenter2).
|
||||
Messages acknowledgements received on DC2 will be automatically related back to DC1.
|
||||
If messages are sent on DC1 (_DataCenter1_) these will be automatically transferred to DC2 (_DataCenter2_).
|
||||
Message acknowledgements received on DC2 will be automatically related back to DC1.
|
||||
|
||||
The only exception to that rule would be if there were already consumers with pending messages on any server, where a mirrored acknowledgement will not prevent the message being consumed by both consumers.
|
||||
[NOTE]
|
||||
====
|
||||
It is recommended to not have active consumers on both servers.
|
||||
====
|
||||
|
||||
== Example
|
||||
=== Example
|
||||
|
||||
There is an example as part of the distribution showing dual broker configuration (or disaster recovery) under ./examples/features/broker-connection/disaster-recovery.
|
||||
There is an example as part of the distribution showing dual broker configuration (or disaster recovery) in `./examples/features/broker-connection/disaster-recovery`.
|
||||
|
||||
On the example two brokers are configured to mirror each other and whatever happens in one broker is immediately copied over to the other broker.
|
||||
On the provided example above, two brokers are configured to mirror each other and whatever happens in one broker is immediately copied over to the other broker.
|
||||
|
||||
== Senders and Receivers
|
||||
|
||||
|
|
@ -262,11 +266,11 @@ Wildcard expressions can be used to match senders and receivers to specific addr
|
|||
When configuring a sender or receiver, the following properties can be set:
|
||||
|
||||
address-match::
|
||||
: Match the sender or receiver to a specific address or *set* of addresses, using a wildcard expression
|
||||
Match the sender or receiver to a specific address or *set* of addresses, using a wildcard expression.
|
||||
queue-name::
|
||||
: Configure the sender or receiver for a specific queue
|
||||
Configure the sender or receiver for a specific queue.
|
||||
|
||||
Some examples are shown below.
|
||||
=== Examples
|
||||
|
||||
Using address expressions:
|
||||
|
||||
|
|
@ -337,7 +341,7 @@ This replaces the traditional need of a router-initiated connection and auto-lin
|
|||
Qpid Dispatch Router offers a lot of advanced networking options that be used together with ActiveMQ Artemis.
|
||||
|
||||
With a peer configuration, the same properties are present as when there are senders and receivers.
|
||||
For example, a configuration where queues with names beginning "queue." act as storage for the matching router waypoint address would be:
|
||||
For example, a configuration where queues with names beginning `queue.` act as storage for the matching router waypoint address would be:
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -381,7 +385,7 @@ The brokers peer configuration replaces these aspects of the router waypoint usa
|
|||
== Address Consideration
|
||||
|
||||
It is highly recommended that `address name` and `queue name` are the same.
|
||||
When a queue with its distinct name (as in the following example) is used, senders and receivers will always use the `address name` when creating the remote endpoint.
|
||||
When a queue with its distinct name (as in the following example) is used senders and receivers will always use the `address` `name` when creating the remote endpoint.
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -399,15 +403,18 @@ When a queue with its distinct name (as in the following example) is used, sende
|
|||
</addresses>
|
||||
----
|
||||
|
||||
In the above example the `broker connection` would create an AMQP sender towards "queues.A".
|
||||
The above example would create an AMQP sender towards `queues.A`.
|
||||
|
||||
IMPORTANT: To avoid confusion it is recommended that `address name` and `queue name` are kept the same.
|
||||
IMPORTANT: To avoid confusion it is recommended that `address` `name` and `queue` `name` are kept the same.
|
||||
|
||||
== Federation
|
||||
|
||||
Broker federation allows the local broker to create remote receivers for addresses or queues that have local demand, conversely the broker connection can send federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection.
|
||||
Broker federation allows the local broker to create remote receivers for addresses or queues that have local demand.
|
||||
Conversely, the broker connection can send federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection.
|
||||
|
||||
Add a `<federation>` element within the `<amqp-connection>` element to configure federation to the broker instance, the `<amqp-connection>` contains all the configuration for authentication and reconnection handling, see the above sections to configure those values.
|
||||
Add a `<federation>` element within the `<amqp-connection>` element to configure federation to the broker instance.
|
||||
The `<amqp-connection>` contains all the configuration for authentication and reconnection handling.
|
||||
See above sections to configure those values.
|
||||
|
||||
The broker connection federation configuration consists of one or more policies that define either local or remote federation configurations for addresses or queues.
|
||||
|
||||
|
|
@ -435,11 +442,14 @@ The broker connection federation configuration consists of one or more policies
|
|||
</broker-connections>
|
||||
----
|
||||
|
||||
===== Local and remote address federation
|
||||
=== Local and remote address federation
|
||||
|
||||
Local or Remote address federation configures the local or remote broker to watch for demand on addresses and when demand exists it will create a consumer on the matching address on the opposing broker. Because the consumers are created on addresses on the opposing broker the authentication credentials supplied to the broker connection must have sufficient access to read from (local address federation) or write to the matching address (remote address federation) on the opposing broker.
|
||||
Local or Remote address federation configures the local or remote broker to watch for demand on addresses.
|
||||
When demand exists it will create a consumer on the matching address on the opposing broker.
|
||||
Because the consumers are created on addresses on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local address federation) or write to the matching address (remote address federation) on the opposing broker.
|
||||
|
||||
An example of address federation configuration is shown below, the local and remote address policies have identical configuration parameters only the policy element names are different.
|
||||
An example of address federation configuration is shown below.
|
||||
Local and remote address policies have identical configuration parameters, only the policy element names are different.
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -463,25 +473,38 @@ An example of address federation configuration is shown below, the local and rem
|
|||
name::
|
||||
The name of the policy, these names should be unique within a broker connection's federation policy elements.
|
||||
include::
|
||||
The address-match pattern to use to match included addresses, multiple of these can be set. If none are set all addresses are matched.
|
||||
The address-match pattern to use to match included addresses, multiple of these can be set.
|
||||
If none are set all addresses are matched.
|
||||
exclude::
|
||||
The address-match pattern to use to match excluded addresses, multiple of these can be set or it can be omitted if no excludes are needed.
|
||||
The address-match pattern to use to match excluded addresses, multiple of these can be set, or it can be omitted if no excludes are needed.
|
||||
max-hops::
|
||||
The number of hops that a message can have made for it to be federated, the default value is zero and will work for most simple federatio0n deployments however is certain topologies a higher value may be required to prevent looping.
|
||||
The maximum number of hops that a message can perform to be federated.
|
||||
Default value is `0` and will work for most simple federation deployments.
|
||||
However, in certain topologies a higher value may be required to prevent looping.
|
||||
auto-delete::
|
||||
For address federation, a durable queue is created on the opposing broker for the matching address. This option is used to mark if the queue should be deleted once the initiating broker disconnects, and the delay and message count parameters have been met. This is useful if you want to automate the clean up, though you may wish to disable this if you want messages to queued for the opposing broker when it disconnects. The default value is `false` and the federation consumer queue will not be auto deleted.
|
||||
For address federation, a durable queue is created on the broker from which messages are being federated.
|
||||
Set this property to `true` to mark the queue for automatic deletion once the initiating broker disconnects and the delay and message count parameters are met.
|
||||
This is a useful option if you want to automate the cleanup of dynamically-created queues.
|
||||
The default value is `false`, which means that the queue is not automatically deleted.
|
||||
|
||||
auto-delete-delay::
|
||||
The amount of time in milliseconds after the initiating broker has disconnected before the created queue can be eligible for `auto-delete`. The default value is zero if the option is omitted.
|
||||
The amount of time in milliseconds after the initiating broker has disconnected before the created queue can be eligible for `auto-delete`.
|
||||
The default value is `0` when option is omitted.
|
||||
auto-delete-message-count::
|
||||
The amount number messages in the remote queue that the message count must be equal or below before the initiating broker has disconnected before the queue can be eligible for auto deletion. The default value is zero if the option is omitted.
|
||||
After the initiating broker has disconnected, the maximum number of messages allowed in the remote queue for the queue to be eligible for automatic deletion.
|
||||
The default value is `0`.
|
||||
|
||||
enable-divert-bindings::
|
||||
Setting to true will enable divert bindings to be listened for demand. If there is a divert binding with an address that matches the included addresses for the federation, any queue bindings that match the forward address of the divert will create demand. The default value for this option is `false`.
|
||||
Setting to `true` enables divert bindings to be listened-to for demand.
|
||||
If a divert binding with an address matches the included addresses for the address policy, any queue bindings that match the forwarding address of the divert creates demand. The default value is `false`.
|
||||
|
||||
===== Local and remote queue federation
|
||||
=== Local and remote queue federation
|
||||
|
||||
Local or Remote queue federation configures the local or remote broker to watch for demand on queues and when demand exists it will create a consumer on the matching queue on the opposing broker. Because the consumers are created on queues on the opposing broker the authentication credentials supplied to the broker connection must have sufficient access to read from (local queue federation) or write to the matching queue (remote queue federation) on the opposing broker.
|
||||
Local or Remote queue federation configures the local or remote broker to watch for demand on queues and when demand exists it will create a consumer on the matching queue on the opposing broker.
|
||||
Because the consumers are created on queues on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local queue federation) or write to the matching queue (remote queue federation) on the opposing broker.
|
||||
|
||||
An example of queue federation configuration is shown below, the local and remote queue policies have identical configuration parameters only the policy element names are different.
|
||||
An example of queue federation configuration is shown below.
|
||||
Local and remote queue policies have identical configuration parameters, only the policy element names are different.
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -498,13 +521,17 @@ An example of queue federation configuration is shown below, the local and remot
|
|||
----
|
||||
|
||||
name::
|
||||
The name of the policy, these names should be unique within a broker connection's federation policy elements.
|
||||
The name of the policy.
|
||||
These names should be unique within a broker connection's federation policy elements.
|
||||
include::
|
||||
The queue-match pattern to use to match included queues, multiple of these can be set. If none are set all queues are matched.
|
||||
The queue-match pattern to use to match included queues.
|
||||
Multiple of these can be set.
|
||||
If none are set all queues are matched.
|
||||
exclude::
|
||||
The queue-match pattern to use to match excluded queues, multiple of these can be set or it can be omitted if no excludes are needed.
|
||||
The queue-match pattern to use to match excluded queues.
|
||||
Multiple of these can be set, or it can be omitted if no excludes are needed.
|
||||
priority-adjustment::
|
||||
When federation consumers are created this value can be used to ensure that those federation consumers have a lower priority value than other local consumers on the same queue. The default value for this configuration option if not set is `-1`.
|
||||
When federation consumers are created this value can be used to ensure that those federation consumers have a lower priority value than other local consumers on the same queue.
|
||||
The default is `-1`.
|
||||
include-federated::
|
||||
Controls if consumers on a queue which come from federation instances should be counted when observing a queue for demand, by default this value is `false` and federation consumers are not counted.
|
||||
|
||||
|
|
|
|||
|
|
@ -2,13 +2,15 @@
|
|||
:idprefix:
|
||||
:idseparator: -
|
||||
|
||||
Address federation is like full multicast over the connected brokers, in that every message sent to address on `Broker-A` will be delivered to every queue on that broker, but like wise will be delivered to `Broker-B` and all attached queues there.
|
||||
Address federation is similar to full multicast over the connected brokers.
|
||||
Every message sent to address on `Broker-A` will be delivered to every queue on that broker, but also will be delivered to `Broker-B` and all their attached queues.
|
||||
|
||||
Address federation dynamically links to other addresses in upstream or downstream brokers.
|
||||
It automatically creates a queue on the remote address for itself, to which then it consumes, copying to the local address, as though they were published directly to it.
|
||||
It automatically creates a queue on the remote address for itself, which it then consumes and copies to the local address, as if they were published directly to it.
|
||||
|
||||
The upstream brokers do not need to be reconfigured or the address, simply permissions to the address need to be given to the address for the downstream broker.
|
||||
Similarly the same applies for downstream configurations.
|
||||
Upstream brokers or addresses do not need to be reconfigured.
|
||||
Just add the needed permissions to the address for the downstream broker.
|
||||
The same applies for downstream configurations.
|
||||
|
||||
.Address Federation
|
||||
image::images/federation-address.png[]
|
||||
|
|
@ -21,9 +23,9 @@ image::images/federation-address.png[]
|
|||
image::images/federation-address-symetric.png[]
|
||||
|
||||
As seen above, a publisher and consumer are connected to each broker.
|
||||
Queues and thus consumers on those queues, can receive messages published by either publisher.
|
||||
Queues _(consumers of those queues)_ can receive messages published by either publisher.
|
||||
|
||||
It is important in this setup to set `max-hops=1` to so that messages are copied only one and avoid cyclic replication.
|
||||
In this setup, it is important to set `max-hops=1`, so messages are copied only once and cyclic replication is avoided.
|
||||
If `max-hops` is not configured correctly, consumers will get multiple copies of the same message.
|
||||
|
||||
=== Full Mesh
|
||||
|
|
@ -31,12 +33,13 @@ If `max-hops` is not configured correctly, consumers will get multiple copies of
|
|||
.Address Federation - Full Mesh
|
||||
image::images/federation-address-complete-graph.png[]
|
||||
|
||||
If not already spotted, the setup is identical to symmetric but simply where all brokers are symmetrically federating each other, creating a full mesh.
|
||||
This setup is identical to symmetric above.
|
||||
All brokers are symmetrically federating each other, creating a full mesh.
|
||||
|
||||
As illustrated, a publisher and consumer are connected to each broker.
|
||||
Queues and thus consumers on those queues, can receive messages published by either publisher.
|
||||
|
||||
As with symmetric setup, it is important in this setup to set `max-hops=1` to so that messages are copied only one and avoid cyclic replication.
|
||||
Again, it is important to set `max-hops=1`, so messages are copied only once and cyclic replication is avoided.
|
||||
If `max-hops` is not configured correctly, consumers will get multiple copies of the same message.
|
||||
|
||||
=== Ring
|
||||
|
|
@ -45,34 +48,37 @@ If `max-hops` is not configured correctly, consumers will get multiple copies of
|
|||
image::images/federation-address-ring.png[]
|
||||
|
||||
In a ring of brokers each federated address is `upstream` to just one other in the ring.
|
||||
To avoid the cyclic issue, it is important to set `max-hops` to `n - 1` where `n` is the number of nodes in the ring.
|
||||
e.g. in the example above property is set to 5 so that every address in the ring sees the message exactly once.
|
||||
To avoid the cyclic issue, it is important to set `max-hops` to `n - 1` where `n` is the number of nodes in the ring.
|
||||
In the example above, property is set to 5, so that every address in the ring receives the message exactly once.
|
||||
|
||||
Whilst this setup is cheap in regards to connections, it is brittle, in that if a single broker fails, the ring fails.
|
||||
Whilst this setup is cheap in regard to connections it is brittle.
|
||||
If a single broker fails then the ring fails.
|
||||
|
||||
=== Fan out
|
||||
|
||||
.Address Federation - Fan Out
|
||||
image::images/federation-address-fan-out.png[]
|
||||
|
||||
One main address (it would required no configuration) is linked to by a tree of downstream federated addresses, the tree can extend to any depth, and can be extended to without needing to re-configure existing brokers.
|
||||
One main address (requires no configuration) is linked to a tree of downstream federated addresses.
|
||||
The tree can extend to any depth and can be extended further without needing to re-configure existing brokers.
|
||||
|
||||
In this case messages published to the main address can be received by any consumer connected to any broker in the tree.
|
||||
In this case, messages published to the main address can be received by any consumer connected to any broker in the tree.
|
||||
|
||||
=== Divert Binding Support
|
||||
|
||||
Divert binding support can be added as part of the address policy configuration.
|
||||
This will allow the federation to respond to divert bindings to create demand.
|
||||
For example, let's say there is one address called "test.federation.source" that is included as a match for the federated address and another address called "test.federation.target" that is not included.
|
||||
Normally when a queue is created on "test.federation.target" this would not cause a federated consumer to be created because the address is not part of the included matches.
|
||||
However, if we create a divert binding such that "test.federation.source" is the source address and "test.federation.target" is the forwarded address then demand will now be created.
|
||||
The source address still must be multicast but the target address can be multicast or anycast.
|
||||
For example, let's say there is one address called `test.federation.source` that is included as a match for the federated address and another address called `test.federation.target` that is not included.
|
||||
Normally, when a queue is created on `test.federation.target`, this would not cause a federated consumer to be created because the address is not part of the included matches.
|
||||
However, if we create a divert binding such that `test.federation.source` is the source address and `test.federation.target` is the forwarded address then demand will now be created.
|
||||
The source address still must be _multicast_ but the target address can be _multicast_ or _anycast_.
|
||||
|
||||
An example use case for this might be a divert that redirects JMS topics (multicast addresses) to a JMS queue (anycast addresses) to allow for load balancing of the messages on a topic for legacy consumers not supporting JMS 2.0 and shared subscriptions.
|
||||
An example use case for this might be a divert, that redirects JMS topics (multicast addresses) to a JMS queue (anycast addresses).
|
||||
This allows message load balancing on a topic for legacy consumers not supporting JMS 2.0 and shared subscriptions.
|
||||
|
||||
== Configuring Address Federation
|
||||
|
||||
Federation is configured in `broker.xml`.
|
||||
Federation is configured in `broker.xml` file.
|
||||
|
||||
Sample Address Federation setup:
|
||||
|
||||
|
|
@ -83,19 +89,19 @@ Sample Address Federation setup:
|
|||
<upstream name="eu-east-1">
|
||||
<static-connectors>
|
||||
<connector-ref>eu-east-connector1</connector-ref>
|
||||
<connector-ref>eu-east-connector1</connector-ref>
|
||||
<connector-ref>eu-east-connector2</connector-ref>
|
||||
</static-connectors>
|
||||
<policy ref="news-address-federation"/>
|
||||
</upstream>
|
||||
<upstream name="eu-west-1" >
|
||||
<static-connectors>
|
||||
<connector-ref>eu-west-connector1</connector-ref>
|
||||
<connector-ref>eu-west-connector1</connector-ref>
|
||||
<connector-ref>eu-west-connector2</connector-ref>
|
||||
</static-connectors>
|
||||
<policy ref="news-address-federation"/>
|
||||
</upstream>
|
||||
|
||||
<address-policy name="news-address-federation" max-hops="1" auto-delete="true" auto-delete-delay="300000" auto-delete-message-count="-1" transformer-ref="federation-transformer-3">
|
||||
<address-policy name="news-address-federation" max-hops="1" auto-delete="true" auto-delete-delay="300000" auto-delete-message-count="-1" transformer-ref="news-transformer">
|
||||
<include address-match="queue.bbc.new" />
|
||||
<include address-match="queue.usatoday" />
|
||||
<include address-match="queue.news.#" />
|
||||
|
|
@ -112,96 +118,102 @@ Sample Address Federation setup:
|
|||
</federations>
|
||||
----
|
||||
|
||||
In the above setup downstream broker `eu-north-1` is configured to connect to two upstream brokers `eu-east-1` and `eu-east-2`, the credentials used for both connections to both brokers in this sample are shared, you can set user and password at the upstream level should they be different per upstream.
|
||||
In the above setup, downstream broker `eu-north-1` is configured to connect to two upstream brokers `eu-east-1` and `eu-west-1`.
|
||||
Credentials used for both connections to brokers in this sample are shared.
|
||||
Should they be different per upstream, you can alter credentials at the upstream level.
|
||||
|
||||
Both upstreams are configured with the same address-policy `news-address-federation`, that is selecting addresses which match any of the include criteria, but will exclude anything that starts `queue.news.sport`.
|
||||
Both upstreams are configured with the same address-policy `news-address-federation`, that is selecting addresses which match any of the include criteria and exclude anything that starts `queue.news.sport`.
|
||||
|
||||
*It is important that federation name is globally unique.*
|
||||
|
||||
Let's take a look at all the `address-policy` parameters in turn, in order of priority.
|
||||
==== address-policy parameters
|
||||
|
||||
name::
|
||||
All address-policies must have a unique name in the server.
|
||||
|
||||
include::
|
||||
The address-match pattern to include addresses.
|
||||
Multiple of these can be set.
|
||||
If none are set all addresses are matched.
|
||||
|
||||
exclude::
|
||||
The address-match pattern to exclude addresses.
|
||||
Multiple of these can be set.
|
||||
The address-match pattern to exclude addresses. Multiple of these can be set.
|
||||
|
||||
max-hops::
|
||||
The number of hops that a message can have made for it to be federated, see <<topology-patterns,Topology Patterns>> above for more details.
|
||||
The maximum number of hops that message can perform across federated addresses. See <<topology-patterns,Topology Patterns>> above for details.
|
||||
|
||||
auto-delete::
|
||||
For address federation, the downstream dynamically creates a durable queue on the upstream address.
|
||||
This is used to mark if the upstream queue should be deleted once downstream disconnects, and the delay and message count params have been met.
|
||||
This is useful if you want to automate the clean up, though you may wish to disable this if you want messages to queued for the downstream when disconnect no matter what.
|
||||
This is used to mark if the upstream queue should be deleted once downstream disconnects and the delay and message count parameters have been met.
|
||||
It is useful in automating cleanup.
|
||||
You may wish to disable this, if you want messages to be queued for the downstream when disconnect no matter what.
|
||||
|
||||
auto-delete-delay::
|
||||
The amount of time in milliseconds after the downstream broker has disconnected before the upstream queue can be eligable for `auto-delete`.
|
||||
The amount of time in milliseconds after the downstream broker has disconnected, before the upstream queue can be eligible for `auto-delete`.
|
||||
|
||||
auto-delete-message-count::
|
||||
The amount number messages in the upstream queue that the message count must be equal or below before the downstream broker has disconnected before the upstream queue can be eligable for `auto-delete`.
|
||||
Maximum number of messages allowed in the upstream queue to be eligible for `auto-delete`, after the downstream broker has disconnected.
|
||||
|
||||
transformer-ref::
|
||||
The ref name for a transformer (see transformer config) that you may wish to configure to transform the message on federation transfer.
|
||||
Reference name of a transformer (see transformer config) that you may wish to configure to transform the message on federation transfer.
|
||||
|
||||
enable-divert-bindings::
|
||||
Setting to true will enable divert bindings to be listened for demand.
|
||||
Setting to `true` will enable divert bindings to be listened for demand.
|
||||
If there is a divert binding with an address that matches the included addresses for the stream, any queue bindings that match the forward address of the divert will create demand.
|
||||
Default is `false`.
|
||||
|
||||
NOTE: `address-policy` and `queue-policy` elements can be defined in the same federation and be linked to the same upstream.
|
||||
|
||||
Now look at all the `transformer` parameters in turn, in order of priority:
|
||||
==== transformer parameters
|
||||
|
||||
name::
|
||||
This must be a unique name in the server, and is used to ref the transformer in `address-policy` and `queue-policy`
|
||||
Unique name in the server used to reference the transformer in `address-policy` and `queue-policy`.
|
||||
|
||||
transformer-class-name::
|
||||
An optional transformer-class-name can be specified.
|
||||
This is the name of a user-defined class which implements the `org.apache.activemq.artemis.core.server.transformer.Transformer` interface.
|
||||
+
|
||||
If this is specified then the transformer's `transform()` method will be invoked with the message before it is transferred.
|
||||
If specified, then the transformer's `transform()` method will be invoked with the message before it is transferred.
|
||||
This gives you the opportunity to transform the message's header or body before it is federated.
|
||||
+
|
||||
* `property` holds key, value pairs that can be used to configure the transformer.
|
||||
+
|
||||
Finally look at `upstream`, this is what defines the upstream broker connection and the policies to use against it.
|
||||
+
|
||||
* `name` attribute.
|
||||
This must be a unique name in the server, and is used to ref the transformer in `address-policy` and `queue-policy`
|
||||
|
||||
property:: holds key-value pairs that can be used to configure the transformer.
|
||||
|
||||
==== Upstream parameters
|
||||
Tag `upstream` defines the upstream broker connection and the policies to use.
|
||||
|
||||
name::
|
||||
Unique name for upstream federated server.
|
||||
|
||||
user::
|
||||
This optional attribute determines the user name to use when creating the upstream connection to the remote server.
|
||||
If it is not specified the shared federation user and password will be used if set.
|
||||
This optional attribute determines the username to use when creating the upstream connection to the remote server.
|
||||
If not specified, the shared federation user and password will be used if they were set.
|
||||
|
||||
password::
|
||||
This optional attribute determines the password to use when creating the upstream connection to the remote server.
|
||||
If it is not specified the shared federation user and password will be used if set.
|
||||
If not specified, the shared federation user and password will be used if they were set.
|
||||
|
||||
static-connectors::
|
||||
Either this or `discovery-group-ref` is used to connect the bridge to the target server.
|
||||
+
|
||||
The `static-connectors` is a list of `connector-ref` elements pointing to `connector` elements defined elsewhere.
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc) as well as the server connection parameters (host, port etc).
|
||||
For more information about what connectors are and how to configure them, please see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc.) as well as the server connection parameters (host, port etc).
|
||||
+
|
||||
For more information about what connectors are and how to configure them, see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
|
||||
discovery-group-ref::
|
||||
Either this or `static-connectors` is used to connect the bridge to the target server.
|
||||
+
|
||||
The `discovery-group-ref` element has one attribute - `discovery-group-name`.
|
||||
This attribute points to a `discovery-group` defined elsewhere.
|
||||
For more information about what discovery-groups are and how to configure them, please see xref:clusters.adoc#clusters[Discovery Groups].
|
||||
For more information about what _discovery-groups_ are and how to configure them, see xref:clusters.adoc#clusters[Discovery Groups].
|
||||
|
||||
ha::
|
||||
This optional parameter determines whether or not this bridge should support high availability.
|
||||
True means it will connect to any available server in a cluster and support failover.
|
||||
Optional parameter determines whether this bridge should support high availability.
|
||||
Using `true` will connect to any available server in a cluster and support failover.
|
||||
The default value is `false`.
|
||||
|
||||
circuit-breaker-timeout::
|
||||
When a connection issue occurs, as the single connection is shared by many federated queue and address consumers, to avoid each one trying to reconnect and possibly causing a thundering heard issue, the first one will try.
|
||||
If unsuccessful the circuit breaker will open, returning the same exception to all.
|
||||
When a connection issue occurs, as the single connection is shared by many federated queue and address consumers, to avoid each one trying to reconnect and possibly causing a thundering herd issue, the first one will be tried.
|
||||
If unsuccessful the circuit breaker will open, returning the same exception to all connections.
|
||||
This is the timeout until the circuit can be closed and connection retried.
|
||||
Measured in milliseconds.
|
||||
|
||||
|
|
@ -230,7 +242,7 @@ This optional parameter determines the period in milliseconds between subsequent
|
|||
The default value is `500` milliseconds.
|
||||
|
||||
retry-interval-multiplier::
|
||||
This is a multiplier used to increase the `retry-interval` after each reconnect attempt, default is 1.
|
||||
Used to increase the `retry-interval` after each reconnect attempt, default is 1.
|
||||
|
||||
max-retry-interval::
|
||||
The maximum delay (in milliseconds) for retries.
|
||||
|
|
@ -238,33 +250,33 @@ Default is `2000`.
|
|||
|
||||
initial-connect-attempts::
|
||||
The number of times the system will try to connect to the remote broker in the federation.
|
||||
If the max-retry is achieved this broker will be considered permanently down and the system will not route messages to this broker.
|
||||
If the max is reached this broker will be considered permanently down and the system will not route messages to this broker.
|
||||
Default is `-1` (infinite retries).
|
||||
|
||||
reconnect-attempts::
|
||||
The number of times the system will try to reconnect to the remote broker in the federation.
|
||||
If the max-retry is achieved this broker will be considered permanently down and the system will stop routing messages to this broker.
|
||||
If the max is reached this broker will be considered permanently down and the system will stop routing messages to this broker.
|
||||
Default is `-1` (infinite retries).
|
||||
|
||||
== Configuring Downstream Federation
|
||||
|
||||
Similarly to `upstream` configuration, a downstream configuration can be configured.
|
||||
This works by sending a command to the `downstream` broker to have it create an `upstream` connection back to the downstream broker.
|
||||
The benefit of this is being able to configure everything for federation on one broker in some cases to make it easier, such as a hub and spoke topology
|
||||
The benefit of this is being able to configure everything for federation on one broker in some cases to make it easier, such as a hub and spoke topology
|
||||
|
||||
All of the same configuration options apply to `downstream` as does `upstream` with the exception of one extra configuration flag that needs to be set:
|
||||
All the same configuration options apply to `downstream` as does `upstream` with the exception of one extra configuration flag that needs to be set:
|
||||
|
||||
The `upstream-connector-ref` is an element pointing to a `connector` elements defined elsewhere.
|
||||
This ref is used to tell the downstream broker what connector to use to create a new upstream connection back to the downstream broker.
|
||||
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc) as well as the server connection parameters (host, port etc).
|
||||
For more information about what connectors are and how to configure them, please see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
upstream-connector-ref::
|
||||
Is an element pointing to a `connector` elements defined elsewhere.
|
||||
This reference is used to tell the downstream broker what connector to use to create a new upstream connection back to the downstream broker.
|
||||
+
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc.) as well as the server connection parameters (host, port etc).
|
||||
For more information about what connectors are and how to configure them, see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
|
||||
Sample Downstream Address Federation setup:
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
||||
<!--Other config Here -->
|
||||
|
||||
<connectors>
|
||||
|
|
@ -283,20 +295,20 @@ Sample Downstream Address Federation setup:
|
|||
<federation name="eu-north-1" user="federation_username" password="32a10275cf4ab4e9">
|
||||
<downstream name="eu-east-1">
|
||||
<static-connectors>
|
||||
<connector-ref>eu-east-connector1</connector-ref>
|
||||
<connector-ref>eu-east-1-connector</connector-ref>
|
||||
</static-connectors>
|
||||
<upstream-connector-ref>netty-connector</upstream-connector-ref>
|
||||
<policy ref="news-address-federation"/>
|
||||
</downstream>
|
||||
<downstream name="eu-west-1" >
|
||||
<static-connectors>
|
||||
<connector-ref>eu-west-connector1</connector-ref>
|
||||
<connector-ref>eu-west-1-connector</connector-ref>
|
||||
</static-connectors>
|
||||
<upstream-connector-ref>netty-connector</upstream-connector-ref>
|
||||
<policy ref="news-address-federation"/>
|
||||
</downstream>
|
||||
|
||||
<address-policy name="news-address-federation" max-hops="1" auto-delete="true" auto-delete-delay="300000" auto-delete-message-count="-1" transformer-ref="federation-transformer-3">
|
||||
<address-policy name="news-address-federation" max-hops="1" auto-delete="true" auto-delete-delay="300000" auto-delete-message-count="-1" transformer-ref="news-transformer">
|
||||
<include address-match="queue.bbc.new" />
|
||||
<include address-match="queue.usatoday" />
|
||||
<include address-match="queue.news.#" />
|
||||
|
|
|
|||
|
|
@ -59,29 +59,29 @@ The batch size is derived from the relevant address settings defaultConsumerWind
|
|||
|
||||
== Configuring Queue Federation
|
||||
|
||||
Federation is configured in `broker.xml`.
|
||||
Federation is configured in `broker.xml` file.
|
||||
|
||||
Sample Queue Federation setup:
|
||||
|
||||
[,xml]
|
||||
----
|
||||
<federations>
|
||||
<federation name="eu-north-1" user="federation_username" password="32a10275cf4ab4e9">
|
||||
<upstream name="eu-east-1">
|
||||
<static-connectors>
|
||||
<connector-ref>eu-east-connector1</connector-ref>
|
||||
<connector-ref>eu-east-connector1</connector-ref>
|
||||
<connector-ref>eu-east-connector2</connector-ref>
|
||||
</static-connectors>
|
||||
<policy ref="news-queue-federation"/>
|
||||
</upstream>
|
||||
<upstream name="eu-west-1" >
|
||||
<static-connectors>
|
||||
<connector-ref>eu-west-connector1</connector-ref>
|
||||
<connector-ref>eu-west-connector1</connector-ref>
|
||||
<connector-ref>eu-west-connector2</connector-ref>
|
||||
</static-connectors>
|
||||
<policy ref="news-queue-federation"/>
|
||||
</upstream>
|
||||
|
||||
<queue-policy name="news-queue-federation" priority-adjustment="-5" include-federated="true" transformer-ref="federation-transformer-3">
|
||||
<queue-policy name="news-queue-federation" priority-adjustment="-5" include-federated="true" transformer-ref="news-transformer">
|
||||
<include queue-match="#" address-match="queue.bbc.new" />
|
||||
<include queue-match="#" address-match="queue.usatoday" />
|
||||
<include queue-match="#" address-match="queue.news.#" />
|
||||
|
|
@ -98,84 +98,87 @@ Sample Queue Federation setup:
|
|||
</federations>
|
||||
----
|
||||
|
||||
In the above setup downstream broker `eu-north-1` is configured to connect to two upstream brokers `eu-east-1` and `eu-east-2`, the credentials used for both connections to both brokers in this sample are shared, you can set user and password at the upstream level should they be different per upstream.
|
||||
In the above setup downstream broker `eu-north-1` is configured to connect to two upstream brokers `eu-east-1` and `eu-west-1`, the credentials used for both connections to both brokers in this sample are shared, you can set user and password at the upstream level should they be different per upstream.
|
||||
|
||||
Both upstreams are configured with the same queue-policy `news-queue-federation`, that is selecting addresses which match any of the include criteria, but will exclude any queues that end with `.local`, keeping these as local queues only.
|
||||
|
||||
*It is important that federation name is globally unique.*
|
||||
|
||||
Let's take a look at all the `queue-policy` parameters in turn, in order of priority.
|
||||
==== Priority ordered queue-policy parameters
|
||||
|
||||
name::
|
||||
All address-policies must have a unique name in the server.
|
||||
|
||||
include::
|
||||
The address-match pattern to include addresses.
|
||||
Multiple of these can be set.
|
||||
If none are set all addresses are matched.
|
||||
The address-match pattern to include addresses. Multiple of these can be set. If none are set all addresses are matched.
|
||||
|
||||
exclude::
|
||||
The address-match pattern to exclude addresses.
|
||||
Multiple of these can be set.
|
||||
The address-match pattern to exclude addresses. Multiple of these can be set.
|
||||
|
||||
priority-adjustment::
|
||||
when a consumer attaches its priority is used to make the upstream consumer, but with an adjustment by default -1, so that local consumers get load balanced first over remote, this enables this to be configurable should it be wanted/needed.
|
||||
|
||||
include-federated::
|
||||
by default this is false, we don't federate a federated consumer, this is to avoid issue, where in symmetric or any closed loop setup you could end up when no "real" consumers attached with messages flowing round and round endlessly.
|
||||
by default this is `false`, we do not federate a federated consumer, this is to avoid issue, where in symmetric or any closed loop setup you could end up when no "real" consumers attached with messages flowing round and round endlessly.
|
||||
+
|
||||
There is though a valid case that if you dont have a close loop setup e.g. three brokers in a chain (A\->B\->C) with producer at broker A and consumer at C, you would want broker B to re-federate the consumer onto A.
|
||||
There is though a valid case that if you do not have a close loop setup e.g. three brokers in a chain (A\->B\->C) with producer at broker A and consumer at C, you would want broker B to re-federate the consumer onto A.
|
||||
|
||||
transformer-ref::
|
||||
The ref name for a transformer (see transformer config) that you may wish to configure to transform the message on federation transfer.
|
||||
|
||||
NOTE: `address-policy` and `queue-policy` elements are able to be defined in the same federation, and they can be linked to the same upstream.
|
||||
|
||||
Now look at all the `transformer` parameters in turn, in order of priority:
|
||||
==== Priority ordered transformer parameters
|
||||
|
||||
name::
|
||||
This must be a unique name in the server, and is used to ref the transformer in `address-policy` and `queue-policy`
|
||||
Unique name in the server used to reference the transformer in `address-policy` and `queue-policy`
|
||||
|
||||
transformer-class-name::
|
||||
An optional transformer-class-name can be specified.
|
||||
An optional `transformer-class-name` can be specified.
|
||||
This is the name of a user-defined class which implements the `org.apache.activemq.artemis.core.server.transformer.Transformer` interface.
|
||||
+
|
||||
If this is specified then the transformer's `transform()` method will be invoked with the message before it is transferred.
|
||||
If specified, then the transformer's `transform()` method will be invoked with the message before it is transferred.
|
||||
This gives you the opportunity to transform the message's header or body before it is federated.
|
||||
|
||||
* `property` holds key, value pairs that can be used to configure the transformer.
|
||||
property:: holds key-value pairs that can be used to configure the transformer.
|
||||
|
||||
Finally look at `upstream`, this is what defines the upstream broker connection and the policies to use against it.
|
||||
==== Upstream parameters
|
||||
Tag `upstream` defines the upstream broker connection and the policies to use.
|
||||
|
||||
* `name` attribute.
|
||||
This must be a unique name in the server, and is used to ref the transformer in `address-policy` and `queue-policy`
|
||||
name::
|
||||
Unique name for upstream federated server.
|
||||
|
||||
user::
|
||||
This optional attribute determines the user name to use when creating the upstream connection to the remote server.
|
||||
If it is not specified the shared federation user and password will be used if set.
|
||||
This optional attribute determines the username to use when creating the upstream connection to the remote server.
|
||||
If not specified, the shared federation user and password will be used if they were set.
|
||||
|
||||
password::
|
||||
This optional attribute determines the password to use when creating the upstream connection to the remote server.
|
||||
If it is not specified the shared federation user and password will be used if set.
|
||||
If not specified, the shared federation user and password will be used if they were set.
|
||||
|
||||
static-connectors::
|
||||
Either this or `discovery-group-ref` is used to connect the bridge to the target server.
|
||||
+
|
||||
The `static-connectors` is a list of `connector-ref` elements pointing to `connector` elements defined elsewhere.
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc) as well as the server connection parameters (host, port etc).
|
||||
For more information about what connectors are and how to configure them, please see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc.) as well as the server connection parameters (host, port etc).
|
||||
+
|
||||
For more information about what connectors are and how to configure them, see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
|
||||
discovery-group-ref::
|
||||
Either this or `static-connectors` is used to connect the bridge to the target server.
|
||||
+
|
||||
The `discovery-group-ref` element has one attribute - `discovery-group-name`.
|
||||
This attribute points to a `discovery-group` defined elsewhere.
|
||||
For more information about what discovery-groups are and how to configure them, please see xref:clusters.adoc#clusters[Discovery Groups].
|
||||
For more information about what discovery-groups are and how to configure them, see xref:clusters.adoc#clusters[Discovery Groups].
|
||||
|
||||
ha::
|
||||
This optional parameter determines whether or not this bridge should support high availability.
|
||||
True means it will connect to any available server in a cluster and support failover.
|
||||
This optional parameter determines whether this bridge should support high availability.
|
||||
`True` means it will connect to any available server in a cluster and support failover.
|
||||
Default is `false`.
|
||||
|
||||
circuit-breaker-timeout::
|
||||
When a connection issue occurs, as the single connection is shared by many federated queue and address consumers, to avoid each one trying to reconnect and possibly causing a thrundering heard issue, the first one will try.
|
||||
If unsuccessful the circuit breaker will open, returning the same exception to all.
|
||||
When a connection issue occurs, as the single connection is shared by many federated queue and address consumers, to avoid each one trying to reconnect and possibly causing a thundering herd issue, the first one will be tried.
|
||||
If unsuccessful the circuit breaker will open, returning the same exception to all connections.
|
||||
This is the timeout until the circuit can be closed and connection retried.
|
||||
Measured in milliseconds.
|
||||
|
||||
|
|
@ -204,7 +207,7 @@ This optional parameter determines the period in milliseconds between subsequent
|
|||
The default value is `500` milliseconds.
|
||||
|
||||
retry-interval-multiplier::
|
||||
This is a multiplier used to increase the `retry-interval` after each reconnect attempt, default is 1.
|
||||
Used to increase the `retry-interval` after each reconnect attempt, default is 1.
|
||||
|
||||
max-retry-interval::
|
||||
The maximum delay (in milliseconds) for retries.
|
||||
|
|
@ -212,27 +215,28 @@ Default is 2000.
|
|||
|
||||
initial-connect-attempts::
|
||||
The number of times the system will try to connect to the remote broker in the federation.
|
||||
If the max-retry is achieved this broker will be considered permanently down and the system will not route messages to this broker.
|
||||
If the _max-retry_ is achieved this broker will be considered permanently down and the system will not route messages to this broker.
|
||||
Default is -1 (infinite retries).
|
||||
|
||||
reconnect-attempts::
|
||||
The number of times the system will try to reconnect to the remote broker in the federation.
|
||||
If the max-retry is achieved this broker will be considered permanently down and the system will stop routing messages to this broker.
|
||||
If the _max-retry_ is achieved this broker will be considered permanently down and the system will stop routing messages to this broker.
|
||||
Default is -1 (infinite retries).
|
||||
|
||||
== Configuring Downstream Federation
|
||||
|
||||
Similarly to `upstream` configuration, a downstream configuration can be configured.
|
||||
This works by sending a command to the `downstream` broker to have it create an `upstream` connection back to the downstream broker.
|
||||
The benefit of this is being able to configure everything for federation on one broker in some cases to make it easier, such as a hub and spoke topology.
|
||||
The benefit of this is being able to configure everything for federation on one broker in some cases to make it easier, such as a hub and spoke topology.
|
||||
|
||||
All of the same configuration options apply to `downstream` as does `upstream` with the exception of one extra configuration flag that needs to be set:
|
||||
All the same configuration options apply to `downstream` as does `upstream` with the exception of one extra configuration flag that needs to be set:
|
||||
|
||||
The `upstream-connector-ref` is an element pointing to a `connector` elements defined elsewhere.
|
||||
This ref is used to tell the downstream broker what connector to use to create a new upstream connection back to the downstream broker.
|
||||
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc) as well as the server connection parameters (host, port etc).
|
||||
For more information about what connectors are and how to configure them, please see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
upstream-connector-ref::
|
||||
Is an element pointing to a `connector` elements defined elsewhere.
|
||||
This reference is used to tell the downstream broker what connector to use to create a new upstream connection back to the downstream broker.
|
||||
+
|
||||
A _connector_ encapsulates knowledge of what transport to use (TCP, SSL, HTTP etc.) as well as the server connection parameters (host, port etc).
|
||||
For more information about what connectors are and how to configure them, see xref:configuring-transports.adoc#configuring-the-transport[Configuring the Transport].
|
||||
|
||||
Sample Downstream Address Federation setup:
|
||||
|
||||
|
|
@ -257,20 +261,20 @@ Sample Downstream Address Federation setup:
|
|||
<federation name="eu-north-1" user="federation_username" password="32a10275cf4ab4e9">
|
||||
<downstream name="eu-east-1">
|
||||
<static-connectors>
|
||||
<connector-ref>eu-east-connector1</connector-ref>
|
||||
<connector-ref>eu-east-1-connector</connector-ref>
|
||||
</static-connectors>
|
||||
<upstream-connector-ref>netty-connector</upstream-connector-ref>
|
||||
<policy ref="news-address-federation"/>
|
||||
</downstream>
|
||||
<downstream name="eu-west-1" >
|
||||
<static-connectors>
|
||||
<connector-ref>eu-west-connector1</connector-ref>
|
||||
<connector-ref>eu-west-1-connector</connector-ref>
|
||||
</static-connectors>
|
||||
<upstream-connector-ref>netty-connector</upstream-connector-ref>
|
||||
<policy ref="news-address-federation"/>
|
||||
</downstream>
|
||||
|
||||
<queue-policy name="news-queue-federation" priority-adjustment="-5" include-federated="true" transformer-ref="federation-transformer-3">
|
||||
<queue-policy name="news-queue-federation" priority-adjustment="-5" include-federated="true" transformer-ref="news-transformer">
|
||||
<include queue-match="#" address-match="queue.bbc.new" />
|
||||
<include queue-match="#" address-match="queue.usatoday" />
|
||||
<include queue-match="#" address-match="queue.news.#" />
|
||||
|
|
|
|||
|
|
@ -16,9 +16,9 @@ A broker can contain federated and local-only components - you don't need to fed
|
|||
|
||||
=== WAN
|
||||
|
||||
The source and target servers do not have to be in the same cluster which makes federation suitable for reliably sending messages from one cluster to another, for instance across a WAN, between cloud regions or there internet and where the connection may be unreliable.
|
||||
The source and target servers do not have to be in the same cluster which makes federation suitable for reliably sending messages from one cluster to another, for instance across a WAN, between cloud regions or where connection may be unreliable.
|
||||
|
||||
Federation has built in resilience to failure so if the target server connection is lost, e.g. due to network failure, federation will retry connecting to the target until it comes back online.
|
||||
Federation has built-in resilience to failure so if the target server connection is lost, e.g. due to network failure, federation will retry connecting to the target until it comes back online.
|
||||
When it comes back online it will resume operation as normal.
|
||||
|
||||
=== Loose Coupling of Brokers
|
||||
|
|
@ -32,47 +32,47 @@ Federation can transmit messages between brokers (or clusters) in different admi
|
|||
|
||||
Federation is applied by policies, that match address and queue names, and then apply.
|
||||
|
||||
This means that federation can dynamically be applied as queues or addresses are added and removed, without needing to hard configure each and every one.
|
||||
This means that federation can dynamically be applied as queues or addresses are added and removed, without need to individually configure them.
|
||||
|
||||
Like wise policies are selective, in that they apply with multiple include and exclude matches.
|
||||
Likewise, policies are selective. They apply with multiple include and exclude matches.
|
||||
|
||||
Mutliple policies can applied directly to multiple upstreams, as well policies can be grouped into policy sets and then applied to upstreams to make managing easier.
|
||||
Multiple policies can be applied directly to multiple upstreams. Policies can be grouped into policy sets and then applied to upstreams to make managing easier.
|
||||
|
||||
== Address Federation
|
||||
|
||||
Address federation is like full multicast over the connected brokers, in that every message sent to address on `Broker-A` will be delivered to every queue on that broker, but like wise will be delivered to `Broker-B` and all attached queues there.
|
||||
Address federation is similar to full multicast over the connected brokers. Every message sent to address on `Broker-A` will be delivered to every queue on that broker, but also will be delivered to `Broker-B` and all their attached queues.
|
||||
|
||||
.Address Federation
|
||||
image:images/federation-address.png[Address Federation]
|
||||
|
||||
For further details please goto xref:federation-address.adoc#address-federation[Address Federation].
|
||||
For further details please see xref:federation-address.adoc#address-federation[Address Federation].
|
||||
|
||||
== Queue Federation
|
||||
|
||||
Effectively, all federated queues act as a single logical queue, with multiple receivers on multiple machines.
|
||||
So federated queues can be used for load balancing.
|
||||
Typically if the brokers are in the same AZ you would look to cluster them, the advantage of queue federation is that it does not require clustering so is suitable for over WAN, cross-region, on-off prem.
|
||||
Federated queues act as a single logical queue with multiple receivers on multiple machines.
|
||||
They can be used for load balancing.
|
||||
If brokers are in the same Availability Zone you would look to cluster them. The advantage of queue federation is that it does not require clustering, so it is suitable for over WAN, cross-region or on/off-premises usage.
|
||||
|
||||
.Queue Federation
|
||||
image:images/federated-queue-symmetric.png[Queue Federation]
|
||||
|
||||
For further details please goto xref:federation-queue.adoc#queue-federation[Queue Federation].
|
||||
For further details please see xref:federation-queue.adoc#queue-federation[Queue Federation].
|
||||
|
||||
== WAN Full Mesh
|
||||
|
||||
With federation it is possible to provide a WAN mesh of brokers, replicating with Address Federation or routing and load balancing with Queue Federation.
|
||||
It is also possible to provide a WAN mesh of brokers, which can
|
||||
|
||||
Linking producers and consumers distant from each other.
|
||||
* replicate by Address Federation
|
||||
* route and load balance using Queue Federation
|
||||
* link distant producers and consumers
|
||||
|
||||
.Example possible full federation mesh
|
||||
.Example of possible full federation mesh
|
||||
image:images/federated-world-wide-mesh.png[WAN Full Mesh]
|
||||
|
||||
|
||||
== Configuring Federation
|
||||
|
||||
Federation is configured in `broker.xml`.
|
||||
|
||||
Sample:
|
||||
Example configuration in `broker.xml`:
|
||||
|
||||
[,xml]
|
||||
----
|
||||
|
|
@ -107,11 +107,15 @@ Sample:
|
|||
|
||||
In the above example we have shown the basic key parameters needed to configure federation for a queue and address to multiple upstream.
|
||||
|
||||
The example shows a broker `eu-north-1` connecting to two upstream brokers `eu-east-1` and `eu-west-1`, and applying queue federation to queue `federated_queue` , and also applying address federation to `federated_address`.
|
||||
The example shows a broker `eu-north-1` connecting to two upstream brokers `eu-east-1` and `eu-west-1`.
|
||||
Applying queue federation to queue `federated_queue` and address federation to address `federated_address`.
|
||||
|
||||
*It is important that federation name is globally unique.*
|
||||
|
||||
There are many configuration options that you can apply these are detailed in the individual docs for xref:federation-address.adoc#address-federation[Address Federation] and xref:federation-queue.adoc#queue-federation[Queue Federation].
|
||||
There are many configuration options that you can apply. These are detailed in the individual docs:
|
||||
|
||||
* xref:federation-address.adoc#address-federation[Address Federation]
|
||||
* xref:federation-queue.adoc#queue-federation[Queue Federation]
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
|
|
@ -120,5 +124,9 @@ Extra parameters from the URI of a connector-ref can be used to override or prov
|
|||
|
||||
=== Large Messages
|
||||
|
||||
If Federation has to process large messages, the default ackBatchSize and consumerWindowSize for the consumer will need to be changed to limit the number of in-flight messages and to enable large message flow.
|
||||
These options can be supplied as parameters on the referenced connector URI, for example: `tcp://<host>:<port>?ackBatchSize=100&consumerWindowSize=-1`
|
||||
If federation has to process large messages, the default `ackBatchSize` and `consumerWindowSize` for the consumer will need to be changed to limit the number of in-flight messages and to enable large message flow.
|
||||
These options can be supplied as parameters on the referenced connector URI, for example:
|
||||
[,url]
|
||||
----
|
||||
tcp://<host>:<port>?ackBatchSize=100&consumerWindowSize=-1
|
||||
----
|
||||
|
|
|
|||
Loading…
Reference in New Issue