ARTEMIS-4495 Improving data retention documentation
This commit is contained in:
Clebert Suconic
clebertsuconic
parent
c858323f07
commit
2efc4967b6
|
|
@ -2000,11 +2000,27 @@ public interface ActiveMQServerControl {
|
|||
@Operation(desc = "forces the broker to reload its configuration file", impact = MBeanOperationInfo.ACTION)
|
||||
void reloadConfigurationFile() throws Exception;
|
||||
|
||||
/**
|
||||
* Replays messages from all files in the retention folder that match an address and filter.
|
||||
* @param address
|
||||
* @param target
|
||||
* @param filter
|
||||
* @throws Exception
|
||||
*/
|
||||
@Operation(desc = "Replays messages from all files in the retention folder that match an address and filter.", impact = MBeanOperationInfo.ACTION)
|
||||
void replay(@Parameter(name = "address", desc = "Name of the address to replay") String address,
|
||||
@Parameter(name = "target", desc = "Where the replay data should be sent") String target,
|
||||
@Parameter(name = "filter", desc = "Filter to apply on message selection. Null means everything matching the address") String filter) throws Exception;
|
||||
|
||||
/**
|
||||
* Replays messages from a configurable subset of the files in the retention folder that match an address and filter.
|
||||
* @param startScan
|
||||
* @param endScan
|
||||
* @param address
|
||||
* @param target
|
||||
* @param filter
|
||||
* @throws Exception
|
||||
*/
|
||||
@Operation(desc = "Replays messages from a configurable subset of the files in the retention folder that match an address and filter.", impact = MBeanOperationInfo.ACTION)
|
||||
void replay(@Parameter(name = "startScanDate", desc = "Start date where we will start scanning for journals to replay. Format YYYYMMDDHHMMSS") String startScan,
|
||||
@Parameter(name = "endScanDate", desc = "Finish date where we will stop scannning for journals to replay. Format YYYYMMDDHHMMSS") String endScan,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,54 @@
|
|||
= Data Retention
|
||||
:idprefix:
|
||||
:idseparator: -
|
||||
|
||||
If you enable `journal-retention` on broker.xml, ActiveMQ Artemis will keep copy of every data that has passed through the broker on this folder.
|
||||
|
||||
[,xml]
|
||||
----
|
||||
...
|
||||
<journal-retention unit="DAYS" directory="history" period="365" storage-limit="10G"/>
|
||||
...
|
||||
----
|
||||
|
||||
ActiveMQ Artemis will keep a copy of each generated journal file, up to the configured retention period, at the unit chose.
|
||||
On the example above the system would keep all the journal files up to 365 days.
|
||||
|
||||
It is also possible to limit the number of files kept on the retention directory.
|
||||
You can keep a storage-limit, and the system will start removing older files when you have more files than the configured storage limit.
|
||||
|
||||
Notice the storage limit is optional however you need to be careful to not run out of disk space at the retention folder or the broker might be shutdown because of a critical IO failure.
|
||||
|
||||
You can use the CLI tools to inspect and recover data from the history, by just passing the journal folder being the retention directory.
|
||||
|
||||
Example:
|
||||
|
||||
[,shell]
|
||||
----
|
||||
./artemis data print --journal ../data/history
|
||||
----
|
||||
|
||||
==== Paging and Large Messages
|
||||
When retention is enabled, paging and large messages are also stored on the retention folder for replaying, so replay will also work for both paged and large messages.
|
||||
|
||||
==== Non Persistent Messages
|
||||
Non persistent messages are never stored anywhere in the broker. If you intend to retain data your producers should be using Durable messages.
|
||||
|
||||
== Recovering data from retention
|
||||
|
||||
To recover data from the retention folder, you can use the method link:https://activemq.apache.org/components/artemis/documentation/javadocs/javadoc-latest/org/apache/activemq/artemis/api/core/management/ActiveMQServerControl.html#replay(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String)[replay] on the management console:
|
||||
|
||||
image::images/replay-method.png[align="center"]
|
||||
|
||||
The broker will scan the set of files accordingly to the parameters you passed and it will send these methods to a target queue that you selected.
|
||||
|
||||
=== CLI recover operation
|
||||
|
||||
The CLI recover operation is intended as a low level operation, where data is read and recovered directly into a set of journal files.
|
||||
Notice uou should never manipulate journal files that are being used by a live broker as that could risk the integrity of your data files and running broker.
|
||||
|
||||
[,shell]
|
||||
----
|
||||
./artemis data recovery --journal ../data/history --target ../data/recovered --large-messages ../data/large-messages
|
||||
----
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 201 KiB |
|
|
@ -29,51 +29,9 @@ The journal also fully supports transactional operation if required, supporting
|
|||
The majority of the journal is written in Java, however we abstract out the interaction with the actual file system to allow different pluggable implementations.
|
||||
Apache ActiveMQ Artemis ships with two implementations:
|
||||
|
||||
=== Journal Retention
|
||||
=== Journal and Data Retention
|
||||
|
||||
If you enable `journal-retention` on broker.xml, ActiveMQ Artemis will keep copy of every data that has passed through the broker on this folder.
|
||||
|
||||
[,xml]
|
||||
----
|
||||
...
|
||||
<journal-retention unit="DAYS" directory="history" period="365" storage-limit="10G"/>
|
||||
...
|
||||
----
|
||||
|
||||
ActiveMQ Artemis will keep a copy of each generated journal file, up to the configured retention period, at the unit chose.
|
||||
On the example above the system would keep all the journal files up to 365 days.
|
||||
|
||||
It is also possible to limit the number of files kept on the retention directory.
|
||||
You can keep a storage-limit, and the system will start removing older files when you have more files than the configured storage limit.
|
||||
|
||||
Notice the storage limit is optional however you need to be careful to not run out of disk space at the retention folder or the broker might be shutdown because of a critical IO failure.
|
||||
|
||||
You can use the CLI tools to inspect and recover data from the history, by just passing the journal folder being the retention directory.
|
||||
|
||||
Example:
|
||||
|
||||
[,shell]
|
||||
----
|
||||
./artemis data print --journal ../data/history
|
||||
----
|
||||
|
||||
To recover the messages from the history:
|
||||
|
||||
[,shell]
|
||||
----
|
||||
./artemis data recovery --journal ../data/history --target ../data/recovered --large-messages ../data/large-messages
|
||||
----
|
||||
|
||||
It is important that you don't call recover into a the journal while the broker is alive.
|
||||
As a matter of fact the current recommendations is to do that on a new journal directory.
|
||||
Perhaps on a new broker so you can inspect and transfer these messages.
|
||||
|
||||
The retention feature is in its current form very simple and intended for emergency situations.
|
||||
If you think it is useful new options to recover the data could be added, perhaps thorugh the admin console and other possibilities.
|
||||
Please share your feedback on this area, and as always Pull Requests are welcomed!
|
||||
|
||||
Also the recovery CLI tool will recover every data on the selected folder.
|
||||
It is important that you do some maintenance and copy the files and interval you need to a new location before you call recover.
|
||||
ActiveMQ Artemis provides a way to store and replay historical data. Refer to xref:data-retention.adoc[Data Retention] chapter for more information.
|
||||
|
||||
=== Java https://en.wikipedia.org/wiki/New_I/O[NIO]
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue