Commit Graph

96 Commits

Author SHA1 Message Date
Clebert Suconic c523458a9a ARTEMIS-4758 Hardening Mirroring
This is a list of improvements done as part of this commit / task:

* Page Transactions on mirror target are now optional.

If you had an interrupt mirror while the target destination was paging, duplicate detection would be ineffective unless you used paged transactions
Users can now configure the ack manager retries intervals.
Say you need some time to remove a consumer from a target mirror. The delivering references would prevent acks from happening. You can allow bigger retry intervals and number of retries by tinkiering with ack manager retry parameters.

* AckManager restarted independent of incoming acks

The ackManager was only restarted when new acks were coming in. If you stopped receiving acks on a target server and restarted that server with pending acks, those acks would never be exercised. The AckManager is now restarted as soon as the server is started.
2024-05-02 17:23:28 -04:00
Clebert Suconic 817dc41ed4 NO-JIRA Test validating multiple mirrors connected to a node 2024-04-30 22:48:38 -04:00
Clebert Suconic 07ba37a74a Revert "NO-JIRA Test validating multiple mirrors connected to a node"
This reverts commit bf81e5fe31.
2024-04-23 23:04:04 -04:00
Clebert Suconic bf81e5fe31 NO-JIRA Test validating multiple mirrors connected to a node 2024-04-23 22:43:51 -04:00
Clebert Suconic fdf2ea874b ARTEMIS-4733 Infinite mirror reflections after CreateAddress 2024-04-20 11:07:14 -04:00
Clebert Suconic a1c6a5c3b6 NO-JIRA making test more reliable 2024-04-17 18:49:03 -04:00
Clebert Suconic d864780293 ARTEMIS-4684 Internal queues should not redistribute 2024-03-13 17:01:08 -04:00
Clebert Suconic 11b7671960 ARTEMIS-4684 Internal Queues should not redistribute
This is particularly true for the Mirrored SNF queue. Redistribution is not meant for internal queues. If an internal queue happens to have the same name on another server, it should not trigger redistribution when consumers are removed.

It would be possible to work around this by adding an address-setting specific to the address with redistribution disabled.

ClusteredMirrorSoakTest was intermittently failing because of this. For a few seconds while the mirror connection is still being made connections could move messages from one node towards another node if both have the same name.
2024-03-13 09:26:47 -04:00
Clebert Suconic 21368cf741 ARTEMIS-4678 JDBC User and password not used by CLI 2024-03-08 18:23:48 -05:00
Clebert Suconic 063968bb4f ARTEMIS-4677 Validating AutoCreate with Mirror and Clustering
There is no semantic change on this commit.

I wrote a test to validate a scenario and this is to keep the test in the codebase.
2024-03-07 15:36:29 -05:00
Clebert Suconic b5791344b4 ARTEIS-4651 Speeding up delivery resume when SNF is paged 2024-03-04 10:47:12 -05:00
Clebert Suconic b4aa652c7b NO-JIRA Improving AuditLoggerTest::testAuditHotLog* tests
These tests are using an asynchronous feature. the check on Log has to use the Wait.assertEquals

I had to make a few changes to the methods to allow the use of Wait
2024-02-27 10:46:43 -05:00
Clebert Suconic db94b18b73 ARTEMIS-4651 Performance improvements on Mirror and Paging 2024-02-26 11:01:04 -05:00
a181321 efe450298d ARTEMIS-4527 - Redistributor race when consumerCount reaches 0 in cluster 2024-01-30 14:22:28 -05:00
Robbie Gemmell 8ad8c9d385 ARTEMIS-4589: consolidate utility code and remove the remaining test-jar creations in tests/ tree 2024-01-30 13:18:18 -05:00
Clebert Suconic ddac006161 ARTEMIS-4585 Previously installed mirror queues with metrics plugin would make mirror non usable 2024-01-26 14:58:40 -05:00
Clebert Suconic 1fef332bea ARTEMIS-4558 Improving test assertions on IdempotentMirrored Ack Test 2024-01-16 12:25:22 -05:00
Clebert Suconic bc7e4639e0 ARTEMIS-4558 Idempotent Mirrored ACKs
Mirror acks should be performed atomically with the storage of the source ACK. Both the send of the ack and the recording of the ack should be part of the same transaction (in case of transactional).

We are also adding support on transactions for an afterWired callback for the proper plug of OperationContext sync.
2024-01-15 19:13:49 -05:00
Clebert Suconic 1887b3fb8e ARTEMIS-4564 Cleanup large message files in case of mirror duplicate detection 2024-01-15 19:13:49 -05:00
Clebert Suconic 5e7a9023d8 ARTEMIS-4560 Fixing defaults on Broker Connections for Broker Properties
ARTEMIS-4566 Allow management of Mirror SNF internal queue
2024-01-15 19:13:49 -05:00
Justin Bertram 85b2f4b126 ARTEMIS-3474 replace non-inclusive terms
This commit does the following:

 - Replaces non-inclusive terms (e.g. master, slave, etc.) in the
   source, docs, & configuration.
 - Supports previous configuration elements, but logs when old elements
   are used.
 - Provides migration documentation.
 - Updates XSD with new config elements and simplifies by combining some
    overlapping complexTypes.
 - Removes ambiguous "live" language that's used with regard to high
   availability.
 - Standardizes use of "primary," "backup," "active," & "passive" as
   nomenclature to describe both configuration & runtime state for high
   availability.
2024-01-08 13:32:31 -05:00
Gary Tully fed01276ba ARTEMIS-4517 - cancel message references in queue sequence order 2023-11-28 16:45:04 -05:00
Clebert Suconic b041f2cde2 ARTEMIS-4476 Client Failures Soak Test
I was not able to reproduce the actual issue here, but I heavily used this test during debugging.
This will not serve as a reproducer to the Ghost consumer issue, but this is a valid test.
2023-11-09 15:09:35 -05:00
Clebert Suconic 9b56d296a3 ARTEMIS-4464 Cleanup on Soak and Smoke tests
- removed a few ignored tests
- removed some artemis maven plugin usage and using the CLI directly now
2023-10-24 14:44:24 -04:00
Clebert Suconic 7c9a15e9b4 ARTEMIS-4447 Add paging prefetch parameters into address settings
we are adding new attributes to determine how many messages (or bytes) we are reading from paging into Queue memory.
2023-09-29 18:43:23 -04:00
Clebert Suconic d01445f485 NO-JIRA removing parameters-paging.sh
use parameters.sh instead
2023-09-29 16:38:23 -04:00
Clebert Suconic 6a8cd175dc ARTEMIS-4401 improving JDBC Performance with Paging by a significant factor 2023-09-12 08:34:09 -04:00
Alexey Markevich 67f9c9d92d ARTEMIS-4391 tests: rework AssertionLoggerHandler 2023-08-17 05:47:24 -04:00
Clebert Suconic 064018a3e9 ARTEMIS-4384 cluster verify CLI command
ARTEMIS-4385 Expand StatQueue to visualize --clustered
2023-08-11 19:12:44 -04:00
Clebert Suconic bce775c98e ARTEMIS-4382 Long Time to process export / import 2023-08-01 13:08:03 -04:00
Justin Bertram b316272e14 ARTEMIS-4311 fix typo 2023-06-20 17:50:19 +01:00
Clebert Suconic 4a202bccfe ARTEMIS-4308 Allowing individual DatabasePagingTest tests, also adding postgres to the list 2023-06-13 04:55:30 -07:00
Clebert Suconic b61ec81656 ARTEMIS-4308 Adding Soak tests for Paging and JDBC
I am also allowing optionally testing with mysql.
The CLI maven plugin is creating a server and downloading the JDBC jar directly into the ./server/lib folder.
Notice this is a test dependency only and it will be used only if mysql is set to true.
2023-06-09 09:48:22 -07:00
Clebert Suconic 88f9fa494b ARTEMIS-4215 Test adjustments 2023-05-26 16:50:12 -04:00
a181321 747e0bd1e4 ARTEMIS-4215 JournalFlush might never happen when journal-sync-* is false 2023-05-26 14:29:20 -04:00
Clebert Suconic f733cac08f ARTEMIS-4254 Improving Transaction test with replication to use 3 nodes 2023-05-03 13:51:56 -07:00
Clebert Suconic 101eabdda8 ARTEMIS-4254 Transactional test with replication 2023-04-25 08:11:22 -07:00
Clebert Suconic 6d3dbc4383 ARTEMIS-4233 Large Message Issues After Failed Clients
- interrupted message breaking reference counting
After the server writing to the client is interrupted in AMQP, the reference counting was broken what would require the server restarted
in order to cleanup the files of any interrupted sends.

- Removed consumer during large message delivery damaging large messages
If the consumer failed to deliver messages for any reason, the message on the queue would be duplicated. what would wipe out the body of the message
and other journal errors would happen because of this.

extra debug capabilities added into RefCountMessage as part of ARTEMIS-4206 in order to identify these issues
2023-04-06 07:40:01 -07:00
Clebert Suconic d9d727b9c2 ARTEMIS-4224 Optimizing memory consumption from MQTT5SoakTest 2023-04-06 07:36:10 -07:00
Clebert Suconic 301aadbf1a ARTEMIS-4237 SoakPagingTest/ReplicationFlowControlTest into soak-tests 2023-04-05 19:11:23 -07:00
Clebert Suconic 880fe86ddc NO-JIRA Moving MQTT5Test::testMaxMessageSize into a soak-test
MQTT5Test::testMaxMessageSize is spiking the memory on the integration testsuite all the way up to 1.5G
what makes this test more like a soak test.

The test is now converted to use a real server like other Soak Tests.
2023-03-27 17:47:50 -07:00
Clebert Suconic d139ad75c2 NO-JIRA Allocating less memory on soak-tests
OWLeakTests is requiring a huge 200MB string to be sent to the clients
which is using too many resources from CI
2023-03-27 10:45:07 -07:00
Clebert Suconic 95cba558e4 NO-JIRA Allocating less memory on soak-tests
These tests are requiring a huge ammount of memory from CI Servers.
I'm tunning down the memory usage and parameters on a few tests.
2023-03-27 11:07:48 -04:00
Clebert Suconic 3a5601572e ARTEMIS-4207 Redistribution could leave messages stranded in the folder
- redistribute received the handle call, it then copies the message
- the routing table changes
- the message is left behind

With the new version of the server these messages will be removed. But we should remove these right away
2023-03-15 12:46:50 -07:00
Clebert Suconic 2c037386b6 ARTEMIS-4206 Unreferenced AMQP Large Messages not removed right away, requiring a reboot 2023-03-15 05:57:51 -07:00
Clebert Suconic 257dd86ae2 ARTEMIS-4193 Large Message Files orphaned after server killed
This fix is scanning journal and paging for existing large messages. We will remove any large messages that do not have a corresponding record in journals or paging.
2023-03-09 08:35:51 -08:00
Clebert Suconic 08a81a7402 I had a mistake reapplying the large message fix. I'm still reworking the commit.
I apologize for this spam

This reverts commit 14536bf311.
2023-03-06 20:05:18 -05:00
Clebert Suconic 14536bf311 Revert "I'm temporarily reverting "ARTEMIS-4193 Large Message Files orphaned after server killed""
This reverts commit 7e6ce5a259.
2023-03-06 17:04:18 -08:00
Clebert Suconic 7e6ce5a259 I'm temporarily reverting "ARTEMIS-4193 Large Message Files orphaned after server killed"
while I figure out a better fix

This reverts commit 9f1927d3fa.
2023-03-06 12:31:54 -05:00
Clebert Suconic 9f1927d3fa ARTEMIS-4193 Large Message Files orphaned after server killed 2023-03-02 19:03:37 -08:00