Commit Graph

233 Commits

Author SHA1 Message Date
Clebert Suconic 6d3dbc4383 ARTEMIS-4233 Large Message Issues After Failed Clients
- interrupted message breaking reference counting
After the server writing to the client is interrupted in AMQP, the reference counting was broken what would require the server restarted
in order to cleanup the files of any interrupted sends.

- Removed consumer during large message delivery damaging large messages
If the consumer failed to deliver messages for any reason, the message on the queue would be duplicated. what would wipe out the body of the message
and other journal errors would happen because of this.

extra debug capabilities added into RefCountMessage as part of ARTEMIS-4206 in order to identify these issues
2023-04-06 07:40:01 -07:00
Clebert Suconic d9d727b9c2 ARTEMIS-4224 Optimizing memory consumption from MQTT5SoakTest 2023-04-06 07:36:10 -07:00
Clebert Suconic 301aadbf1a ARTEMIS-4237 SoakPagingTest/ReplicationFlowControlTest into soak-tests 2023-04-05 19:11:23 -07:00
Clebert Suconic 880fe86ddc NO-JIRA Moving MQTT5Test::testMaxMessageSize into a soak-test
MQTT5Test::testMaxMessageSize is spiking the memory on the integration testsuite all the way up to 1.5G
what makes this test more like a soak test.

The test is now converted to use a real server like other Soak Tests.
2023-03-27 17:47:50 -07:00
Clebert Suconic d139ad75c2 NO-JIRA Allocating less memory on soak-tests
OWLeakTests is requiring a huge 200MB string to be sent to the clients
which is using too many resources from CI
2023-03-27 10:45:07 -07:00
Clebert Suconic 95cba558e4 NO-JIRA Allocating less memory on soak-tests
These tests are requiring a huge ammount of memory from CI Servers.
I'm tunning down the memory usage and parameters on a few tests.
2023-03-27 11:07:48 -04:00
Clebert Suconic 3a5601572e ARTEMIS-4207 Redistribution could leave messages stranded in the folder
- redistribute received the handle call, it then copies the message
- the routing table changes
- the message is left behind

With the new version of the server these messages will be removed. But we should remove these right away
2023-03-15 12:46:50 -07:00
Clebert Suconic 2c037386b6 ARTEMIS-4206 Unreferenced AMQP Large Messages not removed right away, requiring a reboot 2023-03-15 05:57:51 -07:00
Clebert Suconic 257dd86ae2 ARTEMIS-4193 Large Message Files orphaned after server killed
This fix is scanning journal and paging for existing large messages. We will remove any large messages that do not have a corresponding record in journals or paging.
2023-03-09 08:35:51 -08:00
Clebert Suconic 08a81a7402 I had a mistake reapplying the large message fix. I'm still reworking the commit.
I apologize for this spam

This reverts commit 14536bf311.
2023-03-06 20:05:18 -05:00
Clebert Suconic 14536bf311 Revert "I'm temporarily reverting "ARTEMIS-4193 Large Message Files orphaned after server killed""
This reverts commit 7e6ce5a259.
2023-03-06 17:04:18 -08:00
Clebert Suconic 7e6ce5a259 I'm temporarily reverting "ARTEMIS-4193 Large Message Files orphaned after server killed"
while I figure out a better fix

This reverts commit 9f1927d3fa.
2023-03-06 12:31:54 -05:00
Clebert Suconic 9f1927d3fa ARTEMIS-4193 Large Message Files orphaned after server killed 2023-03-02 19:03:37 -08:00
Clebert Suconic ed5f63538e ARTEMIS-4155 Fixing deadlock on LargeMessage conversion and retention 2023-02-06 09:27:18 -05:00
Clebert Suconic 2378d187af [maven-release-plugin] prepare for next development iteration 2023-01-31 07:22:48 -05:00
Clebert Suconic 893e1e7916 [maven-release-plugin] prepare release 2.28.0 2023-01-31 07:22:47 -05:00
Robbie Gemmell f790911c44 ARTEMIS-4110: insert standard ASF licence header comment in various cases, such as to replace a javadoc header 2022-12-07 10:21:25 +00:00
Clebert Suconic eb11b044af ARTEMIS-4108 AMQP Drain fails under load with Large Messages 2022-12-05 16:47:08 -05:00
Clebert Suconic af2b8e4b07 ARTEMIS-4084 Dealing with multi consumers crashes, Improving cached addSorted 2022-11-28 09:54:45 -05:00
Clebert Suconic ae4475201e ARTEMIS-4084 Fixing CriticalAnalyzer policy on Test 2022-11-09 16:49:50 -05:00
Clebert Suconic 03b82142eb ARTEMIS-4084 Fixing addSorted with large transactions
when cancelling a large number of messages, the addSorted could be holding a lock for too long causing the server to crash under CriticalAnalyzer

co-authored: AntonRoskvist <anton.roskvist@volvo.com> (discovering the issue and providing the test ClientCrashMassiveRollbackTest.java)
2022-11-09 15:43:02 -05:00
Justin Bertram dce4ba3c1a [maven-release-plugin] prepare for next development iteration 2022-11-08 13:17:38 -06:00
Justin Bertram 1fd6cb6239 [maven-release-plugin] prepare release 2.27.0 2022-11-08 12:46:20 -06:00
Clebert Suconic d185735e55 ARTEMIS-4073 Page Counters can go off sync when multiple producers are used in the same address 2022-10-25 21:26:14 -04:00
Clebert Suconic 10a93f9203 NO-JIRA log.debug on a test 2022-10-15 00:24:19 -04:00
Clebert Suconic 0275efff29 NO-JIRA Fixing Junit Helpr methods errorprone issue in MegaCleanerPagingTest 2022-10-15 00:15:09 -04:00
Clebert Suconic 3f63e6be82 NO-JIRA Adding a test to validating paging reload
no issues found as part of this test. Just adding a new test.
2022-10-14 16:31:37 -04:00
Clebert Suconic ec00def02b ARTEMIS-4025 Fixing testsuite by adding json missing dependencies 2022-10-14 15:42:42 -04:00
Timothy Bish b900a1e4bd ARTEMIS-4020 Standardize the naming of Logger types for consistency
Attempt to standardize all Logger declaration to a singular variable name
which makes the code more consistent and make finding usages of loggers in
the code a bit easier.
2022-10-13 12:40:45 -04:00
Clebert Suconic 33438c2e44 ARTEMIS-4029 Avoid OME When too many pages are cleared all at once
this is the second part of the fix (I missed the first part) where I am clearig midstream records as well.
2022-10-11 00:26:41 -04:00
Clebert Suconic 42d07458e9 ARTEMIS-4029 Avoid OME When too many pages are cleared all at once
The issue is that depage should not put pages on the used pages as they were not actually intended to read.
instead I should create a newPageObject and not use the RefCounts caching.
2022-10-06 07:53:38 -04:00
Clebert Suconic 20fde76e99 ARTEMIS-4020 Fixing Semaphore on soak-tests/OWLeakTest
I did not intend to have this difference in the semaphore.

The idea is that we never keep messages pending on this case, otherwise such a huge message would use too much memory.
2022-10-04 15:02:10 -04:00
Clebert Suconic 18cfdb7049 ARTEMIS-4024 Avoid excessive NativeMemory allocation when sending OpenWire Multi mega sized messages in openwire 2022-10-04 13:35:50 -04:00
Clebert Suconic d2354ae470 ARTEMIS-4020 Fixing logging on soak-tests 2022-09-29 17:48:46 -04:00
Clebert Suconic 12cc70c5bf ARTEMIS-4020 Using a little trick to create the Loggers
Trick provided by Tim Bish
2022-09-29 17:46:51 -04:00
Robbie Gemmell 9873fccf74 ARTEMIS-4020: switch to using SLF4J API for logging, use Log4J 2 as impl for broker distribution and tests
PR includes work from myself and Clebert Suconic, squashed from the new-logging branch.
2022-09-28 14:01:54 -04:00
Clebert Suconic 5d8e7fe829 [maven-release-plugin] prepare for next development iteration 2022-09-21 13:00:47 -04:00
Clebert Suconic 2d7b1a3ef7 [maven-release-plugin] prepare release 2.26.0 2022-09-21 13:00:46 -04:00
Clebert Suconic f47d592079 ARTEMIS-3988 Moving MMSFactoryTest under soak-tests 2022-09-13 23:29:20 -04:00
Clebert Suconic 9fc5886391 NO-JIRA Making some Test Parameters mandatory
if you miss them, the test would rather fail then silently using less than ideal values
2022-09-13 08:38:23 -04:00
Clebert Suconic 4ad830fb95 Bumping artemis as 3.0
After removing ActiveMQ Artemis rest (per ARTEMIS-3987), we should start promoting Artemis as 3.0
2022-09-12 15:09:31 -04:00
Clebert Suconic 11eab60584 [maven-release-plugin] prepare for next development iteration 2022-08-30 18:04:06 -04:00
Clebert Suconic fb1b362b47 [maven-release-plugin] prepare release 2.25.0 2022-08-30 18:04:05 -04:00
Clebert Suconic d06459df57 ARTEMIS-3943 Adjusting defalut address settings to avoid OME from paging and flow control 2022-08-30 09:26:35 -04:00
Clebert Suconic c96243698d NO-JIRA Fixing suggested script for test variables
These scripts are supposed to be execute with ". ./parameters-paging.sh"
which would incur in -e, and any mistake on the shell will kill your shell and you would be wondering what happened.
2022-08-18 16:24:45 -04:00
Clebert Suconic 252e5b0b14 ARTEMIS-3928 Limiting parallel IO in paging which would allow multiple destinations running all at once
Running HorizontalPagingTest with these variables would make the test to fail unless these changes are applied.

export TEST_HORIZONTAL_SERVER_START_TIMEOUT=300000
export TEST_HORIZONTAL_TIMEOUT_MINUTES=120
export TEST_HORIZONTAL_PROTOCOL_LIST=OPENWIRE

export TEST_HORIZONTAL_OPENWIRE_DESTINATIONS=200
export TEST_HORIZONTAL_OPENWIRE_MESSAGES=1000
export TEST_HORIZONTAL_OPENWIRE_COMMIT_INTERVAL=100
export TEST_HORIZONTAL_OPENWIRE_RECEIVE_COMMIT_INTERVAL=0
export TEST_HORIZONTAL_OPENWIRE_MESSAGE_SIZE=20000
export TEST_HORIZONTAL_OPENWIRE_PARALLEL_SENDS=10
2022-08-10 19:29:29 -04:00
Clebert Suconic 12b81e7a25 ARTEMIS-3928 Adding SoakTest for Paging with in many destinations
Also some organization on soak test to allow this test
adding CLI operation to configure memory size of the broker
2022-08-10 16:38:58 -04:00
Clebert Suconic fa002728f2 [maven-release-plugin] prepare for next development iteration 2022-07-26 11:59:21 -04:00
Clebert Suconic 897d9beaef [maven-release-plugin] prepare release 2.24.0 2022-07-26 11:59:19 -04:00
Clebert Suconic 79daf49105 [maven-release-plugin] prepare for next development iteration 2022-07-21 17:44:35 -04:00