Commit Graph

3422 Commits

Author SHA1 Message Date
Howard Gao 1f79341c05 ARTEMIS-4973 pageSizeBytes/pageLimitBytes combination can cause Address full
Update docs/user-manual/paging.adoc

Co-authored-by: Robbie Gemmell <robbie@apache.org>
2024-08-16 09:41:53 -04:00
Clebert Suconic 583af58e43 ARTEMIS-4986 Tweaks on test: Disable load balancing and check for no messages 2024-08-15 20:47:56 -04:00
Clebert Suconic 209f293922 [maven-release-plugin] prepare for next development iteration 2024-08-14 14:39:34 -04:00
Clebert Suconic de6efc82d1 [maven-release-plugin] prepare release 2.37.0 2024-08-14 14:39:21 -04:00
Josh Byster 8b3874d613
ARTEMIS-4809 Allow configuring initial queue buffer size
In some setups, there could be a few hundred thousand queues that are
created due to many consumers that are connecting. However, most of
these are empty and stay empty for the entire day since there aren't
necessarily messages to be sent. The 8K intermediateMessageReferences
instantiates an 64KB buffer (Object[]). This means we have large
allocation and live heap that ultimately remains empty for almost the
entire day.

In this commit, we introduce initial-queue-buffer-size, which defaults
to the current value of 8192. It can be set programmatically via
QueueConfiguration#setInitialQueueBufferSize(int).

Note that this must be a positive power of 2.
2024-08-14 13:18:29 -05:00
Domenico Francesco Bruscino 7cf6b86bc5 ARTEMIS-4785 ARTEMIS-4702 Add profile and log4j2 files for non-run CLI commands
The run command uses the artemis.profile and log4j2.properties files while all
other CLI commands use the artemis-utility.profile and log4j2-default.properties
files.
2024-08-14 14:59:51 +01:00
Clebert Suconic fb7afa8ff3 ARTEMIS-4986 Allow configuring the target destination as well 2024-08-13 22:16:52 -04:00
Clebert Suconic 5adde5ef43 ARTEMIS-4986 Changing plugin to compile phase to make it simpler on CI 2024-08-13 20:18:11 -04:00
Clebert Suconic 10b6ab9bd3 ARTEMIS-4986 Providing a test that will look for the distribution from a System Variable
if you define TEST_ROLLED_DISTRIBUTION=your artemis home

testRollUpgrade_Provided_Distribution will execute the rolling upgrade from that distribution
2024-08-13 16:33:00 -04:00
Clebert Suconic 537e0023fc ARTEMIS-4986 Compatibility issue on Quorum Voting 2024-08-13 14:45:00 -04:00
Justin Bertram 0ecbae6108 ARTEMIS-4939 allow config of header sizes for embedded web server 2024-08-13 10:38:06 -05:00
Timothy Bish f031481012 ARTEMIS-4982 Cleanup AMQP large message files for rejected sends
When an incoming AMQP large message send is rejected the broker should delete the
large message file as part of the reject handling.
2024-08-12 17:39:51 +01:00
Timothy Bish be4995a530 ARTEMIS-4969 Ensure AMQP senders can target FQQN if authorized
When using targeted FQQN permissions the AMQP sender needs to check that
it can access not only the address but also the queue if sent an FQQN so
that the security can validate if the sender has been granted directed
access to the FQQN as a whole.
2024-08-06 10:21:57 +01:00
Timothy Bish 8250e611df ARTEMIS-4963 Check send auth on openwire producer create
Check that an attaching Openwire producer has SEND permission on the target
destination and reject it if it does not instead of delaying checks until the
actual send. For anonymous producers check early in the send process to reduce
overhead in the JVM handling messages that are going to fail to send.
2024-08-02 16:38:20 +01:00
Clebert Suconic 47acdc4517 ARTEMIS-4964 Fast ACK Test with Mirror
One side of the mirror will send and ack messages one by one.

As the message arrives in the mirror the ack comes before the persistence finishes, so we need to retry and configure retry accordingly.
2024-08-01 23:09:54 -04:00
Clebert Suconic d7078cabf5 NO-JIRA tweak on DivertQueueMirrorTest 2024-07-30 20:44:01 -04:00
Clebert Suconic cbbaa7a8e0 NO-JIRA Mirror and Divert Smoke Test
I was doing some verification with Diverts and Mirror in the form of a smoke test.

no issue found but I'm keeping the test.
2024-07-30 20:41:53 -04:00
Howard Gao 4ce0dfca2d ARTEMIS-4959 moveMessages operation can move more messages than max messageCount 2024-07-30 11:55:53 -04:00
Howard Gao 096a869c1e ARTEMIS-4954 AddressControl.pause() can pause the snf queue 2024-07-25 19:51:30 -04:00
Timothy Bish 284ce80618 ARTEMIS-4952 Use getObjectPropertyForFilter when applying filters
When counting messages with provided filters and grouping use the API
getObjectPropertyForFilter which translates values from AMQP message
annotations that are otherwise missed by the filters.
2024-07-25 19:51:19 -04:00
Clebert Suconic fa917ae3fd [maven-release-plugin] prepare for next development iteration 2024-07-25 13:47:10 -04:00
Clebert Suconic 75f17ba64d [maven-release-plugin] prepare release 2.36.0 2024-07-25 13:47:09 -04:00
Clebert Suconic db0ba73aa4 ARTEMIS-4941 Remove lazy update after application properties only when it's paging
for regular messages it's quite obvious when the message is leaving the queue but for paged messages it becomes a challenge. We should just ignore the update for paged messages.
2024-07-25 13:01:57 -04:00
Clebert Suconic 1a5c2ec51c Revert "ARTEMIS-4941 Remove lazy update after application properties as it's no longer needed"
This reverts commit fb2a57f3ed.
2024-07-25 13:01:57 -04:00
Clebert Suconic ca44ad2a33 ARTEMIS-3509 Improving assertions in StompTest 2024-07-24 12:13:07 -04:00
Clebert Suconic e50c73eac9 Revert "ARTEMIS-3509 Improving assertions on StompTest"
Sorry for the spam, I pushed this commit by mistake / accident

This reverts commit db0a0975bb.
2024-07-24 10:42:19 -04:00
Clebert Suconic db0a0975bb ARTEMIS-3509 Improving assertions on StompTest 2024-07-24 09:59:43 -04:00
Justin Bertram 8d1a66fbbd ARTEMIS-4931 fix bridge config encoding
This commit fixes 3 distinct issues with bridge configuration encoding:

 - The encoding size was calculated incorrectly in three ways:
   - Using 0 instead of `DataConstants.SIZE_NULL` when no transformer is
     defined resulting in a calculated size discrepancy of -1.
   - Using `DataConstants.INT` instead of `DataConstants.SIZE_INT` for
     the number of transformer properties resulting in a calculated size
     discrepancy of +2 when using a configuration with a transformer.
   - Using 0 or `sizeOfNullableInteger` instead of
     `DataConstants.SIZE_INT` for the number of static connectors
     resulting in a variable calculated size discrepancy of either -4 or
     +1 respectively.
 - Encoding was using `writeString` instead of `writeNullableString` for
   the name of the transformer class resulting in an actual buffer size
   discrepancy of -1.

Aside from these fixes this commit also adds new tests specifically for
encoding & decoding include a test to verify known bytes during
encoding.

Lastly, it's worth noting that this *won't fix* bad data that was
already stored to disk by an older broker or bad data that comes over
the wire from an older broker (e.g. if a older primary broker paired
with a newer backup).
2024-07-24 14:51:55 +01:00
Justin Bertram 5c160ae1a8 ARTEMIS-4930 refactor storage manager tests
During the investigation for ARTEMIS-4910 it was noted that some of the
storage manager tests didn't actually force a full reload (i.e.
decoding) of the various configurations from disk. This commit does the
following:

 - refactors these tests so that they all force a reload from disk
 - organizes imports
 - eliminates various bits of repeated code in all the tests
 - improves code formatting

This commit exposes a bug in the encoding/decoding of
BridgeConfiguration via a failing test in
BridgeConfigurationStorageTest. This test is disabled. This failure will
be resolved in a subsequent commit at which point the test will be
enabled again.
2024-07-24 14:51:55 +01:00
Clebert Suconic ae26a8640f NO-JIRA Removing System.out from a test 2024-07-23 16:28:15 -04:00
Clebert Suconic 7c573db6a1 ARTEMIS-4949 TimedBuffer.checkSync could fail on ClosedChannelException 2024-07-23 12:56:33 -04:00
Clebert Suconic 41ec279e22 NO-JIRA Making test more resilient 2024-07-22 18:57:27 -04:00
Clebert Suconic fb2a57f3ed ARTEMIS-4941 Remove lazy update after application properties as it's no longer needed 2024-07-22 15:31:36 -04:00
Clebert Suconic 9e8df4cf7a ARTEMIS-4821 OperationContext leaking on SendAckThread
co-authored: nmeylan (test attached to the JIRA)
2024-07-21 22:23:38 -04:00
Justin Bertram 4e1daccfe3 ARTEMIS-4912 filter not enforced when sending msg to q via web console 2024-07-19 15:00:45 -05:00
Justin Bertram 4053698014
ARTEMIS-4936 make test more resilient 2024-07-19 10:58:31 -05:00
Justin Bertram 9a0797509d ARTEMIS-4936 verify response correlationId when using Core client
If, for whatever reason, the response for a packet sent with blocking
semantics is never returned it's possible that an async response
received in the interventing time will be interpreted as the current
response. This is because ChannelImpl does not verify the correlation
ID set on the response packet when it is received.
2024-07-19 07:05:46 -07:00
Clebert Suconic bcdf626efa NO-JIRA Speeding up test
Removing redundant verification that was just spending time and not being verified.
2024-07-18 16:00:37 -04:00
Clebert Suconic da7dccae87 ARTEMIS-4925 Making assertion to also consider journal and other files from the broker
Small change but say there's ever a leak on the journal. Removing the clause from paging would allow to also capture other leaks.

This is currently not an issue and the test should still pass.
2024-07-18 14:16:02 -04:00
Clebert Suconic 79708749e8 NO-JIRA Speending up a failover test
No need for the iterations, and a few minor improvements to the test.

This method was taking up to 15 seconds on the CI and the same method is reused a couple times, resulting in many minutes wasted
2024-07-18 14:16:02 -04:00
Clebert Suconic 58f9e00874 ARTEMIS-4925 adding missing assertion on the test 2024-07-17 12:02:41 -04:00
Clebert Suconic 569a07c287 ARTEMIS-4925 Allowing server reuse on a test 2024-07-17 10:19:27 -04:00
Justin Bertram 1a83220711 ARTEMIS-4910 fix divert config encoding
This commit fixes 4 distinct issues with divert configuration encoding:

 - The encoding size was calculated incorrectly in three ways:
   - Using 0 instead of `DataConstants.SIZE_NULL` when no transformer is
     defined resulting in a calculated size discrepancy of -1.
   - Using `DataConstants.INT` instead of `DataConstants.SIZE_INT` for
     the number of transformer properties resulting in a calculated size
     discrepancy of +2 when using a configuration with a transformer.
   - Using `BufferHelper.sizeOfNullableBoolean` instead of
     `DataConstants.SIZE_BOOLEAN` for the `exclusive` property resulting
     in a calculated discrepancy of +1.
 - Encoding was using `writeString` instead of `writeNullableString` for
   the name of the transformer class resulting in an actual buffer size
   discrepancy of -1.

Aside from these fixes this commit also:

 - Updates the divert storage test to force a reload from disk which
   will also trigger this problem.
 - Adds new tests specifically for encoding & decoding include a test to
   verify known bytes during encoding.

Lastly, it's worth noting that this *won't fix* bad data that was
already stored to disk by an older broker or bad data that comes over
the wire from an older broker (e.g. if a older primary broker paired
with a newer backup).
2024-07-17 14:11:21 +01:00
Clebert Suconic 4b1952e3aa ARTEMIS-4925 small tweak on test
Using a lambda to close a resource
2024-07-16 19:10:00 -04:00
Clebert Suconic 6c1134388b ARTEMIS-4925 Fixing issue with mirroring and replication
the page writes on replica should include the address where they belong to avoid leaks between the mirrorSNF and the actual addresses.
2024-07-16 17:05:06 -04:00
Clebert Suconic 5af90578cb ARTEMIS-4925 Test with mirror and replication.
Files will leak on the target and messages will not be received after failover.
Also messages will be written to wrong destinations on the replica. The leak is actually between destinations, and the consequence is the file leak.

I usually keep test and fix on the same commit, but in this case I have been heavily validating the server with and without the fix,
so I will open an exception in this case and keep the fix and test separated.
2024-07-16 17:05:06 -04:00
Justin Bertram 19d8059a4e ARTEMIS-4794 configure pending ack behavior for bridge
When a bridge is stopped it doesn't wait for pending send
acknowledgements to arrive. However, when a bridge is paused it does
wait. The behavior should be consistent and more importantly
configurable. This commit implements these improvements and generally
refactors BridgeImpl to clarify and simplify the code. In total, this
commit includes the follow changes:

 - Removes the hard-coded 60-second timeout for pending acks when
   pausing the bridge and adds a new config parameter (i.e.
   "pending-ack-timeout").
 - Applies the new pending-ack-timeout when the bridge is stopped.
 - Updates existing and adds new logging messages for clarity.
 - De-duplicates code for sending bridge-related notifications.
 - Avoids converting bridge name to/from SimpleString.
 - Removes unnecessary comments.
 - Renames variables & functions for clarity.
 - Replaces the `started`, `stopping`, & `active` booleans with a
   single `state` variable which is an enum.
 - Adds `final` to a few variables that were functionally final.
 - Synchronizes `stop` & `pause` methods to add safety when invoked
   concurrently with `handle` (since both deal with `state` and execute
   runnables on the ordered executor).
 - Reorganizes and removes a few methods for clarity.
 - Relocates `connect` method directly into `ConnectRunnable` (mirroring
   the structure of the `StopRunnable` and `PauseRunnable`).
 - Eliminates unnecessary variables in `ConnectRunnable` and
   `ScheduledConnectRunnable`.
 - Adds test to verify pending ack timeout works as expected with both
   `stop` & `pause` with both regular and large messages.
2024-07-15 18:19:39 +01:00
Timothy Bish 1361e29739 ARTEMIS-4919 Handle Exception from proton transport process
Handle any exceptions from the proton transport and set the error on the
transport for processing in the events dispatch cycle by adding in handling
of the transport error event.
2024-07-15 15:34:21 +01:00
Justin Bertram 0277c6498e ARTEMIS-4914 clarify docs on core + client ID
This commit also adds a couple of tests to verify the documented
behavior works.
2024-07-12 11:03:04 -07:00
Justin Bertram b8a9d626cd ARTEMIS-4920 strip leading forward slash from IP address 2024-07-12 11:00:53 -07:00