Commit Graph

2384 Commits

Author SHA1 Message Date
Clebert Suconic d730d1a684 NO-JIRA Testsuite speedup: proper JDBC drop from derby 2023-03-18 15:44:03 -07:00
Domenico Francesco Bruscino aaa8627795 ARTEMIS-4211 Fix record message id for federated queue consumers
The federated queue consumer has to generate a new id for the messages
received from the upstream broker because they have an id generated by
the store manager of the upstream broker.

Co-authored-by: Clebert Suconic <clebertsuconic@apache.org>
2023-03-18 08:52:21 -07:00
Clebert Suconic 4babdab18c NO-JIRA Cleanup MBeanServer usage in the testsuite as it is leaking
Basically I started the testsuite and attached check leak with "java -jar check-leak.jar --pid <pid> --report testsuite-report --sleep 1000" and saw the allocations of this were pretty high.
2023-03-15 12:42:12 -07:00
Clebert Suconic 2c037386b6 ARTEMIS-4206 Unreferenced AMQP Large Messages not removed right away, requiring a reboot 2023-03-15 05:57:51 -07:00
Justin Bertram 68c5bed159 ARTEMIS-4200 configurable link-stealing for MQTT 2023-03-10 17:51:24 -06:00
Justin Bertram 9b4204b345 ARTEMIS-4201 send proper MQTT disconnect code on stolen link 2023-03-10 17:51:24 -06:00
Clebert Suconic 257dd86ae2 ARTEMIS-4193 Large Message Files orphaned after server killed
This fix is scanning journal and paging for existing large messages. We will remove any large messages that do not have a corresponding record in journals or paging.
2023-03-09 08:35:51 -08:00
Domenico Francesco Bruscino 7a0bf52ed8 ARTEMIS-4190 Fix config-delete-queues when address changes 2023-03-09 09:42:57 -06:00
Domenico Francesco Bruscino bb08a573eb ARTEMIS-3640 Use client connectors for HA 2023-03-09 11:20:51 +00:00
Gary Tully 00cae02ca4 ARTEMIS-4196 - set message routing type in mqtt publish 2023-03-08 14:30:30 -06:00
Clebert Suconic ed5322c54f NO-JIRA Fixing intermittent failure on ActiveMQServerControlUsingCoreTest
This failure was because of a noise from the test itself. as the test is creating a producer, and it's measuring for a producer from the test.
it makes no sense to fix it for OverCore.. we just ignore it on UsingCore
2023-03-08 12:13:17 -05:00
Justin Bertram 97c78525da ARTEMIS-4181 make try-with-resources style consistent 2023-03-03 08:40:51 -06:00
Justin Bertram b76c672305 ARTEMIS-4162 support deleting addresses & queues w/o usage check
There are certain use-cases where addresses will be auto-created and
never have a direct binding created on them. Because of this they will
never be auto-deleted. If a large number of these addresses build up
they will consume a problematic amount of heap space.

One specific example of this use-case is an MQTT subscriber with a
wild-card subscription and a large number of MQTT producers sending one
or two messages a large number of different MQTT topics covered by the
wild-card. Since no bindings are ever created on any of these individual
addresses (e.g. from a subscription queue) they will never be
auto-deleted, but they will eventually consume a large amount of heap.
The only way to deal with these addresses is to manually delete them.

There are also situations  where queues may be created and never have
any messages sent to them or never have a consumer connect. These
queues will never be auto-deleted so they must be deleted manually.

This commit adds the ability to configure the broker to skip the usage
check so that these kinds of addresses and queues can be deleted
automatically.
2023-03-02 19:17:03 -06:00
Justin Bertram 6874556f7c ARTEMIS-4188 auto-create + MDB using selector on queue 2023-03-02 16:26:12 -08:00
Justin Bertram fb169bc4af ARTEMIS-4172 sending large msg via core skips plugins & audit log 2023-02-16 13:34:29 -08:00
Justin Bertram 82fc42987a ARTEMIS-4171 potential large message file leak 2023-02-16 13:34:29 -08:00
Clebert Suconic a2ba6ed298 ARTEMIS-4161 AMQP and OpenWire leaking leaking objects in certain conditions
The issue identified with AMQP was under Transaction usage, and while opening and closing sessions.
It seems the leak would be released once the connection is closed.

We added a new testsuite under ./tests/leak-tests To fix and validate these issues
2023-02-16 12:10:54 -08:00
Clebert Suconic 69e21a0eb7 ARTEMIS-4163 Fixing openwire race while chunkSend is happening 2023-02-09 19:58:40 -05:00
Clebert Suconic 1caa406bbf NO-JIRA Adding a test to verify TX in OpenWire Large Message Handling 2023-02-08 09:29:11 -05:00
Justin Bertram 8f30347b18 ARTEMIS-4143 improve mitigation against split-brain with shared-storage
Configurations employing shared-storage with NFS are susceptible to
split-brain in certain scenarios. For example:

  1) Primary loses network connection to NFS.
  2) Backup activates.
  3) Primary reconnects to NFS.
  4) Split-brain.

In reality this situation is pretty unlikely due to the timing involved,
but the possibility still exists. Currently the file lock held by the
primary broker on the NFS share is essentially worthless in this
situation. This commit adds logic by which the timestamp of the lock
file is updated during activation and then routinely checked during
runtime to ensure consistency. This effectively mitigates split-brain in
this situation (and likely others). Here's how it works now.

  1) Primary loses network connection to NFS.
  2) Backup activates.
  3) Primary reconnects to NFS.
  4) Primary detects that the lock file's timestamp has been updated and
     shuts itself down.

When the primary shuts down in step #4 the Topology on the backup can be
damaged. Protections were added for this via ARTEMIS-2868 but only for
the replicated use-case. This commit applies the protection for
removeMember() so that the Topology remains intact.

There are no tests for these changes as I cannot determine how to
properly simulate this use-case. However, there have never been robust,
automated tests for these kinds of NFS use-cases so this is not a
departure from the norm.
2023-02-03 10:40:08 -05:00
Timothy Bish d1b3610f68 ARTEMIS-2431 Stop processing Begin and Attach frames if open fails
For pipelined open cases the events processing should ignore additional begin
and attach events if the open event handler closes the connection to avoid the
processing throwing additional exceptions and replacing the error condition in
the connection with an unrelated error about NPE from the additional events.
2023-02-01 17:53:32 -05:00
Clebert Suconic 2378d187af [maven-release-plugin] prepare for next development iteration 2023-01-31 07:22:48 -05:00
Clebert Suconic 893e1e7916 [maven-release-plugin] prepare release 2.28.0 2023-01-31 07:22:47 -05:00
Clebert Suconic 764db34e9b ARTEMIS-3178 Page Limitting (max messages and max bytes)
I am adding three attributes to Address-settings:

* page-limit-bytes: Number of bytes. We will convert this metric into max number of pages internally by dividing max-bytes / page-size. It will allow a max based on an estimate.
* page-limit-messages: Number of messages
* page-full-message-policy: fail or drop

We will now allow paging, until these max values and then fail or drop messages.

Once these values are retracted, the address will remain full until a period where cleanup is kicked in by paging. So these values may have a certain delay on being applied, but they should always be cleared once cleanup happened.
2023-01-30 18:19:45 -05:00
Justin Bertram bea8d21ecd ARTEMIS-4145 MQTT shared sub queue may be inadvertently removed
o.a.a.a.c.p.m.MQTTSubscriptionManager#removeSubscription() had a chunk
of code from 971f673c60 removed. That code
was added under the assumption that there should only ever be one
consumer per queue. That was true for MQTT 3.x, but it's not always true
for MQTT 5 due to shared subscriptions. However, the tests from that
commit all still pass even with it removed now (as well as all the other
MQTT tests) so I think it's safe.
2023-01-30 12:09:40 -05:00
Justin Bertram 49f8846861 ARTEMIS-4146 reauthenticated subjects are not cached 2023-01-30 12:02:08 -05:00
Timothy Bish 4785995f58 ARTEMIS-3357 Properly compare the subscription address on client re-attach
If the client is using address prefixes to define the routing type along with
durable subscriptions then on re-attach the compairon to check if the subscription
address has changed needs to remove the prefix when comparing against the address
since the prefix isn't propagated when creating the address and will always fail
resulting in the subscription queue being deleted in error.
2023-01-27 16:28:02 -05:00
Timothy Bish 6e10908c50 ARTEMIS-3153 Add tests for address prefixes for AMQP links
Adds some tests to validate that the destination prefixes if set and
are used properly by the client are honored over the default address
auto create routing type condiguration.
2023-01-26 12:56:39 -05:00
Domenico Francesco Bruscino ac7b0e85fb ARTEMIS-4113 Fix NPE for backup brokers with connection routers
The nodeID on backup brokers is available only after they become live.
2023-01-26 09:55:31 -05:00
Timothy Bish ca66028b2a ARTEMIS-4132 AMQP Receiver default to ANYCAST when creating an address
When an AMQP client subscribes to a new address (non-existing) with a receiver link, the
address is created with routing type ANYCAST regardles of the default address creation
configuration of the broker, and ignores even the broker wide default of MULTICAST.
2023-01-23 13:09:36 -05:00
Clebert Suconic 0d3cd8d880 ARTEMIS-4136 Mirrored sync replica
I am adding an option sync=true or false on mirror. if sync, any client blocking operation will wait a roundtrip to the mirror
acting like a sync replica.
2023-01-23 12:38:11 -05:00
Justin Bertram b0ba8cae24 ARTEMIS-4137 MQTT sub-queue clean-up can fail due to auth 2023-01-23 16:18:03 +01:00
Justin Bertram 8a974fe89c ARTEMIS-4133 msg w/null prop value can't be consumed via STOMP 2023-01-17 12:18:49 -06:00
Ryan Highley bb8761fcd2 ARTEMIS-3790 Support masked JMS credentials
Adds standard username and password value unmasking for JMS
ActiveMQConnectionFactory instances
2023-01-17 10:25:46 -06:00
Andy Taylor b02002fc66 ARTEMIS-3875 - adding consumer and producer metrics
https://issues.apache.org/jira/browse/ARTEMIS-3875
2023-01-13 15:49:44 -05:00
Justin Bertram c190d1c72f ARTEMIS-4127 refactor multiprotocol JMS tests in AMQP package
Over time org.apache.activemq.artemis.tests.integration.amqp has become
home to many multi-protocol JMS tests even though the package is really
for AMQP-specific tests. This commit splits those tests out into their
own package.

This is a preliminary step to clarify these tests before I add another
one for a different issue.
2023-01-12 16:19:48 -05:00
Clebert Suconic abd62665ce ARTEMIS-4125 Fixing AutoCreateJmsDestinationTest::testAutoCreateOnSubscribeToTopic 2023-01-12 16:17:28 -05:00
Justin Bertram e531c61c95 ARTEMIS-4126 address not auto-created when sending MQTT msg 2023-01-09 10:46:06 -06:00
Justin Bertram 114302a093 ARTEMIS-4125 address can be removed inadvertently
When the last non-durable subscriber on a JMS topic disconnects the
corresponding queue representing the subscription is deleted as
expected. However, the queue's address will also be deleted no matter
what, which is *not* expected.
2023-01-09 10:45:56 -06:00
Justin Bertram 56167b5e13 ARTEMIS-4122 support timed refresh for LegacyLDAPSecuritySettingPlugin
Some LDAP servers (e.g. OpenLDAP) do not support the "persistent search"
feature and therefore the existing "listener" feature does not actually
fetch updates. This commit implements a "pull" feature controlled by a
configurable interval equivalent to what is implemented in the cached
LDAP authorization module from ActiveMQ "Classic."
2023-01-07 22:54:35 -06:00
Justin Bertram c354c8e642 ARTEMIS-4016 fixing tests 2023-01-07 22:54:26 -06:00
Justin Bertram bf1d81e09e ARTEMIS-3707 fixing tests
A handful of tests started to fail after the original fix was committed.
This commit fixes those failures mainly by using a mock
`TransactionSynchronizationRegistry`.

I changed `o.a.a.a.r.ActiveMQRAManagedConnection#checkTransactionActive`
slightly because `getTransactionStatus` will never return `null` unlike
`getTransaction` would. The semantics should still be the same, though.
2023-01-07 22:54:17 -06:00
Robbie Gemmell 0dcb3aba2e ARTEMIS-4110: update a few more files introduced recently from older PRs being merged 2022-12-20 09:45:52 +00:00
AntonRoskvist 04ddeb647c ARTEMIS-4016 Bridges created by management operations are removed on restart and config reload 2022-12-17 10:10:10 -06:00
AntonRoskvist 6dd7965906
ARTEMIS-3834 include paged messages sending to DLA 2022-12-17 00:03:23 -06:00
Clebert Suconic 9816e1b999 ARTEMIS-3085 Fixing IOCriticalErrorListenerTest 2022-12-16 05:32:59 -05:00
Clebert Suconic f5d9eed9ce ARTEMIS-3085 Fixing test and proper wiring listener 2022-12-16 05:27:18 -05:00
Somdatta 00592d6dd8
ARTEMIS-3085 support custom IOCriticalErrorListener 2022-12-15 16:35:20 -06:00
AntonRoskvist 5c2a0d744c
ARTEMIS-4091 - Make scaleDown target more deterministic 2022-12-15 16:05:26 -06:00
Justin Bertram 038e95adb9 ARTEMIS-4109 unable to auto-delete q for MQTT retained msg 2022-12-15 13:44:55 -06:00