Commit Graph

473 Commits

Author SHA1 Message Date
Clebert Suconic ca4c4068df ARTEMIS-2732 Logging cleanup 2020-04-23 17:49:08 -04:00
Clebert Suconic e8d23f3144 NO-JIRA Retry test on intermittent failure 2020-04-23 16:51:22 -04:00
Clebert Suconic 11a3e810bb [maven-release-plugin] prepare for next development iteration 2020-04-21 17:06:21 -04:00
Clebert Suconic 30272e0c2f [maven-release-plugin] prepare release 2.12.0 2020-04-21 17:06:08 -04:00
Clebert Suconic d231e2ac63 [maven-release-plugin] prepare for next development iteration 2020-04-16 16:16:41 -04:00
Clebert Suconic 9636f4a3b0 [maven-release-plugin] prepare release 2.12.0 2020-04-16 16:16:28 -04:00
Justin Bertram 410a552894 ARTEMIS-2711 use peer host:port for acceptor SSL engine 2020-04-16 10:32:54 -04:00
Francesco Nigro 46060b89ff ARTEMIS-2676 PageCursorProviderImpl::cleanup can save decoding pages without large messages 2020-04-15 22:20:50 -04:00
Justin Bertram 701b4e4794 ARTEMIS-2692 refactor queue creation
This commit changes all tests to use the new methods.
2020-04-13 14:25:30 -05:00
Justin Bertram 2efa44daf5 ARTEMIS-2692 refactor queue creation
This commit does the following:
- Deprecates existing overloaded createQueue, createSharedQueue,
  createTemporaryQueue, & updateQueue methods for ClientSession,
  ServerSession, ActiveMQServer, & ActiveMQServerControl where
  applicable.
- Deprecates QueueAttributes, QueueConfig, & CoreQueueConfiguration.
- Deprecates existing overloaded constructors for QueueImpl.
- Implements QueueConfiguration with JavaDoc to be the single,
  centralized configuration object for both client-side and broker-side
  queue creation including methods to convert to & from JSON for use in
  the management API.
- Implements new createQueue, createSharedQueue & updateQueue methods
  with JavaDoc for ClientSession, ServerSession, ActiveMQServer, &
  ActiveMQServerControl as well as a new constructor for QueueImpl all
  using the new QueueConfiguration object.
- Changes all internal broker code to use the new methods.
2020-04-13 14:25:30 -05:00
Clebert Suconic af796d5ce4 ARTEMIS-2701 Improving DLQ/check over previously removed records 2020-04-06 12:25:35 -04:00
Francesco Nigro 493ce497c7 ARTEMIS-1975 Fix AMQP paged large message usage counter 2020-04-01 17:46:26 -04:00
Justin Bertram 0cb3c96b3e ARTEMIS-1194 fix test 2020-03-10 10:32:30 -04:00
Andrius Dagys f1474ce7c8 ARTEMIS-1194 Add SOCKS support
Add a Netty socks proxy handler during channel initialisation to allow
Artemis to communicate via a SOCKS proxy. Supports SOCKS version 4a & 5.
Even if enabled in configuration, the proxy will not be used when the
target host is a loopback address.
2020-03-07 21:35:38 +00:00
Clebert Suconic ddd8ed4402 ARTEMIS-1975 Real Large Message support into AMQP
This is a Large commit where I am refactoring largeMessage Body out of CoreMessage
which is now reused with AMQP.

I had also to fix Reference Counting to fix how Large Messages are Acked

And I also had to make sure Large Messages are transversing correctly when in cluster.
2020-02-25 15:25:01 +01:00
Francesco Nigro fa0c187ae6 ARTEMIS-2602 Reduce number of copies for non JDBC Journal 2020-01-18 18:45:12 +01:00
Francesco Nigro b10d765139 ARTEMIS-2602 load surviving records into SparseArrayLinkedList 2020-01-17 14:31:52 -05:00
Justin Bertram 97735ca72c [maven-release-plugin] prepare for next development iteration 2020-01-10 09:00:58 -06:00
Justin Bertram 25a947f6cb [maven-release-plugin] prepare release 2.11.0 2020-01-10 08:57:46 -06:00
Francesco Nigro 6cc5749c7b ARTEMIS-2569 LinkedListImpl tests should not rely on the GC mechanism 2020-01-09 13:40:43 -05:00
Justin Bertram 2a452bd7a3 ARTEMIS-2578 clarify storage capacity messages
This is a surprisingly large change just to fix some log messages, but
the changes were necessary in order to get the relevant data to where it
was being logged. The fact that the data wasn't readily available is
probably why it wasn't logged in the first place.
2019-12-17 20:27:10 -05:00
Clebert Suconic 91cbbb8698 ARTEMIS-2559 Connection failure should rollback pending XA TX 2019-11-21 17:52:45 -05:00
Christopher L. Shannon (cshannon) fb54707e2d ARTEMIS-2549 - Add Downstream support to Federation
This commit introduces the ability to configure a downstream connection
for federation.  This works by sending information to the remote broker
and that broker will parse the message and create a new upstream back
to the original broker.
2019-11-20 08:47:16 -05:00
Justin Bertram 84067d8fef ARTEMIS-2504 implement retroactive addresses
A new feature to preserve messages sent to an address for queues that will be
created on the address in the future. This is essentially equivalent to the
"retroactive consumer" feature from 5.x. However, it's implemented in a way
that fits with the address model of Artemis.
2019-10-28 09:01:42 -04:00
Clebert Suconic 63b93499d2 ARTEMIS-2500 Fixing test after test rename 2019-09-23 20:43:54 -04:00
Howard Gao a2504288c6 ARTEMIS-2500 CoreMessage doesn't make a ful copy of its props on moveHeadersAndProperties
When CoreMessage is doing copyHeadersAndProperties() it doesn't
make a full copy of its properties (a TypedProperties object).
It will cause problem when multiple threads/parties are modifying the
properties of the copied messages from the same message.

This will be particular bad if the message is a large message
where moveHeadersAndProperties is being used.
2019-09-23 20:06:41 -04:00
Clebert Suconic 91f4d0a633 [maven-release-plugin] prepare for next development iteration 2019-09-23 11:19:54 -04:00
Clebert Suconic 86a975fdad [maven-release-plugin] prepare release 2.10.1 2019-09-23 11:19:42 -04:00
Clebert Suconic 70c2200c54 ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues into some integration usage
Revert "ARTEMIS-2336 Use zero copy to replicate journal/page/large message file"

This reverts commit 85b93f0883.
2019-09-18 11:55:23 -04:00
Clebert Suconic b846f356bb ARTEMIS-2462 Applying fix on delete SNF queue after ScaleDown 2019-09-17 14:05:00 -04:00
Clebert Suconic d55ec37195 Revert "ARTEMIS-2462 Allow store-forward queue to be deleted afte scaledown"
This reverts commit 397cef699a.
2019-09-17 14:05:00 -04:00
Francesco Nigro 9413925957 ARTEMIS-2473 RemoteQueueBindingImpl should check for empty filters 2019-09-06 14:26:03 -04:00
Wei Yang 4a61d2dc76 ARTEMIS-2380 Fix delivering message in the case of consume close 2019-09-04 11:02:16 +08:00
Howard Gao 397cef699a ARTEMIS-2462 Allow store-forward queue to be deleted afte scaledown
After a node is scaled down to a target node, the sf queue in the
target node is not deleted.

Normally this is fine because may be reused when the scaled down
node is back up.

However in cloud environment many drainer pods can be created and
then shutdown in order to drain the messages to a live node (pod).
Each drainer pod will have a different node-id. Over time the sf
queues in the target broker node grows and those sf queues are
no longer reused.

Although use can use management API/console to manually delete
them, it would be nice to have an option to automatically delete
those sf queue/address resources after scale down.

In this PR it added a boolean configuration parameter called
cleanup-sf-queue to scale down policy so that if the parameter
is "true" the broker will send a message to the
target broker signalling that the SF queue is no longer
needed and should be deleted.

If the parameter is not defined (default) or is "false"
the scale down won't remove the sf queue.
2019-08-28 21:16:38 +08:00
Clebert Suconic 5016f69daa [maven-release-plugin] prepare for next development iteration 2019-08-26 10:42:13 -04:00
Clebert Suconic f092d5396a [maven-release-plugin] prepare release 2.10.0 2019-08-26 10:42:01 -04:00
Justin Bertram 51c2022f38 ARTEMIS-2457 implement ring queue 2019-08-25 23:29:44 -04:00
Michael Pearce 25d0b511ce ARTEMIS-2458 Fix AMQP Transaction Session Close Ordering 2019-08-21 09:34:08 -04:00
Clebert Suconic 61eb379741 ARTEMIS-2458 Fix AMQP Transaction Rollback Ordering by using a sorted add 2019-08-21 09:34:07 -04:00
Wei Yang edace8845e ARTEMIS-2414 Sync before closing file in case data loss 2019-07-29 14:44:26 -04:00
Clebert Suconic 8a1f267bd5 Revert "ARTEMIS-2423 Improving Consumer/Queue Delivery lock"
This reverts commit 7507a9fd4b.
2019-07-27 10:45:08 -04:00
Clebert Suconic 7507a9fd4b ARTEMIS-2423 Improving Consumer/Queue Delivery lock 2019-07-26 15:05:51 -04:00
Maarten Boekhold 7f4f4a5021 ARTEMIS-2412 Allow CF configuration through JNDI references
Most connection related properties, like the SSL ones, currently
have to be encoded in the brokerURL. When configuring connections
purely through JNDI bindings, this is not always desireable.
This commit allows one to configure all properties included
in TransportConstants.ALLOWABLE_CONNECTOR_KEYS to be listed separately
in the JNDI bindings. These properties are then zipped into any
provided brokerURL. For properties that appear in both places,
the one specified separately in the JNDI bindings takes priority.

This commit should not affect any configuration other than those
configure through JNDIReferenceFactory.
2019-07-22 10:44:50 -04:00
Tomas Hofman 1c45d1758d ARTEMIS-2409 Convert HornetQ field names in consumer/queue selector strings 2019-07-17 11:34:04 -04:00
Clebert Suconic 27b151bb0e NO-JIRA moving Wait and removing some duplicate code on artemis-junit module 2019-07-16 22:30:59 -04:00
yang wei 85b93f0883 ARTEMIS-2336 Use zero copy to replicate journal/page/large message file 2019-07-15 23:51:42 +01:00
yang wei fa46647818 ARTEMIS-2425 Message loss due to writing incomplete page file 2019-07-12 19:52:43 +08:00
Howard Gao c66d62e4b0 ARTEMIS-2390 JMSMessageID header can be null when messages are cross-protocol
If a jms client (be it openwire, amqp, or core jms) receives a message that
is from a different protocol, the JMSMessageID maybe null when the
jms client expects it.
2019-06-27 10:26:34 -04:00
Clebert Suconic 9a32286883 ARTEMIS-2381 making setBrokerURL public as requested by Camel Devs 2019-06-19 11:35:28 -05:00
Francesco Nigro 4445261f34 ARTEMIS-2382 Reclaimer doesn't need to be instantiatable 2019-06-17 22:31:30 -04:00
Justin Bertram 3ff75c3353 ARTEMIS-1982 queue metrics can go negative
When redelivery is exhausted and messages are sent to a DLA with bindings
then some queue metrics can go negative.
2019-06-12 17:22:30 -04:00
Clebert Suconic a7e5e6d074 NO-JIRA test improvements 2019-05-31 19:06:15 -04:00
Clebert Suconic b7047faea5 NO-JIRA Adding checking for leaking server socket 2019-05-31 19:06:15 -04:00
Justin Bertram dbb2b54fa7 [maven-release-plugin] prepare for next development iteration 2019-05-29 22:02:00 -05:00
Justin Bertram 88c686cc99 [maven-release-plugin] prepare release 2.9.0 2019-05-29 21:59:18 -05:00
Justin Bertram 5768f6e2f3 ARTEMIS-2308 Support exporting metrics 2019-05-29 15:07:45 -04:00
Clebert Suconic 2ae732cfc8 [maven-release-plugin] prepare for next development iteration 2019-05-16 09:58:27 -04:00
Clebert Suconic 481b9824a8 [maven-release-plugin] prepare release 2.8.1 2019-05-16 09:58:15 -04:00
Francesco Nigro 49836eca52 ARTEMIS-2321 PageCursorProviderImpl can use primitive maps 2019-05-13 10:24:17 -04:00
Emmanuel Hugonnet ee7d5bd0f1 ARTEMIS-2319 Upgrading jboss-logging dependencies
* Upgrading versions
* Adding wildfly-common dependency as jboss-logmanager now depends on it
for simple common operations such as getting hostname or process id
* Updating bootclasspath with wildfly-common
2019-05-07 16:13:48 -04:00
Clebert Suconic 6f572de59e NO-JIRA Fixing random failure on Journal Test 2019-05-07 15:45:48 -04:00
Clebert Suconic 56152a9e7c [maven-release-plugin] prepare for next development iteration 2019-05-02 16:39:10 -04:00
Clebert Suconic a8e7f9d07a [maven-release-plugin] prepare release 2.8.0 2019-05-02 16:38:58 -04:00
Clebert Suconic 228f6af4ca [maven-release-plugin] prepare for next development iteration 2019-04-25 16:52:09 -04:00
Clebert Suconic 09d2ba18a7 [maven-release-plugin] prepare release 2.8.0 2019-04-25 16:51:57 -04:00
Francesco Nigro 881143252c ARTEMIS-2317 Avoid long TTSP caused by Page::read using mmap read
It implements Page::read using chunked reading of SequentialFile
instead of using memory mapped files causing long time to safepoint
issues.
2019-04-25 15:43:01 -04:00
Justin Bertram 73bdbad8b4 ARTEMIS-2310 support system prop sub in xincludes
Historically the broker has read the XML configuration file as a String,
substituted system properties, and then parsed that String into an XML
document. However, this method won't substitute system properties in the
files which are imported via xinclude. In order to substitue system
properties in xincluded files the substitution needs to be performed
after the file is parsed into an XML document. This commit implements
that change and refactors the XMLUtil class a bit to eliminate redundant
code, obsolete comments, etc.
2019-04-25 10:51:12 -04:00
Michael André Pearce 98b3177e6e ARTEMIS-2306 Make key used for group first for consumer configurable
Add tests
Add implementation details
2019-04-16 15:43:16 -04:00
Clebert Suconic da4f95cf71 ARTEMIS-2305 ACK counters to only increment after commit
Also including a new metric for ack attempts that will keep the former semantic.
2019-04-12 14:34:51 -04:00
Howard Gao 70e3888910 NO-JIRA Fix QueueImplTest#testRoundRobinWithQueueing
This test fails occasionally because the queue's delivering thread
may interference with the consumer's iterator during consumers adding.
The result is that the first of the 2 consumers may get iterated
twice and therefore the messages received by the 2 consumers are
not even.

Tha change puts the message add after the consumer add so that
the delivering thread only kicks off after consumers are all added
and messages should be evenly distributed to both consumers.
2019-03-15 12:06:56 +08:00
Clebert Suconic 516b1a1b9f [maven-release-plugin] prepare for next development iteration 2019-03-14 12:41:04 -04:00
Clebert Suconic c3eb6c12ad [maven-release-plugin] prepare release 2.7.0 2019-03-14 12:40:52 -04:00
yb b672cc3ad9 ARTEMIS-2274 Test showing issue with Message overflowing buffer when sizes are too close to TimedBuffer size 2019-03-13 14:56:33 -04:00
Michael André Pearce 4a5af776d8 ARTEMIS-2265 Support Federated Queues and Addresses
Implement Federated Queue (builds on recent consumer priority)
Implement Federated Address (builds on recent queue level auto-delete)
Add Functional Tests
Add Failure Tests (remote and local shutdowns)
Add Documentation
2019-03-07 14:37:00 -05:00
Clebert Suconic 061fb2787e ARTEMIS-1977 Stripping activemq-artemis as a separated proejct 2019-03-07 11:34:15 -05:00
Michael André Pearce 9b01e9521c ARTEMIS-2263 Support queue level auto-delete configuration
Add ability to configure when creating auto created queues at the queue level
Add support for configuring message count check
Add test cases
Update docs
2019-02-27 08:13:19 +00:00
Michael André Pearce c4e68b6046 ARTEMIS-2118 Enhanced Message Groups Support
Support using group buckets on a queue for better local group scaling
Support disabling message groups on a queue
Support rebalancing groups when a consumer is added.
2019-02-21 19:58:34 -05:00
Francesco Nigro 4da9d84311 ARTEMIS-2239 Zero-copy NIO/MAPPED TimedBuffer
NIO/MAPPED journal types can use directly the buffer of TimedBuffer
to perform file writes, avoiding an expensive copy + zeroing.
2019-02-14 00:20:51 +00:00
Clebert Suconic 59ada66033 ARTEMIS-1058 Improving web temporary cleanup
* Using SpawnedVMSupport (used to be on testsuite, moving it to Utils)
* Building the classpath for ./lib, similar to what happens on Bootstrap
* Using Path as much as possible to avoid issues encoding files
2019-02-06 09:36:04 -05:00
Michael André Pearce 7dfa0fe7f4 ARTEMIS-196 Implement Consumer Priority
Add consumer priority support
Includes refactor of consumer iterating in QueueImpl to its own logical class, to be able to implement.
Add OpenWire JMS Test - taken from ActiveMQ5
Add Core JMS Test
Add AMQP Test
Add Docs
2019-01-23 11:19:24 -05:00
Justin Bertram 61e0354b1a ARTEMIS-1867 FQQN for producers
There's a *slight* semantic change with the behavior of the queue query
and binding query to make them consistent with the address query, namely
that they will return the name of the queue and the name of the address
in every case and the returned names will be not use the FQQN syntax but
will be parsed to reflect their actual names in the broker.
2019-01-18 09:03:30 +00:00
Clebert Suconic b3f0a87e0d ARTEMIS-2200 NPE when calling journal.delete from Multiple Threads 2019-01-17 15:38:51 -05:00
Clebert Suconic d79762fa04 ARTEMIS-2205 Refactor AMQP Processing into Netty Thread
These improvements were also part of this task:
- Routing is now cached as much as possible.
- A new Runnable is avoided for each individual message,
  since we use the Netty executor to perform delivery

https://issues.apache.org/jira/browse/ARTEMIS-2205
2019-01-10 16:39:56 +01:00
Francesco Nigro c1eba53b77 ARTEMIS-2220 Fix PageCursorStressTest::testSimpleCursorWithFilter NPE
FakeQueue is not correctly setting the queue on its PageSubscription,
leading to fail the test due to NPEs when PageSubscription::getQueue
is being used.
2019-01-09 16:56:08 -05:00
Martyn Taylor 4445e7ce18 ARTEMIS-2208 Fix Unit Tests Pom 2018-12-19 09:51:27 +00:00
Romain Pelisse 39b177d027 ARTEMIS-2085 - Improve validation of MDB activation config properties values 2018-12-13 11:14:37 -06:00
Justin Bertram f4396da9fd ARTEMIS-2108 fix another potential StackOverflow 2018-11-13 17:37:51 -05:00
Francesco Nigro 270b383e80 ARTEMIS-1710 Allow management msgs to exceed global-max-size limit 2018-11-03 23:34:59 +01:00
Justin Bertram 97612c48d3 ARTEMIS-1856 support delays before deleting addresses & queues 2018-10-26 19:30:06 +01:00
Francesco Nigro 2967df6a99 ARTEMIS-1996 MappedSequentialFileFactory may cause DirectByteBuffer off-heap memory leaks
Compaction is now reusing direct ByteBuffers on both
reading and writing with explicit and deterministic
release to avoid high peak of native memory utilisation
after compaction.
2018-10-22 09:16:15 +02:00
Michael André Pearce 547b2aa592 ARTEMIS-2117 Add custom LVQ Key and Non Destructive Queue into Broker
Implement custom LVQ Key and Non-Destructive in broker - protocol agnostic
Make feature configurable via broker.xml, core apis and activemqservercontrol 
Add last-value-key test cases
Add non-destructive with lvq test cases 
Add non-destructive with expiry-delay test cases
Update documents
Add new methods to support create, update with new attributes
Refactor to pass through queue-attributes in client side methods to reduce further method changes for adding new attributes in future and avoid methods with endless parameters. (note: in future this should prob be done server side too)

Update existing test cases and fake impls for new methods/attributes
2018-10-17 20:53:13 -04:00
yang wei 8f4bd7c621 ARTEMIS-2102 delete paging directory or table if address is removed 2018-10-10 18:50:11 -04:00
Francesco Nigro 3112b4f3db ARTEMIS-2093 NPE thrown by NettyConnector::createConnection
Given that NettyConnector::createConnection isn't happening on the
channel's event loop, it could race with a channel close event, that
would clean the whole channel pipeline, leading to a NPE while
trying to use a configured channel handler of the pipeline.
2018-09-21 15:06:53 +02:00
Clebert Suconic 5d69e3594f ARTEMIS-2074 Fixing RA properties 2018-09-12 16:47:24 -04:00
Michael André Pearce c417d0b5f8 ARTEMIS-2081 listConfiguredQueues returns only queues created by config
Extend test case to reproduce problem of client created queues being incorrectly removed on simple reload of config.
Add a flag/field to the queues created by configuration/broker.xml so we can correctly filter only queues created/managed by config.
Update listConfiguredQueues to use the new queue flag
2018-09-11 13:55:11 -04:00
Michael André Pearce 4b88f38b2d ARTEMIS-2076 Make Filter update-able
Add Tests
Add implementation inline with other queue updatable settings.
Enhance tests to ensure queue is not destroyed during config change and messages in queue already are preserved
2018-09-07 15:49:03 -04:00
Clebert Suconic 34fa0dded6 ARTEMIS-2052 Fixing initial credit negotiation 2018-08-29 13:37:48 -04:00
Clebert Suconic 6b1abd1aad ARTEMIS-2053 avoiding data loss after compacting 2018-08-26 18:17:29 -04:00
Timothy Bish 2c7896cef6 ARTEMIS-1938 Update Qpid JMS along with Proton and Netty
Update the Qpid JMS and Proton dependencies to lastest and sync Netty
with the 4.1.28.Final version used by Qpid JMS to avoid clash that
breaks a test.  Adds override of new Proton-J WritableBuffer API that
allows it to use the Netty String encoder when needed instead of the
slower default version.

Update Qpid JMS to v0.36.0
Proton-J to v0.29.0
Netty to 4.1.28.Final
2018-08-25 07:22:53 +01:00
Clebert Suconic 99d091a0ea ARTEMIS-2030 only use interrupt during shutdown on RA 2018-08-13 20:40:34 -04:00