Commit Graph

453 Commits

Author SHA1 Message Date
Justin Bertram 2a452bd7a3 ARTEMIS-2578 clarify storage capacity messages
This is a surprisingly large change just to fix some log messages, but
the changes were necessary in order to get the relevant data to where it
was being logged. The fact that the data wasn't readily available is
probably why it wasn't logged in the first place.
2019-12-17 20:27:10 -05:00
Clebert Suconic 91cbbb8698 ARTEMIS-2559 Connection failure should rollback pending XA TX 2019-11-21 17:52:45 -05:00
Christopher L. Shannon (cshannon) fb54707e2d ARTEMIS-2549 - Add Downstream support to Federation
This commit introduces the ability to configure a downstream connection
for federation.  This works by sending information to the remote broker
and that broker will parse the message and create a new upstream back
to the original broker.
2019-11-20 08:47:16 -05:00
Justin Bertram 84067d8fef ARTEMIS-2504 implement retroactive addresses
A new feature to preserve messages sent to an address for queues that will be
created on the address in the future. This is essentially equivalent to the
"retroactive consumer" feature from 5.x. However, it's implemented in a way
that fits with the address model of Artemis.
2019-10-28 09:01:42 -04:00
Clebert Suconic 63b93499d2 ARTEMIS-2500 Fixing test after test rename 2019-09-23 20:43:54 -04:00
Howard Gao a2504288c6 ARTEMIS-2500 CoreMessage doesn't make a ful copy of its props on moveHeadersAndProperties
When CoreMessage is doing copyHeadersAndProperties() it doesn't
make a full copy of its properties (a TypedProperties object).
It will cause problem when multiple threads/parties are modifying the
properties of the copied messages from the same message.

This will be particular bad if the message is a large message
where moveHeadersAndProperties is being used.
2019-09-23 20:06:41 -04:00
Clebert Suconic 91f4d0a633 [maven-release-plugin] prepare for next development iteration 2019-09-23 11:19:54 -04:00
Clebert Suconic 86a975fdad [maven-release-plugin] prepare release 2.10.1 2019-09-23 11:19:42 -04:00
Clebert Suconic 70c2200c54 ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues into some integration usage
Revert "ARTEMIS-2336 Use zero copy to replicate journal/page/large message file"

This reverts commit 85b93f0883.
2019-09-18 11:55:23 -04:00
Clebert Suconic b846f356bb ARTEMIS-2462 Applying fix on delete SNF queue after ScaleDown 2019-09-17 14:05:00 -04:00
Clebert Suconic d55ec37195 Revert "ARTEMIS-2462 Allow store-forward queue to be deleted afte scaledown"
This reverts commit 397cef699a.
2019-09-17 14:05:00 -04:00
Francesco Nigro 9413925957 ARTEMIS-2473 RemoteQueueBindingImpl should check for empty filters 2019-09-06 14:26:03 -04:00
Wei Yang 4a61d2dc76 ARTEMIS-2380 Fix delivering message in the case of consume close 2019-09-04 11:02:16 +08:00
Howard Gao 397cef699a ARTEMIS-2462 Allow store-forward queue to be deleted afte scaledown
After a node is scaled down to a target node, the sf queue in the
target node is not deleted.

Normally this is fine because may be reused when the scaled down
node is back up.

However in cloud environment many drainer pods can be created and
then shutdown in order to drain the messages to a live node (pod).
Each drainer pod will have a different node-id. Over time the sf
queues in the target broker node grows and those sf queues are
no longer reused.

Although use can use management API/console to manually delete
them, it would be nice to have an option to automatically delete
those sf queue/address resources after scale down.

In this PR it added a boolean configuration parameter called
cleanup-sf-queue to scale down policy so that if the parameter
is "true" the broker will send a message to the
target broker signalling that the SF queue is no longer
needed and should be deleted.

If the parameter is not defined (default) or is "false"
the scale down won't remove the sf queue.
2019-08-28 21:16:38 +08:00
Clebert Suconic 5016f69daa [maven-release-plugin] prepare for next development iteration 2019-08-26 10:42:13 -04:00
Clebert Suconic f092d5396a [maven-release-plugin] prepare release 2.10.0 2019-08-26 10:42:01 -04:00
Justin Bertram 51c2022f38 ARTEMIS-2457 implement ring queue 2019-08-25 23:29:44 -04:00
Michael Pearce 25d0b511ce ARTEMIS-2458 Fix AMQP Transaction Session Close Ordering 2019-08-21 09:34:08 -04:00
Clebert Suconic 61eb379741 ARTEMIS-2458 Fix AMQP Transaction Rollback Ordering by using a sorted add 2019-08-21 09:34:07 -04:00
Wei Yang edace8845e ARTEMIS-2414 Sync before closing file in case data loss 2019-07-29 14:44:26 -04:00
Clebert Suconic 8a1f267bd5 Revert "ARTEMIS-2423 Improving Consumer/Queue Delivery lock"
This reverts commit 7507a9fd4b.
2019-07-27 10:45:08 -04:00
Clebert Suconic 7507a9fd4b ARTEMIS-2423 Improving Consumer/Queue Delivery lock 2019-07-26 15:05:51 -04:00
Maarten Boekhold 7f4f4a5021 ARTEMIS-2412 Allow CF configuration through JNDI references
Most connection related properties, like the SSL ones, currently
have to be encoded in the brokerURL. When configuring connections
purely through JNDI bindings, this is not always desireable.
This commit allows one to configure all properties included
in TransportConstants.ALLOWABLE_CONNECTOR_KEYS to be listed separately
in the JNDI bindings. These properties are then zipped into any
provided brokerURL. For properties that appear in both places,
the one specified separately in the JNDI bindings takes priority.

This commit should not affect any configuration other than those
configure through JNDIReferenceFactory.
2019-07-22 10:44:50 -04:00
Tomas Hofman 1c45d1758d ARTEMIS-2409 Convert HornetQ field names in consumer/queue selector strings 2019-07-17 11:34:04 -04:00
Clebert Suconic 27b151bb0e NO-JIRA moving Wait and removing some duplicate code on artemis-junit module 2019-07-16 22:30:59 -04:00
yang wei 85b93f0883 ARTEMIS-2336 Use zero copy to replicate journal/page/large message file 2019-07-15 23:51:42 +01:00
yang wei fa46647818 ARTEMIS-2425 Message loss due to writing incomplete page file 2019-07-12 19:52:43 +08:00
Howard Gao c66d62e4b0 ARTEMIS-2390 JMSMessageID header can be null when messages are cross-protocol
If a jms client (be it openwire, amqp, or core jms) receives a message that
is from a different protocol, the JMSMessageID maybe null when the
jms client expects it.
2019-06-27 10:26:34 -04:00
Clebert Suconic 9a32286883 ARTEMIS-2381 making setBrokerURL public as requested by Camel Devs 2019-06-19 11:35:28 -05:00
Francesco Nigro 4445261f34 ARTEMIS-2382 Reclaimer doesn't need to be instantiatable 2019-06-17 22:31:30 -04:00
Justin Bertram 3ff75c3353 ARTEMIS-1982 queue metrics can go negative
When redelivery is exhausted and messages are sent to a DLA with bindings
then some queue metrics can go negative.
2019-06-12 17:22:30 -04:00
Clebert Suconic a7e5e6d074 NO-JIRA test improvements 2019-05-31 19:06:15 -04:00
Clebert Suconic b7047faea5 NO-JIRA Adding checking for leaking server socket 2019-05-31 19:06:15 -04:00
Justin Bertram dbb2b54fa7 [maven-release-plugin] prepare for next development iteration 2019-05-29 22:02:00 -05:00
Justin Bertram 88c686cc99 [maven-release-plugin] prepare release 2.9.0 2019-05-29 21:59:18 -05:00
Justin Bertram 5768f6e2f3 ARTEMIS-2308 Support exporting metrics 2019-05-29 15:07:45 -04:00
Clebert Suconic 2ae732cfc8 [maven-release-plugin] prepare for next development iteration 2019-05-16 09:58:27 -04:00
Clebert Suconic 481b9824a8 [maven-release-plugin] prepare release 2.8.1 2019-05-16 09:58:15 -04:00
Francesco Nigro 49836eca52 ARTEMIS-2321 PageCursorProviderImpl can use primitive maps 2019-05-13 10:24:17 -04:00
Emmanuel Hugonnet ee7d5bd0f1 ARTEMIS-2319 Upgrading jboss-logging dependencies
* Upgrading versions
* Adding wildfly-common dependency as jboss-logmanager now depends on it
for simple common operations such as getting hostname or process id
* Updating bootclasspath with wildfly-common
2019-05-07 16:13:48 -04:00
Clebert Suconic 6f572de59e NO-JIRA Fixing random failure on Journal Test 2019-05-07 15:45:48 -04:00
Clebert Suconic 56152a9e7c [maven-release-plugin] prepare for next development iteration 2019-05-02 16:39:10 -04:00
Clebert Suconic a8e7f9d07a [maven-release-plugin] prepare release 2.8.0 2019-05-02 16:38:58 -04:00
Clebert Suconic 228f6af4ca [maven-release-plugin] prepare for next development iteration 2019-04-25 16:52:09 -04:00
Clebert Suconic 09d2ba18a7 [maven-release-plugin] prepare release 2.8.0 2019-04-25 16:51:57 -04:00
Francesco Nigro 881143252c ARTEMIS-2317 Avoid long TTSP caused by Page::read using mmap read
It implements Page::read using chunked reading of SequentialFile
instead of using memory mapped files causing long time to safepoint
issues.
2019-04-25 15:43:01 -04:00
Justin Bertram 73bdbad8b4 ARTEMIS-2310 support system prop sub in xincludes
Historically the broker has read the XML configuration file as a String,
substituted system properties, and then parsed that String into an XML
document. However, this method won't substitute system properties in the
files which are imported via xinclude. In order to substitue system
properties in xincluded files the substitution needs to be performed
after the file is parsed into an XML document. This commit implements
that change and refactors the XMLUtil class a bit to eliminate redundant
code, obsolete comments, etc.
2019-04-25 10:51:12 -04:00
Michael André Pearce 98b3177e6e ARTEMIS-2306 Make key used for group first for consumer configurable
Add tests
Add implementation details
2019-04-16 15:43:16 -04:00
Clebert Suconic da4f95cf71 ARTEMIS-2305 ACK counters to only increment after commit
Also including a new metric for ack attempts that will keep the former semantic.
2019-04-12 14:34:51 -04:00
Howard Gao 70e3888910 NO-JIRA Fix QueueImplTest#testRoundRobinWithQueueing
This test fails occasionally because the queue's delivering thread
may interference with the consumer's iterator during consumers adding.
The result is that the first of the 2 consumers may get iterated
twice and therefore the messages received by the 2 consumers are
not even.

Tha change puts the message add after the consumer add so that
the delivering thread only kicks off after consumers are all added
and messages should be evenly distributed to both consumers.
2019-03-15 12:06:56 +08:00
Clebert Suconic 516b1a1b9f [maven-release-plugin] prepare for next development iteration 2019-03-14 12:41:04 -04:00
Clebert Suconic c3eb6c12ad [maven-release-plugin] prepare release 2.7.0 2019-03-14 12:40:52 -04:00
yb b672cc3ad9 ARTEMIS-2274 Test showing issue with Message overflowing buffer when sizes are too close to TimedBuffer size 2019-03-13 14:56:33 -04:00
Michael André Pearce 4a5af776d8 ARTEMIS-2265 Support Federated Queues and Addresses
Implement Federated Queue (builds on recent consumer priority)
Implement Federated Address (builds on recent queue level auto-delete)
Add Functional Tests
Add Failure Tests (remote and local shutdowns)
Add Documentation
2019-03-07 14:37:00 -05:00
Clebert Suconic 061fb2787e ARTEMIS-1977 Stripping activemq-artemis as a separated proejct 2019-03-07 11:34:15 -05:00
Michael André Pearce 9b01e9521c ARTEMIS-2263 Support queue level auto-delete configuration
Add ability to configure when creating auto created queues at the queue level
Add support for configuring message count check
Add test cases
Update docs
2019-02-27 08:13:19 +00:00
Michael André Pearce c4e68b6046 ARTEMIS-2118 Enhanced Message Groups Support
Support using group buckets on a queue for better local group scaling
Support disabling message groups on a queue
Support rebalancing groups when a consumer is added.
2019-02-21 19:58:34 -05:00
Francesco Nigro 4da9d84311 ARTEMIS-2239 Zero-copy NIO/MAPPED TimedBuffer
NIO/MAPPED journal types can use directly the buffer of TimedBuffer
to perform file writes, avoiding an expensive copy + zeroing.
2019-02-14 00:20:51 +00:00
Clebert Suconic 59ada66033 ARTEMIS-1058 Improving web temporary cleanup
* Using SpawnedVMSupport (used to be on testsuite, moving it to Utils)
* Building the classpath for ./lib, similar to what happens on Bootstrap
* Using Path as much as possible to avoid issues encoding files
2019-02-06 09:36:04 -05:00
Michael André Pearce 7dfa0fe7f4 ARTEMIS-196 Implement Consumer Priority
Add consumer priority support
Includes refactor of consumer iterating in QueueImpl to its own logical class, to be able to implement.
Add OpenWire JMS Test - taken from ActiveMQ5
Add Core JMS Test
Add AMQP Test
Add Docs
2019-01-23 11:19:24 -05:00
Justin Bertram 61e0354b1a ARTEMIS-1867 FQQN for producers
There's a *slight* semantic change with the behavior of the queue query
and binding query to make them consistent with the address query, namely
that they will return the name of the queue and the name of the address
in every case and the returned names will be not use the FQQN syntax but
will be parsed to reflect their actual names in the broker.
2019-01-18 09:03:30 +00:00
Clebert Suconic b3f0a87e0d ARTEMIS-2200 NPE when calling journal.delete from Multiple Threads 2019-01-17 15:38:51 -05:00
Clebert Suconic d79762fa04 ARTEMIS-2205 Refactor AMQP Processing into Netty Thread
These improvements were also part of this task:
- Routing is now cached as much as possible.
- A new Runnable is avoided for each individual message,
  since we use the Netty executor to perform delivery

https://issues.apache.org/jira/browse/ARTEMIS-2205
2019-01-10 16:39:56 +01:00
Francesco Nigro c1eba53b77 ARTEMIS-2220 Fix PageCursorStressTest::testSimpleCursorWithFilter NPE
FakeQueue is not correctly setting the queue on its PageSubscription,
leading to fail the test due to NPEs when PageSubscription::getQueue
is being used.
2019-01-09 16:56:08 -05:00
Martyn Taylor 4445e7ce18 ARTEMIS-2208 Fix Unit Tests Pom 2018-12-19 09:51:27 +00:00
Romain Pelisse 39b177d027 ARTEMIS-2085 - Improve validation of MDB activation config properties values 2018-12-13 11:14:37 -06:00
Justin Bertram f4396da9fd ARTEMIS-2108 fix another potential StackOverflow 2018-11-13 17:37:51 -05:00
Francesco Nigro 270b383e80 ARTEMIS-1710 Allow management msgs to exceed global-max-size limit 2018-11-03 23:34:59 +01:00
Justin Bertram 97612c48d3 ARTEMIS-1856 support delays before deleting addresses & queues 2018-10-26 19:30:06 +01:00
Francesco Nigro 2967df6a99 ARTEMIS-1996 MappedSequentialFileFactory may cause DirectByteBuffer off-heap memory leaks
Compaction is now reusing direct ByteBuffers on both
reading and writing with explicit and deterministic
release to avoid high peak of native memory utilisation
after compaction.
2018-10-22 09:16:15 +02:00
Michael André Pearce 547b2aa592 ARTEMIS-2117 Add custom LVQ Key and Non Destructive Queue into Broker
Implement custom LVQ Key and Non-Destructive in broker - protocol agnostic
Make feature configurable via broker.xml, core apis and activemqservercontrol 
Add last-value-key test cases
Add non-destructive with lvq test cases 
Add non-destructive with expiry-delay test cases
Update documents
Add new methods to support create, update with new attributes
Refactor to pass through queue-attributes in client side methods to reduce further method changes for adding new attributes in future and avoid methods with endless parameters. (note: in future this should prob be done server side too)

Update existing test cases and fake impls for new methods/attributes
2018-10-17 20:53:13 -04:00
yang wei 8f4bd7c621 ARTEMIS-2102 delete paging directory or table if address is removed 2018-10-10 18:50:11 -04:00
Francesco Nigro 3112b4f3db ARTEMIS-2093 NPE thrown by NettyConnector::createConnection
Given that NettyConnector::createConnection isn't happening on the
channel's event loop, it could race with a channel close event, that
would clean the whole channel pipeline, leading to a NPE while
trying to use a configured channel handler of the pipeline.
2018-09-21 15:06:53 +02:00
Clebert Suconic 5d69e3594f ARTEMIS-2074 Fixing RA properties 2018-09-12 16:47:24 -04:00
Michael André Pearce c417d0b5f8 ARTEMIS-2081 listConfiguredQueues returns only queues created by config
Extend test case to reproduce problem of client created queues being incorrectly removed on simple reload of config.
Add a flag/field to the queues created by configuration/broker.xml so we can correctly filter only queues created/managed by config.
Update listConfiguredQueues to use the new queue flag
2018-09-11 13:55:11 -04:00
Michael André Pearce 4b88f38b2d ARTEMIS-2076 Make Filter update-able
Add Tests
Add implementation inline with other queue updatable settings.
Enhance tests to ensure queue is not destroyed during config change and messages in queue already are preserved
2018-09-07 15:49:03 -04:00
Clebert Suconic 34fa0dded6 ARTEMIS-2052 Fixing initial credit negotiation 2018-08-29 13:37:48 -04:00
Clebert Suconic 6b1abd1aad ARTEMIS-2053 avoiding data loss after compacting 2018-08-26 18:17:29 -04:00
Timothy Bish 2c7896cef6 ARTEMIS-1938 Update Qpid JMS along with Proton and Netty
Update the Qpid JMS and Proton dependencies to lastest and sync Netty
with the 4.1.28.Final version used by Qpid JMS to avoid clash that
breaks a test.  Adds override of new Proton-J WritableBuffer API that
allows it to use the Netty String encoder when needed instead of the
slower default version.

Update Qpid JMS to v0.36.0
Proton-J to v0.29.0
Netty to 4.1.28.Final
2018-08-25 07:22:53 +01:00
Clebert Suconic 99d091a0ea ARTEMIS-2030 only use interrupt during shutdown on RA 2018-08-13 20:40:34 -04:00
Martyn Taylor ad6db74701 ARTEMIS-2024 Enable SharedClientID on ConnectionFactory 2018-08-13 20:41:54 +01:00
Justin Bertram df583922f5 ARTEMIS-2023 Support 1x prefixes for JMS dests created using session
In some cases users who migrate from 1.x to 2.x may still want to keep
the legacy prefixes for their JMS destinations (i.e. "jms.queue.",
"jms.topic.", etc.). This commit adds a boolean on our ConnectionFactory
implementation so that it will use the old prefixes when invoking the
queue/topic creation methods on the Session implementation.
2018-08-13 12:01:51 -04:00
Francesco Nigro 8dd0e9472f ARTEMIS-1999 Broker uses 100% core's CPU time if msg grouping is used
The deliver loop won't give up trying to deliver messages when
back-pressure kicks in (credits and/or TCP) if msg grouping is used and
there are many consumers registered: this change will allow the loop
to exit by instructing the logic that the group consumer is the only
consumer to check.
2018-08-08 17:37:58 +01:00
Clebert Suconic a92324be35 NO-JIRA Fixing MultiThreadAsynchronousFileTest on limited servers
This test was initializing a libaio of 21K, that would fail on limited servers.
This is decreasing maxIO so it would requires less resources to run it.
2018-08-03 13:09:41 -04:00
Clebert Suconic 0e36e072bd ARTEMIS-1732 I simplified some of the changes performed at the previous commit.
Also I changed GlobalDiskFullTest to actually block the senders.
I moved the Runnables from PagingManager into the Util as AtomicRunnable.
2018-07-31 21:39:04 -04:00
Howard Gao 53e1d60160 ARTEMIS-1732 AMQP anonymous producer not blocked on max-disk-usage
Anonymous senders (those created without a target address) are not
blocked when max-disk-usage is reached. The cause is that when such
a sender is created on the broker, the broker doesn't check the
disk/memory usage and gives out the credit immediately.
2018-07-31 21:39:01 -04:00
Michael André Pearce 8a9835a394 ARTEMIS-856 - Support consumersBeforeDispatch and delayBeforeDispatch
https://issues.apache.org/jira/browse/ARTEMIS-856

This is equivalent to consumersBeforeDispatchStarts and timeBeforeDispatchStarts in ActiveMQ 5.x

http://activemq.apache.org/message-groups.html

This is addressing one of the items on the artemis roadmap: http://activemq.apache.org/activemq-artemis-roadmap.html
2018-07-31 16:16:51 +01:00
Clebert Suconic be8c29d4e1 NO-JIRA Improve SpawnVMCheck 2018-07-12 16:51:35 +02:00
Justin Bertram f9af366f07 ARTEMIS-1951 fix comparison bug 2018-06-23 11:57:38 -05:00
Justin Bertram 754a263328 ARTEMIS-1951 allow queue's user to be updated 2018-06-22 10:38:05 -04:00
Justin Bertram 57ed5b0530 ARTEMIS-1926 refactor SSLSupport 2018-06-18 16:15:13 -04:00
Clebert Suconic b6fba64d9e ARTEMIS-1913 Dependency to netty tcpnative should be optional 2018-06-06 16:13:31 -04:00
Johan Stenberg 0ea84ef9ff [ARTEMIS-1890] Fix any-word wildcard matching in AddressImpl to match zero words 2018-05-31 18:57:49 -04:00
Christopher L. Shannon (cshannon) f09a41d433 ARTEMIS-1888 - Add forceSSLParameters flag to override system SSL properties
If true the connection factory will prefer SSL settings set via the connector configuration vs system properties
2018-05-25 12:30:16 -04:00
Justin Bertram 3602713a7e ARTEMIS-1882 verify PKCS12 keystores work 2018-05-24 11:58:43 -04:00
Clebert Suconic af56b67e38 [maven-release-plugin] prepare for next development iteration 2018-05-16 11:03:26 -04:00
Clebert Suconic bcaa11c2ee [maven-release-plugin] prepare release 2.6.0 2018-05-16 11:03:14 -04:00
Christian Sailer 2d59a0d3db ARTEMIS-1746: Fix client side OpenSSL 2018-05-15 09:48:12 -04:00
Clebert Suconic 61d2cc5bb0 NO-JIRA Adding checks for spawned servers on the testsuite 2018-05-04 22:43:56 -04:00
Clebert Suconic e8104586b4 ARTEMIS-1837 Replication Initial catchup deadlocks if clients disconnect
This is fixing existing tests, so no more tests are needed.
2018-04-30 11:27:51 -04:00