346 Commits

Author SHA1 Message Date
Alan Protasio
cd1d5eb785 AMQ-7163 - If the broker had an unclean shutdown and number of free pages is Zero after the recovery, the next shutdown will also be 'unclean' 2019-03-06 15:33:52 -08:00
Alan Protasio
87467dc61e AMQ-7159 - Adding a new attribute on PersistenceAdapterViewMBean to show information about Storage write/read latency 2019-03-06 11:41:08 -08:00
Hoan Nguyen
0262338687
close DataByteArrayOutputStream buff of WriteBatch 2019-02-26 19:07:26 -08:00
Alan Protasio
9e65435517 AMQ-7143 - Temporary transaction file (PageFile) being opened and closed many times, causing poor performance on high latency FS as NFS 2019-01-31 10:31:16 -05:00
Christopher L. Shannon (cshannon)
c3714457f1 AMQ-7136 - Improve recovery of durable subscription metrics in KahaDB
Updated metrics recovery to only have to iterate over the order index 1
time to recovery the pending metrics for the subscriptions instead of
making a pass over the index once per subscription
2019-01-15 14:22:26 -05:00
Alan Protasio
4b51f8b66c AMQ-7132 - ActiveMQ reads lots of index pages upon startup (after a graceful or ungraceful shutdown) 2019-01-14 10:54:26 -05:00
Christopher L. Shannon (cshannon)
a16b227808 AMQ-7129 - code cleanup 2019-01-11 10:31:44 -05:00
Christopher L. Shannon (cshannon)
fa2daa25e9 AMQ-7129 - fix durable message size statistics with individual ack
Make sure that the pending message size for a durable sub only includes
messages part of the ack range
2019-01-11 09:56:57 -05:00
Christopher L. Shannon (cshannon)
25de20c77e AMQ-7129 - Properly recover messages from KahaDB for a durable when there are
messages to recover before the stored lastAck value

With individual ack mode we need to check the durable ackPosition
sequence set in the KahaDB index on subsription load to see if there are
earlier messages before the lastAck value that still haven't been acked.
While this normally wouldn't happen it is possible in individual ack
mode
2019-01-09 14:34:06 -05:00
jgoodyear
4a21edc8d5 [AMQ-7126] Improvement to perf of 5266Test 2019-01-07 21:22:34 -03:30
jgoodyear
b86c666c40 [AMQ-7126] Prevent OOM when recovering KahaDB and memory space is insufficient to load full page. Thank you Heath Kesler and Johan Edstrom for investigation and patch. 2019-01-03 15:46:39 -03:30
jgoodyear
612d4aeeb4 AMQ-7118 This closes #327 - with thanks to Heath Kesler 2018-12-04 12:54:30 -03:30
Alan Protasio
9012a7871b AMQ-7091 - O(n) Memory consumption when broker has inactive durable subscribes causing OOM 2018-11-12 17:48:53 -03:30
gtully
85859fd8dc AMQ-7082 We should make sure that pages managed during recovery are not recovered in error
variation of patch from Alan Protasio <alanprot@gmail.com> closes #317
2018-11-07 11:29:14 +00:00
Alan Protasio
0d34338919 AMQ-7082 - Make sure that the recovery will only mark pages as free if they were created in a previous execution 2018-11-06 04:48:22 -08:00
gtully
bf8eb08aca AMQ-7086 - make kahadb gc/cleanup on shutdown optional to trade availability over disk usage for fast failover 2018-10-31 14:29:05 +00:00
gtully
8a1abd9bb2 AMQ-7084 - ensure allocated and unused free pages are visible to subsequent transactions, fix and test with test updates to reflect proper usage 2018-10-23 16:00:57 +01:00
gtully
fb1e642c15 AMQ-7082 - ensure flush sees free list, use lazySet to avoid store barrier, thanks cshannon 2018-10-22 15:16:26 +01:00
gtully
efa4e683bc AMQ-7082 - fix final ref in test 2018-10-19 16:45:04 +01:00
gtully
79c74998dc AMQ-7082 - recover index free pages in parallel with start, merge in flush, clean shutdown if complete. follow up on AMQ-6590 2018-10-19 16:00:23 +01:00
gtully
ed727d22e1 AMQ-7067 - track xa commit outcomes in ack compaction such that there are no dangling prepared tx on full recovery, fix and test 2018-10-17 18:13:36 +01:00
gtully
5d3a3fcca7 AMQ-6590 - rework fix to take the recovery hit on clean shutdown rather than on restart, trade off availability for short term disk usage 2018-10-17 12:32:04 +01:00
gtully
7c890d4776 AMQ-7067 - test and fix for eager ack compaction moving acks from data files with in progress tx and clearing the link from the prepare record, that now uses the same ack map 2018-10-12 14:51:37 +01:00
gtully
a311139bfe AMQ-7067 - ensure updates to ackMessageFileMap are protected by the index lock 2018-10-09 12:55:11 +01:00
gtully
57c7939534 AMQ-7067 - tidy up tests and add prepare variant, limit rollback location recording to xa case. There is still some work to do for the ack compaction case to make it aware of the tx records such that those are transferred as necessary 2018-10-09 12:01:47 +01:00
jgoodyear
99db9ef301 KahaDB Recovery can experience a dangling transaction when prepare and commit occur on different pagefiles.
Signed-off-by: gtully <gary.tully@gmail.com>
2018-10-09 10:19:35 +01:00
Alan Protasio
8f88dcda09 AMQ-7055 - Optimization on SequenceSet to prevent iterating through the whole set when a value bigger than the last value is added
Signed-off-by: gtully <gary.tully@gmail.com>
2018-09-20 10:38:44 +01:00
Jeff Genender
28819aea4a AMQ-7015 - Changed attribute to purgeRecoveredXATransactionStrategy and
allow NEVER, COMMIT, and ROLLBACK
2018-07-25 12:57:17 -06:00
hkesler
24b9ae2ed3 AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery 2018-07-19 12:01:01 -06:00
gtully
83514ef799 AMQ-7001 - ensure cursor pending cached id list is pruned of futures that end in an exception, fix and test 2018-07-03 21:18:36 +01:00
gtully
4c972d97b9 AMQ-5875 - fix test regression in MultiKahaDBQueueDeletionTest 2018-05-31 10:20:53 +01:00
gtully
ceb97f6baa AMQ-6707 - mKahadb, track recovered tx per store for completion, resolve test regression 2018-04-27 13:48:51 +01:00
gtully
ea70e827c0 AMQ-6707 - JDBC XA recovery and completion.
ensure pending transactions are visible for recovery without restart
sync store and cursor size during and after completion
ensure pending messages are not visible to browsers
retain transaction state on jdbc error
a bunch of new tests around xa completion
2018-04-26 14:58:48 +01:00
gtully
ec6fa19099 [AMQ-6815] rework to drop the batch reference from Location such that batches are free for gc when index pages are agressively cached 2018-01-11 12:56:40 +00:00
gtully
6da08b245e [AMQ-6849] provide a virtualTopic dropOnResourceLimit attribute, default to false.
Enabling will allow dispatch to continue even when end to some consumer queues
fail due to resource constraints.
The existing systemUsage sendFailIfNoSpace and sendFailIfNoSpaceAfterTimeout are
promoted to destination policy options allowing selective destinations to fail a
send with an exception. The global system usage options are still in place.
When dropOnResourceLimit=true dispatch will continue in the event that any
subscription queue throws a resource limit exception on send failure.
2017-10-25 14:53:20 +01:00
gtully
e793260573 [AMQ-6831, AMQ-6771] trigger eofexception on read -1 - AMQ6522Test 2017-10-11 00:20:44 +01:00
gtully
f989992278 [AMQ-6831, AMQ-6771] fix up recovery check to ensure full batch is available in memory, regression from AMQ-6771 2017-10-10 17:24:09 +01:00
gtully
8c3ef6cadb [AMQ-6815] have checkpoint validate status of async writes to avoid stale metadata and validate location size on read to avoid potential oom on restart 2017-09-19 16:51:00 +01:00
gtully
a359d8152c [AMQ-6277] take account of producer audit not being updatated on recovery check, avoid unnecessary partial journal replay 2017-09-15 13:48:03 +01:00
gtully
42bf6e9061 [AMQ-4947] ensure index page file does not skip forcing metadata - size is important to the page file 2017-09-04 16:43:48 +01:00
gtully
8c218ee05d [AMQ-6771] do linear sequential scan of journal when validating checksums - remove batch reads via seek/read which depend on write batch size 2017-07-17 12:18:25 +01:00
Christopher L. Shannon (cshannon)
a9fe4e33c3 [maven-release-plugin] prepare for next development iteration 2017-06-27 13:41:28 -04:00
Christopher L. Shannon (cshannon)
5f0d6943cb [maven-release-plugin] prepare release activemq-5.15.0 2017-06-27 13:41:01 -04:00
gtully
b07821ab64 [AMQ-6625] remove kahadbioexceptionhandler by pushing allowIOResumption into persistence adapter. This allows the lease locker to still be used with kahadb for stopStartConnectors support 2017-06-15 17:28:31 +01:00
gtully
f4c11f73bb [AMQ-6688] catch throwable on add task such that result error result is not lost on failed async send, fix and test 2017-05-29 17:14:05 +01:00
gtully
eb1a3eb77b [AMQ-6670] - harden the test to avoid ci failure 2017-05-10 11:30:30 +01:00
gtully
053c38f99f [AMQ-6670] - disable expiry processing that may page in and change the test 2017-05-08 11:53:55 +01:00
gtully
2518bded67 [AMQ-6670] - trap classcast and marshall errors on load message from journal such that errors are consistently propagated to the ioExceptionHandler 2017-05-05 15:43:09 +01:00
gtully
af03ad4679 [AMQ-6652] add final modifier to test 2017-04-11 11:39:22 +01:00
gtully
ca5e41bb7a [AMQ-6652] ensure checkpoint pagefile flush includeds the checkpoint transaction 2017-04-10 17:15:07 +01:00