Commit Graph

279 Commits

Author SHA1 Message Date
gtully fb1e642c15 AMQ-7082 - ensure flush sees free list, use lazySet to avoid store barrier, thanks cshannon 2018-10-22 15:16:26 +01:00
gtully efa4e683bc AMQ-7082 - fix final ref in test 2018-10-19 16:45:04 +01:00
gtully 79c74998dc AMQ-7082 - recover index free pages in parallel with start, merge in flush, clean shutdown if complete. follow up on AMQ-6590 2018-10-19 16:00:23 +01:00
gtully ed727d22e1 AMQ-7067 - track xa commit outcomes in ack compaction such that there are no dangling prepared tx on full recovery, fix and test 2018-10-17 18:13:36 +01:00
gtully 5d3a3fcca7 AMQ-6590 - rework fix to take the recovery hit on clean shutdown rather than on restart, trade off availability for short term disk usage 2018-10-17 12:32:04 +01:00
gtully 7c890d4776 AMQ-7067 - test and fix for eager ack compaction moving acks from data files with in progress tx and clearing the link from the prepare record, that now uses the same ack map 2018-10-12 14:51:37 +01:00
gtully a311139bfe AMQ-7067 - ensure updates to ackMessageFileMap are protected by the index lock 2018-10-09 12:55:11 +01:00
gtully 57c7939534 AMQ-7067 - tidy up tests and add prepare variant, limit rollback location recording to xa case. There is still some work to do for the ack compaction case to make it aware of the tx records such that those are transferred as necessary 2018-10-09 12:01:47 +01:00
jgoodyear 99db9ef301 KahaDB Recovery can experience a dangling transaction when prepare and commit occur on different pagefiles.
Signed-off-by: gtully <gary.tully@gmail.com>
2018-10-09 10:19:35 +01:00
Alan Protasio 8f88dcda09 AMQ-7055 - Optimization on SequenceSet to prevent iterating through the whole set when a value bigger than the last value is added
Signed-off-by: gtully <gary.tully@gmail.com>
2018-09-20 10:38:44 +01:00
Jeff Genender 28819aea4a AMQ-7015 - Changed attribute to purgeRecoveredXATransactionStrategy and
allow NEVER, COMMIT, and ROLLBACK
2018-07-25 12:57:17 -06:00
hkesler 24b9ae2ed3 AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery 2018-07-19 12:01:01 -06:00
gtully 83514ef799 AMQ-7001 - ensure cursor pending cached id list is pruned of futures that end in an exception, fix and test 2018-07-03 21:18:36 +01:00
gtully 4c972d97b9 AMQ-5875 - fix test regression in MultiKahaDBQueueDeletionTest 2018-05-31 10:20:53 +01:00
gtully ceb97f6baa AMQ-6707 - mKahadb, track recovered tx per store for completion, resolve test regression 2018-04-27 13:48:51 +01:00
gtully ea70e827c0 AMQ-6707 - JDBC XA recovery and completion.
ensure pending transactions are visible for recovery without restart
sync store and cursor size during and after completion
ensure pending messages are not visible to browsers
retain transaction state on jdbc error
a bunch of new tests around xa completion
2018-04-26 14:58:48 +01:00
gtully ec6fa19099 [AMQ-6815] rework to drop the batch reference from Location such that batches are free for gc when index pages are agressively cached 2018-01-11 12:56:40 +00:00
gtully 6da08b245e [AMQ-6849] provide a virtualTopic dropOnResourceLimit attribute, default to false.
Enabling will allow dispatch to continue even when end to some consumer queues
fail due to resource constraints.
The existing systemUsage sendFailIfNoSpace and sendFailIfNoSpaceAfterTimeout are
promoted to destination policy options allowing selective destinations to fail a
send with an exception. The global system usage options are still in place.
When dropOnResourceLimit=true dispatch will continue in the event that any
subscription queue throws a resource limit exception on send failure.
2017-10-25 14:53:20 +01:00
gtully e793260573 [AMQ-6831, AMQ-6771] trigger eofexception on read -1 - AMQ6522Test 2017-10-11 00:20:44 +01:00
gtully f989992278 [AMQ-6831, AMQ-6771] fix up recovery check to ensure full batch is available in memory, regression from AMQ-6771 2017-10-10 17:24:09 +01:00
gtully 8c3ef6cadb [AMQ-6815] have checkpoint validate status of async writes to avoid stale metadata and validate location size on read to avoid potential oom on restart 2017-09-19 16:51:00 +01:00
gtully a359d8152c [AMQ-6277] take account of producer audit not being updatated on recovery check, avoid unnecessary partial journal replay 2017-09-15 13:48:03 +01:00
gtully 42bf6e9061 [AMQ-4947] ensure index page file does not skip forcing metadata - size is important to the page file 2017-09-04 16:43:48 +01:00
gtully 8c218ee05d [AMQ-6771] do linear sequential scan of journal when validating checksums - remove batch reads via seek/read which depend on write batch size 2017-07-17 12:18:25 +01:00
Christopher L. Shannon (cshannon) a9fe4e33c3 [maven-release-plugin] prepare for next development iteration 2017-06-27 13:41:28 -04:00
Christopher L. Shannon (cshannon) 5f0d6943cb [maven-release-plugin] prepare release activemq-5.15.0 2017-06-27 13:41:01 -04:00
gtully b07821ab64 [AMQ-6625] remove kahadbioexceptionhandler by pushing allowIOResumption into persistence adapter. This allows the lease locker to still be used with kahadb for stopStartConnectors support 2017-06-15 17:28:31 +01:00
gtully f4c11f73bb [AMQ-6688] catch throwable on add task such that result error result is not lost on failed async send, fix and test 2017-05-29 17:14:05 +01:00
gtully eb1a3eb77b [AMQ-6670] - harden the test to avoid ci failure 2017-05-10 11:30:30 +01:00
gtully 053c38f99f [AMQ-6670] - disable expiry processing that may page in and change the test 2017-05-08 11:53:55 +01:00
gtully 2518bded67 [AMQ-6670] - trap classcast and marshall errors on load message from journal such that errors are consistently propagated to the ioExceptionHandler 2017-05-05 15:43:09 +01:00
gtully af03ad4679 [AMQ-6652] add final modifier to test 2017-04-11 11:39:22 +01:00
gtully ca5e41bb7a [AMQ-6652] ensure checkpoint pagefile flush includeds the checkpoint transaction 2017-04-10 17:15:07 +01:00
Hiram Chirino 172c29091e AMQ-6651: Add new implementations of the writeUTF8 and readUTF8 methods that are based on Apache Harmony code. This also avoid some code duplication that was occurring. 2017-04-10 10:49:17 -04:00
gtully c1cbf509da [AMQ-6644] - ensure journal files past last update are not included in the logged gc set in error 2017-04-04 12:21:31 +01:00
Christopher L. Shannon (cshannon) 2731f04f1c AMQ-6642: Fix potential NPE on updateMessage
Fixing potential NPE when calling updateMessage on a Queue store in
KahaDB if subscription statistics are enabled.  Also reduced the
visibily from public to protected to subscription statistic related
methods that shouldn't be public and added null pointer checks there as
well.

https://issues.apache.org/jira/browse/AMQ-6642
2017-03-31 09:06:01 -04:00
gtully c5a8b2c8b1 [AMQ-6625] ensure kahadb stops operation on the first IOException to facilitate auto recovery from partial writes 2017-03-13 13:49:41 +00:00
gtully 21ae1ef2e6 [AMQ-6606] - reset next batch so it does not get reused after error and refine fix to sync write batches b/c async locations will already be in the index 2017-03-07 12:28:58 +00:00
Christopher L. Shannon (cshannon) 0d824a8e68 AMQ-6377: Storing JournalSyncStrategy as an enum
Switching MessageDatabase to store JournalSyncStrategy as its enum value
instead of String to avoid the performance penalty of comparing Strings
everytime isEnableJournalDiskSyncs() is called which currently happens
on every write.
2017-03-06 07:54:39 -05:00
gtully d53b8f8d42 [AMQ-6606] avoid partial writes to the end of the journal - revert offset increment on ioexception, fix and test 2017-02-21 17:04:00 +00:00
gtully 9b64e188b5 [AMQ-6432] issue was journal scan on newly created ack file. I left the relevant braces from AMQ-6288 in place. Fix and test 2017-02-03 16:58:52 +00:00
gtully e3b529129a [AMQ-6432] improve the logging to see the current offset 2017-02-02 12:14:17 +00:00
Christopher L. Shannon (cshannon) 83511c96e5 https://issues.apache.org/jira/browse/AMQ-6590
Removing un-needed imports
2017-02-02 07:09:57 -05:00
Christopher L. Shannon (cshannon) 38d85be476 https://issues.apache.org/jira/browse/AMQ-6590
Fix KahaDB index free page recovery on unclean shutdown so that existing
free pages will be tracked and not lost.
2017-02-02 07:06:11 -05:00
Christopher L. Shannon (cshannon) 0ad62f722f https://issues.apache.org/jira/browse/AMQ-6572
On journal start use lastFileLength instead of maxFileLength when
checking for unused space in the last journal file as it is possible the
length of the last journal file is not the same length as maxFileLength
2017-01-23 11:01:20 -05:00
gtully 20522394cc [AMQ-6567] ensure gc file removal/move is completed after index updates to avoid dangling index referenced on partial failure. Fix and test 2017-01-17 14:05:04 +00:00
Timothy Bish 281d600ae2 https://issues.apache.org/jira/browse/AMQ-6563
ensure that the lock is always released in the load method.
2017-01-13 11:48:19 -05:00
gtully 0ae90c0897 https://issues.apache.org/jira/browse/AMQ-6547 - add storeUsage attribute to mKahadb filtered adapter such that disk utilisation is available as a QOS 2016-12-20 14:47:22 +00:00
Christopher L. Shannon (cshannon) 5fd63a0e4e https://issues.apache.org/jira/browse/AMQ-6545
Fixing the order of locks acquired in the KahaDB journal to prevent a
deadlock during file rotation
2016-12-16 13:44:18 -05:00
Christopher L. Shannon (cshannon) 0f0bdb21ef https://issues.apache.org/jira/browse/AMQ-6524
Fixing a thread safety issue with memoryUsage when using concurrentStoreAndDispatch
that was causing memory usage to get out of sync.

The InnerFutureTask class inside KahaDB was not thread safe which was
the root cause of the problem.
2016-12-05 07:10:20 -05:00