Commit Graph

392 Commits

Author SHA1 Message Date
Christopher L. Shannon (cshannon) c3714457f1 AMQ-7136 - Improve recovery of durable subscription metrics in KahaDB
Updated metrics recovery to only have to iterate over the order index 1
time to recovery the pending metrics for the subscriptions instead of
making a pass over the index once per subscription
2019-01-15 14:22:26 -05:00
Alan Protasio 4b51f8b66c AMQ-7132 - ActiveMQ reads lots of index pages upon startup (after a graceful or ungraceful shutdown) 2019-01-14 10:54:26 -05:00
Christopher L. Shannon (cshannon) a16b227808 AMQ-7129 - code cleanup 2019-01-11 10:31:44 -05:00
Christopher L. Shannon (cshannon) fa2daa25e9 AMQ-7129 - fix durable message size statistics with individual ack
Make sure that the pending message size for a durable sub only includes
messages part of the ack range
2019-01-11 09:56:57 -05:00
Christopher L. Shannon (cshannon) 25de20c77e AMQ-7129 - Properly recover messages from KahaDB for a durable when there are
messages to recover before the stored lastAck value

With individual ack mode we need to check the durable ackPosition
sequence set in the KahaDB index on subsription load to see if there are
earlier messages before the lastAck value that still haven't been acked.
While this normally wouldn't happen it is possible in individual ack
mode
2019-01-09 14:34:06 -05:00
jgoodyear 4a21edc8d5 [AMQ-7126] Improvement to perf of 5266Test 2019-01-07 21:22:34 -03:30
jgoodyear b86c666c40 [AMQ-7126] Prevent OOM when recovering KahaDB and memory space is insufficient to load full page. Thank you Heath Kesler and Johan Edstrom for investigation and patch. 2019-01-03 15:46:39 -03:30
jgoodyear 612d4aeeb4 AMQ-7118 This closes #327 - with thanks to Heath Kesler 2018-12-04 12:54:30 -03:30
Alan Protasio 9012a7871b AMQ-7091 - O(n) Memory consumption when broker has inactive durable subscribes causing OOM 2018-11-12 17:48:53 -03:30
gtully 85859fd8dc AMQ-7082 We should make sure that pages managed during recovery are not recovered in error
variation of patch from Alan Protasio <alanprot@gmail.com> closes #317
2018-11-07 11:29:14 +00:00
Alan Protasio 0d34338919 AMQ-7082 - Make sure that the recovery will only mark pages as free if they were created in a previous execution 2018-11-06 04:48:22 -08:00
gtully bf8eb08aca AMQ-7086 - make kahadb gc/cleanup on shutdown optional to trade availability over disk usage for fast failover 2018-10-31 14:29:05 +00:00
gtully 8a1abd9bb2 AMQ-7084 - ensure allocated and unused free pages are visible to subsequent transactions, fix and test with test updates to reflect proper usage 2018-10-23 16:00:57 +01:00
gtully fb1e642c15 AMQ-7082 - ensure flush sees free list, use lazySet to avoid store barrier, thanks cshannon 2018-10-22 15:16:26 +01:00
gtully efa4e683bc AMQ-7082 - fix final ref in test 2018-10-19 16:45:04 +01:00
gtully 79c74998dc AMQ-7082 - recover index free pages in parallel with start, merge in flush, clean shutdown if complete. follow up on AMQ-6590 2018-10-19 16:00:23 +01:00
gtully ed727d22e1 AMQ-7067 - track xa commit outcomes in ack compaction such that there are no dangling prepared tx on full recovery, fix and test 2018-10-17 18:13:36 +01:00
gtully 5d3a3fcca7 AMQ-6590 - rework fix to take the recovery hit on clean shutdown rather than on restart, trade off availability for short term disk usage 2018-10-17 12:32:04 +01:00
gtully 7c890d4776 AMQ-7067 - test and fix for eager ack compaction moving acks from data files with in progress tx and clearing the link from the prepare record, that now uses the same ack map 2018-10-12 14:51:37 +01:00
gtully a311139bfe AMQ-7067 - ensure updates to ackMessageFileMap are protected by the index lock 2018-10-09 12:55:11 +01:00
gtully 57c7939534 AMQ-7067 - tidy up tests and add prepare variant, limit rollback location recording to xa case. There is still some work to do for the ack compaction case to make it aware of the tx records such that those are transferred as necessary 2018-10-09 12:01:47 +01:00
jgoodyear 99db9ef301 KahaDB Recovery can experience a dangling transaction when prepare and commit occur on different pagefiles.
Signed-off-by: gtully <gary.tully@gmail.com>
2018-10-09 10:19:35 +01:00
Alan Protasio 8f88dcda09 AMQ-7055 - Optimization on SequenceSet to prevent iterating through the whole set when a value bigger than the last value is added
Signed-off-by: gtully <gary.tully@gmail.com>
2018-09-20 10:38:44 +01:00
Jeff Genender 28819aea4a AMQ-7015 - Changed attribute to purgeRecoveredXATransactionStrategy and
allow NEVER, COMMIT, and ROLLBACK
2018-07-25 12:57:17 -06:00
hkesler 24b9ae2ed3 AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery 2018-07-19 12:01:01 -06:00
gtully 83514ef799 AMQ-7001 - ensure cursor pending cached id list is pruned of futures that end in an exception, fix and test 2018-07-03 21:18:36 +01:00
gtully 4c972d97b9 AMQ-5875 - fix test regression in MultiKahaDBQueueDeletionTest 2018-05-31 10:20:53 +01:00
gtully ceb97f6baa AMQ-6707 - mKahadb, track recovered tx per store for completion, resolve test regression 2018-04-27 13:48:51 +01:00
gtully ea70e827c0 AMQ-6707 - JDBC XA recovery and completion.
ensure pending transactions are visible for recovery without restart
sync store and cursor size during and after completion
ensure pending messages are not visible to browsers
retain transaction state on jdbc error
a bunch of new tests around xa completion
2018-04-26 14:58:48 +01:00
gtully ec6fa19099 [AMQ-6815] rework to drop the batch reference from Location such that batches are free for gc when index pages are agressively cached 2018-01-11 12:56:40 +00:00
gtully 6da08b245e [AMQ-6849] provide a virtualTopic dropOnResourceLimit attribute, default to false.
Enabling will allow dispatch to continue even when end to some consumer queues
fail due to resource constraints.
The existing systemUsage sendFailIfNoSpace and sendFailIfNoSpaceAfterTimeout are
promoted to destination policy options allowing selective destinations to fail a
send with an exception. The global system usage options are still in place.
When dropOnResourceLimit=true dispatch will continue in the event that any
subscription queue throws a resource limit exception on send failure.
2017-10-25 14:53:20 +01:00
gtully e793260573 [AMQ-6831, AMQ-6771] trigger eofexception on read -1 - AMQ6522Test 2017-10-11 00:20:44 +01:00
gtully f989992278 [AMQ-6831, AMQ-6771] fix up recovery check to ensure full batch is available in memory, regression from AMQ-6771 2017-10-10 17:24:09 +01:00
gtully 8c3ef6cadb [AMQ-6815] have checkpoint validate status of async writes to avoid stale metadata and validate location size on read to avoid potential oom on restart 2017-09-19 16:51:00 +01:00
gtully a359d8152c [AMQ-6277] take account of producer audit not being updatated on recovery check, avoid unnecessary partial journal replay 2017-09-15 13:48:03 +01:00
gtully 42bf6e9061 [AMQ-4947] ensure index page file does not skip forcing metadata - size is important to the page file 2017-09-04 16:43:48 +01:00
gtully 8c218ee05d [AMQ-6771] do linear sequential scan of journal when validating checksums - remove batch reads via seek/read which depend on write batch size 2017-07-17 12:18:25 +01:00
Christopher L. Shannon (cshannon) a9fe4e33c3 [maven-release-plugin] prepare for next development iteration 2017-06-27 13:41:28 -04:00
Christopher L. Shannon (cshannon) 5f0d6943cb [maven-release-plugin] prepare release activemq-5.15.0 2017-06-27 13:41:01 -04:00
gtully b07821ab64 [AMQ-6625] remove kahadbioexceptionhandler by pushing allowIOResumption into persistence adapter. This allows the lease locker to still be used with kahadb for stopStartConnectors support 2017-06-15 17:28:31 +01:00
gtully f4c11f73bb [AMQ-6688] catch throwable on add task such that result error result is not lost on failed async send, fix and test 2017-05-29 17:14:05 +01:00
gtully eb1a3eb77b [AMQ-6670] - harden the test to avoid ci failure 2017-05-10 11:30:30 +01:00
gtully 053c38f99f [AMQ-6670] - disable expiry processing that may page in and change the test 2017-05-08 11:53:55 +01:00
gtully 2518bded67 [AMQ-6670] - trap classcast and marshall errors on load message from journal such that errors are consistently propagated to the ioExceptionHandler 2017-05-05 15:43:09 +01:00
gtully af03ad4679 [AMQ-6652] add final modifier to test 2017-04-11 11:39:22 +01:00
gtully ca5e41bb7a [AMQ-6652] ensure checkpoint pagefile flush includeds the checkpoint transaction 2017-04-10 17:15:07 +01:00
Hiram Chirino 172c29091e AMQ-6651: Add new implementations of the writeUTF8 and readUTF8 methods that are based on Apache Harmony code. This also avoid some code duplication that was occurring. 2017-04-10 10:49:17 -04:00
gtully c1cbf509da [AMQ-6644] - ensure journal files past last update are not included in the logged gc set in error 2017-04-04 12:21:31 +01:00
Christopher L. Shannon (cshannon) 2731f04f1c AMQ-6642: Fix potential NPE on updateMessage
Fixing potential NPE when calling updateMessage on a Queue store in
KahaDB if subscription statistics are enabled.  Also reduced the
visibily from public to protected to subscription statistic related
methods that shouldn't be public and added null pointer checks there as
well.

https://issues.apache.org/jira/browse/AMQ-6642
2017-03-31 09:06:01 -04:00
gtully c5a8b2c8b1 [AMQ-6625] ensure kahadb stops operation on the first IOException to facilitate auto recovery from partial writes 2017-03-13 13:49:41 +00:00
gtully 21ae1ef2e6 [AMQ-6606] - reset next batch so it does not get reused after error and refine fix to sync write batches b/c async locations will already be in the index 2017-03-07 12:28:58 +00:00
Christopher L. Shannon (cshannon) 0d824a8e68 AMQ-6377: Storing JournalSyncStrategy as an enum
Switching MessageDatabase to store JournalSyncStrategy as its enum value
instead of String to avoid the performance penalty of comparing Strings
everytime isEnableJournalDiskSyncs() is called which currently happens
on every write.
2017-03-06 07:54:39 -05:00
gtully d53b8f8d42 [AMQ-6606] avoid partial writes to the end of the journal - revert offset increment on ioexception, fix and test 2017-02-21 17:04:00 +00:00
gtully 9b64e188b5 [AMQ-6432] issue was journal scan on newly created ack file. I left the relevant braces from AMQ-6288 in place. Fix and test 2017-02-03 16:58:52 +00:00
gtully e3b529129a [AMQ-6432] improve the logging to see the current offset 2017-02-02 12:14:17 +00:00
Christopher L. Shannon (cshannon) 83511c96e5 https://issues.apache.org/jira/browse/AMQ-6590
Removing un-needed imports
2017-02-02 07:09:57 -05:00
Christopher L. Shannon (cshannon) 38d85be476 https://issues.apache.org/jira/browse/AMQ-6590
Fix KahaDB index free page recovery on unclean shutdown so that existing
free pages will be tracked and not lost.
2017-02-02 07:06:11 -05:00
Christopher L. Shannon (cshannon) 0ad62f722f https://issues.apache.org/jira/browse/AMQ-6572
On journal start use lastFileLength instead of maxFileLength when
checking for unused space in the last journal file as it is possible the
length of the last journal file is not the same length as maxFileLength
2017-01-23 11:01:20 -05:00
gtully 20522394cc [AMQ-6567] ensure gc file removal/move is completed after index updates to avoid dangling index referenced on partial failure. Fix and test 2017-01-17 14:05:04 +00:00
Timothy Bish 281d600ae2 https://issues.apache.org/jira/browse/AMQ-6563
ensure that the lock is always released in the load method.
2017-01-13 11:48:19 -05:00
gtully 0ae90c0897 https://issues.apache.org/jira/browse/AMQ-6547 - add storeUsage attribute to mKahadb filtered adapter such that disk utilisation is available as a QOS 2016-12-20 14:47:22 +00:00
Christopher L. Shannon (cshannon) 5fd63a0e4e https://issues.apache.org/jira/browse/AMQ-6545
Fixing the order of locks acquired in the KahaDB journal to prevent a
deadlock during file rotation
2016-12-16 13:44:18 -05:00
Christopher L. Shannon (cshannon) 0f0bdb21ef https://issues.apache.org/jira/browse/AMQ-6524
Fixing a thread safety issue with memoryUsage when using concurrentStoreAndDispatch
that was causing memory usage to get out of sync.

The InnerFutureTask class inside KahaDB was not thread safe which was
the root cause of the problem.
2016-12-05 07:10:20 -05:00
gtully dad629e889 [AMQ-6522] - remove hardcoded 32k batch limit from recovery check of the journal, fix and test 2016-11-29 11:32:03 +00:00
gtully 0a29533ed7 [AMQ-6520] respect kahadb indexDirectory for perDestination mKahaDB - fix and test 2016-11-25 16:44:28 +00:00
Timothy Bish 2c828b23df https://issues.apache.org/jira/browse/AMQ-6504
Fix intermittent test failure
2016-11-21 12:37:52 -05:00
Timothy Bish 980162233f https://issues.apache.org/jira/browse/AMQ-6504
Round the start time value not truncate to ensure delay falls on the
correct side of the scheduling block.
2016-11-11 14:48:49 -05:00
gtully 3b7613d930 AMQ-6451 - catch errors on preallocation. allocate direct buffer on start and reuse to ensure resource availability at runtime 2016-11-08 14:05:46 +00:00
Hadrian Zbarcea e725451521 This closes #162. Minor improvement in using the Logger api 2016-11-03 17:26:00 -04:00
Hadrian Zbarcea be24a133ef This closes #88 (merged as a65ac586). More file permissions fixes. 2016-11-03 14:26:34 -04:00
Hadrian Zbarcea d5a929c0d2 This closes #205. Won't fix. (fix even more file permissions) 2016-11-03 14:11:40 -04:00
Hadrian Zbarcea d54d046b8a This closes #185. Won't fix. (fix more file permissions) 2016-11-03 11:54:33 -04:00
Timothy Bish 1a91decf9f https://issues.apache.org/jira/browse/AMQ-6483
Fix conversion from long to int and back to long again.
2016-10-26 13:27:24 -04:00
gtully 538ed74510 NO-JIRA - ensure stack trace in incuded with exception that terminates data file appender thread 2016-10-07 16:26:07 +01:00
Christopher L. Shannon (cshannon) 351faf2699 https://issues.apache.org/jira/browse/AMQ-6445
Fixing RandomAccessFile usage so that the file will be properly closed
even if there is an exception
2016-09-29 06:16:40 -04:00
Christopher L. Shannon (cshannon) a82c95cd29 https://issues.apache.org/jira/browse/AMQ-6436
The temporary store will now delete the old temp directory on start up
if lazyInit is true instead of waiting for the store to initialize to
clear up space.  This prevents space on the disk from being wasted with
old data if the temp store isn't initialized
2016-09-23 15:47:27 -04:00
Christopher L. Shannon (cshannon) 18571ce09b https://issues.apache.org/jira/browse/AMQ-6430
Modifying patch so that only stores that persist the noLocal flag will
check if this flag has changed to prevent a subscription from being
deleted by mistake
2016-09-23 10:59:32 -04:00
gtully f8bc19b96d AMQ-6413 - ensure audit update on skipped store add for kahadb concurrentStoreAndDispatch. Fix and test 2016-09-01 16:46:21 +01:00
gtully 5a874816b7 AMQ-6403 - add indexDirectory attribute to kahadb plist impl - settable via broker service tempDataStore 2016-08-24 11:39:22 +01:00
gtully b37b9eb5af fix up cancelledTaskModMetric reporting in kahadb 2016-08-19 11:48:42 +01:00
gtully 330f0ce734 AMQ-5815 ensure async write latch is released on exception 2016-08-12 16:44:23 +01:00
gtully 51166dc7af AMQ-6377 - fix deprecation java doc ref 2016-08-10 12:05:21 +01:00
gtully fcd9047b9f additional test to validate stats on missing messages and outstanding acks on recovery 2016-08-10 11:54:12 +01:00
Christopher L. Shannon (cshannon) 06a21dbbb3 [maven-release-plugin] prepare for next development iteration 2016-08-02 09:03:37 -04:00
Christopher L. Shannon (cshannon) 71cbc65283 [maven-release-plugin] prepare release activemq-5.14.0 2016-08-02 09:02:54 -04:00
Christopher L. Shannon (cshannon) 71bb54f94c https://issues.apache.org/jira/browse/AMQ-6372
Fixing test so that test DataFile is written to a temporary directory
instead of the main activemq-kahadb-store directory.  This prevents git
from picking up the test file
2016-07-30 12:59:10 -04:00
Christopher L. Shannon (cshannon) 1a598277cf https://issues.apache.org/jira/browse/AMQ-6377
Reworking patch so that a periodic disk sync uses a journal trace
command to trigger the sync so that everything is done in the same
thread for the writes
2016-07-28 10:07:10 -04:00
Christopher L. Shannon (cshannon) 498e2184be https://issues.apache.org/jira/browse/AMQ-6377
Adding new setters to KahaDBPersistenceAdapter
2016-07-27 17:33:45 -04:00
Christopher L. Shannon (cshannon) dd0ed17e59 https://issues.apache.org/jira/browse/AMQ-6377
Introducing JournalDiskSyncStrategy to allow a peridic disk sync mode
instead of always syncing after every write or never syncing.
2016-07-27 14:31:07 -04:00
gtully 822e2be90e AMQ-6378 force recovery on corrupt metadata location info, relates to AMQ-6376 2016-07-27 15:56:28 +01:00
gtully 6b8e743b08 AMQ-6372 - add IOExceptionHandler to kahadb read path to ensure fast shutdown on disk access errors 2016-07-27 11:58:22 +01:00
Christopher L. Shannon (cshannon) cf3d419528 https://issues.apache.org/jira/browse/AMQ-6375
Adding non-blocking metrics to KahaDB to track message counts and sizes
for subscriptions.  Turned off by default but can be enabled on the
KahaDBPersistenceAdapter
2016-07-26 12:02:44 -04:00
gtully d427952b19 AMQ-6372 - fix up accessor pool cleanupUnused and use the minimum number of open files for recovery 2016-07-25 13:33:24 +01:00
Timothy Bish 8cc5c56d5e https://issues.apache.org/jira/browse/AMQ-6358
Wait until recover is done before starting the checkpoint tasks.
2016-07-14 10:49:59 -04:00
Christopher L. Shannon (cshannon) a5050a8bc5 https://issues.apache.org/jira/browse/AMQ-6356
Fixing store size calculation on KahaUpdateMessageCommand processing so
that the size won't increase inadvertently if the existing location of
the command in the journal is the same as the new location
2016-07-14 09:21:23 -04:00
Christopher L. Shannon (cshannon) d635a36940 https://issues.apache.org/jira/browse/AMQ-6352
Fixing PList size tracking by moving the null assignment on removal to
after the onRemoval method is called to make sure that the decrement is
done properly.
2016-07-12 10:27:57 -04:00
gtully 4386750fdb for mkahadb populate mdc activemq.persistenceDir such that logging for a particular instance can be identified and correlated 2016-07-04 11:19:56 +01:00
Christopher L. Shannon (cshannon) 4d6cc4b460 https://issues.apache.org/jira/browse/AMQ-6303
Properly setting typeCode value for new journal files used for ack
compaction
2016-05-23 18:42:29 +00:00
Timothy Bish 15405af2e6 https://issues.apache.org/jira/browse/AMQ-6292
allow for batch writes of forwarded acks.
2016-05-13 14:56:01 -04:00
Christopher L. Shannon (cshannon) c8a6171d04 https://issues.apache.org/jira/browse/AMQ-6288
Switching the checkpoint lock to a readlock when forwarding acks to
prevent other journal updates from being blocked.
2016-05-12 14:39:12 +00:00
Christopher L. Shannon (cshannon) e53e340262 https://issues.apache.org/jira/browse/AMQ-6288
The ack compaction task now acquires the checkpoint lock while it runs
to prevent a checkpoint from running at the same time unintentionally.
Also, getJournalLocation is now protected by a try/catch to handle
errors.
2016-05-11 13:16:57 +00:00
Christopher L. Shannon (cshannon) 91213010b3 https://issues.apache.org/jira/browse/AMQ-6285
Added an extra isShutdown() failsafe check when detecting if the
checkpoint thread should start
2016-05-09 19:35:02 +00:00
Christopher L. Shannon (cshannon) 7bdcca1bda https://issues.apache.org/jira/browse/AMQ-6285
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the
thread will properly restart.
2016-05-09 19:06:40 +00:00
Christopher L. Shannon (cshannon) db3f8b3554 Revert "https://issues.apache.org/jira/browse/AMQ-6285"
This reverts commit 60b0c4f85a.

Inadvertantly commited a bunch of changes by mistake
2016-05-09 19:05:52 +00:00
Christopher L. Shannon (cshannon) 60b0c4f85a https://issues.apache.org/jira/browse/AMQ-6285
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the thread will
properly restart.
2016-05-09 19:03:38 +00:00
gtully 65cef69130 https://issues.apache.org/jira/browse/AMQ-5603 - reverting default preallocatonScope to entire_journal b/c async only really works for ssd 2016-05-05 13:06:42 +01:00
gtully 1c4108545c https://issues.apache.org/jira/browse/AMQ-6277 - journal getNextLocation needs too passes to skip past if target is not initialized 2016-05-04 22:09:06 +01:00
gtully 555cd2b6bb https://issues.apache.org/jira/browse/AMQ-5603 - catch interrupted on chunked zeros case 2016-05-03 17:25:14 +01:00
gtully ba77b9f55a https://issues.apache.org/jira/browse/AMQ-6277 - tidy up logic that determines recovery location so that we don't recovery from the end of the journal in error on normal restart. This avoids suprious recovery logging 2016-05-03 12:47:49 +01:00
gtully 62bdbb0db5 https://issues.apache.org/jira/browse/AMQ-5603 - add preallocationScope=full_journal_async that will preallocate a journal in advance or use to avoid latency jitter on journal rotation. Added none option to disable preallocation 2016-04-29 16:57:28 +01:00
gtully 43d8205cf0 reduce the throughput for ci 2016-04-22 14:53:14 +01:00
gtully 930a74c696 https://issues.apache.org/jira/browse/AMQ-5578 - enable org.apache.activemq.kahaDB.files.skipMetadataUpdate for tests to validate. remove some redundant call to limit and add latencey test to validate the need for outof band or pool allocation of journals 2016-04-22 12:29:31 +01:00
Christopher L. Shannon (cshannon) 11622b3af3 https://issues.apache.org/jira/browse/AMQ-6256
Moving beforeMarshall back to the store implementations because we don't
want all store implementations to marshall (such as memory store)

This reverts commit b9b98a45ce.
2016-04-18 12:34:25 +00:00
Christopher L. Shannon (cshannon) 1ffac14f6e https://issues.apache.org/jira/browse/AMQ-6256
cleanup imports
2016-04-15 14:23:54 +00:00
Christopher L. Shannon (cshannon) b9b98a45ce https://issues.apache.org/jira/browse/AMQ-6256
Moving beforeMarshall call out of the store and into the actual
destination
2016-04-15 14:17:00 +00:00
Christopher L. Shannon (cshannon) b9f9f03829 https://issues.apache.org/jira/browse/AMQ-6256
Calling beforeMarshall on messages when they async stored before the
store task is run and before consumer dispatch to prevent two threads
from trying to mutate the message state at the same time.
2016-04-15 13:01:21 +00:00
Christopher L. Shannon (cshannon) 092c56d70c https://issues.apache.org/jira/browse/AMQ-6250
Checking for null before closing the journal in
MultiKahaDBPersistentAdapter.  Also protecting against multiple calls to
start and stop.
2016-04-14 18:39:14 +00:00
Christopher L. Shannon (cshannon) cbad8babe5 https://issues.apache.org/jira/browse/AMQ-6203
Adding a flag to enable or disable Ack Compaction
Currently defaults to enabled for 5.14.0 and above
2016-04-12 13:30:10 +00:00
Timothy Bish a9521dcebf https://issues.apache.org/jira/browse/AMQ-6203
Expose configuration on the persistence adapter level.
Double the test timeout and increase the compaction frequency to account
for very slow CI boxes.
2016-03-15 12:02:40 -04:00
Timothy Bish 946e62d702 https://issues.apache.org/jira/browse/AMQ-6203
Rewrite older acks that can be preventing GC of log files.
2016-03-14 11:04:57 -04:00
Timothy Bish b39ab7867c https://issues.apache.org/jira/browse/AMQ-6207
Perform writes / sync in predicatable order first to the recovery file
and then into the main index file.
2016-03-11 09:59:07 -05:00
gtully 13ec994939 Revert "https://issues.apache.org/jira/browse/AMQ-4495 - revisit. Reinstate check for space on pagein, so that highWaterMark is respected and full state is not reached, hense pfc is not triggered in error"
This reverts commit d8cf54b0a9.
2016-03-07 16:26:25 +00:00
Timothy Bish 5e0f49332b https://issues.apache.org/jira/browse/AMQ-5578
Remove unnecessary buffer initialization after confirmation that it is
not needed.
2016-03-03 17:35:21 -05:00
Timothy Bish edfc23ee9d https://issues.apache.org/jira/browse/AMQ-5578
Add additional preallocation strategy [CHUNKED contributed by user
jtahlborn.
2016-03-03 17:34:07 -05:00
Timothy Bish be4ad3d1c8 https://issues.apache.org/jira/browse/AMQ-5578
Replace system.out with a trace log.
2016-03-03 17:00:11 -05:00
Timothy Bish e8f44a29dd https://issues.apache.org/jira/browse/AMQ-5578
Revert to previous behavior of explicit init.
2016-03-02 17:18:30 -05:00
Timothy Bish e89b7a57f1 https://issues.apache.org/jira/browse/AMQ-5578
Remove unnecessary zero init block as the java spec requires the
allocations to do this already.  Fix some warnings.
2016-03-02 16:17:43 -05:00
gtully d8cf54b0a9 https://issues.apache.org/jira/browse/AMQ-4495 - revisit. Reinstate check for space on pagein, so that highWaterMark is respected and full state is not reached, hense pfc is not triggered in error 2016-03-01 11:44:41 +00:00
gtully 499e39e52c https://issues.apache.org/jira/browse/AMQ-6164 - allow journal write batching on a single destination 2016-02-09 12:50:27 +00:00
Christopher L. Shannon (cshannon) 25ff5699f1 https://issues.apache.org/jira/browse/AMQ-6158
Computing messageSize for a durable subscription in KahaDB now runs much
faster (n vs n^2) which is noticable when there are a large number of
pending messages for a durable subscription.
2016-02-04 15:17:49 +00:00
Timothy Bish 8c4b5f485d https://issues.apache.org/jira/browse/AMQ-6152
Ensure that when add / remove commands are colocated they don't prevent
the log from being GC'd once it is unreferenced.
2016-02-02 20:28:24 -05:00
Timothy Bish 4ecff14030 NO-JIRA Update master version to 5.14.0-SNAPSHOT to match the eventual
release version.
2016-01-20 12:40:20 -05:00
Christopher L. Shannon (cshannon) b4aa53d806 https://issues.apache.org/jira/browse/AMQ-6133
Fixing the processing of KahaUpdateMessageCommand to add the message to
the store if it doesn't exist, instead of ignoring it.  This will
prevent message loss in certain cases when the KahaDB index has to be
rebuilt.
2016-01-20 16:59:07 +00:00
Timothy Bish 193f6be687 https://issues.apache.org/jira/browse/AMQ-6131
Fix check for durable sub with no pending messages during checkpoint
cleanup.
2016-01-18 17:43:18 -05:00
Christopher L. Shannon (cshannon) 507d40a79f https://issues.apache.org/jira/browse/AMQ-6097
Fixing JournalCorruptionEofIndexRecoveryTest class

Because of the size being stored in the message order index, the test
testRecoveryAftercorruptionCheckSumExistingIndex was no longer passing
as that test was corrupting the size value in the journal, but not the
size value in the index.  The test now corrupts both values so that the
expect exception occurs again.
2015-12-21 16:46:47 +00:00
Christopher L. Shannon (cshannon) 246ccb8e04 https://issues.apache.org/jira/browse/AMQ-6097
Fixing KahaDB so that the correct marshaller is used for the message
keys inside of the message order index.  This will ensure that message
size metrics are accurate.
2015-12-18 19:26:24 +00:00
gtully 31d99b6aa9 https://issues.apache.org/jira/browse/AMQ-6083 - resolve some test regressions - AMQ2149Test - fail to start with missing data file. Ensure ack map is always saved on modification 2015-12-11 13:30:55 +00:00
gtully 768fa17085 https://issues.apache.org/jira/browse/AMQ-6005 - have plist whack it existing state on start. Start the temp store after the primary persistence adapter which does the locking 2015-12-10 14:24:03 +00:00
gtully 5db5f3e39a https://issues.apache.org/jira/browse/AMQ-6083 - ignoreMissingJournalfiles false - check for missing acks and corruption anywhere and error out so that corruption does not go unnoticed - fix and test 2015-12-09 12:30:29 +00:00
Christopher L. Shannon (cshannon) 7a7c70ad75 https://issues.apache.org/jira/browse/AMQ-6082
Propertly re-setting the storeOpenWireVersion from the BrokerService
on the KahaDB Metadata if a corrupted index is detected and the
Metadata has to be recreated.
2015-12-08 20:10:11 +00:00
Torsten Mielke 66c348b1b8 https://issues.apache.org/jira/browse/AMQ-6071 - Log info about corrupted journal records at WARN level 2015-12-04 11:29:55 +00:00
Christopher L. Shannon (cshannon) cfb4ca9f54 [maven-release-plugin] prepare for next development iteration 2015-11-30 13:48:58 +00:00
Christopher L. Shannon (cshannon) abfe038ddf [maven-release-plugin] prepare release activemq-5.13.0 2015-11-30 13:48:11 +00:00
Christopher L. Shannon (cshannon) 8f30866fdf https://issues.apache.org/jira/browse/AMQ-6063
Fixing potential NullPointerException during KahaDB index recovery.
2015-11-25 16:02:52 +00:00
Christopher L. Shannon (cshannon) e07809dec6 NO JIRA - Clear journalSize counter
The journalSize counter should be set to 0 after KahaDB is shut down.
Usually, a store is not restarted again after stopping, but if it ever
was the value would be wrong since it wasn't reset back to 0.
2015-11-24 17:53:34 +00:00
gtully fd0f71a4c8 https://issues.apache.org/jira/browse/AMQ-6033 - fix shadowing error, additional test 2015-11-02 15:08:22 +00:00
gtully dc06c8dc75 https://issues.apache.org/jira/browse/AMQ-5994 https://issues.apache.org/jira/browse/AMQ-4000 - proper fix for duplicate sub info from the store on recovery failure from AMQ2149Test. Additional test from Christopher L 2015-10-06 12:01:24 +01:00
gtully f6e1054a4b https://issues.apache.org/jira/browse/AMQ-5984 - fix sync on get for access ordered LinkedHashMap 2015-09-28 14:20:45 +01:00
gtully fcabcd282d https://issues.apache.org/jira/browse/AMQ-5960 - rework fix to reset the next sequence so that the next ack position and message reference gets cleared up in normal operation 2015-09-23 13:15:29 +01:00
Dejan Bosanac ee4672baaf https://issues.apache.org/jira/browse/AMQ-5972 - wildcard policy entries not applied in correct order 2015-09-17 11:08:21 +02:00