4b51f8b66c
AMQ-7132 - ActiveMQ reads lots of index pages upon startup (after a graceful or ungraceful shutdown)
2019-01-14 10:54:26 -05:00
a16b227808
AMQ-7129 - code cleanup
2019-01-11 10:31:44 -05:00
fa2daa25e9
AMQ-7129 - fix durable message size statistics with individual ack
...
Make sure that the pending message size for a durable sub only includes
messages part of the ack range
2019-01-11 09:56:57 -05:00
25de20c77e
AMQ-7129 - Properly recover messages from KahaDB for a durable when there are
...
messages to recover before the stored lastAck value
With individual ack mode we need to check the durable ackPosition
sequence set in the KahaDB index on subsription load to see if there are
earlier messages before the lastAck value that still haven't been acked.
While this normally wouldn't happen it is possible in individual ack
mode
2019-01-09 14:34:06 -05:00
4a21edc8d5
[AMQ-7126] Improvement to perf of 5266Test
2019-01-07 21:22:34 -03:30
b86c666c40
[AMQ-7126] Prevent OOM when recovering KahaDB and memory space is insufficient to load full page. Thank you Heath Kesler and Johan Edstrom for investigation and patch.
2019-01-03 15:46:39 -03:30
612d4aeeb4
AMQ-7118 This closes #327 - with thanks to Heath Kesler
2018-12-04 12:54:30 -03:30
9012a7871b
AMQ-7091 - O(n) Memory consumption when broker has inactive durable subscribes causing OOM
2018-11-12 17:48:53 -03:30
85859fd8dc
AMQ-7082 We should make sure that pages managed during recovery are not recovered in error
...
variation of patch from Alan Protasio <alanprot@gmail.com> closes #317
2018-11-07 11:29:14 +00:00
0d34338919
AMQ-7082 - Make sure that the recovery will only mark pages as free if they were created in a previous execution
2018-11-06 04:48:22 -08:00
bf8eb08aca
AMQ-7086 - make kahadb gc/cleanup on shutdown optional to trade availability over disk usage for fast failover
2018-10-31 14:29:05 +00:00
8a1abd9bb2
AMQ-7084 - ensure allocated and unused free pages are visible to subsequent transactions, fix and test with test updates to reflect proper usage
2018-10-23 16:00:57 +01:00
fb1e642c15
AMQ-7082 - ensure flush sees free list, use lazySet to avoid store barrier, thanks cshannon
2018-10-22 15:16:26 +01:00
efa4e683bc
AMQ-7082 - fix final ref in test
2018-10-19 16:45:04 +01:00
79c74998dc
AMQ-7082 - recover index free pages in parallel with start, merge in flush, clean shutdown if complete. follow up on AMQ-6590
2018-10-19 16:00:23 +01:00
ed727d22e1
AMQ-7067 - track xa commit outcomes in ack compaction such that there are no dangling prepared tx on full recovery, fix and test
2018-10-17 18:13:36 +01:00
5d3a3fcca7
AMQ-6590 - rework fix to take the recovery hit on clean shutdown rather than on restart, trade off availability for short term disk usage
2018-10-17 12:32:04 +01:00
7c890d4776
AMQ-7067 - test and fix for eager ack compaction moving acks from data files with in progress tx and clearing the link from the prepare record, that now uses the same ack map
2018-10-12 14:51:37 +01:00
a311139bfe
AMQ-7067 - ensure updates to ackMessageFileMap are protected by the index lock
2018-10-09 12:55:11 +01:00
57c7939534
AMQ-7067 - tidy up tests and add prepare variant, limit rollback location recording to xa case. There is still some work to do for the ack compaction case to make it aware of the tx records such that those are transferred as necessary
2018-10-09 12:01:47 +01:00
99db9ef301
KahaDB Recovery can experience a dangling transaction when prepare and commit occur on different pagefiles.
...
Signed-off-by: gtully <gary.tully@gmail.com>
2018-10-09 10:19:35 +01:00
8f88dcda09
AMQ-7055 - Optimization on SequenceSet to prevent iterating through the whole set when a value bigger than the last value is added
...
Signed-off-by: gtully <gary.tully@gmail.com>
2018-09-20 10:38:44 +01:00
28819aea4a
AMQ-7015 - Changed attribute to purgeRecoveredXATransactionStrategy and
...
allow NEVER, COMMIT, and ROLLBACK
2018-07-25 12:57:17 -06:00
24b9ae2ed3
AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery
2018-07-19 12:01:01 -06:00
83514ef799
AMQ-7001 - ensure cursor pending cached id list is pruned of futures that end in an exception, fix and test
2018-07-03 21:18:36 +01:00
4c972d97b9
AMQ-5875 - fix test regression in MultiKahaDBQueueDeletionTest
2018-05-31 10:20:53 +01:00
ceb97f6baa
AMQ-6707 - mKahadb, track recovered tx per store for completion, resolve test regression
2018-04-27 13:48:51 +01:00
ea70e827c0
AMQ-6707 - JDBC XA recovery and completion.
...
ensure pending transactions are visible for recovery without restart
sync store and cursor size during and after completion
ensure pending messages are not visible to browsers
retain transaction state on jdbc error
a bunch of new tests around xa completion
2018-04-26 14:58:48 +01:00
ec6fa19099
[AMQ-6815] rework to drop the batch reference from Location such that batches are free for gc when index pages are agressively cached
2018-01-11 12:56:40 +00:00
6da08b245e
[AMQ-6849] provide a virtualTopic dropOnResourceLimit attribute, default to false.
...
Enabling will allow dispatch to continue even when end to some consumer queues
fail due to resource constraints.
The existing systemUsage sendFailIfNoSpace and sendFailIfNoSpaceAfterTimeout are
promoted to destination policy options allowing selective destinations to fail a
send with an exception. The global system usage options are still in place.
When dropOnResourceLimit=true dispatch will continue in the event that any
subscription queue throws a resource limit exception on send failure.
2017-10-25 14:53:20 +01:00
e793260573
[AMQ-6831, AMQ-6771] trigger eofexception on read -1 - AMQ6522Test
2017-10-11 00:20:44 +01:00
f989992278
[AMQ-6831, AMQ-6771] fix up recovery check to ensure full batch is available in memory, regression from AMQ-6771
2017-10-10 17:24:09 +01:00
8c3ef6cadb
[AMQ-6815] have checkpoint validate status of async writes to avoid stale metadata and validate location size on read to avoid potential oom on restart
2017-09-19 16:51:00 +01:00
a359d8152c
[AMQ-6277] take account of producer audit not being updatated on recovery check, avoid unnecessary partial journal replay
2017-09-15 13:48:03 +01:00
42bf6e9061
[AMQ-4947] ensure index page file does not skip forcing metadata - size is important to the page file
2017-09-04 16:43:48 +01:00
8c218ee05d
[AMQ-6771] do linear sequential scan of journal when validating checksums - remove batch reads via seek/read which depend on write batch size
2017-07-17 12:18:25 +01:00
b07821ab64
[AMQ-6625] remove kahadbioexceptionhandler by pushing allowIOResumption into persistence adapter. This allows the lease locker to still be used with kahadb for stopStartConnectors support
2017-06-15 17:28:31 +01:00
f4c11f73bb
[AMQ-6688] catch throwable on add task such that result error result is not lost on failed async send, fix and test
2017-05-29 17:14:05 +01:00
eb1a3eb77b
[AMQ-6670] - harden the test to avoid ci failure
2017-05-10 11:30:30 +01:00
053c38f99f
[AMQ-6670] - disable expiry processing that may page in and change the test
2017-05-08 11:53:55 +01:00
2518bded67
[AMQ-6670] - trap classcast and marshall errors on load message from journal such that errors are consistently propagated to the ioExceptionHandler
2017-05-05 15:43:09 +01:00
af03ad4679
[AMQ-6652] add final modifier to test
2017-04-11 11:39:22 +01:00
ca5e41bb7a
[AMQ-6652] ensure checkpoint pagefile flush includeds the checkpoint transaction
2017-04-10 17:15:07 +01:00
172c29091e
AMQ-6651: Add new implementations of the writeUTF8 and readUTF8 methods that are based on Apache Harmony code. This also avoid some code duplication that was occurring.
2017-04-10 10:49:17 -04:00
c1cbf509da
[AMQ-6644] - ensure journal files past last update are not included in the logged gc set in error
2017-04-04 12:21:31 +01:00
2731f04f1c
AMQ-6642: Fix potential NPE on updateMessage
...
Fixing potential NPE when calling updateMessage on a Queue store in
KahaDB if subscription statistics are enabled. Also reduced the
visibily from public to protected to subscription statistic related
methods that shouldn't be public and added null pointer checks there as
well.
https://issues.apache.org/jira/browse/AMQ-6642
2017-03-31 09:06:01 -04:00
c5a8b2c8b1
[AMQ-6625] ensure kahadb stops operation on the first IOException to facilitate auto recovery from partial writes
2017-03-13 13:49:41 +00:00
21ae1ef2e6
[AMQ-6606] - reset next batch so it does not get reused after error and refine fix to sync write batches b/c async locations will already be in the index
2017-03-07 12:28:58 +00:00
0d824a8e68
AMQ-6377: Storing JournalSyncStrategy as an enum
...
Switching MessageDatabase to store JournalSyncStrategy as its enum value
instead of String to avoid the performance penalty of comparing Strings
everytime isEnableJournalDiskSyncs() is called which currently happens
on every write.
2017-03-06 07:54:39 -05:00
d53b8f8d42
[AMQ-6606] avoid partial writes to the end of the journal - revert offset increment on ioexception, fix and test
2017-02-21 17:04:00 +00:00
9b64e188b5
[AMQ-6432] issue was journal scan on newly created ack file. I left the relevant braces from AMQ-6288 in place. Fix and test
2017-02-03 16:58:52 +00:00
e3b529129a
[AMQ-6432] improve the logging to see the current offset
2017-02-02 12:14:17 +00:00
83511c96e5
https://issues.apache.org/jira/browse/AMQ-6590
...
Removing un-needed imports
2017-02-02 07:09:57 -05:00
38d85be476
https://issues.apache.org/jira/browse/AMQ-6590
...
Fix KahaDB index free page recovery on unclean shutdown so that existing
free pages will be tracked and not lost.
2017-02-02 07:06:11 -05:00
0ad62f722f
https://issues.apache.org/jira/browse/AMQ-6572
...
On journal start use lastFileLength instead of maxFileLength when
checking for unused space in the last journal file as it is possible the
length of the last journal file is not the same length as maxFileLength
2017-01-23 11:01:20 -05:00
20522394cc
[AMQ-6567] ensure gc file removal/move is completed after index updates to avoid dangling index referenced on partial failure. Fix and test
2017-01-17 14:05:04 +00:00
281d600ae2
https://issues.apache.org/jira/browse/AMQ-6563
...
ensure that the lock is always released in the load method.
2017-01-13 11:48:19 -05:00
0ae90c0897
https://issues.apache.org/jira/browse/AMQ-6547 - add storeUsage attribute to mKahadb filtered adapter such that disk utilisation is available as a QOS
2016-12-20 14:47:22 +00:00
5fd63a0e4e
https://issues.apache.org/jira/browse/AMQ-6545
...
Fixing the order of locks acquired in the KahaDB journal to prevent a
deadlock during file rotation
2016-12-16 13:44:18 -05:00
0f0bdb21ef
https://issues.apache.org/jira/browse/AMQ-6524
...
Fixing a thread safety issue with memoryUsage when using concurrentStoreAndDispatch
that was causing memory usage to get out of sync.
The InnerFutureTask class inside KahaDB was not thread safe which was
the root cause of the problem.
2016-12-05 07:10:20 -05:00
dad629e889
[AMQ-6522] - remove hardcoded 32k batch limit from recovery check of the journal, fix and test
2016-11-29 11:32:03 +00:00
0a29533ed7
[AMQ-6520] respect kahadb indexDirectory for perDestination mKahaDB - fix and test
2016-11-25 16:44:28 +00:00
2c828b23df
https://issues.apache.org/jira/browse/AMQ-6504
...
Fix intermittent test failure
2016-11-21 12:37:52 -05:00
980162233f
https://issues.apache.org/jira/browse/AMQ-6504
...
Round the start time value not truncate to ensure delay falls on the
correct side of the scheduling block.
2016-11-11 14:48:49 -05:00
3b7613d930
AMQ-6451 - catch errors on preallocation. allocate direct buffer on start and reuse to ensure resource availability at runtime
2016-11-08 14:05:46 +00:00
e725451521
This closes #162 . Minor improvement in using the Logger api
2016-11-03 17:26:00 -04:00
be24a133ef
This closes #88 (merged as a65ac586). More file permissions fixes.
2016-11-03 14:26:34 -04:00
d54d046b8a
This closes #185 . Won't fix. (fix more file permissions)
2016-11-03 11:54:33 -04:00
1a91decf9f
https://issues.apache.org/jira/browse/AMQ-6483
...
Fix conversion from long to int and back to long again.
2016-10-26 13:27:24 -04:00
538ed74510
NO-JIRA - ensure stack trace in incuded with exception that terminates data file appender thread
2016-10-07 16:26:07 +01:00
351faf2699
https://issues.apache.org/jira/browse/AMQ-6445
...
Fixing RandomAccessFile usage so that the file will be properly closed
even if there is an exception
2016-09-29 06:16:40 -04:00
a82c95cd29
https://issues.apache.org/jira/browse/AMQ-6436
...
The temporary store will now delete the old temp directory on start up
if lazyInit is true instead of waiting for the store to initialize to
clear up space. This prevents space on the disk from being wasted with
old data if the temp store isn't initialized
2016-09-23 15:47:27 -04:00
18571ce09b
https://issues.apache.org/jira/browse/AMQ-6430
...
Modifying patch so that only stores that persist the noLocal flag will
check if this flag has changed to prevent a subscription from being
deleted by mistake
2016-09-23 10:59:32 -04:00
f8bc19b96d
AMQ-6413 - ensure audit update on skipped store add for kahadb concurrentStoreAndDispatch. Fix and test
2016-09-01 16:46:21 +01:00
5a874816b7
AMQ-6403 - add indexDirectory attribute to kahadb plist impl - settable via broker service tempDataStore
2016-08-24 11:39:22 +01:00
b37b9eb5af
fix up cancelledTaskModMetric reporting in kahadb
2016-08-19 11:48:42 +01:00
330f0ce734
AMQ-5815 ensure async write latch is released on exception
2016-08-12 16:44:23 +01:00
51166dc7af
AMQ-6377 - fix deprecation java doc ref
2016-08-10 12:05:21 +01:00
fcd9047b9f
additional test to validate stats on missing messages and outstanding acks on recovery
2016-08-10 11:54:12 +01:00
71bb54f94c
https://issues.apache.org/jira/browse/AMQ-6372
...
Fixing test so that test DataFile is written to a temporary directory
instead of the main activemq-kahadb-store directory. This prevents git
from picking up the test file
2016-07-30 12:59:10 -04:00
1a598277cf
https://issues.apache.org/jira/browse/AMQ-6377
...
Reworking patch so that a periodic disk sync uses a journal trace
command to trigger the sync so that everything is done in the same
thread for the writes
2016-07-28 10:07:10 -04:00
498e2184be
https://issues.apache.org/jira/browse/AMQ-6377
...
Adding new setters to KahaDBPersistenceAdapter
2016-07-27 17:33:45 -04:00
dd0ed17e59
https://issues.apache.org/jira/browse/AMQ-6377
...
Introducing JournalDiskSyncStrategy to allow a peridic disk sync mode
instead of always syncing after every write or never syncing.
2016-07-27 14:31:07 -04:00
822e2be90e
AMQ-6378 force recovery on corrupt metadata location info, relates to AMQ-6376
2016-07-27 15:56:28 +01:00
6b8e743b08
AMQ-6372 - add IOExceptionHandler to kahadb read path to ensure fast shutdown on disk access errors
2016-07-27 11:58:22 +01:00
cf3d419528
https://issues.apache.org/jira/browse/AMQ-6375
...
Adding non-blocking metrics to KahaDB to track message counts and sizes
for subscriptions. Turned off by default but can be enabled on the
KahaDBPersistenceAdapter
2016-07-26 12:02:44 -04:00
d427952b19
AMQ-6372 - fix up accessor pool cleanupUnused and use the minimum number of open files for recovery
2016-07-25 13:33:24 +01:00
8cc5c56d5e
https://issues.apache.org/jira/browse/AMQ-6358
...
Wait until recover is done before starting the checkpoint tasks.
2016-07-14 10:49:59 -04:00
a5050a8bc5
https://issues.apache.org/jira/browse/AMQ-6356
...
Fixing store size calculation on KahaUpdateMessageCommand processing so
that the size won't increase inadvertently if the existing location of
the command in the journal is the same as the new location
2016-07-14 09:21:23 -04:00
d635a36940
https://issues.apache.org/jira/browse/AMQ-6352
...
Fixing PList size tracking by moving the null assignment on removal to
after the onRemoval method is called to make sure that the decrement is
done properly.
2016-07-12 10:27:57 -04:00
4386750fdb
for mkahadb populate mdc activemq.persistenceDir such that logging for a particular instance can be identified and correlated
2016-07-04 11:19:56 +01:00
4d6cc4b460
https://issues.apache.org/jira/browse/AMQ-6303
...
Properly setting typeCode value for new journal files used for ack
compaction
2016-05-23 18:42:29 +00:00
15405af2e6
https://issues.apache.org/jira/browse/AMQ-6292
...
allow for batch writes of forwarded acks.
2016-05-13 14:56:01 -04:00
c8a6171d04
https://issues.apache.org/jira/browse/AMQ-6288
...
Switching the checkpoint lock to a readlock when forwarding acks to
prevent other journal updates from being blocked.
2016-05-12 14:39:12 +00:00
e53e340262
https://issues.apache.org/jira/browse/AMQ-6288
...
The ack compaction task now acquires the checkpoint lock while it runs
to prevent a checkpoint from running at the same time unintentionally.
Also, getJournalLocation is now protected by a try/catch to handle
errors.
2016-05-11 13:16:57 +00:00
91213010b3
https://issues.apache.org/jira/browse/AMQ-6285
...
Added an extra isShutdown() failsafe check when detecting if the
checkpoint thread should start
2016-05-09 19:35:02 +00:00
7bdcca1bda
https://issues.apache.org/jira/browse/AMQ-6285
...
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the
thread will properly restart.
2016-05-09 19:06:40 +00:00
db3f8b3554
Revert " https://issues.apache.org/jira/browse/AMQ-6285 "
...
This reverts commit 60b0c4f85a
2016-05-09 19:05:52 +00:00
60b0c4f85a
https://issues.apache.org/jira/browse/AMQ-6285
...
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the thread will
properly restart.
2016-05-09 19:03:38 +00:00
65cef69130
https://issues.apache.org/jira/browse/AMQ-5603 - reverting default preallocatonScope to entire_journal b/c async only really works for ssd
2016-05-05 13:06:42 +01:00
Alan Protasio
Christopher L. Shannon (cshannon)
jgoodyear
jgoodyear
gtully
Jeff Genender
hkesler
Hiram Chirino
Timothy Bish
Hadrian Zbarcea