2731f04f1c
AMQ-6642: Fix potential NPE on updateMessage
...
Fixing potential NPE when calling updateMessage on a Queue store in
KahaDB if subscription statistics are enabled. Also reduced the
visibily from public to protected to subscription statistic related
methods that shouldn't be public and added null pointer checks there as
well.
https://issues.apache.org/jira/browse/AMQ-6642
2017-03-31 09:06:01 -04:00
c5a8b2c8b1
[AMQ-6625] ensure kahadb stops operation on the first IOException to facilitate auto recovery from partial writes
2017-03-13 13:49:41 +00:00
21ae1ef2e6
[AMQ-6606] - reset next batch so it does not get reused after error and refine fix to sync write batches b/c async locations will already be in the index
2017-03-07 12:28:58 +00:00
0d824a8e68
AMQ-6377: Storing JournalSyncStrategy as an enum
...
Switching MessageDatabase to store JournalSyncStrategy as its enum value
instead of String to avoid the performance penalty of comparing Strings
everytime isEnableJournalDiskSyncs() is called which currently happens
on every write.
2017-03-06 07:54:39 -05:00
d53b8f8d42
[AMQ-6606] avoid partial writes to the end of the journal - revert offset increment on ioexception, fix and test
2017-02-21 17:04:00 +00:00
9b64e188b5
[AMQ-6432] issue was journal scan on newly created ack file. I left the relevant braces from AMQ-6288 in place. Fix and test
2017-02-03 16:58:52 +00:00
e3b529129a
[AMQ-6432] improve the logging to see the current offset
2017-02-02 12:14:17 +00:00
83511c96e5
https://issues.apache.org/jira/browse/AMQ-6590
...
Removing un-needed imports
2017-02-02 07:09:57 -05:00
38d85be476
https://issues.apache.org/jira/browse/AMQ-6590
...
Fix KahaDB index free page recovery on unclean shutdown so that existing
free pages will be tracked and not lost.
2017-02-02 07:06:11 -05:00
0ad62f722f
https://issues.apache.org/jira/browse/AMQ-6572
...
On journal start use lastFileLength instead of maxFileLength when
checking for unused space in the last journal file as it is possible the
length of the last journal file is not the same length as maxFileLength
2017-01-23 11:01:20 -05:00
20522394cc
[AMQ-6567] ensure gc file removal/move is completed after index updates to avoid dangling index referenced on partial failure. Fix and test
2017-01-17 14:05:04 +00:00
281d600ae2
https://issues.apache.org/jira/browse/AMQ-6563
...
ensure that the lock is always released in the load method.
2017-01-13 11:48:19 -05:00
0ae90c0897
https://issues.apache.org/jira/browse/AMQ-6547 - add storeUsage attribute to mKahadb filtered adapter such that disk utilisation is available as a QOS
2016-12-20 14:47:22 +00:00
5fd63a0e4e
https://issues.apache.org/jira/browse/AMQ-6545
...
Fixing the order of locks acquired in the KahaDB journal to prevent a
deadlock during file rotation
2016-12-16 13:44:18 -05:00
0f0bdb21ef
https://issues.apache.org/jira/browse/AMQ-6524
...
Fixing a thread safety issue with memoryUsage when using concurrentStoreAndDispatch
that was causing memory usage to get out of sync.
The InnerFutureTask class inside KahaDB was not thread safe which was
the root cause of the problem.
2016-12-05 07:10:20 -05:00
dad629e889
[AMQ-6522] - remove hardcoded 32k batch limit from recovery check of the journal, fix and test
2016-11-29 11:32:03 +00:00
0a29533ed7
[AMQ-6520] respect kahadb indexDirectory for perDestination mKahaDB - fix and test
2016-11-25 16:44:28 +00:00
2c828b23df
https://issues.apache.org/jira/browse/AMQ-6504
...
Fix intermittent test failure
2016-11-21 12:37:52 -05:00
980162233f
https://issues.apache.org/jira/browse/AMQ-6504
...
Round the start time value not truncate to ensure delay falls on the
correct side of the scheduling block.
2016-11-11 14:48:49 -05:00
3b7613d930
AMQ-6451 - catch errors on preallocation. allocate direct buffer on start and reuse to ensure resource availability at runtime
2016-11-08 14:05:46 +00:00
e725451521
This closes #162 . Minor improvement in using the Logger api
2016-11-03 17:26:00 -04:00
be24a133ef
This closes #88 (merged as a65ac586). More file permissions fixes.
2016-11-03 14:26:34 -04:00
d5a929c0d2
This closes #205 . Won't fix. (fix even more file permissions)
2016-11-03 14:11:40 -04:00
d54d046b8a
This closes #185 . Won't fix. (fix more file permissions)
2016-11-03 11:54:33 -04:00
1a91decf9f
https://issues.apache.org/jira/browse/AMQ-6483
...
Fix conversion from long to int and back to long again.
2016-10-26 13:27:24 -04:00
538ed74510
NO-JIRA - ensure stack trace in incuded with exception that terminates data file appender thread
2016-10-07 16:26:07 +01:00
351faf2699
https://issues.apache.org/jira/browse/AMQ-6445
...
Fixing RandomAccessFile usage so that the file will be properly closed
even if there is an exception
2016-09-29 06:16:40 -04:00
a82c95cd29
https://issues.apache.org/jira/browse/AMQ-6436
...
The temporary store will now delete the old temp directory on start up
if lazyInit is true instead of waiting for the store to initialize to
clear up space. This prevents space on the disk from being wasted with
old data if the temp store isn't initialized
2016-09-23 15:47:27 -04:00
18571ce09b
https://issues.apache.org/jira/browse/AMQ-6430
...
Modifying patch so that only stores that persist the noLocal flag will
check if this flag has changed to prevent a subscription from being
deleted by mistake
2016-09-23 10:59:32 -04:00
f8bc19b96d
AMQ-6413 - ensure audit update on skipped store add for kahadb concurrentStoreAndDispatch. Fix and test
2016-09-01 16:46:21 +01:00
5a874816b7
AMQ-6403 - add indexDirectory attribute to kahadb plist impl - settable via broker service tempDataStore
2016-08-24 11:39:22 +01:00
b37b9eb5af
fix up cancelledTaskModMetric reporting in kahadb
2016-08-19 11:48:42 +01:00
330f0ce734
AMQ-5815 ensure async write latch is released on exception
2016-08-12 16:44:23 +01:00
51166dc7af
AMQ-6377 - fix deprecation java doc ref
2016-08-10 12:05:21 +01:00
fcd9047b9f
additional test to validate stats on missing messages and outstanding acks on recovery
2016-08-10 11:54:12 +01:00
06a21dbbb3
[maven-release-plugin] prepare for next development iteration
2016-08-02 09:03:37 -04:00
71cbc65283
[maven-release-plugin] prepare release activemq-5.14.0
2016-08-02 09:02:54 -04:00
71bb54f94c
https://issues.apache.org/jira/browse/AMQ-6372
...
Fixing test so that test DataFile is written to a temporary directory
instead of the main activemq-kahadb-store directory. This prevents git
from picking up the test file
2016-07-30 12:59:10 -04:00
1a598277cf
https://issues.apache.org/jira/browse/AMQ-6377
...
Reworking patch so that a periodic disk sync uses a journal trace
command to trigger the sync so that everything is done in the same
thread for the writes
2016-07-28 10:07:10 -04:00
498e2184be
https://issues.apache.org/jira/browse/AMQ-6377
...
Adding new setters to KahaDBPersistenceAdapter
2016-07-27 17:33:45 -04:00
dd0ed17e59
https://issues.apache.org/jira/browse/AMQ-6377
...
Introducing JournalDiskSyncStrategy to allow a peridic disk sync mode
instead of always syncing after every write or never syncing.
2016-07-27 14:31:07 -04:00
822e2be90e
AMQ-6378 force recovery on corrupt metadata location info, relates to AMQ-6376
2016-07-27 15:56:28 +01:00
6b8e743b08
AMQ-6372 - add IOExceptionHandler to kahadb read path to ensure fast shutdown on disk access errors
2016-07-27 11:58:22 +01:00
cf3d419528
https://issues.apache.org/jira/browse/AMQ-6375
...
Adding non-blocking metrics to KahaDB to track message counts and sizes
for subscriptions. Turned off by default but can be enabled on the
KahaDBPersistenceAdapter
2016-07-26 12:02:44 -04:00
d427952b19
AMQ-6372 - fix up accessor pool cleanupUnused and use the minimum number of open files for recovery
2016-07-25 13:33:24 +01:00
8cc5c56d5e
https://issues.apache.org/jira/browse/AMQ-6358
...
Wait until recover is done before starting the checkpoint tasks.
2016-07-14 10:49:59 -04:00
a5050a8bc5
https://issues.apache.org/jira/browse/AMQ-6356
...
Fixing store size calculation on KahaUpdateMessageCommand processing so
that the size won't increase inadvertently if the existing location of
the command in the journal is the same as the new location
2016-07-14 09:21:23 -04:00
d635a36940
https://issues.apache.org/jira/browse/AMQ-6352
...
Fixing PList size tracking by moving the null assignment on removal to
after the onRemoval method is called to make sure that the decrement is
done properly.
2016-07-12 10:27:57 -04:00
4386750fdb
for mkahadb populate mdc activemq.persistenceDir such that logging for a particular instance can be identified and correlated
2016-07-04 11:19:56 +01:00
4d6cc4b460
https://issues.apache.org/jira/browse/AMQ-6303
...
Properly setting typeCode value for new journal files used for ack
compaction
2016-05-23 18:42:29 +00:00
15405af2e6
https://issues.apache.org/jira/browse/AMQ-6292
...
allow for batch writes of forwarded acks.
2016-05-13 14:56:01 -04:00
c8a6171d04
https://issues.apache.org/jira/browse/AMQ-6288
...
Switching the checkpoint lock to a readlock when forwarding acks to
prevent other journal updates from being blocked.
2016-05-12 14:39:12 +00:00
e53e340262
https://issues.apache.org/jira/browse/AMQ-6288
...
The ack compaction task now acquires the checkpoint lock while it runs
to prevent a checkpoint from running at the same time unintentionally.
Also, getJournalLocation is now protected by a try/catch to handle
errors.
2016-05-11 13:16:57 +00:00
91213010b3
https://issues.apache.org/jira/browse/AMQ-6285
...
Added an extra isShutdown() failsafe check when detecting if the
checkpoint thread should start
2016-05-09 19:35:02 +00:00
7bdcca1bda
https://issues.apache.org/jira/browse/AMQ-6285
...
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the
thread will properly restart.
2016-05-09 19:06:40 +00:00
db3f8b3554
Revert " https://issues.apache.org/jira/browse/AMQ-6285 "
...
This reverts commit 60b0c4f85a
2016-05-09 19:05:52 +00:00
60b0c4f85a
https://issues.apache.org/jira/browse/AMQ-6285
...
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the thread will
properly restart.
2016-05-09 19:03:38 +00:00
65cef69130
https://issues.apache.org/jira/browse/AMQ-5603 - reverting default preallocatonScope to entire_journal b/c async only really works for ssd
2016-05-05 13:06:42 +01:00
1c4108545c
https://issues.apache.org/jira/browse/AMQ-6277 - journal getNextLocation needs too passes to skip past if target is not initialized
2016-05-04 22:09:06 +01:00
555cd2b6bb
https://issues.apache.org/jira/browse/AMQ-5603 - catch interrupted on chunked zeros case
2016-05-03 17:25:14 +01:00
ba77b9f55a
https://issues.apache.org/jira/browse/AMQ-6277 - tidy up logic that determines recovery location so that we don't recovery from the end of the journal in error on normal restart. This avoids suprious recovery logging
2016-05-03 12:47:49 +01:00
62bdbb0db5
https://issues.apache.org/jira/browse/AMQ-5603 - add preallocationScope=full_journal_async that will preallocate a journal in advance or use to avoid latency jitter on journal rotation. Added none option to disable preallocation
2016-04-29 16:57:28 +01:00
43d8205cf0
reduce the throughput for ci
2016-04-22 14:53:14 +01:00
930a74c696
https://issues.apache.org/jira/browse/AMQ-5578 - enable org.apache.activemq.kahaDB.files.skipMetadataUpdate for tests to validate. remove some redundant call to limit and add latencey test to validate the need for outof band or pool allocation of journals
2016-04-22 12:29:31 +01:00
11622b3af3
https://issues.apache.org/jira/browse/AMQ-6256
...
Moving beforeMarshall back to the store implementations because we don't
want all store implementations to marshall (such as memory store)
This reverts commit b9b98a45ce
2016-04-18 12:34:25 +00:00
1ffac14f6e
https://issues.apache.org/jira/browse/AMQ-6256
...
cleanup imports
2016-04-15 14:23:54 +00:00
b9b98a45ce
https://issues.apache.org/jira/browse/AMQ-6256
...
Moving beforeMarshall call out of the store and into the actual
destination
2016-04-15 14:17:00 +00:00
b9f9f03829
https://issues.apache.org/jira/browse/AMQ-6256
...
Calling beforeMarshall on messages when they async stored before the
store task is run and before consumer dispatch to prevent two threads
from trying to mutate the message state at the same time.
2016-04-15 13:01:21 +00:00
092c56d70c
https://issues.apache.org/jira/browse/AMQ-6250
...
Checking for null before closing the journal in
MultiKahaDBPersistentAdapter. Also protecting against multiple calls to
start and stop.
2016-04-14 18:39:14 +00:00
cbad8babe5
https://issues.apache.org/jira/browse/AMQ-6203
...
Adding a flag to enable or disable Ack Compaction
Currently defaults to enabled for 5.14.0 and above
2016-04-12 13:30:10 +00:00
a9521dcebf
https://issues.apache.org/jira/browse/AMQ-6203
...
Expose configuration on the persistence adapter level.
Double the test timeout and increase the compaction frequency to account
for very slow CI boxes.
2016-03-15 12:02:40 -04:00
946e62d702
https://issues.apache.org/jira/browse/AMQ-6203
...
Rewrite older acks that can be preventing GC of log files.
2016-03-14 11:04:57 -04:00
b39ab7867c
https://issues.apache.org/jira/browse/AMQ-6207
...
Perform writes / sync in predicatable order first to the recovery file
and then into the main index file.
2016-03-11 09:59:07 -05:00
13ec994939
Revert " https://issues.apache.org/jira/browse/AMQ-4495 - revisit. Reinstate check for space on pagein, so that highWaterMark is respected and full state is not reached, hense pfc is not triggered in error"
...
This reverts commit d8cf54b0a9
2016-03-07 16:26:25 +00:00
5e0f49332b
https://issues.apache.org/jira/browse/AMQ-5578
...
Remove unnecessary buffer initialization after confirmation that it is
not needed.
2016-03-03 17:35:21 -05:00
edfc23ee9d
https://issues.apache.org/jira/browse/AMQ-5578
...
Add additional preallocation strategy [CHUNKED contributed by user
jtahlborn.
2016-03-03 17:34:07 -05:00
be4ad3d1c8
https://issues.apache.org/jira/browse/AMQ-5578
...
Replace system.out with a trace log.
2016-03-03 17:00:11 -05:00
e8f44a29dd
https://issues.apache.org/jira/browse/AMQ-5578
...
Revert to previous behavior of explicit init.
2016-03-02 17:18:30 -05:00
e89b7a57f1
https://issues.apache.org/jira/browse/AMQ-5578
...
Remove unnecessary zero init block as the java spec requires the
allocations to do this already. Fix some warnings.
2016-03-02 16:17:43 -05:00
d8cf54b0a9
https://issues.apache.org/jira/browse/AMQ-4495 - revisit. Reinstate check for space on pagein, so that highWaterMark is respected and full state is not reached, hense pfc is not triggered in error
2016-03-01 11:44:41 +00:00
499e39e52c
https://issues.apache.org/jira/browse/AMQ-6164 - allow journal write batching on a single destination
2016-02-09 12:50:27 +00:00
25ff5699f1
https://issues.apache.org/jira/browse/AMQ-6158
...
Computing messageSize for a durable subscription in KahaDB now runs much
faster (n vs n^2) which is noticable when there are a large number of
pending messages for a durable subscription.
2016-02-04 15:17:49 +00:00
8c4b5f485d
https://issues.apache.org/jira/browse/AMQ-6152
...
Ensure that when add / remove commands are colocated they don't prevent
the log from being GC'd once it is unreferenced.
2016-02-02 20:28:24 -05:00
4ecff14030
NO-JIRA Update master version to 5.14.0-SNAPSHOT to match the eventual
...
release version.
2016-01-20 12:40:20 -05:00
b4aa53d806
https://issues.apache.org/jira/browse/AMQ-6133
...
Fixing the processing of KahaUpdateMessageCommand to add the message to
the store if it doesn't exist, instead of ignoring it. This will
prevent message loss in certain cases when the KahaDB index has to be
rebuilt.
2016-01-20 16:59:07 +00:00
193f6be687
https://issues.apache.org/jira/browse/AMQ-6131
...
Fix check for durable sub with no pending messages during checkpoint
cleanup.
2016-01-18 17:43:18 -05:00
507d40a79f
https://issues.apache.org/jira/browse/AMQ-6097
...
Fixing JournalCorruptionEofIndexRecoveryTest class
Because of the size being stored in the message order index, the test
testRecoveryAftercorruptionCheckSumExistingIndex was no longer passing
as that test was corrupting the size value in the journal, but not the
size value in the index. The test now corrupts both values so that the
expect exception occurs again.
2015-12-21 16:46:47 +00:00
246ccb8e04
https://issues.apache.org/jira/browse/AMQ-6097
...
Fixing KahaDB so that the correct marshaller is used for the message
keys inside of the message order index. This will ensure that message
size metrics are accurate.
2015-12-18 19:26:24 +00:00
31d99b6aa9
https://issues.apache.org/jira/browse/AMQ-6083 - resolve some test regressions - AMQ2149Test - fail to start with missing data file. Ensure ack map is always saved on modification
2015-12-11 13:30:55 +00:00
768fa17085
https://issues.apache.org/jira/browse/AMQ-6005 - have plist whack it existing state on start. Start the temp store after the primary persistence adapter which does the locking
2015-12-10 14:24:03 +00:00
5db5f3e39a
https://issues.apache.org/jira/browse/AMQ-6083 - ignoreMissingJournalfiles false - check for missing acks and corruption anywhere and error out so that corruption does not go unnoticed - fix and test
2015-12-09 12:30:29 +00:00
7a7c70ad75
https://issues.apache.org/jira/browse/AMQ-6082
...
Propertly re-setting the storeOpenWireVersion from the BrokerService
on the KahaDB Metadata if a corrupted index is detected and the
Metadata has to be recreated.
2015-12-08 20:10:11 +00:00
66c348b1b8
https://issues.apache.org/jira/browse/AMQ-6071 - Log info about corrupted journal records at WARN level
2015-12-04 11:29:55 +00:00
cfb4ca9f54
[maven-release-plugin] prepare for next development iteration
2015-11-30 13:48:58 +00:00
abfe038ddf
[maven-release-plugin] prepare release activemq-5.13.0
2015-11-30 13:48:11 +00:00
8f30866fdf
https://issues.apache.org/jira/browse/AMQ-6063
...
Fixing potential NullPointerException during KahaDB index recovery.
2015-11-25 16:02:52 +00:00
e07809dec6
NO JIRA - Clear journalSize counter
...
The journalSize counter should be set to 0 after KahaDB is shut down.
Usually, a store is not restarted again after stopping, but if it ever
was the value would be wrong since it wasn't reset back to 0.
2015-11-24 17:53:34 +00:00
fd0f71a4c8
https://issues.apache.org/jira/browse/AMQ-6033 - fix shadowing error, additional test
2015-11-02 15:08:22 +00:00
dc06c8dc75
https://issues.apache.org/jira/browse/AMQ-5994 https://issues.apache.org/jira/browse/AMQ-4000 - proper fix for duplicate sub info from the store on recovery failure from AMQ2149Test. Additional test from Christopher L
2015-10-06 12:01:24 +01:00
f6e1054a4b
https://issues.apache.org/jira/browse/AMQ-5984 - fix sync on get for access ordered LinkedHashMap
2015-09-28 14:20:45 +01:00
Christopher L. Shannon (cshannon)
gtully
Timothy Bish
Hadrian Zbarcea
Torsten Mielke