Commit Graph

195 Commits

Author SHA1 Message Date
Christopher L. Shannon (cshannon) 4d6cc4b460 https://issues.apache.org/jira/browse/AMQ-6303
Properly setting typeCode value for new journal files used for ack
compaction
2016-05-23 18:42:29 +00:00
Timothy Bish 15405af2e6 https://issues.apache.org/jira/browse/AMQ-6292
allow for batch writes of forwarded acks.
2016-05-13 14:56:01 -04:00
Christopher L. Shannon (cshannon) c8a6171d04 https://issues.apache.org/jira/browse/AMQ-6288
Switching the checkpoint lock to a readlock when forwarding acks to
prevent other journal updates from being blocked.
2016-05-12 14:39:12 +00:00
Christopher L. Shannon (cshannon) e53e340262 https://issues.apache.org/jira/browse/AMQ-6288
The ack compaction task now acquires the checkpoint lock while it runs
to prevent a checkpoint from running at the same time unintentionally.
Also, getJournalLocation is now protected by a try/catch to handle
errors.
2016-05-11 13:16:57 +00:00
Christopher L. Shannon (cshannon) 91213010b3 https://issues.apache.org/jira/browse/AMQ-6285
Added an extra isShutdown() failsafe check when detecting if the
checkpoint thread should start
2016-05-09 19:35:02 +00:00
Christopher L. Shannon (cshannon) 7bdcca1bda https://issues.apache.org/jira/browse/AMQ-6285
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the
thread will properly restart.
2016-05-09 19:06:40 +00:00
Christopher L. Shannon (cshannon) db3f8b3554 Revert "https://issues.apache.org/jira/browse/AMQ-6285"
This reverts commit 60b0c4f85a.

Inadvertantly commited a bunch of changes by mistake
2016-05-09 19:05:52 +00:00
Christopher L. Shannon (cshannon) 60b0c4f85a https://issues.apache.org/jira/browse/AMQ-6285
Properly nulling out the scheduler service in MessageDatabase after it
is shutdown on a store close so that if the store is restarted the thread will
properly restart.
2016-05-09 19:03:38 +00:00
gtully 65cef69130 https://issues.apache.org/jira/browse/AMQ-5603 - reverting default preallocatonScope to entire_journal b/c async only really works for ssd 2016-05-05 13:06:42 +01:00
gtully 1c4108545c https://issues.apache.org/jira/browse/AMQ-6277 - journal getNextLocation needs too passes to skip past if target is not initialized 2016-05-04 22:09:06 +01:00
gtully 555cd2b6bb https://issues.apache.org/jira/browse/AMQ-5603 - catch interrupted on chunked zeros case 2016-05-03 17:25:14 +01:00
gtully ba77b9f55a https://issues.apache.org/jira/browse/AMQ-6277 - tidy up logic that determines recovery location so that we don't recovery from the end of the journal in error on normal restart. This avoids suprious recovery logging 2016-05-03 12:47:49 +01:00
gtully 62bdbb0db5 https://issues.apache.org/jira/browse/AMQ-5603 - add preallocationScope=full_journal_async that will preallocate a journal in advance or use to avoid latency jitter on journal rotation. Added none option to disable preallocation 2016-04-29 16:57:28 +01:00
gtully 43d8205cf0 reduce the throughput for ci 2016-04-22 14:53:14 +01:00
gtully 930a74c696 https://issues.apache.org/jira/browse/AMQ-5578 - enable org.apache.activemq.kahaDB.files.skipMetadataUpdate for tests to validate. remove some redundant call to limit and add latencey test to validate the need for outof band or pool allocation of journals 2016-04-22 12:29:31 +01:00
Christopher L. Shannon (cshannon) 11622b3af3 https://issues.apache.org/jira/browse/AMQ-6256
Moving beforeMarshall back to the store implementations because we don't
want all store implementations to marshall (such as memory store)

This reverts commit b9b98a45ce.
2016-04-18 12:34:25 +00:00
Christopher L. Shannon (cshannon) 1ffac14f6e https://issues.apache.org/jira/browse/AMQ-6256
cleanup imports
2016-04-15 14:23:54 +00:00
Christopher L. Shannon (cshannon) b9b98a45ce https://issues.apache.org/jira/browse/AMQ-6256
Moving beforeMarshall call out of the store and into the actual
destination
2016-04-15 14:17:00 +00:00
Christopher L. Shannon (cshannon) b9f9f03829 https://issues.apache.org/jira/browse/AMQ-6256
Calling beforeMarshall on messages when they async stored before the
store task is run and before consumer dispatch to prevent two threads
from trying to mutate the message state at the same time.
2016-04-15 13:01:21 +00:00
Christopher L. Shannon (cshannon) 092c56d70c https://issues.apache.org/jira/browse/AMQ-6250
Checking for null before closing the journal in
MultiKahaDBPersistentAdapter.  Also protecting against multiple calls to
start and stop.
2016-04-14 18:39:14 +00:00
Christopher L. Shannon (cshannon) cbad8babe5 https://issues.apache.org/jira/browse/AMQ-6203
Adding a flag to enable or disable Ack Compaction
Currently defaults to enabled for 5.14.0 and above
2016-04-12 13:30:10 +00:00
Timothy Bish a9521dcebf https://issues.apache.org/jira/browse/AMQ-6203
Expose configuration on the persistence adapter level.
Double the test timeout and increase the compaction frequency to account
for very slow CI boxes.
2016-03-15 12:02:40 -04:00
Timothy Bish 946e62d702 https://issues.apache.org/jira/browse/AMQ-6203
Rewrite older acks that can be preventing GC of log files.
2016-03-14 11:04:57 -04:00
Timothy Bish b39ab7867c https://issues.apache.org/jira/browse/AMQ-6207
Perform writes / sync in predicatable order first to the recovery file
and then into the main index file.
2016-03-11 09:59:07 -05:00
gtully 13ec994939 Revert "https://issues.apache.org/jira/browse/AMQ-4495 - revisit. Reinstate check for space on pagein, so that highWaterMark is respected and full state is not reached, hense pfc is not triggered in error"
This reverts commit d8cf54b0a9.
2016-03-07 16:26:25 +00:00
Timothy Bish 5e0f49332b https://issues.apache.org/jira/browse/AMQ-5578
Remove unnecessary buffer initialization after confirmation that it is
not needed.
2016-03-03 17:35:21 -05:00
Timothy Bish edfc23ee9d https://issues.apache.org/jira/browse/AMQ-5578
Add additional preallocation strategy [CHUNKED contributed by user
jtahlborn.
2016-03-03 17:34:07 -05:00
Timothy Bish be4ad3d1c8 https://issues.apache.org/jira/browse/AMQ-5578
Replace system.out with a trace log.
2016-03-03 17:00:11 -05:00
Timothy Bish e8f44a29dd https://issues.apache.org/jira/browse/AMQ-5578
Revert to previous behavior of explicit init.
2016-03-02 17:18:30 -05:00
Timothy Bish e89b7a57f1 https://issues.apache.org/jira/browse/AMQ-5578
Remove unnecessary zero init block as the java spec requires the
allocations to do this already.  Fix some warnings.
2016-03-02 16:17:43 -05:00
gtully d8cf54b0a9 https://issues.apache.org/jira/browse/AMQ-4495 - revisit. Reinstate check for space on pagein, so that highWaterMark is respected and full state is not reached, hense pfc is not triggered in error 2016-03-01 11:44:41 +00:00
gtully 499e39e52c https://issues.apache.org/jira/browse/AMQ-6164 - allow journal write batching on a single destination 2016-02-09 12:50:27 +00:00
Christopher L. Shannon (cshannon) 25ff5699f1 https://issues.apache.org/jira/browse/AMQ-6158
Computing messageSize for a durable subscription in KahaDB now runs much
faster (n vs n^2) which is noticable when there are a large number of
pending messages for a durable subscription.
2016-02-04 15:17:49 +00:00
Timothy Bish 8c4b5f485d https://issues.apache.org/jira/browse/AMQ-6152
Ensure that when add / remove commands are colocated they don't prevent
the log from being GC'd once it is unreferenced.
2016-02-02 20:28:24 -05:00
Timothy Bish 4ecff14030 NO-JIRA Update master version to 5.14.0-SNAPSHOT to match the eventual
release version.
2016-01-20 12:40:20 -05:00
Christopher L. Shannon (cshannon) b4aa53d806 https://issues.apache.org/jira/browse/AMQ-6133
Fixing the processing of KahaUpdateMessageCommand to add the message to
the store if it doesn't exist, instead of ignoring it.  This will
prevent message loss in certain cases when the KahaDB index has to be
rebuilt.
2016-01-20 16:59:07 +00:00
Timothy Bish 193f6be687 https://issues.apache.org/jira/browse/AMQ-6131
Fix check for durable sub with no pending messages during checkpoint
cleanup.
2016-01-18 17:43:18 -05:00
Christopher L. Shannon (cshannon) 507d40a79f https://issues.apache.org/jira/browse/AMQ-6097
Fixing JournalCorruptionEofIndexRecoveryTest class

Because of the size being stored in the message order index, the test
testRecoveryAftercorruptionCheckSumExistingIndex was no longer passing
as that test was corrupting the size value in the journal, but not the
size value in the index.  The test now corrupts both values so that the
expect exception occurs again.
2015-12-21 16:46:47 +00:00
Christopher L. Shannon (cshannon) 246ccb8e04 https://issues.apache.org/jira/browse/AMQ-6097
Fixing KahaDB so that the correct marshaller is used for the message
keys inside of the message order index.  This will ensure that message
size metrics are accurate.
2015-12-18 19:26:24 +00:00
gtully 31d99b6aa9 https://issues.apache.org/jira/browse/AMQ-6083 - resolve some test regressions - AMQ2149Test - fail to start with missing data file. Ensure ack map is always saved on modification 2015-12-11 13:30:55 +00:00
gtully 768fa17085 https://issues.apache.org/jira/browse/AMQ-6005 - have plist whack it existing state on start. Start the temp store after the primary persistence adapter which does the locking 2015-12-10 14:24:03 +00:00
gtully 5db5f3e39a https://issues.apache.org/jira/browse/AMQ-6083 - ignoreMissingJournalfiles false - check for missing acks and corruption anywhere and error out so that corruption does not go unnoticed - fix and test 2015-12-09 12:30:29 +00:00
Christopher L. Shannon (cshannon) 7a7c70ad75 https://issues.apache.org/jira/browse/AMQ-6082
Propertly re-setting the storeOpenWireVersion from the BrokerService
on the KahaDB Metadata if a corrupted index is detected and the
Metadata has to be recreated.
2015-12-08 20:10:11 +00:00
Torsten Mielke 66c348b1b8 https://issues.apache.org/jira/browse/AMQ-6071 - Log info about corrupted journal records at WARN level 2015-12-04 11:29:55 +00:00
Christopher L. Shannon (cshannon) cfb4ca9f54 [maven-release-plugin] prepare for next development iteration 2015-11-30 13:48:58 +00:00
Christopher L. Shannon (cshannon) abfe038ddf [maven-release-plugin] prepare release activemq-5.13.0 2015-11-30 13:48:11 +00:00
Christopher L. Shannon (cshannon) 8f30866fdf https://issues.apache.org/jira/browse/AMQ-6063
Fixing potential NullPointerException during KahaDB index recovery.
2015-11-25 16:02:52 +00:00
Christopher L. Shannon (cshannon) e07809dec6 NO JIRA - Clear journalSize counter
The journalSize counter should be set to 0 after KahaDB is shut down.
Usually, a store is not restarted again after stopping, but if it ever
was the value would be wrong since it wasn't reset back to 0.
2015-11-24 17:53:34 +00:00
gtully fd0f71a4c8 https://issues.apache.org/jira/browse/AMQ-6033 - fix shadowing error, additional test 2015-11-02 15:08:22 +00:00
gtully dc06c8dc75 https://issues.apache.org/jira/browse/AMQ-5994 https://issues.apache.org/jira/browse/AMQ-4000 - proper fix for duplicate sub info from the store on recovery failure from AMQ2149Test. Additional test from Christopher L 2015-10-06 12:01:24 +01:00