Commit Graph

86 Commits

Author SHA1 Message Date
Hiram Chirino b720b264e3 The the hostname on the replicated leveldb store test so that the test works better on machines with weird DNS settings. 2013-10-02 12:15:36 -04:00
Hiram Chirino 0b66abcbfb Fixing leveldb replication bug where loosing quorum did not properly restart the master in a clean way. 2013-10-02 12:15:36 -04:00
Hiram Chirino 8804341c19 Adding more debug info to help trouble shoot election issues with leveldb replication elections. 2013-10-02 10:45:58 -04:00
Hiram Chirino f75520fc8b Implementing AMQ-4744: Support using LevelDB as a nested store in mKahaDB 2013-09-27 09:19:48 -04:00
gtully efaa351db7 https://issues.apache.org/jira/browse/AMQ-4365 - allow lease locker to be used by kahadb - remove deps on jdbc pa. LockableService now passes a reference to a locker so it can pull the brokerService, extracted some of the jdbc lock common stuff, additional test kahadb with jdbc lease 2013-09-20 15:33:24 +01:00
Hiram Chirino d771ebb97e Fixes bug in replicated leveldb where log files on slaves were not getting GCed. 2013-09-16 11:58:03 -04:00
Hiram Chirino 1eca031356 leveldb replication Master was failing to give up being master after it's process is suspended by using ctrl-z. 2013-09-03 10:41:33 -04:00
Hiram R. Chirino f4d51e092b Adding a LevelDB version of the RedeliveryRestartTest. Implemented redelivery tracking in the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1518289 13f79535-47bb-0310-9956-ffa450edef68
2013-08-28 17:20:25 +00:00
Hiram R. Chirino 45e1462609 Persist the latest producer position across restarts in the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1515799 13f79535-47bb-0310-9956-ffa450edef68
2013-08-20 12:38:32 +00:00
Gary Tully 7c50c1c736 fix up failure - still leveldb variant problem that needs work - testQueueTransactionalOrderWithRestart - org.apache.activemq.bugs.AMQ2149LevelDBTest
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1512332 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 15:11:11 +00:00
Hiram R. Chirino 563cc1bc6a Warn when there are too many replication nodes attached.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1506965 13f79535-47bb-0310-9956-ffa450edef68
2013-07-25 13:40:59 +00:00
Timothy A. Bish 194c6535cd [LevelDB]
https://issues.apache.org/jira/browse/AMQ-4296

Fixes remainder of failing unit tests.  The LevelDB wasn't incrementing or decrementing reference counts on messages added to the store which causes the expectations of certain memory limit based tests to fail as the memory usage was being updates after the store add instead of during so a message could get placed into the batch list of a cursor when we did not expect that it would.  This could also cause a browse to return fewer message than we want as the in memory messages would top out the usage limit so we'd never page in one batch of messages. 

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1505805 13f79535-47bb-0310-9956-ffa450edef68
2013-07-22 20:53:27 +00:00
Timothy A. Bish 11ed6b0e18 {LevelDB] fix for StoreQueueCursorLevelDBNoDuplicateTest failure. When the cursor calls setBatch and then recovering the next batch its expecting the batch to start at lastMsgId + 1 so we were seeing a single duplicate message and since audit was disabled it was delivered.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1504668 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 21:21:38 +00:00
Hiram R. Chirino ae5abf143b Adding a container property to the zk node data used by the leveldb replication.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1504626 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 20:04:45 +00:00
Hiram R. Chirino 86e2426d1b Improve the replicated leveldb bits: Avoid dependencies on fabric-group stuff. Makes it easier to embed in different versions of a fabric osgi env.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1501877 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 17:49:03 +00:00
Timothy A. Bish 5f0dd8ab53 Fix for failing LveelDB unit tests where only non-persistent messages are sent in a TX. The preCommit wasn't being run so the Queue's orderIndexUpdates structure wasn't getting updated with the TX to process in the postCommit phase.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1501420 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 18:15:26 +00:00
Hiram R. Chirino 2019a21d96 Simplify and improve the leveldb replication MBean
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1499754 13f79535-47bb-0310-9956-ffa450edef68
2013-07-04 13:47:27 +00:00
Hiram R. Chirino 34f3329370 When the leveldb replicated master was shutting down the client would get notified of a failure and it would not be hidden from the client app. We now suppress sending failure messages to clients when a broker is shutting down so that the client failover logic can kick in an reconnect the client to another server gracefully.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1498978 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 15:28:42 +00:00
Hiram R. Chirino b98067732b Improve the replicated leveldb behavior when the number of nodes in the cluster falls below the required minimum. The master node will switch to electing mode. The master store startup will now also block until it syncs up with slaves so that we don't accept connections the master is fully online.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1498601 13f79535-47bb-0310-9956-ffa450edef68
2013-07-01 17:38:13 +00:00
Hiram R. Chirino 16cd8c3954 Expose the replicated store status via JMX.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1497843 13f79535-47bb-0310-9956-ffa450edef68
2013-06-28 16:57:29 +00:00
Hiram R. Chirino baf8c499b3 Fixes AMQ-4599: Only package the pure java LevelDB driver by default
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1496514 13f79535-47bb-0310-9956-ffa450edef68
2013-06-25 15:12:23 +00:00
Bosanac Dejan 06dae40d94 https://issues.apache.org/jira/browse/AMQ-4593 - upgrade snappy java
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1495392 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 11:31:23 +00:00
Hiram R. Chirino 7ef79f2d2a Display who the slave leveldb store thinks is the master.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1494218 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 17:25:09 +00:00
Timothy A. Bish 6ee4c66fc3 m2e updates
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1492832 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 19:46:00 +00:00
Hiram R. Chirino 7db8987ca6 Fixup the assembly packaging so that the replicated leveldb store can be used.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1491287 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 20:41:24 +00:00
Hiram R. Chirino 5f301c40fa Help the xbean generate the proper xsd for the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1491278 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 20:16:22 +00:00
Hiram R. Chirino c161033abb Make the fabric dependencies of the leveldb store needed for replication as provided.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1490271 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 13:20:28 +00:00
Hiram R. Chirino 50e8795464 Fixes a ghost messages issue where the queue cursor goes out of sync /w the leveldb store when transactions are being used.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1490065 13f79535-47bb-0310-9956-ffa450edef68
2013-06-05 23:01:13 +00:00
Hiram R. Chirino bb30abf774 Try to avoid the 'short record at position' LevelDB error reported at: http://activemq.2283324.n4.nabble.com/Activemq-5-9-leveldb-replication-issue-tp4667495p4667674.html
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1489019 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 14:55:57 +00:00
Hiram R. Chirino ec9975c36e Additional fixes related to AMQ-4563: You can now configure the storeOpenWireVersion property of a broker to control which version of openwire is used by the persistence stores. This needs to be set to version 10 to preserve the original AMQP message ids.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1488375 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 19:56:03 +00:00
Hiram R. Chirino 300a26d350 No need for the uber jar anymore since we are part of the distro.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1486243 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 22:33:38 +00:00
Hiram R. Chirino b41121e359 This should fixed the problem with the delayed leveldb index updates.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1486242 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 22:33:32 +00:00
Hiram R. Chirino 75245da626 Seems like on some machines leveldb index updates are delayed.. looping seems to fix it.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1485897 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 23:17:26 +00:00
Hiram R. Chirino 148909357f related to AMQ-4296 : Fixes leveldb store cursoring. It was recovering too many messages and sometimes not the right messages.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1485810 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 18:33:06 +00:00
Hiram R. Chirino 49a2d9656e Convert some tests to junit 4 style and add test timeouts.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1484496 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 15:02:09 +00:00
Hiram R. Chirino 3e2dcc1202 No need to sync to the remote machines if a remote sync option is not enabled.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1484480 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 13:38:56 +00:00
Hiram R. Chirino 57617b862f Try initializing the replicated leveldb response_callbacks early on to avoid potential NPE.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1484049 13f79535-47bb-0310-9956-ffa450edef68
2013-05-18 03:07:35 +00:00
Hiram R. Chirino b6703d6bd7 Fixes failing ThreeBrokerVirtualTopicNetwork test associated /w AMQ-4296
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483878 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 15:47:04 +00:00
Hiram R. Chirino 1558cbbb55 Support multiple local/remote syncing styles.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483810 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 14:04:12 +00:00
Hiram R. Chirino ef6df68831 Fixes issue in AMQ-4296: Implement a pure LevelDB test for the StoreQueueCursorNoDuplicateTest and fix the bug that was causing it to fail.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483582 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 21:50:25 +00:00
Hiram R. Chirino b5505d462d Fix for AMQ-4296: Don't GC past LevelDB records pointed to by prepared XA transactions.
Also avoid double logging a LOG_ADD_ENTRY.

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483530 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 19:34:48 +00:00
Hiram R. Chirino e40d280115 Reduce how often forced leveldb compactions occur.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483395 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 14:49:51 +00:00
Hiram R. Chirino 086e9de9ac Additional fix for AMQ-4535: Seems store was getting out of sync /w cursor due to thread unsafe access in the leveldb store.
Switched to a concurrent HashMap to track the transaction since that will be getting accessed by multiple threads.

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483369 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 13:50:35 +00:00
Hiram R. Chirino 2ecb5eaa1b Don't delay flushes for now. Seems to cause more harm than good.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482845 13f79535-47bb-0310-9956-ffa450edef68
2013-05-15 13:58:25 +00:00
Hiram R. Chirino 8461158178 Fixes AMQ-4535: Activemq configured with leveldb commit fail when accessed by PutGet from IBM Perf Harness
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482789 13f79535-47bb-0310-9956-ffa450edef68
2013-05-15 12:30:45 +00:00
Hiram R. Chirino 842630c8c7 Implement some tracking of producer positions in the LevelDB store to help the broker to filter out dups.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482788 13f79535-47bb-0310-9956-ffa450edef68
2013-05-15 12:30:34 +00:00
Christian Posta 63c0ef179a Fix for https://issues.apache.org/jira/browse/AMQ-4534 When using ElectingLevelDBStore cannot delete all messages on startup
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482180 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 03:37:39 +00:00
Hiram R. Chirino 875885dfa9 Increase the default auto compaction ratio.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1481528 13f79535-47bb-0310-9956-ffa450edef68
2013-05-12 11:23:40 +00:00
Hiram R. Chirino 30038957b6 Add a storeConentAndClear() method to message so that we can lower the memory impact of embedded broker usage.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1481527 13f79535-47bb-0310-9956-ffa450edef68
2013-05-12 11:23:30 +00:00
Hiram R. Chirino 2572e2bd04 Add a little main class to the leveldb store module to make it easier to run a broker from your IDE.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1481014 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 14:03:39 +00:00