Commit Graph

127 Commits

Author SHA1 Message Date
Gary Tully 7c50c1c736 fix up failure - still leveldb variant problem that needs work - testQueueTransactionalOrderWithRestart - org.apache.activemq.bugs.AMQ2149LevelDBTest
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1512332 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 15:11:11 +00:00
Hiram R. Chirino 563cc1bc6a Warn when there are too many replication nodes attached.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1506965 13f79535-47bb-0310-9956-ffa450edef68
2013-07-25 13:40:59 +00:00
Timothy A. Bish 194c6535cd [LevelDB]
https://issues.apache.org/jira/browse/AMQ-4296

Fixes remainder of failing unit tests.  The LevelDB wasn't incrementing or decrementing reference counts on messages added to the store which causes the expectations of certain memory limit based tests to fail as the memory usage was being updates after the store add instead of during so a message could get placed into the batch list of a cursor when we did not expect that it would.  This could also cause a browse to return fewer message than we want as the in memory messages would top out the usage limit so we'd never page in one batch of messages. 

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1505805 13f79535-47bb-0310-9956-ffa450edef68
2013-07-22 20:53:27 +00:00
Timothy A. Bish 11ed6b0e18 {LevelDB] fix for StoreQueueCursorLevelDBNoDuplicateTest failure. When the cursor calls setBatch and then recovering the next batch its expecting the batch to start at lastMsgId + 1 so we were seeing a single duplicate message and since audit was disabled it was delivered.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1504668 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 21:21:38 +00:00
Hiram R. Chirino ae5abf143b Adding a container property to the zk node data used by the leveldb replication.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1504626 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 20:04:45 +00:00
Hiram R. Chirino 86e2426d1b Improve the replicated leveldb bits: Avoid dependencies on fabric-group stuff. Makes it easier to embed in different versions of a fabric osgi env.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1501877 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 17:49:03 +00:00
Timothy A. Bish 5f0dd8ab53 Fix for failing LveelDB unit tests where only non-persistent messages are sent in a TX. The preCommit wasn't being run so the Queue's orderIndexUpdates structure wasn't getting updated with the TX to process in the postCommit phase.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1501420 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 18:15:26 +00:00
Hiram R. Chirino 2019a21d96 Simplify and improve the leveldb replication MBean
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1499754 13f79535-47bb-0310-9956-ffa450edef68
2013-07-04 13:47:27 +00:00
Hiram R. Chirino 34f3329370 When the leveldb replicated master was shutting down the client would get notified of a failure and it would not be hidden from the client app. We now suppress sending failure messages to clients when a broker is shutting down so that the client failover logic can kick in an reconnect the client to another server gracefully.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1498978 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 15:28:42 +00:00
Hiram R. Chirino b98067732b Improve the replicated leveldb behavior when the number of nodes in the cluster falls below the required minimum. The master node will switch to electing mode. The master store startup will now also block until it syncs up with slaves so that we don't accept connections the master is fully online.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1498601 13f79535-47bb-0310-9956-ffa450edef68
2013-07-01 17:38:13 +00:00
Hiram R. Chirino 16cd8c3954 Expose the replicated store status via JMX.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1497843 13f79535-47bb-0310-9956-ffa450edef68
2013-06-28 16:57:29 +00:00
Hiram R. Chirino baf8c499b3 Fixes AMQ-4599: Only package the pure java LevelDB driver by default
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1496514 13f79535-47bb-0310-9956-ffa450edef68
2013-06-25 15:12:23 +00:00
Bosanac Dejan 06dae40d94 https://issues.apache.org/jira/browse/AMQ-4593 - upgrade snappy java
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1495392 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 11:31:23 +00:00
Hiram R. Chirino 7ef79f2d2a Display who the slave leveldb store thinks is the master.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1494218 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 17:25:09 +00:00
Timothy A. Bish 6ee4c66fc3 m2e updates
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1492832 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 19:46:00 +00:00
Hiram R. Chirino 7db8987ca6 Fixup the assembly packaging so that the replicated leveldb store can be used.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1491287 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 20:41:24 +00:00
Hiram R. Chirino 5f301c40fa Help the xbean generate the proper xsd for the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1491278 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 20:16:22 +00:00
Hiram R. Chirino c161033abb Make the fabric dependencies of the leveldb store needed for replication as provided.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1490271 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 13:20:28 +00:00
Hiram R. Chirino 50e8795464 Fixes a ghost messages issue where the queue cursor goes out of sync /w the leveldb store when transactions are being used.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1490065 13f79535-47bb-0310-9956-ffa450edef68
2013-06-05 23:01:13 +00:00
Hiram R. Chirino bb30abf774 Try to avoid the 'short record at position' LevelDB error reported at: http://activemq.2283324.n4.nabble.com/Activemq-5-9-leveldb-replication-issue-tp4667495p4667674.html
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1489019 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 14:55:57 +00:00
Hiram R. Chirino ec9975c36e Additional fixes related to AMQ-4563: You can now configure the storeOpenWireVersion property of a broker to control which version of openwire is used by the persistence stores. This needs to be set to version 10 to preserve the original AMQP message ids.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1488375 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 19:56:03 +00:00
Hiram R. Chirino 300a26d350 No need for the uber jar anymore since we are part of the distro.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1486243 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 22:33:38 +00:00
Hiram R. Chirino b41121e359 This should fixed the problem with the delayed leveldb index updates.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1486242 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 22:33:32 +00:00
Hiram R. Chirino 75245da626 Seems like on some machines leveldb index updates are delayed.. looping seems to fix it.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1485897 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 23:17:26 +00:00
Hiram R. Chirino 148909357f related to AMQ-4296 : Fixes leveldb store cursoring. It was recovering too many messages and sometimes not the right messages.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1485810 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 18:33:06 +00:00
Hiram R. Chirino 49a2d9656e Convert some tests to junit 4 style and add test timeouts.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1484496 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 15:02:09 +00:00
Hiram R. Chirino 3e2dcc1202 No need to sync to the remote machines if a remote sync option is not enabled.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1484480 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 13:38:56 +00:00
Hiram R. Chirino 57617b862f Try initializing the replicated leveldb response_callbacks early on to avoid potential NPE.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1484049 13f79535-47bb-0310-9956-ffa450edef68
2013-05-18 03:07:35 +00:00
Hiram R. Chirino b6703d6bd7 Fixes failing ThreeBrokerVirtualTopicNetwork test associated /w AMQ-4296
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483878 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 15:47:04 +00:00
Hiram R. Chirino 1558cbbb55 Support multiple local/remote syncing styles.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483810 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 14:04:12 +00:00
Hiram R. Chirino ef6df68831 Fixes issue in AMQ-4296: Implement a pure LevelDB test for the StoreQueueCursorNoDuplicateTest and fix the bug that was causing it to fail.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483582 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 21:50:25 +00:00
Hiram R. Chirino b5505d462d Fix for AMQ-4296: Don't GC past LevelDB records pointed to by prepared XA transactions.
Also avoid double logging a LOG_ADD_ENTRY.

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483530 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 19:34:48 +00:00
Hiram R. Chirino e40d280115 Reduce how often forced leveldb compactions occur.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483395 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 14:49:51 +00:00
Hiram R. Chirino 086e9de9ac Additional fix for AMQ-4535: Seems store was getting out of sync /w cursor due to thread unsafe access in the leveldb store.
Switched to a concurrent HashMap to track the transaction since that will be getting accessed by multiple threads.

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1483369 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 13:50:35 +00:00
Hiram R. Chirino 2ecb5eaa1b Don't delay flushes for now. Seems to cause more harm than good.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482845 13f79535-47bb-0310-9956-ffa450edef68
2013-05-15 13:58:25 +00:00
Hiram R. Chirino 8461158178 Fixes AMQ-4535: Activemq configured with leveldb commit fail when accessed by PutGet from IBM Perf Harness
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482789 13f79535-47bb-0310-9956-ffa450edef68
2013-05-15 12:30:45 +00:00
Hiram R. Chirino 842630c8c7 Implement some tracking of producer positions in the LevelDB store to help the broker to filter out dups.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482788 13f79535-47bb-0310-9956-ffa450edef68
2013-05-15 12:30:34 +00:00
Christian Posta 63c0ef179a Fix for https://issues.apache.org/jira/browse/AMQ-4534 When using ElectingLevelDBStore cannot delete all messages on startup
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1482180 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 03:37:39 +00:00
Hiram R. Chirino 875885dfa9 Increase the default auto compaction ratio.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1481528 13f79535-47bb-0310-9956-ffa450edef68
2013-05-12 11:23:40 +00:00
Hiram R. Chirino 30038957b6 Add a storeConentAndClear() method to message so that we can lower the memory impact of embedded broker usage.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1481527 13f79535-47bb-0310-9956-ffa450edef68
2013-05-12 11:23:30 +00:00
Hiram R. Chirino 2572e2bd04 Add a little main class to the leveldb store module to make it easier to run a broker from your IDE.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1481014 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 14:03:39 +00:00
Hiram R. Chirino d344aa7a42 Fixes AMQ-4529: leveldb store NPEs when you send to a composite destination.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1481013 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 14:03:31 +00:00
Hiram R. Chirino 2c06326506 Fixes AMQ-4251: Scala compile warnings - Compiling activemq-leveldb-store
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1480811 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 22:17:47 +00:00
Hiram R. Chirino 25356f2695 Fixes issue identified in APLO-245 where index does not seem to get cleaned up / compacted. Fix ported over from Apollo.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1480711 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 16:11:52 +00:00
Hiram R. Chirino d98ac90f28 Suspend/Resume index compactions instead of closing out the index.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1480710 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 16:11:36 +00:00
Hiram R. Chirino 3ef7911a1d Fixes bug identified in APLO-284. Ported fix over from Apollo.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1480709 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 16:11:26 +00:00
Christian Posta 5a299b76eb fix for https://issues.apache.org/jira/browse/AMQ-4519 - MasterLevelDBStore does not shutdown its protocol server when stopped
made fix and added test

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1480060 13f79535-47bb-0310-9956-ffa450edef68
2013-05-07 20:29:56 +00:00
Christian Posta ee97df7422 fixed some small code mistypes in replicated leveldb.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1480059 13f79535-47bb-0310-9956-ffa450edef68
2013-05-07 20:29:45 +00:00
Bosanac Dejan 7450a32ae7 https://issues.apache.org/jira/browse/AMQ-4495 - always get a max batch of messages from the store
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1478823 13f79535-47bb-0310-9956-ffa450edef68
2013-05-03 14:50:56 +00:00
Hiram R. Chirino ede1705e34 The replicated leveldb store will now stash the last known good replica before starting to replicate with a new master. If the replication does not fully synchronize before a slave failure occurs, the store will revert back to the the stashed state.
If a slave connection encounters an error, try to reconnect again after 1 second.

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1478548 13f79535-47bb-0310-9956-ffa450edef68
2013-05-02 21:07:04 +00:00
Hiram R. Chirino e507461dbc Get the replicatedLevelDB element working in the activemq standalone config.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1477709 13f79535-47bb-0310-9956-ffa450edef68
2013-04-30 16:11:52 +00:00
Hiram R. Chirino 1eeae77339 Added an new ElectingLevelDBStore which handles the M/S election bits using ZooKeeper.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1477387 13f79535-47bb-0310-9956-ffa450edef68
2013-04-29 22:33:26 +00:00
Hiram R. Chirino ca814802ca Add more replication tests.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1476786 13f79535-47bb-0310-9956-ffa450edef68
2013-04-28 15:03:24 +00:00
Hiram R. Chirino 5c6b8ba11f Adding an initial spike of a M/S replicated leveldb store. Replication protocol is working nicely, just need automated M/S election bits.
Move tests in a test package.
Rename the HA* classes to DFS*.

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1476719 13f79535-47bb-0310-9956-ffa450edef68
2013-04-28 03:53:57 +00:00
Gary Tully 30b20b3f38 https://issues.apache.org/jira/browse/AMQ-4341 - apply patch from ioannis with thanks
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1450550 13f79535-47bb-0310-9956-ffa450edef68
2013-02-27 00:36:48 +00:00
Hiram R. Chirino b2fca26213 Remove deprecated modules. Converted many AMQ store tests to be LevelDB tests.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1445633 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 14:47:40 +00:00
Hiram R. Chirino 4a8033ed97 Upgrade the leveldbjni version used.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1443114 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 18:27:23 +00:00
Gary Tully 56174e770d [maven-release-plugin] prepare for next development iteration
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1443026 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 15:45:26 +00:00
Gary Tully 5e257808f3 [maven-release-plugin] prepare release activemq-5.8.0
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1443024 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 15:43:41 +00:00
Hiram R. Chirino 960481c7e8 Improve the osgi module so that the leveldb store can be loaded.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1439910 13f79535-47bb-0310-9956-ffa450edef68
2013-01-29 14:36:39 +00:00
Timothy A. Bish a55aae00e7 Scrub the pom files and remove old cruft and leftovers from the recent refactoring
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1439680 13f79535-47bb-0310-9956-ffa450edef68
2013-01-28 22:44:25 +00:00
Bosanac Dejan 2e622a4e15 https://issues.apache.org/jira/browse/AMQ-4034 - initial work on leveldb integration
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1439086 13f79535-47bb-0310-9956-ffa450edef68
2013-01-27 14:03:41 +00:00
Bosanac Dejan f5360a727d resolve leveldb store jmx issues and use the proper mbean naming
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1435163 13f79535-47bb-0310-9956-ffa450edef68
2013-01-18 14:38:01 +00:00
Timothy A. Bish 86afe187c1 fix warnings
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1431699 13f79535-47bb-0310-9956-ffa450edef68
2013-01-10 22:17:24 +00:00
Hiram R. Chirino 1949846c0a Switch to scala 2.10
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1429079 13f79535-47bb-0310-9956-ffa450edef68
2013-01-04 20:02:27 +00:00
Hiram R. Chirino 450b509d58 Add the slf4j-api as a dependency of the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1428535 13f79535-47bb-0310-9956-ffa450edef68
2013-01-03 18:37:04 +00:00
Gary Tully 6335b7e331 fix up imports - resolve warn re permanently hidden defs
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1428409 13f79535-47bb-0310-9956-ffa450edef68
2013-01-03 15:41:43 +00:00
Robert Davies 4724913279 git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1425879 13f79535-47bb-0310-9956-ffa450edef68 2012-12-26 11:03:33 +00:00
Gary Tully d1292441a2 fix leveldb variant of org.apache.activemq.bugs.AMQ2584Test - add rudimentary size calculation to leveldb
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1423854 13f79535-47bb-0310-9956-ffa450edef68
2012-12-19 13:09:26 +00:00
Hiram R. Chirino 4531001b23 Make sure we close the plist db.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1418748 13f79535-47bb-0310-9956-ffa450edef68
2012-12-08 20:11:28 +00:00
Hiram R. Chirino 3ef55fa82b Simplify LevelDB store build, use latest versions of leveldb.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1418698 13f79535-47bb-0310-9956-ffa450edef68
2012-12-08 16:01:55 +00:00
Hiram R. Chirino 32e63dce28 Fixes AMQ-4215: Simplify PList interface and provide a LevelDB store implementation.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1418686 13f79535-47bb-0310-9956-ffa450edef68
2012-12-08 15:21:14 +00:00
Hiram R. Chirino 2751497e38 AMQ-4005 : Also support pluggable storage lockers for the LevelDB store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1411901 13f79535-47bb-0310-9956-ffa450edef68
2012-11-20 22:02:00 +00:00
Hiram R. Chirino aa2a6ddf31 Default the directory setting on the LevelDB store to avoid test failure in AMQ2580Test.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1407589 13f79535-47bb-0310-9956-ffa450edef68
2012-11-09 18:47:38 +00:00
Hiram R. Chirino 1a2cbe7641 Fixup failing LevelDB XA test.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1407018 13f79535-47bb-0310-9956-ffa450edef68
2012-11-08 11:25:49 +00:00
Hiram R. Chirino cd19db289b Moving spring support classes into the activemq-spring module.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1406795 13f79535-47bb-0310-9956-ffa450edef68
2012-11-07 20:41:40 +00:00
Hiram R. Chirino bfbe90b399 rename the activemq-leveldb module to activemq-leveldb-store to be consistent with other store modules. Move more spring/xbean support bits into the activemq-spring module.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1406726 13f79535-47bb-0310-9956-ffa450edef68
2012-11-07 17:26:31 +00:00