Commit Graph

204 Commits

Author SHA1 Message Date
Hiram Chirino 4e3499e41b Implements https://issues.apache.org/jira/browse/AMQ-5458 - Lets make sure we register the Test mbean when in slave mode. 2014-11-26 10:57:13 -05:00
Hiram Chirino f4149c0a46 Implements https://issues.apache.org/jira/browse/AMQ-5458 - Also needed to update the slave since it does writes slightly differently. 2014-11-26 08:36:43 -05:00
Hiram Chirino ebafd5c193 Implements https://issues.apache.org/jira/browse/AMQ-5458 2014-11-25 11:41:00 -05:00
gtully 350889c1eb https://issues.apache.org/jira/browse/AMQ-5266 - fix regression in levedlb xa recovery and durable sub scan - org.apache.activemq.broker.mLevelDBXARecoveryBrokerTest and org.apache.activemq.bugs.AMQ2149LevelDBTest 2014-10-24 14:28:00 +01:00
gtully 3042797b41 https://issues.apache.org/jira/browse/AMQ-5266 tidy up leveldb impl with additional scenario tests 2014-10-23 16:29:57 +01:00
gtully 1a0bd45a4e https://issues.apache.org/jira/browse/AMQ-4485 - leveldb version - shows the need to flag async adds so they can get suppressed as duplicates \(and not go to the dlq\) when the cursor order is out of sync on cache full 2014-10-17 13:35:31 +01:00
gtully 51a2596c52 run leveldb tests on osx 2014-10-16 23:53:44 +01:00
gtully 8a37f97315 AMQ4677Test.testSendAndReceiveAllMessages - demoed the lack of reference increment for transacted send and the non completion of transacted futures in leveldb 2014-08-30 23:36:05 +01:00
gtully 54e2e3bef2 rework cursor store sync w.r.t to index order. resolve issues with skipped dispatch and duplicate dispatch. https://issues.apache.org/jira/browse/AMQ-4485 https://issues.apache.org/jira/browse/AMQ-5266 2014-08-30 00:51:21 +01:00
Hiram Chirino b76d8318d7 Support configuring a connectUrl on the leveldb store in case your running in a system like OpenShift where clients connect to a different ip:port from the one that the sever binds. 2014-08-27 13:08:01 -04:00
gtully 66a689df0e fix up karaf-itests , related commits https://issues.apache.org/jira/browse/AMQ-5224 https://issues.apache.org/jira/browse/AMQ-5105 Update poms to fully load into M2E - 06bb7a2716 2014-07-28 15:30:12 +01:00
Dejan Bosanac 3c69127524 https://issues.apache.org/jira/browse/AMQ-4349 - stopping leveldb throws NPE 2014-07-18 15:12:02 +02:00
Dejan Bosanac 4da588d4fc https://issues.apache.org/jira/browse/AMQ-5105 - problem starting broker with leveldb store 2014-07-17 17:12:52 +02:00
Daniel Kulp 06bb7a2716 Update poms to fully load into M2E 2014-07-10 13:47:55 -04:00
Kevin Earls 13581c9aac Exclude leveldb tests on AIX, Solaris, HP-UX, and Windows as they cause problems with CI 2014-07-10 11:10:11 +02:00
Timothy Bish e62e90abaf https://issues.apache.org/jira/browse/AMQ-5125
Fix for potential deadlock when external classes synchronize on the
LevelDBStore instance which can deadlock the hawtDispatch runner thread
if a task also attempts to take the lock to protect some mutable state
values.
2014-07-07 17:53:46 -04:00
Timothy Bish 74846bb2b4 https://issues.apache.org/jira/browse/AMQ-3758
Refactor the scheduler store into a more KahaDB style store that can
recover from various problems like missing journal files or corruption
as well as rebuild its index when needed.  Move the scheduler store into
a more configurable style that allows for users to plug in their own
implementations.  Store update from legacy versions is automatic.
2014-07-07 12:28:11 -04:00
Kevin Earls bbd61084e0 Change logging from TRACE to WARN to save space on CI nodes 2014-06-24 11:21:34 +02:00
Dejan Bosanac fa086396a7 [maven-release-plugin] prepare for next development iteration 2014-06-05 11:59:29 +02:00
Dejan Bosanac 58dd93e42e [maven-release-plugin] prepare release activemq-5.10.0 2014-06-05 11:59:17 +02:00
Dejan Bosanac c2cf78542f [maven-release-plugin] prepare for next development iteration 2014-06-04 11:33:17 +02:00
Dejan Bosanac 5bc741f227 [maven-release-plugin] prepare release activemq-5.10.0 2014-06-04 11:33:03 +02:00
Hiram Chirino bdb24ee7c7 Fixes AMQ-5176: Support building ActiveMQ using Java 8 JDK 2014-05-07 10:53:51 -04:00
gtully ad1f751a41 https://issues.apache.org/jira/browse/AMQ-5077 - reduce reader thread work when client uses async send; async store updates can now queue up to the destination memory limit b/c they don't block the send thread. Pending store writes are now tracked in memory usage. This allows a client to quickly provide a burst of messages to fill the destination cache bounded only by network bandwidth 2014-04-30 16:13:18 +01:00
gtully 266d23ef79 revisit https://issues.apache.org/jira/browse/AMQ-3519 with more durable solution https://issues.apache.org/jira/browse/AMQ-5068 - JMSRedelivered header now persisted before dispatch so that it is a reliable indication of a possible duplicate delivery. The option is enabled via destination policy persistJMSRedelivered 2014-03-26 11:16:42 +00:00
Hiram Chirino db321727c9 Fixes AMQ-5115: LevelDB sync=true is not being honored. 2014-03-21 12:54:55 -04:00
gtully cb6941ee0b https://issues.apache.org/jira/browse/AMQ-4886 - think the sync on shutdown issue is resolved and the test will no longer hang 2014-03-20 14:14:31 +00:00
Kevin Earls 4ca5519e9b Reduced timeouts from 60 to 10 minutes to avoid long CI hangs 2014-02-20 14:21:02 +01:00
Daniel Kulp 43a88a9367 Get mvn eclipse:eclipse work for the leveldb stuff 2014-02-05 11:41:22 -05:00
Hadrian Zbarcea d36e3c0e95 AMQ-5005. Fix for scala plugin 2014-01-30 15:43:45 -05:00
Kevin Earls c027552035 Upgraded surefire plugin to 2.16. AMQ-5006 2014-01-30 14:22:44 +01:00
gtully 0f0c0d676a fix leveldb version of org.apache.activemq.bugs.AMQ2870Test#testSize 2014-01-07 16:54:18 +00:00
Hiram Chirino 30f3a9b7f3 Default flushDelay on the replicated leveldb store did not have the correct default. 2013-12-05 13:38:52 -05:00
Hiram Chirino ed8e4eae8f Fixes https://issues.apache.org/jira/browse/AMQ-4923: Replicated LevelDB: Loss of broker Quorum fails to fully stop the master 2013-12-05 13:38:52 -05:00
Hiram Chirino 3b5fa4bf89 Implementing a dirty stop method for the leveldb store that might come in handy for testing. 2013-12-05 09:24:10 -05:00
Hiram Chirino 8378cb1ffc Fixing https://issues.apache.org/jira/browse/AMQ-4917 : LevelDB store can fail when using durable subs.
We were browsing durable sub entries which had been concurrently GCed causing leveldb store failures which then caused the broker to restart.
2013-12-04 12:09:47 -05:00
Hiram Chirino 5fa462a08a Adding assertions to make sure that we only append to the log from the write thread. Found a code path that was appending to the log from a different thread. This might have been affecting https://issues.apache.org/jira/browse/AMQ-4882 2013-12-03 12:05:16 -05:00
Hiram Chirino 61e13fa6ce Fixing regression caused by changes to SuppressRelyException in commit b0e91d47f5. 2013-12-02 12:29:21 -05:00
Hiram Chirino b0e91d47f5 Have the leveldb store thorw SuppressReplyExceptions instead of IOExceptions so that the clients retry try the operations instead of giving up. Also retry the problemantic getMessage() call which seems to fail at times. 2013-11-25 13:17:58 -05:00
Claus Ibsen 00cb9a5668 AMQ-4858: Moved scala source file from java to scala dir. Thanks to Jaromir Hamala for patch. 2013-11-22 11:25:08 +01:00
Claus Ibsen c0090f6f0c AMQ-4885: Renamed some API methods in MessageStore due typos. 2013-11-22 11:21:42 +01:00
Timothy Bish 38ab4b10a4 https://issues.apache.org/jira/browse/AMQ-4855
Deprecate the misspelled subcription named getters and use the correctly
spelled versions where the older ones are in use.
2013-11-11 11:40:07 -05:00
Hiram Chirino 42e1c463d4 Replicated leveldb slaves index snapshots were being labeled with higher journal positions than what they really contained. 2013-11-08 10:34:21 -05:00
Hiram Chirino 4367ec1b82 If a replicated leveldb slave's connection gets slow, lets merge together journal write events to avoid them queuing up on the master side. 2013-11-05 11:53:37 -05:00
Hiram Chirino a907fc9e94 leveldb store: Add more log traces for when we need to get more details on what's going on. 2013-11-04 09:26:01 -05:00
Hiram Chirino b1d8cbe4cd Fixes for leveldb replication: make sure we only apply index updates when we encounter a UOW_END_RECORD so that we don't end up with an inconsistent index if a partial UOW is replicated. 2013-11-04 09:26:01 -05:00
Hiram Chirino dea38e62f8 leveldb replication: Lets always download the current append log just to be safe 2013-11-04 09:26:01 -05:00
Hiram Chirino 2824a94af4 Continue to append to the last leveldb log file on a store restart. 2013-11-04 09:26:01 -05:00
Hiram Chirino 190da29691 Fix for: https://issues.apache.org/jira/browse/AMQ-4837 : LevelDB corrupted in AMQ cluster.
- The tracked flushed offset was not getting initialized to match the initial size of the log file.  This invalid flushed offset was being used to do the initial sync /w the slave and then things went downhill.
2013-11-01 12:13:33 -04:00
Hiram Chirino 24d5490e57 Fix for: https://issues.apache.org/jira/browse/AMQ-4837 : LevelDB corrupted in AMQ cluster.
- Log rotation was causing a pre-mature index snapshot to be taken on the slave (snapshot while the slave was still synchronizing).
- Also fix issue with the append position displayed in JMX for the master not being correct.
2013-10-31 12:52:50 -04:00
Hiram Chirino 3ed52ef8a2 Working on a test case for https://issues.apache.org/jira/browse/AMQ-4837 : LevelDB corrupted in AMQ cluster. 2013-10-30 15:44:06 -04:00
Hiram Chirino 17415ceecd Only register the leveldb store MBeans in JMX if jmx is enabled on the broker. 2013-10-30 15:34:29 -04:00
Hiram Chirino aaa1a74dfb Add a toString() method to the replicated leveldb store so that it displays a nicer message when the broker starts it up. 2013-10-30 12:22:28 -04:00
Hiram Chirino 1896d27409 Update to the latest leveldbjni release. 2013-10-17 09:30:07 -04:00
Hiram Chirino ef45a5f8ff Avoid NullPointerExceptions that can occur during leveldb replication M/S state transitions. 2013-10-15 12:43:00 -04:00
gtully 864aa6798f [maven-release-plugin] prepare for next development iteration 2013-10-14 23:38:35 +01:00
gtully 87c9bbebac [maven-release-plugin] prepare release activemq-5.9.0 2013-10-14 23:38:22 +01:00
Hiram Chirino d44c8968c4 Expose a JMX attribute to show when the last leveldb replication log entry was recorded.
This can be used to get a time estimate of far behind a master a slave is.
2013-10-14 08:45:54 -04:00
gtully ee0933c71e [maven-release-plugin] prepare for next development iteration 2013-10-11 14:51:36 +01:00
gtully b276006911 [maven-release-plugin] prepare release activemq-5.9.0 2013-10-11 14:51:24 +01:00
Claus Ibsen 59af7479b2 Polished the logging 2013-10-11 11:20:22 +02:00
Hiram Chirino 0214bfbfcd Support weighting the leveldb store replication nodes so that some have a better chance of becoming the master than others. 2013-10-10 15:42:02 -04:00
Hiram Chirino e90ce1aabb Don't display leveldb replication recovery progress on slave nodes. 2013-10-10 14:52:07 -04:00
gtully 4f108cead5 fix up missing license headers - keep mr. rat happy :-) 2013-10-10 11:41:58 +01:00
Hiram Chirino d609d2ae38 Make sure the leveldb append log file is flushed to disk before it's closed to avoid data loss. 2013-10-09 11:49:11 -04:00
Hiram Chirino 5e63ddd337 Fixing bug which caused replicated leveldb nodes to not recover from a ZooKeeper failure. 2013-10-09 10:56:34 -04:00
Hiram Chirino 119fdab1d0 Fixes leveldb replication bug that can cause read errors: make sure that log is flushed for records that are being read. 2013-10-09 09:15:48 -04:00
Hiram Chirino a5ac1a3630 Avoid logging ZooKeeper related exceptions during a replicated leveldb store shutdown. 2013-10-03 10:42:45 -04:00
Hiram Chirino b720b264e3 The the hostname on the replicated leveldb store test so that the test works better on machines with weird DNS settings. 2013-10-02 12:15:36 -04:00
Hiram Chirino 0b66abcbfb Fixing leveldb replication bug where loosing quorum did not properly restart the master in a clean way. 2013-10-02 12:15:36 -04:00
Hiram Chirino 8804341c19 Adding more debug info to help trouble shoot election issues with leveldb replication elections. 2013-10-02 10:45:58 -04:00
Hiram Chirino f75520fc8b Implementing AMQ-4744: Support using LevelDB as a nested store in mKahaDB 2013-09-27 09:19:48 -04:00
gtully efaa351db7 https://issues.apache.org/jira/browse/AMQ-4365 - allow lease locker to be used by kahadb - remove deps on jdbc pa. LockableService now passes a reference to a locker so it can pull the brokerService, extracted some of the jdbc lock common stuff, additional test kahadb with jdbc lease 2013-09-20 15:33:24 +01:00
Hiram Chirino d771ebb97e Fixes bug in replicated leveldb where log files on slaves were not getting GCed. 2013-09-16 11:58:03 -04:00
Hiram Chirino 1eca031356 leveldb replication Master was failing to give up being master after it's process is suspended by using ctrl-z. 2013-09-03 10:41:33 -04:00
Hiram R. Chirino f4d51e092b Adding a LevelDB version of the RedeliveryRestartTest. Implemented redelivery tracking in the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1518289 13f79535-47bb-0310-9956-ffa450edef68
2013-08-28 17:20:25 +00:00
Hiram R. Chirino 45e1462609 Persist the latest producer position across restarts in the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1515799 13f79535-47bb-0310-9956-ffa450edef68
2013-08-20 12:38:32 +00:00
Gary Tully 7c50c1c736 fix up failure - still leveldb variant problem that needs work - testQueueTransactionalOrderWithRestart - org.apache.activemq.bugs.AMQ2149LevelDBTest
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1512332 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 15:11:11 +00:00
Hiram R. Chirino 563cc1bc6a Warn when there are too many replication nodes attached.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1506965 13f79535-47bb-0310-9956-ffa450edef68
2013-07-25 13:40:59 +00:00
Timothy A. Bish 194c6535cd [LevelDB]
https://issues.apache.org/jira/browse/AMQ-4296

Fixes remainder of failing unit tests.  The LevelDB wasn't incrementing or decrementing reference counts on messages added to the store which causes the expectations of certain memory limit based tests to fail as the memory usage was being updates after the store add instead of during so a message could get placed into the batch list of a cursor when we did not expect that it would.  This could also cause a browse to return fewer message than we want as the in memory messages would top out the usage limit so we'd never page in one batch of messages. 

git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1505805 13f79535-47bb-0310-9956-ffa450edef68
2013-07-22 20:53:27 +00:00
Timothy A. Bish 11ed6b0e18 {LevelDB] fix for StoreQueueCursorLevelDBNoDuplicateTest failure. When the cursor calls setBatch and then recovering the next batch its expecting the batch to start at lastMsgId + 1 so we were seeing a single duplicate message and since audit was disabled it was delivered.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1504668 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 21:21:38 +00:00
Hiram R. Chirino ae5abf143b Adding a container property to the zk node data used by the leveldb replication.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1504626 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 20:04:45 +00:00
Hiram R. Chirino 86e2426d1b Improve the replicated leveldb bits: Avoid dependencies on fabric-group stuff. Makes it easier to embed in different versions of a fabric osgi env.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1501877 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 17:49:03 +00:00
Timothy A. Bish 5f0dd8ab53 Fix for failing LveelDB unit tests where only non-persistent messages are sent in a TX. The preCommit wasn't being run so the Queue's orderIndexUpdates structure wasn't getting updated with the TX to process in the postCommit phase.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1501420 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 18:15:26 +00:00
Hiram R. Chirino 2019a21d96 Simplify and improve the leveldb replication MBean
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1499754 13f79535-47bb-0310-9956-ffa450edef68
2013-07-04 13:47:27 +00:00
Hiram R. Chirino 34f3329370 When the leveldb replicated master was shutting down the client would get notified of a failure and it would not be hidden from the client app. We now suppress sending failure messages to clients when a broker is shutting down so that the client failover logic can kick in an reconnect the client to another server gracefully.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1498978 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 15:28:42 +00:00
Hiram R. Chirino b98067732b Improve the replicated leveldb behavior when the number of nodes in the cluster falls below the required minimum. The master node will switch to electing mode. The master store startup will now also block until it syncs up with slaves so that we don't accept connections the master is fully online.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1498601 13f79535-47bb-0310-9956-ffa450edef68
2013-07-01 17:38:13 +00:00
Hiram R. Chirino 16cd8c3954 Expose the replicated store status via JMX.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1497843 13f79535-47bb-0310-9956-ffa450edef68
2013-06-28 16:57:29 +00:00
Hiram R. Chirino baf8c499b3 Fixes AMQ-4599: Only package the pure java LevelDB driver by default
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1496514 13f79535-47bb-0310-9956-ffa450edef68
2013-06-25 15:12:23 +00:00
Bosanac Dejan 06dae40d94 https://issues.apache.org/jira/browse/AMQ-4593 - upgrade snappy java
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1495392 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 11:31:23 +00:00
Hiram R. Chirino 7ef79f2d2a Display who the slave leveldb store thinks is the master.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1494218 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 17:25:09 +00:00
Timothy A. Bish 6ee4c66fc3 m2e updates
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1492832 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 19:46:00 +00:00
Hiram R. Chirino 7db8987ca6 Fixup the assembly packaging so that the replicated leveldb store can be used.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1491287 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 20:41:24 +00:00
Hiram R. Chirino 5f301c40fa Help the xbean generate the proper xsd for the leveldb store.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1491278 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 20:16:22 +00:00
Hiram R. Chirino c161033abb Make the fabric dependencies of the leveldb store needed for replication as provided.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1490271 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 13:20:28 +00:00
Hiram R. Chirino 50e8795464 Fixes a ghost messages issue where the queue cursor goes out of sync /w the leveldb store when transactions are being used.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1490065 13f79535-47bb-0310-9956-ffa450edef68
2013-06-05 23:01:13 +00:00
Hiram R. Chirino bb30abf774 Try to avoid the 'short record at position' LevelDB error reported at: http://activemq.2283324.n4.nabble.com/Activemq-5-9-leveldb-replication-issue-tp4667495p4667674.html
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1489019 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 14:55:57 +00:00
Hiram R. Chirino ec9975c36e Additional fixes related to AMQ-4563: You can now configure the storeOpenWireVersion property of a broker to control which version of openwire is used by the persistence stores. This needs to be set to version 10 to preserve the original AMQP message ids.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1488375 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 19:56:03 +00:00
Hiram R. Chirino 300a26d350 No need for the uber jar anymore since we are part of the distro.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1486243 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 22:33:38 +00:00
Hiram R. Chirino b41121e359 This should fixed the problem with the delayed leveldb index updates.
git-svn-id: https://svn.apache.org/repos/asf/activemq/trunk@1486242 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 22:33:32 +00:00