Commit Graph

610 Commits

Author SHA1 Message Date
Christopher L. Shannon 39a20bd788 https://issues.apache.org/jira/browse/AMQ-6065
Fixed test case so that a System.exit call will not take down the
JVM and cause the test suite to abort in Jenkins early
2015-11-29 10:05:27 -05:00
Christopher L. Shannon 673c2928a1 https://issues.apache.org/jira/browse/AMQ-6066
Improving performance of OrderedPendingList contains method

Thanks for David Sitsky for the patch
2015-11-28 13:25:53 -05:00
gtully 2aa172f905 https://issues.apache.org/jira/browse/AMQ-6065 - allow ioexception handler to flip the systemExitOnShutdown broker attribute before calling stop, i.e: exit on store io errors 2015-11-27 14:21:20 +00:00
gtully 0a12bcb928 https://issues.apache.org/jira/browse/AMQ-2191 https://issues.apache.org/jira/browse/AMQ-3529 - rework fixes to remove uncertanty from dealing with intettuptedexception. Sync requests will trap interrupts that ocurr while waiting for responses and fail the connection with an interruptedioexception. Interrupts pending before requests will be suppressed, allowing possible clean shutdown. It is not safe to replay openwire ops b/c they are not idempotent, the only safe option is to have a teardown of the broker side state from a close 2015-11-27 12:21:56 +00:00
Christopher L. Shannon (cshannon) 29d943429c https://issues.apache.org/jira/browse/AMQ-5393
Reverting commit because allowing 0 doesn't actually set the usage to
0 size.  It ends up disabling the check all together which was the
opposite intention of the commit.

This reverts commit 3b658f8e1a.
2015-11-25 17:59:41 +00:00
Christopher L. Shannon (cshannon) 35b7ac250b https://issues.apache.org/jira/browse/AMQ-5898
Removing assertion in VirtualDestinationInterceptor to allow
multiple composite destinations to forward to a physical destination
2015-11-25 13:33:32 +00:00
Christopher L. Shannon (cshannon) 3b658f8e1a https://issues.apache.org/jira/browse/AMQ-5393
Allowing disk limit percentages to be 0
2015-11-25 13:18:59 +00:00
Christopher L. Shannon (cshannon) bc9edf00d1 https://issues.apache.org/jira/browse/AMQ-6058
Updating patch to make sure SelectorAwareVirtualTopics are covered
and code cleanup
2015-11-23 20:35:32 +00:00
Chad Zobrisky dca6e4506e AMQ-6058 added a check to make sure the destination starts with the virtual topic prefix. 2015-11-23 20:10:26 +00:00
gtully 4318eba4a3 https://issues.apache.org/jira/browse/AMQ-6014 - topic store prefetch should take ownership like in the non persistent case - DurableSubscriberNonPersistentMessageTest regression 2015-11-23 12:27:03 +00:00
Christopher L. Shannon (cshannon) cc6213ebf2 https://issues.apache.org/jira/browse/AMQ-5712
Switching addMessageLast to tryAddMessageLast when messages are added
to a Queue pending cursor to allow a potential deadlock to be
avoided. There is more work to be done here but this will at least
prevent a deadlock from occurring.

Fix and test based off of a patch created by Timothy Bish.
2015-11-20 20:58:27 +00:00
gtully 15412bae64 AMQ4221Test - remove system exit as it breaks surefire fork, https://issues.apache.org/jira/browse/AMQ-6004 interrupt causes errors on shutdown. Two are sorted but the check for errors in the logs may still cause intermittent failures 2015-11-18 16:08:29 +00:00
Christopher L. Shannon (cshannon) 09054fc4a8 https://issues.apache.org/jira/browse/AMQ-6050
Clearing the subscription from the local map in
DemandForwardingBridgeSupport to make sure that demand can be properly
recreated again.
2015-11-18 15:06:33 +00:00
Timothy Bish 078a101cf7 https://issues.apache.org/jira/browse/AMQ-6000
Fix and test, wakeup queue dispatch when resume is called.
2015-11-10 15:39:51 -05:00
Timothy Bish 938aa626c2 https://issues.apache.org/jira/browse/AMQ-6039
If the rename option fails, then we must try a complete copy since the
move can cross file systems.
2015-11-10 11:12:50 -05:00
Christopher L. Shannon (cshannon) cc81680e10 https://issues.apache.org/jira/browse/AMQ-6027
Adding support for consumers on virtual destinations to create network
demand. This behavior is turned off by default but can be enabled.

For example, if a consumer comes online for a queue that is part of a
VirtualTopic, this will cause a network of brokers to forward messages
because a demand subscription will be created. Same for if a consumer
comes online for a forwarded destination from a composite
destination.

There is also an option to enable flow based on the existence of a
virtual destination if the virtual destination is forwarding to a
Queue.

Full configuration instructions for this feature will be on the wiki page.
2015-11-09 20:07:43 +00:00
Timothy Bish 014d5594d6 AMQ-6032 Repack message prior to scheduler dispatch to store updated
properties in marshaled form.
2015-10-30 14:11:06 -04:00
Christopher L. Shannon (cshannon) c5506f5b17 Switching getter method to public
Switching isReduceMethodFootprint() to public access so it can be
accessed.
2015-10-30 15:39:24 +00:00
gtully 5d697cff3b https://issues.apache.org/jira/browse/AMQ-6016 - rework fix for https://issues.apache.org/jira/browse/AMQ-2106 - account group assignment on a per destination basis to prevent modification during consumer ordering 2015-10-21 14:02:56 +01:00
gtully a01578ad4c fix stop/stop or stop no start NPE on sharedfilelocker 2015-10-20 13:02:57 +01:00
gtully dc19d28af9 https://issues.apache.org/jira/browse/AMQ-3573 https://issues.apache.org/jira/browse/AMQ-2834 - only validate limits if they are set 2015-10-20 12:17:23 +01:00
gtully 1ad0117932 https://issues.apache.org/jira/browse/AMQ-6014 - fix up reference counting for durable subs such that ack on one sub does not decrement usage for offline subs. Fix and test 2015-10-19 16:22:57 +01:00
gtully b285d10188 https://issues.apache.org/jira/browse/AMQ-4705 - ensure jvm lock system property is cleared in the event of lock release throwing exception. 2015-10-14 15:08:59 +01:00
Christopher L. Shannon (cshannon) af09b4586b Improving disk usage log message
Improving the warning message for a percent disk usage check so it is
more clear what the original limit was set to and what the current
limit is being set to.
2015-10-14 11:35:18 +00:00
Christopher L. Shannon (cshannon) 6fef9eea65 Cleaning up logging
Fixing one of the logging statements for the disk usage check process
that warns when the amount of free space is too low
2015-10-07 11:55:20 +00:00
Timothy Bish 81b4b9ae3d https://issues.apache.org/jira/browse/AMQ-5996
When possible check for noLocl changes on durable subscription
reactivation and recreate the durable sub if it changes.  For both
selector change and noLocal change also update the AbstractSubscription
selectorExpression so it matches with what was requested.
2015-10-01 19:23:58 -04:00
gtully 86c826c461 https://issues.apache.org/jira/browse/AMQ-5568 - verify delete return code for win platform failure. Thanks to Erik Wramner for the heads up 2015-09-22 12:56:13 +01:00
Christopher L. Shannon (cshannon) 00d19e7b66 Removing inadvertantly checked in file 2015-09-21 17:13:36 +00:00
gtully cc9b9b0843 https://issues.apache.org/jira/browse/AMQ-5938 - remove unnecessary topic cast - breaks auth scenario. Fix and test 2015-09-21 14:48:44 +01:00
Christopher L. Shannon (cshannon) d2bd152960 Tweaking some unit tests to improve reliability 2015-09-17 11:28:14 +00:00
Dejan Bosanac ee4672baaf https://issues.apache.org/jira/browse/AMQ-5972 - wildcard policy entries not applied in correct order 2015-09-17 11:08:21 +02:00
Christopher L. Shannon (cshannon) 4cddd2c015 https://issues.apache.org/jira/browse/AMQ-5963
Disk limits can now be specified as a percentage of the partition size
that the store is located on.  The usual checks of the max size
exceeding the available space will still apply.  When using a
percentage, the store can always regrow itself if space becomes
available.

https://issues.apache.org/jira/browse/AMQ-5964
https://issues.apache.org/jira/browse/AMQ-5965
https://issues.apache.org/jira/browse/AMQ-5969
2015-09-15 14:50:18 +00:00
Christopher L. Shannon (cshannon) 4f8d56aaf6 https://issues.apache.org/jira/browse/AMQ-5967
Updating temporary limit disk check to take into account space being
used by the temporary store itself
2015-09-14 19:46:57 +00:00
Christopher L. Shannon (cshannon) 734fb7dda3 https://issues.apache.org/jira/browse/AMQ-5923
Adding metrics to track the pending message size for a queue and for
subscribers.  This is useful so that not only the pending count is
known but also the total message size left to consume. Also improving
the message size store tests as well.
2015-09-09 18:12:15 +00:00
Timothy Bish f58683ea07 https://issues.apache.org/jira/browse/AMQ-5959
Add check to avoid NPE
2015-09-08 10:03:23 -04:00
gtully c7b93d1232 https://issues.apache.org/jira/browse/AMQ-5953 - transaction broker, fix sync of xatransaction get with insert 2015-09-03 16:56:20 +01:00
Timothy Bish 505916b927 AMQ-5949 Treat maxSendRetries of -1 as infinite. 2015-09-02 16:37:24 -04:00
Dejan Bosanac ee54f09303 https://issues.apache.org/jira/browse/AMQ-5831 - revisit topic subscriptions 2015-08-26 12:28:39 +02:00
gtully 10a74ce737 https://issues.apache.org/jira/browse/AMQ-5943 - patch, with test, applied with thanks to Torsten Mielke 2015-08-25 12:07:02 +01:00
gtully 11579bb918 https://issues.apache.org/jira/browse/AMQ-5938 - add remove(messageId) op to offline durable subs jmx view 2015-08-21 13:17:09 +01:00
gtully 3c2ed9352b AMQ-5933 - add test and fix 2015-08-19 15:02:41 +01:00
gtully ffdaeb2bd1 AMQ-5920 - make using a vt transaction configurable, a transaction negates concurrentstoreanddispatch and imposes local 2pc on mKahadb so needs to be off by default 2015-08-17 15:35:36 +01:00
Christopher L. Shannon (cshannon) 3c358b32fd AMQ-5889 - refactoring and cleanup 2015-08-13 15:49:50 +00:00
Christopher L. Shannon (cshannon) e14aca871c https://issues.apache.org/jira/browse/AMQ-5889
-Cleaned up missing license headers and refactored packages.
-Added configuration options for a protocol detection timeout and
for the max number of connections accepted at the same time.
-Fixed a regression with connection counts
-Also added some more tests
2015-08-13 14:41:29 +00:00
Christopher L. Shannon (cshannon) 04ee70a161 https://issues.apache.org/jira/browse/AMQ-5889
Adding support for auto detection of wire protocols over a transport.
OpenWire, AMQP, STOMP, and MQTT can all be detected and the broker
will properly handle each one over a given Transport.  Currently
auto TCP, NIO, SSL, and NIO+SSL transports can handle auto-detection
of the wire format and client but support could be added in the
future for other transports like websockets.
2015-08-11 19:39:29 +00:00
Timothy Bish 6cdd750ddc [maven-release-plugin] prepare for next development iteration 2015-08-10 11:04:26 -04:00
Timothy Bish a9eeb03520 [maven-release-plugin] prepare release activemq-5.12.0 2015-08-10 11:04:17 -04:00
Christopher L. Shannon c6542a921b AMQ-5748 - Cleanup and clearing cache on shutdown 2015-08-09 09:39:14 -04:00
Christopher L. Shannon (cshannon) a49d46e3ca https://issues.apache.org/jira/browse/AMQ-5748
Updating MemoryTopicMessageStore to decrement store statistics on cache
eviction.  Updating KahaDBMessageStoreSizeStatTest to account for the
fact that a LRU cache is used so the last 100 messages are kept in
memroy.
2015-08-08 17:59:19 +00:00
Timothy Bish a3c8bee1f0 https://issues.apache.org/jira/browse/AMQ-5605
Attempt to resolve high CPU usage after error by ensuring that the
transport is disposed along with the fire of the local error.
2015-08-07 16:21:52 -04:00
Christopher L. Shannon (cshannon) 2b7bb6f81b https://issues.apache.org/jira/browse/AMQ-5837
Switching to a List to track dispatched messages in a TopicSubscription
to be consistent with a PrefetchSubscription and to prevent an error
in case acks come back out of order.
2015-08-06 17:14:09 +00:00
gtully 340728f2d1 AMQ-5920 - use implicit broker transaction for virtual topic fanout and add concurrentSend=true option to use an executor to fanout. Combination gives a 3x reduction in fanout roundtrip for small persistent messages to 100 consumer queues 2015-08-06 11:56:47 +01:00
Christopher L. Shannon (cshannon) 457dbd8b64 https://issues.apache.org/jira/browse/AMQ-5919
Converting printStackTrace statements to slf4j so that exceptions
are logged properly
2015-08-05 16:29:29 +00:00
Timothy Bish 82e3ce805e https://issues.apache.org/jira/browse/AMQ-5916
Apply patch from Ganesh Murthy to resolve issue of creation of durable
subscriber fails via broker AdminView due to lack of SecurityContext on
ConnectionContext
2015-08-04 14:51:15 -04:00
Timothy Bish 5e05df1cb7 Fix a bunch of javadoc warnings produced during release profile builds. 2015-08-04 11:54:46 -04:00
Christopher L. Shannon (cshannon) 5684d093c0 https://issues.apache.org/jira/browse/AMQ-5875
Reworking DurableConduitBridge to look up subscriptions from the
TopicRegion  instead of the RegionBroker since inactive subscriptions
need to be looked at as well when dynamicOnly is false for a
network bridge.
2015-07-29 12:10:05 +01:00
gtully a439a0c6bf AMQ-5875 - rework the patch a bit to only call createDemandSubscription when we have a match to avoid trcking state in error via the bridge subscriptionMap* from configureDemandSubscription 2015-07-17 16:56:22 +01:00
Christopher L. Shannon (cshannon) 2117768e0a https://issues.apache.org/jira/browse/AMQ-5875
Fixing a regression that caused a network bridge to recreate durable
demand improperly.
2015-07-17 16:34:06 +01:00
gtully b1ea29ed14 AMQ-5892 - remove default stack trace from service warn to debug level on async error; makes it consistent with transport logging 2015-07-16 15:18:23 +01:00
gtully 2b5b890db9 AMQ-5888 add patch from GMURTHY with thanks - fix and test 2015-07-16 11:53:12 +01:00
Christopher L. Shannon (cshannon) f10aab6428 https://issues.apache.org/jira/browse/AMQ-5851
resolves #123

This commit resolves an issue where unmatched acknowledgement
messages could be received when running a MDB consumer and
sending messages with a short TTL.  The expiration logic when
receiveing an expired Message Ack will now only expire messages
in dispatch relating to the received ack, not all expired messages
in the dispatch list.
2015-07-09 16:46:21 +01:00
gtully a05a9a2129 https://issues.apache.org/jira/browse/AMQ-5552 - added profile for activemq.tests=smoke to run a fast subset of tests that gives a fast verification of a full build 2015-07-09 14:10:38 +01:00
Timothy Bish 13044decce https://issues.apache.org/jira/browse/AMQ-5848
Use the latest openwire version marshallers in the KahaDB store when
starting from a clean install, drop back to the version used in the
existing store if one is found.
2015-07-08 17:29:32 -04:00
Christopher L. Shannon bd28c3b0ba https://issues.apache.org/jira/browse/AMQ-5748
Fixing a potential Null pointer exception in  MemoryMessageStore
2015-07-07 18:54:06 -04:00
Christopher L. Shannon (cshannon) 46055034c9 https://issues.apache.org/jira/browse/AMQ-5837
This commit adds an Inflight message size statistic to SubscriptionStatistics
so we can know the size of all the messages that are inflight, besides just the count.
2015-07-07 16:33:58 -04:00
Christopher L. Shannon (cshannon) 785b16bf9e https://issues.apache.org/jira/browse/AMQ-5748
Added a getMessageSize method to MessageStore to support retrieving the
total message size of all stored messages for a destination.  Added a
new storeMessageSize statistic to DestinationStatistics.
2015-07-07 18:03:27 +00:00
Timothy Bish 7a68ad5d98 https://issues.apache.org/jira/browse/AMQ-4814
Remove of a durable subscription when the keepDurableSubActive flag is
true (default) did not actually stop and cleanup the still active cursor
due to the fact that the cursor checks the flag of its subscription to
see if it should stay active when its stop method is called.  The
subscription needs to disable that flag when the subscription is being
removed so that the cursor will correctly shutdown.
2015-07-06 18:21:20 -04:00
Timothy Bish c38a61d7ae https://issues.apache.org/jira/browse/AMQ-5340
Clean up a bit, extend test timeout to account for slow CI machines,
remove System prints and replace with LOG.
2015-07-06 16:47:58 -04:00
Christopher L. Shannon 174dcbff2a https://issues.apache.org/jira/browse/AMQ-5340
A QueueBrowser no longer checks expiry on messages to prevent a
browser from hanging in between the hasMoreElements check and actually
getting a message.  This means that if messages were in the queue when
the browser started it will receive messages even if they are now
expired. Even though the browser will get the expired message, the
broker will still expire it to prevent future access to it.

Thanks to Henno Vermeulen for providing a test case.
2015-07-06 16:13:52 -04:00
Timothy Bish e4af2eb635 https://issues.apache.org/jira/browse/AMQ-5814
Use the context of the subscription when the producer adds a destination
that matches its wildcard.
2015-07-02 11:36:53 -04:00
Christopher L. Shannon (cshannon) 12b26b641b https://issues.apache.org/jira/browse/AMQ-5745
This commit is based off a patch submitted by admin@int3solutions.com.
It fixes several locations in the code where there are potential resource leaks.
2015-06-30 13:41:48 -04:00
Timothy Bish 1b493749a1 https://issues.apache.org/jira/browse/AMQ-5859
fix for incorrect logic in reconnect attempt limits and also fix a case
where an NPE could occur (saw in the tests).   Based on patch provided
by Benoit Wiart
2015-06-30 13:00:04 -04:00
gtully 002ade79b0 https://issues.apache.org/jira/browse/AMQ-5639 - the duplex case needed work. All advisories were being acked async in duplex mode, that code needed to be more selective to forward advisories that dont terminate at the bridge. Fix and test 2015-06-26 14:54:29 +01:00
Dejan Bosanac 13c471cc11 https://issues.apache.org/jira/browse/AMQ-5656 - add support for selective mbean suppression; refactor a bit so we can support patterns and wildcards 2015-06-26 12:56:37 +02:00
gtully b6fea83126 https://issues.apache.org/jira/browse/AMQ-5863 - fix and test - sequenceId state in messageid needs to be cleared before doing a store 2015-06-25 12:49:52 +01:00
Christian Schneider 97cd60fb7e [AMQ-5821] Use wiring to check for extensions
This closes #119
2015-06-24 12:10:10 -04:00
gtully 37c46b9b42 https://issues.apache.org/jira/browse/AMQ-5705 - fix test regression. Make PublishedAddressPolicy vm scheme aware so such that is won't attempt a transform 2015-06-23 11:25:43 +01:00
Timothy Bish 833d30837b https://issues.apache.org/jira/browse/AMQ-5848
Introduce OpenWire v11 which adds noLocal to the SubscriptionInfo, if
configured KahaDB can store that version and the noLocal value is
recovered from the store.
2015-06-18 14:25:07 -04:00
Christopher L. Shannon (cshannon) b40dc4cc54 https://issues.apache.org/jira/browse/AMQ-5668
This commit fixes a race condition in AbstractStoreCursor setLastCacheId that could
cause a null pointer exception in certain cases.
2015-06-17 19:09:53 -04:00
Christopher L. Shannon (cshannon) fc3e026122 https://issues.apache.org/jira/browse/AMQ-5705
Modifing the AdivsoryBroker to set the originBrokerURL to the transport connector's
URL if it has been set versus using a default URL.
2015-06-17 17:47:24 +00:00
Christopher L. Shannon (cshannon) edacc2a840 https://issues.apache.org/jira/browse/AMQ-5843
Adding a new property on PolicyEntry called includeBodyForAdvisory which will
include the original message body when sending advisory messages that include
the original message, instead of clearing it out.  This is turned off by
default.
2015-06-16 12:48:38 -04:00
Christopher L. Shannon (cshannon) 61fd811adc https://issues.apache.org/jira/browse/AMQ-5393
Adding a property called schedulePeriodForDiskUsageCheck which can be set
to a time period to periodically check disk usage limits and adjust if
the amount of disk space has been reduced.
2015-06-12 10:35:20 -04:00
gtully 3100909041 https://issues.apache.org/jira/browse/AMQ-5830 - ensure duplex inbound connection sets network=true flag, fix and test 2015-06-09 12:19:49 +01:00
Christopher L. Shannon (cshannon) 886e2d4d97 Adding a configuration option to PolicyEntry to enable setting the maximum number of created destinations by policy on the broker.
This resolves https://issues.apache.org/jira/browse/AMQ-5751
2015-06-08 16:00:08 -04:00
Christopher L. Shannon (cshannon) ea03bb1f8c https://issues.apache.org/jira/browse/AMQ-5813
Fixing countBeforeFull for TopicSubscriptions to report a positive value
2015-06-05 14:08:15 -04:00
gtully 928e815a02 https://issues.apache.org/jira/browse/AMQ-5656 - add support for selective mbean suppression based on objectName attribute match. Avoids mbeanserver contention on systems where producer/consumers/dests are dynamic 2015-06-04 14:27:18 +01:00
Timothy Bish 35b82e5f93 https://issues.apache.org/jira/browse/AMQ-5599
Ensure that persistence adapter start is not missed to avoid infinite
wait for it.
2015-06-02 11:36:47 -04:00
mfrazier dcc7b11347 AMQ-5713 - making accessors public to remove startup error.
Includes unit test DestinationBridgeAccessorsTest which replicated the problem and now passes.
2015-05-29 16:13:41 -07:00
Christopher L. Shannon (cshannon) eece576b19 https://issues.apache.org/jira/browse/AMQ-5792
Adding SubscriptionStatistics to group together all metrics in a bean
for a subscription.
2015-05-28 15:47:30 -04:00
Dejan Bosanac ea2746b121 https://issues.apache.org/jira/browse/AMQ-5697 - adding acl fields as some plugins can use those directly instead of setting roles 2015-05-25 13:24:36 +02:00
Christopher L. Shannon 84737f7749 https://issues.apache.org/jira/browse/AMQ-5771
Including remote ip address when logging authentication failures
2015-05-22 15:37:48 -04:00
gtully 9bc602be43 https://issues.apache.org/jira/browse/AMQ-4705 - close window where both master and slave can be active - slave waits for possible keepAlivePeriod with the lock before startin. Ensures a keepAlive has completed on the master 2015-05-22 11:40:40 +01:00
Christopher L. Shannon (cshannon) e99c814830 Fixing missing volatile on references in VMTransport to prevent a synchronization bug.
This resolves https://issues.apache.org/jira/browse/AMQ-5787
2015-05-20 15:09:17 -04:00
gtully 9ad65c62ed https://issues.apache.org/jira/browse/AMQ-5164 https://issues.apache.org/jira/browse/AMQ-4842 - master slave advisory needed to wait for failover reconnect - exponential backoff was sometimes giving to long a delay. Retroactive override needs to applied after policies. Tests reinstated 2015-05-20 09:57:05 +01:00
gtully 3fdf9861b2 https://issues.apache.org/jira/browse/AMQ-5783 - properly remove free topic from the store when there are no remaining subs. Issue: release of the index but remaining metadata refs to the old state. Mbean was also left dangling. test thanks to pat fox 2015-05-20 09:57:05 +01:00
gtully 17f4f349f3 more tidy up of derby usage in tests to ensure proper cleanup; ci failures 2015-05-20 09:57:05 +01:00
Timothy Bish fd2805fe1c https://issues.apache.org/jira/browse/AMQ-5441
Don't dip into the store to recover durable topic subscriptions or queue
subscriptions depending on the subscription strategy in use.  The needed
information is always kept in either TopicRegion or QueueRegion even
after restart as all stored destinations are reloaded as the broker is
started.
2015-05-15 16:01:54 -04:00
gtully 5e36f65e0e https://issues.apache.org/jira/browse/AMQ-4705 - lastmod granualarity is second on some nix. Also write on lock acquire to ensure modification time change 2015-05-15 13:23:12 +01:00
gtully 181e4d4617 https://issues.apache.org/jira/browse/AMQ-4705 - ensure cached lastModified is externally visible - hardened test showed window for missed mod 2015-05-14 13:48:14 +01:00
gtully b8a20e9ef6 skipped first dispatch on vmtransport needed a better test - reworked to avoid busy loop on full and ensured sync on started for enqueue. Sort FailoverStaticNetworkTest and NetworkOfTwentyBrokersTest intermittent failures 2015-05-13 13:56:24 +01:00