Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource. 

See stack traces here, from current master: https://gist.github.com/gianm/bd9a66c826995f97fc8f

1. The thread "qtp925672150-62" holds the lock on InternalInjectorCreator.class,
   used by Scopes.SINGLETON, and wants the lock on "handlers" in Lifecycle.addMaybeStartHandler
   called by DiscoveryModule.getServiceAnnouncer.
2. The main thread holds the lock on "handlers" in Lifecycle.addMaybeStartHandler, which it
   took because it's trying to add the ExecutorLifecycle to the lifecycle. main is trying
   to get the InternalInjectorCreator.class lock because it's running ExecutorLifecycle.start,
   which does some Jackson deserialization, and Jackson needs that lock in order to inject
   stuff into the Task it's deserializing.

This patch eagerly instantiates ChatHandlerResource (which I believe is what's trying to
create the ServiceAnnouncer in the qtp925672150-62 jetty thread) and the ExecutorLifecycle.
This commit is contained in:
Gian Merlino 2016-03-02 10:01:14 -08:00
parent c82b680cd4
commit 7557eb2800
2 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
server/src/main/java/io/druid/server/initialization/jetty/ChatHandlerServerModule.java
View File

@ -55,6 +55,7 @@ public class ChatHandlerServerModule implements Module
public void configure(Binder binder)
{
Jerseys.addResource(binder, ChatHandlerResource.class);
LifecycleModule.register(binder, ChatHandlerResource.class);
if (properties.containsKey(MAX_CHAT_REQUESTS_PROPERTY)) {
final int maxRequests = Integer.parseInt(properties.getProperty(MAX_CHAT_REQUESTS_PROPERTY));

2
services/src/main/java/io/druid/cli/CliPeon.java
View File

@ -72,7 +72,6 @@ import io.druid.segment.loading.OmniDataSegmentMover;
import io.druid.segment.loading.SegmentLoaderConfig;
import io.druid.segment.loading.StorageLocationConfig;
import io.druid.segment.realtime.firehose.ChatHandlerProvider;
import io.druid.segment.realtime.firehose.ChatHandlerResource;
import io.druid.segment.realtime.firehose.NoopChatHandlerProvider;
import io.druid.segment.realtime.firehose.ServiceAnnouncingChatHandlerProvider;
import io.druid.segment.realtime.plumber.CoordinatorBasedSegmentHandoffNotifierConfig;
@ -157,6 +156,7 @@ public class CliPeon extends GuiceRunnable
binder.bind(DataSegmentArchiver.class).to(OmniDataSegmentArchiver.class).in(LazySingleton.class);
binder.bind(ExecutorLifecycle.class).in(ManageLifecycle.class);
LifecycleModule.register(binder, ExecutorLifecycle.class);
binder.bind(ExecutorLifecycleConfig.class).toInstance(
new ExecutorLifecycleConfig()
.setTaskFile(new File(taskAndStatusFile.get(0)))