Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,302 Commits 111 Branches 503 Tags 728 MiB
Commit Graph

439 Commits

Author SHA1 Message Date
cheddar cc632cdb30 Make Jetty Server lazy 
JettyServerModule had been eagerly instantiating the Server object, which was causing things that didn't care about an HTTP interface to all of a sudden require host and port parameters.  The change makes the JettyServerModule only setup the bindings without eagerly instantiating the Jetty Server.  Each cli needs to register the Server class with the Lifecycle in order to make it actually get used.
 2013-09-23 16:02:24 -05:00
fjy dc8a119787 fix broken unit tests are a result of the last merge 2013-09-23 12:56:01 -07:00
cheddar fefc71e34f Refix the things that were lost in the merge. 2013-09-20 18:01:08 -05:00
cheddar b13ae3f405 Merge branch 'master' into fix-extensionization 
Conflicts:
	services/src/main/java/io/druid/cli/CliBroker.java
	services/src/main/java/io/druid/cli/CliCoordinator.java
 2013-09-20 17:49:13 -05:00
cheddar 5712b29c8c Fix issues with bindings and handling extensions 
The way the Guice bindings were setup previously, each process only had bindings
for the things it cared about.  This became problematic when adding extension modules
that bound everything that they could possibly need expecting that the processes would
only instantiate what they actually do need.  Guice tries to fail-fast and verifies that all
 bindings exist before it does anything, which is a problem because the extension bind
 some objects that don't necessarily have all of their dependencies bound in all processes.

The fix for this is to build a single Injector with all bindings in it and let each of the
 processes only load the things that they care about.  This also requires the use of
 Module overrides and other such interesting things, which are node done.

 In doing the fix, I also swapped out the way that the DataSegmentPusher/Puller stuff is bound, as well as made the Cassandra stuff fail if its settings are not provided.  This all of a sudden made all of the things require Cassandra's settings, so I migrated the Cassandra deep storage stuff into its own module.

 In doing these changes, I also discovered that some properties weren't properly converting for the ConvertProperties command (specifically, the properties related to data segment loading and pushing), so I fixed that.
 2013-09-20 17:45:01 -05:00
fjy f26e13c61a fix broken http endpoints 2013-09-19 17:33:58 -07:00
cheddar 6bfe2310bb Merge branch 'master' into guice 
Conflicts:
	client/pom.xml
	client/src/main/java/com/metamx/druid/query/segment/LegacySegmentSpec.java
	indexing-common/pom.xml
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolboxFactory.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskQueueTest.java
	realtime/src/main/java/com/metamx/druid/realtime/RealtimeNode.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/IrcFirehoseFactory.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/WikipediaIrcDecoder.java
	realtime/src/main/java/com/metamx/druid/realtime/plumber/RealtimePlumberSchool.java
	server/src/main/java/com/metamx/druid/BaseServerNode.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/index/v1/IncrementalIndexStorageAdapter.java
	server/src/main/java/com/metamx/druid/query/group/GroupByQueryEngine.java
 2013-09-18 16:46:54 -05:00
fjy f7df951610 broker needs to announce itself in service discovery 2013-09-18 09:23:46 -07:00
fjy cabae7993d port over multi threaded realtime and also fix broken realtime nodes that can't start up 2013-09-16 16:03:47 -07:00
fjy 5a8b71ddec fix how druid services is assembled 2013-09-13 18:41:33 -07:00
cheddar e0738b3b22 1) Add tool to convert properties from the old settings to the new ones. 2013-09-12 17:59:23 -05:00
cheddar a2dcc45a8e 1) Remove SingleSegmentLoader and replace with OmniSegmentLoader 2013-09-12 11:47:03 -05:00
cheddar 3e4a4f5566 1) Adjustments to allow for the addition of new Runnables via modules. Modules should implement CliCommandCreator and add their stuff there. 2013-09-11 17:41:20 -05:00
cheddar 6c9a107356 1) remove duplicate package initialization.initialization 2013-09-09 17:02:57 -05:00
Russell Jurney a11eae4144 Edited assembly.xml to exclude the selfcontained jars, and edited path of run_example_server to use the lib/ directory instead of selfcontained jars, thus reducing the size of the assembly tarball from 150MB to 50MB. 2013-09-04 12:53:38 -07:00
cheddar 740c70ed06 1) It's good when unit tests actually pass... 2013-08-31 17:27:15 -05:00
cheddar 5ab671050e No more com.metamx.druid, it is now all io.druid! 2013-08-30 19:42:12 -05:00
cheddar bd0756e360 More stuff moved, things still compiling and tests still passing. Yay! 2013-08-30 18:58:35 -05:00
cheddar 56e2b956d0 OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.! 2013-08-30 18:21:04 -05:00
cheddar 3617ac17fc 1) Eliminate ExecutorMain and have it run using the new Main! 2013-08-27 14:11:05 -05:00
cheddar 269997dc94 1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful 2013-08-26 18:08:41 -05:00
cheddar 55dbda2046 1) Worker appears to be running! It's also now known as the MiddleManager 2013-08-23 17:59:48 -05:00
cheddar b897c2cb22 1) IndexCoordinator appears to work as the CliOverlord now, yay! 2013-08-23 14:11:34 -05:00
fjy 6a8c160740 update code according to code review 2013-08-22 10:46:05 -07:00
fjy 85ee8bb267 port realtime to guice 2013-08-13 17:08:45 -07:00
cheddar 51f648ec11 1) Rename QueryToolChestModule back 2013-08-06 12:05:56 -07:00
cheddar 69f7831d82 1) Rename QueryToolChestModule because there was a case sensitivity issue, grrr 2013-08-06 12:04:04 -07:00
cheddar 9f71d42314 1) Broker works with Guice 
2) Extract ServerViewModule to handle various ServerView thingies
3) Extract QueryToolChestModule and QueryRunnerFactoryModule to reuse code for configuration of Query stuff
4) Extract QueryJettyServerInitializer to reuse between Historical and Broker nodes
5) Remove ClientMain, BrokerMain and BrokerNode
 2013-08-06 12:01:52 -07:00
cheddar 293c0e8418 1) Clean up the servlet registration stuff, resource registrations on CliHistorical weren't working, they do now. 2013-08-05 13:58:45 -07:00
cheddar d64bc2f762 1) Eliminate Old mains() that aren't needed anymore. 2013-08-05 13:33:43 -07:00
cheddar 9e78bb38f5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
	client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
	client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
	client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
	indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ServerManager.java
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
	server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
	server/src/main/java/com/metamx/druid/master/DruidMaster.java
 2013-08-01 16:42:47 -07:00
cheddar 019bb5d453 1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working! 
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
 2013-08-01 15:28:08 -07:00
cheddar f4fc8b2f6e 1) A whole boondoggle of changes to Guicify the instantiation of things for the Historical nodes 
2) Add GuiceInjectableValues that supports @JacksonInject resolution via Guice
3) Refactor RequestLogger stuff and setup polymorphic deserialization method of creating objects
 2013-07-31 18:36:47 -07:00
cheddar 97fb971f84 1) Start the journey of making the historical nodes work with Guice. Doesn't work yet, but at least it builds. 2013-07-30 11:58:05 -07:00
cheddar 2b587cba85 1) Introduce airlift/airline for CLI stuff and add a Coordinator CLI verb to make it all happiness 2013-07-29 15:14:20 -07:00
Russell Jurney 28af252bca Include multiple configurations, one per node type, in assembly package for examples. 2013-07-01 16:36:30 -07:00
Steve Harris 5a9bf98b6c Some changes to automatically build a kit with samples. 2013-05-04 20:04:16 -07:00
cheddar 12fe145cc8 1) Setup assembly to grab all dependencies -- it's broken right now, http://jira.codehaus.org/browse/MASSEMBLY-646 2013-05-02 16:35:06 -05:00
cheddar d1a8b93b3f 1) druid-services -> services 2013-05-01 17:37:38 -05:00