Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,273 Commits 113 Branches 504 Tags 726 MiB
Commit Graph

866 Commits

Author SHA1 Message Date
Gian Merlino 9609314765 ForkingTaskRunner: Make TaskInfo into ForkingTaskRunnerWorkItem 
This allows the API/GUI to return reasonable results when the primary
task runner is a ForkingTaskRunner.
 2013-08-20 14:04:28 -07:00
Gian Merlino 4e8325f963 Better tests and error messages for TaskResource 2013-08-20 14:01:38 -07:00
Gian Merlino b102d67173 Fix getResourceManagementScheduler for non-autoscaling configs 2013-08-20 13:39:05 -07:00
Gian Merlino 25e330780c Simplify AbstractTask constructor 2013-08-20 13:38:52 -07:00
Gian Merlino d8493f8e26 RealtimeIndexTask: Fix "resource" serde 2013-08-20 13:02:52 -07:00
fjy 5c600c7012 change error msg to alert 2013-08-15 13:27:41 -07:00
fjy 4e7dac18b9 fix condition where status would be returned even if worker not running task 2013-08-15 13:12:57 -07:00
fjy 1fb6107a37 fix the case where RTR does not clean up a completed task on startup 2013-08-15 13:09:02 -07:00
Gian Merlino 8d7a4f4493 Retries for S3TaskLogs, S3DataSegmentPusher 2013-08-12 14:27:34 -07:00
fjy df883a9823 learn to type 2013-08-06 21:18:36 -07:00
fjy 795657aedf fix bug where workers with same capacity would not be unique 2013-08-06 21:04:13 -07:00
fjy ebf1ac47f0 Merge branch 'master' of github.com:metamx/druid 2013-08-06 15:38:25 -07:00
fjy 9d0e4a94f0 alert when task fails in RTR assign 2013-08-06 15:38:17 -07:00
cheddar 4a64ce37ed Finish the merging, wtf IntelliJ? 2013-08-06 13:34:24 -07:00
cheddar eee1efdcb5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
 2013-08-06 13:33:31 -07:00
Gian Merlino a1904c9b3b ChatHandlerResource: Fix Guice type errors 2013-08-05 19:56:03 -07:00
fjy d1b2a5a4b3 fix indexer console serde of running tasks 2013-08-05 18:22:12 -07:00
fjy 479f0cefca fix bug with RTR not assigning tasks when a new worker is available 2013-08-05 17:57:59 -07:00
fjy 626cf14a6e fix bug where the curator config name was changed in one place but not another; make some info msgs into debug msgs; fix zkworker serialization 2013-08-05 16:02:26 -07:00
fjy 66c658305f Merge branch 'master' of github.com:metamx/druid 2013-08-05 14:44:09 -07:00
fjy 35f89d7232 make RTR idempotent to multiple run requests for same task, because higher level things in the indexing service require this behaviour 2013-08-05 14:44:01 -07:00
Gian Merlino efd34f3a8b TaskRunner: Fix comment 2013-08-05 14:20:31 -07:00
cheddar 2361e0112a Make it all compile again... 2013-08-02 10:14:46 -07:00
fjy c33f2f06ff fix logic of how to assign tasks to workers 2013-08-02 09:01:02 -07:00
fjy 584ccac833 move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better 2013-08-01 17:50:05 -07:00
cheddar 9e78bb38f5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
	client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
	client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
	client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
	indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ServerManager.java
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
	server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
	server/src/main/java/com/metamx/druid/master/DruidMaster.java
 2013-08-01 16:42:47 -07:00
cheddar 019bb5d453 1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working! 
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
 2013-08-01 15:28:08 -07:00
fjy a4edc2221d fix RTR comments 2013-07-31 15:28:52 -07:00
fjy 215d147a69 Merge branch 'worker-resource' of github.com:metamx/druid into worker-resource 2013-07-31 15:23:49 -07:00
fjy e2b5cd6067 Merge branch 'master' of github.com:metamx/druid into worker-resource 
Conflicts:
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
 2013-07-31 15:23:13 -07:00
Gian Merlino eaddce06d5 Call TaskRunner.bootstrap immediately after starting it 2013-07-30 15:26:11 -07:00
fjy 50836798fa toggle between compressed and non compressed service discovery 2013-07-29 15:40:45 -07:00
fjy ad65c8111d fix logs 2013-07-29 11:41:42 -07:00
fjy 4ae8395538 1) on bootstrap, load all initial data and do a compare with bootstrapped tasks, delete any that are extra out there 
2) change autoscaling logic such that it only works with remote task runnrs
3) zk workers use their own status caches to determine what they are running
 2013-07-26 14:32:08 -07:00
cheddar 6b9963b472 1) Move the @Json and @Smile annotations to com.metamx.druid.guice.annotations 
2) Changed ServiceAnnouncer to take a DruidNode object
3) Remove ServiceInstanceFactory interface and implementations
4) Add registrations to DiscoveryModule so that you can register nodes that should be announced on startup
5) Make the set of default Monitors configurable
 2013-07-26 14:03:51 -07:00
fjy a1262760b2 Merge branch 'master' into worker-resource 2013-07-26 10:02:15 -07:00
Gian Merlino 4dd42c7177 RemoteTaskActionClient: Log service uri when submitting action 2013-07-25 14:48:25 -07:00
Gian Merlino 5d44f0f15b ForkingTaskRunner: Use guava Closer to closer stuff 2013-07-25 14:37:45 -07:00
Gian Merlino 952b8ce06b RemoteTaskActionClient: Fix http path 2013-07-25 14:37:20 -07:00
Gian Merlino f38d7cf964 Fix busted condition 2013-07-24 16:33:01 -07:00
Gian Merlino 6d1cb1bfc1 IndexerCoordinatorResource: Log when remote task action fails 2013-07-24 16:13:12 -07:00
Gian Merlino 30d98f56c1 RealtimeIndexTask: Allow configurable rejection policies 2013-07-24 16:12:54 -07:00
Gian Merlino 0eebb0a149 Add RealtimeMetricsMonitor to RealtimeIndexTask 2013-07-24 14:39:59 -07:00
Gian Merlino 71704f7ee8 Replace graceful/hard shutdown combo with simple exit 2013-07-24 10:27:21 -07:00
fjy b9578a1ada 1) remove retry logic from RTR 
2) simplify configs
3) introduce task resource
4) make worker versions match coordinator version by default
 2013-07-12 12:51:12 -07:00
fjy d3eb491155 Merge branch 'batch-zk' into worker-resource 
Conflicts:
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
 2013-07-02 16:24:59 -07:00
fjy e21aa41bdd Merge branch 'master' into worker-resource 
Conflicts:
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
 2013-07-02 16:23:53 -07:00
fjy 5ac2d1db0b Merge branch 'master' of github.com:metamx/druid into batch-zk 
Conflicts:
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
 2013-07-02 15:57:31 -07:00
fjy 7219ed15d3 fix according to code review 2013-07-02 15:56:12 -07:00
cheddar 797a083b69 1) Adjust SingleSegmentLoader to allow for storing segments on multiple different mount points. The specification language is really janky right now, so this is remaining a stealth feature for the time being. 2013-07-01 14:56:57 -07:00
fjy a6787ab201 fix up configs 2013-06-27 16:40:17 -07:00
fjy caa68e101a first commit; things working right now 2013-06-19 15:56:45 -07:00
cheddar 11ea15fc1a 1) Introduce Jetty 8 
2) Fix up modules based on exceptions at startup for MasterMain
 2013-06-17 10:53:50 -07:00
cheddar 2f56c24259 1) Inject IndexingServiceClient 
2) Switch all the DBI references to IDBI
 2013-06-07 17:37:33 -07:00
fjy 37a3994003 add delegating executor service and fix bug with query priortization 2013-06-07 14:08:51 -07:00
cheddar f68df7ab69 1) Make tests work and continue trying to make the DruidMaster start up with just Guice 2013-06-07 12:01:46 -07:00
Gian Merlino d69fb61a6e Fix getImplicitLockInterval check 2013-06-06 18:22:41 -07:00
fjy 14f9b0409e fix pushing down too many properties 2013-06-06 17:58:40 -07:00
cheddar 9df458a065 1) Initial commit of conversion to using Guice modules for bootstrapping. Things don't actually completely work yet. 2013-06-06 15:43:15 -07:00
fjy e4ea357b52 multiple bug fixes for indexing service; scv quotes 2013-06-06 14:10:18 -07:00
fjy 451d3d358b some fixes according to code review comments 2013-06-06 11:02:08 -07:00
fjy 06931ee0f5 introduce availability groups 2013-06-04 17:12:19 -07:00
fjy 42cc87a294 Merge branch 'master' into refactor-indexing 
Conflicts:
	indexing-service/src/main/java/com/metamx/druid/indexing/common/task/IndexTask.java
	pom.xml
 2013-05-31 17:28:59 -07:00
fjy c6ad0753f0 bug fixes 2013-05-16 16:02:20 -07:00
fjy 6213c0b63c Merge branch 'master' into refactor-indexing 2013-05-15 17:14:40 -07:00
fjy 20ae1d8b6b lots of cleanups and refactorings 2013-05-15 15:37:04 -07:00