cheddar
613ebd54b5
1) Delete unused things
2013-08-23 14:32:14 -05:00
cheddar
b897c2cb22
1) IndexCoordinator appears to work as the CliOverlord now, yay!
2013-08-23 14:11:34 -05:00
fjy
d92ab8bb58
more logs for RTR
2013-08-21 21:47:59 -07:00
fjy
54f00479cc
add explicit null check for moving tasks from pending to running
2013-08-21 13:02:35 -07:00
fjy
88661b26a0
bug fix for RTR removing workers race condition and partition chunks not being sorted by chunk number
2013-08-21 11:14:54 -07:00
fjy
e283de6831
fix another bug with RTR to remove things correctly from running tasks
2013-08-20 19:34:30 -07:00
fjy
d02be16245
fix RTR closing PCC too early
2013-08-20 19:25:16 -07:00
Gian Merlino
70ab225770
Add missing license headers
2013-08-20 17:50:10 -07:00
Gian Merlino
455645e723
Workers announce TaskAnnouncement rather than TaskStatus
2013-08-20 16:14:36 -07:00
Gian Merlino
9609314765
ForkingTaskRunner: Make TaskInfo into ForkingTaskRunnerWorkItem
...
This allows the API/GUI to return reasonable results when the primary
task runner is a ForkingTaskRunner.
2013-08-20 14:04:28 -07:00
Gian Merlino
4e8325f963
Better tests and error messages for TaskResource
2013-08-20 14:01:38 -07:00
Gian Merlino
b102d67173
Fix getResourceManagementScheduler for non-autoscaling configs
2013-08-20 13:39:05 -07:00
Gian Merlino
25e330780c
Simplify AbstractTask constructor
2013-08-20 13:38:52 -07:00
Gian Merlino
d8493f8e26
RealtimeIndexTask: Fix "resource" serde
2013-08-20 13:02:52 -07:00
fjy
5c600c7012
change error msg to alert
2013-08-15 13:27:41 -07:00
fjy
4e7dac18b9
fix condition where status would be returned even if worker not running task
2013-08-15 13:12:57 -07:00
fjy
1fb6107a37
fix the case where RTR does not clean up a completed task on startup
2013-08-15 13:09:02 -07:00
Gian Merlino
8d7a4f4493
Retries for S3TaskLogs, S3DataSegmentPusher
2013-08-12 14:27:34 -07:00
fjy
df883a9823
learn to type
2013-08-06 21:18:36 -07:00
fjy
795657aedf
fix bug where workers with same capacity would not be unique
2013-08-06 21:04:13 -07:00
fjy
ebf1ac47f0
Merge branch 'master' of github.com:metamx/druid
2013-08-06 15:38:25 -07:00
fjy
9d0e4a94f0
alert when task fails in RTR assign
2013-08-06 15:38:17 -07:00
cheddar
4a64ce37ed
Finish the merging, wtf IntelliJ?
2013-08-06 13:34:24 -07:00
cheddar
eee1efdcb5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
2013-08-06 13:33:31 -07:00
Gian Merlino
a1904c9b3b
ChatHandlerResource: Fix Guice type errors
2013-08-05 19:56:03 -07:00
fjy
d1b2a5a4b3
fix indexer console serde of running tasks
2013-08-05 18:22:12 -07:00
fjy
479f0cefca
fix bug with RTR not assigning tasks when a new worker is available
2013-08-05 17:57:59 -07:00
fjy
626cf14a6e
fix bug where the curator config name was changed in one place but not another; make some info msgs into debug msgs; fix zkworker serialization
2013-08-05 16:02:26 -07:00
fjy
66c658305f
Merge branch 'master' of github.com:metamx/druid
2013-08-05 14:44:09 -07:00
fjy
35f89d7232
make RTR idempotent to multiple run requests for same task, because higher level things in the indexing service require this behaviour
2013-08-05 14:44:01 -07:00
Gian Merlino
efd34f3a8b
TaskRunner: Fix comment
2013-08-05 14:20:31 -07:00
cheddar
2361e0112a
Make it all compile again...
2013-08-02 10:14:46 -07:00
fjy
c33f2f06ff
fix logic of how to assign tasks to workers
2013-08-02 09:01:02 -07:00
fjy
584ccac833
move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better
2013-08-01 17:50:05 -07:00
cheddar
9e78bb38f5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
pom.xml
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-08-01 16:42:47 -07:00
cheddar
019bb5d453
1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working!
...
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
2013-08-01 15:28:08 -07:00
fjy
a4edc2221d
fix RTR comments
2013-07-31 15:28:52 -07:00
fjy
215d147a69
Merge branch 'worker-resource' of github.com:metamx/druid into worker-resource
2013-07-31 15:23:49 -07:00
fjy
e2b5cd6067
Merge branch 'master' of github.com:metamx/druid into worker-resource
...
Conflicts:
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
2013-07-31 15:23:13 -07:00
Gian Merlino
eaddce06d5
Call TaskRunner.bootstrap immediately after starting it
2013-07-30 15:26:11 -07:00
fjy
50836798fa
toggle between compressed and non compressed service discovery
2013-07-29 15:40:45 -07:00
fjy
ad65c8111d
fix logs
2013-07-29 11:41:42 -07:00
fjy
4ae8395538
1) on bootstrap, load all initial data and do a compare with bootstrapped tasks, delete any that are extra out there
...
2) change autoscaling logic such that it only works with remote task runnrs
3) zk workers use their own status caches to determine what they are running
2013-07-26 14:32:08 -07:00
cheddar
6b9963b472
1) Move the @Json and @Smile annotations to com.metamx.druid.guice.annotations
...
2) Changed ServiceAnnouncer to take a DruidNode object
3) Remove ServiceInstanceFactory interface and implementations
4) Add registrations to DiscoveryModule so that you can register nodes that should be announced on startup
5) Make the set of default Monitors configurable
2013-07-26 14:03:51 -07:00
fjy
a1262760b2
Merge branch 'master' into worker-resource
2013-07-26 10:02:15 -07:00
Gian Merlino
4dd42c7177
RemoteTaskActionClient: Log service uri when submitting action
2013-07-25 14:48:25 -07:00
Gian Merlino
5d44f0f15b
ForkingTaskRunner: Use guava Closer to closer stuff
2013-07-25 14:37:45 -07:00
Gian Merlino
952b8ce06b
RemoteTaskActionClient: Fix http path
2013-07-25 14:37:20 -07:00
Gian Merlino
f38d7cf964
Fix busted condition
2013-07-24 16:33:01 -07:00
Gian Merlino
6d1cb1bfc1
IndexerCoordinatorResource: Log when remote task action fails
2013-07-24 16:13:12 -07:00
