703b674800
add availability zone info to autoscaling
2013-10-07 12:16:50 -07:00
ac330f72bb
first set of changes to standarize the naming convention we use in druid
2013-10-03 16:36:48 -07:00
8bc56daa66
fix things up according to code review comments
2013-09-26 11:35:45 -07:00
87259321b6
port hadoop druid indexer to new guice framework
2013-09-26 11:04:42 -07:00
dc8a119787
fix broken unit tests are a result of the last merge
2013-09-23 12:56:01 -07:00
cabae7993d
port over multi threaded realtime and also fix broken realtime nodes that can't start up
2013-09-16 16:03:47 -07:00
f7c10e3594
rework tests in indexing service to be more unit testy
2013-09-12 16:37:58 -07:00
a2dcc45a8e
1) Remove SingleSegmentLoader and replace with OmniSegmentLoader
2013-09-12 11:47:03 -05:00
6c9a107356
1) remove duplicate package initialization.initialization
2013-09-09 17:02:57 -05:00
3c39f90c89
1) Move Firehose interface and dependencies to druid-api
...
2) Move DataSegment* interfaces and dependencies to druid-api
2013-08-31 16:43:28 -05:00
5ab671050e
No more com.metamx.druid, it is now all io.druid!
2013-08-30 19:42:12 -05:00
bd0756e360
More stuff moved, things still compiling and tests still passing. Yay!
2013-08-30 18:58:35 -05:00
56e2b956d0
OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.!
2013-08-30 18:21:04 -05:00
9c30ced5ea
1) Move various "api" classes to io.druid packages and make sure things compile and stuff
2013-08-28 15:51:02 -05:00
ee1e73cfa1
1) Make it compile again after the merge
2013-08-27 14:36:01 -05:00
5fa944dd26
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
pom.xml
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/http/MasterServletModule.java
server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
2013-08-27 14:27:32 -05:00
3617ac17fc
1) Eliminate ExecutorMain and have it run using the new Main!
2013-08-27 14:11:05 -05:00
269997dc94
1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful
2013-08-26 18:08:41 -05:00
55dbda2046
1) Worker appears to be running! It's also now known as the MiddleManager
2013-08-23 17:59:48 -05:00
b897c2cb22
1) IndexCoordinator appears to work as the CliOverlord now, yay!
2013-08-23 14:11:34 -05:00
d92ab8bb58
more logs for RTR
2013-08-21 21:47:59 -07:00
88661b26a0
bug fix for RTR removing workers race condition and partition chunks not being sorted by chunk number
2013-08-21 11:14:54 -07:00
455645e723
Workers announce TaskAnnouncement rather than TaskStatus
2013-08-20 16:14:36 -07:00
4e8325f963
Better tests and error messages for TaskResource
2013-08-20 14:01:38 -07:00
d8493f8e26
RealtimeIndexTask: Fix "resource" serde
2013-08-20 13:02:52 -07:00
1fb6107a37
fix the case where RTR does not clean up a completed task on startup
2013-08-15 13:09:02 -07:00
4a64ce37ed
Finish the merging, wtf IntelliJ?
2013-08-06 13:34:24 -07:00
eee1efdcb5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
2013-08-06 13:33:31 -07:00
479f0cefca
fix bug with RTR not assigning tasks when a new worker is available
2013-08-05 17:57:59 -07:00
35f89d7232
make RTR idempotent to multiple run requests for same task, because higher level things in the indexing service require this behaviour
2013-08-05 14:44:01 -07:00
2361e0112a
Make it all compile again...
2013-08-02 10:14:46 -07:00
584ccac833
move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better
2013-08-01 17:50:05 -07:00
9e78bb38f5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
pom.xml
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-08-01 16:42:47 -07:00
4ae8395538
1) on bootstrap, load all initial data and do a compare with bootstrapped tasks, delete any that are extra out there
...
2) change autoscaling logic such that it only works with remote task runnrs
3) zk workers use their own status caches to determine what they are running
2013-07-26 14:32:08 -07:00
a1262760b2
Merge branch 'master' into worker-resource
2013-07-26 10:02:15 -07:00
30d98f56c1
RealtimeIndexTask: Allow configurable rejection policies
2013-07-24 16:12:54 -07:00
0eebb0a149
Add RealtimeMetricsMonitor to RealtimeIndexTask
2013-07-24 14:39:59 -07:00
b9578a1ada
1) remove retry logic from RTR
...
2) simplify configs
3) introduce task resource
4) make worker versions match coordinator version by default
2013-07-12 12:51:12 -07:00
7219ed15d3
fix according to code review
2013-07-02 15:56:12 -07:00
caa68e101a
first commit; things working right now
2013-06-19 15:56:45 -07:00
f68df7ab69
1) Make tests work and continue trying to make the DruidMaster start up with just Guice
2013-06-07 12:01:46 -07:00
e4ea357b52
multiple bug fixes for indexing service; scv quotes
2013-06-06 14:10:18 -07:00
06931ee0f5
introduce availability groups
2013-06-04 17:12:19 -07:00
42cc87a294
Merge branch 'master' into refactor-indexing
...
Conflicts:
indexing-service/src/main/java/com/metamx/druid/indexing/common/task/IndexTask.java
pom.xml
2013-05-31 17:28:59 -07:00
6213c0b63c
Merge branch 'master' into refactor-indexing
2013-05-15 17:14:40 -07:00
20ae1d8b6b
lots of cleanups and refactorings
2013-05-15 15:37:04 -07:00
fjy
cheddar
Gian Merlino