Apache/druid
1
0
Fork 0
mirror of https://github.com/apache/druid.git synced 2025-02-08 02:58:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
395 Commits 116 Branches 507 Tags
Commit Graph

133 Commits

Author SHA1 Message Date
Gian Merlino
4ec67f612e Merger: Emit indexer/segment/bytes once per segment. No longer emit indexer/segment/count. 2013-02-01 16:29:58 -08:00
Fangjin Yang
d252059e4a Merge branch 'task-stuff' of github.com:metamx/druid into task-stuff 2013-02-01 12:59:04 -08:00
Fangjin Yang
58bc305e26 changes to remote task runner to kill nodes better 2013-02-01 12:58:38 -08:00
Gian Merlino
393bec0539 TaskQueue: Behavior tweaks, simplification 
- Lock tasks on add if possible
- Detect "already added" using exception test instead of bookkeeping map
- Update task status after commitRunnable instead of before commitRunnable
 2013-02-01 12:16:06 -08:00
Gian Merlino
455bae2c76 Merge branch 'master' into task-stuff 2013-01-31 11:28:23 -08:00
Fangjin Yang
f50c214514 Merge branch 'master' into task-stuff 2013-01-31 09:59:52 -08:00
Gian Merlino
f946fc3ee6 TaskConsumer: Commit to segment db in one transaction 2013-01-31 08:22:10 -08:00
Gian Merlino
1e35e6ad46 TaskQueue: Tweak to implementation of findTaskGroupForTask 2013-01-31 08:22:10 -08:00
Gian Merlino
34e76f868f TaskQueue: Clarify "Naptime!" log message 2013-01-31 08:22:10 -08:00
Gian Merlino
86ec3a372e TaskGroup: Add javadocs 2013-01-31 08:22:10 -08:00
Gian Merlino
65576cbade TaskStorageQueryAdapter: Rename some methods and tweak javadocs for clarity 2013-01-31 08:22:10 -08:00
Gian Merlino
779c54d8f2 TaskToolbox: Replace IndexerCoordinatorConfig with TaskConfig 2013-01-31 08:22:09 -08:00
Gian Merlino
fe38ed2547 Introduce TaskExistsException, thrown by TaskStorage.insert when appropriate 2013-01-31 08:22:09 -08:00
Gian Merlino
0e469c6f4c TaskStatus: Fix javadocs 2013-01-31 08:09:22 -08:00
Fangjin Yang
00e6ac1e10 [maven-release-plugin] prepare for next development iteration 2013-01-30 17:09:23 -08:00
Fangjin Yang
ef19de8270 [maven-release-plugin] prepare release druid-0.2.2 2013-01-30 17:09:18 -08:00
Fangjin Yang
26ed96f05d bug fix for autoscaling termination 2013-01-30 17:06:02 -08:00
Fangjin Yang
fa733565e8 [maven-release-plugin] prepare for next development iteration 2013-01-30 16:28:28 -08:00
Fangjin Yang
0b116a8fe4 [maven-release-plugin] prepare release druid-0.2.1 2013-01-30 16:28:22 -08:00
Fangjin Yang
2a0de2090b [maven-release-plugin] prepare for next development iteration 2013-01-30 13:47:15 -08:00
Fangjin Yang
049ca2bad4 [maven-release-plugin] prepare release druid-0.2.0 2013-01-30 13:47:08 -08:00
Fangjin Yang
74057600f9 fix worker node still using db connection 2013-01-27 09:57:37 -08:00
Gian Merlino
1cff766e2a RemoteTaskRunner: Only cleanup paths during bootstrapping if task is complete 2013-01-25 16:01:17 -08:00
Fangjin Yang
efdff7b9f1 fix unit test bug 2013-01-25 14:33:05 -08:00
Gian Merlino
0f4746450b RemoteTaskRunner: null check on workerData 2013-01-25 13:40:20 -08:00
Gian Merlino
bb68091cef Index service: TaskConsumer tests 2013-01-25 13:15:55 -08:00
Gian Merlino
82d77a8b72 Index service: Fix kill task 2013-01-25 13:15:49 -08:00
Gian Merlino
e6a618ca76 Merge branch 'killsegments' into task-stuff 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
	merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/IndexTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/LocalTaskRunner.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/exec/TaskConsumer.java
	merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
	merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
	merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
 2013-01-25 11:30:10 -08:00
Gian Merlino
a14200d779 Index service support for early returns and choice of commit semantics. 
Task:
- Add TaskCallback to run method (for early returns)

TaskStatus:
- Remove CONTINUED status
- Add segmentsNuked (placeholder for future deletion support)
- Add more builder methods
- Add validations to constructor

TaskStorage:
- Add TaskStorageQueryAdapter, a concrete class that wraps TaskStorages and
  provides various read-only convenience methods
- Add getTask method for benefit of TaskStorageQueryAdapter

TaskQueue:
- Rename "done" to "notify"
- notify is responsible for deciding if we should commit
- Add optional commitRunnable to "notify", which gets called when it's time to commit
- Allow nextTasks and commits to run early (statusCode RUNNING)
- Move getStatus, collapseStatus functionality to TaskStorageQueryAdapter
 2013-01-25 11:05:34 -08:00
Fangjin Yang
272d737517 cleaning up some interactions with RTR and workers 2013-01-22 16:21:38 -08:00
Fangjin Yang
bab9ee8827 Merge branch 'master' into killsegments 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
 2013-01-21 14:47:49 -08:00
cheddar
07131c51ed Merge pull request #53 from metamx/autoscaling 
Use a database to store configurations for indexer workers
 2013-01-18 17:08:52 -08:00
Gian Merlino
7166534666 YeOldePlumberSchool: Tweak for IndexIO changes 2013-01-17 16:03:18 -08:00
Gian Merlino
5ce53eb2ac Merge branch 'master' into yops-descriptor-dimensions 2013-01-17 16:02:32 -08:00
Fangjin Yang
38b2041ad9 key/value config table 2013-01-17 14:56:48 -08:00
Fangjin Yang
21613bc73b initial commit to hard delete segments 2013-01-16 17:31:01 -08:00
Eric Tschetter
c8cb96b006 1) Remove vast majority of usages of IndexIO.mapDir() and deprecated it. IndexIO.loadIndex() is the new IndexIO.mapDir() 
2) Fix bug with IndexMerger and null columns
3) Add QueryableIndexIndexableAdapter so that QueryableIndexes can be merged
4) Adjust twitter example to have multiple values for each hash tag
5) Adjusted GroupByQueryEngine to just drop dimensions that don't exist instead of throwing an NPE
 2013-01-16 17:10:33 -06:00
Gian Merlino
6fc350bfba YeOldePlumberSchool: Populate dimension metadata in segment descriptor 2013-01-16 11:30:24 -08:00
Fangjin Yang
3c0880fe28 Merge branch 'master' into autoscaling 2013-01-15 13:36:39 -08:00
Fangjin Yang
7e074e8158 fix pom breakage 2013-01-15 12:04:12 -08:00
Fangjin Yang
d1f4317af7 updates to autoscaling config based on code review comments 2013-01-14 14:55:04 -08:00
Fangjin Yang
4c2da93389 bug fixes with ec2 auto scaling 2013-01-09 14:51:35 -08:00
Fangjin Yang
3685839376 Merge branch 'master' into autoscaling 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
 2013-01-07 18:04:14 -08:00
Fangjin Yang
79e9ff94a6 add missing headers 2013-01-07 14:07:47 -08:00
Fangjin Yang
aff4a05ca3 db based configs for indexer workers 2013-01-07 13:41:26 -08:00
Eric Tschetter
b184c46ae0 1) Introduce idea of value types and generic columns to the serialization format 
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
 2013-01-04 18:12:49 -06:00
xvrl
9627fdcd1c [maven-release-plugin] prepare for next development iteration 2013-01-03 10:09:52 -08:00
xvrl
f2c2d3f4cf [maven-release-plugin] prepare release druid-0.1.25 2013-01-03 10:09:43 -08:00
xvrl
2a188996cd [maven-release-plugin] prepare for next development iteration 2012-12-31 14:56:10 -08:00
xvrl
6cdcc2becc [maven-release-plugin] prepare release druid-0.1.24 2012-12-31 14:56:03 -08:00