bf0d9c601d
Merge branch 'task-stuff' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/KillTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskMasterLifecycle.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunnerWorkQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorServletModule.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-14 15:42:07 -08:00
dd44f5cbf0
Merge branch 'master' into task-stuff
2013-02-14 15:11:40 -08:00
ee4236dac2
1) Update to mmx build of jackson 2.1 branch to get bug-fix for jackson-databind #167
2013-02-14 17:10:27 -06:00
ef768feeb1
new resource
2013-02-14 14:20:20 -08:00
ad1de9ba9d
Merge branch 'master' into task-stuff
2013-02-14 14:16:46 -08:00
77fbe974d8
fix version conflicts of old jackson
2013-02-14 13:52:19 -08:00
de4fc98128
TaskMasterLifecycle: Stop leaderLifecycle if it dies while starting
2013-02-14 13:36:43 -08:00
18daa19fc9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskHolder.java
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/AbstractTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskContext.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-14 13:10:29 -08:00
ff325b1f65
Merger: Changes to support rolling locks and a task-action API
...
Generally, TaskStatus is no longer used as the sole way for tasks to
communicate status and metadata update requests back to the
coordinator. It is still used to communicate success or failure, but a
TaskActionClient is used for all other communication. In remote mode
this uses an http api.
TaskContext no longer exists, as its purpose is now served by
the TaskActionClient.
The TaskQueue and TaskConsumer are no longer in charge of committing
stuff for a task. Instead, it happens when an action is sent in. Tasks
are expected to ensure that this is OK by making their behavior
idempotent (since they may be retried after partially completing).
One new kind of request afforded to tasks is the ability to release
and acquire arbitrary locks. This will be useful for long-running tasks.
Short-lived tasks can still be given a lock automatically upon startup
by providing a "fixed interval".
New tables when using db storage:
- tasklocks, to support persistence of arbitrary locks.
- tasklogs, to log actions taken by tasks
2013-02-14 12:46:40 -08:00
6998d604a2
fixes #80
2013-02-14 12:41:34 -08:00
a616f86128
add DateTime import back
2013-02-14 09:09:22 -08:00
754b065c54
merge master
2013-02-14 09:08:25 -08:00
98de0382d5
Merge branch 'master' into speedup-timeboundary
2013-02-13 21:27:27 -08:00
1b85eaad3e
simplify query toolchest interface
2013-02-13 21:25:16 -08:00
e3822f6ca7
DeterminePartitionsJob: Fix docs
2013-02-13 18:51:33 -08:00
ee193f0ca7
DeterminePartitionsJob: Select partition dimension to minimize segment size variance when cardinality is low
2013-02-13 18:51:32 -08:00
c5b78e75cf
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-13 18:51:32 -08:00
82f02a4fe4
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-13 16:34:14 -08:00
1963bfe83f
cleanup
2013-02-13 16:04:43 -08:00
c7b4973700
bug fixes for RTR
2013-02-13 14:19:11 -08:00
ce1d90788d
DeterminePartitionsJob: Fix docs
2013-02-13 10:16:03 -08:00
0eb65f69d4
Merge pull request #86 from metamx/dimpartition-by-variance
...
Select partition dimension by variance when cardinality is low
2013-02-13 09:54:34 -08:00
a665bfa2ef
DeterminePartitionsJob: Select partition dimension to minimize segment size variance when cardinality is low
2013-02-13 09:51:04 -08:00
429e23d3b2
change break to continue in BalancerCostAnalyzer to skip calculating cost when it's not necessary
2013-02-13 08:57:39 -08:00
a0b159fed7
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-12 20:58:17 -06:00
fcbac96193
revert change to findNewSegmentHome: now return single ServerHolder
2013-02-12 10:47:30 -08:00
07b0b4611b
bump up to 0.3.0
2013-02-12 09:52:21 -08:00
68cd2e10a7
remove unnecessary segment grouping
2013-02-12 09:45:26 -08:00
7ab2e0552a
cleanup imports
2013-02-11 18:42:43 -08:00
633cccce69
bump up to 0.3.0
2013-02-11 18:41:25 -08:00
d4009c8c1c
refactor filterSegments to use `TimelineObjectHolder`s
2013-02-11 18:41:25 -08:00
b27a459f71
more refactoring and UT passing at this point
2013-02-11 18:13:01 -08:00
6b87ef2921
assume segment intervals have already been filtered based on query interval
2013-02-11 16:56:52 -08:00
e9f546c4ca
add default implementations to QueryToolChest methods
2013-02-11 16:51:40 -08:00
695b78b19c
make findNewSegmentHome return a priority queue so assignment can assign replicants to other servers
2013-02-11 15:28:17 -08:00
e10025c841
filter out unneeded segments for TimeBoundary query
2013-02-11 13:21:37 -08:00
69d0d98df6
t Merge branch 'master' into indexing_refactor
2013-02-11 12:37:20 -08:00
a9176e738a
Merge branch 'master' into indexing_refactor
2013-02-11 12:36:49 -08:00
1b9764fffb
add binary version to datasegments for balancer test
2013-02-11 11:36:45 -08:00
e12da51057
merge master
2013-02-11 11:28:50 -08:00
6708029043
1) Remove class level comments auto-created by IntelliJ
2013-02-11 13:13:52 -06:00
05168808c2
Merge pull request #77 from metacret/master
...
Adding LinearShardSpec and ACL on S3 object
2013-02-11 11:12:31 -08:00
e519f82e87
[maven-release-plugin] prepare for next development iteration
2013-02-11 10:58:51 -08:00
73f7468a23
[maven-release-plugin] prepare release druid-0.2.7
2013-02-11 10:58:43 -08:00
07c005a0f7
Merge branch 'master' of github.com:metamx/druid
2013-02-11 10:56:39 -08:00
ac93f752b0
fix a case where a segment may get stuck in the terminate queue
2013-02-11 10:56:30 -08:00
875df7b989
Merge pull request #81 from metamx/fix-timeboundary
...
fix cut-n-paste mistake in error message
2013-02-11 10:39:56 -08:00
111df9a90e
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-07 10:35:32 -08:00
6ff42f379c
fix cut-n-paste mistake in error message
2013-02-07 10:11:36 -08:00
9ac5eeebb3
1) Fix NPE in DeterminePartitionsJob when it fails
2013-02-06 23:34:46 -08:00
Fangjin Yang
Gian Merlino
Eric Tschetter
xvrl
Nelson Ray
cheddar