Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,185 Commits 111 Branches 503 Tags 726 MiB
Commit Graph

351 Commits

Author SHA1 Message Date
Gian Merlino be25d51a2c RemoteTaskRunner: Fix issues leading to failing tests 2013-12-12 13:49:49 -08:00
Gian Merlino c60158a21a RemoteTaskRunner: Remove task from pendingTaskPayloads on shutdown if needed 2013-12-12 10:59:16 -08:00
Gian Merlino 0129ea99cf RemoteTaskRunner changes to make bootstrapping actually work. 
- Workers are not added to zkWorkers until caches have been initialized.
- Worker status we haven't heard about will be added to runningTasks or
  completeTasks as appropriate. 
- TaskRunnerWorkItem now only needs a taskId, not the entire Task. This makes
  it possible to create them from TaskStatus objects, if that's all we have.
- Also remove some dead code.
 2013-12-12 10:44:46 -08:00
Gian Merlino d92b88718c OverlordResource: Fix comment 2013-12-12 08:46:24 -08:00
Gian Merlino b6a52610bc IndexTask: Call plumber.startJob() 2013-12-12 08:46:10 -08:00
Gian Merlino db9b515e71 IndexTask: Remove unnecessary args to determinePartitions. 2013-12-12 08:46:00 -08:00
Gian Merlino f4a09d4ee3 TaskAction: Add JsonSubType for LockTryAcquireAction 2013-12-12 08:45:23 -08:00
Gian Merlino b17dc6f744 Task interval, isReady hygiene 2013-12-11 22:42:20 -08:00
Gian Merlino 05e24bd85c RemoteTaskRunner: Fix typo 2013-12-11 22:38:04 -08:00
Gian Merlino bed263efa5 VersionConverterTask: Less goofy import for Preconditions 2013-12-11 22:37:55 -08:00
Gian Merlino 53d90efe30 TaskQueueConfig: Copyright header 2013-12-11 22:37:40 -08:00
Gian Merlino 0adda97776 AbstractFixedIntervalTask: Copyright header 2013-12-11 22:37:28 -08:00
Gian Merlino c4b8c8bc6f Rework indexing service internals to hopefully be more reliable. 
The TaskQueue directly manages the TaskRunner. The main management loop runs
periodically and checks that the runner is doing reasonable things. If not, it
attempts to adjust the runner. The management loop also runs on-demand when a
task is added to keep task assignment relatively low latency. The TaskConsumer
is no longer necessary and so it no longer exists.

Task interval locks are handled differently. Instead of some tasks acquiring
locks at runtime and some tasks having implicit fixed lock intervals, all tasks
ask for locks explicitly. This occurs either in "isReady" (which runs on the
overlord) or in "run" (which runs on the peon).

Other changes:
- The TaskQueue is attached to the leader lifecycle, instead of global
- The TaskLockbox is able to sync itself from storage and is no longer
  bootstrapped by the TaskQueue.
- RemoteTaskRunner does not clean up zk paths until asked to. This will
  prevent deletion of statuses that have not yet been committed.
- Added retries on DbTaskStorage operations.
- Removed SpawnTasksAction (no more subtasks)
- Removed obsolete EventReceiverFirehose configs
- Removed obsolete OldOverlordResource
- Removed TaskStorageQueryAdapter methods related to subtasks
 2013-12-11 15:05:16 -08:00
fjy 96f679f31c clean up for merge 2013-12-10 17:51:13 -08:00
Gian Merlino f3cfd1d781 Introduce FileTaskLogs, and move TaskLogs module from server to indexing-service 2013-12-10 17:39:43 -08:00
Gian Merlino 47c1c8cab2 TaskStorage: Rename getRunningTasks -> getActiveTasks 2013-12-10 17:39:42 -08:00
fjy a7cc559623 [maven-release-plugin] prepare for next development iteration 2013-12-09 20:07:38 -08:00
fjy 54755b8864 [maven-release-plugin] prepare release druid-0.6.30 2013-12-09 20:07:34 -08:00
fjy 7a134272b3 [maven-release-plugin] prepare for next development iteration 2013-12-09 18:45:27 -08:00
fjy 829658f171 [maven-release-plugin] prepare release druid-0.6.29 2013-12-09 18:45:23 -08:00
fjy 98a0f40227 [maven-release-plugin] prepare for next development iteration 2013-12-09 18:35:22 -08:00
fjy 939d2313c0 [maven-release-plugin] prepare release druid-0.6.28 2013-12-09 18:35:18 -08:00
fjy c03d573e84 [maven-release-plugin] prepare for next development iteration 2013-12-09 18:30:15 -08:00
fjy 71fe8a43da [maven-release-plugin] prepare release druid-0.6.27 2013-12-09 18:30:11 -08:00
fjy 303f6ff334 fix worker config setup problems 2013-12-09 18:25:29 -08:00
fjy 8aba4ad3d2 Merge pull request #311 from metamx/versions 
Add versions to static page
 2013-12-09 17:18:16 -08:00
nishantmonu51 2186bd6cb7 Minor fixes and documentation changes 2013-12-09 19:07:48 +05:30
fjy 7c7cfdf819 [maven-release-plugin] prepare for next development iteration 2013-12-04 18:57:30 -08:00
fjy 6bfb3b8d41 [maven-release-plugin] prepare release druid-0.6.26 2013-12-04 18:57:25 -08:00
Himadri Singh 47278ad109 Merge relevant changes from 'druid-version-info' 2013-12-04 10:21:19 +05:30
fjy 1a3a1221c2 [maven-release-plugin] prepare for next development iteration 2013-12-02 14:22:54 -08:00
fjy b18b65108a [maven-release-plugin] prepare release druid-0.6.25 2013-12-02 14:22:50 -08:00
fjy 730e3a2303 [maven-release-plugin] prepare for next development iteration 2013-11-25 17:55:01 -08:00
fjy c7171e3311 [maven-release-plugin] prepare release druid-0.6.24 2013-11-25 17:54:57 -08:00
fjy 5c6e647148 [maven-release-plugin] prepare for next development iteration 2013-11-20 12:00:15 -08:00
fjy 5b94895a36 [maven-release-plugin] prepare release druid-0.6.23 2013-11-20 12:00:11 -08:00
fjy ea95566cd9 [maven-release-plugin] prepare for next development iteration 2013-11-20 10:45:21 -08:00
fjy fe7122b717 [maven-release-plugin] prepare release druid-0.6.22 2013-11-20 10:45:17 -08:00
fjy 88378eb036 [maven-release-plugin] prepare for next development iteration 2013-11-20 10:28:57 -08:00
fjy 2e6056f537 [maven-release-plugin] prepare release druid-0.6.21 2013-11-20 10:28:52 -08:00
fjy 7fec5e6ff7 [maven-release-plugin] prepare for next development iteration 2013-11-19 09:59:48 -08:00
fjy 265ef23cd5 [maven-release-plugin] prepare release druid-0.6.20 2013-11-19 09:59:44 -08:00
fjy 5d7173ac98 fix autoscaling termination duration bug 2013-11-15 19:21:25 -08:00
fjy 48d275e5cb [maven-release-plugin] prepare for next development iteration 2013-11-14 18:20:29 -08:00
fjy c31c5ce1ac [maven-release-plugin] prepare release druid-0.6.19 2013-11-14 18:20:25 -08:00
fjy 346cf0e04c fix out of order in urls for hadoop classpath 2013-11-14 18:18:39 -08:00
fjy d935e1b47b [maven-release-plugin] prepare for next development iteration 2013-11-14 15:08:12 -08:00
fjy 32dbbe4f63 [maven-release-plugin] prepare release druid-0.6.18 2013-11-14 15:08:09 -08:00
fjy 6b41681424 fix ordering of urls, ugh, need to write more tests 2013-11-14 15:06:07 -08:00
fjy 373251a476 [maven-release-plugin] prepare for next development iteration 2013-11-14 14:31:26 -08:00