Xavier Léauté
4a291fdf30
better naming
2013-12-16 13:59:16 -08:00
Xavier Léauté
a417cd5df2
add archive task
2013-12-16 13:59:15 -08:00
fjy
3812bf7215
[maven-release-plugin] prepare for next development iteration
2013-12-13 16:57:07 -08:00
fjy
a7ff58f877
[maven-release-plugin] prepare release druid-0.6.36
2013-12-13 16:57:02 -08:00
fjy
87b83bceb1
fix task storage config serde and prepare for next release
2013-12-13 16:55:22 -08:00
fjy
41d64f4aa6
[maven-release-plugin] prepare for next development iteration
2013-12-13 16:47:21 -08:00
fjy
1c050a02f8
[maven-release-plugin] prepare release druid-0.6.35
2013-12-13 16:47:17 -08:00
fjy
01f9c1df31
fix broken task storage config and prepare for next release
2013-12-13 16:45:32 -08:00
fjy
7891e0a2e2
[maven-release-plugin] prepare for next development iteration
2013-12-13 16:20:23 -08:00
fjy
1d75738f81
[maven-release-plugin] prepare release druid-0.6.34
2013-12-13 16:20:18 -08:00
Gian Merlino
600dc7546f
Configurability of recency threshold
2013-12-13 16:02:54 -08:00
fjy
4a8140be81
better messaging to console again
2013-12-13 15:04:25 -08:00
fjy
52cdb20f10
add better messaging and error handling
2013-12-13 15:01:07 -08:00
Gian Merlino
e63c69dd57
TaskStorage: Return recently complete tasks in reverse chronological order
2013-12-13 12:27:45 -08:00
Gian Merlino
6c993d87bf
Indexing service API and GUI improvements!
...
- New APIs: waitingTasks, completeTasks, task payload
- GUI for the above, and for task logs + status
2013-12-13 11:38:18 -08:00
Gian Merlino
f36a5b677c
TaskLifecycleTest: Add test for noop task
2013-12-13 07:48:28 -08:00
Gian Merlino
3b053a66ff
TaskLifecycleTest: Add test for never-ready task
2013-12-13 07:48:27 -08:00
Gian Merlino
863012c384
TaskQueue: Exception during isReady does not warrant an alert.
2013-12-13 07:48:27 -08:00
Gian Merlino
6227963af9
TaskQueue: Copy task list before management loop.
2013-12-13 07:48:27 -08:00
fjy
aef7726c33
[maven-release-plugin] prepare for next development iteration
2013-12-12 18:53:18 -08:00
fjy
1d1baf5976
[maven-release-plugin] prepare release druid-0.6.33
2013-12-12 18:53:14 -08:00
fjy
f920dd6fb2
[maven-release-plugin] prepare for next development iteration
2013-12-12 17:28:38 -08:00
fjy
857d24a4ca
[maven-release-plugin] prepare release druid-0.6.32
2013-12-12 17:28:34 -08:00
fjy
7c744ffda0
[maven-release-plugin] prepare for next development iteration
2013-12-12 14:53:01 -08:00
fjy
365da245c2
[maven-release-plugin] prepare release druid-0.6.31
2013-12-12 14:52:57 -08:00
Gian Merlino
70c153592f
CliPeon: Fix local mode
2013-12-12 14:22:57 -08:00
Gian Merlino
370e2f855a
TaskSerdeTest: Fix IndexTask test by including an actual firehoseFactory
2013-12-12 13:58:44 -08:00
Gian Merlino
169f149cf9
TaskLifecycleTest: Fix broken setUp and broken assumptions.
2013-12-12 13:51:13 -08:00
Gian Merlino
ba757b1e5a
IndexTask: Actually make and publish segments for the correct intervals.
2013-12-12 13:50:53 -08:00
Gian Merlino
be25d51a2c
RemoteTaskRunner: Fix issues leading to failing tests
2013-12-12 13:49:49 -08:00
Gian Merlino
c60158a21a
RemoteTaskRunner: Remove task from pendingTaskPayloads on shutdown if needed
2013-12-12 10:59:16 -08:00
Gian Merlino
0129ea99cf
RemoteTaskRunner changes to make bootstrapping actually work.
...
- Workers are not added to zkWorkers until caches have been initialized.
- Worker status we haven't heard about will be added to runningTasks or
completeTasks as appropriate.
- TaskRunnerWorkItem now only needs a taskId, not the entire Task. This makes
it possible to create them from TaskStatus objects, if that's all we have.
- Also remove some dead code.
2013-12-12 10:44:46 -08:00
Gian Merlino
d92b88718c
OverlordResource: Fix comment
2013-12-12 08:46:24 -08:00
Gian Merlino
b6a52610bc
IndexTask: Call plumber.startJob()
2013-12-12 08:46:10 -08:00
Gian Merlino
db9b515e71
IndexTask: Remove unnecessary args to determinePartitions.
2013-12-12 08:46:00 -08:00
Gian Merlino
f4a09d4ee3
TaskAction: Add JsonSubType for LockTryAcquireAction
2013-12-12 08:45:23 -08:00
Gian Merlino
b17dc6f744
Task interval, isReady hygiene
2013-12-11 22:42:20 -08:00
Gian Merlino
05e24bd85c
RemoteTaskRunner: Fix typo
2013-12-11 22:38:04 -08:00
Gian Merlino
bed263efa5
VersionConverterTask: Less goofy import for Preconditions
2013-12-11 22:37:55 -08:00
Gian Merlino
53d90efe30
TaskQueueConfig: Copyright header
2013-12-11 22:37:40 -08:00
Gian Merlino
0adda97776
AbstractFixedIntervalTask: Copyright header
2013-12-11 22:37:28 -08:00
Gian Merlino
c4b8c8bc6f
Rework indexing service internals to hopefully be more reliable.
...
The TaskQueue directly manages the TaskRunner. The main management loop runs
periodically and checks that the runner is doing reasonable things. If not, it
attempts to adjust the runner. The management loop also runs on-demand when a
task is added to keep task assignment relatively low latency. The TaskConsumer
is no longer necessary and so it no longer exists.
Task interval locks are handled differently. Instead of some tasks acquiring
locks at runtime and some tasks having implicit fixed lock intervals, all tasks
ask for locks explicitly. This occurs either in "isReady" (which runs on the
overlord) or in "run" (which runs on the peon).
Other changes:
- The TaskQueue is attached to the leader lifecycle, instead of global
- The TaskLockbox is able to sync itself from storage and is no longer
bootstrapped by the TaskQueue.
- RemoteTaskRunner does not clean up zk paths until asked to. This will
prevent deletion of statuses that have not yet been committed.
- Added retries on DbTaskStorage operations.
- Removed SpawnTasksAction (no more subtasks)
- Removed obsolete EventReceiverFirehose configs
- Removed obsolete OldOverlordResource
- Removed TaskStorageQueryAdapter methods related to subtasks
2013-12-11 15:05:16 -08:00
fjy
96f679f31c
clean up for merge
2013-12-10 17:51:13 -08:00
Gian Merlino
f3cfd1d781
Introduce FileTaskLogs, and move TaskLogs module from server to indexing-service
2013-12-10 17:39:43 -08:00
Gian Merlino
47c1c8cab2
TaskStorage: Rename getRunningTasks -> getActiveTasks
2013-12-10 17:39:42 -08:00
fjy
a7cc559623
[maven-release-plugin] prepare for next development iteration
2013-12-09 20:07:38 -08:00
fjy
54755b8864
[maven-release-plugin] prepare release druid-0.6.30
2013-12-09 20:07:34 -08:00
fjy
7a134272b3
[maven-release-plugin] prepare for next development iteration
2013-12-09 18:45:27 -08:00
fjy
829658f171
[maven-release-plugin] prepare release druid-0.6.29
2013-12-09 18:45:23 -08:00
fjy
98a0f40227
[maven-release-plugin] prepare for next development iteration
2013-12-09 18:35:22 -08:00
