fjy
1ecc9d0f98
fix the edge case where autoscaling tries to terminate node without ip
2014-01-20 16:44:19 -08:00
fjy
c1f0027a36
Merge pull request #361 from metamx/worker-oome-fix
...
Worker oome fix
2014-01-20 11:34:45 -08:00
fjy
57e022f446
[maven-release-plugin] prepare for next development iteration
2014-01-17 15:53:54 -08:00
fjy
6540b2a58e
[maven-release-plugin] prepare release druid-0.6.52
2014-01-17 15:53:50 -08:00
nishantmonu51
3fb72aff93
rename maxPendingPersistBatches to maxPendingPersists
2014-01-17 12:59:21 +05:30
nishantmonu51
fb819abd6f
make maxPendingPersistPeriod configurable
2014-01-17 11:01:55 +05:30
fjy
26a0f862fb
[maven-release-plugin] prepare for next development iteration
2014-01-14 17:35:45 -08:00
fjy
3737ac4966
[maven-release-plugin] prepare release druid-0.6.51
2014-01-14 17:35:40 -08:00
fjy
528efedf52
[maven-release-plugin] prepare for next development iteration
2014-01-14 16:08:47 -08:00
fjy
b4e4a5df96
[maven-release-plugin] prepare release druid-0.6.50
2014-01-14 16:08:43 -08:00
fjy
4a9c255cde
[maven-release-plugin] prepare for next development iteration
2014-01-14 09:56:34 -08:00
fjy
d8ac5575bc
[maven-release-plugin] prepare release druid-0.6.49
2014-01-14 09:56:30 -08:00
fjy
396311a547
[maven-release-plugin] prepare for next development iteration
2014-01-13 18:04:29 -08:00
fjy
9c9d6287a2
[maven-release-plugin] prepare release druid-0.6.48
2014-01-13 18:04:24 -08:00
Gian Merlino
1331f2ce56
TaskStorage.add() now throws TaskExistsException, and the servlet respects it
...
The servlet will throw 400 rather than 500 when a task already exists, to
signify that the request has no hope of ever working.
2014-01-13 15:42:05 -08:00
Gian Merlino
a72c4429f7
RemoteTaskRunner: Fix NPE on cleanup due to missing withWorker
2014-01-13 15:41:04 -08:00
fjy
f3476f40e1
fix typo
2014-01-10 18:08:33 -08:00
fjy
f4e3f02c3b
more exceptions
2014-01-10 18:06:42 -08:00
fjy
1ecc94cfb6
another attempt at index task
2014-01-10 17:56:22 -08:00
fjy
f0b4d0c1e4
fix small bug with unusable dims
2014-01-10 14:59:09 -08:00
fjy
fe50104053
fix the index task and more docs
2014-01-10 14:47:18 -08:00
nishantmonu51
da01c4a78a
Add registration for backward compatibility
2014-01-10 02:02:06 +05:30
nishantmonu51
d28f9daccb
Remove duplicate registration of service
...
If the serviceName does not contain ":" this leads to duplicate
registration of firehose with same name
2014-01-10 01:17:41 +05:30
Gian Merlino
9037141c00
IndexTask: Better logging at the end of each segment
2014-01-08 15:22:12 -08:00
Gian Merlino
2c53af4d66
ForkingTaskRunner: Upload task logs even when job fails
2014-01-08 14:46:18 -08:00
Gian Merlino
7f430d9fde
RealtimeIndexTask: If a Throwable was thrown it is not a normalExit
2014-01-08 14:45:35 -08:00
Gian Merlino
83b4641e31
ForkingTaskRunnerConfig: Add java.io.tmpdir to allowedPrefixes
2014-01-07 16:12:24 -08:00
Gian Merlino
bf158102c4
IndexTask: Print metrics even if finishJob fails
2014-01-07 07:17:19 -08:00
fjy
2b46c1d292
[maven-release-plugin] prepare for next development iteration
2014-01-02 19:53:41 -08:00
fjy
7a0f634b02
[maven-release-plugin] prepare release druid-0.6.47
2014-01-02 19:53:35 -08:00
Xavier Léauté
ef05312c97
cleanup dependencies
2013-12-30 16:31:12 -08:00
Gian Merlino
668cf009a0
[maven-release-plugin] prepare for next development iteration
2013-12-20 12:28:54 -08:00
Gian Merlino
bd58def933
[maven-release-plugin] prepare release druid-0.6.46
2013-12-20 12:28:49 -08:00
Gian Merlino
26991b5a2a
Indexing service: Fix termination related log message
2013-12-20 12:05:42 -08:00
Gian Merlino
4d83837e88
RealtimeIndexTask: Clean up imports and comments
2013-12-20 11:37:16 -08:00
Gian Merlino
9043c211b2
[maven-release-plugin] prepare for next development iteration
2013-12-20 11:25:36 -08:00
Gian Merlino
29c0fa1df9
[maven-release-plugin] prepare release druid-0.6.45
2013-12-20 11:25:31 -08:00
Gian Merlino
17ad4ee2f0
Fix RemoteTaskRunnerTest
2013-12-20 11:23:28 -08:00
Gian Merlino
5711ac4aff
[maven-release-plugin] prepare for next development iteration
2013-12-20 11:17:01 -08:00
Gian Merlino
3b15f6a834
[maven-release-plugin] prepare release druid-0.6.44
2013-12-20 11:17:01 -08:00
Gian Merlino
e5b8546d19
Autoscaling fixes.
...
- Initial targetWorkerCount must be subject to pool size limits
- Use consistent workerSetupData for the entire autoscaling run
- Don't call terminate() when we have nothing to terminate
- Terminate obsolete workers even faster
2013-12-20 11:17:01 -08:00
Gian Merlino
c503f5e9c5
[maven-release-plugin] prepare for next development iteration
2013-12-20 11:17:01 -08:00
Gian Merlino
97095ee3db
[maven-release-plugin] prepare release druid-0.6.43
2013-12-20 11:17:00 -08:00
fjy
3ec2766cd3
Merge pull request #339 from metamx/autoscaling
...
Autoscaling: Move target count independent of actual count.
2013-12-20 10:04:26 -08:00
Gian Merlino
6224577ed1
Autoscaling: Terminate obsolete workers faster
2013-12-20 10:01:32 -08:00
Gian Merlino
4a722c0a6d
Autoscaling changes from code review.
...
- Log and return immediately when workerSetupData is null
- Allow provisioning more nodes while other nodes are still provisioning
- Add tests for bumping up the minimum version
2013-12-20 08:59:35 -08:00
Gian Merlino
0ee6136ea3
NoopTask: Fix things that should be static. Add simple factory method.
2013-12-20 08:56:17 -08:00
Gian Merlino
3dd9a25546
Fix import
2013-12-19 16:18:16 -08:00
Gian Merlino
0ff7f0e8e0
TaskActionToolbox: Combine adjacent ifs
2013-12-19 16:16:34 -08:00
Gian Merlino
f86342f7dc
DbTaskStorage: Protect against invalid lock_payload
2013-12-19 16:16:20 -08:00
Gian Merlino
1f4b99634f
Autoscaling: Move target count independent of actual count.
...
This should let us grow and shrink the worker pool in chunks when necessary
(like when a bunch of them go offline, or when there is a worker version
change).
2013-12-19 16:11:30 -08:00
Gian Merlino
846c3da4ab
Empty task intervals, and empty lock intervals, aren't useful.
...
So prevent them from being created, through checks in AbstractFixedIntervalTask
and TaskLockbox.tryLock.
2013-12-19 13:21:41 -08:00
Gian Merlino
566a3a6112
Indexing service: Break up segment actions
...
Each one now one operates on at most a collection of segments that comprise
a single partition. The main purpose of this change is to prevent audit log
payload sizes from getting out of control.
2013-12-19 13:10:40 -08:00
Gian Merlino
6fbe67eeea
IndexerDBCoordinator: Work around SELECT -> INSERT races when adding segments
2013-12-19 13:10:40 -08:00
Gian Merlino
25c3f28907
[maven-release-plugin] prepare for next development iteration
2013-12-18 18:38:27 -08:00
Gian Merlino
6a740768d6
[maven-release-plugin] prepare release druid-0.6.42
2013-12-18 18:38:22 -08:00
Gian Merlino
8d8ec51c9c
[maven-release-plugin] prepare for next development iteration
2013-12-18 16:05:33 -08:00
Gian Merlino
00d651bf6f
[maven-release-plugin] prepare release druid-0.6.41
2013-12-18 16:05:27 -08:00
Gian Merlino
d0fd58bbae
[maven-release-plugin] prepare for next development iteration
2013-12-18 15:35:55 -08:00
Gian Merlino
231d7a6d1f
[maven-release-plugin] prepare release druid-0.6.40
2013-12-18 15:35:50 -08:00
Gian Merlino
1ff855d744
Fix MoveTask serde and ArchiveTask id creation
2013-12-18 15:17:12 -08:00
fjy
494ec53050
[maven-release-plugin] prepare for next development iteration
2013-12-17 17:48:46 -08:00
fjy
2a98a4d3e0
[maven-release-plugin] prepare release druid-0.6.39
2013-12-17 17:48:42 -08:00
fjy
5932150d24
[maven-release-plugin] prepare for next development iteration
2013-12-17 16:29:53 -08:00
fjy
ed9f4b4cf2
[maven-release-plugin] prepare release druid-0.6.38
2013-12-17 16:29:49 -08:00
Gian Merlino
58d1262edf
Indexing console: Clarify "Complete" with "recently completed"
2013-12-17 08:16:49 -08:00
fjy
48d677d135
[maven-release-plugin] prepare for next development iteration
2013-12-16 16:17:45 -08:00
fjy
7e0b4d4fd2
[maven-release-plugin] prepare release druid-0.6.37
2013-12-16 16:17:41 -08:00
Xavier Léauté
e333776aca
rename SegmentMoveAction to SegmentMetadataUpdateAction
2013-12-16 14:00:56 -08:00
Xavier Léauté
ac2ca0e46c
separate move and archive tasks
2013-12-16 14:00:55 -08:00
Xavier Léauté
123bddd615
update for new interfaces
2013-12-16 13:59:16 -08:00
Xavier Léauté
4a291fdf30
better naming
2013-12-16 13:59:16 -08:00
Xavier Léauté
a417cd5df2
add archive task
2013-12-16 13:59:15 -08:00
fjy
3812bf7215
[maven-release-plugin] prepare for next development iteration
2013-12-13 16:57:07 -08:00
fjy
a7ff58f877
[maven-release-plugin] prepare release druid-0.6.36
2013-12-13 16:57:02 -08:00
fjy
87b83bceb1
fix task storage config serde and prepare for next release
2013-12-13 16:55:22 -08:00
fjy
41d64f4aa6
[maven-release-plugin] prepare for next development iteration
2013-12-13 16:47:21 -08:00
fjy
1c050a02f8
[maven-release-plugin] prepare release druid-0.6.35
2013-12-13 16:47:17 -08:00
fjy
01f9c1df31
fix broken task storage config and prepare for next release
2013-12-13 16:45:32 -08:00
fjy
7891e0a2e2
[maven-release-plugin] prepare for next development iteration
2013-12-13 16:20:23 -08:00
fjy
1d75738f81
[maven-release-plugin] prepare release druid-0.6.34
2013-12-13 16:20:18 -08:00
Gian Merlino
600dc7546f
Configurability of recency threshold
2013-12-13 16:02:54 -08:00
fjy
4a8140be81
better messaging to console again
2013-12-13 15:04:25 -08:00
fjy
52cdb20f10
add better messaging and error handling
2013-12-13 15:01:07 -08:00
Gian Merlino
e63c69dd57
TaskStorage: Return recently complete tasks in reverse chronological order
2013-12-13 12:27:45 -08:00
Gian Merlino
6c993d87bf
Indexing service API and GUI improvements!
...
- New APIs: waitingTasks, completeTasks, task payload
- GUI for the above, and for task logs + status
2013-12-13 11:38:18 -08:00
Gian Merlino
f36a5b677c
TaskLifecycleTest: Add test for noop task
2013-12-13 07:48:28 -08:00
Gian Merlino
3b053a66ff
TaskLifecycleTest: Add test for never-ready task
2013-12-13 07:48:27 -08:00
Gian Merlino
863012c384
TaskQueue: Exception during isReady does not warrant an alert.
2013-12-13 07:48:27 -08:00
Gian Merlino
6227963af9
TaskQueue: Copy task list before management loop.
2013-12-13 07:48:27 -08:00
fjy
aef7726c33
[maven-release-plugin] prepare for next development iteration
2013-12-12 18:53:18 -08:00
fjy
1d1baf5976
[maven-release-plugin] prepare release druid-0.6.33
2013-12-12 18:53:14 -08:00
fjy
f920dd6fb2
[maven-release-plugin] prepare for next development iteration
2013-12-12 17:28:38 -08:00
fjy
857d24a4ca
[maven-release-plugin] prepare release druid-0.6.32
2013-12-12 17:28:34 -08:00
fjy
7c744ffda0
[maven-release-plugin] prepare for next development iteration
2013-12-12 14:53:01 -08:00
fjy
365da245c2
[maven-release-plugin] prepare release druid-0.6.31
2013-12-12 14:52:57 -08:00
Gian Merlino
70c153592f
CliPeon: Fix local mode
2013-12-12 14:22:57 -08:00
Gian Merlino
370e2f855a
TaskSerdeTest: Fix IndexTask test by including an actual firehoseFactory
2013-12-12 13:58:44 -08:00
Gian Merlino
169f149cf9
TaskLifecycleTest: Fix broken setUp and broken assumptions.
2013-12-12 13:51:13 -08:00
Gian Merlino
ba757b1e5a
IndexTask: Actually make and publish segments for the correct intervals.
2013-12-12 13:50:53 -08:00