Commit Graph

1452 Commits

Author SHA1 Message Date
fjy db695f49c2 [maven-release-plugin] prepare for next development iteration 2013-08-05 18:26:20 -07:00
fjy c22e5b7ee0 [maven-release-plugin] prepare release druid-0.5.27 2013-08-05 18:26:10 -07:00
fjy d1b2a5a4b3 fix indexer console serde of running tasks 2013-08-05 18:22:12 -07:00
fjy 8ff0b53df4 [maven-release-plugin] prepare for next development iteration 2013-08-05 18:00:46 -07:00
fjy 1d3729906d [maven-release-plugin] prepare release druid-0.5.26 2013-08-05 18:00:36 -07:00
fjy 479f0cefca fix bug with RTR not assigning tasks when a new worker is available 2013-08-05 17:57:59 -07:00
fjy 90e082463b [maven-release-plugin] prepare for next development iteration 2013-08-05 16:10:19 -07:00
fjy eb276d34d7 [maven-release-plugin] prepare release druid-0.5.25 2013-08-05 16:10:11 -07:00
fjy 626cf14a6e fix bug where the curator config name was changed in one place but not another; make some info msgs into debug msgs; fix zkworker serialization 2013-08-05 16:02:26 -07:00
fjy a95d9c46e2 [maven-release-plugin] prepare for next development iteration 2013-08-05 14:47:35 -07:00
fjy 8e1f4d3256 [maven-release-plugin] prepare release druid-0.5.24 2013-08-05 14:47:26 -07:00
fjy 66c658305f Merge branch 'master' of github.com:metamx/druid 2013-08-05 14:44:09 -07:00
fjy 35f89d7232 make RTR idempotent to multiple run requests for same task, because higher level things in the indexing service require this behaviour 2013-08-05 14:44:01 -07:00
Gian Merlino efd34f3a8b TaskRunner: Fix comment 2013-08-05 14:20:31 -07:00
fjy fbb1211cbc [maven-release-plugin] prepare for next development iteration 2013-08-05 11:44:36 -07:00
fjy 22a90d36b7 [maven-release-plugin] prepare release druid-0.5.23 2013-08-05 11:44:28 -07:00
cheddar 3c808b15c3 1) Fix HadoopDruidIndexerConfigTest to actually verify the current correct behavior. 2013-08-05 11:37:20 -07:00
cheddar d6ac75cff5 Merge branch 'master' of github.com:metamx/druid 2013-08-05 11:22:37 -07:00
cheddar 2b71505421 1) Fix HadoopDruidIndexerConfig to no longer replace ":" with "_" on the segmentOutputDir. The segmentOutputDir is user-supplied so they should have the ability to just not set a bad directory. 2013-08-05 11:22:26 -07:00
cheddar e1d65e736c Merge pull request #203 from metamx/worker-resource
Remove retry logic out of the Remote Task Runner and introduce Task resources
2013-08-05 11:11:21 -07:00
fjy c33f2f06ff fix logic of how to assign tasks to workers 2013-08-02 09:01:02 -07:00
fjy 584ccac833 move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better 2013-08-01 17:50:05 -07:00
cheddar fba2bc9623 Merge pull request #204 from metamx/close-handle
add the ability to unmap mmapped files
2013-08-01 14:09:46 -07:00
fjy 8f39db7804 more data to exception msgs 2013-08-01 14:04:27 -07:00
fjy 6d54ae9f81 more unit tests to test for multiple closes in different threads 2013-08-01 13:51:56 -07:00
fjy dc169d9fcd hide decrement and update close logic to call decrement 2013-08-01 13:24:56 -07:00
fjy e92e0c35a4 put all concurrency test back into server manager test, introduce better concurrency for reference counting 2013-08-01 13:13:03 -07:00
fjy b86258cc14 fix broken master UT based on changes that were discarded as part of a pull req 2013-08-01 10:44:39 -07:00
fjy 2f8351b314 fix broken ut 2013-08-01 09:41:36 -07:00
fjy a4edc2221d fix RTR comments 2013-07-31 15:28:52 -07:00
fjy 215d147a69 Merge branch 'worker-resource' of github.com:metamx/druid into worker-resource 2013-07-31 15:23:49 -07:00
fjy e2b5cd6067 Merge branch 'master' of github.com:metamx/druid into worker-resource
Conflicts:
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
2013-07-31 15:23:13 -07:00
fjy f76540c16c move reference counting classes out of server manager and more tests for concurrency 2013-07-31 15:14:39 -07:00
fjy 5d1fe50703 additional ut for reference counting 2013-07-30 17:55:57 -07:00
fjy 404b84823e reference count to make sure we arent accessing any instances of a closed mmapped segment 2013-07-30 17:36:56 -07:00
Gian Merlino eaddce06d5 Call TaskRunner.bootstrap immediately after starting it 2013-07-30 15:26:11 -07:00
cheddar 843b6650a2 Merge pull request #206 from metamx/peontimeout
Add a timeout in LoadQueuePeon to fail an assign if it takes too long
2013-07-30 15:08:46 -07:00
fjy e7c6dd8112 get and reset failed assign count 2013-07-30 14:50:24 -07:00
fjy 35d8a82879 move the close method to the segment object 2013-07-30 12:24:01 -07:00
cheddar 1c5c2e0ef5 Merge pull request #205 from metamx/compress-curator
Toggle between compressed and non compressed service discovery
2013-07-30 11:33:43 -07:00
cheddar 66adef5d64 Merge pull request #179 from zanox/master
Fix path used when storing segments to HDFS
2013-07-30 11:13:29 -07:00
fjy f55e12040f fix according to code review 2013-07-29 18:39:14 -07:00
fjy 6a96c1fb76 fix according to code review comments 2013-07-29 17:56:19 -07:00
fjy af6db21264 fix configs based on code review 2013-07-29 17:04:27 -07:00
fjy 091dce11c8 Add a timeout in LQP to fail an assign if it takes too long 2013-07-29 15:47:05 -07:00
fjy 50836798fa toggle between compressed and non compressed service discovery 2013-07-29 15:40:45 -07:00
fjy 1226190279 add the ability to close mmapped files 2013-07-29 15:22:34 -07:00
fjy ad65c8111d fix logs 2013-07-29 11:41:42 -07:00
fjy 4ae8395538 1) on bootstrap, load all initial data and do a compare with bootstrapped tasks, delete any that are extra out there
2) change autoscaling logic such that it only works with remote task runnrs
3) zk workers use their own status caches to determine what they are running
2013-07-26 14:32:08 -07:00
fjy a1262760b2 Merge branch 'master' into worker-resource 2013-07-26 10:02:15 -07:00