Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,268 Commits 113 Branches 506 Tags 732 MiB
Commit Graph

1041 Commits

Author SHA1 Message Date
cheddar c02d887cfe 1) Rewrite SearchQueryRunner to not require StorageAdapter to be "Searchable" 
2) Extract SearchQueryRunner out of SearchQueryRunnerFactory
3) Extract ColumnSelectorBitmapIndexSelector out to make it reusable
 2013-08-28 18:50:40 -05:00
cheddar 1f37e962f6 1) Delete the files that have moved to druid-api and get them through the druid-api dependency. Still need to fix Searchable on StorageAdapter, but other than that, everything should compile. 2013-08-28 16:18:40 -05:00
cheddar 9c30ced5ea 1) Move various "api" classes to io.druid packages and make sure things compile and stuff 2013-08-28 15:51:02 -05:00
cheddar fc60158273 1) What is an .Rhistory file and why is it checked in? It's not! (anymore) 2013-08-27 14:37:12 -05:00
cheddar ee1e73cfa1 1) Make it compile again after the merge 2013-08-27 14:36:01 -05:00
cheddar 8097450d8c Some things that didn't get committed with the merge for some reason!? 2013-08-27 14:29:03 -05:00
cheddar 5fa944dd26 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
	client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
	client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	pom.xml
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/http/MasterServletModule.java
	server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
	server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
	server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
 2013-08-27 14:27:32 -05:00
cheddar 3617ac17fc 1) Eliminate ExecutorMain and have it run using the new Main! 2013-08-27 14:11:05 -05:00
cheddar 269997dc94 1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful 2013-08-26 18:08:41 -05:00
cheddar 55dbda2046 1) Worker appears to be running! It's also now known as the MiddleManager 2013-08-23 17:59:48 -05:00
cheddar b897c2cb22 1) IndexCoordinator appears to work as the CliOverlord now, yay! 2013-08-23 14:11:34 -05:00
cheddar 344ec1ec2f [maven-release-plugin] prepare for next development iteration 2013-08-23 13:20:09 -05:00
cheddar 5b0de6eb5c [maven-release-plugin] prepare release druid-0.5.48 2013-08-23 13:20:02 -05:00
fjy 6a8c160740 update code according to code review 2013-08-22 10:46:05 -07:00
fjy 3f51ef63a4 Merge branch 'guice' into guice-rt 2013-08-22 10:20:53 -07:00
fjy 4102dbd1e0 [maven-release-plugin] prepare for next development iteration 2013-08-21 21:50:55 -07:00
fjy 0daa2a95fe [maven-release-plugin] prepare release druid-0.5.47 2013-08-21 21:50:46 -07:00
fjy b1d0f989ec [maven-release-plugin] prepare for next development iteration 2013-08-21 20:28:15 -07:00
fjy ab467bf42a [maven-release-plugin] prepare release druid-0.5.46 2013-08-21 20:28:06 -07:00
fjy 1037a48a8e [maven-release-plugin] prepare for next development iteration 2013-08-21 18:55:49 -07:00
fjy 6b9ec95f01 [maven-release-plugin] prepare release druid-0.5.45 2013-08-21 18:55:40 -07:00
fjy ed666d9d5f [maven-release-plugin] prepare for next development iteration 2013-08-20 19:37:52 -07:00
fjy 66dcfe8c3e [maven-release-plugin] prepare release druid-0.5.44 2013-08-20 19:37:44 -07:00
fjy baedacf970 [maven-release-plugin] prepare for next development iteration 2013-08-20 19:29:05 -07:00
fjy a4eb025ba7 [maven-release-plugin] prepare release druid-0.5.43 2013-08-20 19:28:56 -07:00
Gian Merlino a3793eeba4 [maven-release-plugin] prepare for next development iteration 2013-08-20 17:58:28 -07:00
Gian Merlino b792bf2ba5 [maven-release-plugin] prepare release druid-0.5.42 2013-08-20 17:58:22 -07:00
fjy 47ec240703 [maven-release-plugin] prepare for next development iteration 2013-08-15 13:47:57 -07:00
fjy 33e2fb3e21 [maven-release-plugin] prepare release druid-0.5.41 2013-08-15 13:47:48 -07:00
fjy 1fb6107a37 fix the case where RTR does not clean up a completed task on startup 2013-08-15 13:09:02 -07:00
cheddar 0c5a906a1b Merge branch 'master' of github.com:metamx/druid 2013-08-14 17:34:22 -07:00
cheddar ba5ed4c9d4 1) Fix bug with NPE thrown when requesting a dimension on a GroupBy query that is not dictionary encoded 2013-08-14 17:34:11 -07:00
Gian Merlino a0fcd02b3c [maven-release-plugin] prepare for next development iteration 2013-08-14 17:03:40 -07:00
Gian Merlino b6c445c726 [maven-release-plugin] prepare release druid-0.5.40 2013-08-14 17:03:28 -07:00
fjy 36d02ab943 [maven-release-plugin] prepare for next development iteration 2013-08-14 14:05:49 -07:00
fjy 8c841e90a4 [maven-release-plugin] prepare release druid-0.5.39 2013-08-14 14:05:39 -07:00
fjy fc1018bc3f revert removal of master loadstatus 2013-08-14 14:02:47 -07:00
fjy 761a411b16 [maven-release-plugin] prepare for next development iteration 2013-08-14 10:13:50 -07:00
fjy 4dd727d974 [maven-release-plugin] prepare release druid-0.5.38 2013-08-14 10:13:34 -07:00
cheddar c0624d9591 Merge branch 'master' of github.com:metamx/druid 2013-08-14 10:04:04 -07:00
cheddar a7ef5b8b70 1) Fix bug with SingleSegmentLoader.StorageLocation keeping track of its storage size incorrectly. Add unit test ftw. 2013-08-14 10:03:47 -07:00
cheddar 7ebe053ac1 Fix bug with segment loader counting thingie 2013-08-14 09:37:22 -07:00
cheddar 4791d0466d Merge pull request #198 from metamx/loadBalancer 
Improve runtime of segment distribution algorithm
 2013-08-13 18:47:43 -07:00
cheddar c2305a82f9 Merge pull request #209 from zanox/kafka-protobuf 
Kafka protobuf
 2013-08-13 18:38:27 -07:00
fjy 9870cab855 [maven-release-plugin] prepare for next development iteration 2013-08-13 17:21:56 -07:00
fjy e4c998cfef [maven-release-plugin] prepare release druid-0.5.37 2013-08-13 17:21:46 -07:00
fjy 85ee8bb267 port realtime to guice 2013-08-13 17:08:45 -07:00
cheddar 145e08682c 1) Add check in ServerManagerTest to make sure that the Segment has been "checked out" before the factory ever sees it. 
2) Some code readability changes to ReferenceCountingSegment
 2013-08-13 17:01:24 -07:00
cheddar a5855fb749 1) Fix ServerManager to make sure that it "checks out" the Segment before it starts processing 2013-08-13 14:51:14 -07:00
cheddar 28c7e78a91 1) Fix ReferenceCountingSegmentTest to actually work again 2013-08-13 14:28:44 -07:00
fjy 41347c82bb fix broken server manager tests 2013-08-13 12:07:45 -07:00
fjy 070d822345 fix broken UT 2013-08-13 12:01:24 -07:00
fjy 08318af7f7 Merge branch 'master' of github.com:metamx/druid 2013-08-13 11:54:32 -07:00
fjy 13e522c8a7 update the server manager unit tests to check for the case where a segment may be accessed in toYield calls before an increment occurs 2013-08-13 11:53:41 -07:00
Gian Merlino 246bd0fbaa [maven-release-plugin] prepare for next development iteration 2013-08-13 07:27:24 -07:00
Gian Merlino a2e9441693 [maven-release-plugin] prepare release druid-0.5.36 2013-08-13 07:27:18 -07:00
fjy d4fc069e01 [maven-release-plugin] prepare for next development iteration 2013-08-12 18:58:14 -07:00
fjy dd8c96c734 [maven-release-plugin] prepare release druid-0.5.35 2013-08-12 18:58:06 -07:00
cheddar bb18b162ab 1) Include new version of java-util to fix issue with resources not getting closed when an exception is thrown sometimes 
2) Whitespace changes to GroupByQueryEngine
 2013-08-12 17:57:30 -07:00
Gian Merlino 8d7a4f4493 Retries for S3TaskLogs, S3DataSegmentPusher 2013-08-12 14:27:34 -07:00
fjy 1f4e0ea40c [maven-release-plugin] prepare for next development iteration 2013-08-12 13:21:29 -07:00
fjy 1c2ad4f08c [maven-release-plugin] prepare release druid-0.5.34 2013-08-12 13:21:21 -07:00
fjy a39eb65edb change summary of segment failures to one per line 2013-08-12 13:15:35 -07:00
fjy dd20950f8a adding missing continue 2013-08-12 13:07:29 -07:00
fjy 2ec2957207 clean up the error reporting code according to code review 2013-08-12 13:04:48 -07:00
fjy 0c360c05c2 better handling of exceptions during startup 2013-08-12 12:56:59 -07:00
fjy 2f728f3478 fix zk coord to not fail during startup exceptions 2013-08-12 11:33:42 -07:00
Jan Rudert 3f32ef7bbc Merge branch 'master' of github.com:metamx/druid into kafka-protobuf 2013-08-12 17:42:40 +02:00
jan 55267ca0e4 removed @author tags 2013-08-11 13:44:13 +02:00
Dhruv Parthasarathy 01db89ac6e fixed issue with web console. Cleaned up code 2013-08-09 00:33:40 -07:00
fjy 68a3f1ab79 [maven-release-plugin] prepare for next development iteration 2013-08-08 22:15:04 -07:00
fjy ce5679554d [maven-release-plugin] prepare release druid-0.5.33 2013-08-08 22:14:55 -07:00
fjy a190269faa fix druid master segments getting stuck bug 2013-08-08 22:06:41 -07:00
Dhruv Parthasarathy fc5e15f72b merged changes with master 2013-08-08 18:11:33 -07:00
Dhruv Parthasarathy abe347efd3 merged with master 2013-08-08 17:21:57 -07:00
Dhruv Parthasarathy 764863d6f5 made volatile. fixed parens 2013-08-08 16:40:10 -07:00
Dhruv Parthasarathy 8a8ea5d028 removed test annotation from profiler 2013-08-08 16:38:24 -07:00
Dhruv Parthasarathy 01b35ec819 all tests passed. Ready for merge 2013-08-08 16:23:26 -07:00
Dhruv Parthasarathy eed9e1b373 merged with master 2013-08-08 16:14:06 -07:00
Dhruv Parthasarathy 0b24ea710c fixed paths 2013-08-08 16:10:31 -07:00
Dhruv Parthasarathy 3a298681e4 fixed formatting 2013-08-07 10:59:54 -07:00
Dhruv Parthasarathy c4f0e4d96d fixed with Erics feedback 2013-08-07 10:53:37 -07:00
fjy ddc2e87c4c [maven-release-plugin] prepare for next development iteration 2013-08-06 21:21:28 -07:00
fjy ffbcd18cdc [maven-release-plugin] prepare release druid-0.5.32 2013-08-06 21:21:18 -07:00
fjy 4617d37e28 [maven-release-plugin] prepare for next development iteration 2013-08-06 21:08:02 -07:00
fjy 53b248dc28 [maven-release-plugin] prepare release druid-0.5.31 2013-08-06 21:07:52 -07:00
Gian Merlino 99a3de1597 [maven-release-plugin] prepare for next development iteration 2013-08-06 17:41:58 -07:00
Gian Merlino c0ccfd66ea [maven-release-plugin] prepare release druid-0.5.30 2013-08-06 17:41:51 -07:00
fjy 930b089f7f [maven-release-plugin] prepare for next development iteration 2013-08-06 16:20:37 -07:00
fjy dd5f5ae49b [maven-release-plugin] prepare release druid-0.5.29 2013-08-06 16:20:29 -07:00
fjy 5f292143e0 disable merging segments with linear shard spec 2013-08-06 16:17:49 -07:00
Dhruv Parthasarathy f31ee552d0 fixed with fj comments 2013-08-06 15:03:07 -07:00
Dhruv Parthasarathy 0d5b222f8f removed SlickGrid :( 2013-08-06 14:30:35 -07:00
Dhruv Parthasarathy 6124ecd9cb fied some tests 2013-08-06 14:26:08 -07:00
cheddar eee1efdcb5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
 2013-08-06 13:33:31 -07:00
cheddar 51f648ec11 1) Rename QueryToolChestModule back 2013-08-06 12:05:56 -07:00
cheddar 69f7831d82 1) Rename QueryToolChestModule because there was a case sensitivity issue, grrr 2013-08-06 12:04:04 -07:00
cheddar 9f71d42314 1) Broker works with Guice 
2) Extract ServerViewModule to handle various ServerView thingies
3) Extract QueryToolChestModule and QueryRunnerFactoryModule to reuse code for configuration of Query stuff
4) Extract QueryJettyServerInitializer to reuse between Historical and Broker nodes
5) Remove ClientMain, BrokerMain and BrokerNode
 2013-08-06 12:01:52 -07:00
Dhruv Parthasarathy ebba2d66a3 results good 2013-08-06 11:46:18 -07:00
Dhruv Parthasarathy e40d5b55ef testing speed for one run 2013-08-06 11:40:49 -07:00
Gian Merlino ba847bff32 [maven-release-plugin] prepare for next development iteration 2013-08-05 19:58:40 -07:00
Gian Merlino 7101d9ad2e [maven-release-plugin] prepare release druid-0.5.28 2013-08-05 19:58:34 -07:00
fjy db695f49c2 [maven-release-plugin] prepare for next development iteration 2013-08-05 18:26:20 -07:00
fjy c22e5b7ee0 [maven-release-plugin] prepare release druid-0.5.27 2013-08-05 18:26:10 -07:00
fjy 8ff0b53df4 [maven-release-plugin] prepare for next development iteration 2013-08-05 18:00:46 -07:00
fjy 1d3729906d [maven-release-plugin] prepare release druid-0.5.26 2013-08-05 18:00:36 -07:00
fjy 90e082463b [maven-release-plugin] prepare for next development iteration 2013-08-05 16:10:19 -07:00
fjy eb276d34d7 [maven-release-plugin] prepare release druid-0.5.25 2013-08-05 16:10:11 -07:00
Dhruv Parthasarathy 575dfe3679 replication working too 2013-08-05 14:51:58 -07:00
fjy a95d9c46e2 [maven-release-plugin] prepare for next development iteration 2013-08-05 14:47:35 -07:00
fjy 8e1f4d3256 [maven-release-plugin] prepare release druid-0.5.24 2013-08-05 14:47:26 -07:00
Dhruv Parthasarathy 1045aac4b8 finished profiling here 2013-08-05 13:53:40 -07:00
cheddar d64bc2f762 1) Eliminate Old mains() that aren't needed anymore. 2013-08-05 13:33:43 -07:00
Dhruv Parthasarathy 540c4885d7 fixes to make profiler work 2013-08-05 12:36:43 -07:00
fjy fbb1211cbc [maven-release-plugin] prepare for next development iteration 2013-08-05 11:44:36 -07:00
fjy 22a90d36b7 [maven-release-plugin] prepare release druid-0.5.23 2013-08-05 11:44:28 -07:00
Jan Rudert 7195637a41 use same hadoop version as druid master (0.20.2) 2013-08-05 09:16:57 +02:00
Dhruv Parthasarathy 1c19fb4a02 changed number of replicants 2013-08-02 17:41:47 -07:00
cheddar d66af7625c 1) Eliminate the need for SingleSegmentLoader 
2) Setup the configuration glue for ServerInventoryViews and DataSegmentAnnouncer
3) Make processes run and work again!
 2013-08-02 17:05:01 -07:00
cheddar 2361e0112a Make it all compile again... 2013-08-02 10:14:46 -07:00
Jan Rudert 37ab5ef28b Merge branch 'master' of github.com:metamx/druid into HEAD 2013-08-02 09:23:19 +02:00
Dhruv Parthasarathy 6f6a335a40 adding profiling for 55k servers 2013-08-01 23:38:44 -07:00
Dhruv Parthasarathy 9f17e8fc43 all working 2013-08-01 21:01:02 -07:00
Dhruv Parthasarathy 696b04020b wrote regression test 2013-08-01 18:28:48 -07:00
fjy 584ccac833 move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better 2013-08-01 17:50:05 -07:00
Dhruv Parthasarathy 6f28eac995 fixed problem of balancer ignoring home server 2013-08-01 17:28:32 -07:00
Dhruv Parthasarathy cd3b62e123 fix to runtime params 2013-08-01 17:01:50 -07:00
cheddar 9e78bb38f5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
	client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
	client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
	client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
	indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ServerManager.java
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
	server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
	server/src/main/java/com/metamx/druid/master/DruidMaster.java
 2013-08-01 16:42:47 -07:00
cheddar 019bb5d453 1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working! 
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
 2013-08-01 15:28:08 -07:00
cheddar fba2bc9623 Merge pull request #204 from metamx/close-handle 
add the ability to unmap mmapped files
 2013-08-01 14:09:46 -07:00
Dhruv Parthasarathy 9b340d9f6c simplified console code. Added Builder. Fixed urls 2013-08-01 14:06:22 -07:00
fjy 8f39db7804 more data to exception msgs 2013-08-01 14:04:27 -07:00
fjy 6d54ae9f81 more unit tests to test for multiple closes in different threads 2013-08-01 13:51:56 -07:00
fjy dc169d9fcd hide decrement and update close logic to call decrement 2013-08-01 13:24:56 -07:00
fjy e92e0c35a4 put all concurrency test back into server manager test, introduce better concurrency for reference counting 2013-08-01 13:13:03 -07:00
fjy b86258cc14 fix broken master UT based on changes that were discarded as part of a pull req 2013-08-01 10:44:39 -07:00
fjy 2f8351b314 fix broken ut 2013-08-01 09:41:36 -07:00
Jan Rudert 73d5ff4b94 Merge branch 'master' of github.com:metamx/druid into HEAD 2013-08-01 18:00:29 +02:00
Jan Rudert 31c048ccb6 use hadoop cdh4-4.1.1 and protobuf 2.4.0 2013-08-01 17:27:33 +02:00
cheddar f4fc8b2f6e 1) A whole boondoggle of changes to Guicify the instantiation of things for the Historical nodes 
2) Add GuiceInjectableValues that supports @JacksonInject resolution via Guice
3) Refactor RequestLogger stuff and setup polymorphic deserialization method of creating objects
 2013-07-31 18:36:47 -07:00
fjy e2b5cd6067 Merge branch 'master' of github.com:metamx/druid into worker-resource 
Conflicts:
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
 2013-07-31 15:23:13 -07:00
fjy f76540c16c move reference counting classes out of server manager and more tests for concurrency 2013-07-31 15:14:39 -07:00
Dhruv Parthasarathy dd689ca828 changed urls 2013-07-31 11:43:24 -07:00
fjy 5d1fe50703 additional ut for reference counting 2013-07-30 17:55:57 -07:00
fjy 404b84823e reference count to make sure we arent accessing any instances of a closed mmapped segment 2013-07-30 17:36:56 -07:00
cheddar 843b6650a2 Merge pull request #206 from metamx/peontimeout 
Add a timeout in LoadQueuePeon to fail an assign if it takes too long
 2013-07-30 15:08:46 -07:00
fjy e7c6dd8112 get and reset failed assign count 2013-07-30 14:50:24 -07:00
fjy 35d8a82879 move the close method to the segment object 2013-07-30 12:24:01 -07:00
cheddar 97fb971f84 1) Start the journey of making the historical nodes work with Guice. Doesn't work yet, but at least it builds. 2013-07-30 11:58:05 -07:00
cheddar 1c5c2e0ef5 Merge pull request #205 from metamx/compress-curator 
Toggle between compressed and non compressed service discovery
 2013-07-30 11:33:43 -07:00
cheddar 66adef5d64 Merge pull request #179 from zanox/master 
Fix path used when storing segments to HDFS
 2013-07-30 11:13:29 -07:00
fjy f55e12040f fix according to code review 2013-07-29 18:39:14 -07:00
Dhruv Parthasarathy b8940b4524 added support for dynamicConfigs for Master 2013-07-29 18:16:57 -07:00
Dhruv Parthasarathy 4e4d582e07 formatted files 2013-07-29 18:10:08 -07:00
Dhruv Parthasarathy ac0f0afdbf turned Iterable to ArrayList, fixed potential null pointer in reservoir sampling 2013-07-29 18:07:49 -07:00
fjy 6a96c1fb76 fix according to code review comments 2013-07-29 17:56:19 -07:00
Dhruv Parthasarathy b0d9889c83 added web console for changing dynamicConfigs 2013-07-29 17:49:09 -07:00
fjy 091dce11c8 Add a timeout in LQP to fail an assign if it takes too long 2013-07-29 15:47:05 -07:00
fjy 50836798fa toggle between compressed and non compressed service discovery 2013-07-29 15:40:45 -07:00
fjy 1226190279 add the ability to close mmapped files 2013-07-29 15:22:34 -07:00
cheddar 4c9be81912 1) Pull try/catch check in the JettyServerInitializer out of the initializer. 2013-07-29 14:35:30 -07:00
cheddar 9cd26c70c0 1) Adjust the JettyServerModule and other things such that Jetty will actually serve the static artifacts and the various resources of the Master 2013-07-29 14:26:51 -07:00
Dhruv Parthasarathy 31ffed466c first set of changes 2013-07-29 11:39:24 -07:00
Jan Rudert bba83ec532 introduced a ByteBufferInputRowParser to keep the old StringInputRowParser in places outside the KafkaFireHose 2013-07-29 11:59:08 +02:00
cheddar 6b9963b472 1) Move the @Json and @Smile annotations to com.metamx.druid.guice.annotations 
2) Changed ServiceAnnouncer to take a DruidNode object
3) Remove ServiceInstanceFactory interface and implementations
4) Add registrations to DiscoveryModule so that you can register nodes that should be announced on startup
5) Make the set of default Monitors configurable
 2013-07-26 14:03:51 -07:00
Dhruv Parthasarathy ba4a9c28a7 removed unecessary line 2013-07-26 10:52:50 -07:00
Dhruv Parthasarathy 1e4f64b119 redid testing 2013-07-26 10:50:06 -07:00
fjy a1262760b2 Merge branch 'master' into worker-resource 2013-07-26 10:02:15 -07:00
fjy b0090a1de6 [maven-release-plugin] prepare for next development iteration 2013-07-25 17:26:44 -07:00
fjy 6868153905 [maven-release-plugin] prepare release druid-0.5.22 2013-07-25 17:26:36 -07:00
fjy cf0cb3b703 [maven-release-plugin] prepare for next development iteration 2013-07-25 16:55:28 -07:00
fjy 8297f068c3 [maven-release-plugin] prepare release druid-0.5.21 2013-07-25 16:55:20 -07:00
fjy f68a68112f [maven-release-plugin] prepare for next development iteration 2013-07-25 16:21:59 -07:00
fjy b6e8d11aec [maven-release-plugin] prepare release druid-0.5.20 2013-07-25 16:21:49 -07:00
fjy 217894d44a Merge branch 'master' into worker-resource 
Conflicts:
	pom.xml
 2013-07-25 16:18:42 -07:00
Dhruv Parthasarathy 8e4fba431c moved profiler 2013-07-25 14:37:03 -07:00
Dhruv Parthasarathy 34a9525966 added tests for random sampler and changed abstract class to interface 2013-07-25 14:14:16 -07:00
Dhruv Parthasarathy 8ee71c8ceb added strategy pattern and RandomBalancerStrategy 2013-07-25 00:00:38 -07:00
Stefán Freyr Stefánsson b3530fb15d Adding a configuration property to disable setting an ACL on the segment pushed to S3. 2013-07-24 23:37:21 +00:00
Dhruv Parthasarathy a03dcc6429 cleaned up comments 2013-07-24 15:34:08 -07:00
Dhruv Parthasarathy bcb3e089d8 profiler added, commented out initialTotalCost normalization and normalizedInitialCost 2013-07-24 14:57:02 -07:00
fjy cfc635de53 [maven-release-plugin] prepare for next development iteration 2013-07-24 09:41:30 -07:00
fjy 09576ccd09 [maven-release-plugin] prepare release druid-0.5.19 2013-07-24 09:41:20 -07:00
Dhruv Parthasarathy 6dc8d09b00 partial changes 2013-07-23 17:25:36 -07:00
cheddar 02ffd805c8 1) Move DbConnector into its own module 
2) Move JacksonConfig stuff into its own module
3) Fix LogEmitterModule to inject properly with a Supplier<>
4) Set Guice to require explicit bindings, to protect from weirdness like was initially experienced with LoggingEmitterConfig
5) Master fires up and appears to all work with Guice!  Whoot!
 2013-07-23 14:55:36 -07:00
fjy 980b09d903 fix broken test 2013-07-23 10:19:26 -07:00
fjy ea7c7d1d66 Merge branch 'master' into batch-zk 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
 2013-07-23 10:14:26 -07:00
fjy ef27810458 rename to fix case issue 2013-07-22 18:02:10 -07:00
fjy 3f97ac9253 add read to batch zk 2013-07-22 18:01:13 -07:00
cheddar 7504f3ab65 1) Adjust the Announcer to cleanup parent nodes that it creates. This stops it from leaving little turdlings lying around on zookeeper as nodes exit 
2) Rename *CuratorDataSegmentAnnouncer because neither of them depend directly on Curator.
 2013-07-22 16:31:01 -07:00
cheddar 85a35201b7 1) The Master compiles and runs with Guice bindings!!!! 
2) The service discovery stuff really needs to be reworked.  It's unhappy now.
 2013-07-19 16:29:54 -07:00
Jan Rudert 527c6c81c5 refactored StringInputRowParser to handle a ByteBuffer, better wiring and configuration of KafkaFirehose 2013-07-19 14:12:53 +02:00
fjy 1b56fa5416 [maven-release-plugin] prepare for next development iteration 2013-07-18 15:37:54 -07:00
fjy 0c11602623 [maven-release-plugin] prepare release druid-0.5.18 2013-07-18 15:37:16 -07:00
fjy 816f673a47 [maven-release-plugin] prepare for next development iteration 2013-07-18 15:14:47 -07:00
fjy bcb1b99410 [maven-release-plugin] prepare release druid-0.5.17 2013-07-18 15:14:32 -07:00
fjy 357d77aca2 [maven-release-plugin] prepare for next development iteration 2013-07-18 14:53:56 -07:00
fjy b5b6cd6594 [maven-release-plugin] prepare release druid-0.5.16 2013-07-18 14:53:46 -07:00
fjy 08f56feee3 [maven-release-plugin] prepare for next development iteration 2013-07-18 14:48:53 -07:00
fjy 74b4c577f0 [maven-release-plugin] prepare release druid-0.5.15 2013-07-18 14:48:42 -07:00
fjy 5b55db6a42 [maven-release-plugin] prepare for next development iteration 2013-07-18 14:28:54 -07:00
fjy 3792a54869 [maven-release-plugin] prepare release druid-0.5.14 2013-07-18 14:28:46 -07:00
fjy 2f63140f2f [maven-release-plugin] prepare for next development iteration 2013-07-18 14:17:16 -07:00
fjy 4481b10efc [maven-release-plugin] prepare release druid-0.5.13 2013-07-18 14:16:55 -07:00
fjy df88274d9e fix poms 2013-07-18 14:14:16 -07:00
fjy 325a5fb4c0 [maven-release-plugin] prepare release druid-0.5.12 2013-07-18 14:09:56 -07:00
fjy 4e2f6380e6 Merge branch 'master' of github.com:metamx/druid 2013-07-18 13:25:08 -07:00
fjy 7d58a2d8db fix bug with enabling segments and improve replication logic 2013-07-18 13:24:52 -07:00
Jan Rudert d3412c851b Merge branch 'master' of github.com:metamx/druid into kafka-protobuf 2013-07-18 15:14:02 +02:00
cheddar e906481704 Merge pull request #191 from metamx/s3-puller-retries 
S3 puller retries
 2013-07-17 17:07:57 -07:00
fjy cef0e50881 [maven-release-plugin] prepare for next development iteration 2013-07-17 09:14:37 -07:00
fjy 22526a6c68 [maven-release-plugin] prepare release druid-0.5.11 2013-07-17 09:14:28 -07:00
fjy 43bdb99593 [maven-release-plugin] prepare for next development iteration 2013-07-16 17:55:42 -07:00
fjy d5fee45bf4 [maven-release-plugin] prepare release druid-0.5.10 2013-07-16 17:55:29 -07:00
fjy f3187b67a5 [maven-release-plugin] prepare for next development iteration 2013-07-16 17:32:05 -07:00
fjy eb89857bdc [maven-release-plugin] prepare release druid-0.5.9 2013-07-16 17:31:55 -07:00
fjy 0f492e17c3 [maven-release-plugin] prepare for next development iteration 2013-07-16 16:53:51 -07:00
fjy a3879d1151 [maven-release-plugin] prepare release druid-0.5.8 2013-07-16 16:53:41 -07:00
fjy 4e84fb5f03 better logic for exiting load rule assign if replicants cannot be created 2013-07-16 16:50:52 -07:00
fjy f422bfae8c [maven-release-plugin] prepare for next development iteration 2013-07-16 13:35:38 -07:00
fjy cd174dbd80 [maven-release-plugin] prepare release druid-0.5.7 2013-07-16 13:35:29 -07:00
fjy afac15d631 fix version problem 2013-07-16 13:32:50 -07:00
fjy 36756aa557 [maven-release-plugin] prepare release druid-0.5.6 2013-07-16 13:30:16 -07:00
fjy 49a80572b7 fix master speed issue with immutable copy of in druid server, better exception handling in moving and dropping segments 2013-07-16 13:24:43 -07:00
Gian Merlino 6258d77398 Retry s3 operations on non-wrapped IOExceptions 
Can happen if we get a socket related mishap while fetching an s3 object.
 2013-07-16 08:25:23 -07:00
Gian Merlino 320f1fe840 Move retryS3Operation to S3Utils 2013-07-16 08:07:14 -07:00
Gian Merlino 945bc9a370 Limited retries on s3 segment puller exceptions 
Useful for reducing spurious alerts due to general s3 flakiness.
 2013-07-16 08:03:47 -07:00
Jan Rudert 0961d8f149 Merge branch 'master' of github.com:metamx/druid 2013-07-11 16:02:11 +02:00
cheddar e892d14f77 1) Configuration fixes, slowly gonna make the MasterMain actually start up! 2013-07-09 16:16:56 -07:00
fjy cc35841e23 Merge branch 'batch-zk' into worker-resource 2013-07-08 11:04:05 -07:00
fjy 454f23d7c0 Merge branch 'master' of github.com:metamx/druid into batch-zk 2013-07-08 10:40:30 -07:00
Jan Rudert d5b4417280 Merge branch 'master' of github.com:metamx/druid 2013-07-08 11:50:15 +02:00
Jan Rudert 18ec081855 moved getHdfsStorageDir() to DataSegmentPusherUtil, extended test 2013-07-08 08:39:08 +02:00
cheddar b42c5743c9 Merge pull request #177 from nebrera/master 
Postgresql adaptation
 2013-07-05 16:42:57 -07:00
Jan Rudert 6f06d701d9 Building a correct segment path for HDFS 2013-07-04 16:00:20 +02:00
fjy be2f153985 Merge branch 'bug-fix' into worker-resource 2013-07-02 16:25:07 -07:00
fjy d3eb491155 Merge branch 'batch-zk' into worker-resource 
Conflicts:
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
 2013-07-02 16:24:59 -07:00
fjy e21aa41bdd Merge branch 'master' into worker-resource 
Conflicts:
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
 2013-07-02 16:23:53 -07:00
fjy fde874ea56 fix according to code review 2013-07-02 16:11:12 -07:00
fjy 9818d28a26 Merge branch 'master' into bug-fix 2013-07-02 15:59:52 -07:00
fjy 5ac2d1db0b Merge branch 'master' of github.com:metamx/druid into batch-zk 
Conflicts:
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
 2013-07-02 15:57:31 -07:00
fjy 7219ed15d3 fix according to code review 2013-07-02 15:56:12 -07:00
cheddar fd1d73e83a [maven-release-plugin] prepare for next development iteration 2013-07-01 16:04:31 -07:00
cheddar 090921b43e [maven-release-plugin] prepare release druid-0.5.5 2013-07-01 16:04:25 -07:00
cheddar 797a083b69 1) Adjust SingleSegmentLoader to allow for storing segments on multiple different mount points. The specification language is really janky right now, so this is remaining a stealth feature for the time being. 2013-07-01 14:56:57 -07:00
cheddar e8afeda046 Merge branch 'master' of github.com:metamx/druid 2013-07-01 14:54:32 -07:00
fjy 5a57539736 fix hdfs config 2013-06-28 10:18:10 -07:00
Pablo Nebrera e9ce6b596c PostgresSQL integration with druid 2013-06-28 12:49:24 +02:00
fjy 2969d9a713 Merge branch 'master' into batch-zk 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
 2013-06-27 16:46:37 -07:00
fjy a6787ab201 fix up configs 2013-06-27 16:40:17 -07:00