Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,489 Commits 113 Branches 504 Tags 726 MiB
Commit Graph

657 Commits

Author SHA1 Message Date
fjy dd20950f8a adding missing continue 2013-08-12 13:07:29 -07:00
fjy 2ec2957207 clean up the error reporting code according to code review 2013-08-12 13:04:48 -07:00
fjy 0c360c05c2 better handling of exceptions during startup 2013-08-12 12:56:59 -07:00
fjy 2f728f3478 fix zk coord to not fail during startup exceptions 2013-08-12 11:33:42 -07:00
Jan Rudert 3f32ef7bbc Merge branch 'master' of github.com:metamx/druid into kafka-protobuf 2013-08-12 17:42:40 +02:00
jan 55267ca0e4 removed @author tags 2013-08-11 13:44:13 +02:00
Dhruv Parthasarathy 01db89ac6e fixed issue with web console. Cleaned up code 2013-08-09 00:33:40 -07:00
fjy a190269faa fix druid master segments getting stuck bug 2013-08-08 22:06:41 -07:00
Dhruv Parthasarathy fc5e15f72b merged changes with master 2013-08-08 18:11:33 -07:00
Dhruv Parthasarathy abe347efd3 merged with master 2013-08-08 17:21:57 -07:00
Dhruv Parthasarathy 764863d6f5 made volatile. fixed parens 2013-08-08 16:40:10 -07:00
Dhruv Parthasarathy 8a8ea5d028 removed test annotation from profiler 2013-08-08 16:38:24 -07:00
Dhruv Parthasarathy 01b35ec819 all tests passed. Ready for merge 2013-08-08 16:23:26 -07:00
Dhruv Parthasarathy eed9e1b373 merged with master 2013-08-08 16:14:06 -07:00
Dhruv Parthasarathy 0b24ea710c fixed paths 2013-08-08 16:10:31 -07:00
Dhruv Parthasarathy 3a298681e4 fixed formatting 2013-08-07 10:59:54 -07:00
Dhruv Parthasarathy c4f0e4d96d fixed with Erics feedback 2013-08-07 10:53:37 -07:00
fjy 5f292143e0 disable merging segments with linear shard spec 2013-08-06 16:17:49 -07:00
Dhruv Parthasarathy f31ee552d0 fixed with fj comments 2013-08-06 15:03:07 -07:00
Dhruv Parthasarathy 0d5b222f8f removed SlickGrid :( 2013-08-06 14:30:35 -07:00
Dhruv Parthasarathy 6124ecd9cb fied some tests 2013-08-06 14:26:08 -07:00
cheddar eee1efdcb5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
 2013-08-06 13:33:31 -07:00
cheddar 51f648ec11 1) Rename QueryToolChestModule back 2013-08-06 12:05:56 -07:00
cheddar 69f7831d82 1) Rename QueryToolChestModule because there was a case sensitivity issue, grrr 2013-08-06 12:04:04 -07:00
cheddar 9f71d42314 1) Broker works with Guice 
2) Extract ServerViewModule to handle various ServerView thingies
3) Extract QueryToolChestModule and QueryRunnerFactoryModule to reuse code for configuration of Query stuff
4) Extract QueryJettyServerInitializer to reuse between Historical and Broker nodes
5) Remove ClientMain, BrokerMain and BrokerNode
 2013-08-06 12:01:52 -07:00
Dhruv Parthasarathy ebba2d66a3 results good 2013-08-06 11:46:18 -07:00
Dhruv Parthasarathy e40d5b55ef testing speed for one run 2013-08-06 11:40:49 -07:00
Dhruv Parthasarathy 575dfe3679 replication working too 2013-08-05 14:51:58 -07:00
Dhruv Parthasarathy 1045aac4b8 finished profiling here 2013-08-05 13:53:40 -07:00
cheddar d64bc2f762 1) Eliminate Old mains() that aren't needed anymore. 2013-08-05 13:33:43 -07:00
Dhruv Parthasarathy 540c4885d7 fixes to make profiler work 2013-08-05 12:36:43 -07:00
Dhruv Parthasarathy 1c19fb4a02 changed number of replicants 2013-08-02 17:41:47 -07:00
cheddar d66af7625c 1) Eliminate the need for SingleSegmentLoader 
2) Setup the configuration glue for ServerInventoryViews and DataSegmentAnnouncer
3) Make processes run and work again!
 2013-08-02 17:05:01 -07:00
cheddar 2361e0112a Make it all compile again... 2013-08-02 10:14:46 -07:00
Dhruv Parthasarathy 6f6a335a40 adding profiling for 55k servers 2013-08-01 23:38:44 -07:00
Dhruv Parthasarathy 9f17e8fc43 all working 2013-08-01 21:01:02 -07:00
Dhruv Parthasarathy 696b04020b wrote regression test 2013-08-01 18:28:48 -07:00
fjy 584ccac833 move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better 2013-08-01 17:50:05 -07:00
Dhruv Parthasarathy 6f28eac995 fixed problem of balancer ignoring home server 2013-08-01 17:28:32 -07:00
Dhruv Parthasarathy cd3b62e123 fix to runtime params 2013-08-01 17:01:50 -07:00
cheddar 9e78bb38f5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
	client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
	client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
	client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
	indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ServerManager.java
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
	server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
	server/src/main/java/com/metamx/druid/master/DruidMaster.java
 2013-08-01 16:42:47 -07:00
cheddar 019bb5d453 1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working! 
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
 2013-08-01 15:28:08 -07:00
cheddar fba2bc9623 Merge pull request #204 from metamx/close-handle 
add the ability to unmap mmapped files
 2013-08-01 14:09:46 -07:00
Dhruv Parthasarathy 9b340d9f6c simplified console code. Added Builder. Fixed urls 2013-08-01 14:06:22 -07:00
fjy 8f39db7804 more data to exception msgs 2013-08-01 14:04:27 -07:00
fjy 6d54ae9f81 more unit tests to test for multiple closes in different threads 2013-08-01 13:51:56 -07:00
fjy dc169d9fcd hide decrement and update close logic to call decrement 2013-08-01 13:24:56 -07:00
fjy e92e0c35a4 put all concurrency test back into server manager test, introduce better concurrency for reference counting 2013-08-01 13:13:03 -07:00
fjy b86258cc14 fix broken master UT based on changes that were discarded as part of a pull req 2013-08-01 10:44:39 -07:00
fjy 2f8351b314 fix broken ut 2013-08-01 09:41:36 -07:00
cheddar f4fc8b2f6e 1) A whole boondoggle of changes to Guicify the instantiation of things for the Historical nodes 
2) Add GuiceInjectableValues that supports @JacksonInject resolution via Guice
3) Refactor RequestLogger stuff and setup polymorphic deserialization method of creating objects
 2013-07-31 18:36:47 -07:00
fjy e2b5cd6067 Merge branch 'master' of github.com:metamx/druid into worker-resource 
Conflicts:
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
 2013-07-31 15:23:13 -07:00
fjy f76540c16c move reference counting classes out of server manager and more tests for concurrency 2013-07-31 15:14:39 -07:00
Dhruv Parthasarathy dd689ca828 changed urls 2013-07-31 11:43:24 -07:00
fjy 5d1fe50703 additional ut for reference counting 2013-07-30 17:55:57 -07:00
fjy 404b84823e reference count to make sure we arent accessing any instances of a closed mmapped segment 2013-07-30 17:36:56 -07:00
cheddar 843b6650a2 Merge pull request #206 from metamx/peontimeout 
Add a timeout in LoadQueuePeon to fail an assign if it takes too long
 2013-07-30 15:08:46 -07:00
fjy e7c6dd8112 get and reset failed assign count 2013-07-30 14:50:24 -07:00
fjy 35d8a82879 move the close method to the segment object 2013-07-30 12:24:01 -07:00
cheddar 97fb971f84 1) Start the journey of making the historical nodes work with Guice. Doesn't work yet, but at least it builds. 2013-07-30 11:58:05 -07:00
cheddar 1c5c2e0ef5 Merge pull request #205 from metamx/compress-curator 
Toggle between compressed and non compressed service discovery
 2013-07-30 11:33:43 -07:00
cheddar 66adef5d64 Merge pull request #179 from zanox/master 
Fix path used when storing segments to HDFS
 2013-07-30 11:13:29 -07:00
fjy f55e12040f fix according to code review 2013-07-29 18:39:14 -07:00
Dhruv Parthasarathy b8940b4524 added support for dynamicConfigs for Master 2013-07-29 18:16:57 -07:00
Dhruv Parthasarathy 4e4d582e07 formatted files 2013-07-29 18:10:08 -07:00
Dhruv Parthasarathy ac0f0afdbf turned Iterable to ArrayList, fixed potential null pointer in reservoir sampling 2013-07-29 18:07:49 -07:00
fjy 6a96c1fb76 fix according to code review comments 2013-07-29 17:56:19 -07:00
Dhruv Parthasarathy b0d9889c83 added web console for changing dynamicConfigs 2013-07-29 17:49:09 -07:00
fjy 091dce11c8 Add a timeout in LQP to fail an assign if it takes too long 2013-07-29 15:47:05 -07:00
fjy 50836798fa toggle between compressed and non compressed service discovery 2013-07-29 15:40:45 -07:00
fjy 1226190279 add the ability to close mmapped files 2013-07-29 15:22:34 -07:00
cheddar 4c9be81912 1) Pull try/catch check in the JettyServerInitializer out of the initializer. 2013-07-29 14:35:30 -07:00
cheddar 9cd26c70c0 1) Adjust the JettyServerModule and other things such that Jetty will actually serve the static artifacts and the various resources of the Master 2013-07-29 14:26:51 -07:00
Dhruv Parthasarathy 31ffed466c first set of changes 2013-07-29 11:39:24 -07:00
cheddar 6b9963b472 1) Move the @Json and @Smile annotations to com.metamx.druid.guice.annotations 
2) Changed ServiceAnnouncer to take a DruidNode object
3) Remove ServiceInstanceFactory interface and implementations
4) Add registrations to DiscoveryModule so that you can register nodes that should be announced on startup
5) Make the set of default Monitors configurable
 2013-07-26 14:03:51 -07:00
Dhruv Parthasarathy ba4a9c28a7 removed unecessary line 2013-07-26 10:52:50 -07:00
Dhruv Parthasarathy 1e4f64b119 redid testing 2013-07-26 10:50:06 -07:00
fjy 217894d44a Merge branch 'master' into worker-resource 
Conflicts:
	pom.xml
 2013-07-25 16:18:42 -07:00
Dhruv Parthasarathy 8e4fba431c moved profiler 2013-07-25 14:37:03 -07:00
Dhruv Parthasarathy 34a9525966 added tests for random sampler and changed abstract class to interface 2013-07-25 14:14:16 -07:00
Dhruv Parthasarathy 8ee71c8ceb added strategy pattern and RandomBalancerStrategy 2013-07-25 00:00:38 -07:00
Stefán Freyr Stefánsson b3530fb15d Adding a configuration property to disable setting an ACL on the segment pushed to S3. 2013-07-24 23:37:21 +00:00
Dhruv Parthasarathy a03dcc6429 cleaned up comments 2013-07-24 15:34:08 -07:00
Dhruv Parthasarathy bcb3e089d8 profiler added, commented out initialTotalCost normalization and normalizedInitialCost 2013-07-24 14:57:02 -07:00
Dhruv Parthasarathy 6dc8d09b00 partial changes 2013-07-23 17:25:36 -07:00
cheddar 02ffd805c8 1) Move DbConnector into its own module 
2) Move JacksonConfig stuff into its own module
3) Fix LogEmitterModule to inject properly with a Supplier<>
4) Set Guice to require explicit bindings, to protect from weirdness like was initially experienced with LoggingEmitterConfig
5) Master fires up and appears to all work with Guice!  Whoot!
 2013-07-23 14:55:36 -07:00
fjy 980b09d903 fix broken test 2013-07-23 10:19:26 -07:00
fjy ea7c7d1d66 Merge branch 'master' into batch-zk 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
 2013-07-23 10:14:26 -07:00
fjy ef27810458 rename to fix case issue 2013-07-22 18:02:10 -07:00
fjy 3f97ac9253 add read to batch zk 2013-07-22 18:01:13 -07:00
cheddar 7504f3ab65 1) Adjust the Announcer to cleanup parent nodes that it creates. This stops it from leaving little turdlings lying around on zookeeper as nodes exit 
2) Rename *CuratorDataSegmentAnnouncer because neither of them depend directly on Curator.
 2013-07-22 16:31:01 -07:00
cheddar 85a35201b7 1) The Master compiles and runs with Guice bindings!!!! 
2) The service discovery stuff really needs to be reworked.  It's unhappy now.
 2013-07-19 16:29:54 -07:00
fjy 4e2f6380e6 Merge branch 'master' of github.com:metamx/druid 2013-07-18 13:25:08 -07:00
fjy 7d58a2d8db fix bug with enabling segments and improve replication logic 2013-07-18 13:24:52 -07:00
cheddar e906481704 Merge pull request #191 from metamx/s3-puller-retries 
S3 puller retries
 2013-07-17 17:07:57 -07:00
fjy 4e84fb5f03 better logic for exiting load rule assign if replicants cannot be created 2013-07-16 16:50:52 -07:00
fjy 49a80572b7 fix master speed issue with immutable copy of in druid server, better exception handling in moving and dropping segments 2013-07-16 13:24:43 -07:00
Gian Merlino 6258d77398 Retry s3 operations on non-wrapped IOExceptions 
Can happen if we get a socket related mishap while fetching an s3 object.
 2013-07-16 08:25:23 -07:00
Gian Merlino 320f1fe840 Move retryS3Operation to S3Utils 2013-07-16 08:07:14 -07:00
Gian Merlino 945bc9a370 Limited retries on s3 segment puller exceptions 
Useful for reducing spurious alerts due to general s3 flakiness.
 2013-07-16 08:03:47 -07:00
cheddar e892d14f77 1) Configuration fixes, slowly gonna make the MasterMain actually start up! 2013-07-09 16:16:56 -07:00
fjy cc35841e23 Merge branch 'batch-zk' into worker-resource 2013-07-08 11:04:05 -07:00
fjy 454f23d7c0 Merge branch 'master' of github.com:metamx/druid into batch-zk 2013-07-08 10:40:30 -07:00
Jan Rudert d5b4417280 Merge branch 'master' of github.com:metamx/druid 2013-07-08 11:50:15 +02:00
Jan Rudert 18ec081855 moved getHdfsStorageDir() to DataSegmentPusherUtil, extended test 2013-07-08 08:39:08 +02:00
cheddar b42c5743c9 Merge pull request #177 from nebrera/master 
Postgresql adaptation
 2013-07-05 16:42:57 -07:00
Jan Rudert 6f06d701d9 Building a correct segment path for HDFS 2013-07-04 16:00:20 +02:00
fjy be2f153985 Merge branch 'bug-fix' into worker-resource 2013-07-02 16:25:07 -07:00
fjy d3eb491155 Merge branch 'batch-zk' into worker-resource 
Conflicts:
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
 2013-07-02 16:24:59 -07:00
fjy e21aa41bdd Merge branch 'master' into worker-resource 
Conflicts:
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
 2013-07-02 16:23:53 -07:00
fjy fde874ea56 fix according to code review 2013-07-02 16:11:12 -07:00
fjy 9818d28a26 Merge branch 'master' into bug-fix 2013-07-02 15:59:52 -07:00
fjy 5ac2d1db0b Merge branch 'master' of github.com:metamx/druid into batch-zk 
Conflicts:
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
 2013-07-02 15:57:31 -07:00
fjy 7219ed15d3 fix according to code review 2013-07-02 15:56:12 -07:00
cheddar 797a083b69 1) Adjust SingleSegmentLoader to allow for storing segments on multiple different mount points. The specification language is really janky right now, so this is remaining a stealth feature for the time being. 2013-07-01 14:56:57 -07:00
cheddar e8afeda046 Merge branch 'master' of github.com:metamx/druid 2013-07-01 14:54:32 -07:00
fjy 5a57539736 fix hdfs config 2013-06-28 10:18:10 -07:00
Pablo Nebrera e9ce6b596c PostgresSQL integration with druid 2013-06-28 12:49:24 +02:00
fjy 2969d9a713 Merge branch 'master' into batch-zk 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
 2013-06-27 16:46:37 -07:00
fjy a6787ab201 fix up configs 2013-06-27 16:40:17 -07:00
fjy d5306c5dd9 Merge branch 'master' into worker-resource 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
 2013-06-27 12:52:58 -07:00
fjy cce62cbd8b The highly important commit of changing a log line 2013-06-26 17:27:04 -07:00
fjy 15a4dae2b1 bunch of issue fixes 2013-06-26 10:45:54 -07:00
cheddar cd0fa34a73 1) Log exception when there is an error removing a segment 2013-06-25 11:18:04 -07:00
cheddar 149c2e3560 1) A bunch more changes to make things actually run. Not quite done, but close! 2013-06-20 17:19:51 -07:00
fjy caa68e101a first commit; things working right now 2013-06-19 15:56:45 -07:00
cheddar 11ea15fc1a 1) Introduce Jetty 8 
2) Fix up modules based on exceptions at startup for MasterMain
 2013-06-17 10:53:50 -07:00
cheddar 06f7e7e665 1) Fix DatabaseSegmentManager so that Unit Test passes 2013-06-10 14:13:28 -07:00
fjy 3312d9b802 fix merging problem with linear shard specs 2013-06-10 14:07:36 -07:00
cheddar 2f56c24259 1) Inject IndexingServiceClient 
2) Switch all the DBI references to IDBI
 2013-06-07 17:37:33 -07:00
cheddar 06f90bc2e1 Merge branch 'master' of github.com:metamx/druid 2013-06-07 14:15:39 -07:00
cheddar a949f1672d 1) Fix bug with Master stopping trying to be Master after he losses it once 2013-06-07 14:14:58 -07:00
fjy 37a3994003 add delegating executor service and fix bug with query priortization 2013-06-07 14:08:51 -07:00
cheddar f68df7ab69 1) Make tests work and continue trying to make the DruidMaster start up with just Guice 2013-06-07 12:01:46 -07:00
fjy f1dea971f2 Merge pull request #156 from metamx/query-prioritization 
Query prioritization
 2013-06-06 16:05:47 -07:00
fjy 19c5dacf3a more code review fixes 2013-06-06 15:57:18 -07:00
cheddar 9df458a065 1) Initial commit of conversion to using Guice modules for bootstrapping. Things don't actually completely work yet. 2013-06-06 15:43:15 -07:00
fjy e20f547261 fix according to code review 2013-06-06 15:37:19 -07:00
cheddar b8feb98616 1) Make Master honor the druid.monitoring.monitorSystem property to turn off Sigar 2013-06-06 14:55:23 -07:00
fjy d48b4d8d97 Merge branch 'master' into query-prioritization 2013-06-06 14:19:50 -07:00
fjy 8202bc70eb optimize spatial and update poms everywhere 2013-06-05 15:26:40 -07:00
fjy 61074daa67 fix merge error 2013-06-04 18:56:20 -07:00
fjy 63b379ae90 Merge branch 'master' into refactor-indexing 2013-06-04 17:25:16 -07:00
fjy 06931ee0f5 introduce availability groups 2013-06-04 17:12:19 -07:00
cheddar 1a02637bea 1) add full parameter to master endpoint /info/rules/dataSourceName that returns the fully resolved rules, not just the specified rules 2013-06-04 17:10:25 -07:00
fjy 42cc87a294 Merge branch 'master' into refactor-indexing 
Conflicts:
	indexing-service/src/main/java/com/metamx/druid/indexing/common/task/IndexTask.java
	pom.xml
 2013-05-31 17:28:59 -07:00
fjy 09f7b181d4 first commit, working UT 2013-05-29 17:04:25 -07:00
xvrl 192b24c355 enable gzip compression for HTTP requests 2013-05-22 23:02:37 -07:00
fjy 1fb8c9db7f fix spatial bugs and more tests 2013-05-22 13:35:12 -07:00
fjy 47f2d3b0aa fix druidsetup properties path bug 2013-05-22 10:25:11 -07:00
fjy 39f55a010c additional bug fixes for hadoop based spatial indexing 2013-05-21 21:33:06 -07:00
fjy 6e5b893e6d make spatial filters work 2013-05-21 17:10:44 -07:00
fjy 69c7131a7b cleanup interval determination in II 2013-05-20 21:06:32 -07:00
fjy efc92e7ca6 bug fixes for segment metadata queries and spatial indexing 2013-05-20 20:19:18 -07:00
fjy 1b4a543f25 add default cassandra configs 2013-05-20 13:44:33 -07:00
fjy a792525063 Merge tag 'druid-0.4.12.5' 
[maven-release-plugin]  copy for tag druid-0.4.12.5

Conflicts:
	client/pom.xml
	common/pom.xml
	examples/pom.xml
	index-common/pom.xml
	indexer/pom.xml
	merger/pom.xml
	pom.xml
	realtime/pom.xml
	server/pom.xml
	services/pom.xml
 2013-05-20 12:14:23 -07:00
fjy 54c48aac6d fix DruidSetup not working 2013-05-20 12:06:21 -07:00
fjy 8af9598157 fix currSize bug and maxTime bugs 2013-05-18 13:21:45 -07:00
fjy b0a58af05d Merge pull request #140 from boneill42/cassandra-segments 
Initial implementation of Cassandra Data Segment Pusher and Puller
 2013-05-17 09:56:44 -07:00
Brian O'Neill c8ff5ca2fb Using string formatter in log statements. 
Clean up of imports on the pusher.
 2013-05-17 12:54:03 -04:00
fjy c6ad0753f0 bug fixes 2013-05-16 16:02:20 -07:00
Brian O'Neill 61c014c49f Cleaned up imports. 2013-05-16 13:33:39 -04:00
Brian O'Neill 23998f3f01 - Added cleanup to the puller. 
- Edited the documentation to remove reference to real-time node.
 2013-05-16 13:04:46 -04:00
fjy 6213c0b63c Merge branch 'master' into refactor-indexing 2013-05-15 17:14:40 -07:00
fjy 20ae1d8b6b lots of cleanups and refactorings 2013-05-15 15:37:04 -07:00
fjy 803e8ff69e add configurable spatial dimensions to hadoop indexer 2013-05-14 17:38:05 -07:00
fjy 5af188f18d Merge branch 'master' into spatial 2013-05-14 16:10:18 -07:00
cheddar 86415e70ea Merge pull request #142 from metamx/improve-bard 
Enable the broker to select a compute node will the least number of open connections
 2013-05-14 15:18:04 -07:00
cheddar fe5627e119 1) Left some log lines in the tests... That's no good. 2013-05-14 16:59:34 -05:00
cheddar 8bd19e9d04 1) Take in g9yuayon's pull request, fix merge conflicts and formatting 2013-05-14 16:55:43 -05:00
fjy e8ab0893ff fix column size reporting 2013-05-14 14:52:38 -07:00
fjy 1a978fe525 fix predicates 2013-05-12 15:26:19 -07:00
fjy bcfeac2d8c fix NPE 2013-05-12 15:14:57 -07:00
Brian O'Neill 8e0c0e6d47 Switched to default the aws properties. 2013-05-08 16:02:11 -04:00
Brian O'Neill 3eb0f4dfca Fix hierarchy in Pusher (to use CassandraStorage superclass) 2013-05-08 15:50:58 -04:00
fjy 986bb2482d Merge branch 'master' into spatial 2013-05-08 10:37:38 -07:00
fjy 43d3c53806 fix order by double sum agg 2013-05-08 10:14:40 -07:00
fjy 32f82364f1 Merge branch 'master' into spatial 2013-05-07 14:12:05 -07:00
Brian O'Neill 863b8808cc Merge branch 'master' of https://github.com/metamx/druid into cassandra-segments 2013-05-07 11:36:49 -04:00
Brian O'Neill 41e9f8fcb4 Working Push & Pull. 2013-05-07 11:35:14 -04:00
Brian O'Neill 8e8736291f Added puller. 2013-05-06 23:14:18 -04:00
cheddar 01c1257cee Merge pull request #134 from metamx/event-push 
Event push
 2013-05-06 15:10:44 -07:00
Brian O'Neill 142668cda5 Merge branch 'master' of https://github.com/metamx/druid into cassandra-segments 2013-05-06 17:54:56 -04:00
Brian O'Neill 79d802fede Working data segment pushed for C*. 2013-05-06 17:54:12 -04:00
fjy 8902986069 Merge branch 'master' into spatial 2013-05-03 23:09:30 -07:00
fjy 344a2b5d24 numerous bug fixes and improvements according to code review 2013-05-03 23:09:05 -07:00
cheddar 665f3ef0fd 1) Unit tests for order by and limit stuff for groupBy queries 
2) Adjustments to order by and limit stuff in light of unit tests-n-stuff
 2013-05-03 17:47:18 -05:00
fjy d57141f46e checking stuff in but stuff not really working yet 2013-05-03 10:35:26 -07:00
Gian Merlino af08ea7617 ServiceAnnouncer: 
- Interface with two impls, Noop and Curator
- Used wherever fine services are advertised

Initialization:
- Don't automatically register with service discovery
- Add makeServiceAnnouncer, announceDefaultService methods
- Replace serviceInstance with makeServiceInstanceFactory
 2013-05-03 14:29:17 +03:00
Brian O'Neill b8c8ed8d68 Formatting. 2013-05-01 15:17:15 -04:00
Brian O'Neill bf01399a2a Cassandra Data Segment Pusher, initial commit. 2013-05-01 15:11:50 -04:00
fjy 6d4c0850ca Merge branch 'master' into spatial 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
 2013-05-01 11:49:00 -07:00
fjy 63ccb24ef6 Merge branch 'master' into spatial 
Conflicts:
	pom.xml
	server/src/main/java/com/metamx/druid/http/InfoResource.java
 2013-05-01 10:12:04 -07:00
cheddar 0a3492c9f5 Move all the code for examples under the examples directory. Also add initial take on flights demo. 2013-04-30 18:27:38 -05:00
cheddar 8922adb1f2 1) Make the demos work again. 
2) First step in restructuring the demos into a single repository
 2013-04-30 18:13:55 -05:00
cheddar 60b279b0d3 1) Add support for storing segments in HDFS 2013-04-30 12:47:43 -05:00
cheddar a665e8b3b7 whitespace! 2013-04-29 14:02:30 -05:00
cheddar 2630d75cd3 More explicit comments around Zookeeper race condition 2013-04-29 13:58:57 -05:00
cheddar 405eed7c60 Better name for ServerInventoryThingie 2013-04-29 11:48:42 -05:00
cheddar 99305b29c8 Fixorz unit tests! 2013-04-26 19:19:57 -05:00
cheddar 163a73bed1 More logging stuff 2013-04-26 19:19:57 -05:00
cheddar de5c8c9d64 It's better when things compile 2013-04-26 19:19:56 -05:00
cheddar 1eb2dfe17e 1) Don't need a ServiceEmitter in ZkCoordinator anymore. 2013-04-26 19:19:56 -05:00
cheddar 38b5d94ffc 1) Imports... 2013-04-26 19:19:56 -05:00
cheddar 67ce1b6f26 1) Adjust all of the uses of zookeeper paths to be based on ZkPathsConfig 2013-04-26 19:19:56 -05:00
cheddar 3e94204c44 1) Switch from com.netflix.curator to org.apache.curator 2013-04-26 19:19:55 -05:00
cheddar b8ba9138ff 1) Fix bug with Master not starting up without an indexer specified 
2) Fix bug with PotentiallyGzippedCompressionProvider only catching ZipExceptions and not IOException (java 6 throws IO, java 7 throws Zip)
3) Create DruidServerMetadata and use that instead of DruidServer to represent the current server
 2013-04-26 19:19:55 -05:00
cheddar f71b941a1a 1) Refactor the announcement of segments to all exist inside the DataSegmentAnnouncer 
2) Adjust the ExecutorNodes to expose the correct nodeType given the refactorings
 2013-04-26 19:19:54 -05:00
cheddar 7370b0f2fc 1) Cleanup TODO comments 2013-04-26 19:19:54 -05:00
cheddar 57c31656d1 1) Eliminate PhoneBook 2013-04-26 19:19:53 -05:00
cheddar dde50a0d87 1) Remove references to zkclient 
2) Eradicate zkclient from the poms!
 2013-04-26 19:19:53 -05:00
cheddar 43d630c098 1) Replace InventoryManagers with Curator-based Inventory Managers 
2) Replace ZkSegmentAnnouncer with CuratorSegmentAnnouncer
3) Adjust LoadQueuePeons to operate using Curator
4) Remove ZkPhoneBook

Things left to do:
1) Unit tests
2) Remove references to zkclient library
3) Remove references to PhoneBook interface
 2013-04-26 19:19:23 -05:00
fjy e01a78515e many more fixes and unit tests 2013-04-25 19:01:45 -07:00
fjy 13b617cfa9 fix some interval bugs 2013-04-23 16:53:57 -07:00
fjy 7184e528fd fix some config bugs 2013-04-23 13:16:50 -07:00
fjy 70937af7bd Merge branch 'master' of github.com:metamx/druid into spatial 
Conflicts:
	server/src/main/java/com/metamx/druid/index/v1/IndexStorageAdapter.java
	server/src/main/java/com/metamx/druid/index/v1/QueryableIndexStorageAdapter.java
 2013-04-23 11:40:44 -07:00
fjy 9b9c516e30 more tests 2013-04-22 18:39:41 -07:00
fjy 6c4e844f97 semi working spatial 2013-04-19 16:56:56 -07:00
Fangjin Yang 71269d7e88 somehow tests are passing 2013-04-17 17:13:24 -07:00
xvrl 50998671ae formatting 2013-04-16 09:33:50 -07:00
xvrl 4597d7f44e remove IndexStorageAdapter 2013-04-16 09:33:50 -07:00
xvrl 2afe2bdeae formatting 2013-04-16 09:33:50 -07:00
xvrl c83631928c object column selectors 2013-04-16 09:33:50 -07:00
xvrl 0c68bd1b1d rename MetricSelectorFactory to ColumnSelectorFactory 2013-04-16 09:33:49 -07:00
Fangjin Yang cc929b2a07 slightly better configuration when talking to indexing service 2013-04-15 11:09:04 -07:00
Eric Tschetter 55648c47a7 1) Adjust the GroupByQuery to also be able to merge results on the local node. Fixes #116 
2) Make the GroupByQuery operate in a multi-threaded fashion by default (this is configurable via druid.query.groupBy.singleThreaded).  Fixes #96
3) Fix up some post aggregation computation stuff.  I believe #72 is fixed
4) Fix case sensitivity issue with post aggregations on GroupBy queries
 2013-04-09 19:13:08 -07:00
cheddar a678f08b67 Merge pull request #118 from darxriggs/master 
cleanup & refactoring
 2013-04-09 18:56:59 -07:00
Fangjin Yang 120b822c18 Merge branch 'master' of github.com:metamx/druid 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
 2013-04-08 11:24:17 -07:00
Fangjin Yang 925c104dd2 update emitter version and fix some NPEs 2013-04-08 11:23:08 -07:00
René Scheibe 6c27cd2e8e cleanup - removed unused imports 2013-04-07 23:31:22 +02:00
René Scheibe a6dc29cf85 cleanup - removed extra empty lines 
- there should not be any adjacent empty lines
 2013-04-07 14:32:09 +02:00
Nelson Ray b1f2186c1f use reservoir sampling in pickSegmentToMove to avoid IndexOutOfBoundsException when a segment gets dropped mid-run 2013-04-02 12:22:11 -07:00
René Scheibe c9d541364e fix cut-n-paste mistake in error message (issue #81) 2013-04-02 18:42:30 +02:00
Fangjin Yang 0730976c80 fix broken UT 2013-04-01 17:16:19 -07:00
Fangjin Yang 258f18471e nodes will not try and download segments during startup 2013-04-01 16:55:03 -07:00
Fangjin Yang b56ab64572 fix segment partition deletion bug 2013-04-01 14:26:02 -07:00
Fangjin Yang 5bdcc3613c fix inconsistent druid http paths + fix master console bugs 2013-03-27 10:40:44 -07:00
Fangjin Yang a7d836a0dd fix NPE for master balancing; introduce max number of workers for indexing service 2013-03-26 10:25:52 -07:00
fjy 05fdf08644 Merge pull request #107 from metamx/aggregator-close 
add close method to aggregators
 2013-03-25 13:09:19 -07:00
fjy 669f792668 Merge pull request #109 from metamx/fix-groupby-allgran-timestamp 
fix output timestamps in groupby queries with granularity "all".
 2013-03-25 12:58:32 -07:00
Nelson Ray e57249bf15 refactor balancer metric emission code 2013-03-25 11:49:46 -07:00
Nelson Ray a28de5fa88 fix per tier metrics emission for new balancer 2013-03-25 10:32:02 -07:00
xvrl c420fe3b56 fix output timestamps in groupby queries with granularity "all". 2013-03-23 23:42:43 -07:00
xvrl 3687cba18e close resources as we iterate over them 2013-03-22 17:42:10 -07:00
xvrl 16e4cd4291 fix cursor overshooting granularity boundaries in IncrementalIndexStorageAdapter as well 2013-03-21 10:38:39 -07:00
xvrl c0397aa67c add test case for granularities not aligned with segment boundaries 2013-03-21 10:38:06 -07:00
xvrl d722c99530 fix cursor overshooting granularity boundaries, due to incorrect gran.next() usage 2013-03-20 23:36:52 -07:00
xvrl 57f8810721 properly close rowiterators 2013-03-20 14:07:06 -07:00
xvrl 0b04114c12 more squigglies 2013-03-20 12:17:20 -07:00
xvrl 8eec41f934 squigglies ftw 2013-03-20 11:42:17 -07:00