Commit Graph

5177 Commits

Author SHA1 Message Date
Fangjin Yang 88bb02963a fix compile breakage introduced by scaling strategies 2012-11-08 10:32:18 -08:00
cheddar f8a5d08dc3 Merge pull request #20 from metamx/autoscaling
Improvements to distributed indexing service and auto scaling
2012-11-08 10:11:33 -08:00
Fangjin Yang 7083821fbf no param for scaling provision and minor fix for initialization 2012-11-08 10:08:40 -08:00
Fangjin Yang cdd9cdb2bd use PropUtils for indexer coordinator node 2012-11-08 09:31:22 -08:00
Fangjin Yang f39af83be4 make switching between scaling strategies config based 2012-11-08 09:30:32 -08:00
Fangjin Yang 4d6cfdc774 Merge branch 'master' into autoscaling 2012-11-07 17:27:42 -08:00
Fangjin Yang 52214d7355 additional improvements according to code review; unit tests; bug fixes for retry policies 2012-11-07 17:27:23 -08:00
cheddar 9c3b70068e Merge pull request #22 from metamx/histograms
HistogramVisual: add support for quantiles
2012-11-06 20:20:34 -08:00
Eric Tschetter b1475219a9 Merge branch 'master' of git://github.com/tralfamadude/druid-1
Added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon.
2012-11-06 20:15:57 -08:00
Nelson Ray 217caa3a04 HistogramVisual: put min. and max. in quantile array 2012-11-06 16:55:35 -08:00
Paul Baclace cb02c6841b removed recursive zpath listing from DruidSetup dump command. 2012-11-06 15:30:46 -08:00
Fangjin Yang 34cb352cf8 working indexer with registererers 2012-11-06 14:26:53 -08:00
Fangjin Yang 5698f640d7 fix last commit with version 2012-11-06 12:40:53 -08:00
Fangjin Yang 0b6dd99452 set default version if one is not set 2012-11-06 12:36:55 -08:00
Fangjin Yang 34a221a586 fix bug with jackson conversion 2012-11-06 11:56:48 -08:00
Fangjin Yang 68e5adde33 register registererers in the config 2012-11-06 11:49:17 -08:00
Paul Baclace 86d2fb3550 added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon. 2012-11-06 00:16:27 -08:00
Fangjin Yang eb2b5a61fa fix setters for hadoop node 2012-11-05 18:40:54 -08:00
Fangjin Yang 2ae0a15b5a add register abilities to mapper 2012-11-05 18:31:23 -08:00
Fangjin Yang 9fbee29eb4 change hadoop indexer to be node based 2012-11-05 18:19:04 -08:00
Fangjin Yang 7b2522ff3f allow hadoop druid indexer to register registererers 2012-11-05 16:13:50 -08:00
Fangjin Yang a356447767 Merge branch 'moarunittests' into misc
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
2012-11-05 16:13:20 -08:00
Nelson Ray 0a42f18cc3 HistogramVisual: add support for quantiles 2012-11-05 14:33:15 -08:00
Fangjin Yang 51cd361fbe some changes to how handlers are registered 2012-11-05 13:39:16 -08:00
Fangjin Yang 499263386c timeseries for druid + unit tests 2012-11-05 11:38:03 -08:00
Fangjin Yang 6b247f1599 cleanup redundant code 2012-11-05 11:19:55 -08:00
Fangjin Yang b4e3f98f4b bug fixes for retries without sufficient capacity 2012-11-05 11:18:11 -08:00
Fangjin Yang e9dadcae49 more sensical logic around when to terminate worker nodes 2012-11-02 16:59:49 -07:00
Fangjin Yang a5c3eab2f1 fix merge break 2012-11-02 15:59:55 -07:00
Fangjin Yang b1b611735e Merge branch 'master' into autoscaling
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2012-11-02 15:57:16 -07:00
Fangjin Yang abab35194a general cleanup and improvments for auto scaling 2012-11-02 15:56:18 -07:00
Eric Tschetter 88a3fb842c 1) New attempt at making OpenJDK 6 happy 2012-11-02 14:24:43 -05:00
Eric Tschetter 38ed1d8b75 1) Revert failed attempt at making OpenJDK6 happy. 2012-11-02 14:24:09 -05:00
Eric Tschetter 7921be790b 1) Bump up the maven version pre-requisite to 3.0.3 2012-11-02 14:17:50 -05:00
Eric Tschetter 63b2f502ac 1) Make TimelineEntry in VersionedIntervalTimeline a static class with its own generic parameters to hopefully make OpenJDK happier. 2012-11-02 11:35:23 -05:00
Fangjin Yang 69b929fad5 Merge branch 'master' into moarunittests
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
	merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
2012-11-01 16:41:13 -07:00
Fangjin Yang 09cc2e3353 Merge branch 'master' into moarunittests
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
	merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
	merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2012-11-01 16:39:28 -07:00
Eric Tschetter f07b15db1f Undo massive import re-ordering 2012-11-01 18:38:27 -05:00
Eric Tschetter 8dd6f5c059 1) Refactor the BrokerMain into a BrokerMain and a BrokerNode to allow for extensions
2) Rename ServerMain to ComputeMain
3) Refactor ComputeMain to ComputeMain and ComputeNode to allow for extensions
4) Remove "TheSizeAdjuster", was old legacy stuff that's not relevant anymore
5) Fix bug with registering different IndexIO handlers
6) Adjust it so that when a query type is unknown, it returns an error message instead of just emitting an alert and returning nothing
7) Create super classes for the various *Node classes that allow for code sharing on pieces that are common
2012-11-01 18:20:38 -05:00
cheddar 4928978870 Merge pull request #17 from ianbrandt/codecleanup
Minor code cleanup and other miscellanea.
2012-11-01 16:14:17 -07:00
Fangjin Yang 83a51a801e cleanup redirects and reannounce workers on connection loss 2012-10-31 16:43:47 -07:00
Ian Brandt e7ec073d75 Cleaned Maven dependency configuration. In particular the 'mvn dependency:analyze' report no longer warns of used but undeclared dependencies. 2012-10-31 16:10:07 -07:00
Fangjin Yang 9547ea494d fix bug with provision 2012-10-31 14:45:44 -07:00
Fangjin Yang 11c64593ae improvements to the indexing service and auto scaling 2012-10-31 14:35:09 -07:00
Ian Brandt e8f02219bb Streamlined the Maven POMs, in particular the plugin configurations. The 'mvn versions:display-plugin-updates' report is now warning free, and there are no more Maven warnings reported in Eclipse. 2012-10-31 12:58:16 -07:00
Fangjin Yang fe9cb397a1 remove unncessarily created file 2012-10-31 10:29:03 -07:00
Fangjin Yang d0f66c1fb3 bug fix for merging with empty strings 2012-10-31 10:28:24 -07:00
Ian Brandt e24948319b Removed some dead/unused code. 2012-10-31 00:00:30 -07:00
Ian Brandt c5e2a88187 Resolved dead code warning by null checking the correct variable according to the associated log statement. 2012-10-30 23:59:31 -07:00
Ian Brandt 35f04b4df6 Fixed some minor potential resource leaks. 2012-10-30 23:26:21 -07:00