From 2ddf3c4ef621b48b096e8e959cf7f53ac4adc78f Mon Sep 17 00:00:00 2001 From: Igal Levy Date: Thu, 9 Jan 2014 15:44:54 -0800 Subject: [PATCH] first stab at overhauling this doc to bring it up to date --- docs/content/Booting-a-production-cluster.md | 35 +++++++++----------- 1 file changed, 15 insertions(+), 20 deletions(-) diff --git a/docs/content/Booting-a-production-cluster.md b/docs/content/Booting-a-production-cluster.md index 3f57ce13d1c..52e9b9dea0c 100644 --- a/docs/content/Booting-a-production-cluster.md +++ b/docs/content/Booting-a-production-cluster.md @@ -1,30 +1,25 @@ --- layout: doc_page --- -# Booting a Single Node Cluster # -[Loading Your Data](Tutorial%3A-Loading-Your-Data-Part-2.html) and [All About Queries](Tutorial%3A-All-About-Queries.html) contain recipes to boot a small druid cluster on localhost. Here we will boot a small cluster on EC2. You can checkout the code, or download a tarball from [here](http://static.druid.io/artifacts/druid-services-0.6.46-bin.tar.gz). +# Booting a Druid Cluster +[Loading Your Data](Tutorial%3A-Loading-Your-Data-Part-2.html) and [All About Queries](Tutorial%3A-All-About-Queries.html) contain recipes to boot a small druid cluster on localhost. However, when it's time to run a more realistic setup, for production or just for testing production, you'll want to find a way to start the cluster on multiple hosts. This document describes two different ways to do this: manually, or as a cloud service via Apache Whirr. -The [ec2 run script](https://github.com/metamx/druid/blob/master/examples/bin/run_ec2.sh), run_ec2.sh, is located at 'examples/bin' if you have checked out the code, or at the root of the project if you've downloaded a tarball. The scripts rely on the [Amazon EC2 API Tools](http://aws.amazon.com/developertools/351), and you will need to set three environment variables: +## Manually Booting a Druid Cluster +You can provision individual servers, loading Druid onto each machine (or building it) and setting the required configuration for each type of node. You'll also have to set up required external dependencies. Then you'll have to start each node. this process is outlined in [Tutorial: The Druid Cluster](Tutorial:-The-Druid-Cluster.html). -```bash -# Setup environment for ec2-api-tools -export EC2_HOME=/path/to/ec2-api-tools-1.6.7.4/ -export PATH=$PATH:$EC2_HOME/bin -export AWS_ACCESS_KEY= -export AWS_SECRET_KEY= -``` +## Apache Whirr -Then, booting an ec2 instance running one node of each type is as simple as running the script, run_ec2.sh :) +[Apache Whirr](http://whirr.apache.org/) is a set of libraries for launching cloud services. For Druid, Whirr serves as an easy way to launch a cluster in Amazon AWS by using simple commands and configuration files (called *recipes*). -# Apache Whirr # +You'll need an AWS account, and an EC2 key pair from that account so that Whirr can connect to it. If you haven't generated a key pair, see the [AWS documentation](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html) or see this [Whirr FAQ](http://whirr.apache.org/faq.html#how-do-i-find-my-cloud-credentials). -Apache Whirr is a set of libraries for launching cloud services. You can clone a version of Whirr that includes Druid as a service from git@github.com:rjurney/whirr.git: +### Installing Whirr +To get a version of Whirr that includes and supports a Druid recipe, clone the code from [https://github.com/rjurney/whirr/tree/trunk](https://github.com/rjurney/whirr/tree/trunk) and build Whirr: -```bash -git clone git@github.com:rjurney/whirr.git -cd whirr -git checkout trunk -mvn clean install -Dmaven.test.failure.ignore=true -Dcheckstyle.skip -sp;bin/whirr launch-cluster --config recipes/druid.properties -``` + git clone git@github.com:rjurney/whirr.git + cd whirr + git checkout trunk + mvn clean install -Dmaven.test.failure.ignore=true + +### \ No newline at end of file