hbase/contrib/ec2/README.txt

HBase EC2

This collection of scripts allows you to run HBase clusters on Amazon.com's Elastic Compute Cloud (EC2) service described at:

  http://aws.amazon.com/ec2
  
To get help, type the following in a shell:
  
  bin/hbase-ec2

You need both the EC2 API and AMI tools 

  http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351

  http://developer.amazonwebservices.com/connect/entry.jspa?externalID=368&categoryID=88

installed and on the path. 

When setting up keypairs on EC2, be sure to name your keypair as 'root'. 

Quick Start:

1) Download and unzip the EC2 AMI and API tools zipfiles.

   For Ubuntu, "apt-get install ec2-ami-tools ec2-api-tools".

2) Put the tools on the path and set EC2_HOME in the environment to point to
   the top level directory of the API tools.

3) Configure src/contrib/ec2/bin/hbase-ec2-env.sh

   Fill in AWS_ACCOUNT_ID with your EC2 account number.

   Fill in AWS_ACCESS_KEY_ID with your EC2 access key.

   Fill in AWS_SECRET_ACCESS_KEY with your EC2 secret access key.

   Fill in EC2_PRIVATE_KEY with the location of your AWS private key file --
   must begin with 'pk' and end with '.pem'.

   Fill in EC2_CERT with the location of your AWS certificate -- must begin
   with 'cert' and end with '.pem'.

   Make sure the private part of your AWS SSH keypair exists in the same
   directory as EC2_PRIVATE_KEY with the name id_rsa_root. Also, insure that
   the permissions on the private key file are 600 (ONLY owner readable/
   writable).

4) ./bin/hbase-ec2 launch-cluster <name> <nr-zoos> <nr-slaves>, e.g

       ./bin/hbase-ec2 launch-cluster testcluster 3 3

5) Once the above command has finished without error, ./bin/hbase-ec2 login
   <name>, e.g.

       ./bin/hbase-ec2 login testcluster

6) Check that the cluster is up and functional:

       hbase shell
       > status 'simple'

   You should see something like:

       3 live servers
         domU-12-31-39-09-75-11.compute-1.internal:60020 1258653694915
           requests=0, regions=1, usedHeap=29, maxHeap=987
         domU-12-31-39-01-AC-31.compute-1.internal:60020 1258653709041
           requests=0, regions=1, usedHeap=29, maxHeap=987
         domU-12-31-39-01-B0-91.compute-1.internal:60020 1258653706411
           requests=0, regions=0, usedHeap=27, maxHeap=987
       0 dead servers


Extra Packages:

It is possible to specify that extra packages be downloaded and installed on
demand when the master and slave instances start. 

   1. Set up a YUM repository. See: http://yum.baseurl.org/wiki/RepoCreate

   2. Host the repository somewhere public. For example, build the
      repository locally and then copy it up to an S3 bucket.

   3. Create a YUM repository descriptor (.repo file). See:
      http://yum.baseurl.org/wiki/RepoCreate

        [myrepo]
        name = MyRepo
        baseurl = http://mybucket.s3.amazonaws.com/myrepo
        enabled = 1

      Upload the .repo file somewhere public, for example, in the root
      directory of the repository,
      mybucket.s3.amazonaws.com/myrepo/myrepo.repo

   4. Configure hbase-ec2-env.sh thus:

      EXTRA_PACKAGES="http://mybucket.s3.amazonaws.com/myrepo/myrepo.repo \
        pkg1 pkg2 pkg3"

When the master and slave instances start, the .repo file will be added to
the Yum repository list and then Yum will be invoked to pull the packages
listed after the URL.