Merge pull request #5686 from skierkowski/guides-building-in-cicd
Adds a guide on using Packer with CI/CD
This commit is contained in:
Matthew Hooker
GitHub
commit
8703a71917
Binary file not shown.
|
After Width: | Height: | Size: 325 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 225 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 243 KiB |
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
layout: guides
|
||||
sidebar_current: guides-packer-on-cicd-build-image
|
||||
page_title: Build Images in CI/CD
|
||||
---
|
||||
|
||||
# Build Images in CI/CD
|
||||
|
||||
The following guides from our partners show how to use their services to build
|
||||
images with Packer.
|
||||
|
||||
- How to Build Immutable Infrastructure with Packer and CircleCI Workflows (coming soon)
|
||||
- [Using Packer and Ansible to Build Immutable Infrastructure in CodeShip](https://blog.codeship.com/packer-ansible/)
|
||||
|
||||
The majority of the [Packer Builders](/docs/builders/index.html) can run in
|
||||
a container or VM, a common model used by most CI/CD services. However, the
|
||||
[QEMU builder](/docs/builders/qemu.html) for
|
||||
[KVM](https://www.linux-kvm.org/page/Main_Page) and
|
||||
[Xen](https://www.xenproject.org/) virtual machine images, [VirtualBox
|
||||
builder](/docs/builders/virtualbox.html) for OVA or OVF virtual machines and
|
||||
[VMWare builder](/docs/builders/vmware.html) for use with VMware products
|
||||
require running on a bare-metal machine.
|
||||
|
||||
The [Building a VirtualBox Image with Packer in
|
||||
TeamCity](/guides/packer-on-cicd/building-virtualbox-image.html) guide shows
|
||||
how to create a VirtualBox image using TeamCity's support for running scripts
|
||||
on bare-metal machines.
|
||||
|
|
@ -0,0 +1,142 @@
|
|||
---
|
||||
layout: guides
|
||||
sidebar_current: guides-packer-on-cicd-build-virtualbox
|
||||
page_title: Build a VirtualBox Image with Packer in TeamCity
|
||||
---
|
||||
|
||||
# Build a VirtualBox Image with Packer in TeamCity
|
||||
|
||||
This guide walks through the process of building a VirtualBox image using
|
||||
Packer on a new TeamCity Agent. Before getting started you should have access
|
||||
to a TeamCity Server.
|
||||
|
||||
The Packer VirtualBox builder requires access to VirtualBox, which needs to run
|
||||
on a bare-metal machine, as virtualization is generally not supported on cloud
|
||||
instances. This is also true for the [VMWare](/docs/builders/vmware.html) and
|
||||
the [QEMU](/docs/builders/qemu.html) Packer builders.
|
||||
|
||||
We will use Chef's [Bento boxes](https://github.com/chef/bento) to provision an
|
||||
Ubuntu image on VirtualBox. We will use a fork of this repository as the
|
||||
project we will build.
|
||||
|
||||
## 1. Provision a Bare-metal Machine
|
||||
|
||||
The Packer VirtualBox builder requires running on bare-metal (hardware). If you
|
||||
do not have access to a bare-metal machine, we recommend using
|
||||
[Packet.net](https://www.packet.net/) to obtain a new machine. If you are
|
||||
a first time user of Packet.net, the Packet.net team has provided HashiCorp the
|
||||
coupon code `hash25` which you can use for $25 off to test out this guide. You
|
||||
can use a `baremetal_0` server type for testing, but for regular use, the
|
||||
`baremetal_1` instance may be a better option.
|
||||
|
||||
There is also a [Packet
|
||||
Provider](https://www.terraform.io/docs/providers/packet/index.html) in
|
||||
Terraform you can use to provision the project and instance.
|
||||
|
||||
```hcl
|
||||
provider "packet" { }
|
||||
|
||||
resource "packet_project" "teamcity_agents" {
|
||||
name = "TeamCity"
|
||||
}
|
||||
|
||||
resource "packet_device" "agent" {
|
||||
hostname = "teamcity-agent"
|
||||
plan = "baremetal_0"
|
||||
facility = "ams1"
|
||||
operating_system = "ubuntu_16_04"
|
||||
billing_cycle = "hourly"
|
||||
project_id = "${packet_project.teamcity_project.id}"
|
||||
}
|
||||
```
|
||||
|
||||
## 2. Install VirtualBox and TeamCity dependencies
|
||||
|
||||
VirtualBox must be installed on the new instance, and TeamCity requires the JDK
|
||||
prior to installation. This guide uses Ubuntu as the Linux distribution, so you
|
||||
may need to adjust these commands for your distribution of choice.
|
||||
|
||||
**Install Teamcity Dependencies**
|
||||
|
||||
```shell
|
||||
apt-get upgrade
|
||||
apt-get install -y zip linux-headers-generic linux-headers-4.13.0-16-generic build-essential openjdk-8-jdk
|
||||
```
|
||||
|
||||
**Install VirtualBox**
|
||||
|
||||
```
|
||||
curl -OL "http://download.virtualbox.org/virtualbox/5.2.2/virtualbox-5.2_5.2.2-119230~Ubuntu~xenial_amd64.deb"
|
||||
dpkg -i virtualbox-5.2_5.2.2-119230~Ubuntu~xenial_amd64.deb
|
||||
```
|
||||
|
||||
You can also use the [`remote-exec`
|
||||
provisioner](https://www.terraform.io/docs/provisioners/remote-exec.html) in
|
||||
your Terraform configuration to automatically run these commands when
|
||||
provisioning the new instance.
|
||||
|
||||
## 3. Install Packer
|
||||
|
||||
The TeamCity Agent machine will also need Packer Installed. You can find the
|
||||
latest download link from the [Packer
|
||||
Download](https://www.packer.io/downloads.html) page.
|
||||
|
||||
```shell
|
||||
curl -OL "https://releases.hashicorp.com/packer/1.1.2/packer_1.1.2_linux_amd64.zip"
|
||||
unzip ./packer_1.1.2_linux_amd64.zip
|
||||
```
|
||||
|
||||
Packer is installed at the `/root/packer` path which is used in subsequent
|
||||
steps. If it is installed elsewhere, take note of the path.
|
||||
|
||||
## 4. Install TeamCity Agent
|
||||
|
||||
This guide assume you already have a running instance of TeamCity Server. The
|
||||
new TeamCity Agent can be installed by [downloading a zip file and installing
|
||||
manually](https://confluence.jetbrains.com/display/TCD10//Setting+up+and+Running+Additional+Build+Agents#SettingupandRunningAdditionalBuildAgents-InstallingAdditionalBuildAgents),
|
||||
or using [Agent
|
||||
Push](https://confluence.jetbrains.com/display/TCD10//Setting+up+and+Running+Additional+Build+Agents#SettingupandRunningAdditionalBuildAgents-InstallingviaAgentPush).
|
||||
Once it is installed it should appear in TeamCity as a new Agent.
|
||||
|
||||
Create a new Agent Pool for agents responsible for VirtualBox Packer builds and
|
||||
assign the new Agent to it.
|
||||
|
||||
## 5. Create a New Build in TeamCity
|
||||
|
||||
In TeamCity Server, create a new build, and configure the Version Control
|
||||
Settings to download the Packer build configuration from the VCS repository.
|
||||
|
||||
Add one **Build Step: Command Line** to the build.
|
||||
|
||||
|
||||
|
||||
In the **Script content** field add the following:
|
||||
|
||||
```shell
|
||||
#!/usr/bin/env bash
|
||||
/root/packer build -only=virtualbox-iso -var "headless=true" ubuntu/ubuntu-16.04-amd64.json
|
||||
```
|
||||
|
||||
This will use the `build` command in Packer to build the image defined in
|
||||
`ubuntu/ubuntu-16.04-amd64.json`. It assumes that the VCS repository you're
|
||||
using is a fork of [Chef/Bento](https://github.com/chef/bento). Packer defaults
|
||||
to building VirtualBox machines by launching a GUI that shows the console.
|
||||
Since this will run in CI/CD, use the [`headless`
|
||||
variable](/docs/builders/virtualbox-iso.html#headless) to instruct Packer to
|
||||
start the machine without the console. Packer can build multiple image types,
|
||||
so the [`-only=virtualbox-iso`
|
||||
option](/docs/commands/build.html#only-foo-bar-baz) instructs Packer to only
|
||||
build the builds with the name `virtualbox-iso`.
|
||||
|
||||
## 6. Run a build in TeamCity
|
||||
|
||||
The entire configuration is ready for a new build. Start a new run in TeamCity
|
||||
by pressing “Run”.
|
||||
|
||||
The new run should be triggered and the virtual box image will be built.
|
||||
|
||||
|
||||
|
||||
Once complete, the build status should be updated to complete and successful.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
---
|
||||
layout: guides
|
||||
sidebar_current: guides-packer-on-cicd-index
|
||||
page_title: Build Immutable Infrastructure with Packer in CI/CD
|
||||
---
|
||||
|
||||
# Build Immutable Infrastructure with Packer in CI/CD
|
||||
|
||||
This guide focuses on the following workflow for building immutable
|
||||
infrastructure. This workflow can be manual or automated and it can be
|
||||
implemented with a variety of technologies. The goal of this guide is to show
|
||||
how this workflow can be fully automated using Packer for building images from
|
||||
a continuous integration/continuous deployment (CI/CD) pipeline.
|
||||
|
||||
1. [Build Images using Packer in CI/CD](/guides/packer-on-cicd/build-image-in-cicd.html)
|
||||
2. [Upload the new image to S3](/guides/packer-on-cicd/upload-images-to-artifact.html) for future deployment or use during development
|
||||
3. [Create new Terraform Enterprise runs](/guides/packer-on-cicd/trigger-tfe.html) to provision new instances with the images
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
layout: guides
|
||||
sidebar_current: guides-packer-on-cicd-trigger-tfe-run
|
||||
page_title: Trigger Terraform Enterprise runs
|
||||
---
|
||||
|
||||
# Create Terraform Enterprise Runs
|
||||
|
||||
Once an image is built and uploaded to an artifact store, the next step is to
|
||||
use this new image. In some cases the image will be downloaded by the dev team
|
||||
and used locally in development, like is often done with VirtualBox images with
|
||||
Vagrant. In most other cases, the new image will be used to provision new
|
||||
infrastructure.
|
||||
|
||||
[Terraform](https://www.terraform.io/) is an open source tool that is ideal for
|
||||
provisioning new infrastructure with images generated by Packer, and [Terraform
|
||||
Enterprise](https://www.hashicorp.com/products/terraform/) is the best way to
|
||||
perform automated Terraform runs.
|
||||
|
||||
## Create a Terraform Configuration and Workspace
|
||||
|
||||
The following is a sample Terraform configuration which provisions a new AWS
|
||||
EC2 instance. The `aws_ami_id` is a variable which will be provided when
|
||||
running `terraform plan` and `terraform apply`. This variable references the
|
||||
latest AMI generated with the Packer build in CI/CD.
|
||||
|
||||
```hcl
|
||||
variable "aws_ami_id" { }
|
||||
|
||||
provider "aws" {
|
||||
region = "us-west-2"
|
||||
}
|
||||
|
||||
resource "aws_instance" "web" {
|
||||
ami = "${var.aws_ami_id}"
|
||||
instance_type = "t2.micro"
|
||||
}
|
||||
```
|
||||
|
||||
Terraform Enterprise should have a workspace with this terraform configuration
|
||||
and a placeholder variable `aws_ami_id`.
|
||||
|
||||
## Include Terraform Enterprise in Your CI Builds
|
||||
|
||||
Follow these steps to create a new run from CI/CD after a Packer build is
|
||||
complete and uploaded.
|
||||
|
||||
1. Add a new step to the CI/CD pipeline.
|
||||
2. In the new step add a `curl` call to update the variables in the workspace
|
||||
using the [update variables
|
||||
API](https://www.terraform.io/docs/enterprise-beta/api/variables.html#update-variables),
|
||||
so that Terraform has a reference to the latest image. For the sample
|
||||
configuration above, the `aws_ami_id` variable should be updated to the AMI
|
||||
ID of the latest image.
|
||||
3. In that same step, add another `curl` call to [create a new run via the
|
||||
API](https://www.terraform.io/docs/enterprise-beta/api/run.html#create-a-run).
|
||||
A run performs a plan and apply on the last configuration version created,
|
||||
using the variables set in the workspace. In the previous step we update the
|
||||
variables, so the new run can be created using the previous configuration
|
||||
version.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
layout: guides
|
||||
sidebar_current: guides-packer-on-cicd-upload-image-to-artifact-store
|
||||
page_title: Upload VirtualBox Image to S3
|
||||
---
|
||||
|
||||
# Upload VirtualBox Image to S3
|
||||
|
||||
Once the image is generated it will be used by other parts of your operations
|
||||
workflow. For example, it is common to build VirtualBox images with Packer to
|
||||
be used as base boxes in Vagrant.
|
||||
|
||||
The exact process for uploading images depends on the artifact store and CI
|
||||
system you use. TeamCity provides a [Build
|
||||
Artifacts](https://confluence.jetbrains.com/display/TCD9/Build+Artifact)
|
||||
feature which can be used to store the newly generated image. Other CI/CD
|
||||
services also have similar build artifacts features built in, like [Circle CI
|
||||
Build Artifacts](https://circleci.com/docs/2.0/artifacts/). In addition to the
|
||||
built in artifact stores in CI/CD tools, there are also dedicated universal
|
||||
artifact storage services like
|
||||
[Artifactory](https://confluence.jetbrains.com/display/TCD9/Build+Artifact).
|
||||
All of these are great options for image artifact storage.
|
||||
|
||||
The following example uses TeamCity and Amazon S3.
|
||||
|
||||
## Example: Uploading to S3 in a TeamCity Build
|
||||
|
||||
On the agent machine responsible for building images, install the [AWS Command
|
||||
Line Tool](https://aws.amazon.com/cli/). Since this is a one-time operation,
|
||||
this can be incorporated into the initial agent provisioning step when
|
||||
installing other dependencies. The AWS Command Line tool may require installing
|
||||
additional
|
||||
[dependencies](http://docs.aws.amazon.com/cli/latest/userguide/installing.html)
|
||||
prior.
|
||||
|
||||
```shell
|
||||
pip install awscli
|
||||
```
|
||||
|
||||
In your build configuration in TeamCity Server, add an additional **Build Step:
|
||||
Command Line** and set the **Script content** field to the following:
|
||||
|
||||
```shell
|
||||
awscli s3 cp . s3://bucket/ --exclude “*” --include “*.iso”
|
||||
```
|
||||
|
|
@ -4,6 +4,23 @@
|
|||
<li<%= sidebar_current("guides-veewee-to-packer") %>>
|
||||
<a href="/guides/veewee-to-packer.html">Veewee to Packer</a>
|
||||
</li>
|
||||
<li<%= sidebar_current("guides-packer-on-cicd") %>>
|
||||
<a href="/guides/packer-on-cicd/index.html">Build Immutable Infrastructure with Packer in CI/CD</a>
|
||||
<ul class="nav">
|
||||
<li<%= sidebar_current("guides-packer-on-cicd-build-image") %>>
|
||||
<a href="/guides/packer-on-cicd/build-image-in-cicd.html">Build Images in CI/CD</a>
|
||||
</li>
|
||||
<li<%= sidebar_current("guides-packer-on-cicd-build-virtualbox") %>>
|
||||
<a href="/guides/packer-on-cicd/build-virtualbox-image.html">Build a VirtualBox Image with Packer in TeamCity</a>
|
||||
</li>
|
||||
<li<%= sidebar_current("guides-packer-on-cicd-upload-image-to-artifact-store") %>>
|
||||
<a href="/guides/packer-on-cicd/upload-images-to-artifact.html">Upload a VirtualBox Image to S3</a>
|
||||
</li>
|
||||
<li<%= sidebar_current("guides-packer-on-cicd-trigger-tfe-run") %>>
|
||||
<a href="/guides/packer-on-cicd/trigger-tfe.html">Trigger Terraform Enterprise runs</a>
|
||||
</li>
|
||||
</li>
|
||||
</li>
|
||||
</ul>
|
||||
<% end %>
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue