iSharkFly-Docs
/
packer-cn
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge

This commit is contained in:
bugbuilder 2018-07-24 20:54:08 -04:00
parent f7b45312f1 c8970b86eb
commit 2c4f703ff8
2353 changed files with 237353 additions and 40779 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.gitattributes vendored
View File

@ -1,2 +1,4 @@
* text=auto
*.go text eol=lf
*.sh text eol=lf
common/test-fixtures/root/* eol=lf

217
.github/CONTRIBUTING.md vendored Normal file
View File

@ -0,0 +1,217 @@
# Contributing to Packer
**First:** if you're unsure or afraid of _anything_, just ask or submit the
issue or pull request anyways. You won't be yelled at for giving your best
effort. The worst that can happen is that you'll be politely asked to change
something. We appreciate any sort of contributions, and don't want a wall of
rules to get in the way of that.
However, for those individuals who want a bit more guidance on the best way to
contribute to the project, read on. This document will cover what we're looking
for. By addressing all the points we're looking for, it raises the chances we
can quickly merge or address your contributions.
## Issues
### Reporting an Issue
* Make sure you test against the latest released version. It is possible we
already fixed the bug you're experiencing.
* Run the command with debug output with the environment variable `PACKER_LOG`.
For example: `PACKER_LOG=1 packer build template.json`. Take the _entire_
output and create a [gist](https://gist.github.com) for linking to in your
issue. Packer should strip sensitive keys from the output, but take a look
through just in case.
* Provide a reproducible test case. If a contributor can't reproduce an issue,
then it dramatically lowers the chances it'll get fixed. And in some cases,
the issue will eventually be closed.
* Respond promptly to any questions made by the Packer team to your issue. Stale
issues will be closed.
### Issue Lifecycle
1. The issue is reported.
2. The issue is verified and categorized by a Packer collaborator.
Categorization is done via tags. For example, bugs are marked as "bugs" and
easy fixes are marked as "easy".
3. Unless it is critical, the issue is left for a period of time (sometimes many
weeks), giving outside contributors a chance to address the issue.
4. The issue is addressed in a pull request or commit. The issue will be
referenced in the commit message so that the code that fixes it is clearly
linked.
5. The issue is closed.
## Setting up Go to work on Packer
If you have never worked with Go before, you will have to complete the following
steps in order to be able to compile and test Packer. These instructions target
POSIX-like environments (Mac OS X, Linux, Cygwin, etc.) so you may need to
adjust them for Windows or other shells.
1. [Download](https://golang.org/dl) and install Go. The instructions below are
for go 1.7. Earlier versions of Go are no longer supported.
2. Set and export the `GOPATH` environment variable and update your `PATH`. For
example, you can add the following to your `.bash_profile` (or comparable
shell startup scripts):
```
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
```
3. Download the Packer source (and its dependencies) by running
`go get github.com/hashicorp/packer`. This will download the Packer source to
`$GOPATH/src/github.com/hashicorp/packer`.
4. When working on Packer, first `cd $GOPATH/src/github.com/hashicorp/packer`
so you can run `make` and easily access other files. Run `make help` to get
information about make targets.
5. Make your changes to the Packer source. You can run `make` in
`$GOPATH/src/github.com/hashicorp/packer` to run tests and build the Packer
binary. Any compilation errors will be shown when the binaries are
rebuilding. If you don't have `make` you can simply run
`go build -o bin/packer .` from the project root.
6. After running building Packer successfully, use
`$GOPATH/src/github.com/hashicorp/packer/bin/packer` to build a machine and
verify your changes work. For instance:
`$GOPATH/src/github.com/hashicorp/packer/bin/packer build template.json`.
7. If everything works well and the tests pass, run `go fmt` on your code before
submitting a pull-request.
### Opening an Pull Request
Thank you for contributing! When you are ready to open a pull-request, you will
need to [fork
Packer](https://github.com/hashicorp/packer#fork-destination-box), push your
changes to your fork, and then open a pull-request.
For example, my github username is `cbednarski`, so I would do the following:
```
git checkout -b f-my-feature
# Develop a patch.
git push https://github.com/cbednarski/Packer f-my-feature
```
From there, open your fork in your browser to open a new pull-request.
**Note:** Go infers package names from their file paths. This means `go build`
will break if you `git clone` your fork instead of using `go get` on the main
Packer project.
### Pull Request Lifecycle
1. You are welcome to submit your pull request for commentary or review before
it is fully completed. Please prefix the title of your pull request with
"[WIP]" to indicate this. It's also a good idea to include specific questions
or items you'd like feedback on.
1. Once you believe your pull request is ready to be merged, you can remove any
"[WIP]" prefix from the title and a core team member will review.
1. One of Packer's core team members will look over your contribution and
either provide comments letting you know if there is anything left to do. We
do our best to provide feedback in a timely manner, but it may take some time
for us to respond.
1. Once all outstanding comments and checklist items have been addressed, your
contribution will be merged! Merged PRs will be included in the next
Packer release. The core team takes care of updating the CHANGELOG as they
merge.
1. In rare cases, we might decide that a PR should be closed. We'll make sure to
provide clear reasoning when this happens.
### Tips for Working on Packer
#### Working on forks
The easiest way to work on a fork is to set it as a remote of the Packer
project. After following the steps in "Setting up Go to work on Packer":
1. Navigate to `$GOPATH/src/github.com/hashicorp/packer`
2. Add the remote by running
`git remote add <name of remote> <github url of fork>`. For example:
`git remote add mwhooker https://github.com/mwhooker/packer.git`.
3. Checkout a feature branch: `git checkout -b new-feature`
4. Make changes
5. (Optional) Push your changes to the fork:
`git push -u <name of remote> new-feature`
This way you can push to your fork to create a PR, but the code on disk still
lives in the spot where the go cli tools are expecting to find it.
#### Govendor
If you are submitting a change that requires new or updated dependencies, please
include them in `vendor/vendor.json` and in the `vendor/` folder. This helps
everything get tested properly in CI.
Note that you will need to use [govendor](https://github.com/kardianos/govendor)
to do this. This step is recommended but not required; if you don't use govendor
please indicate in your PR which dependencies have changed and to what versions.
Use `govendor fetch <project>` to add dependencies to the project. See
[govendor quick start](https://github.com/kardianos/govendor#quick-start-also-see-the-faq)
for examples.
Please only apply the minimal vendor changes to get your PR to work. Packer does
not attempt to track the latest version for each dependency.
#### Running Unit Tests
You can run tests for individual packages using commands like this:
```
make test TEST=./builder/amazon/...
```
#### Running Acceptance Tests
Packer has [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing)
for various builders. These typically require an API key (AWS, GCE), or
additional software to be installed on your computer (VirtualBox, VMware).
If you're working on a new builder or builder feature and want verify it is
functioning (and also hasn't broken anything else), we recommend running the
acceptance tests.
**Warning:** The acceptance tests create/destroy/modify _real resources_, which
may incur costs for real money. In the presence of a bug, it is possible that
resources may be left behind, which can cost money even though you were not
using them. We recommend running tests in an account used only for that purpose
so it is easy to see if there are any dangling resources, and so production
resources are not accidentally destroyed or overwritten during testing.
To run the acceptance tests, invoke `make testacc`:
```
make testacc TEST=./builder/amazon/ebs
...
```
The `TEST` variable lets you narrow the scope of the acceptance tests to a
specific package / folder. The `TESTARGS` variable is recommended to filter down
to a specific resource to test, since testing all of them at once can sometimes
take a very long time.
To run only a specific test, use the `-run` argument:
```
make testacc TEST=./builder/amazon/ebs TESTARGS="-run TestBuilderAcc_forceDeleteSnapshot"
```
Acceptance tests typically require other environment variables to be set for
things such as API tokens and keys. Each test should error and tell you which
credentials are missing, so those are not documented here.

1
.gitignore vendored
View File

@ -24,3 +24,4 @@ packer-test*.log
*.iml
Thumbs.db
/packer.exe
.project

8
.travis.yml
View File

@ -6,9 +6,9 @@ sudo: false
language: go
go:
- 1.7.4
- 1.8.3
- 1.9
- 1.9.x
- 1.10.x
- master
install:
- make deps
@ -21,4 +21,6 @@ branches:
- master
matrix:
allow_failures:
- go: master
fast_finish: true

509
CHANGELOG.md
View File

@ -1,35 +1,460 @@
## (UNRELEASED)
## 1.2.6 (Unreleased)
### IMRPOVEMENTS:
### IMPROVEMENTS:
* post-processor/docker-push: Add `aws_profile` option to control the aws profile for ECR. [GH-5470]
* builder/docker: Add `aws_profile` option to control the aws profile for ECR. [GH-5470]
* post-processor/vsphere: Properly capture `ovftool` output. [GH-5499]
* builder/hyper-v: Also disable automatic checkpoints for gen 2 VMs. [GH-5517]
* builder/hyper-v: Add `disk_additional_size` option to allow for up to 64 additional disks. [GH-5491]
* builder/amazon: correctly deregister AMIs when `force_deregister` is set. [GH-5525]
* builder/digitalocean: Add `ipv6` option to enable on droplet. [GH-5534]
* builder/triton: Add `source_machine_image_filter` option to select an image ID based on a variety of parameters. [GH-5538]
* communicator/ssh: Add socks 5 proxy support. [GH-5439]
* builder/lxc: Add new `publish_properties` field to set image properties. [GH-5475]
* builder/virtualbox-ovf: Retry while removing VM to solve for transient errors. [GH-5512]
* builder/google: Interpolate network and subnetwork values, rather than relying on an API call that packer may not have permission for. [GH-5343]
* builder/lxc: Add three new configuration option categories to LXC builder: create_options, start_options, and attach_options. [GH-5530]
* core: Rewrite vagrantfile code to make cross-platform development easier. [5539]
* builder/triton: Update triton-go sdk. [GH-5531]
* builder/google: Add clean_image_name template engine. [GH-5463]
* command/validate: Warn users if config needs fixing. [GH-6423]
### BUG FIXES:
## 1.2.5 (July 16, 2018)
### BUG FIXES:
* builder/alickoud: Fix issue where internet_max_bandwidth_out template option
was not being passed to builder. [GH-6416]
* builder/alicloud: Fix an issue with VPC cleanup. [GH-6418]
* builder/amazon-chroot: Fix communicator bug that broke chroot builds.
[GH-6363]
* builder/amazon: Replace packer's waiters with those from the AWS sdk, solving
several timeout bugs. [GH-6332]
* builder/azure: update azure-sdk-for-go, fixing 32-bit build errors. [GH-6479]
* builder/azure: update the max length of managed_image_resource_group to match
new increased length of 90 characters. [GH-6477]
* builder/hyper-v: Fix secure boot template feature so that it properly passes
the temolate for MicrosoftUEFICertificateAuthority. [GH-6415]
* builder/hyperv: Fix bug in HyperV IP lookups that was causing breakages in
FreeBSD/OpenBSD builds. [GH-6416]
* builder/qemu: Fix error race condition in qemu builder that caused convert to
fail on ubuntu 18.x [GH-6437]
* builder/qemu: vnc_bind_address was not being passed to qemu. [GH-6467]
* builder/virtualbox: Allow iso_url to be a symlink. [GH-6370]
* builder/vmware: Don't fail on DHCP lease files that cannot be read, fixing
bug where builder failed on NAT networks that don't serve DHCP. [GH-6415]
* builder/vmware: Fix bug where we couldn't discover IP if vm_name differed
from the vmx displayName. [GH-6448]
* builder/vmware: Fix validation to prevent hang when remopte_password is not
sent but vmware is building on esxi. [GH-6424]
* builder/vmware:Correctly default the vm export format to ovf; this is what
the docs claimed we already did, but we didn't. [GH-4538]
* communicator/winrm: Revert an attempt to determine whether remote upload
destinations were files or directories, as this broke uploads on systems
without Powershell installed. [GH-6481]
* core: Fix bug in parsing of iso checksum files that arose when setting
iso_url to a relative filepath. [GH-6488]
* core: Fix Packer crash caused by improper error handling in the downloader.
[GH-6381]
* fix: Fix bug where fixer for ssh_private_ip that failed when boolean values
are passed as strings. [GH-6458]
* provisioner/powershell: Make upload of powershell variables retryable, in
case of system restarts. [GH-6388]
### IMPROVEMENTS:
* builder/amazon: Add the ap-northeast-3 region. [GH-6385]
* builder/amazon: Spot requests may now have tags applied using the `spot_tags`
option [GH-5452]
* builder/cloudstack: Add support for Projectid and new config option
prevent_firewall_changes. [GH-6487]
* builder/openstack: Add support for token authorization and cloud.yaml.
[GH-6362]
* builder/oracle-oci: Add new "instance_name" template option. [GH-6408]
* builder/scaleway: Add new "bootscript" parameter, allowing the user to not
use the default local bootscript [GH-6439]
* builder/vmware: Add support for linked clones to vmware-vmx. [GH-6394]
* debug: The -debug flag will now cause Packer to pause between provisioner
scripts in addition to Packer steps. [GH-4663]
* post-processor/googlecompute-import: Added new googlecompute-import post-
processor [GH-6451]
* provisioner/ansible: Add new "playbook_files" option to execute multiple
playbooks within one provisioner call. [GH-5086]
## 1.2.4 (May 29, 2018)
### BUG FIXES:
* builder/amazon: Can now force the chroot builder to mount an entire block
device instead of a partition [GH-6194]
* builder/azure: windows-sql-cloud is now in the default list of projects to
check for provided images. [GH-6210]
* builder/chroot: A new template option, `nvme_device_path` has been added to
provide a workaround for users who need the amazon-chroot builder to mount
a NVMe volume on their instances. [GH-6295]
* builder/hyper-v: Fix command for mounting multiple disks [GH-6267]
* builder/hyperv: Enable IP retrieval for Server 2008 R2 hosts. [GH-6219]
* builder/hyperv: Fix bug in MAC address specification on Hyper-V. [GH-6187]
* builder/parallels-pvm: Add missing disk compaction step. [GH-6202]
* builder/vmware-esxi: Remove floppy files from the remote server on cleanup.
[GH-6206]
* communicator/winrm: Updated dependencies to fix a race condition [GH-6261]
* core: When using `-on-error=[abort|ask]`, output the error to the user.
[GH-6252]
* provisioner/puppet: Extra-Arguments are no longer prematurely
interpolated.[GH-6215]
* provisioner/shell: Remove file stat that was causing problems uploading files
[GH-6239]
### IMPROVEMENTS:
* builder/amazon: Amazon builders other than `chroot` now support T2 unlimited
instances [GH-6265]
* builder/azure: Allow device login for US government cloud. [GH-6105]
* builder/azure: Devicelogin Support for Windows [GH-6285]
* builder/azure: Enable simultaneous builds within one resource group.
[GH-6231]
* builder/azure: Faster deletion of Azure Resource Groups. [GH-6269]
* builder/azure: Updated Azure SDK to v15.0.0 [GH-6224]
* builder/hyper-v: Hyper-V builds now connect to vnc display by default when
building [GH-6243]
* builder/hyper-v: New `use_fixed_vhd_format` allows vm export in an Azure-
compatible format [GH-6101]
* builder/hyperv: New config option for specifying what secure boot template to
use, allowing secure boot of linux vms. [GH-5883]
* builder/qemu: Add support for hvf accelerator. [GH-6193]
* builder/scaleway: Fix SSH communicator connection issue. [GH-6238]
* core: Add opt-in Packer top-level command autocomplete [GH-5454]
* post-processor/shell-local: New options have been added to create feature
parity with the shell-local provisioner. This feature now works on Windows
hosts. [GH-5956]
* provisioner/chef: New config option allows user to skip cleanup of chef
client staging directory. [GH-4300]
* provisioner/shell-local: Can now access automatically-generated WinRM
password as variable [GH-6251]
* provisoner/shell-local: New options have been added to create feature parity
with the shell-local post-processor. This feature now works on Windows
hosts. [GH-5956]
* builder/virtualbox: Use HTTPS to download guest editions, now that it's
available. [GH-6406]
## 1.2.3 (April 25, 2018)
### BUG FIXES:
* builder/azure: Azure CLI may now be logged into several accounts. [GH-6087]
* builder/ebssurrogate: Snapshot all launch devices. [GH-6056]
* builder/hyper-v: Fix CopyExportedVirtualMachine script so it works with
links. [GH-6082]
* builder/oracle-classic: Fix panics when cleaning up resources that haven't
been created. [GH-6095]
* builder/parallels: Allow user to cancel build while the OS is starting up.
[GH-6166]
* builder/qemu: Avoid warning when using raw format. [GH-6080]
* builder/scaleway: Fix compilation issues on solaris/amd64. [GH-6069]
* builder/virtualbox: Fix broken scancodes in boot_command. [GH-6067]
* builder/vmware-iso: Fail in validation if user gives wrong remote_type value.
[GH-4563]
* builder/vmware: Fixed a case-sensitivity issue when determing the network
type during the cloning step in the vmware-vmx builder. [GH-6057]
* builder/vmware: Fixes the DHCP lease and configuration pathfinders for VMware
Player. [GH-6096]
* builder/vmware: Multi-disk VM's can be properly handled by the compacting
stage. [GH-6074]
* common/bootcommand: Fix numerous bugs in the boot command code, and make
supported features consistent across builders. [GH-6129]
* communicator/ssh: Stop trying to discover whether destination is a directory
from uploader. [GH-6124]
* post-processor/vagrant: Large VMDKs should no longer show a 0-byte size on OS
X. [GH-6084]
* post-processor/vsphere: Fix encoding of spaces in passwords for upload.
[GH-6110]
* provisioner/ansible: Pass the inventory_directory configuration option to
ansible -i when it is set. [GH-6065]
* provisioner/powershell: fix bug with SSH communicator + cygwin. [GH-6160]
* provisioner/powershell: The {{.WinRMPassword}} template variable now works
with parallel builders. [GH-6144]
### IMPROVEMENTS:
* builder/alicloud: Update aliyungo common package. [GH-6157]
* builder/amazon: Expose more source ami data as template variables. [GH-6088]
* builder/amazon: Setting `force_delete` will only delete AMIs owned by the
user. This should prevent failures where we try to delete an AMI with a
matching name, but owned by someone else. [GH-6111]
* builder/azure: Users of Powershell provisioner may access the randomly-
generated winrm password using the template variable {{.WinRMPassword}}.
[GH-6113]
* builder/google: Users of Powershell provisioner may access the randomly-
generated winrm password using the template variable {{.WinRMPassword}}.
[GH-6141]
* builder/hyper-v: User can now configure hyper-v disk block size. [GH-5941]
* builder/openstack: Add configuration option for `instance_name`. [GH-6041]
* builder/oracle-classic: Better validation of destination image name.
[GH-6089]
* builder/oracle-oci: New config options for user data and user data file.
[GH-6079]
* builder/oracle-oci: use the official OCI sdk instead of handcrafted client.
[GH-6142]
* builder/triton: Add support to Skip TLS Verification of Triton Certificate.
[GH-6039]
* provisioner/ansible: Ansible users may provide a custom inventory file.
[GH-6107]
* provisioner/file: New `generated` tag allows users to upload files created
during Packer run. [GH-3891]
## 1.2.2 (March 26, 2018)
### BUG FIXES:
* builder/amazon: Fix AWS credential defaulting [GH-6019]
* builder/LXC: make sleep timeout easily configurable [GH-6038]
* builder/virtualbox: Correctly send multi-byte scancodes when typing boot
command. [GH-5987]
* builder/virtualbox: Special boot-commands no longer overwrite previous
commands [GH-6002]
* builder/vmware: Default to disabling XHCI bus for USB on the vmware-iso
builder. [GH-5975]
* builder/vmware: Handle multiple devices per VMware network type [GH-5985]
* communicator/ssh: Handle errors uploading files more gracefully [GH-6033]
* provisioner/powershell: Fix environment variable file escaping. [GH-5973]
### IMPROVEMENTS:
* builder/amazon: Added new region `cn-northwest-1`. [GH-5960]
* builder/amazon: Users may now access the amazon-generated administrator
password [GH-5998]
* builder/azure: Add support concurrent deployments in the same resource group.
[GH-6005]
* builder/azure: Add support for building with additional disks. [GH-5944]
* builder/azure: Add support for marketplace plan information. [GH-5970]
* builder/azure: Make all command output human readable. [GH-5967]
* builder/azure: Respect `-force` for managed image deletion. [GH-6003]
* builder/google: Add option to specify a service account, or to run without
one. [GH-5991] [GH-5928]
* builder/oracle-oci: Add new "use_private_ip" option. [GH-5893]
* post-processor/vagrant: Add LXC support. [GH-5980]
* provisioner/salt-masterless: Added Windows support. [GH-5702]
* provisioner/salt: Add windows support to salt provisioner [GH-6012] [GH-6012]
## 1.2.1 (February 23, 2018)
### BUG FIXES:
* builder/amazon: Fix authorization using assume role. [GH-5914]
* builder/hyper-v: Fix command collisions with VMWare PowerCLI. [GH-5861]
* builder/vmware-iso: Fix panic when building on esx5 remotes. [GH-5931]
* builder/vmware: Fix issue detecting host IP. [GH-5898] [GH-5900]
* provisioner/ansible-local: Fix conflicting escaping schemes for vars provided
via `--extra-vars`. [GH-5888]
### IMPROVEMENTS:
* builder/oracle-classic: Add `snapshot_timeout` option to control how long we
wait for the snapshot to be created. [GH-5932]
* builder/oracle-classic: Add support for WinRM connections. [GH-5929]
## 1.2.0 (February 9, 2018)
### BACKWARDS INCOMPATIBILITIES:
* 3rd party plugins: We have moved internal dependencies, meaning your 3rd
party plugins will no longer compile (however existing builds will still
work fine); the work to fix them is minimal and documented in GH-5810.
[GH-5810]
* builder/amazon: The `ssh_private_ip` option has been removed. Instead, please
use `"ssh_interface": "private"`. A fixer has been written for this, which
can be invoked with `packer fix`. [GH-5876]
* builder/openstack: Extension support has been removed. To use OpenStack
builder with the OpenStack Newton (Oct 2016) or earlier, we recommend you
use Packer v1.1.2 or earlier version.
* core: Affects Windows guests: User variables containing Powershell special
characters no longer need to be escaped.[GH-5376]
* provisioner/file: We've made destination semantics more consistent across the
various communicators. In general, if the destination is a directory, files
will be uploaded into the directory instead of failing. This mirrors the
behavior of `rsync`. There's a chance some users might be depending on the
previous buggy behavior, so it's worth ensuring your configuration is
correct. [GH-5426]
* provisioner/powershell: Regression from v1.1.1 forcing extra escaping of
environment variables in the non-elevated provisioner has been fixed.
[GH-5515] [GH-5872]
### IMPROVEMENTS:
* **New builder:** `ncloud` for building server images using the NAVER Cloud
Platform. [GH-5791]
* **New builder:** `oci-classic` for building new custom images for use with
Oracle Cloud Infrastructure Classic Compute. [GH-5819]
* **New builder:** `scaleway` - The Scaleway Packer builder is able to create
new images for use with Scaleway BareMetal and Virtual cloud server.
[GH-4770]
* builder/amazon: Add `kms_key_id` option to block device mappings. [GH-5774]
* builder/amazon: Add `skip_metadata_api_check` option to skip consulting the
amazon metadata service. [GH-5764]
* builder/amazon: Add Paris region (eu-west-3) [GH-5718]
* builder/amazon: Give better error messages if we have trouble during
authentication. [GH-5764]
* builder/amazon: Remove Session Token (STS) from being shown in the log.
[GH-5665]
* builder/amazon: Replace `InstanceStatusOK` check with `InstanceReady`. This
reduces build times universally while still working for all instance types.
[GH-5678]
* builder/amazon: Report which authentication provider we're using. [GH-5764]
* builder/amazon: Timeout early if metadata service can't be reached. [GH-5764]
* builder/amazon: Warn during prepare if we didn't get both an access key and a
secret key when we were expecting one. [GH-5762]
* builder/azure: Add validation for incorrect VHD URLs [GH-5695]
* builder/docker: Remove credentials from being shown in the log. [GH-5666]
* builder/google: Support specifying licenses for images. [GH-5842]
* builder/hyper-v: Allow MAC address specification. [GH-5709]
* builder/hyper-v: New option to use differential disks and Inline disk
creation to improve build time and reduce disk usage [GH-5631]
* builder/qemu: Add Intel HAXM support to QEMU builder [GH-5738]
* builder/triton: Triton RBAC is now supported. [GH-5741]
* builder/triton: Updated triton-go dependencies, allowing better error
handling. [GH-5795]
* builder/vmware-iso: Add support for cdrom and disk adapter types. [GH-3417]
* builder/vmware-iso: Add support for setting network type and network adapter
type. [GH-3417]
* builder/vmware-iso: Add support for usb/serial/parallel ports. [GH-3417]
* builder/vmware-iso: Add support for virtual soundcards. [GH-3417]
* builder/vmware-iso: More reliably retrieve the guest networking
configuration. [GH-3417]
* builder/vmware: Add support for "super" key in `boot_command`. [GH-5681]
* communicator/ssh: Add session-level keep-alives [GH-5830]
* communicator/ssh: Detect dead connections. [GH-4709]
* core: Gracefully clean up resources on SIGTERM. [GH-5318]
* core: Improved error logging in floppy file handling. [GH-5802]
* core: Improved support for downloading and validating a uri containing a
Windows UNC path or a relative file:// scheme. [GH-2906]
* post-processor/amazon-import: Allow user to specify role name in amazon-
import [GH-5817]
* post-processor/docker: Remove credentials from being shown in the log.
[GH-5666]
* post-processor/google-export: Synchronize credential semantics with the
Google builder. [GH-4148]
* post-processor/vagrant: Add vagrant post-processor support for Google
[GH-5732]
* post-processor/vsphere-template: Now accepts artifacts from the vSphere post-
processor. [GH-5380]
* provisioner/amazon: Use Amazon SDK's InstanceRunning waiter instead of
InstanceStatusOK waiter [GH-5773]
* provisioner/ansible: Improve user retrieval. [GH-5758]
* provisioner/chef: Add support for 'trusted_certs_dir' chef-client
configuration option [GH-5790]
* provisioner/chef: Added Policyfile support to chef-client provisioner.
[GH-5831]
### BUG FIXES:
* builder/alicloud-ecs: Attach keypair before starting instance in alicloud
builder [GH-5739]
* builder/amazon: Fix tagging support when building in us-gov/china. [GH-5841]
* builder/amazon: NewSession now inherits MaxRetries and other settings.
[GH-5719]
* builder/virtualbox: Fix interpolation ordering so that edge cases around
guest_additions_url are handled correctly [GH-5757]
* builder/virtualbox: Fix regression affecting users running Packer on a
Windows host that kept Packer from finding Virtualbox guest additions if
Packer ran on a different drive from the one where the guest additions were
stored. [GH-5761]
* builder/vmware: Fix case where artifacts might not be cleaned up correctly.
[GH-5835]
* builder/vmware: Fixed file handle leak that may have caused race conditions
in vmware builder [GH-5767]
* communicator/ssh: Add deadline to SSH connection to prevent Packer hangs
after script provisioner reboots vm [GH-4684]
* communicator/winrm: Fix issue copying empty directories. [GH-5763]
* provisioner/ansible-local: Fix support for `--extra-vars` in
`extra_arguments`. [GH-5703]
* provisioner/ansible-remote: Fixes an error where Packer's private key can be
overridden by inherited `ansible_ssh_private_key` options. [GH-5869]
* provisioner/ansible: The "default extra variables" feature added in Packer
v1.0.1 caused the ansible-local provisioner to fail when an --extra-vars
argument was specified in the extra_arguments configuration option; this
has been fixed. [GH-5335]
* provisioner/powershell: Regression from v1.1.1 forcing extra escaping of
environment variables in the non-elevated provisioner has been fixed.
[GH-5515] [GH-5872]
## 1.1.3 (December 8, 2017)
### IMPROVEMENTS:
* builder/alicloud-ecs: Add security token support and set TLS handshake
timeout through environment variable. [GH-5641]
* builder/amazon: Add a new parameter `ssh_interface`. Valid values include
`public_ip`, `private_ip`, `public_dns` or `private_dns`. [GH-5630]
* builder/azure: Add sanity checks for resource group names [GH-5599]
* builder/azure: Allow users to specify an existing resource group to use,
instead of creating a new one for every run. [GH-5548]
* builder/hyper-v: Add support for differencing disk. [GH-5458]
* builder/vmware-iso: Improve logging of network errors. [GH-5456]
* core: Add new `packer_version` template engine. [GH-5619]
* core: Improve logic checking for downloaded ISOs in case where user has
provided more than one URL in `iso_urls` [GH-5632]
* provisioner/ansible-local: Add ability to clean staging directory. [GH-5618]
### BUG FIXES:
* builder/amazon: Allow `region` to appear in `ami_regions`. [GH-5660]
* builder/amazon: `C5` instance types now build more reliably. [GH-5678]
* builder/amazon: Correctly set AWS region if given in template along with a
profile. [GH-5676]
* builder/amazon: Prevent `sriov_support` and `ena_support` from being used
with spot instances, which would cause a build failure. [GH-5679]
* builder/hyper-v: Fix interpolation context for user variables in
`boot_command` [GH-5547]
* builder/qemu: Set default disk size to 40960 MB to prevent boot failures.
[GH-5588]
* builder/vmware: Correctly detect Windows boot on vmware workstation.
[GH-5672]
* core: Fix windows path regression when downloading ISOs. [GH-5591]
* provisioner/chef: Fix chef installs on Windows. [GH-5649]
## 1.1.2 (November 15, 2017)
### IMPROVEMENTS:
* builder/amazon: Correctly deregister AMIs when `force_deregister` is set.
[GH-5525]
* builder/digitalocean: Add `ipv6` option to enable on droplet. [GH-5534]
* builder/docker: Add `aws_profile` option to control the aws profile for ECR.
[GH-5470]
* builder/google: Add `clean_image_name` template engine. [GH-5463]
* builder/google: Allow selecting container optimized images. [GH-5576]
* builder/google: Interpolate network and subnetwork values, rather than
relying on an API call that packer may not have permission for. [GH-5343]
* builder/hyper-v: Add `disk_additional_size` option to allow for up to 64
additional disks. [GH-5491]
* builder/hyper-v: Also disable automatic checkpoints for gen 2 VMs. [GH-5517]
* builder/lxc: Add new `publish_properties` field to set image properties.
[GH-5475]
* builder/lxc: Add three new configuration option categories to LXC builder:
`create_options`, `start_options`, and `attach_options`. [GH-5530]
* builder/triton: Add `source_machine_image_filter` option to select an image
ID based on a variety of parameters. [GH-5538]
* builder/virtualbox-ovf: Error during prepare if source path doesn't exist.
[GH-5573]
* builder/virtualbox-ovf: Retry while removing VM to solve for transient
errors. [GH-5512]
* communicator/ssh: Add socks 5 proxy support. [GH-5439]
* core/iso_config: Support relative paths in checksum file. [GH-5578]
* core: Rewrite vagrantfile code to make cross-platform development easier.
[GH-5539]
* post-processor/docker-push: Add `aws_profile` option to control the aws
profile for ECR. [GH-5470]
* post-processor/vsphere: Properly capture `ovftool` output. [GH-5499]
### BUG FIXES:
* builder/docker: Remove `login_email`, which no longer exists in the docker client. [GH-5511]
* builder/triton: Fix a bug where partially created images can be reported as complete. [GH-5566]
* builder/amazon: region is set from profile, if profile is set, rather than being overridden by metadata. [GH-5562]
* provisioner/windows-restart: Wait for restart no longer endlessly loops if user specifies a custom restart check command. [GH-5563]
* post-processor/vsphere: Use the vm disk path information to re-create the vmx datastore path. [GH-5567]
* builder/amazon: Add a delay option to security group waiter. [GH-5536]
* builder/amazon: Fix regressions relating to spot instances and EBS volumes. [GH-5495]
* builder/hyperv: Fix admin check that was causing powershell failures. [GH-5510]
* builder/oracle: Defaulting of OCI builder region will first check the packer template and the OCI config file. [GH-5407]
* builder/amazon: Fix regressions relating to spot instances and EBS volumes.
[GH-5495]
* builder/amazon: Set region from profile, if profile is set, rather than being
overridden by metadata. [GH-5562]
* builder/docker: Remove `login_email`, which no longer exists in the docker
client. [GH-5511]
* builder/hyperv: Fix admin check that was causing powershell failures.
[GH-5510]
* builder/oracle: Defaulting of OCI builder region will first check the packer
template and the OCI config file. [GH-5407]
* builder/triton: Fix a bug where partially created images can be reported as
complete. [GH-5566]
* post-processor/vsphere: Use the vm disk path information to re-create the vmx
datastore path. [GH-5567]
* provisioner/windows-restart: Wait for restart no longer endlessly loops if
user specifies a custom restart check command. [GH-5563]
## 1.1.1 (October 13, 2017)
@ -212,7 +637,7 @@
* builder/cloudstack: Properly report back errors. [GH-5103] [GH-5123]
* builder/docker: Fix windows filepath in docker-toolbox call [GH-4887]
* builder/docker: Fix windows filepath in docker-toolbox call. [GH-4887]
* builder/hyperv: Use SID to verify membersip in Admin group, fixing for non-
* builder/hyperv: Use SID to verify membership in Admin group, fixing for non-
english users. [GH-5022]
* builder/hyperv: Verify membership in the group Hyper-V Administrators by SID
not name. [GH-5022]
@ -238,7 +663,7 @@
* core: Remove logging that shouldn't be there when running commands. [GH-5042]
* provisioner/shell: Fix bug where scripts were being run under `sh`. [GH-5043]
### IMRPOVEMENTS:
### IMPROVEMENTS:
* provisioner/windows-restart: make it clear that timeouts come from the
provisioner, not winrm. [GH-5040]
@ -459,7 +884,7 @@
* builder/amazon: Crashes when new EBS vols are used. [GH-4308]
* builder/amazon: Fix crash in amazon-instance. [GH-4372]
* builder/amazon: fix run volume tagging [GH-4420]
* builder/amazon: fix when using non-existant security\_group\_id. [GH-4425]
* builder/amazon: fix when using non-existent security\_group\_id. [GH-4425]
* builder/amazon: Properly error if we don't have the
ec2:DescribeSecurityGroups permission. [GH-4304]
* builder/amazon: Properly wait for security group to exist. [GH-4369]
@ -1122,7 +1547,7 @@
* builder/parallels: Support Parallels Desktop 11. [GH-2199]
* builder/openstack: Add `rackconnect_wait` for Rackspace customers to wait for
RackConnect data to appear
* buidler/openstack: Add `ssh_interface` option for rackconnect for users that
* builder/openstack: Add `ssh_interface` option for rackconnect for users that
have prohibitive firewalls
* builder/openstack: Flavor names can be used as well as refs
* builder/openstack: Add `availability_zone` [GH-2016]
@ -1153,7 +1578,7 @@
* core: Fix potential panic for post-processor plugin exits. [GH-2098]
* core: `PACKER_CONFIG` may point to a non-existent file. [GH-2226]
* builder/amazon: Allow spaces in AMI names when using `clean_ami_name` [GH-2182]
* builder/amazon: Remove deprecated ec2-upload-bundle paramger. [GH-1931]
* builder/amazon: Remove deprecated ec2-upload-bundle parameter. [GH-1931]
* builder/amazon: Use IAM Profile to upload bundle if provided. [GH-1985]
* builder/amazon: Use correct exit code after SSH authentication failed. [GH-2004]
* builder/amazon: Retry finding created instance for eventual
@ -1236,7 +1661,7 @@
* builder/googlecompute: Support for ubuntu-os-cloud project
* builder/googlecompute: Support for OAuth2 to avoid client secrets file
* builder/googlecompute: GCE image from persistant disk instead of tarball
* builder/googlecompute: GCE image from persistent disk instead of tarball
* builder/qemu: Checksum type "none" can be used
* provisioner/chef: Generate a node name if none available
* provisioner/chef: Added ssl_verify_mode configuration
@ -1370,7 +1795,7 @@
* builder/docker: Can now specify login credentials to pull images.
* builder/docker: Support mounting additional volumes. [GH-1430]
* builder/parallels/all: Path to tools ISO is calculated automatically. [GH-1455]
* builder/parallels-pvm: `reassign_mac` option to choose wehther or not
* builder/parallels-pvm: `reassign_mac` option to choose whether or not
to generate a new MAC address. [GH-1461]
* builder/qemu: Can specify "none" acceleration type. [GH-1395]
* builder/qemu: Can specify "tcg" acceleration type. [GH-1395]
@ -1399,7 +1824,7 @@
manager certs. [GH-1137]
* builder/amazon/all: `delete_on_termination` set to false will work.
* builder/amazon/all: Fix race condition on setting tags. [GH-1367]
* builder/amazon/all: More desctriptive error messages if Amazon only
* builder/amazon/all: More descriptive error messages if Amazon only
sends an error code. [GH-1189]
* builder/docker: Error if `DOCKER_HOST` is set.
* builder/docker: Remove the container during cleanup. [GH-1206]
@ -1813,7 +2238,7 @@
* builder/digitalocean: scrub API keys from config debug output. [GH-516]
* builder/virtualbox: error if VirtualBox version cant be detected. [GH-488]
* builder/virtualbox: detect if vboxdrv isn't properly setup. [GH-488]
* builder/virtualbox: sleep a bit before export to ensure the sesssion
* builder/virtualbox: sleep a bit before export to ensure the session
is unlocked. [GH-512]
* builder/virtualbox: create SATA drives properly on VirtualBox 4.3. [GH-547]
* builder/virtualbox: support user templates in SSH key path. [GH-539]
@ -1970,7 +2395,7 @@
* builder/virtualbox,vmware: Support SHA512 as a checksum type. [GH-356]
* builder/vmware: The root hard drive type can now be specified with
"disk_type_id" for advanced users. [GH-328]
* provisioner/salt-masterless: Ability to specfy a minion config. [GH-264]
* provisioner/salt-masterless: Ability to specify a minion config. [GH-264]
* provisioner/salt-masterless: Ability to upload pillars. [GH-353]
### IMPROVEMENTS:
@ -2029,7 +2454,7 @@
* core: All HTTP downloads across Packer now support the standard
proxy environmental variables (`HTTP_PROXY`, `NO_PROXY`, etc.) [GH-252]
* builder/amazon: API requests will use HTTP proxy if specified by
enviromental variables.
environmental variables.
* builder/digitalocean: API requests will use HTTP proxy if specified
by environmental variables.
@ -2075,11 +2500,11 @@
* builder/amazon-instance: send IAM instance profile data. [GH-294]
* builder/digitalocean: API request parameters are properly URL
encoded. [GH-281]
* builder/virtualbox: dowload progress won't be shown until download
* builder/virtualbox: download progress won't be shown until download
actually starts. [GH-288]
* builder/virtualbox: floppy files names of 13 characters are now properly
written to the FAT12 filesystem. [GH-285]
* builder/vmware: dowload progress won't be shown until download
* builder/vmware: download progress won't be shown until download
actually starts. [GH-288]
* builder/vmware: interrupt works while typing commands over VNC.
* builder/virtualbox: floppy files names of 13 characters are now properly
@ -2201,7 +2626,7 @@
### BUG FIXES:
* builder/amazon/all: Gracefully handle when AMI appears to not exist
while AWS state is propogating. [GH-207]
while AWS state is propagating. [GH-207]
* builder/virtualbox: Trim carriage returns for Windows to properly
detect VM state on Windows. [GH-218]
* core: build names no longer cause invalid config errors. [GH-197]
@ -2226,7 +2651,7 @@
* Amazon EBS builder can now optionally use a pre-made security group
instead of randomly generating one.
* DigitalOcean API key and client IDs can now be passed in as
environmental variables. See the documentatin for more details.
environmental variables. See the documentation for more details.
* VirtualBox and VMware can now have `floppy_files` specified to attach
floppy disks when booting. This allows for unattended Windows installs.
* `packer build` has a new `-force` flag that forces the removal of
@ -2283,7 +2708,7 @@
* core: Non-200 response codes on downloads now show proper errors.
[GH-141]
* amazon-ebs: SSH handshake is retried. [GH-130]
* vagrant: The `BuildName` template propery works properly in
* vagrant: The `BuildName` template property works properly in
the output path.
* vagrant: Properly configure the provider-specific post-processors so
things like `vagrantfile_template` work. [GH-129]

2
CODEOWNERS
View File

@ -13,6 +13,8 @@
/builder/oracle/ @prydie @owainlewis
/builder/profitbricks/ @jasmingacic
/builder/triton/ @jen20 @sean-
/builder/ncloud/ @YuSungDuk
/builder/scaleway/ @dimtion @edouardb
# provisioners

158
CONTRIBUTING.md
View File

@ -1,158 +0,0 @@
# Contributing to Packer
**First:** if you're unsure or afraid of _anything_, just ask
or submit the issue or pull request anyways. You won't be yelled at for
giving your best effort. The worst that can happen is that you'll be
politely asked to change something. We appreciate any sort of contributions,
and don't want a wall of rules to get in the way of that.
However, for those individuals who want a bit more guidance on the
best way to contribute to the project, read on. This document will cover
what we're looking for. By addressing all the points we're looking for,
it raises the chances we can quickly merge or address your contributions.
## Issues
### Reporting an Issue
* Make sure you test against the latest released version. It is possible
we already fixed the bug you're experiencing.
* Run the command with debug ouput with the environment variable
`PACKER_LOG`. For example: `PACKER_LOG=1 packer build template.json`. Take
the *entire* output and create a [gist](https://gist.github.com) for linking
to in your issue. Packer should strip sensitive keys from the output,
but take a look through just in case.
* Provide a reproducible test case. If a contributor can't reproduce an
issue, then it dramatically lowers the chances it'll get fixed. And in
some cases, the issue will eventually be closed.
* Respond promptly to any questions made by the Packer team to your issue.
Stale issues will be closed.
### Issue Lifecycle
1. The issue is reported.
2. The issue is verified and categorized by a Packer collaborator.
Categorization is done via tags. For example, bugs are marked as "bugs"
and easy fixes are marked as "easy".
3. Unless it is critical, the issue is left for a period of time (sometimes
many weeks), giving outside contributors a chance to address the issue.
4. The issue is addressed in a pull request or commit. The issue will be
referenced in the commit message so that the code that fixes it is clearly
linked.
5. The issue is closed.
## Setting up Go to work on Packer
If you have never worked with Go before, you will have to complete the
following steps in order to be able to compile and test Packer. These instructions target POSIX-like environments (Mac OS X, Linux, Cygwin, etc.) so you may need to adjust them for Windows or other shells.
1. [Download](https://golang.org/dl) and install Go. The instructions below
are for go 1.7. Earlier versions of Go are no longer supported.
2. Set and export the `GOPATH` environment variable and update your `PATH`. For
example, you can add to your `.bash_profile`.
```
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
```
3. Download the Packer source (and its dependencies) by running `go get
github.com/hashicorp/packer`. This will download the Packer source to
`$GOPATH/src/github.com/hashicorp/packer`.
4. When working on packer `cd $GOPATH/src/github.com/hashicorp/packer` so you
can run `make` and easily access other files. Run `make help` to get
information about make targets.
5. Make your changes to the Packer source. You can run `make` in
`$GOPATH/src/github.com/hashicorp/packer` to run tests and build the packer
binary. Any compilation errors will be shown when the binaries are
rebuilding. If you don't have `make` you can simply run `go build -o bin/packer .` from the project root.
6. After running building packer successfully, use
`$GOPATH/src/github.com/hashicorp/packer/bin/packer` to build a machine and
verify your changes work. For instance: `$GOPATH/src/github.com/hashicorp/packer/bin/packer build template.json`.
7. If everything works well and the tests pass, run `go fmt` on your code
before submitting a pull-request.
### Opening an Pull Request
When you are ready to open a pull-request, you will need to [fork packer](https://github.com/hashicorp/packer#fork-destination-box), push your changes to your fork, and then open a pull-request.
For example, my github username is `cbednarski` so I would do the following:
git checkout -b f-my-feature
// develop a patch
git push https://github.com/cbednarski/packer f-my-feature
From there, open your fork in your browser to open a new pull-request.
**Note** Go infers package names from their filepaths. This means `go build` will break if you `git clone` your fork instead of using `go get` on the main packer project.
### Tips for Working on Packer
#### Working on forks
The easiest way to work on a fork is to set it as a remote of the packer project. After following the steps in "Setting up Go to work on Packer":
1. Navigate to $GOPATH/src/github.com/hashicorp/packer
2. Add the remote `git remote add <name of remote> <github url of fork>`. For example `git remote add mwhooker https://github.com/mwhooker/packer.git`.
3. Checkout a feature branch: `git checkout -b new-feature`
4. Make changes
5. (Optional) Push your changes to the fork: `git push -u <name of remote> new-feature`
This way you can push to your fork to create a PR, but the code on disk still lives in the spot where the go cli tools are expecting to find it.
#### Govendor
If you are submitting a change that requires new or updated dependencies, please include them in `vendor/vendor.json` and in the `vendor/` folder. This helps everything get tested properly in CI.
Note that you will need to use [govendor](https://github.com/kardianos/govendor) to do this. This step is recommended but not required; if you don't use govendor please indicate in your PR which dependencies have changed and to what versions.
Use `govendor fetch <project>` to add dependencies to the project. See
[govendor quick
start](https://github.com/kardianos/govendor#quick-start-also-see-the-faq) for
examples.
Please only apply the minimal vendor changes to get your PR to work. Packer does not attempt to track the latest version for each dependency.
#### Running Unit Tests
You can run tests for individual packages using commands like this:
$ make test TEST=./builder/amazon/...
#### Running Acceptance Tests
Packer has [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing)
for various builders. These typically require an API key (AWS, GCE), or
additional software to be installed on your computer (VirtualBox, VMware).
If you're working on a new builder or builder feature and want verify it is functioning (and also hasn't broken anything else), we recommend running the
acceptance tests.
**Warning:** The acceptance tests create/destroy/modify *real resources*, which
may incur costs for real money. In the presence of a bug, it is possible that resources may be left behind, which can cost money even though you were not using them. We recommend running tests in an account used only for that purpose so it is easy to see if there are any dangling resources, and so production resources are not accidentally destroyed or overwritten during testing.
To run the acceptance tests, invoke `make testacc`:
$ make testacc TEST=./builder/amazon/ebs
...
The `TEST` variable lets you narrow the scope of the acceptance tests to a
specific package / folder. The `TESTARGS` variable is recommended to filter
down to a specific resource to test, since testing all of them at once can
sometimes take a very long time.
Acceptance tests typically require other environment variables to be set for
things such as API tokens and keys. Each test should error and tell you which
credentials are missing, so those are not documented here.

23
Makefile
View File

@ -4,11 +4,13 @@ VET?=$(shell ls -d */ | grep -v vendor | grep -v website)
GITSHA:=$(shell git rev-parse HEAD)
# Get the current local branch name from git (if we can, this may be blank)
GITBRANCH:=$(shell git symbolic-ref --short HEAD 2>/dev/null)
GOFMT_FILES?=$$(find . -not -path "./vendor/*" -name "*.go")
GOOS=$(shell go env GOOS)
GOARCH=$(shell go env GOARCH)
GOPATH=$(shell go env GOPATH)
# gofmt
UNFORMATTED_FILES=$(shell find . -not -path "./vendor/*" -name "*.go" | xargs gofmt -s -l)
# Get the git commit
GIT_DIRTY=$(shell test -n "`git status --porcelain`" && echo "+CHANGES" || true)
GIT_COMMIT=$(shell git rev-parse --short HEAD)
@ -41,8 +43,11 @@ package:
@sh -c "$(CURDIR)/scripts/dist.sh $(VERSION)"
deps:
@go get golang.org/x/tools/cmd/goimports
@go get golang.org/x/tools/cmd/stringer
@go get -u github.com/mna/pigeon
@go get github.com/kardianos/govendor
@go get golang.org/x/tools/cmd/goimports
@govendor sync
dev: deps ## Build and install a development build
@ -57,10 +62,18 @@ dev: deps ## Build and install a development build
@cp $(GOPATH)/bin/packer pkg/$(GOOS)_$(GOARCH)
fmt: ## Format Go code
@gofmt -w -s $(GOFMT_FILES)
@gofmt -w -s main.go $(UNFORMATTED_FILES)
fmt-check: ## Check go code formatting
$(CURDIR)/scripts/gofmtcheck.sh $(GOFMT_FILES)
@echo "==> Checking that code complies with gofmt requirements..."
@if [ ! -z "$(UNFORMATTED_FILES)" ]; then \
echo "gofmt needs to be run on the following files:"; \
echo "$(UNFORMATTED_FILES)" | xargs -n1; \
echo "You can use the command: \`make fmt\` to reformat code."; \
exit 1; \
else \
echo "Check passed."; \
fi
fmt-docs:
@find ./website/source/docs -name "*.md" -exec pandoc --wrap auto --columns 79 --atx-headers -s -f "markdown_github+yaml_metadata_block" -t "markdown_github+yaml_metadata_block" {} -o {} \;
@ -73,6 +86,8 @@ fmt-examples:
# source files.
generate: deps ## Generate dynamically generated code
go generate .
gofmt -w common/bootcommand/boot_command.go
goimports -w common/bootcommand/boot_command.go
gofmt -w command/plugin.go
test: deps fmt-check ## Run unit tests
@ -91,7 +106,7 @@ testrace: deps ## Test for race conditions
@go test -race $(TEST) $(TESTARGS) -timeout=2m
updatedeps:
@echo "INFO: Packer deps are managed by govendor. See CONTRIBUTING.md"
@echo "INFO: Packer deps are managed by govendor. See .github/CONTRIBUTING.md"
help:
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

43
README.md
View File

@ -9,10 +9,10 @@
[travis]: https://travis-ci.org/hashicorp/packer
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/miavlgnp989e5obc/branch/master?svg=true
[appveyor]: https://ci.appveyor.com/project/hashicorp/packer
[godoc-badge]: https://godoc.org/github.com/mitchellh/packer?status.svg
[godoc]: https://godoc.org/github.com/mitchellh/packer
[report-badge]: https://goreportcard.com/badge/github.com/mitchellh/packer
[report]: https://goreportcard.com/report/github.com/mitchellh/packer
[godoc-badge]: https://godoc.org/github.com/hashicorp/packer?status.svg
[godoc]: https://godoc.org/github.com/hashicorp/packer
[report-badge]: https://goreportcard.com/badge/github.com/hashicorp/packer
[report]: https://goreportcard.com/report/github.com/hashicorp/packer
* Website: https://www.packer.io
* IRC: `#packer-tool` on Freenode
@ -23,24 +23,8 @@ from a single source configuration.
Packer is lightweight, runs on every major operating system, and is highly
performant, creating machine images for multiple platforms in parallel. Packer
comes out of the box with support for the following platforms:
* Amazon EC2 (AMI). Both EBS-backed and instance-store AMIs
* Azure
* CloudStack
* DigitalOcean
* Docker
* Google Compute Engine
* Hyper-V
* 1&1
* OpenStack
* Oracle Cloud Infrastructure
* Parallels
* ProfitBricks
* QEMU. Both KVM and Xen images.
* Triton (Joyent Public Cloud)
* VMware
* VirtualBox
comes out of the box with support for many platforms, the full list of which can
be found at https://www.packer.io/docs/builders/index.html.
Support for other platforms can be added via plugins.
@ -48,10 +32,6 @@ The images that Packer creates can easily be turned into
[Vagrant](http://www.vagrantup.com) boxes.
## Quick Start
Download and install packages and dependencies
```
go get github.com/hashicorp/packer
```
**Note:** There is a great
[introduction and getting started guide](https://www.packer.io/intro)
@ -59,8 +39,10 @@ for those with a bit more patience. Otherwise, the quick start below
will get you up and running quickly, at the sacrifice of not explaining some
key points.
First, [download a pre-built Packer binary](https://www.packer.io/downloads.html)
for your operating system or [compile Packer yourself](CONTRIBUTING.md#setting-up-go-to-work-on-packer).
First, [download a pre-built Packer
binary](https://www.packer.io/downloads.html) for your operating system or
[compile Packer
yourself](https://github.com/hashicorp/packer/blob/master/.github/CONTRIBUTING.md#setting-up-go-to-work-on-packer).
After Packer is installed, create your first template, which tells Packer
what platforms to build images for and how you want to build them. In our
@ -108,4 +90,7 @@ https://www.packer.io/docs
## Developing Packer
See [CONTRIBUTING.md](https://github.com/hashicorp/packer/blob/master/CONTRIBUTING.md) for best practices and instructions on setting up your development environment to work on Packer.
See
[CONTRIBUTING.md](https://github.com/hashicorp/packer/blob/master/.github/CONTRIBUTING.md)
for best practices and instructions on setting up your development environment
to work on Packer.

9
Vagrantfile vendored
View File

@ -5,6 +5,10 @@ LINUX_BASE_BOX = "bento/ubuntu-16.04"
FREEBSD_BASE_BOX = "jen20/FreeBSD-12.0-CURRENT"
Vagrant.configure(2) do |config|
if Vagrant.has_plugin?("vagrant-cachier")
config.cache.scope = :box
end
# Compilation and development boxes
config.vm.define "linux", autostart: true, primary: true do |vmCfg|
vmCfg.vm.box = LINUX_BASE_BOX
@ -69,11 +73,6 @@ def configureProviders(vmCfg, cpus: "2", memory: "2048")
end
end
vmCfg.vm.provider "virtualbox" do |v|
v.memory = memory
v.cpus = cpus
end
return vmCfg
end

18
azure-merge.sh
View File

@ -1,18 +0,0 @@
PACKER=$GOPATH/src/github.com/mitchellh/packer
AZURE=/tmp/packer-azure
ls $AZURE >/dev/null || git clone https://github.com/Azure/packer-azure /tmp/packer-azure
PWD=`pwd`
cd $AZURE && git pull
cd $PWD
# copy things
cp -r $AZURE/packer/builder/azure $PACKER/builder/
cp -r $AZURE/packer/communicator/* $PACKER/communicator/
cp -r $AZURE/packer/provisioner/azureVmCustomScriptExtension $PACKER/provisioner/
# remove legacy API client
rm -rf $PACKER/builder/azure/smapi
# fix imports
find $PACKER/builder/azure/ -type f | grep ".go" | xargs sed -e 's/Azure\/packer-azure\/packer\/builder\/azure/mitchellh\/packer\/builder\/azure/g' -i ''

8
builder/alicloud/ecs/access_config.go
View File

@ -15,6 +15,7 @@ type AlicloudAccessConfig struct {
AlicloudSecretKey string `mapstructure:"secret_key"`
AlicloudRegion string `mapstructure:"region"`
AlicloudSkipValidation bool `mapstructure:"skip_region_validation"`
SecurityToken string `mapstructure:"security_token"`
}
// Client for AlicloudClient
@ -22,7 +23,12 @@ func (c *AlicloudAccessConfig) Client() (*ecs.Client, error) {
if err := c.loadAndValidate(); err != nil {
return nil, err
}
client := ecs.NewECSClient(c.AlicloudAccessKey, c.AlicloudSecretKey, common.Region(c.AlicloudRegion))
if c.SecurityToken == "" {
c.SecurityToken = os.Getenv("SECURITY_TOKEN")
}
client := ecs.NewECSClientWithSecurityToken(c.AlicloudAccessKey, c.AlicloudSecretKey,
c.SecurityToken, common.Region(c.AlicloudRegion))
client.SetBusinessInfo("Packer")
if _, err := client.DescribeRegions(); err != nil {
return nil, err

22
builder/alicloud/ecs/builder.go
View File

@ -6,12 +6,13 @@ import (
"log"
"fmt"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
// The unique ID for this builder
@ -88,12 +89,12 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
steps = []multistep.Step{
&stepPreValidate{
AlicloudDestImageName: b.config.AlicloudImageName,
ForceDelete: b.config.AlicloudImageForceDetele,
ForceDelete: b.config.AlicloudImageForceDelete,
},
&stepCheckAlicloudSourceImage{
SourceECSImageId: b.config.AlicloudSourceImage,
},
&StepConfigAlicloudKeyPair{
&stepConfigAlicloudKeyPair{
Debug: b.config.PackerDebug,
KeyPairName: b.config.SSHKeyPairName,
PrivateKeyFile: b.config.Comm.SSHPrivateKey,
@ -131,14 +132,15 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
RegionId: b.config.AlicloudRegion,
InternetChargeType: b.config.InternetChargeType,
InternetMaxBandwidthOut: b.config.InternetMaxBandwidthOut,
InstnaceName: b.config.InstanceName,
InstanceName: b.config.InstanceName,
ZoneId: b.config.ZoneId,
})
if b.chooseNetworkType() == VpcNet {
steps = append(steps, &setpConfigAlicloudEIP{
steps = append(steps, &stepConfigAlicloudEIP{
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
RegionId: b.config.AlicloudRegion,
InternetChargeType: b.config.InternetChargeType,
InternetMaxBandwidthOut: b.config.InternetMaxBandwidthOut,
})
} else {
steps = append(steps, &stepConfigAlicloudPublicIP{
@ -146,9 +148,9 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
})
}
steps = append(steps,
&stepAttachKeyPair{},
&stepRunAlicloudInstance{},
&stepMountAlicloudDisk{},
&stepAttachKeyPar{},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: SSHHost(
@ -164,17 +166,17 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
ForceStop: b.config.ForceStopInstance,
},
&stepDeleteAlicloudImageSnapshots{
AlicloudImageForceDeteleSnapshots: b.config.AlicloudImageForceDeteleSnapshots,
AlicloudImageForceDetele: b.config.AlicloudImageForceDetele,
AlicloudImageForceDeleteSnapshots: b.config.AlicloudImageForceDeleteSnapshots,
AlicloudImageForceDelete: b.config.AlicloudImageForceDelete,
AlicloudImageName: b.config.AlicloudImageName,
},
&stepCreateAlicloudImage{},
&setpRegionCopyAlicloudImage{
&stepRegionCopyAlicloudImage{
AlicloudImageDestinationRegions: b.config.AlicloudImageDestinationRegions,
AlicloudImageDestinationNames: b.config.AlicloudImageDestinationNames,
RegionId: b.config.AlicloudRegion,
},
&setpShareAlicloudImage{
&stepShareAlicloudImage{
AlicloudImageShareAccounts: b.config.AlicloudImageShareAccounts,
AlicloudImageUNShareAccounts: b.config.AlicloudImageUNShareAccounts,
RegionId: b.config.AlicloudRegion,

9
builder/alicloud/ecs/image_config.go
View File

@ -3,10 +3,11 @@ package ecs
import (
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/hashicorp/packer/template/interpolate"
"regexp"
"strings"
"github.com/denverdino/aliyungo/common"
"github.com/hashicorp/packer/template/interpolate"
)
type AlicloudDiskDevice struct {
@ -31,8 +32,8 @@ type AlicloudImageConfig struct {
AlicloudImageUNShareAccounts []string `mapstructure:"image_unshare_account"`
AlicloudImageDestinationRegions []string `mapstructure:"image_copy_regions"`
AlicloudImageDestinationNames []string `mapstructure:"image_copy_names"`
AlicloudImageForceDetele bool `mapstructure:"image_force_delete"`
AlicloudImageForceDeteleSnapshots bool `mapstructure:"image_force_delete_snapshots"`
AlicloudImageForceDelete bool `mapstructure:"image_force_delete"`
AlicloudImageForceDeleteSnapshots bool `mapstructure:"image_force_delete_snapshots"`
AlicloudImageForceDeleteInstances bool `mapstructure:"image_force_delete_instances"`
AlicloudImageSkipRegionValidation bool `mapstructure:"skip_region_validation"`
AlicloudDiskDevices `mapstructure:",squash"`

2
builder/alicloud/ecs/packer_helper.go
View File

@ -3,8 +3,8 @@ package ecs
import (
"fmt"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
func message(state multistep.StateBag, module string) {

2
builder/alicloud/ecs/run_config.go
View File

@ -57,7 +57,7 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
}
if c.InstanceType == "" {
errs = append(errs, errors.New("An aliclod_instance_type must be specified"))
errs = append(errs, errors.New("An alicloud_instance_type must be specified"))
}
if c.UserData != "" && c.UserDataFile != "" {

3
builder/alicloud/ecs/run_config_test.go
View File

@ -2,6 +2,7 @@ package ecs
import (
"io/ioutil"
"os"
"testing"
"github.com/hashicorp/packer/helper/communicator"
@ -68,6 +69,7 @@ func TestRunConfigPrepare_UserData(t *testing.T) {
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserData = "foo"
@ -92,6 +94,7 @@ func TestRunConfigPrepare_UserDataFile(t *testing.T) {
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserDataFile = tf.Name()

2
builder/alicloud/ecs/ssh_helper.go
View File

@ -7,7 +7,7 @@ import (
"time"
packerssh "github.com/hashicorp/packer/communicator/ssh"
"github.com/mitchellh/multistep"
"github.com/hashicorp/packer/helper/multistep"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/agent"
)

12
builder/alicloud/ecs/step_attach_keypair.go
View File

@ -1,19 +1,21 @@
package ecs
import (
"context"
"fmt"
"time"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
"time"
)
type stepAttachKeyPar struct {
type stepAttachKeyPair struct {
}
func (s *stepAttachKeyPar) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepAttachKeyPair) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
keyPairName := state.Get("keyPair").(string)
if keyPairName == "" {
return multistep.ActionContinue
@ -48,7 +50,7 @@ func (s *stepAttachKeyPar) Run(state multistep.StateBag) multistep.StepAction {
return multistep.ActionContinue
}
func (s *stepAttachKeyPar) Cleanup(state multistep.StateBag) {
func (s *stepAttachKeyPair) Cleanup(state multistep.StateBag) {
keyPairName := state.Get("keyPair").(string)
if keyPairName == "" {
return

5
builder/alicloud/ecs/step_check_source_image.go
View File

@ -1,19 +1,20 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepCheckAlicloudSourceImage struct {
SourceECSImageId string
}
func (s *stepCheckAlicloudSourceImage) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepCheckAlicloudSourceImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(Config)
ui := state.Get("ui").(packer.Ui)

11
builder/alicloud/ecs/step_config_eip.go
View File

@ -1,28 +1,31 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type setpConfigAlicloudEIP struct {
type stepConfigAlicloudEIP struct {
AssociatePublicIpAddress bool
RegionId string
InternetChargeType string
InternetMaxBandwidthOut int
allocatedId string
}
func (s *setpConfigAlicloudEIP) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepConfigAlicloudEIP) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
ui.Say("Allocating eip")
ipaddress, allocateId, err := client.AllocateEipAddress(&ecs.AllocateEipAddressArgs{
RegionId: common.Region(s.RegionId), InternetChargeType: common.InternetChargeType(s.InternetChargeType),
Bandwidth: s.InternetMaxBandwidthOut,
})
if err != nil {
state.Put("error", err)
@ -54,7 +57,7 @@ func (s *setpConfigAlicloudEIP) Run(state multistep.StateBag) multistep.StepActi
return multistep.ActionContinue
}
func (s *setpConfigAlicloudEIP) Cleanup(state multistep.StateBag) {
func (s *stepConfigAlicloudEIP) Cleanup(state multistep.StateBag) {
if len(s.allocatedId) == 0 {
return
}

9
builder/alicloud/ecs/step_config_key_pair.go
View File

@ -1,6 +1,7 @@
package ecs
import (
"context"
"fmt"
"io/ioutil"
"os"
@ -8,11 +9,11 @@ import (
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepConfigAlicloudKeyPair struct {
type stepConfigAlicloudKeyPair struct {
Debug bool
SSHAgentAuth bool
DebugKeyPath string
@ -24,7 +25,7 @@ type StepConfigAlicloudKeyPair struct {
keyName string
}
func (s *StepConfigAlicloudKeyPair) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepConfigAlicloudKeyPair) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
if s.PrivateKeyFile != "" {
@ -107,7 +108,7 @@ func (s *StepConfigAlicloudKeyPair) Run(state multistep.StateBag) multistep.Step
return multistep.ActionContinue
}
func (s *StepConfigAlicloudKeyPair) Cleanup(state multistep.StateBag) {
func (s *stepConfigAlicloudKeyPair) Cleanup(state multistep.StateBag) {
// If no key name is set, then we never created it, so just return
// If we used an SSH private key file, do not go about deleting
// keypairs

11
builder/alicloud/ecs/step_config_public_ip.go
View File

@ -1,19 +1,20 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepConfigAlicloudPublicIP struct {
publicIPAdress string
RegionId string
publicIPAddress string
RegionId string
}
func (s *stepConfigAlicloudPublicIP) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepConfigAlicloudPublicIP) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
@ -24,7 +25,7 @@ func (s *stepConfigAlicloudPublicIP) Run(state multistep.StateBag) multistep.Ste
ui.Say(fmt.Sprintf("Error allocating public ip: %s", err))
return multistep.ActionHalt
}
s.publicIPAdress = ipaddress
s.publicIPAddress = ipaddress
ui.Say(fmt.Sprintf("Allocated public ip address %s.", ipaddress))
state.Put("ipaddress", ipaddress)
return multistep.ActionContinue

5
builder/alicloud/ecs/step_config_security_group.go
View File

@ -1,14 +1,15 @@
package ecs
import (
"context"
"errors"
"fmt"
"time"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepConfigAlicloudSecurityGroup struct {
@ -20,7 +21,7 @@ type stepConfigAlicloudSecurityGroup struct {
isCreate bool
}
func (s *stepConfigAlicloudSecurityGroup) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepConfigAlicloudSecurityGroup) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
networkType := state.Get("networktype").(InstanceNetWork)

8
builder/alicloud/ecs/step_config_vpc.go
View File

@ -1,14 +1,15 @@
package ecs
import (
"context"
"errors"
"fmt"
"time"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepConfigAlicloudVPC struct {
@ -18,7 +19,7 @@ type stepConfigAlicloudVPC struct {
isCreate bool
}
func (s *stepConfigAlicloudVPC) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepConfigAlicloudVPC) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(Config)
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
@ -84,7 +85,8 @@ func (s *stepConfigAlicloudVPC) Cleanup(state multistep.StateBag) {
e, _ := err.(*common.Error)
if (e.Code == "DependencyViolation.Instance" || e.Code == "DependencyViolation.RouteEntry" ||
e.Code == "DependencyViolation.VSwitch" ||
e.Code == "DependencyViolation.SecurityGroup") && time.Now().Before(timeoutPoint) {
e.Code == "DependencyViolation.SecurityGroup" ||
e.Code == "Forbbiden") && time.Now().Before(timeoutPoint) {
time.Sleep(1 * time.Second)
continue
}

7
builder/alicloud/ecs/step_config_vswitch.go
View File

@ -1,14 +1,15 @@
package ecs
import (
"context"
"errors"
"fmt"
"time"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepConfigAlicloudVSwitch struct {
@ -19,7 +20,7 @@ type stepConfigAlicloudVSwitch struct {
VSwitchName string
}
func (s *stepConfigAlicloudVSwitch) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepConfigAlicloudVSwitch) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
vpcId := state.Get("vpcid").(string)
@ -136,7 +137,7 @@ func (s *stepConfigAlicloudVSwitch) Cleanup(state multistep.StateBag) {
e, _ := err.(*common.Error)
if (e.Code == "IncorrectVSwitchStatus" || e.Code == "DependencyViolation" ||
e.Code == "DependencyViolation.HaVip" ||
e.Code == "IncorretRouteEntryStatus") && time.Now().Before(timeoutPoint) {
e.Code == "IncorrectRouteEntryStatus") && time.Now().Before(timeoutPoint) {
time.Sleep(1 * time.Second)
continue
}

5
builder/alicloud/ecs/step_create_image.go
View File

@ -1,19 +1,20 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepCreateAlicloudImage struct {
image *ecs.ImageType
}
func (s *stepCreateAlicloudImage) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepCreateAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(Config)
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)

11
builder/alicloud/ecs/step_create_instance.go
View File

@ -1,14 +1,15 @@
package ecs
import (
"context"
"fmt"
"io/ioutil"
"log"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepCreateAlicloudInstance struct {
@ -20,12 +21,12 @@ type stepCreateAlicloudInstance struct {
RegionId string
InternetChargeType string
InternetMaxBandwidthOut int
InstnaceName string
InstanceName string
ZoneId string
instance *ecs.InstanceAttributesType
}
func (s *stepCreateAlicloudInstance) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepCreateAlicloudInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(Config)
ui := state.Get("ui").(packer.Ui)
@ -62,7 +63,7 @@ func (s *stepCreateAlicloudInstance) Run(state multistep.StateBag) multistep.Ste
IoOptimized: ioOptimized,
VSwitchId: vswitchId,
SecurityGroupId: securityGroupId,
InstanceName: s.InstnaceName,
InstanceName: s.InstanceName,
Password: password,
ZoneId: s.ZoneId,
DataDisk: diskDeviceToDiskType(config.AlicloudImageConfig.ECSImagesDiskMappings),
@ -88,7 +89,7 @@ func (s *stepCreateAlicloudInstance) Run(state multistep.StateBag) multistep.Ste
InternetMaxBandwidthOut: s.InternetMaxBandwidthOut,
IoOptimized: ioOptimized,
SecurityGroupId: securityGroupId,
InstanceName: s.InstnaceName,
InstanceName: s.InstanceName,
Password: password,
ZoneId: s.ZoneId,
DataDisk: diskDeviceToDiskType(config.AlicloudImageConfig.ECSImagesDiskMappings),

13
builder/alicloud/ecs/step_delete_images_snapshots.go
View File

@ -1,28 +1,29 @@
package ecs
import (
"context"
"fmt"
"log"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepDeleteAlicloudImageSnapshots struct {
AlicloudImageForceDetele bool
AlicloudImageForceDeteleSnapshots bool
AlicloudImageForceDelete bool
AlicloudImageForceDeleteSnapshots bool
AlicloudImageName string
}
func (s *stepDeleteAlicloudImageSnapshots) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepDeleteAlicloudImageSnapshots) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
config := state.Get("config").(Config)
ui.Say("Deleting image snapshots.")
// Check for force delete
if s.AlicloudImageForceDetele {
if s.AlicloudImageForceDelete {
images, _, err := client.DescribeImages(&ecs.DescribeImagesArgs{
RegionId: common.Region(config.AlicloudRegion),
ImageName: s.AlicloudImageName,
@ -42,7 +43,7 @@ func (s *stepDeleteAlicloudImageSnapshots) Run(state multistep.StateBag) multist
ui.Error(err.Error())
return multistep.ActionHalt
}
if s.AlicloudImageForceDeteleSnapshots {
if s.AlicloudImageForceDeleteSnapshots {
for _, diskDevice := range image.DiskDeviceMappings.DiskDeviceMapping {
if err := client.DeleteSnapshot(diskDevice.SnapshotId); err != nil {
err := fmt.Errorf("Deleting ECS snapshot failed: %s", err)

5
builder/alicloud/ecs/step_mount_disk.go
View File

@ -1,17 +1,18 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepMountAlicloudDisk struct {
}
func (s *stepMountAlicloudDisk) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepMountAlicloudDisk) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
config := state.Get("config").(Config)
ui := state.Get("ui").(packer.Ui)

5
builder/alicloud/ecs/step_pre_validate.go
View File

@ -1,12 +1,13 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepPreValidate struct {
@ -14,7 +15,7 @@ type stepPreValidate struct {
ForceDelete bool
}
func (s *stepPreValidate) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepPreValidate) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
if s.ForceDelete {
ui.Say("Force delete flag found, skipping prevalidating image name.")

9
builder/alicloud/ecs/step_region_copy_image.go
View File

@ -1,21 +1,22 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type setpRegionCopyAlicloudImage struct {
type stepRegionCopyAlicloudImage struct {
AlicloudImageDestinationRegions []string
AlicloudImageDestinationNames []string
RegionId string
}
func (s *setpRegionCopyAlicloudImage) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepRegionCopyAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
if len(s.AlicloudImageDestinationRegions) == 0 {
return multistep.ActionContinue
}
@ -51,7 +52,7 @@ func (s *setpRegionCopyAlicloudImage) Run(state multistep.StateBag) multistep.St
return multistep.ActionContinue
}
func (s *setpRegionCopyAlicloudImage) Cleanup(state multistep.StateBag) {
func (s *stepRegionCopyAlicloudImage) Cleanup(state multistep.StateBag) {
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if cancelled || halted {

9
builder/alicloud/ecs/step_run_instance.go
View File

@ -1,17 +1,18 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepRunAlicloudInstance struct {
}
func (s *stepRunAlicloudInstance) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepRunAlicloudInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
@ -42,8 +43,8 @@ func (s *stepRunAlicloudInstance) Cleanup(state multistep.StateBag) {
ui := state.Get("ui").(packer.Ui)
client := state.Get("client").(*ecs.Client)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
instanceAttrubite, _ := client.DescribeInstanceAttribute(instance.InstanceId)
if instanceAttrubite.Status == ecs.Starting || instanceAttrubite.Status == ecs.Running {
instanceAttribute, _ := client.DescribeInstanceAttribute(instance.InstanceId)
if instanceAttribute.Status == ecs.Starting || instanceAttribute.Status == ecs.Running {
if err := client.StopInstance(instance.InstanceId, true); err != nil {
ui.Say(fmt.Sprintf("Error stopping instance %s, it may still be around %s", instance.InstanceId, err))
return

9
builder/alicloud/ecs/step_share_image.go
View File

@ -1,21 +1,22 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/common"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type setpShareAlicloudImage struct {
type stepShareAlicloudImage struct {
AlicloudImageShareAccounts []string
AlicloudImageUNShareAccounts []string
RegionId string
}
func (s *setpShareAlicloudImage) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepShareAlicloudImage) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
ui := state.Get("ui").(packer.Ui)
alicloudImages := state.Get("alicloudimages").(map[string]string)
@ -36,7 +37,7 @@ func (s *setpShareAlicloudImage) Run(state multistep.StateBag) multistep.StepAct
return multistep.ActionContinue
}
func (s *setpShareAlicloudImage) Cleanup(state multistep.StateBag) {
func (s *stepShareAlicloudImage) Cleanup(state multistep.StateBag) {
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if cancelled || halted {

5
builder/alicloud/ecs/step_stop_instance.go
View File

@ -1,18 +1,19 @@
package ecs
import (
"context"
"fmt"
"github.com/denverdino/aliyungo/ecs"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepStopAlicloudInstance struct {
ForceStop bool
}
func (s *stepStopAlicloudInstance) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepStopAlicloudInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
client := state.Get("client").(*ecs.Client)
instance := state.Get("instance").(*ecs.InstanceAttributesType)
ui := state.Get("ui").(packer.Ui)

18
builder/amazon/chroot/builder.go
View File

@ -13,9 +13,9 @@ import (
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
// The unique ID for this builder
@ -33,9 +33,10 @@ type Config struct {
CommandWrapper string `mapstructure:"command_wrapper"`
CopyFiles []string `mapstructure:"copy_files"`
DevicePath string `mapstructure:"device_path"`
NVMEDevicePath string `mapstructure:"nvme_device_path"`
FromScratch bool `mapstructure:"from_scratch"`
MountOptions []string `mapstructure:"mount_options"`
MountPartition int `mapstructure:"mount_partition"`
MountPartition string `mapstructure:"mount_partition"`
MountPath string `mapstructure:"mount_path"`
PostMountCommands []string `mapstructure:"post_mount_commands"`
PreMountCommands []string `mapstructure:"pre_mount_commands"`
@ -43,6 +44,7 @@ type Config struct {
RootVolumeSize int64 `mapstructure:"root_volume_size"`
SourceAmi string `mapstructure:"source_ami"`
SourceAmiFilter awscommon.AmiFilterOptions `mapstructure:"source_ami_filter"`
RootVolumeTags awscommon.TagMap `mapstructure:"root_volume_tags"`
ctx interpolate.Context
}
@ -66,6 +68,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"ami_description",
"snapshot_tags",
"tags",
"root_volume_tags",
"command_wrapper",
"post_mount_commands",
"pre_mount_commands",
@ -112,8 +115,8 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
b.config.MountPath = "/mnt/packer-amazon-chroot-volumes/{{.Device}}"
}
if b.config.MountPartition == 0 {
b.config.MountPartition = 1
if b.config.MountPartition == "" {
b.config.MountPartition = "1"
}
// Accumulate any errors or warnings
@ -173,7 +176,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
return warns, errs
}
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey))
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return warns, nil
}
@ -198,6 +201,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("ec2", ec2conn)
state.Put("awsSession", session)
state.Put("hook", hook)
state.Put("ui", ui)
state.Put("wrappedCommand", CommandWrapper(wrappedCommand))
@ -228,6 +232,8 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
&StepPrepareDevice{},
&StepCreateVolume{
RootVolumeSize: b.config.RootVolumeSize,
RootVolumeTags: b.config.RootVolumeTags,
Ctx: b.config.ctx,
},
&StepAttachVolume{},
&StepEarlyUnflock{},
@ -305,7 +311,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
artifact := &awscommon.Artifact{
Amis: state.Get("amis").(map[string]string),
BuilderIdValue: BuilderId,
Conn: ec2conn,
Session: session,
}
return artifact, nil

14
builder/amazon/chroot/builder_test.go
View File

@ -71,11 +71,16 @@ func TestBuilderPrepare_ChrootMounts(t *testing.T) {
if err != nil {
t.Errorf("err: %s", err)
}
}
func TestBuilderPrepare_ChrootMountsBadDefaults(t *testing.T) {
b := &Builder{}
config := testConfig()
config["chroot_mounts"] = [][]string{
{"bad"},
}
warnings, err = b.Prepare(config)
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}
@ -135,9 +140,14 @@ func TestBuilderPrepare_CopyFiles(t *testing.T) {
if len(b.config.CopyFiles) != 1 && b.config.CopyFiles[0] != "/etc/resolv.conf" {
t.Errorf("Was expecting default value for copy_files.")
}
}
func TestBuilderPrepare_CopyFilesNoDefault(t *testing.T) {
b := &Builder{}
config := testConfig()
config["copy_files"] = []string{}
warnings, err = b.Prepare(config)
warnings, err := b.Prepare(config)
if len(warnings) > 0 {
t.Fatalf("bad: %#v", warnings)
}

2
builder/amazon/chroot/cleanup.go
View File

@ -1,7 +1,7 @@
package chroot
import (
"github.com/mitchellh/multistep"
"github.com/hashicorp/packer/helper/multistep"
)
// Cleanup is an interface that some steps implement for early cleanup.

3
builder/amazon/chroot/communicator_test.go
View File

@ -1,8 +1,9 @@
package chroot
import (
"github.com/hashicorp/packer/packer"
"testing"
"github.com/hashicorp/packer/packer"
)
func TestCommunicator_ImplementsCommunicator(t *testing.T) {

10
builder/amazon/chroot/device_test.go Normal file
View File

@ -0,0 +1,10 @@
package chroot
import "testing"
func TestDevicePrefixMatch(t *testing.T) {
/*
if devicePrefixMatch("nvme0n1") != "" {
}
*/
}

6
builder/amazon/chroot/run_local_commands.go
View File

@ -3,8 +3,8 @@ package chroot
import (
"fmt"
sl "github.com/hashicorp/packer/common/shell-local"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/post-processor/shell-local"
"github.com/hashicorp/packer/template/interpolate"
)
@ -21,7 +21,9 @@ func RunLocalCommands(commands []string, wrappedCommand CommandWrapper, ctx inte
}
ui.Say(fmt.Sprintf("Executing command: %s", command))
comm := &shell_local.Communicator{}
comm := &sl.Communicator{
ExecuteCommand: []string{"sh", "-c", command},
}
cmd := &packer.RemoteCmd{Command: command}
if err := cmd.StartWithUi(comm, ui); err != nil {
return fmt.Errorf("Error executing command: %s", err)

59
builder/amazon/chroot/step_attach_volume.go
View File

@ -1,15 +1,15 @@
package chroot
import (
"errors"
"context"
"fmt"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepAttachVolume attaches the previously created volume to an
@ -23,7 +23,7 @@ type StepAttachVolume struct {
volumeId string
}
func (s *StepAttachVolume) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepAttachVolume) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
device := state.Get("device").(string)
instance := state.Get("instance").(*ec2.Instance)
@ -51,35 +51,7 @@ func (s *StepAttachVolume) Run(state multistep.StateBag) multistep.StepAction {
s.volumeId = volumeId
// Wait for the volume to become attached
stateChange := awscommon.StateChangeConf{
Pending: []string{"attaching"},
StepState: state,
Target: "attached",
Refresh: func() (interface{}, string, error) {
attempts := 0
for attempts < 30 {
resp, err := ec2conn.DescribeVolumes(&ec2.DescribeVolumesInput{VolumeIds: []*string{&volumeId}})
if err != nil {
return nil, "", err
}
if len(resp.Volumes[0].Attachments) > 0 {
a := resp.Volumes[0].Attachments[0]
return a, *a.State, nil
}
// When Attachment on volume is not present sleep for 2s and retry
attempts += 1
ui.Say(fmt.Sprintf(
"Volume %s show no attachments. Attempt %d/30. Sleeping for 2s and will retry.",
volumeId, attempts))
time.Sleep(2 * time.Second)
}
// Attachment on volume is not present after all attempts
return nil, "", errors.New("No attachments on volume.")
},
}
_, err = awscommon.WaitForState(&stateChange)
err = awscommon.WaitUntilVolumeAttached(ctx, ec2conn, s.volumeId)
if err != nil {
err := fmt.Errorf("Error waiting for volume: %s", err)
state.Put("error", err)
@ -115,26 +87,7 @@ func (s *StepAttachVolume) CleanupFunc(state multistep.StateBag) error {
s.attached = false
// Wait for the volume to detach
stateChange := awscommon.StateChangeConf{
Pending: []string{"attaching", "attached", "detaching"},
StepState: state,
Target: "detached",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeVolumes(&ec2.DescribeVolumesInput{VolumeIds: []*string{&s.volumeId}})
if err != nil {
return nil, "", err
}
v := resp.Volumes[0]
if len(v.Attachments) > 0 {
return v, *v.Attachments[0].State, nil
} else {
return v, "detached", nil
}
},
}
_, err = awscommon.WaitForState(&stateChange)
err = awscommon.WaitUntilVolumeDetached(aws.BackgroundContext(), ec2conn, s.volumeId)
if err != nil {
return fmt.Errorf("Error waiting for volume: %s", err)
}

5
builder/amazon/chroot/step_check_root_device.go
View File

@ -1,17 +1,18 @@
package chroot
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepCheckRootDevice makes sure the root device on the AMI is EBS-backed.
type StepCheckRootDevice struct{}
func (s *StepCheckRootDevice) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepCheckRootDevice) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
image := state.Get("source_image").(*ec2.Image)
ui := state.Get("ui").(packer.Ui)

5
builder/amazon/chroot/step_chroot_provision.go
View File

@ -1,17 +1,18 @@
package chroot
import (
"context"
"log"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepChrootProvision provisions the instance within a chroot.
type StepChrootProvision struct {
}
func (s *StepChrootProvision) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepChrootProvision) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
hook := state.Get("hook").(packer.Hook)
mountPath := state.Get("mount_path").(string)
ui := state.Get("ui").(packer.Ui)

8
builder/amazon/chroot/step_copy_files.go
View File

@ -2,11 +2,13 @@ package chroot
import (
"bytes"
"context"
"fmt"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
"log"
"path/filepath"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
// StepCopyFiles copies some files from the host into the chroot environment.
@ -18,7 +20,7 @@ type StepCopyFiles struct {
files []string
}
func (s *StepCopyFiles) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepCopyFiles) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
mountPath := state.Get("mount_path").(string)
ui := state.Get("ui").(packer.Ui)

49
builder/amazon/chroot/step_create_volume.go
View File

@ -1,14 +1,16 @@
package chroot
import (
"context"
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
"github.com/hashicorp/packer/template/interpolate"
)
// StepCreateVolume creates a new volume from the snapshot of the root
@ -19,14 +21,36 @@ import (
type StepCreateVolume struct {
volumeId string
RootVolumeSize int64
RootVolumeTags awscommon.TagMap
Ctx interpolate.Context
}
func (s *StepCreateVolume) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepCreateVolume) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
ui := state.Get("ui").(packer.Ui)
volTags, err := s.RootVolumeTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging volumes: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
// Collect tags for tagging on resource creation
var tagSpecs []*ec2.TagSpecification
if len(volTags) > 0 {
runVolTags := &ec2.TagSpecification{
ResourceType: aws.String("volume"),
Tags: volTags,
}
tagSpecs = append(tagSpecs, runVolTags)
}
var createVolume *ec2.CreateVolumeInput
if config.FromScratch {
createVolume = &ec2.CreateVolumeInput{
@ -68,6 +92,10 @@ func (s *StepCreateVolume) Run(state multistep.StateBag) multistep.StepAction {
}
}
if len(tagSpecs) > 0 {
createVolume.SetTagSpecifications(tagSpecs)
volTags.Report(ui)
}
log.Printf("Create args: %+v", createVolume)
createVolumeResp, err := ec2conn.CreateVolume(createVolume)
@ -83,22 +111,7 @@ func (s *StepCreateVolume) Run(state multistep.StateBag) multistep.StepAction {
log.Printf("Volume ID: %s", s.volumeId)
// Wait for the volume to become ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"creating"},
StepState: state,
Target: "available",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeVolumes(&ec2.DescribeVolumesInput{VolumeIds: []*string{&s.volumeId}})
if err != nil {
return nil, "", err
}
v := resp.Volumes[0]
return v, *v.State, nil
},
}
_, err = awscommon.WaitForState(&stateChange)
err = awscommon.WaitUntilVolumeAvailable(ctx, ec2conn, s.volumeId)
if err != nil {
err := fmt.Errorf("Error waiting for volume: %s", err)
state.Put("error", err)

8
builder/amazon/chroot/step_early_cleanup.go
View File

@ -1,17 +1,19 @@
package chroot
import (
"context"
"fmt"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
"log"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
// StepEarlyCleanup performs some of the cleanup steps early in order to
// prepare for snapshotting and creating an AMI.
type StepEarlyCleanup struct{}
func (s *StepEarlyCleanup) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepEarlyCleanup) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
cleanupKeys := []string{
"copy_files_cleanup",

8
builder/amazon/chroot/step_early_unflock.go
View File

@ -1,16 +1,18 @@
package chroot
import (
"context"
"fmt"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
"log"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
// StepEarlyUnflock unlocks the flock.
type StepEarlyUnflock struct{}
func (s *StepEarlyUnflock) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepEarlyUnflock) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
cleanup := state.Get("flock_cleanup").(Cleanup)
ui := state.Get("ui").(packer.Ui)

8
builder/amazon/chroot/step_flock.go
View File

@ -1,12 +1,14 @@
package chroot
import (
"context"
"fmt"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
"log"
"os"
"path/filepath"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
// StepFlock provisions the instance within a chroot.
@ -17,7 +19,7 @@ type StepFlock struct {
fh *os.File
}
func (s *StepFlock) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepFlock) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
lockfile := "/var/lock/packer-chroot/lock"

9
builder/amazon/chroot/step_instance_info.go
View File

@ -1,28 +1,29 @@
package chroot
import (
"context"
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepInstanceInfo verifies that this builder is running on an EC2 instance.
type StepInstanceInfo struct{}
func (s *StepInstanceInfo) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepInstanceInfo) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
session := state.Get("awsSession").(*session.Session)
ui := state.Get("ui").(packer.Ui)
// Get our own instance ID
ui.Say("Gathering information about this EC2 instance...")
sess := session.New()
ec2meta := ec2metadata.New(sess)
ec2meta := ec2metadata.New(session)
identity, err := ec2meta.GetInstanceIdentityDocument()
if err != nil {
err := fmt.Errorf(

19
builder/amazon/chroot/step_mount_device.go
View File

@ -2,6 +2,7 @@ package chroot
import (
"bytes"
"context"
"fmt"
"log"
"os"
@ -9,9 +10,9 @@ import (
"strings"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
type mountPathData struct {
@ -25,15 +26,19 @@ type mountPathData struct {
// mount_device_cleanup CleanupFunc - To perform early cleanup
type StepMountDevice struct {
MountOptions []string
MountPartition int
MountPartition string
mountPath string
}
func (s *StepMountDevice) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepMountDevice) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ui := state.Get("ui").(packer.Ui)
device := state.Get("device").(string)
if config.NVMEDevicePath != "" {
// customizable device path for mounting NVME block devices on c5 and m5 HVM
device = config.NVMEDevicePath
}
wrappedCommand := state.Get("wrappedCommand").(CommandWrapper)
var virtualizationType string
@ -46,6 +51,7 @@ func (s *StepMountDevice) Run(state multistep.StateBag) multistep.StepAction {
}
ctx := config.ctx
ctx.Data = &mountPathData{Device: filepath.Base(device)}
mountPath, err := interpolate.Render(config.MountPath, &ctx)
@ -74,8 +80,9 @@ func (s *StepMountDevice) Run(state multistep.StateBag) multistep.StepAction {
}
deviceMount := device
if virtualizationType == "hvm" {
deviceMount = fmt.Sprintf("%s%d", device, s.MountPartition)
if virtualizationType == "hvm" && s.MountPartition != "0" {
deviceMount = fmt.Sprintf("%s%s", device, s.MountPartition)
}
state.Put("deviceMount", deviceMount)
@ -96,7 +103,7 @@ func (s *StepMountDevice) Run(state multistep.StateBag) multistep.StepAction {
ui.Error(err.Error())
return multistep.ActionHalt
}
log.Printf("[DEBUG] (step mount) mount command is %s", mountCommand)
cmd := ShellCommand(mountCommand)
cmd.Stderr = stderr
if err := cmd.Run(); err != nil {

8
builder/amazon/chroot/step_mount_extra.go
View File

@ -2,12 +2,14 @@ package chroot
import (
"bytes"
"context"
"fmt"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
"os"
"os/exec"
"syscall"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
// StepMountExtra mounts the attached device.
@ -18,7 +20,7 @@ type StepMountExtra struct {
mounts []string
}
func (s *StepMountExtra) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepMountExtra) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
mountPath := state.Get("mount_path").(string)
ui := state.Get("ui").(packer.Ui)

6
builder/amazon/chroot/step_post_mount_commands.go
View File

@ -1,8 +1,10 @@
package chroot
import (
"context"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type postMountCommandsData struct {
@ -16,7 +18,7 @@ type StepPostMountCommands struct {
Commands []string
}
func (s *StepPostMountCommands) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepPostMountCommands) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
device := state.Get("device").(string)
mountPath := state.Get("mount_path").(string)

6
builder/amazon/chroot/step_pre_mount_commands.go
View File

@ -1,8 +1,10 @@
package chroot
import (
"context"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type preMountCommandsData struct {
@ -14,7 +16,7 @@ type StepPreMountCommands struct {
Commands []string
}
func (s *StepPreMountCommands) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepPreMountCommands) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
device := state.Get("device").(string)
ui := state.Get("ui").(packer.Ui)

5
builder/amazon/chroot/step_prepare_device.go
View File

@ -1,19 +1,20 @@
package chroot
import (
"context"
"fmt"
"log"
"os"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepPrepareDevice finds an available device and sets it.
type StepPrepareDevice struct {
}
func (s *StepPrepareDevice) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepPrepareDevice) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ui := state.Get("ui").(packer.Ui)

15
builder/amazon/chroot/step_register_ami.go
View File

@ -1,13 +1,14 @@
package chroot
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepRegisterAMI creates the AMI.
@ -17,7 +18,7 @@ type StepRegisterAMI struct {
EnableAMISriovNetSupport bool
}
func (s *StepRegisterAMI) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
snapshotId := state.Get("snapshot_id").(string)
@ -101,16 +102,8 @@ func (s *StepRegisterAMI) Run(state multistep.StateBag) multistep.StepAction {
amis[*ec2conn.Config.Region] = *registerResp.ImageId
state.Put("amis", amis)
// Wait for the image to become ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: awscommon.AMIStateRefreshFunc(ec2conn, *registerResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI to become ready...")
if _, err := awscommon.WaitForState(&stateChange); err != nil {
if err := awscommon.WaitUntilAMIAvailable(ctx, ec2conn, *registerResp.ImageId); err != nil {
err := fmt.Errorf("Error waiting for AMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())

27
builder/amazon/chroot/step_snapshot.go
View File

@ -1,14 +1,14 @@
package chroot
import (
"errors"
"context"
"fmt"
"time"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepSnapshot creates a snapshot of the created volume.
@ -19,7 +19,7 @@ type StepSnapshot struct {
snapshotId string
}
func (s *StepSnapshot) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepSnapshot) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
volumeId := state.Get("volume_id").(string)
@ -43,26 +43,7 @@ func (s *StepSnapshot) Run(state multistep.StateBag) multistep.StepAction {
ui.Message(fmt.Sprintf("Snapshot ID: %s", s.snapshotId))
// Wait for the snapshot to be ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
StepState: state,
Target: "completed",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeSnapshots(&ec2.DescribeSnapshotsInput{SnapshotIds: []*string{&s.snapshotId}})
if err != nil {
return nil, "", err
}
if len(resp.Snapshots) == 0 {
return nil, "", errors.New("No snapshots found.")
}
s := resp.Snapshots[0]
return s, *s.State, nil
},
}
_, err = awscommon.WaitForState(&stateChange)
err = awscommon.WaitUntilSnapshotDone(ctx, ec2conn, s.snapshotId)
if err != nil {
err := fmt.Errorf("Error waiting for snapshot: %s", err)
state.Put("error", err)

92
builder/amazon/common/access_config.go
View File

@ -3,10 +3,11 @@ package common
import (
"fmt"
"log"
"os"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
@ -16,15 +17,16 @@ import (
// AccessConfig is for common configuration related to AWS access
type AccessConfig struct {
AccessKey string `mapstructure:"access_key"`
CustomEndpointEc2 string `mapstructure:"custom_endpoint_ec2"`
MFACode string `mapstructure:"mfa_code"`
ProfileName string `mapstructure:"profile"`
RawRegion string `mapstructure:"region"`
SecretKey string `mapstructure:"secret_key"`
SkipValidation bool `mapstructure:"skip_region_validation"`
Token string `mapstructure:"token"`
session *session.Session
AccessKey string `mapstructure:"access_key"`
CustomEndpointEc2 string `mapstructure:"custom_endpoint_ec2"`
MFACode string `mapstructure:"mfa_code"`
ProfileName string `mapstructure:"profile"`
RawRegion string `mapstructure:"region"`
SecretKey string `mapstructure:"secret_key"`
SkipValidation bool `mapstructure:"skip_region_validation"`
SkipMetadataApiCheck bool `mapstructure:"skip_metadata_api_check"`
Token string `mapstructure:"token"`
session *session.Session
}
// Config returns a valid aws.Config object for access to AWS services, or
@ -34,13 +36,14 @@ func (c *AccessConfig) Session() (*session.Session, error) {
return c.session, nil
}
config := aws.NewConfig().WithMaxRetries(11).WithCredentialsChainVerboseErrors(true)
config := aws.NewConfig().WithCredentialsChainVerboseErrors(true)
if c.ProfileName != "" {
if err := os.Setenv("AWS_PROFILE", c.ProfileName); err != nil {
return nil, fmt.Errorf("Set env error: %s", err)
}
} else if c.RawRegion != "" {
staticCreds := credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, c.Token)
if _, err := staticCreds.Get(); err != credentials.ErrStaticCredentialsEmpty {
config.WithCredentials(staticCreds)
}
if c.RawRegion != "" {
config = config.WithRegion(c.RawRegion)
} else if region := c.metadataRegion(); region != "" {
config = config.WithRegion(region)
@ -50,25 +53,15 @@ func (c *AccessConfig) Session() (*session.Session, error) {
config = config.WithEndpoint(c.CustomEndpointEc2)
}
if c.AccessKey != "" {
creds := credentials.NewChainCredentials(
[]credentials.Provider{
&credentials.StaticProvider{
Value: credentials.Value{
AccessKeyID: c.AccessKey,
SecretAccessKey: c.SecretKey,
SessionToken: c.Token,
},
},
})
config = config.WithCredentials(creds)
}
opts := session.Options{
SharedConfigState: session.SharedConfigEnable,
Config: *config,
}
if c.ProfileName != "" {
opts.Profile = c.ProfileName
}
if c.MFACode != "" {
opts.AssumeRoleTokenProvider = func() (string, error) {
return c.MFACode, nil
@ -82,11 +75,37 @@ func (c *AccessConfig) Session() (*session.Session, error) {
} else {
log.Printf("Found region %s", *sess.Config.Region)
c.session = sess
}
cp, err := c.session.Config.Credentials.Get()
if err != nil {
if awsErr, ok := err.(awserr.Error); ok && awsErr.Code() == "NoCredentialProviders" {
return nil, fmt.Errorf("No valid credential sources found for AWS Builder. " +
"Please see https://www.packer.io/docs/builders/amazon.html#specifying-amazon-credentials " +
"for more information on providing credentials for the AWS Builder.")
} else {
return nil, fmt.Errorf("Error loading credentials for AWS Provider: %s", err)
}
}
log.Printf("[INFO] AWS Auth provider used: %q", cp.ProviderName)
}
return c.session, nil
}
func (c *AccessConfig) SessionRegion() string {
if c.session == nil {
panic("access config session should be set.")
}
return aws.StringValue(c.session.Config.Region)
}
func (c *AccessConfig) IsGovCloud() bool {
return strings.HasPrefix(c.SessionRegion(), "us-gov-")
}
func (c *AccessConfig) IsChinaCloud() bool {
return strings.HasPrefix(c.SessionRegion(), "cn-")
}
// metadataRegion returns the region from the metadata service
func (c *AccessConfig) metadataRegion() string {
@ -108,6 +127,17 @@ func (c *AccessConfig) metadataRegion() string {
func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
var errs []error
if c.SkipMetadataApiCheck {
log.Println("(WARN) skip_metadata_api_check ignored.")
}
// Either both access and secret key must be set or neither of them should
// be.
if (len(c.AccessKey) > 0) != (len(c.SecretKey) > 0) {
errs = append(errs,
fmt.Errorf("`access_key` and `secret_key` must both be either set or not set."))
}
if c.RawRegion != "" && !c.SkipValidation {
if valid := ValidateRegion(c.RawRegion); !valid {
errs = append(errs, fmt.Errorf("Unknown region: %s", c.RawRegion))

20
builder/amazon/common/access_config_test.go
View File

@ -2,6 +2,9 @@ package common
import (
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
)
func testAccessConfig() *AccessConfig {
@ -38,3 +41,20 @@ func TestAccessConfigPrepare_Region(t *testing.T) {
c.SkipValidation = false
}
func TestAccessConfigPrepare_RegionRestricted(t *testing.T) {
c := testAccessConfig()
// Create a Session with a custom region
c.session = session.Must(session.NewSession(&aws.Config{
Region: aws.String("us-gov-west-1"),
}))
if err := c.Prepare(nil); err != nil {
t.Fatalf("shouldn't have err: %s", err)
}
if !c.IsGovCloud() {
t.Fatal("We should be in gov region.")
}
}

43
builder/amazon/common/ami_config.go
View File

@ -2,6 +2,7 @@ package common
import (
"fmt"
"log"
"github.com/hashicorp/packer/template/interpolate"
)
@ -16,7 +17,7 @@ type AMIConfig struct {
AMIProductCodes []string `mapstructure:"ami_product_codes"`
AMIRegions []string `mapstructure:"ami_regions"`
AMISkipRegionValidation bool `mapstructure:"skip_region_validation"`
AMITags map[string]string `mapstructure:"tags"`
AMITags TagMap `mapstructure:"tags"`
AMIENASupport bool `mapstructure:"ena_support"`
AMISriovNetSupport bool `mapstructure:"sriov_support"`
AMIForceDeregister bool `mapstructure:"force_deregister"`
@ -24,7 +25,7 @@ type AMIConfig struct {
AMIEncryptBootVolume bool `mapstructure:"encrypt_boot"`
AMIKmsKeyId string `mapstructure:"kms_key_id"`
AMIRegionKMSKeyIDs map[string]string `mapstructure:"region_kms_key_ids"`
SnapshotTags map[string]string `mapstructure:"snapshot_tags"`
SnapshotTags TagMap `mapstructure:"snapshot_tags"`
SnapshotUsers []string `mapstructure:"snapshot_users"`
SnapshotGroups []string `mapstructure:"snapshot_groups"`
}
@ -41,22 +42,20 @@ func stringInSlice(s []string, searchstr string) bool {
func (c *AMIConfig) Prepare(accessConfig *AccessConfig, ctx *interpolate.Context) []error {
var errs []error
if accessConfig != nil {
session, err := accessConfig.Session()
if err != nil {
errs = append(errs, err)
} else {
region := *session.Config.Region
if stringInSlice(c.AMIRegions, region) {
errs = append(errs, fmt.Errorf("Cannot copy AMI to AWS session region '%s', please remove it from `ami_regions`.", region))
}
}
}
if c.AMIName == "" {
errs = append(errs, fmt.Errorf("ami_name must be specified"))
}
// Make sure that if we have region_kms_key_ids defined,
// the regions in region_kms_key_ids are also in ami_regions
if len(c.AMIRegionKMSKeyIDs) > 0 {
for kmsKeyRegion := range c.AMIRegionKMSKeyIDs {
if !stringInSlice(c.AMIRegions, kmsKeyRegion) {
errs = append(errs, fmt.Errorf("Region %s is in region_kms_key_ids but not in ami_regions", kmsKeyRegion))
}
}
}
if len(c.AMIRegions) > 0 {
regionSet := make(map[string]struct{})
regions := make([]string, 0, len(c.AMIRegions))
@ -84,21 +83,17 @@ func (c *AMIConfig) Prepare(accessConfig *AccessConfig, ctx *interpolate.Context
errs = append(errs, fmt.Errorf("Region %s is in ami_regions but not in region_kms_key_ids", region))
}
}
if (accessConfig != nil) && (region == accessConfig.RawRegion) {
// make sure we don't try to copy to the region we originally
// create the AMI in.
log.Printf("Cannot copy AMI to AWS session region '%s', deleting it from `ami_regions`.", region)
continue
}
regions = append(regions, region)
}
c.AMIRegions = regions
}
// Make sure that if we have region_kms_key_ids defined,
// the regions in region_kms_key_ids are also in ami_regions
if len(c.AMIRegionKMSKeyIDs) > 0 {
for kmsKeyRegion := range c.AMIRegionKMSKeyIDs {
if !stringInSlice(c.AMIRegions, kmsKeyRegion) {
errs = append(errs, fmt.Errorf("Region %s is in region_kms_key_ids but not in ami_regions", kmsKeyRegion))
}
}
}
if len(c.AMIUsers) > 0 && c.AMIEncryptBootVolume {
errs = append(errs, fmt.Errorf("Cannot share AMI with encrypted boot volume"))

15
builder/amazon/common/ami_config_test.go
View File

@ -11,6 +11,12 @@ func testAMIConfig() *AMIConfig {
}
}
func getFakeAccessConfig(region string) *AccessConfig {
return &AccessConfig{
RawRegion: region,
}
}
func TestAMIConfigPrepare_name(t *testing.T) {
c := testAMIConfig()
if err := c.Prepare(nil, nil); err != nil {
@ -118,6 +124,15 @@ func TestAMIConfigPrepare_regions(t *testing.T) {
if err := c.Prepare(nil, nil); err == nil {
t.Fatal("should have error b/c theres a region in in ami_regions that isn't in the key map")
}
// allow rawregion to exist in ami_regions list.
accessConf := getFakeAccessConfig("us-east-1")
c.AMIRegions = []string{"us-east-1", "us-west-1", "us-east-2"}
c.AMIRegionKMSKeyIDs = nil
if err := c.Prepare(accessConf, nil); err != nil {
t.Fatal("should allow user to have the raw region in ami_regions")
}
}
func TestAMIConfigPrepare_Share_EncryptedBoot(t *testing.T) {

14
builder/amazon/common/artifact.go
View File

@ -21,7 +21,7 @@ type Artifact struct {
BuilderIdValue string
// EC2 connection for performing API stuff.
Conn *ec2.EC2
Session *session.Session
}
func (a *Artifact) BuilderId() string {
@ -69,15 +69,9 @@ func (a *Artifact) Destroy() error {
for region, imageId := range a.Amis {
log.Printf("Deregistering image ID (%s) from region (%s)", imageId, region)
regionConfig := &aws.Config{
Credentials: a.Conn.Config.Credentials,
Region: aws.String(region),
}
session, err := session.NewSession(regionConfig)
if err != nil {
return err
}
regionConn := ec2.New(session)
regionConn := ec2.New(a.Session, &aws.Config{
Region: aws.String(region),
})
// Get image metadata
imageResp, err := regionConn.DescribeImages(&ec2.DescribeImagesInput{

27
builder/amazon/common/block_device.go
View File

@ -1,6 +1,7 @@
package common
import (
"fmt"
"strings"
"github.com/aws/aws-sdk-go/aws"
@ -19,6 +20,7 @@ type BlockDevice struct {
VirtualName string `mapstructure:"virtual_name"`
VolumeType string `mapstructure:"volume_type"`
VolumeSize int64 `mapstructure:"volume_size"`
KmsKeyId string `mapstructure:"kms_key_id"`
}
type BlockDevices struct {
@ -73,6 +75,10 @@ func buildBlockDevices(b []BlockDevice) []*ec2.BlockDeviceMapping {
ebsBlockDevice.Encrypted = aws.Bool(blockDevice.Encrypted)
}
if blockDevice.KmsKeyId != "" {
ebsBlockDevice.KmsKeyId = aws.String(blockDevice.KmsKeyId)
}
mapping.Ebs = ebsBlockDevice
}
@ -81,10 +87,29 @@ func buildBlockDevices(b []BlockDevice) []*ec2.BlockDeviceMapping {
return blockDevices
}
func (b *BlockDevices) Prepare(ctx *interpolate.Context) []error {
func (b *BlockDevice) Prepare(ctx *interpolate.Context) error {
// Warn that encrypted must be true when setting kms_key_id
if b.KmsKeyId != "" && b.Encrypted == false {
return fmt.Errorf("The device %v, must also have `encrypted: "+
"true` when setting a kms_key_id.", b.DeviceName)
}
return nil
}
func (b *BlockDevices) Prepare(ctx *interpolate.Context) (errs []error) {
for _, d := range b.AMIMappings {
if err := d.Prepare(ctx); err != nil {
errs = append(errs, fmt.Errorf("AMIMapping: %s", err.Error()))
}
}
for _, d := range b.LaunchMappings {
if err := d.Prepare(ctx); err != nil {
errs = append(errs, fmt.Errorf("LaunchMapping: %s", err.Error()))
}
}
return errs
}
func (b *AMIBlockDevices) BuildAMIDevices() []*ec2.BlockDeviceMapping {
return buildBlockDevices(b.AMIMappings)
}

21
builder/amazon/common/block_device_test.go
View File

@ -84,6 +84,27 @@ func TestBlockDevice(t *testing.T) {
},
},
},
{
Config: &BlockDevice{
DeviceName: "/dev/sdb",
VolumeType: "gp2",
VolumeSize: 8,
DeleteOnTermination: true,
Encrypted: true,
KmsKeyId: "2Fa48a521f-3aff-4b34-a159-376ac5d37812",
},
Result: &ec2.BlockDeviceMapping{
DeviceName: aws.String("/dev/sdb"),
Ebs: &ec2.EbsBlockDevice{
VolumeType: aws.String("gp2"),
VolumeSize: aws.Int64(8),
DeleteOnTermination: aws.Bool(true),
Encrypted: aws.Bool(true),
KmsKeyId: aws.String("2Fa48a521f-3aff-4b34-a159-376ac5d37812"),
},
},
},
{
Config: &BlockDevice{
DeviceName: "/dev/sdb",

34
builder/amazon/common/interpolate_build_info.go
View File

@ -1,6 +1,36 @@
package common
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
)
type BuildInfoTemplate struct {
SourceAMI string
BuildRegion string
BuildRegion string
SourceAMI string
SourceAMIName string
SourceAMITags map[string]string
}
func extractBuildInfo(region string, state multistep.StateBag) *BuildInfoTemplate {
rawSourceAMI, hasSourceAMI := state.GetOk("source_image")
if !hasSourceAMI {
return &BuildInfoTemplate{
BuildRegion: region,
}
}
sourceAMI := rawSourceAMI.(*ec2.Image)
sourceAMITags := make(map[string]string, len(sourceAMI.Tags))
for _, tag := range sourceAMI.Tags {
sourceAMITags[aws.StringValue(tag.Key)] = aws.StringValue(tag.Value)
}
return &BuildInfoTemplate{
BuildRegion: region,
SourceAMI: aws.StringValue(sourceAMI.ImageId),
SourceAMIName: aws.StringValue(sourceAMI.Name),
SourceAMITags: sourceAMITags,
}
}

63
builder/amazon/common/interpolate_build_info_test.go Normal file
View File

@ -0,0 +1,63 @@
package common
import (
"reflect"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
)
func testImage() *ec2.Image {
return &ec2.Image{
ImageId: aws.String("ami-abcd1234"),
Name: aws.String("ami_test_name"),
Tags: []*ec2.Tag{
{
Key: aws.String("key-1"),
Value: aws.String("value-1"),
},
{
Key: aws.String("key-2"),
Value: aws.String("value-2"),
},
},
}
}
func testState() multistep.StateBag {
state := new(multistep.BasicStateBag)
return state
}
func TestInterpolateBuildInfo_extractBuildInfo_noSourceImage(t *testing.T) {
state := testState()
buildInfo := extractBuildInfo("foo", state)
expected := BuildInfoTemplate{
BuildRegion: "foo",
}
if !reflect.DeepEqual(*buildInfo, expected) {
t.Fatalf("Unexpected BuildInfoTemplate: expected %#v got %#v\n", expected, *buildInfo)
}
}
func TestInterpolateBuildInfo_extractBuildInfo_withSourceImage(t *testing.T) {
state := testState()
state.Put("source_image", testImage())
buildInfo := extractBuildInfo("foo", state)
expected := BuildInfoTemplate{
BuildRegion: "foo",
SourceAMI: "ami-abcd1234",
SourceAMIName: "ami_test_name",
SourceAMITags: map[string]string{
"key-1": "value-1",
"key-2": "value-2",
},
}
if !reflect.DeepEqual(*buildInfo, expected) {
t.Fatalf("Unexpected BuildInfoTemplate: expected %#v got %#v\n", expected, *buildInfo)
}
}

3
builder/amazon/common/regions.go
View File

@ -5,6 +5,7 @@ func listEC2Regions() []string {
return []string{
"ap-northeast-1",
"ap-northeast-2",
"ap-northeast-3",
"ap-south-1",
"ap-southeast-1",
"ap-southeast-2",
@ -12,6 +13,7 @@ func listEC2Regions() []string {
"eu-central-1",
"eu-west-1",
"eu-west-2",
"eu-west-3",
"sa-east-1",
"us-east-1",
"us-east-2",
@ -20,6 +22,7 @@ func listEC2Regions() []string {
// not part of autogenerated list
"us-gov-west-1",
"cn-north-1",
"cn-northwest-1",
}
}

68
builder/amazon/common/run_config.go
View File

@ -1,11 +1,11 @@
package common
import (
"errors"
"fmt"
"net"
"os"
"regexp"
"strings"
"time"
"github.com/hashicorp/packer/common/uuid"
@ -30,30 +30,32 @@ func (d *AmiFilterOptions) Empty() bool {
type RunConfig struct {
AssociatePublicIpAddress bool `mapstructure:"associate_public_ip_address"`
AvailabilityZone string `mapstructure:"availability_zone"`
DisableStopInstance bool `mapstructure:"disable_stop_instance"`
EbsOptimized bool `mapstructure:"ebs_optimized"`
EnableT2Unlimited bool `mapstructure:"enable_t2_unlimited"`
IamInstanceProfile string `mapstructure:"iam_instance_profile"`
InstanceInitiatedShutdownBehavior string `mapstructure:"shutdown_behavior"`
InstanceType string `mapstructure:"instance_type"`
RunTags map[string]string `mapstructure:"run_tags"`
SecurityGroupId string `mapstructure:"security_group_id"`
SecurityGroupIds []string `mapstructure:"security_group_ids"`
SourceAmi string `mapstructure:"source_ami"`
SourceAmiFilter AmiFilterOptions `mapstructure:"source_ami_filter"`
SpotPrice string `mapstructure:"spot_price"`
SpotPriceAutoProduct string `mapstructure:"spot_price_auto_product"`
DisableStopInstance bool `mapstructure:"disable_stop_instance"`
SecurityGroupId string `mapstructure:"security_group_id"`
SecurityGroupIds []string `mapstructure:"security_group_ids"`
TemporarySGSourceCidr string `mapstructure:"temporary_security_group_source_cidr"`
SpotTags map[string]string `mapstructure:"spot_tags"`
SubnetId string `mapstructure:"subnet_id"`
TemporaryKeyPairName string `mapstructure:"temporary_key_pair_name"`
TemporarySGSourceCidr string `mapstructure:"temporary_security_group_source_cidr"`
UserData string `mapstructure:"user_data"`
UserDataFile string `mapstructure:"user_data_file"`
WindowsPasswordTimeout time.Duration `mapstructure:"windows_password_timeout"`
VpcId string `mapstructure:"vpc_id"`
InstanceInitiatedShutdownBehavior string `mapstructure:"shutdown_behavior"`
WindowsPasswordTimeout time.Duration `mapstructure:"windows_password_timeout"`
// Communicator settings
Comm communicator.Config `mapstructure:",squash"`
SSHKeyPairName string `mapstructure:"ssh_keypair_name"`
SSHPrivateIp bool `mapstructure:"ssh_private_ip"`
SSHInterface string `mapstructure:"ssh_interface"`
}
func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
@ -77,29 +79,53 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
// Validation
errs := c.Comm.Prepare(ctx)
// Validating ssh_interface
if c.SSHInterface != "public_ip" &&
c.SSHInterface != "private_ip" &&
c.SSHInterface != "public_dns" &&
c.SSHInterface != "private_dns" &&
c.SSHInterface != "" {
errs = append(errs, fmt.Errorf("Unknown interface type: %s", c.SSHInterface))
}
if c.SSHKeyPairName != "" {
if c.Comm.Type == "winrm" && c.Comm.WinRMPassword == "" && c.Comm.SSHPrivateKey == "" {
errs = append(errs, errors.New("A private_key_file must be provided to retrieve the winrm password when using ssh_keypair_name."))
errs = append(errs, fmt.Errorf("ssh_private_key_file must be provided to retrieve the winrm password when using ssh_keypair_name."))
} else if c.Comm.SSHPrivateKey == "" && !c.Comm.SSHAgentAuth {
errs = append(errs, errors.New("A private_key_file must be provided or ssh_agent_auth enabled when ssh_keypair_name is specified."))
errs = append(errs, fmt.Errorf("ssh_private_key_file must be provided or ssh_agent_auth enabled when ssh_keypair_name is specified."))
}
}
if c.SourceAmi == "" && c.SourceAmiFilter.Empty() {
errs = append(errs, errors.New("A source_ami or source_ami_filter must be specified"))
errs = append(errs, fmt.Errorf("A source_ami or source_ami_filter must be specified"))
}
if c.InstanceType == "" {
errs = append(errs, errors.New("An instance_type must be specified"))
errs = append(errs, fmt.Errorf("An instance_type must be specified"))
}
if c.SpotPrice == "auto" {
if c.SpotPriceAutoProduct == "" {
errs = append(errs, errors.New(
errs = append(errs, fmt.Errorf(
"spot_price_auto_product must be specified when spot_price is auto"))
}
}
if c.SpotPriceAutoProduct != "" {
if c.SpotPrice != "auto" {
errs = append(errs, fmt.Errorf(
"spot_price should be set to auto when spot_price_auto_product is specified"))
}
}
if c.SpotTags != nil {
if c.SpotPrice == "" || c.SpotPrice == "0" {
errs = append(errs, fmt.Errorf(
"spot_tags should not be set when not requesting a spot instance"))
}
}
if c.UserData != "" && c.UserDataFile != "" {
errs = append(errs, fmt.Errorf("Only one of user_data or user_data_file can be specified."))
} else if c.UserDataFile != "" {
@ -131,5 +157,21 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
errs = append(errs, fmt.Errorf("shutdown_behavior only accepts 'stop' or 'terminate' values."))
}
if c.EnableT2Unlimited {
if c.SpotPrice != "" {
errs = append(errs, fmt.Errorf("Error: T2 Unlimited cannot be used in conjuction with Spot Instances"))
}
firstDotIndex := strings.Index(c.InstanceType, ".")
if firstDotIndex == -1 {
errs = append(errs, fmt.Errorf("Error determining main Instance Type from: %s", c.InstanceType))
} else if c.InstanceType[0:firstDotIndex] != "t2" {
errs = append(errs, fmt.Errorf("Error: T2 Unlimited enabled with a non-T2 Instance Type: %s", c.InstanceType))
}
}
return errs
}
func (c *RunConfig) IsSpotInstance() bool {
return c.SpotPrice != "" && c.SpotPrice != "0"
}

55
builder/amazon/common/run_config_test.go
View File

@ -48,7 +48,7 @@ func TestRunConfigPrepare_InstanceType(t *testing.T) {
c := testConfig()
c.InstanceType = ""
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("err: %s", err)
t.Fatalf("Should error if an instance_type is not specified")
}
}
@ -56,14 +56,14 @@ func TestRunConfigPrepare_SourceAmi(t *testing.T) {
c := testConfig()
c.SourceAmi = ""
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("err: %s", err)
t.Fatalf("Should error if a source_ami (or source_ami_filter) is not specified")
}
}
func TestRunConfigPrepare_SourceAmiFilterBlank(t *testing.T) {
c := testConfigFilter()
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("err: %s", err)
t.Fatalf("Should error if source_ami_filter is empty or not specified (and source_ami is not specified)")
}
}
@ -79,17 +79,58 @@ func TestRunConfigPrepare_SourceAmiFilterGood(t *testing.T) {
}
}
func TestRunConfigPrepare_EnableT2UnlimitedGood(t *testing.T) {
c := testConfig()
// Must have a T2 instance type if T2 Unlimited is enabled
c.InstanceType = "t2.micro"
c.EnableT2Unlimited = true
err := c.Prepare(nil)
if len(err) > 0 {
t.Fatalf("err: %s", err)
}
}
func TestRunConfigPrepare_EnableT2UnlimitedBadInstanceType(t *testing.T) {
c := testConfig()
// T2 Unlimited cannot be used with instance types other than T2
c.InstanceType = "m5.large"
c.EnableT2Unlimited = true
err := c.Prepare(nil)
if len(err) != 1 {
t.Fatalf("Should error if T2 Unlimited is enabled with non-T2 instance_type")
}
}
func TestRunConfigPrepare_EnableT2UnlimitedBadWithSpotInstanceRequest(t *testing.T) {
c := testConfig()
// T2 Unlimited cannot be used with Spot Instances
c.InstanceType = "t2.micro"
c.EnableT2Unlimited = true
c.SpotPrice = "auto"
c.SpotPriceAutoProduct = "Linux/UNIX"
err := c.Prepare(nil)
if len(err) != 1 {
t.Fatalf("Should error if T2 Unlimited has been used in conjuntion with a Spot Price request")
}
}
func TestRunConfigPrepare_SpotAuto(t *testing.T) {
c := testConfig()
c.SpotPrice = "auto"
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("err: %s", err)
t.Fatalf("Should error if spot_price_auto_product is not set and spot_price is set to auto")
}
// Good - SpotPrice and SpotPriceAutoProduct are correctly set
c.SpotPriceAutoProduct = "foo"
if err := c.Prepare(nil); len(err) != 0 {
t.Fatalf("err: %s", err)
}
c.SpotPrice = ""
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("Should error if spot_price is not set to auto and spot_price_auto_product is set")
}
}
func TestRunConfigPrepare_SSHPort(t *testing.T) {
@ -119,12 +160,13 @@ func TestRunConfigPrepare_UserData(t *testing.T) {
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserData = "foo"
c.UserDataFile = tf.Name()
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("err: %s", err)
t.Fatalf("Should error if user_data string and user_data_file have both been specified")
}
}
@ -136,13 +178,14 @@ func TestRunConfigPrepare_UserDataFile(t *testing.T) {
c.UserDataFile = "idontexistidontthink"
if err := c.Prepare(nil); len(err) != 1 {
t.Fatalf("err: %s", err)
t.Fatalf("Should error if the file specified by user_data_file does not exist")
}
tf, err := ioutil.TempFile("", "packer")
if err != nil {
t.Fatalf("err: %s", err)
}
defer os.Remove(tf.Name())
defer tf.Close()
c.UserDataFile = tf.Name()

33
builder/amazon/common/ssh.go
View File

@ -9,7 +9,7 @@ import (
"github.com/aws/aws-sdk-go/service/ec2"
packerssh "github.com/hashicorp/packer/communicator/ssh"
"github.com/mitchellh/multistep"
"github.com/hashicorp/packer/helper/multistep"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/agent"
)
@ -25,21 +25,40 @@ var (
// SSHHost returns a function that can be given to the SSH communicator
// for determining the SSH address based on the instance DNS name.
func SSHHost(e ec2Describer, private bool) func(multistep.StateBag) (string, error) {
func SSHHost(e ec2Describer, sshInterface string) func(multistep.StateBag) (string, error) {
return func(state multistep.StateBag) (string, error) {
const tries = 2
// <= with current structure to check result of describing `tries` times
for j := 0; j <= tries; j++ {
var host string
i := state.Get("instance").(*ec2.Instance)
if i.VpcId != nil && *i.VpcId != "" {
if i.PublicIpAddress != nil && *i.PublicIpAddress != "" && !private {
if sshInterface != "" {
switch sshInterface {
case "public_ip":
if i.PublicIpAddress != nil {
host = *i.PublicIpAddress
}
case "private_ip":
if i.PrivateIpAddress != nil {
host = *i.PrivateIpAddress
}
case "public_dns":
if i.PublicDnsName != nil {
host = *i.PublicDnsName
}
case "private_dns":
if i.PrivateDnsName != nil {
host = *i.PrivateDnsName
}
default:
panic(fmt.Sprintf("Unknown interface type: %s", sshInterface))
}
} else if i.VpcId != nil && *i.VpcId != "" {
if i.PublicIpAddress != nil && *i.PublicIpAddress != "" {
host = *i.PublicIpAddress
} else if i.PrivateIpAddress != nil && *i.PrivateIpAddress != "" {
host = *i.PrivateIpAddress
}
} else if private && i.PrivateIpAddress != nil && *i.PrivateIpAddress != "" {
host = *i.PrivateIpAddress
} else if i.PublicDnsName != nil && *i.PublicDnsName != "" {
host = *i.PublicDnsName
}
@ -63,7 +82,7 @@ func SSHHost(e ec2Describer, private bool) func(multistep.StateBag) (string, err
time.Sleep(sshHostSleepDuration)
}
return "", errors.New("couldn't determine IP address for instance")
return "", errors.New("couldn't determine address for instance")
}
}

62
builder/amazon/common/ssh_test.go
View File

@ -5,13 +5,14 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mitchellh/multistep"
"github.com/hashicorp/packer/helper/multistep"
)
const (
privateIP = "10.0.0.1"
publicIP = "192.168.1.1"
publicDNS = "public.dns.test"
privateIP = "10.0.0.1"
publicIP = "192.168.1.1"
privateDNS = "private.dns.test"
publicDNS = "public.dns.test"
)
func TestSSHHost(t *testing.T) {
@ -20,44 +21,54 @@ func TestSSHHost(t *testing.T) {
sshHostSleepDuration = 0
var cases = []struct {
allowTries int
vpcId string
private bool
allowTries int
vpcId string
sshInterface string
ok bool
wantHost string
}{
{1, "", false, true, publicDNS},
{1, "", true, true, privateIP},
{1, "vpc-id", false, true, publicIP},
{1, "vpc-id", true, true, privateIP},
{2, "", false, true, publicDNS},
{2, "", true, true, privateIP},
{2, "vpc-id", false, true, publicIP},
{2, "vpc-id", true, true, privateIP},
{3, "", false, false, ""},
{3, "", true, false, ""},
{3, "vpc-id", false, false, ""},
{3, "vpc-id", true, false, ""},
{1, "", "", true, publicDNS},
{1, "", "private_ip", true, privateIP},
{1, "vpc-id", "", true, publicIP},
{1, "vpc-id", "private_ip", true, privateIP},
{1, "vpc-id", "private_dns", true, privateDNS},
{1, "vpc-id", "public_dns", true, publicDNS},
{1, "vpc-id", "public_ip", true, publicIP},
{2, "", "", true, publicDNS},
{2, "", "private_ip", true, privateIP},
{2, "vpc-id", "", true, publicIP},
{2, "vpc-id", "private_ip", true, privateIP},
{2, "vpc-id", "private_dns", true, privateDNS},
{2, "vpc-id", "public_dns", true, publicDNS},
{2, "vpc-id", "public_ip", true, publicIP},
{3, "", "", false, ""},
{3, "", "private_ip", false, ""},
{3, "vpc-id", "", false, ""},
{3, "vpc-id", "private_ip", false, ""},
{3, "vpc-id", "private_dns", false, ""},
{3, "vpc-id", "public_dns", false, ""},
{3, "vpc-id", "public_ip", false, ""},
}
for _, c := range cases {
testSSHHost(t, c.allowTries, c.vpcId, c.private, c.ok, c.wantHost)
testSSHHost(t, c.allowTries, c.vpcId, c.sshInterface, c.ok, c.wantHost)
}
}
func testSSHHost(t *testing.T, allowTries int, vpcId string, private, ok bool, wantHost string) {
t.Logf("allowTries=%d vpcId=%s private=%t ok=%t wantHost=%q", allowTries, vpcId, private, ok, wantHost)
func testSSHHost(t *testing.T, allowTries int, vpcId string, sshInterface string, ok bool, wantHost string) {
t.Logf("allowTries=%d vpcId=%s sshInterface=%s ok=%t wantHost=%q", allowTries, vpcId, sshInterface, ok, wantHost)
e := &fakeEC2Describer{
allowTries: allowTries,
vpcId: vpcId,
privateIP: privateIP,
publicIP: publicIP,
privateDNS: privateDNS,
publicDNS: publicDNS,
}
f := SSHHost(e, private)
f := SSHHost(e, sshInterface)
st := &multistep.BasicStateBag{}
st.Put("instance", &ec2.Instance{
InstanceId: aws.String("instance-id"),
@ -85,8 +96,8 @@ type fakeEC2Describer struct {
allowTries int
tries int
vpcId string
privateIP, publicIP, publicDNS string
vpcId string
privateIP, publicIP, privateDNS, publicDNS string
}
func (d *fakeEC2Describer) DescribeInstances(in *ec2.DescribeInstancesInput) (*ec2.DescribeInstancesOutput, error) {
@ -104,6 +115,7 @@ func (d *fakeEC2Describer) DescribeInstances(in *ec2.DescribeInstancesInput) (*e
instance.PublicIpAddress = aws.String(d.publicIP)
instance.PrivateIpAddress = aws.String(d.privateIP)
instance.PublicDnsName = aws.String(d.publicDNS)
instance.PrivateDnsName = aws.String(d.privateDNS)
}
out := &ec2.DescribeInstancesOutput{

463
builder/amazon/common/state.go
View File

@ -1,18 +1,15 @@
package common
import (
"errors"
"fmt"
"log"
"net"
"os"
"strconv"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mitchellh/multistep"
"github.com/hashicorp/packer/helper/multistep"
)
// StateRefreshFunc is a function type used for StateChangeConf that is
@ -35,228 +32,304 @@ type StateChangeConf struct {
Target string
}
// AMIStateRefreshFunc returns a StateRefreshFunc that is used to watch
// an AMI for state changes.
func AMIStateRefreshFunc(conn *ec2.EC2, imageId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeImages(&ec2.DescribeImagesInput{
ImageIds: []*string{&imageId},
})
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && ec2err.Code() == "InvalidAMIID.NotFound" {
// Set this to nil as if we didn't find anything.
resp = nil
} else if isTransientNetworkError(err) {
// Transient network error, treat it as if we didn't find anything
resp = nil
} else {
log.Printf("Error on AMIStateRefresh: %s", err)
return nil, "", err
}
}
// Following are wrapper functions that use Packer's environment-variables to
// determing retry logic, then call the AWS SDK's built-in waiters.
if resp == nil || len(resp.Images) == 0 {
// Sometimes AWS has consistency issues and doesn't see the
// AMI. Return an empty state.
return nil, "", nil
}
i := resp.Images[0]
return i, *i.State, nil
func WaitUntilAMIAvailable(ctx aws.Context, conn *ec2.EC2, imageId string) error {
imageInput := ec2.DescribeImagesInput{
ImageIds: []*string{&imageId},
}
err := conn.WaitUntilImageAvailableWithContext(
ctx,
&imageInput,
getWaiterOptions()...)
return err
}
// InstanceStateRefreshFunc returns a StateRefreshFunc that is used to watch
// an EC2 instance.
func InstanceStateRefreshFunc(conn *ec2.EC2, instanceId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeInstances(&ec2.DescribeInstancesInput{
InstanceIds: []*string{&instanceId},
})
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && ec2err.Code() == "InvalidInstanceID.NotFound" {
// Set this to nil as if we didn't find anything.
resp = nil
} else if isTransientNetworkError(err) {
// Transient network error, treat it as if we didn't find anything
resp = nil
} else {
log.Printf("Error on InstanceStateRefresh: %s", err)
return nil, "", err
}
}
func WaitUntilInstanceTerminated(ctx aws.Context, conn *ec2.EC2, instanceId string) error {
if resp == nil || len(resp.Reservations) == 0 || len(resp.Reservations[0].Instances) == 0 {
// Sometimes AWS just has consistency issues and doesn't see
// our instance yet. Return an empty state.
return nil, "", nil
}
i := resp.Reservations[0].Instances[0]
return i, *i.State.Name, nil
instanceInput := ec2.DescribeInstancesInput{
InstanceIds: []*string{&instanceId},
}
err := conn.WaitUntilInstanceTerminatedWithContext(
ctx,
&instanceInput,
getWaiterOptions()...)
return err
}
// SpotRequestStateRefreshFunc returns a StateRefreshFunc that is used to watch
// a spot request for state changes.
func SpotRequestStateRefreshFunc(conn *ec2.EC2, spotRequestId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeSpotInstanceRequests(&ec2.DescribeSpotInstanceRequestsInput{
SpotInstanceRequestIds: []*string{&spotRequestId},
})
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && ec2err.Code() == "InvalidSpotInstanceRequestID.NotFound" {
// Set this to nil as if we didn't find anything.
resp = nil
} else if isTransientNetworkError(err) {
// Transient network error, treat it as if we didn't find anything
resp = nil
} else {
log.Printf("Error on SpotRequestStateRefresh: %s", err)
return nil, "", err
}
}
if resp == nil || len(resp.SpotInstanceRequests) == 0 {
// Sometimes AWS has consistency issues and doesn't see the
// SpotRequest. Return an empty state.
return nil, "", nil
}
i := resp.SpotInstanceRequests[0]
return i, *i.State, nil
// This function works for both requesting and cancelling spot instances.
func WaitUntilSpotRequestFulfilled(ctx aws.Context, conn *ec2.EC2, spotRequestId string) error {
spotRequestInput := ec2.DescribeSpotInstanceRequestsInput{
SpotInstanceRequestIds: []*string{&spotRequestId},
}
err := conn.WaitUntilSpotInstanceRequestFulfilledWithContext(
ctx,
&spotRequestInput,
getWaiterOptions()...)
return err
}
func ImportImageRefreshFunc(conn *ec2.EC2, importTaskId string) StateRefreshFunc {
return func() (interface{}, string, error) {
resp, err := conn.DescribeImportImageTasks(&ec2.DescribeImportImageTasksInput{
ImportTaskIds: []*string{
&importTaskId,
func WaitUntilVolumeAvailable(ctx aws.Context, conn *ec2.EC2, volumeId string) error {
volumeInput := ec2.DescribeVolumesInput{
VolumeIds: []*string{&volumeId},
}
err := conn.WaitUntilVolumeAvailableWithContext(
ctx,
&volumeInput,
getWaiterOptions()...)
return err
}
func WaitUntilSnapshotDone(ctx aws.Context, conn *ec2.EC2, snapshotID string) error {
snapInput := ec2.DescribeSnapshotsInput{
SnapshotIds: []*string{&snapshotID},
}
err := conn.WaitUntilSnapshotCompletedWithContext(
ctx,
&snapInput,
getWaiterOptions()...)
return err
}
// Wrappers for our custom AWS waiters
func WaitUntilVolumeAttached(ctx aws.Context, conn *ec2.EC2, volumeId string) error {
volumeInput := ec2.DescribeVolumesInput{
VolumeIds: []*string{&volumeId},
}
err := WaitForVolumeToBeAttached(conn,
ctx,
&volumeInput,
getWaiterOptions()...)
return err
}
func WaitUntilVolumeDetached(ctx aws.Context, conn *ec2.EC2, volumeId string) error {
volumeInput := ec2.DescribeVolumesInput{
VolumeIds: []*string{&volumeId},
}
err := WaitForVolumeToBeDetached(conn,
ctx,
&volumeInput,
getWaiterOptions()...)
return err
}
func WaitUntilImageImported(ctx aws.Context, conn *ec2.EC2, taskID string) error {
importInput := ec2.DescribeImportImageTasksInput{
ImportTaskIds: []*string{&taskID},
}
err := WaitForImageToBeImported(conn,
ctx,
&importInput,
getWaiterOptions()...)
return err
}
// Custom waiters using AWS's request.Waiter
func WaitForVolumeToBeAttached(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeVolumesInput, opts ...request.WaiterOption) error {
w := request.Waiter{
Name: "DescribeVolumes",
MaxAttempts: 40,
Delay: request.ConstantWaiterDelay(5 * time.Second),
Acceptors: []request.WaiterAcceptor{
{
State: request.SuccessWaiterState,
Matcher: request.PathAllWaiterMatch,
Argument: "Volumes[].Attachments[].State",
Expected: "attached",
},
},
)
if err != nil {
if ec2err, ok := err.(awserr.Error); ok && strings.HasPrefix(ec2err.Code(), "InvalidConversionTaskId") {
resp = nil
} else if isTransientNetworkError(err) {
resp = nil
} else {
log.Printf("Error on ImportImageRefresh: %s", err)
return nil, "", err
Logger: c.Config.Logger,
NewRequest: func(opts []request.Option) (*request.Request, error) {
var inCpy *ec2.DescribeVolumesInput
if input != nil {
tmp := *input
inCpy = &tmp
}
}
if resp == nil || len(resp.ImportImageTasks) == 0 {
return nil, "", nil
}
i := resp.ImportImageTasks[0]
return i, *i.Status, nil
req, _ := c.DescribeVolumesRequest(inCpy)
req.SetContext(ctx)
req.ApplyOptions(opts...)
return req, nil
},
}
return w.WaitWithContext(ctx)
}
// WaitForState watches an object and waits for it to achieve a certain
// state.
func WaitForState(conf *StateChangeConf) (i interface{}, err error) {
log.Printf("Waiting for state to become: %s", conf.Target)
sleepSeconds := SleepSeconds()
maxTicks := TimeoutSeconds()/sleepSeconds + 1
notfoundTick := 0
for {
var currentState string
i, currentState, err = conf.Refresh()
if err != nil {
return
}
if i == nil {
// If we didn't find the resource, check if we have been
// not finding it for awhile, and if so, report an error.
notfoundTick += 1
if notfoundTick > maxTicks {
return nil, errors.New("couldn't find resource")
func WaitForVolumeToBeDetached(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeVolumesInput, opts ...request.WaiterOption) error {
w := request.Waiter{
Name: "DescribeVolumes",
MaxAttempts: 40,
Delay: request.ConstantWaiterDelay(5 * time.Second),
Acceptors: []request.WaiterAcceptor{
{
State: request.SuccessWaiterState,
Matcher: request.PathAllWaiterMatch,
Argument: "length(Volumes[].Attachments[]) == `0`",
Expected: true,
},
},
Logger: c.Config.Logger,
NewRequest: func(opts []request.Option) (*request.Request, error) {
var inCpy *ec2.DescribeVolumesInput
if input != nil {
tmp := *input
inCpy = &tmp
}
} else {
// Reset the counter for when a resource isn't found
notfoundTick = 0
if currentState == conf.Target {
return
}
if conf.StepState != nil {
if _, ok := conf.StepState.GetOk(multistep.StateCancelled); ok {
return nil, errors.New("interrupted")
}
}
found := false
for _, allowed := range conf.Pending {
if currentState == allowed {
found = true
break
}
}
if !found {
err := fmt.Errorf("unexpected state '%s', wanted target '%s'", currentState, conf.Target)
return nil, err
}
}
time.Sleep(time.Duration(sleepSeconds) * time.Second)
req, _ := c.DescribeVolumesRequest(inCpy)
req.SetContext(ctx)
req.ApplyOptions(opts...)
return req, nil
},
}
return w.WaitWithContext(ctx)
}
func isTransientNetworkError(err error) bool {
if nerr, ok := err.(net.Error); ok && nerr.Temporary() {
return true
func WaitForImageToBeImported(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeImportImageTasksInput, opts ...request.WaiterOption) error {
w := request.Waiter{
Name: "DescribeImages",
MaxAttempts: 300,
Delay: request.ConstantWaiterDelay(5 * time.Second),
Acceptors: []request.WaiterAcceptor{
{
State: request.SuccessWaiterState,
Matcher: request.PathAllWaiterMatch,
Argument: "ImportImageTasks[].Status",
Expected: "completed",
},
},
Logger: c.Config.Logger,
NewRequest: func(opts []request.Option) (*request.Request, error) {
var inCpy *ec2.DescribeImportImageTasksInput
if input != nil {
tmp := *input
inCpy = &tmp
}
req, _ := c.DescribeImportImageTasksRequest(inCpy)
req.SetContext(ctx)
req.ApplyOptions(opts...)
return req, nil
},
}
return false
return w.WaitWithContext(ctx)
}
// Returns 300 seconds (5 minutes) by default
// Some AWS operations, like copying an AMI to a distant region, take a very long time
// Allow user to override with AWS_TIMEOUT_SECONDS environment variable
func TimeoutSeconds() (seconds int) {
seconds = 300
// This helper function uses the environment variables AWS_TIMEOUT_SECONDS and
// AWS_POLL_DELAY_SECONDS to generate waiter options that can be passed into any
// request.Waiter function. These options will control how many times the waiter
// will retry the request, as well as how long to wait between the retries.
override := os.Getenv("AWS_TIMEOUT_SECONDS")
// DEFAULTING BEHAVIOR:
// if AWS_POLL_DELAY_SECONDS is set but the others are not, Packer will set this
// poll delay and use the waiter-specific default
// if AWS_TIMEOUT_SECONDS is set but AWS_MAX_ATTEMPTS is not, Packer will use
// AWS_TIMEOUT_SECONDS and _either_ AWS_POLL_DELAY_SECONDS _or_ 2 if the user has not set AWS_POLL_DELAY_SECONDS, to determine a max number of attempts to make.
// if AWS_TIMEOUT_SECONDS, _and_ AWS_MAX_ATTEMPTS are both set,
// AWS_TIMEOUT_SECONDS will be ignored.
// if AWS_MAX_ATTEMPTS is set but AWS_POLL_DELAY_SECONDS is not, then we will
// use waiter-specific defaults.
type envInfo struct {
envKey string
Val int
overridden bool
}
type overridableWaitVars struct {
awsPollDelaySeconds envInfo
awsMaxAttempts envInfo
awsTimeoutSeconds envInfo
}
func getWaiterOptions() []request.WaiterOption {
envOverrides := getEnvOverrides()
waitOpts := applyEnvOverrides(envOverrides)
return waitOpts
}
func getOverride(varInfo envInfo) envInfo {
override := os.Getenv(varInfo.envKey)
if override != "" {
n, err := strconv.Atoi(override)
if err != nil {
log.Printf("Invalid timeout seconds '%s', using default", override)
log.Printf("Invalid %s '%s', using default", varInfo.envKey, override)
} else {
seconds = n
varInfo.overridden = true
varInfo.Val = n
}
}
log.Printf("Allowing %ds to complete (change with AWS_TIMEOUT_SECONDS)", seconds)
return seconds
return varInfo
}
// Returns 2 seconds by default
// AWS async operations sometimes takes long times, if there are multiple parallel builds,
// polling at 2 second frequency will exceed the request limit. Allow 2 seconds to be
// overwritten with AWS_POLL_DELAY_SECONDS
func SleepSeconds() (seconds int) {
seconds = 2
override := os.Getenv("AWS_POLL_DELAY_SECONDS")
if override != "" {
n, err := strconv.Atoi(override)
if err != nil {
log.Printf("Invalid sleep seconds '%s', using default", override)
} else {
seconds = n
}
func getEnvOverrides() overridableWaitVars {
// Load env vars from environment, and use them to override defaults
envValues := overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 0, false},
envInfo{"AWS_TIMEOUT_SECONDS", 300, false},
}
log.Printf("Using %ds as polling delay (change with AWS_POLL_DELAY_SECONDS)", seconds)
return seconds
envValues.awsMaxAttempts = getOverride(envValues.awsMaxAttempts)
envValues.awsPollDelaySeconds = getOverride(envValues.awsPollDelaySeconds)
envValues.awsTimeoutSeconds = getOverride(envValues.awsTimeoutSeconds)
return envValues
}
func applyEnvOverrides(envOverrides overridableWaitVars) []request.WaiterOption {
waitOpts := make([]request.WaiterOption, 0)
// If user has set poll delay seconds, overwrite it. If user has NOT,
// default to a poll delay of 2 seconds
if envOverrides.awsPollDelaySeconds.overridden {
delaySeconds := request.ConstantWaiterDelay(time.Duration(envOverrides.awsPollDelaySeconds.Val) * time.Second)
waitOpts = append(waitOpts, request.WithWaiterDelay(delaySeconds))
}
// If user has set max attempts, overwrite it. If user hasn't set max
// attempts, default to whatever the waiter has set as a default.
if envOverrides.awsMaxAttempts.overridden {
waitOpts = append(waitOpts, request.WithWaiterMaxAttempts(envOverrides.awsMaxAttempts.Val))
}
if envOverrides.awsMaxAttempts.overridden && envOverrides.awsTimeoutSeconds.overridden {
log.Printf("WARNING: AWS_MAX_ATTEMPTS and AWS_TIMEOUT_SECONDS are" +
" both set. Packer will be using AWS_MAX_ATTEMPTS and discarding " +
"AWS_TIMEOUT_SECONDS. If you have not set AWS_POLL_DELAY_SECONDS, " +
"Packer will default to a 2 second poll delay.")
} else if envOverrides.awsTimeoutSeconds.overridden {
log.Printf("DEPRECATION WARNING: env var AWS_TIMEOUT_SECONDS is " +
"deprecated in favor of AWS_MAX_ATTEMPTS. If you have not " +
"explicitly set AWS_POLL_DELAY_SECONDS, we are defaulting to a " +
"poll delay of 2 seconds, regardless of the AWS waiter's default.")
maxAttempts := envOverrides.awsTimeoutSeconds.Val / envOverrides.awsPollDelaySeconds.Val
// override the delay so we can get the timeout right
if !envOverrides.awsPollDelaySeconds.overridden {
delaySeconds := request.ConstantWaiterDelay(time.Duration(envOverrides.awsPollDelaySeconds.Val) * time.Second)
waitOpts = append(waitOpts, request.WithWaiterDelay(delaySeconds))
}
waitOpts = append(waitOpts, request.WithWaiterMaxAttempts(maxAttempts))
}
if len(waitOpts) == 0 {
log.Printf("No AWS timeout and polling overrides have been set. " +
"Packer will default to waiter-specific delays and timeouts. If you would " +
"like to customize the length of time between retries and max " +
"number of retries you may do so by setting the environment " +
"variables AWS_POLL_DELAY_SECONDS and AWS_MAX_ATTEMPTS to your " +
"desired values.")
}
return waitOpts
}

66
builder/amazon/common/state_test.go Normal file
View File

@ -0,0 +1,66 @@
package common
import (
"reflect"
"testing"
"time"
"github.com/aws/aws-sdk-go/aws/request"
)
func testGetWaiterOptions(t *testing.T) {
// no vars are set
envValues := overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 0, false},
envInfo{"AWS_TIMEOUT_SECONDS", 300, false},
}
options := applyEnvOverrides(envValues)
if len(options) > 0 {
t.Fatalf("Did not expect any waiter options to be generated; actual: %#v", options)
}
// all vars are set
envValues = overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 1, true},
envInfo{"AWS_MAX_ATTEMPTS", 800, true},
envInfo{"AWS_TIMEOUT_SECONDS", 20, true},
}
options = applyEnvOverrides(envValues)
expected := []request.WaiterOption{
request.WithWaiterDelay(request.ConstantWaiterDelay(time.Duration(1) * time.Second)),
request.WithWaiterMaxAttempts(800),
}
if !reflect.DeepEqual(options, expected) {
t.Fatalf("expected != actual!! Expected: %#v; Actual: %#v.", expected, options)
}
// poll delay is not set
envValues = overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 800, true},
envInfo{"AWS_TIMEOUT_SECONDS", 300, false},
}
options = applyEnvOverrides(envValues)
expected = []request.WaiterOption{
request.WithWaiterMaxAttempts(800),
}
if !reflect.DeepEqual(options, expected) {
t.Fatalf("expected != actual!! Expected: %#v; Actual: %#v.", expected, options)
}
// poll delay is not set but timeout seconds is
envValues = overridableWaitVars{
envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
envInfo{"AWS_MAX_ATTEMPTS", 0, false},
envInfo{"AWS_TIMEOUT_SECONDS", 20, true},
}
options = applyEnvOverrides(envValues)
expected = []request.WaiterOption{
request.WithWaiterDelay(request.ConstantWaiterDelay(time.Duration(2) * time.Second)),
request.WithWaiterMaxAttempts(10),
}
if !reflect.DeepEqual(options, expected) {
t.Fatalf("expected != actual!! Expected: %#v; Actual: %#v.", expected, options)
}
}

19
builder/amazon/common/step_ami_region_copy.go
View File

@ -1,14 +1,15 @@
package common
import (
"context"
"fmt"
"sync"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepAMIRegionCopy struct {
@ -19,7 +20,7 @@ type StepAMIRegionCopy struct {
Name string
}
func (s *StepAMIRegionCopy) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepAMIRegionCopy) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
amis := state.Get("amis").(map[string]string)
@ -52,7 +53,7 @@ func (s *StepAMIRegionCopy) Run(state multistep.StateBag) multistep.StepAction {
go func(region string) {
defer wg.Done()
id, snapshotIds, err := amiRegionCopy(state, s.AccessConfig, s.Name, ami, region, *ec2conn.Config.Region, regKeyID)
id, snapshotIds, err := amiRegionCopy(ctx, state, s.AccessConfig, s.Name, ami, region, *ec2conn.Config.Region, regKeyID)
lock.Lock()
defer lock.Unlock()
amis[region] = id
@ -84,7 +85,7 @@ func (s *StepAMIRegionCopy) Cleanup(state multistep.StateBag) {
// amiRegionCopy does a copy for the given AMI to the target region and
// returns the resulting ID and snapshot IDs, or error.
func amiRegionCopy(state multistep.StateBag, config *AccessConfig, name string, imageId string,
func amiRegionCopy(ctx context.Context, state multistep.StateBag, config *AccessConfig, name string, imageId string,
target string, source string, keyID string) (string, []string, error) {
snapshotIds := []string{}
isEncrypted := false
@ -115,14 +116,8 @@ func amiRegionCopy(state multistep.StateBag, config *AccessConfig, name string,
imageId, target, err)
}
stateChange := StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: AMIStateRefreshFunc(regionconn, *resp.ImageId),
StepState: state,
}
if _, err := WaitForState(&stateChange); err != nil {
// Wait for the image to become ready
if err := WaitUntilAMIAvailable(ctx, regionconn, *resp.ImageId); err != nil {
return "", snapshotIds, fmt.Errorf("Error waiting for AMI (%s) in region (%s): %s",
*resp.ImageId, target, err)
}

82
builder/amazon/common/step_create_tags.go
View File

@ -1,6 +1,7 @@
package common
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
@ -8,30 +9,24 @@ import (
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
retry "github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
type StepCreateTags struct {
Tags map[string]string
SnapshotTags map[string]string
Tags TagMap
SnapshotTags TagMap
Ctx interpolate.Context
}
func (s *StepCreateTags) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepCreateTags) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
session := state.Get("awsSession").(*session.Session)
ui := state.Get("ui").(packer.Ui)
amis := state.Get("amis").(map[string]string)
var sourceAMI string
if rawSourceAMI, hasSourceAMI := state.GetOk("source_image"); hasSourceAMI {
sourceAMI = *rawSourceAMI.(*ec2.Image).ImageId
} else {
sourceAMI = ""
}
if len(s.Tags) == 0 && len(s.SnapshotTags) == 0 {
if !s.Tags.IsSet() && !s.SnapshotTags.IsSet() {
return multistep.ActionContinue
}
@ -39,23 +34,13 @@ func (s *StepCreateTags) Run(state multistep.StateBag) multistep.StepAction {
for region, ami := range amis {
ui.Say(fmt.Sprintf("Adding tags to AMI (%s)...", ami))
// Declare list of resources to tag
awsConfig := aws.Config{
Credentials: ec2conn.Config.Credentials,
Region: aws.String(region),
}
session, err := session.NewSession(&awsConfig)
if err != nil {
err := fmt.Errorf("Error creating AWS session: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
regionconn := ec2.New(session)
regionConn := ec2.New(session, &aws.Config{
Region: aws.String(region),
})
// Retrieve image list for given AMI
resourceIds := []*string{&ami}
imageResp, err := regionconn.DescribeImages(&ec2.DescribeImagesInput{
imageResp, err := regionConn.DescribeImages(&ec2.DescribeImagesInput{
ImageIds: resourceIds,
})
@ -87,27 +72,27 @@ func (s *StepCreateTags) Run(state multistep.StateBag) multistep.StepAction {
// Convert tags to ec2.Tag format
ui.Say("Creating AMI tags")
amiTags, err := ConvertToEC2Tags(s.Tags, *ec2conn.Config.Region, sourceAMI, s.Ctx)
amiTags, err := s.Tags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ReportTags(ui, amiTags)
amiTags.Report(ui)
ui.Say("Creating snapshot tags")
snapshotTags, err := ConvertToEC2Tags(s.SnapshotTags, *ec2conn.Config.Region, sourceAMI, s.Ctx)
snapshotTags, err := s.SnapshotTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ReportTags(ui, snapshotTags)
snapshotTags.Report(ui)
// Retry creating tags for about 2.5 minutes
err = retry.Retry(0.2, 30, 11, func(_ uint) (bool, error) {
// Tag images and snapshots
_, err := regionconn.CreateTags(&ec2.CreateTagsInput{
_, err := regionConn.CreateTags(&ec2.CreateTagsInput{
Resources: resourceIds,
Tags: amiTags,
})
@ -120,7 +105,7 @@ func (s *StepCreateTags) Run(state multistep.StateBag) multistep.StepAction {
// Override tags on snapshots
if len(snapshotTags) > 0 {
_, err = regionconn.CreateTags(&ec2.CreateTagsInput{
_, err = regionConn.CreateTags(&ec2.CreateTagsInput{
Resources: snapshotIds,
Tags: snapshotTags,
})
@ -150,36 +135,3 @@ func (s *StepCreateTags) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepCreateTags) Cleanup(state multistep.StateBag) {
// No cleanup...
}
func ReportTags(ui packer.Ui, tags []*ec2.Tag) {
for _, tag := range tags {
ui.Message(fmt.Sprintf("Adding tag: \"%s\": \"%s\"",
aws.StringValue(tag.Key), aws.StringValue(tag.Value)))
}
}
func ConvertToEC2Tags(tags map[string]string, region, sourceAmiId string, ctx interpolate.Context) ([]*ec2.Tag, error) {
var ec2Tags []*ec2.Tag
for key, value := range tags {
ctx.Data = &BuildInfoTemplate{
SourceAMI: sourceAmiId,
BuildRegion: region,
}
interpolatedKey, err := interpolate.Render(key, &ctx)
if err != nil {
return ec2Tags, fmt.Errorf("Error processing tag: %s:%s - %s", key, value, err)
}
interpolatedValue, err := interpolate.Render(value, &ctx)
if err != nil {
return ec2Tags, fmt.Errorf("Error processing tag: %s:%s - %s", key, value, err)
}
ec2Tags = append(ec2Tags, &ec2.Tag{
Key: aws.String(interpolatedKey),
Value: aws.String(interpolatedValue),
})
}
return ec2Tags, nil
}

8
builder/amazon/common/step_deregister_ami.go
View File

@ -1,12 +1,13 @@
package common
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepDeregisterAMI struct {
@ -17,7 +18,7 @@ type StepDeregisterAMI struct {
Regions []string
}
func (s *StepDeregisterAMI) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepDeregisterAMI) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
// Check for force deregister
if !s.ForceDeregister {
return multistep.ActionContinue
@ -40,9 +41,10 @@ func (s *StepDeregisterAMI) Run(state multistep.StateBag) multistep.StepAction {
))
resp, err := regionconn.DescribeImages(&ec2.DescribeImagesInput{
Owners: aws.StringSlice([]string{"self"}),
Filters: []*ec2.Filter{{
Name: aws.String("name"),
Values: []*string{aws.String(s.AMIName)},
Values: aws.StringSlice([]string{s.AMIName}),
}}})
if err != nil {

14
builder/amazon/common/step_encrypted_ami.go
View File

@ -1,13 +1,14 @@
package common
import (
"context"
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepCreateEncryptedAMICopy struct {
@ -18,7 +19,7 @@ type StepCreateEncryptedAMICopy struct {
AMIMappings []BlockDevice
}
func (s *StepCreateEncryptedAMICopy) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepCreateEncryptedAMICopy) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
kmsKeyId := s.KeyID
@ -64,15 +65,8 @@ func (s *StepCreateEncryptedAMICopy) Run(state multistep.StateBag) multistep.Ste
}
// Wait for the copy to become ready
stateChange := StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: AMIStateRefreshFunc(ec2conn, *copyResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI copy to become ready...")
if _, err := WaitForState(&stateChange); err != nil {
if err := WaitUntilAMIAvailable(ctx, ec2conn, *copyResp.ImageId); err != nil {
err := fmt.Errorf("Error waiting for AMI Copy: %s", err)
state.Put("error", err)
ui.Error(err.Error())

19
builder/amazon/common/step_get_password.go
View File

@ -1,6 +1,7 @@
package common
import (
"context"
"crypto/rsa"
"crypto/x509"
"encoding/base64"
@ -11,20 +12,22 @@ import (
"time"
"github.com/aws/aws-sdk-go/service/ec2"
commonhelper "github.com/hashicorp/packer/helper/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepGetPassword reads the password from a Windows server and sets it
// on the WinRM config.
type StepGetPassword struct {
Debug bool
Comm *communicator.Config
Timeout time.Duration
Debug bool
Comm *communicator.Config
Timeout time.Duration
BuildName string
}
func (s *StepGetPassword) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepGetPassword) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
// Skip if we're not using winrm
@ -91,11 +94,15 @@ WaitLoop:
ui.Message(fmt.Sprintf(
"Password (since debug is enabled): %s", s.Comm.WinRMPassword))
}
// store so that we can access this later during provisioning
commonhelper.SetSharedState("winrm_password", s.Comm.WinRMPassword, s.BuildName)
return multistep.ActionContinue
}
func (s *StepGetPassword) Cleanup(multistep.StateBag) {}
func (s *StepGetPassword) Cleanup(multistep.StateBag) {
commonhelper.RemoveSharedStateFile("winrm_password", s.BuildName)
}
func (s *StepGetPassword) waitForPassword(state multistep.StateBag, cancel <-chan struct{}) (string, error) {
ec2conn := state.Get("ec2").(*ec2.EC2)

5
builder/amazon/common/step_key_pair.go
View File

@ -1,14 +1,15 @@
package common
import (
"context"
"fmt"
"io/ioutil"
"os"
"runtime"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepKeyPair struct {
@ -22,7 +23,7 @@ type StepKeyPair struct {
doCleanup bool
}
func (s *StepKeyPair) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepKeyPair) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
if s.PrivateKeyFile != "" {

46
builder/amazon/common/step_modify_ami_attributes.go
View File

@ -1,14 +1,15 @@
package common
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
type StepModifyAMIAttributes struct {
@ -21,17 +22,11 @@ type StepModifyAMIAttributes struct {
Ctx interpolate.Context
}
func (s *StepModifyAMIAttributes) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepModifyAMIAttributes) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
session := state.Get("awsSession").(*session.Session)
ui := state.Get("ui").(packer.Ui)
amis := state.Get("amis").(map[string]string)
var sourceAMI string
if rawSourceAMI, hasSourceAMI := state.GetOk("source_image"); hasSourceAMI {
sourceAMI = *rawSourceAMI.(*ec2.Image).ImageId
} else {
sourceAMI = ""
}
snapshots := state.Get("snapshots").(map[string][]string)
// Determine if there is any work to do.
@ -48,10 +43,7 @@ func (s *StepModifyAMIAttributes) Run(state multistep.StateBag) multistep.StepAc
}
var err error
s.Ctx.Data = &BuildInfoTemplate{
SourceAMI: sourceAMI,
BuildRegion: *ec2conn.Config.Region,
}
s.Ctx.Data = extractBuildInfo(*ec2conn.Config.Region, state)
s.Description, err = interpolate.Render(s.Description, &s.Ctx)
if err != nil {
err = fmt.Errorf("Error interpolating AMI description: %s", err)
@ -152,22 +144,13 @@ func (s *StepModifyAMIAttributes) Run(state multistep.StateBag) multistep.StepAc
// Modifying image attributes
for region, ami := range amis {
ui.Say(fmt.Sprintf("Modifying attributes on AMI (%s)...", ami))
awsConfig := aws.Config{
Credentials: ec2conn.Config.Credentials,
Region: aws.String(region),
}
session, err := session.NewSession(&awsConfig)
if err != nil {
err := fmt.Errorf("Error creating AWS session: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
regionconn := ec2.New(session)
regionConn := ec2.New(session, &aws.Config{
Region: aws.String(region),
})
for name, input := range options {
ui.Message(fmt.Sprintf("Modifying: %s", name))
input.ImageId = &ami
_, err := regionconn.ModifyImageAttribute(input)
_, err := regionConn.ModifyImageAttribute(input)
if err != nil {
err := fmt.Errorf("Error modify AMI attributes: %s", err)
state.Put("error", err)
@ -181,16 +164,13 @@ func (s *StepModifyAMIAttributes) Run(state multistep.StateBag) multistep.StepAc
for region, region_snapshots := range snapshots {
for _, snapshot := range region_snapshots {
ui.Say(fmt.Sprintf("Modifying attributes on snapshot (%s)...", snapshot))
awsConfig := aws.Config{
Credentials: ec2conn.Config.Credentials,
Region: aws.String(region),
}
session := session.New(&awsConfig)
regionconn := ec2.New(session)
regionConn := ec2.New(session, &aws.Config{
Region: aws.String(region),
})
for name, input := range snapshotOptions {
ui.Message(fmt.Sprintf("Modifying: %s", name))
input.SnapshotId = &snapshot
_, err := regionconn.ModifySnapshotAttribute(input)
_, err := regionConn.ModifySnapshotAttribute(input)
if err != nil {
err := fmt.Errorf("Error modify snapshot attributes: %s", err)
state.Put("error", err)

5
builder/amazon/common/step_modify_ebs_instance.go
View File

@ -1,12 +1,13 @@
package common
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepModifyEBSBackedInstance struct {
@ -14,7 +15,7 @@ type StepModifyEBSBackedInstance struct {
EnableAMISriovNetSupport bool
}
func (s *StepModifyEBSBackedInstance) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepModifyEBSBackedInstance) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
ui := state.Get("ui").(packer.Ui)

5
builder/amazon/common/step_pre_validate.go
View File

@ -1,12 +1,13 @@
package common
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepPreValidate provides an opportunity to pre-validate any configuration for
@ -17,7 +18,7 @@ type StepPreValidate struct {
ForceDeregister bool
}
func (s *StepPreValidate) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepPreValidate) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
if s.ForceDeregister {
ui.Say("Force Deregister flag found, skipping prevalidating AMI Name")

124
builder/amazon/common/step_run_source_instance.go
View File

@ -1,41 +1,46 @@
package common
import (
"context"
"encoding/base64"
"fmt"
"io/ioutil"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/ec2"
retry "github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
type StepRunSourceInstance struct {
AssociatePublicIpAddress bool
AvailabilityZone string
BlockDevices BlockDevices
Ctx interpolate.Context
Debug bool
EbsOptimized bool
EnableT2Unlimited bool
ExpectedRootDevice string
IamInstanceProfile string
InstanceInitiatedShutdownBehavior string
InstanceType string
IsRestricted bool
SourceAMI string
SubnetId string
Tags map[string]string
VolumeTags map[string]string
Tags TagMap
UserData string
UserDataFile string
Ctx interpolate.Context
VolumeTags TagMap
instanceId string
}
func (s *StepRunSourceInstance) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepRunSourceInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
var keyName string
if name, ok := state.GetOk("keyPair"); ok {
@ -84,16 +89,15 @@ func (s *StepRunSourceInstance) Run(state multistep.StateBag) multistep.StepActi
s.Tags["Name"] = "Packer Builder"
}
ec2Tags, err := ConvertToEC2Tags(s.Tags, *ec2conn.Config.Region, s.SourceAMI, s.Ctx)
ec2Tags, err := s.Tags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging source instance: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ReportTags(ui, ec2Tags)
volTags, err := ConvertToEC2Tags(s.VolumeTags, *ec2conn.Config.Region, s.SourceAMI, s.Ctx)
volTags, err := s.VolumeTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging volumes: %s", err)
state.Put("error", err)
@ -113,6 +117,12 @@ func (s *StepRunSourceInstance) Run(state multistep.StateBag) multistep.StepActi
EbsOptimized: &s.EbsOptimized,
}
if s.EnableT2Unlimited {
creditOption := "unlimited"
runOpts.CreditSpecification = &ec2.CreditSpecificationRequest{CpuCredits: &creditOption}
}
// Collect tags for tagging on resource creation
var tagSpecs []*ec2.TagSpecification
if len(ec2Tags) > 0 {
@ -133,8 +143,11 @@ func (s *StepRunSourceInstance) Run(state multistep.StateBag) multistep.StepActi
tagSpecs = append(tagSpecs, runVolTags)
}
if len(tagSpecs) > 0 {
// If our region supports it, set tag specifications
if len(tagSpecs) > 0 && !s.IsRestricted {
runOpts.SetTagSpecifications(tagSpecs)
ec2Tags.Report(ui)
volTags.Report(ui)
}
if keyName != "" {
@ -174,21 +187,26 @@ func (s *StepRunSourceInstance) Run(state multistep.StateBag) multistep.StepActi
ui.Message(fmt.Sprintf("Instance ID: %s", instanceId))
ui.Say(fmt.Sprintf("Waiting for instance (%v) to become ready...", instanceId))
stateChange := StateChangeConf{
Pending: []string{"pending"},
Target: "running",
Refresh: InstanceStateRefreshFunc(ec2conn, instanceId),
StepState: state,
describeInstance := &ec2.DescribeInstancesInput{
InstanceIds: []*string{aws.String(instanceId)},
}
latestInstance, err := WaitForState(&stateChange)
if err != nil {
if err := ec2conn.WaitUntilInstanceRunningWithContext(ctx, describeInstance); err != nil {
err := fmt.Errorf("Error waiting for instance (%s) to become ready: %s", instanceId, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
instance := latestInstance.(*ec2.Instance)
r, err := ec2conn.DescribeInstances(describeInstance)
if err != nil || len(r.Reservations) == 0 || len(r.Reservations[0].Instances) == 0 {
err := fmt.Errorf("Error finding source instance.")
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
instance := r.Reservations[0].Instances[0]
if s.Debug {
if instance.PublicDnsName != nil && *instance.PublicDnsName != "" {
@ -206,6 +224,70 @@ func (s *StepRunSourceInstance) Run(state multistep.StateBag) multistep.StepActi
state.Put("instance", instance)
// If we're in a region that doesn't support tagging on instance creation,
// do that now.
if s.IsRestricted {
ec2Tags.Report(ui)
// Retry creating tags for about 2.5 minutes
err = retry.Retry(0.2, 30, 11, func(_ uint) (bool, error) {
_, err := ec2conn.CreateTags(&ec2.CreateTagsInput{
Tags: ec2Tags,
Resources: []*string{instance.InstanceId},
})
if err == nil {
return true, nil
}
if awsErr, ok := err.(awserr.Error); ok {
if awsErr.Code() == "InvalidInstanceID.NotFound" {
return false, nil
}
}
return true, err
})
if err != nil {
err := fmt.Errorf("Error tagging source instance: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
// Now tag volumes
volumeIds := make([]*string, 0)
for _, v := range instance.BlockDeviceMappings {
if ebs := v.Ebs; ebs != nil {
volumeIds = append(volumeIds, ebs.VolumeId)
}
}
if len(volumeIds) > 0 && s.VolumeTags.IsSet() {
ui.Say("Adding tags to source EBS Volumes")
volumeTags, err := s.VolumeTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging source EBS Volumes on %s: %s", *instance.InstanceId, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
volumeTags.Report(ui)
_, err = ec2conn.CreateTags(&ec2.CreateTagsInput{
Resources: volumeIds,
Tags: volumeTags,
})
if err != nil {
err := fmt.Errorf("Error tagging source EBS Volumes on %s: %s", *instance.InstanceId, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
}
}
return multistep.ActionContinue
}
@ -221,14 +303,8 @@ func (s *StepRunSourceInstance) Cleanup(state multistep.StateBag) {
ui.Error(fmt.Sprintf("Error terminating instance, may still be around: %s", err))
return
}
stateChange := StateChangeConf{
Pending: []string{"pending", "running", "shutting-down", "stopped", "stopping"},
Refresh: InstanceStateRefreshFunc(ec2conn, s.instanceId),
Target: "terminated",
}
_, err := WaitForState(&stateChange)
if err != nil {
if err := WaitUntilInstanceTerminated(aws.BackgroundContext(), ec2conn, s.instanceId); err != nil {
ui.Error(err.Error())
}
}

95
builder/amazon/common/step_run_spot_instance.go
View File

@ -1,6 +1,7 @@
package common
import (
"context"
"encoding/base64"
"fmt"
"io/ioutil"
@ -13,9 +14,9 @@ import (
"github.com/aws/aws-sdk-go/service/ec2"
retry "github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
type StepRunSpotInstance struct {
@ -31,9 +32,10 @@ type StepRunSpotInstance struct {
SourceAMI string
SpotPrice string
SpotPriceProduct string
SpotTags TagMap
SubnetId string
Tags map[string]string
VolumeTags map[string]string
Tags TagMap
VolumeTags TagMap
UserData string
UserDataFile string
Ctx interpolate.Context
@ -42,7 +44,7 @@ type StepRunSpotInstance struct {
spotRequest *ec2.SpotInstanceRequest
}
func (s *StepRunSpotInstance) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
var keyName string
if name, ok := state.GetOk("keyPair"); ok {
@ -142,14 +144,14 @@ func (s *StepRunSpotInstance) Run(state multistep.StateBag) multistep.StepAction
s.Tags["Name"] = "Packer Builder"
}
ec2Tags, err := ConvertToEC2Tags(s.Tags, *ec2conn.Config.Region, s.SourceAMI, s.Ctx)
ec2Tags, err := s.Tags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging source instance: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ReportTags(ui, ec2Tags)
ec2Tags.Report(ui)
ui.Message(fmt.Sprintf(
"Requesting spot instance '%s' for: %s",
@ -201,13 +203,7 @@ func (s *StepRunSpotInstance) Run(state multistep.StateBag) multistep.StepAction
spotRequestId := s.spotRequest.SpotInstanceRequestId
ui.Message(fmt.Sprintf("Waiting for spot request (%s) to become active...", *spotRequestId))
stateChange := StateChangeConf{
Pending: []string{"open"},
Target: "active",
Refresh: SpotRequestStateRefreshFunc(ec2conn, *spotRequestId),
StepState: state,
}
_, err = WaitForState(&stateChange)
err = WaitUntilSpotRequestFulfilled(ctx, ec2conn, *spotRequestId)
if err != nil {
err := fmt.Errorf("Error waiting for spot request (%s) to become ready: %s", *spotRequestId, err)
state.Put("error", err)
@ -226,26 +222,58 @@ func (s *StepRunSpotInstance) Run(state multistep.StateBag) multistep.StepAction
}
instanceId = *spotResp.SpotInstanceRequests[0].InstanceId
// Tag spot instance request
spotTags, err := s.SpotTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging spot request: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
spotTags.Report(ui)
if len(spotTags) > 0 && s.SpotTags.IsSet() {
// Retry creating tags for about 2.5 minutes
err = retry.Retry(0.2, 30, 11, func(_ uint) (bool, error) {
_, err := ec2conn.CreateTags(&ec2.CreateTagsInput{
Tags: spotTags,
Resources: []*string{spotRequestId},
})
return true, err
})
if err != nil {
err := fmt.Errorf("Error tagging spot request: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
}
// Set the instance ID so that the cleanup works properly
s.instanceId = instanceId
ui.Message(fmt.Sprintf("Instance ID: %s", instanceId))
ui.Say(fmt.Sprintf("Waiting for instance (%v) to become ready...", instanceId))
stateChangeSpot := StateChangeConf{
Pending: []string{"pending"},
Target: "running",
Refresh: InstanceStateRefreshFunc(ec2conn, instanceId),
StepState: state,
describeInstance := &ec2.DescribeInstancesInput{
InstanceIds: []*string{aws.String(instanceId)},
}
latestInstance, err := WaitForState(&stateChangeSpot)
if err != nil {
if err := ec2conn.WaitUntilInstanceRunningWithContext(ctx, describeInstance); err != nil {
err := fmt.Errorf("Error waiting for instance (%s) to become ready: %s", instanceId, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
instance := latestInstance.(*ec2.Instance)
r, err := ec2conn.DescribeInstances(&ec2.DescribeInstancesInput{
InstanceIds: []*string{aws.String(instanceId)},
})
if err != nil || len(r.Reservations) == 0 || len(r.Reservations[0].Instances) == 0 {
err := fmt.Errorf("Error finding source instance.")
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
instance := r.Reservations[0].Instances[0]
// Retry creating tags for about 2.5 minutes
err = retry.Retry(0.2, 30, 11, func(_ uint) (bool, error) {
@ -278,21 +306,21 @@ func (s *StepRunSpotInstance) Run(state multistep.StateBag) multistep.StepAction
}
}
if len(volumeIds) > 0 && len(s.VolumeTags) > 0 {
if len(volumeIds) > 0 && s.VolumeTags.IsSet() {
ui.Say("Adding tags to source EBS Volumes")
tags, err := ConvertToEC2Tags(s.VolumeTags, *ec2conn.Config.Region, s.SourceAMI, s.Ctx)
volumeTags, err := s.VolumeTags.EC2Tags(s.Ctx, *ec2conn.Config.Region, state)
if err != nil {
err := fmt.Errorf("Error tagging source EBS Volumes on %s: %s", *instance.InstanceId, err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
ReportTags(ui, tags)
volumeTags.Report(ui)
_, err = ec2conn.CreateTags(&ec2.CreateTagsInput{
Resources: volumeIds,
Tags: tags,
Tags: volumeTags,
})
if err != nil {
@ -338,13 +366,8 @@ func (s *StepRunSpotInstance) Cleanup(state multistep.StateBag) {
ui.Error(fmt.Sprintf("Error cancelling the spot request, may still be around: %s", err))
return
}
stateChange := StateChangeConf{
Pending: []string{"active", "open"},
Refresh: SpotRequestStateRefreshFunc(ec2conn, *s.spotRequest.SpotInstanceRequestId),
Target: "cancelled",
}
_, err := WaitForState(&stateChange)
err := WaitUntilSpotRequestFulfilled(aws.BackgroundContext(), ec2conn, *s.spotRequest.SpotInstanceRequestId)
if err != nil {
ui.Error(err.Error())
}
@ -358,14 +381,8 @@ func (s *StepRunSpotInstance) Cleanup(state multistep.StateBag) {
ui.Error(fmt.Sprintf("Error terminating instance, may still be around: %s", err))
return
}
stateChange := StateChangeConf{
Pending: []string{"pending", "running", "shutting-down", "stopped", "stopping"},
Refresh: InstanceStateRefreshFunc(ec2conn, s.instanceId),
Target: "terminated",
}
_, err := WaitForState(&stateChange)
if err != nil {
if err := WaitUntilInstanceTerminated(aws.BackgroundContext(), ec2conn, s.instanceId); err != nil {
ui.Error(err.Error())
}
}

5
builder/amazon/common/step_security_group.go
View File

@ -1,6 +1,7 @@
package common
import (
"context"
"fmt"
"log"
"time"
@ -10,8 +11,8 @@ import (
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/common/uuid"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepSecurityGroup struct {
@ -23,7 +24,7 @@ type StepSecurityGroup struct {
createdGroupId string
}
func (s *StepSecurityGroup) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepSecurityGroup) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)

5
builder/amazon/common/step_source_ami_info.go
View File

@ -1,14 +1,15 @@
package common
import (
"context"
"fmt"
"log"
"sort"
"time"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepSourceAMIInfo extracts critical information from the source AMI
@ -52,7 +53,7 @@ func mostRecentAmi(images []*ec2.Image) *ec2.Image {
return sortedImages[len(sortedImages)-1]
}
func (s *StepSourceAMIInfo) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepSourceAMIInfo) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)

20
builder/amazon/common/step_stop_ebs_instance.go
View File

@ -1,13 +1,14 @@
package common
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type StepStopEBSBackedInstance struct {
@ -15,7 +16,7 @@ type StepStopEBSBackedInstance struct {
DisableStopInstance bool
}
func (s *StepStopEBSBackedInstance) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepStopEBSBackedInstance) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
ui := state.Get("ui").(packer.Ui)
@ -75,15 +76,14 @@ func (s *StepStopEBSBackedInstance) Run(state multistep.StateBag) multistep.Step
ui.Say("Automatic instance stop disabled. Please stop instance manually.")
}
// Wait for the instance to actual stop
// Wait for the instance to actually stop
ui.Say("Waiting for the instance to stop...")
stateChange := StateChangeConf{
Pending: []string{"running", "pending", "stopping"},
Target: "stopped",
Refresh: InstanceStateRefreshFunc(ec2conn, *instance.InstanceId),
StepState: state,
}
_, err = WaitForState(&stateChange)
err = ec2conn.WaitUntilInstanceStoppedWithContext(ctx,
&ec2.DescribeInstancesInput{
InstanceIds: []*string{instance.InstanceId},
},
getWaiterOptions()...)
if err != nil {
err := fmt.Errorf("Error waiting for instance to stop: %s", err)
state.Put("error", err)

46
builder/amazon/common/tags.go Normal file
View File

@ -0,0 +1,46 @@
package common
import (
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
)
type TagMap map[string]string
type EC2Tags []*ec2.Tag
func (t EC2Tags) Report(ui packer.Ui) {
for _, tag := range t {
ui.Message(fmt.Sprintf("Adding tag: \"%s\": \"%s\"",
aws.StringValue(tag.Key), aws.StringValue(tag.Value)))
}
}
func (t TagMap) IsSet() bool {
return len(t) > 0
}
func (t TagMap) EC2Tags(ctx interpolate.Context, region string, state multistep.StateBag) (EC2Tags, error) {
var ec2Tags []*ec2.Tag
ctx.Data = extractBuildInfo(region, state)
for key, value := range t {
interpolatedKey, err := interpolate.Render(key, &ctx)
if err != nil {
return nil, fmt.Errorf("Error processing tag: %s:%s - %s", key, value, err)
}
interpolatedValue, err := interpolate.Render(value, &ctx)
if err != nil {
return nil, fmt.Errorf("Error processing tag: %s:%s - %s", key, value, err)
}
ec2Tags = append(ec2Tags, &ec2.Tag{
Key: aws.String(interpolatedKey),
Value: aws.String(interpolatedValue),
})
}
return ec2Tags, nil
}

98
builder/amazon/ebs/builder.go
View File

@ -14,9 +14,9 @@ import (
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
// The unique ID for this builder
@ -28,7 +28,7 @@ type Config struct {
awscommon.AMIConfig `mapstructure:",squash"`
awscommon.BlockDevices `mapstructure:",squash"`
awscommon.RunConfig `mapstructure:",squash"`
VolumeRunTags map[string]string `mapstructure:"run_volume_tags"`
VolumeRunTags awscommon.TagMap `mapstructure:"run_volume_tags"`
ctx interpolate.Context
}
@ -48,6 +48,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"ami_description",
"run_tags",
"run_volume_tags",
"spot_tags",
"snapshot_tags",
"tags",
},
@ -69,11 +70,18 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs = packer.MultiErrorAppend(errs, b.config.BlockDevices.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
if b.config.IsSpotInstance() && (b.config.AMIENASupport || b.config.AMISriovNetSupport) {
errs = packer.MultiErrorAppend(errs,
fmt.Errorf("Spot instances do not support modification, which is required "+
"when either `ena_support` or `sriov_support` are set. Please ensure "+
"you use an AMI that already has either SR-IOV or ENA enabled."))
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
}
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey))
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return nil, nil
}
@ -106,51 +114,54 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
state := new(multistep.BasicStateBag)
state.Put("config", b.config)
state.Put("ec2", ec2conn)
state.Put("awsSession", session)
state.Put("hook", hook)
state.Put("ui", ui)
var instanceStep multistep.Step
isSpotInstance := b.config.SpotPrice != "" && b.config.SpotPrice != "0"
if isSpotInstance {
if b.config.IsSpotInstance() {
instanceStep = &awscommon.StepRunSpotInstance{
Debug: b.config.PackerDebug,
ExpectedRootDevice: "ebs",
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
InstanceType: b.config.InstanceType,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
SourceAMI: b.config.SourceAmi,
IamInstanceProfile: b.config.IamInstanceProfile,
SubnetId: b.config.SubnetId,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
EbsOptimized: b.config.EbsOptimized,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Tags: b.config.RunTags,
VolumeTags: b.config.VolumeRunTags,
Ctx: b.config.ctx,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotTags: b.config.SpotTags,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
VolumeTags: b.config.VolumeRunTags,
}
} else {
instanceStep = &awscommon.StepRunSourceInstance{
Debug: b.config.PackerDebug,
ExpectedRootDevice: "ebs",
InstanceType: b.config.InstanceType,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
SourceAMI: b.config.SourceAmi,
IamInstanceProfile: b.config.IamInstanceProfile,
SubnetId: b.config.SubnetId,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
EbsOptimized: b.config.EbsOptimized,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Tags: b.config.RunTags,
VolumeTags: b.config.VolumeRunTags,
Ctx: b.config.ctx,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
IsRestricted: b.config.IsChinaCloud() || b.config.IsGovCloud(),
SourceAMI: b.config.SourceAmi,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
VolumeTags: b.config.VolumeRunTags,
}
}
@ -185,15 +196,16 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
},
instanceStep,
&awscommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.SSHPrivateIp),
b.config.SSHInterface),
SSHConfig: awscommon.SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
@ -201,7 +213,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
},
&common.StepProvision{},
&awscommon.StepStopEBSBackedInstance{
Skip: isSpotInstance,
Skip: b.config.IsSpotInstance(),
DisableStopInstance: b.config.DisableStopInstance,
},
&awscommon.StepModifyEBSBackedInstance{
@ -263,7 +275,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
artifact := &awscommon.Artifact{
Amis: state.Get("amis").(map[string]string),
BuilderIdValue: BuilderId,
Conn: ec2conn,
Session: session,
}
return artifact, nil

32
builder/amazon/ebs/builder_acc_test.go
View File

@ -1,8 +1,11 @@
/*
Deregister the test image with
aws ec2 deregister-image --image-id $(aws ec2 describe-images --output text --filters "Name=name,Values=packer-test-packer-test-dereg" --query 'Images[*].{ID:ImageId}')
*/
package ebs
import (
"fmt"
"os"
"testing"
"github.com/aws/aws-sdk-go/aws"
@ -58,14 +61,14 @@ func TestBuilderAcc_forceDeleteSnapshot(t *testing.T) {
// Get image data by AMI name
ec2conn, _ := testEC2Conn()
imageResp, _ := ec2conn.DescribeImages(
&ec2.DescribeImagesInput{Filters: []*ec2.Filter{
{
Name: aws.String("name"),
Values: []*string{aws.String(amiName)},
},
}},
)
describeInput := &ec2.DescribeImagesInput{Filters: []*ec2.Filter{
{
Name: aws.String("name"),
Values: []*string{aws.String(amiName)},
},
}}
ec2conn.WaitUntilImageExists(describeInput)
imageResp, _ := ec2conn.DescribeImages(describeInput)
image := imageResp.Images[0]
// Get snapshot ids for image
@ -243,13 +246,6 @@ func checkBootEncrypted() builderT.TestCheckFunc {
}
func testAccPreCheck(t *testing.T) {
if v := os.Getenv("AWS_ACCESS_KEY_ID"); v == "" {
t.Fatal("AWS_ACCESS_KEY_ID must be set for acceptance tests")
}
if v := os.Getenv("AWS_SECRET_ACCESS_KEY"); v == "" {
t.Fatal("AWS_SECRET_ACCESS_KEY must be set for acceptance tests")
}
}
func testEC2Conn() (*ec2.EC2, error) {
@ -298,7 +294,7 @@ const testBuilderAccForceDeregister = `
"source_ami": "ami-76b2a71e",
"ssh_username": "ubuntu",
"force_deregister": "%s",
"ami_name": "packer-test-%s"
"ami_name": "%s"
}]
}
`
@ -313,7 +309,7 @@ const testBuilderAccForceDeleteSnapshot = `
"ssh_username": "ubuntu",
"force_deregister": "%s",
"force_delete_snapshot": "%s",
"ami_name": "packer-test-%s"
"ami_name": "%s"
}]
}
`

5
builder/amazon/ebs/step_cleanup_volumes.go
View File

@ -1,13 +1,14 @@
package ebs
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// stepCleanupVolumes cleans up any orphaned volumes that were not designated to
@ -17,7 +18,7 @@ type stepCleanupVolumes struct {
BlockDevices common.BlockDevices
}
func (s *stepCleanupVolumes) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepCleanupVolumes) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
// stepCleanupVolumes is for Cleanup only
return multistep.ActionContinue
}

26
builder/amazon/ebs/step_create_ami.go
View File

@ -1,19 +1,21 @@
package ebs
import (
"context"
"fmt"
"log"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
type stepCreateAMI struct {
image *ec2.Image
}
func (s *stepCreateAMI) Run(state multistep.StateBag) multistep.StepAction {
func (s *stepCreateAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
instance := state.Get("instance").(*ec2.Instance)
@ -42,16 +44,18 @@ func (s *stepCreateAMI) Run(state multistep.StateBag) multistep.StepAction {
state.Put("amis", amis)
// Wait for the image to become ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: awscommon.AMIStateRefreshFunc(ec2conn, *createResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI to become ready...")
if _, err := awscommon.WaitForState(&stateChange); err != nil {
err := fmt.Errorf("Error waiting for AMI: %s", err)
if err := awscommon.WaitUntilAMIAvailable(ctx, ec2conn, *createResp.ImageId); err != nil {
log.Printf("Error waiting for AMI: %s", err)
imagesResp, err := ec2conn.DescribeImages(&ec2.DescribeImagesInput{ImageIds: []*string{createResp.ImageId}})
if err != nil {
log.Printf("Unable to determine reason waiting for AMI failed: %s", err)
err = fmt.Errorf("Unknown error waiting for AMI.")
} else {
stateReason := imagesResp.Images[0].StateReason
err = fmt.Errorf("Error waiting for AMI. Reason: %s", stateReason)
}
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt

110
builder/amazon/ebssurrogate/builder.go
View File

@ -12,9 +12,9 @@ import (
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
const BuilderId = "mitchellh.amazon.ebssurrogate"
@ -26,8 +26,8 @@ type Config struct {
awscommon.BlockDevices `mapstructure:",squash"`
awscommon.AMIConfig `mapstructure:",squash"`
RootDevice RootBlockDevice `mapstructure:"ami_root_device"`
VolumeRunTags map[string]string `mapstructure:"run_volume_tags"`
RootDevice RootBlockDevice `mapstructure:"ami_root_device"`
VolumeRunTags awscommon.TagMap `mapstructure:"run_volume_tags"`
ctx interpolate.Context
}
@ -48,6 +48,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
"run_tags",
"run_volume_tags",
"snapshot_tags",
"spot_tags",
"tags",
},
},
@ -84,11 +85,18 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("no volume with name '%s' is found", b.config.RootDevice.SourceDeviceName))
}
if b.config.IsSpotInstance() && (b.config.AMIENASupport || b.config.AMISriovNetSupport) {
errs = packer.MultiErrorAppend(errs,
fmt.Errorf("Spot instances do not support modification, which is required "+
"when either `ena_support` or `sriov_support` are set. Please ensure "+
"you use an AMI that already has either SR-IOV or ENA enabled."))
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
}
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey))
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return nil, nil
}
@ -120,54 +128,60 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
state := new(multistep.BasicStateBag)
state.Put("config", &b.config)
state.Put("ec2", ec2conn)
state.Put("awsSession", session)
state.Put("hook", hook)
state.Put("ui", ui)
var instanceStep multistep.Step
isSpotInstance := b.config.SpotPrice != "" && b.config.SpotPrice != "0"
if isSpotInstance {
if b.config.IsSpotInstance() {
instanceStep = &awscommon.StepRunSpotInstance{
Debug: b.config.PackerDebug,
ExpectedRootDevice: "ebs",
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
InstanceType: b.config.InstanceType,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
SourceAMI: b.config.SourceAmi,
IamInstanceProfile: b.config.IamInstanceProfile,
SubnetId: b.config.SubnetId,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
EbsOptimized: b.config.EbsOptimized,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Tags: b.config.RunTags,
VolumeTags: b.config.VolumeRunTags,
Ctx: b.config.ctx,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotTags: b.config.SpotTags,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
VolumeTags: b.config.VolumeRunTags,
}
} else {
instanceStep = &awscommon.StepRunSourceInstance{
Debug: b.config.PackerDebug,
ExpectedRootDevice: "ebs",
InstanceType: b.config.InstanceType,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
SourceAMI: b.config.SourceAmi,
IamInstanceProfile: b.config.IamInstanceProfile,
SubnetId: b.config.SubnetId,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
EbsOptimized: b.config.EbsOptimized,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Tags: b.config.RunTags,
VolumeTags: b.config.VolumeRunTags,
Ctx: b.config.ctx,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.BlockDevices,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
IsRestricted: b.config.IsChinaCloud() || b.config.IsGovCloud(),
SourceAMI: b.config.SourceAmi,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
VolumeTags: b.config.VolumeRunTags,
}
}
amiDevices := b.config.BuildAMIDevices()
launchDevices := b.config.BuildLaunchDevices()
// Build the steps
steps := []multistep.Step{
&awscommon.StepPreValidate{
@ -196,15 +210,16 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
},
instanceStep,
&awscommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.SSHPrivateIp),
b.config.SSHInterface),
SSHConfig: awscommon.SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
@ -212,15 +227,15 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
},
&common.StepProvision{},
&awscommon.StepStopEBSBackedInstance{
Skip: isSpotInstance,
Skip: b.config.IsSpotInstance(),
DisableStopInstance: b.config.DisableStopInstance,
},
&awscommon.StepModifyEBSBackedInstance{
EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
EnableAMIENASupport: b.config.AMIENASupport,
},
&StepSnapshotNewRootVolume{
NewRootMountPoint: b.config.RootDevice.SourceDeviceName,
&StepSnapshotVolumes{
LaunchDevices: launchDevices,
},
&awscommon.StepDeregisterAMI{
AccessConfig: &b.config.AccessConfig,
@ -231,7 +246,8 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
},
&StepRegisterAMI{
RootDevice: b.config.RootDevice,
BlockDevices: b.config.BlockDevices.BuildAMIDevices(),
AMIDevices: amiDevices,
LaunchDevices: launchDevices,
EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
EnableAMIENASupport: b.config.AMIENASupport,
},
@ -277,7 +293,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
artifact := &awscommon.Artifact{
Amis: amis.(map[string]string),
BuilderIdValue: BuilderId,
Conn: ec2conn,
Session: session,
}
return artifact, nil

21
builder/amazon/ebssurrogate/root_block_device.go
View File

@ -2,8 +2,7 @@ package ebssurrogate
import (
"errors"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/hashicorp/packer/template/interpolate"
)
@ -45,21 +44,3 @@ func (c *RootBlockDevice) Prepare(ctx *interpolate.Context) []error {
return nil
}
func (d *RootBlockDevice) createBlockDeviceMapping(snapshotId string) *ec2.BlockDeviceMapping {
rootBlockDevice := &ec2.EbsBlockDevice{
SnapshotId: aws.String(snapshotId),
VolumeType: aws.String(d.VolumeType),
VolumeSize: aws.Int64(d.VolumeSize),
DeleteOnTermination: aws.Bool(d.DeleteOnTermination),
}
if d.IOPS != 0 {
rootBlockDevice.Iops = aws.Int64(d.IOPS)
}
return &ec2.BlockDeviceMapping{
DeviceName: aws.String(d.DeviceName),
Ebs: rootBlockDevice,
}
}

60
builder/amazon/ebssurrogate/step_register_ami.go
View File

@ -1,40 +1,42 @@
package ebssurrogate
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepRegisterAMI creates the AMI.
type StepRegisterAMI struct {
RootDevice RootBlockDevice
BlockDevices []*ec2.BlockDeviceMapping
AMIDevices []*ec2.BlockDeviceMapping
LaunchDevices []*ec2.BlockDeviceMapping
EnableAMIENASupport bool
EnableAMISriovNetSupport bool
image *ec2.Image
}
func (s *StepRegisterAMI) Run(state multistep.StateBag) multistep.StepAction {
func (s *StepRegisterAMI) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
config := state.Get("config").(*Config)
ec2conn := state.Get("ec2").(*ec2.EC2)
snapshotId := state.Get("snapshot_id").(string)
snapshotIds := state.Get("snapshot_ids").(map[string]string)
ui := state.Get("ui").(packer.Ui)
ui.Say("Registering the AMI...")
blockDevicesExcludingRoot := DeduplicateRootVolume(s.BlockDevices, s.RootDevice, snapshotId)
blockDevices := s.combineDevices(snapshotIds)
registerOpts := &ec2.RegisterImageInput{
Name: &config.AMIName,
Architecture: aws.String(ec2.ArchitectureValuesX8664),
RootDeviceName: aws.String(s.RootDevice.DeviceName),
VirtualizationType: aws.String(config.AMIVirtType),
BlockDeviceMappings: blockDevicesExcludingRoot,
BlockDeviceMappings: blockDevices,
}
if s.EnableAMISriovNetSupport {
@ -61,15 +63,8 @@ func (s *StepRegisterAMI) Run(state multistep.StateBag) multistep.StepAction {
state.Put("amis", amis)
// Wait for the image to become ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
Target: "available",
Refresh: awscommon.AMIStateRefreshFunc(ec2conn, *registerResp.ImageId),
StepState: state,
}
ui.Say("Waiting for AMI to become ready...")
if _, err := awscommon.WaitForState(&stateChange); err != nil {
if err := awscommon.WaitUntilAMIAvailable(ctx, ec2conn, *registerResp.ImageId); err != nil {
err := fmt.Errorf("Error waiting for AMI: %s", err)
state.Put("error", err)
ui.Error(err.Error())
@ -119,17 +114,34 @@ func (s *StepRegisterAMI) Cleanup(state multistep.StateBag) {
}
}
func DeduplicateRootVolume(BlockDevices []*ec2.BlockDeviceMapping, RootDevice RootBlockDevice, snapshotId string) []*ec2.BlockDeviceMapping {
// Defensive coding to make sure we only add the root volume once
blockDevicesExcludingRoot := make([]*ec2.BlockDeviceMapping, 0, len(BlockDevices))
for _, blockDevice := range BlockDevices {
if *blockDevice.DeviceName == RootDevice.SourceDeviceName {
continue
}
func (s *StepRegisterAMI) combineDevices(snapshotIds map[string]string) []*ec2.BlockDeviceMapping {
devices := map[string]*ec2.BlockDeviceMapping{}
blockDevicesExcludingRoot = append(blockDevicesExcludingRoot, blockDevice)
for _, device := range s.AMIDevices {
devices[*device.DeviceName] = device
}
blockDevicesExcludingRoot = append(blockDevicesExcludingRoot, RootDevice.createBlockDeviceMapping(snapshotId))
return blockDevicesExcludingRoot
// Devices in launch_block_device_mappings override any with
// the same name in ami_block_device_mappings, except for the
// one designated as the root device in ami_root_device
for _, device := range s.LaunchDevices {
snapshotId, ok := snapshotIds[*device.DeviceName]
if ok {
device.Ebs.SnapshotId = aws.String(snapshotId)
// Block devices with snapshot inherit
// encryption settings from the snapshot
device.Ebs.Encrypted = nil
device.Ebs.KmsKeyId = nil
}
if *device.DeviceName == s.RootDevice.SourceDeviceName {
device.DeviceName = aws.String(s.RootDevice.DeviceName)
}
devices[*device.DeviceName] = device
}
blockDevices := []*ec2.BlockDeviceMapping{}
for _, device := range devices {
blockDevices = append(blockDevices, device)
}
return blockDevices
}

262
builder/amazon/ebssurrogate/step_register_ami_test.go
View File

@ -1,37 +1,247 @@
package ebssurrogate
import (
"reflect"
"sort"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
)
func GetStringPointer() *string {
tmp := "/dev/name"
return &tmp
}
const sourceDeviceName = "/dev/xvdf"
const rootDeviceName = "/dev/xvda"
func GetTestDevice() *ec2.BlockDeviceMapping {
TestDev := ec2.BlockDeviceMapping{
DeviceName: GetStringPointer(),
}
return &TestDev
}
func TestStepRegisterAmi_DeduplicateRootVolume(t *testing.T) {
TestRootDevice := RootBlockDevice{}
TestRootDevice.SourceDeviceName = "/dev/name"
blockDevices := []*ec2.BlockDeviceMapping{}
blockDevicesExcludingRoot := DeduplicateRootVolume(blockDevices, TestRootDevice, "12342351")
if len(blockDevicesExcludingRoot) != 1 {
t.Fatalf("Unexpected length of block devices list")
}
TestBlockDevice := GetTestDevice()
blockDevices = append(blockDevices, TestBlockDevice)
blockDevicesExcludingRoot = DeduplicateRootVolume(blockDevices, TestRootDevice, "12342351")
if len(blockDevicesExcludingRoot) != 1 {
t.Fatalf("Unexpected length of block devices list")
func newStepRegisterAMI(amiDevices, launchDevices []*ec2.BlockDeviceMapping) *StepRegisterAMI {
return &StepRegisterAMI{
RootDevice: RootBlockDevice{
SourceDeviceName: sourceDeviceName,
DeviceName: rootDeviceName,
DeleteOnTermination: true,
VolumeType: "ebs",
VolumeSize: 10,
},
AMIDevices: amiDevices,
LaunchDevices: launchDevices,
}
}
func sorted(devices []*ec2.BlockDeviceMapping) []*ec2.BlockDeviceMapping {
sort.SliceStable(devices, func(i, j int) bool {
return *devices[i].DeviceName < *devices[j].DeviceName
})
return devices
}
func TestStepRegisterAmi_combineDevices(t *testing.T) {
cases := []struct {
snapshotIds map[string]string
amiDevices []*ec2.BlockDeviceMapping
launchDevices []*ec2.BlockDeviceMapping
allDevices []*ec2.BlockDeviceMapping
}{
{
snapshotIds: map[string]string{},
amiDevices: []*ec2.BlockDeviceMapping{},
launchDevices: []*ec2.BlockDeviceMapping{},
allDevices: []*ec2.BlockDeviceMapping{},
},
{
snapshotIds: map[string]string{},
amiDevices: []*ec2.BlockDeviceMapping{},
launchDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String(sourceDeviceName),
},
},
allDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String(rootDeviceName),
},
},
},
{
// Minimal single device
snapshotIds: map[string]string{
sourceDeviceName: "snap-0123456789abcdef1",
},
amiDevices: []*ec2.BlockDeviceMapping{},
launchDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String(sourceDeviceName),
},
},
allDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef1"),
},
DeviceName: aws.String(rootDeviceName),
},
},
},
{
// Single launch device with AMI device
snapshotIds: map[string]string{
sourceDeviceName: "snap-0123456789abcdef1",
},
amiDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String("/dev/xvdg"),
},
},
launchDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String(sourceDeviceName),
},
},
allDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef1"),
},
DeviceName: aws.String(rootDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String("/dev/xvdg"),
},
},
},
{
// Multiple launch devices
snapshotIds: map[string]string{
sourceDeviceName: "snap-0123456789abcdef1",
"/dev/xvdg": "snap-0123456789abcdef2",
},
amiDevices: []*ec2.BlockDeviceMapping{},
launchDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String(sourceDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{},
DeviceName: aws.String("/dev/xvdg"),
},
},
allDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef1"),
},
DeviceName: aws.String(rootDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef2"),
},
DeviceName: aws.String("/dev/xvdg"),
},
},
},
{
// Multiple launch devices with encryption
snapshotIds: map[string]string{
sourceDeviceName: "snap-0123456789abcdef1",
"/dev/xvdg": "snap-0123456789abcdef2",
},
amiDevices: []*ec2.BlockDeviceMapping{},
launchDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
Encrypted: aws.Bool(true),
},
DeviceName: aws.String(sourceDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{
Encrypted: aws.Bool(true),
},
DeviceName: aws.String("/dev/xvdg"),
},
},
allDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef1"),
// Encrypted: true stripped from snapshotted devices
},
DeviceName: aws.String(rootDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef2"),
},
DeviceName: aws.String("/dev/xvdg"),
},
},
},
{
// Multiple launch devices and AMI devices with encryption
snapshotIds: map[string]string{
sourceDeviceName: "snap-0123456789abcdef1",
"/dev/xvdg": "snap-0123456789abcdef2",
},
amiDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
Encrypted: aws.Bool(true),
KmsKeyId: aws.String("keyId"),
},
// Source device name can be used in AMI devices
// since launch device of same name gets renamed
// to root device name
DeviceName: aws.String(sourceDeviceName),
},
},
launchDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
Encrypted: aws.Bool(true),
},
DeviceName: aws.String(sourceDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{
Encrypted: aws.Bool(true),
},
DeviceName: aws.String("/dev/xvdg"),
},
},
allDevices: []*ec2.BlockDeviceMapping{
{
Ebs: &ec2.EbsBlockDevice{
Encrypted: aws.Bool(true),
KmsKeyId: aws.String("keyId"),
},
DeviceName: aws.String(sourceDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef1"),
},
DeviceName: aws.String(rootDeviceName),
},
{
Ebs: &ec2.EbsBlockDevice{
SnapshotId: aws.String("snap-0123456789abcdef2"),
},
DeviceName: aws.String("/dev/xvdg"),
},
},
},
}
for _, tc := range cases {
stepRegisterAmi := newStepRegisterAMI(tc.amiDevices, tc.launchDevices)
allDevices := stepRegisterAmi.combineDevices(tc.snapshotIds)
if !reflect.DeepEqual(sorted(allDevices), sorted(tc.allDevices)) {
t.Fatalf("Unexpected output from combineDevices")
}
}
}

102
builder/amazon/ebssurrogate/step_snapshot_new_root.go
View File

@ -1,102 +0,0 @@
package ebssurrogate
import (
"errors"
"fmt"
"time"
"github.com/aws/aws-sdk-go/service/ec2"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/packer"
"github.com/mitchellh/multistep"
)
// StepSnapshotNewRootVolume creates a snapshot of the created volume.
//
// Produces:
// snapshot_id string - ID of the created snapshot
type StepSnapshotNewRootVolume struct {
NewRootMountPoint string
snapshotId string
}
func (s *StepSnapshotNewRootVolume) Run(state multistep.StateBag) multistep.StepAction {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ec2.Instance)
var newRootVolume string
for _, volume := range instance.BlockDeviceMappings {
if *volume.DeviceName == s.NewRootMountPoint {
newRootVolume = *volume.Ebs.VolumeId
}
}
ui.Say(fmt.Sprintf("Creating snapshot of EBS Volume %s...", newRootVolume))
description := fmt.Sprintf("Packer: %s", time.Now().String())
createSnapResp, err := ec2conn.CreateSnapshot(&ec2.CreateSnapshotInput{
VolumeId: &newRootVolume,
Description: &description,
})
if err != nil {
err := fmt.Errorf("Error creating snapshot: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
// Set the snapshot ID so we can delete it later
s.snapshotId = *createSnapResp.SnapshotId
ui.Message(fmt.Sprintf("Snapshot ID: %s", s.snapshotId))
// Wait for the snapshot to be ready
stateChange := awscommon.StateChangeConf{
Pending: []string{"pending"},
StepState: state,
Target: "completed",
Refresh: func() (interface{}, string, error) {
resp, err := ec2conn.DescribeSnapshots(&ec2.DescribeSnapshotsInput{SnapshotIds: []*string{&s.snapshotId}})
if err != nil {
return nil, "", err
}
if len(resp.Snapshots) == 0 {
return nil, "", errors.New("No snapshots found.")
}
s := resp.Snapshots[0]
return s, *s.State, nil
},
}
_, err = awscommon.WaitForState(&stateChange)
if err != nil {
err := fmt.Errorf("Error waiting for snapshot: %s", err)
state.Put("error", err)
ui.Error(err.Error())
return multistep.ActionHalt
}
state.Put("snapshot_id", s.snapshotId)
return multistep.ActionContinue
}
func (s *StepSnapshotNewRootVolume) Cleanup(state multistep.StateBag) {
if s.snapshotId == "" {
return
}
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if cancelled || halted {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
ui.Say("Removing snapshot since we cancelled or halted...")
_, err := ec2conn.DeleteSnapshot(&ec2.DeleteSnapshotInput{SnapshotId: &s.snapshotId})
if err != nil {
ui.Error(fmt.Sprintf("Error: %s", err))
}
}
}

107
builder/amazon/ebssurrogate/step_snapshot_volumes.go Normal file
View File

@ -0,0 +1,107 @@
package ebssurrogate
import (
"context"
"fmt"
"sync"
"time"
"github.com/aws/aws-sdk-go/service/ec2"
multierror "github.com/hashicorp/go-multierror"
awscommon "github.com/hashicorp/packer/builder/amazon/common"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
)
// StepSnapshotVolumes creates snapshots of the created volumes.
//
// Produces:
// snapshot_ids map[string]string - IDs of the created snapshots
type StepSnapshotVolumes struct {
LaunchDevices []*ec2.BlockDeviceMapping
snapshotIds map[string]string
}
func (s *StepSnapshotVolumes) snapshotVolume(ctx context.Context, deviceName string, state multistep.StateBag) error {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
instance := state.Get("instance").(*ec2.Instance)
var volumeId string
for _, volume := range instance.BlockDeviceMappings {
if *volume.DeviceName == deviceName {
volumeId = *volume.Ebs.VolumeId
}
}
if volumeId == "" {
return fmt.Errorf("Volume ID for device %s not found", deviceName)
}
ui.Say(fmt.Sprintf("Creating snapshot of EBS Volume %s...", volumeId))
description := fmt.Sprintf("Packer: %s", time.Now().String())
createSnapResp, err := ec2conn.CreateSnapshot(&ec2.CreateSnapshotInput{
VolumeId: &volumeId,
Description: &description,
})
if err != nil {
return err
}
// Set the snapshot ID so we can delete it later
s.snapshotIds[deviceName] = *createSnapResp.SnapshotId
// Wait for snapshot to be created
err = awscommon.WaitUntilSnapshotDone(ctx, ec2conn, *createSnapResp.SnapshotId)
return err
}
func (s *StepSnapshotVolumes) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
ui := state.Get("ui").(packer.Ui)
s.snapshotIds = map[string]string{}
var wg sync.WaitGroup
var errs *multierror.Error
for _, device := range s.LaunchDevices {
wg.Add(1)
go func(device *ec2.BlockDeviceMapping) {
defer wg.Done()
if err := s.snapshotVolume(ctx, *device.DeviceName, state); err != nil {
errs = multierror.Append(errs, err)
}
}(device)
}
wg.Wait()
if errs != nil {
state.Put("error", errs)
ui.Error(errs.Error())
return multistep.ActionHalt
}
state.Put("snapshot_ids", s.snapshotIds)
return multistep.ActionContinue
}
func (s *StepSnapshotVolumes) Cleanup(state multistep.StateBag) {
if len(s.snapshotIds) == 0 {
return
}
_, cancelled := state.GetOk(multistep.StateCancelled)
_, halted := state.GetOk(multistep.StateHalted)
if cancelled || halted {
ec2conn := state.Get("ec2").(*ec2.EC2)
ui := state.Get("ui").(packer.Ui)
ui.Say("Removing snapshots since we cancelled or halted...")
for _, snapshotId := range s.snapshotIds {
_, err := ec2conn.DeleteSnapshot(&ec2.DeleteSnapshotInput{SnapshotId: &snapshotId})
if err != nil {
ui.Error(fmt.Sprintf("Error: %s", err))
}
}
}
}

2
builder/amazon/ebsvolume/block_device.go
View File

@ -7,7 +7,7 @@ import (
type BlockDevice struct {
awscommon.BlockDevice `mapstructure:"-,squash"`
Tags map[string]string `mapstructure:"tags"`
Tags awscommon.TagMap `mapstructure:"tags"`
}
func commonBlockDevices(mappings []BlockDevice, ctx *interpolate.Context) (awscommon.BlockDevices, error) {

97
builder/amazon/ebsvolume/builder.go
View File

@ -11,9 +11,9 @@ import (
"github.com/hashicorp/packer/common"
"github.com/hashicorp/packer/helper/communicator"
"github.com/hashicorp/packer/helper/config"
"github.com/hashicorp/packer/helper/multistep"
"github.com/hashicorp/packer/packer"
"github.com/hashicorp/packer/template/interpolate"
"github.com/mitchellh/multistep"
)
const BuilderId = "mitchellh.amazon.ebsvolume"
@ -44,6 +44,7 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
InterpolateFilter: &interpolate.RenderFilter{
Exclude: []string{
"run_tags",
"spot_tags",
"ebs_volumes",
},
},
@ -56,17 +57,31 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
var errs *packer.MultiError
errs = packer.MultiErrorAppend(errs, b.config.AccessConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.RunConfig.Prepare(&b.config.ctx)...)
errs = packer.MultiErrorAppend(errs, b.config.launchBlockDevices.Prepare(&b.config.ctx)...)
for _, d := range b.config.VolumeMappings {
if err := d.Prepare(&b.config.ctx); err != nil {
errs = packer.MultiErrorAppend(errs, fmt.Errorf("AMIMapping: %s", err.Error()))
}
}
b.config.launchBlockDevices, err = commonBlockDevices(b.config.VolumeMappings, &b.config.ctx)
if err != nil {
errs = packer.MultiErrorAppend(errs, err)
}
if b.config.IsSpotInstance() && (b.config.AMIENASupport || b.config.AMISriovNetSupport) {
errs = packer.MultiErrorAppend(errs,
fmt.Errorf("Spot instances do not support modification, which is required "+
"when either `ena_support` or `sriov_support` are set. Please ensure "+
"you use an AMI that already has either SR-IOV or ENA enabled."))
}
if errs != nil && len(errs.Errors) > 0 {
return nil, errs
}
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey))
log.Println(common.ScrubConfig(b.config, b.config.AccessKey, b.config.SecretKey, b.config.Token))
return nil, nil
}
@ -103,45 +118,46 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
var instanceStep multistep.Step
isSpotInstance := b.config.SpotPrice != "" && b.config.SpotPrice != "0"
if isSpotInstance {
if b.config.IsSpotInstance() {
instanceStep = &awscommon.StepRunSpotInstance{
Debug: b.config.PackerDebug,
ExpectedRootDevice: "ebs",
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
InstanceType: b.config.InstanceType,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
SourceAMI: b.config.SourceAmi,
IamInstanceProfile: b.config.IamInstanceProfile,
SubnetId: b.config.SubnetId,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
EbsOptimized: b.config.EbsOptimized,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.launchBlockDevices,
Tags: b.config.RunTags,
Ctx: b.config.ctx,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.launchBlockDevices,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
SourceAMI: b.config.SourceAmi,
SpotPrice: b.config.SpotPrice,
SpotPriceProduct: b.config.SpotPriceAutoProduct,
SpotTags: b.config.SpotTags,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
}
} else {
instanceStep = &awscommon.StepRunSourceInstance{
Debug: b.config.PackerDebug,
ExpectedRootDevice: "ebs",
InstanceType: b.config.InstanceType,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
SourceAMI: b.config.SourceAmi,
IamInstanceProfile: b.config.IamInstanceProfile,
SubnetId: b.config.SubnetId,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
EbsOptimized: b.config.EbsOptimized,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.launchBlockDevices,
Tags: b.config.RunTags,
Ctx: b.config.ctx,
AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
AvailabilityZone: b.config.AvailabilityZone,
BlockDevices: b.config.launchBlockDevices,
Ctx: b.config.ctx,
Debug: b.config.PackerDebug,
EbsOptimized: b.config.EbsOptimized,
EnableT2Unlimited: b.config.EnableT2Unlimited,
ExpectedRootDevice: "ebs",
IamInstanceProfile: b.config.IamInstanceProfile,
InstanceInitiatedShutdownBehavior: b.config.InstanceInitiatedShutdownBehavior,
InstanceType: b.config.InstanceType,
IsRestricted: b.config.IsChinaCloud() || b.config.IsGovCloud(),
SourceAMI: b.config.SourceAmi,
SubnetId: b.config.SubnetId,
Tags: b.config.RunTags,
UserData: b.config.UserData,
UserDataFile: b.config.UserDataFile,
}
}
@ -173,15 +189,16 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
Ctx: b.config.ctx,
},
&awscommon.StepGetPassword{
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
Debug: b.config.PackerDebug,
Comm: &b.config.RunConfig.Comm,
Timeout: b.config.WindowsPasswordTimeout,
BuildName: b.config.PackerBuildName,
},
&communicator.StepConnect{
Config: &b.config.RunConfig.Comm,
Host: awscommon.SSHHost(
ec2conn,
b.config.SSHPrivateIp),
b.config.SSHInterface),
SSHConfig: awscommon.SSHConfig(
b.config.RunConfig.Comm.SSHAgentAuth,
b.config.RunConfig.Comm.SSHUsername,
@ -189,7 +206,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
},
&common.StepProvision{},
&awscommon.StepStopEBSBackedInstance{
Skip: isSpotInstance,
Skip: b.config.IsSpotInstance(),
DisableStopInstance: b.config.DisableStopInstance,
},
&awscommon.StepModifyEBSBackedInstance{

Some files were not shown because too many files have changed in this diff Show More