Commit Graph

241 Commits

Author SHA1 Message Date
Rickard von Essen ba9cae5078 builder/amazon: Add SourceAMI and BuildRegion template
Added {{ .SourceAMI }} and {{ .BuildRegion }} template values availible
in `ami_description`, `run_tags`, `run_volume_tags`, `tags`, and
`snapshot_tags`.
2017-01-21 20:39:58 +01:00
Matthew Hooker fc30f6cc0a
log tag creation.
closes #4427
2017-01-19 12:00:41 -08:00
Matthew Hooker f5bf588393
tag all volumes 2017-01-18 11:34:46 -08:00
Matthew Hooker 800351b7ca
fix panic and don't print each tag we add. 2017-01-18 11:34:36 -08:00
Roman Zhuzha ae016a1f25 simplify code flow: skip the step if no run_volume_tags provided 2017-01-18 13:03:05 +01:00
Roman Zhuzha 36d8a1df1d use common.ConvertToEC2Tags to apply tags to source EBS volumes, change Say message 2017-01-18 12:52:59 +01:00
Matthew Hooker c1f5545986
compress pr #4395 2017-01-13 11:42:20 -08:00
Richard Moore 9435fa62e3 Output error causing ebs volume deletion failure
When an ebs volume fails to delete, Packer reports that there
was an error deleting the volume and the volume id. But it doesn't
give you the details of what that error is. This commit adds the
error reported back to the standard output.
2017-01-13 15:33:47 +00:00
Rickard von Essen 4b1e014be8
builder/amazon: Crashes when multiple EBS vols are used
If you use a new EBS vol (which hasn't any snapshot) Packer crashes when
creating the AMI.

Closes #4303
2016-12-21 15:37:08 +01:00
Matthew Hooker e4f6eaae9b Merge pull request #4231 from artburkart/force-delete-snapshot-patch
Fixes acceptance test for PR #4223
2016-12-17 11:43:26 -08:00
Rickard von Essen d3c2239b9e
builder/amazon: Change shutdown_behaviour to shutdown_behavior 2016-12-14 21:59:16 +01:00
Rickard von Essen 4ce3b8712a
Replace unencrypted EBS snapshots with encrypted
To ensure that groups and users attributes are added to the encrypted
snapshots.
2016-12-06 03:31:09 +01:00
Rickard von Essen c3a352955e amazon: Added snapshot_users and snapshot_groups 2016-12-04 16:03:01 +01:00
Ari Aviran 46f217f255 amazon - Snapshot permissions correctly applied
Fixes #3344
2016-12-04 16:03:01 +01:00
Rickard von Essen d7a7d8c852
It's not possible to share encrypted AMI's 2016-12-04 15:52:23 +01:00
poida 3eed6fd508 Added KMS CMK support to EBS builder
Added the 'kms_key_id' parameter. This supports supplying a customer master key (CMK) when encrypting the EBS volume.

The parameter is optional and only takes effect when 'encrypted' is true. When 'encrypted' is true but 'kms_key_id' is missing the 'aws/ebs' key will be used.
2016-12-03 07:49:34 +11:00
Arthur Burkart 576f6f1b39 Fixes acceptance test
The original acceptance test required a manual check. This one does
not.
2016-11-30 22:58:19 -05:00
Matthew Hooker 832de2104b
Revert "Merge pull request #4230 from mitchellh/revert-4223-closes_3320"
This reverts commit bda89b0c42, reversing
changes made to f9f9d82419.
2016-11-30 13:28:34 -08:00
Matthew Hooker f5c3c25d2b Revert "Adds `force_delete_snapshot` flag" 2016-11-30 13:10:22 -08:00
Matthew Hooker f9f9d82419 Merge pull request #4223 from artburkart/closes_3320
Adds `force_delete_snapshot` flag
2016-11-30 12:53:34 -08:00
Arthur Burkart 5b59f56cdb Adds `force_delete_snapshot` flag
This PR adds the ability for Packer to clean up snapshots in addition to
deregistering AMIs at build time.

To test this, I used the following `test.json` file:

```json
{
  "builders": [
    {
      "type": "amazon-ebs",
      "region": "us-east-1",
      "source_ami": "ami-fce3c696",
      "ami_name": "packer-test",
      "instance_type": "m3.medium",
      "ssh_username": "ubuntu",
      "vpc_id": "some-vpc-id",
      "subnet_id": "some-subnet-routed-through-igw",
      "security_group_id": "some-security-group-with-port-22-access",
      "force_delete_snapshot": true
    }
  ],
  "provisioners": [
    {
      "type": "shell-local",
      "command": "echo 'hello'"
    }
  ]
}

```

I appreciate any constructive feedbakc that can be given. Cheers!
2016-11-29 23:39:36 -05:00
Rickard von Essen 99ee610afa
builder/amazon: Fix SSH Agent
Closes #4220
2016-11-29 22:33:53 +01:00
Arthur Burkart 0c7e73b1cf
Implements Snapshot tagging
While implementing my acceptance test, I stumbled upon a comment stating
that snapshot deletion should also be implemented, so I snuck that in. I
can't help but wonder if there is some generic logic that is implemented
a few times throughout the packer code base that could maybe better serve
us if it were abstracted to the common package.
2016-11-26 17:48:26 +01:00
Arthur Burkart 2e65867cba Closes #3908: Adds snapshot tag overrides
This commit adds the ability to configure unique tags on snapshots
that are separate from the tags defined on the AMI. Anything applied
to the AMI will also be applied to the snapshots, but `snapshot_tags`
will override and append tags to the tags already applied to the snapshots
2016-11-15 23:16:27 -05:00
Matthew Hooker 60c459d6c8 Merge pull request #4098 from mitchellh/newsession
aws-sdk: use session.NewSession
2016-11-02 12:25:14 -07:00
James Nugent b1ff8c3bfc builder/amazon: Add `ebs-volume` builder
This commit adds a builder that works like EBS builders, except does not
create an AMI, and instead is intended to create EBS volumes in an
initialized state. For example, the following template can be used to
create and export a set of 3 EBS Volumes in a ZFS zpool named `data` for
importing by instances running production systems:

```
{
	"variables": {
		"aws_access_key_id": "{{ env `AWS_ACCESS_KEY_ID` }}",
		"aws_secret_access_key": "{{ env `AWS_SECRET_ACCESS_KEY` }}",
		"region": "{{ env `AWS_REGION` }}",
		"source_ami": "{{ env `PACKER_SOURCE_AMI` }}",
		"vpc_id": "{{ env `PACKER_VPC_ID` }}",
		"subnet_id": "{{ env `PACKER_SUBNET_ID` }}"
	},
	"builders": [{
		"type": "amazon-ebs-volume",
		"access_key": "{{ user `aws_access_key_id` }}",
		"secret_key": "{{ user `aws_secret_access_key` }}",
		"region": "{{user `region`}}",
		"spot_price_auto_product": "Linux/UNIX (Amazon VPC)",

		"ssh_pty": true,
		"instance_type": "t2.medium",
		"vpc_id": "{{user `vpc_id` }}",
		"subnet_id": "{{user `subnet_id` }}",
		"associate_public_ip_address": true,
		"source_ami": "{{user `source_ami` }}",
		"ssh_username": "ubuntu",
		"ssh_timeout": "5m",

		"ebs_volumes": [
			{
				"device_name": "/dev/xvdf",
				"delete_on_termination": false,
				"volume_size": 10,
				"volume_type": "gp2",
				"tags": {
					"Name": "TeamCity-Data1",
					"zpool": "data",
					"Component": "TeamCity"
				}
			},
			{
				"device_name": "/dev/xvdg",
				"delete_on_termination": false,
				"volume_size": 10,
				"volume_type": "gp2",
				"tags": {
					"Name": "TeamCity-Data2",
					"zpool": "data",
					"Component": "TeamCity"
				}
			},
			{
				"device_name": "/dev/xvdh",
				"delete_on_termination": false,
				"volume_size": 10,
				"volume_type": "gp2",
				"tags": {
					"Name": "TeamCity-Data3",
					"zpool": "data",
					"Component": "TeamCity"
				}
			}
		]
	}],
	"provisioners": [
	{
		"type": "shell",
		"start_retry_timeout": "10m",
		"inline": [
			"DEBIAN_FRONTEND=noninteractive sudo apt-get update",
			"DEBIAN_FRONTEND=noninteractive sudo apt-get install -y zfs",
			"lsblk",
			"sudo parted /dev/xvdf --script mklabel GPT",
			"sudo parted /dev/xvdg --script mklabel GPT",
			"sudo parted /dev/xvdh --script mklabel GPT",
			"sudo zpool create -m none data raidz xvdf xvdg xvdh",
			"sudo zpool status",
			"sudo zpool export data",
			"sudo zpool status"
		]
	}
	]
}
```

StepModifyInstance and StepStopInstance are now shared between EBS and
EBS-Volume builders - move them into the AWS common directory and rename
them to indicate that they only apply to EBS-backed builders.
2016-11-02 12:56:39 -04:00
Matthew Hooker 970b37077e Merge pull request #4050 from jen20/ssh-agent
builder/amazon: Allow use of local SSH Agent
2016-11-01 17:27:37 -07:00
Matthew Hooker 48bdae93d1 aws-sdk: use session.NewSession 2016-11-01 15:53:04 -07:00
Matthew Hooker d920b3fbf4 run gofmt 2016-11-01 14:08:04 -07:00
Rickard von Essen d16d5d9686 Merge pull request #3817 from ChrisLundquist/dynamic-source-ami
Dynamic source ami
2016-10-25 21:24:41 +02:00
Richard Owen adddc60a1d Fix typo in encrypted ami step log message 2016-10-25 16:01:01 +01:00
James Nugent 7425fef2c7 builder/amazon: Allow use of local SSH Agent
This commit adds an option to use the local SSH Agent to authenticate
connections to source instances started by the the EBS and Instance
Store builders.

This is of use when the source AMI _already_ has configuration for
authorized SSH keys - for example if one uses an SSH certificate
authority.

A further extension (not implemented in this commit) is to allow SSH
agent use with a pre-defined key pair, in order to allow keys with
passphrases to be used without giving the passphrase to Packer.
2016-10-23 21:43:47 -05:00
Rickard von Essen c273bd6beb Merge pull request #3789 from jeremy-asher/amazon-ebs-delete-volumes-on-failure
clean up EBS volumes on failure
2016-10-15 14:57:15 +02:00
Rickard von Essen 0be3034e82 Merge pull request #3953 from rickard-von-essen/amz-ssh
builder/amazon: Allow using ssh_private_key_file and ssh_password
2016-10-03 10:22:15 +02:00
Rickard von Essen acc2689b49
builder/amazon: Allow using ssh_private_key_file and ssh_password
This adds support for using amazon-ebs/amazon-instance builder without
a keypair. If a ssh_private_key_file is supplied without a
ssh_keypair_name no temporary ssh keypair is created. If ssh_password is
used no temporary ssh keypair is created and the password is used when
trying to connect.

Closes #2301
Closes #3156
2016-10-02 22:20:36 +02:00
Rickard von Essen cffc8e892c Merge pull request #3382 from ahamidi/encrypted-boot-volume
Create AMI with encrypted boot volume
2016-10-02 01:02:43 +02:00
Chris Lundquist 1b4895c684 Merge remote-tracking branch 'origin/master' into dynamic-source-ami 2016-10-01 22:20:17 +00:00
Orivej Desh 6762965696 Add -on-error command line argument to allow preserving artifacts on builder errors
Resolves #409
2016-09-16 12:15:00 +00:00
Chris Lundquist defdd1ecf3 Merge remote-tracking branch 'origin/master' into dynamic-source-ami 2016-09-02 01:43:03 +00:00
Krzysztof Wilczynski d01fb65dcf Fix. Use temporary_key_pair_name when specified. (#3739)
Fix: Use temporary_key_pair_name when specified.

This is to fix an issue where the temporary_key_pair_name configuration option
for amazon-ebs and amazon-instance builders would be ignored and replaced with
an automatically generated value using UUID, even when the option was explicitly
specified.

Remove space from the auto-generated SSH key-pair name.

Resolves #3736
2016-08-25 09:17:57 +02:00
Chris Lundquist 95afaa58ca [dynamic-source-ami] rename to source_ami_filter 2016-08-20 23:34:22 +00:00
Chris Lundquist ec1abb9448 [dynamic-source-ami] proof of concept 2016-08-20 18:58:36 +00:00
Jeremy Asher 6130b8588d remove check for AMIs when deleting vols
This removes the unnecessary check for AMIs before cleaning up volumes.
AMIs will exist on a successful run, but not when a build fails after
instance creation (during provisioning for example).  The following
check for instance being nil should be sufficient.
2016-08-08 14:16:43 -07:00
Ali Hamidi 1cecda639a add encrypted boot volume acceptance test 2016-07-29 15:01:24 -07:00
Ali Hamidi 902ae375b3 delete unencrypted snapshot 2016-07-29 15:01:24 -07:00
Ali Hamidi 53b3867c50 allow packer to create an encrypted copy of the AMI 2016-07-29 15:01:24 -07:00
Patrick Robinson b1d6d28a90 Add unit tests for shutdown behaviour 2016-06-14 14:29:10 -07:00
Patrick Robinson a9f914ea97 Add shutdown_behaviour option
This means an instance will terminate upon shutdown.
2016-06-14 14:29:10 -07:00
Chris Chalfant 601b833aaa style and documentation changes per PR review comments 2016-03-15 08:01:20 -04:00
Chris Chalfant 8163e16e02 gofmt code 2016-03-14 13:49:42 -04:00
Chris Chalfant 72a7123a0b Add option to tell packer not to stop the instance
It is sometimes desirable to sysprep a windows machine before creating
an EC2 image. The AWS-approved way to do this is to run
ec2configservice.exe -sysprep and let ec2configservice shut down the instance.
This change adds an option to disable the stop instance call issued by packer
so that the user can control when the machine is stopped.
2016-03-14 12:54:03 -04:00
Chris Bednarski 1a5d5acea4 Merge pull request #2737 from jen20/ebs-volume-tags
Tag EBS volumes with volume_tags in EBS builder
2016-01-21 13:29:49 -08:00
Chris Bednarski d6fad30162 Merge pull request #2955 from yayugu/feature/ami_product_codes
Add AMI Product Codes modification for amazon-ebs & amazon-chroot
2016-01-20 09:13:09 -08:00
Colin Hebert 48d242a8f8 Use pointers in the AWS API 2016-01-06 08:12:20 +01:00
Colin Hebert 9385a277c6 Fix API calls and remove az detection in chroot 2016-01-06 08:02:29 +01:00
Colin Hebert e4930e7b57 Merge branch 'master' into subnet_az_discovery 2016-01-06 07:43:03 +01:00
Yuya YAGUCHI 9c8405b2c6 Add AMI Product Codes modification for amazon-ebs & amazon-chroot 2015-11-19 18:28:04 +09:00
Mark Peek ca19688316 aws: fix test breakage due to upstream breaking change #2891 2015-10-31 11:04:50 -07:00
Luke Amdor 8682dec178 aws: build after upstream breaking change
see 1a69d06935
2015-10-30 13:58:56 -05:00
Mark Peek 4e1e9b788a Merge pull request #2845 from abridgett/feature/spot_price_zero
spot_price of "0" to mean on demand
2015-10-26 21:51:44 -07:00
Adrian Bridgett 3257b26fa7 one more place to check SpotPrice 2015-10-26 10:20:49 +00:00
Adrian Bridgett 8349e22df9 Add ebs_optimized support 2015-10-06 21:36:21 +01:00
James Nugent 632d38a1c5 Tag EBS volumes with volume_tags in EBS builder
This change allows specification of a new "volume_tags" array in the
amazon-ebs builder in order to allow tagging of volumes used on the
source instance. It is implemented as a new step which is skipped if
there are no tags specified.
2015-09-11 16:39:34 -04:00
Chris Bednarski 1a775c05d9 Update calls to amazon to match the upstream
- see http://aws.amazon.com/releasenotes/2948141298714307
- run awsmigrate-renamer on each amazon module (chroot, instance, etc.)
2015-08-17 17:44:01 -07:00
Chris Bednarski 28bf1877c2 Updated AWS SDK calls to match the 0.7.0 release of the AWS SDK 2015-07-28 17:10:21 -07:00
Mitchell Hashimoto b315e470c8 builder/amazon: set debug flag for get password 2015-06-29 09:40:58 -07:00
Tom Cahill 016d916d83 Fix adding groups to AMI launch permissions.
Previously, groups were granted launch permissions by submitting a
ModifyImageAttribute request with the UserGroups parameter set
appropriately. This is no longer valid, as the LaunchPermission
parameter must be set.
2015-06-26 10:52:27 -07:00
Clint Shryock 7af5d5419a builder/amazon: Fix issue with sharing AMIs with ami_users 2015-06-24 11:41:58 -05:00
Clint 6579219013 Merge pull request #2282 from mitchellh/f-amazon-cleanup-volumes
builder/amazon-ebs: Clean up orphan volumes
2015-06-22 12:56:26 -05:00
Mitchell Hashimoto 84189f7a28 builder/*: properly save interpolation context 2015-06-22 09:22:42 -07:00
Clint Shryock dff6cf1a83 code tweak after review 2015-06-22 10:48:54 -05:00
Clint Shryock c9714ce69e builder/amazon-ebs: Clean up orphan volumes
Fixes #1783
2015-06-19 10:22:39 -05:00
Mitchell Hashimoto 5c290fdbc7 builder/amazon use proper key pair name 2015-06-19 06:15:16 +02:00
Mitchell Hashimoto de2004c1c9 Merge branch 'master' of https://github.com/nyetsche/packer into nyetsche-master 2015-06-15 15:21:58 -07:00
Clint 2372393f64 Merge pull request #2212 from mitchellh/f-amazon-add-tags-to-snapshots
Add EBS snapshot tags #846 #1219
2015-06-15 10:05:57 -05:00
Clint 8fca582278 Merge pull request #2221 from mitchellh/f-amazon-force-deregister-artifacts
builder/amazon: Add force_deregister option
2015-06-15 10:02:30 -05:00
Mitchell Hashimoto 8f6ecfd9e3 builder/amazon: various fixes (minor) to get things going 2015-06-13 23:12:59 -07:00
Mitchell Hashimoto fd4e0e9da4 builder/amazon: StepGetPassword 2015-06-13 22:35:45 -07:00
Mitchell Hashimoto 115d583cff helper/communicator: make host more generic 2015-06-13 19:23:33 -04:00
Mitchell Hashimoto e557928119 builder/amazon: use helper/communicator 2015-06-13 18:16:12 -04:00
Clint Shryock bec59b535d builder/amazon: Add force_deregister option, to automatically deregister artifacts with name conflicts 2015-06-12 13:23:47 -05:00
Clint Shryock 86206e316d add tags test 2015-06-12 10:39:37 -05:00
Mitchell Hashimoto be4a82dfae amazon/*: fix some merge conflicts 2015-06-10 10:33:01 -07:00
Mitchell Hashimoto 8e1db0d94c Merge branch 'fix_amazon_instance_waits' of https://github.com/renat-sabitov/packer into renat-sabitov-fix_amazon_instance_waits 2015-06-10 10:31:21 -07:00
Clint 455d57a87f Merge pull request #2187 from mitchellh/f-aws-add-prevalidate-step
builder/aws: Add pre validate step, to validate things before building.
2015-06-08 22:02:36 -05:00
Clint Shryock 231f01cd35 builder/aws: Add pre validate step, to validate things before building. 2015-06-08 17:08:39 -05:00
Mitchell Hashimoto f7e49d92ac Merge pull request #2178 from henrysher/fix-ami-copy-name
"Name" parameter required for copying ami across regions [GH-2172]
2015-06-08 09:16:17 -07:00
Mitchell Hashimoto 769b7d20b9 amazon/ebs: acceptance test for region copy 2015-06-08 09:16:01 -07:00
Henry Huang 052b5e6699 "Name" parameter required for copying ami across regions [GH-2172] 2015-06-05 11:15:48 +00:00
Seth Vargo 33ca8b7fb5 Migrate to new AWS repo 2015-06-03 17:13:52 -04:00
Mitchell Hashimoto b78b119a11 amazon/*: fix merge issues with lib switch 2015-05-28 08:31:22 -07:00
Mitchell Hashimoto 44b980e659 Merge branch 'aws-sdk-go' of https://github.com/jen20/packer into jen20-aws-sdk-go 2015-05-28 08:24:41 -07:00
Mitchell Hashimoto 034e4e676c amazon/*: use new interpolation functions 2015-05-27 11:47:45 -07:00
Mitchell Hashimoto 50d7c598e9 amazon/ebs: use new interpolation stuff 2015-05-27 11:35:56 -07:00
Mitchell Hashimoto fea8bb3566 amazon/ebs: basic acceptance test 2015-05-26 13:47:24 -07:00
James Nugent e99cd56b6c Migrate from mitchellh/goamz to awslabs/aws-sdk-go
This commit moves the Amazon builders of Packer away from the Hashicorp
fork of the goamz library to the official AWS SDK for Go, in order that
third party plugins may depend on the more complete official library
more easily.
2015-04-12 21:33:09 -04:00
renat-sabitov-sirca d046e1cc5b Fixing transient AWS errors during EBS builds
Relates to #1539

AWS is eventually consistent and instance can be not visibile for
some time after creation. This fix eliminates describe-instances
call before going to the proper wait loop
2015-03-11 10:34:21 +11:00
Colin Hebert a08725f7c5 Add auto discovery of AZ based on the subnet 2015-03-01 00:00:45 +11:00
Emil Hessman 505f51ff27 builder/amazon/ebs: fix incorrect printf verb type
Fixes the following vet report:

builder/amazon/ebs/step_create_ami.go:90: arg resp.Return for printf verb %s of wrong type: bool
2015-02-25 05:29:53 +01:00
Lesko, Matthew (NIH/NLM/NCBI) [C] 43f08b2664 go fmt all the things 2015-01-13 22:58:41 -05:00