Commit Graph

2127 Commits

Author SHA1 Message Date
Matthew Hooker 5d0892bd37 Merge pull request #4101 from lwander/f-googlecompute-ssh-private-key-file
builder/googlecompute: Use ssh_private_key_file if provided
2016-11-03 11:04:02 -07:00
Lars Wander 583e341ae7 builder/googlecompute: Store empty public_key 2016-11-03 12:49:09 -04:00
Lars Wander 7f474d8f78 builder/googlecompute: Tests added 2016-11-03 12:45:52 -04:00
Scott Crunkleton 85210afe92 googlecompute: conditionally omit the wait for startup script step if no startup script is provided. 2016-11-02 15:20:21 -07:00
Lars Wander d70e783455 builder/googlecompute: Use ssh_private_key_file if provided
This seemed to be missing from the googlecompute provider. Now if the
ssh_private_key_file is provided, that will be used in place of a temporary
key. I didn't update the googlecompute specific docs under `./website/`, since
this parameter is already documented under the communicators templates page.
2016-11-02 17:04:34 -04:00
Matthew Hooker f3f5f89fe6 Merge pull request #4096 from mitchellh/filterday
builder/amazon-ebs: show ami id found from filter
2016-11-02 12:25:46 -07:00
Matthew Hooker 60c459d6c8 Merge pull request #4098 from mitchellh/newsession
aws-sdk: use session.NewSession
2016-11-02 12:25:14 -07:00
Matthew Hooker 9bc23ea512 Merge pull request #4099 from mitchellh/waitfail
Waitfail
2016-11-02 12:21:51 -07:00
James Nugent 36c09af992 builder/amazon: Add tests for the `ebs-volume` builder
These smoke tests are effectively a mirror of the EBS builder tests.
2016-11-02 12:56:39 -04:00
James Nugent b1ff8c3bfc builder/amazon: Add `ebs-volume` builder
This commit adds a builder that works like EBS builders, except does not
create an AMI, and instead is intended to create EBS volumes in an
initialized state. For example, the following template can be used to
create and export a set of 3 EBS Volumes in a ZFS zpool named `data` for
importing by instances running production systems:

```
{
	"variables": {
		"aws_access_key_id": "{{ env `AWS_ACCESS_KEY_ID` }}",
		"aws_secret_access_key": "{{ env `AWS_SECRET_ACCESS_KEY` }}",
		"region": "{{ env `AWS_REGION` }}",
		"source_ami": "{{ env `PACKER_SOURCE_AMI` }}",
		"vpc_id": "{{ env `PACKER_VPC_ID` }}",
		"subnet_id": "{{ env `PACKER_SUBNET_ID` }}"
	},
	"builders": [{
		"type": "amazon-ebs-volume",
		"access_key": "{{ user `aws_access_key_id` }}",
		"secret_key": "{{ user `aws_secret_access_key` }}",
		"region": "{{user `region`}}",
		"spot_price_auto_product": "Linux/UNIX (Amazon VPC)",

		"ssh_pty": true,
		"instance_type": "t2.medium",
		"vpc_id": "{{user `vpc_id` }}",
		"subnet_id": "{{user `subnet_id` }}",
		"associate_public_ip_address": true,
		"source_ami": "{{user `source_ami` }}",
		"ssh_username": "ubuntu",
		"ssh_timeout": "5m",

		"ebs_volumes": [
			{
				"device_name": "/dev/xvdf",
				"delete_on_termination": false,
				"volume_size": 10,
				"volume_type": "gp2",
				"tags": {
					"Name": "TeamCity-Data1",
					"zpool": "data",
					"Component": "TeamCity"
				}
			},
			{
				"device_name": "/dev/xvdg",
				"delete_on_termination": false,
				"volume_size": 10,
				"volume_type": "gp2",
				"tags": {
					"Name": "TeamCity-Data2",
					"zpool": "data",
					"Component": "TeamCity"
				}
			},
			{
				"device_name": "/dev/xvdh",
				"delete_on_termination": false,
				"volume_size": 10,
				"volume_type": "gp2",
				"tags": {
					"Name": "TeamCity-Data3",
					"zpool": "data",
					"Component": "TeamCity"
				}
			}
		]
	}],
	"provisioners": [
	{
		"type": "shell",
		"start_retry_timeout": "10m",
		"inline": [
			"DEBIAN_FRONTEND=noninteractive sudo apt-get update",
			"DEBIAN_FRONTEND=noninteractive sudo apt-get install -y zfs",
			"lsblk",
			"sudo parted /dev/xvdf --script mklabel GPT",
			"sudo parted /dev/xvdg --script mklabel GPT",
			"sudo parted /dev/xvdh --script mklabel GPT",
			"sudo zpool create -m none data raidz xvdf xvdg xvdh",
			"sudo zpool status",
			"sudo zpool export data",
			"sudo zpool status"
		]
	}
	]
}
```

StepModifyInstance and StepStopInstance are now shared between EBS and
EBS-Volume builders - move them into the AWS common directory and rename
them to indicate that they only apply to EBS-backed builders.
2016-11-02 12:56:39 -04:00
Matthew Hooker d47e47cf1f rewrite to not use private aws package 2016-11-01 18:29:18 -07:00
Matthew Hooker c4c46c349e implement security group waiter 2016-11-01 18:10:42 -07:00
Matthew Hooker 970b37077e Merge pull request #4050 from jen20/ssh-agent
builder/amazon: Allow use of local SSH Agent
2016-11-01 17:27:37 -07:00
Matthew Hooker 48bdae93d1 aws-sdk: use session.NewSession 2016-11-01 15:53:04 -07:00
Matthew Hooker 83b57b98ac show ami id found from filter 2016-11-01 15:26:43 -07:00
Matthew Hooker d920b3fbf4 run gofmt 2016-11-01 14:08:04 -07:00
Matthew Hooker 0dd7d7ff3b fix error message 2016-10-31 16:44:18 -07:00
Ganesh kumar Sankaran b9c6139d67 AWS async operations sometimes takes long times, if there are multiple parallel builds, polling at 2 second frequency will exceed the request limit. Allow 2 seconds to be overwritten with AWS_POLL_DELAY_SECONDS 2016-10-31 16:44:18 -07:00
Matthew Hooker 01be917450 Merge pull request #3660 from StackPointCloud/packer-builder-profitbricks
Packer Builder ProfitBricks
2016-10-31 15:45:24 -07:00
Matthew Hooker 93f0436766 Merge pull request #3909 from svanharmelen/f-cloudstack-builder
Add a CloudStack builder
2016-10-31 13:34:20 -07:00
Matthew Hooker f69ee4634f Merge pull request #4041 from mitchellh/vmwareshutdown
VIP: VMware shutdown fails
2016-10-31 11:37:00 -07:00
Rickard von Essen d66490ebfd
Properly cleanup AWS temporary key pairs
Closes #4057 - Amazon key pair no longer cleaned up at end of build
2016-10-27 23:06:13 +02:00
Rickard von Essen d16d5d9686 Merge pull request #3817 from ChrisLundquist/dynamic-source-ami
Dynamic source ami
2016-10-25 21:24:41 +02:00
Richard Owen adddc60a1d Fix typo in encrypted ami step log message 2016-10-25 16:01:01 +01:00
James Nugent 395d88941c amazon: Allow SSH Agent auth for existing key pair
This commit allows SSH Agent authentication to be used with an existing
key pair defined in AWS.
2016-10-23 22:26:14 -05:00
James Nugent 7425fef2c7 builder/amazon: Allow use of local SSH Agent
This commit adds an option to use the local SSH Agent to authenticate
connections to source instances started by the the EBS and Instance
Store builders.

This is of use when the source AMI _already_ has configuration for
authorized SSH keys - for example if one uses an SSH certificate
authority.

A further extension (not implemented in this commit) is to allow SSH
agent use with a pre-defined key pair, in order to allow keys with
passphrases to be used without giving the passphrase to Packer.
2016-10-23 21:43:47 -05:00
Matthew Hooker d6bed79429 wait for shutdown command to end so we get stderr/out 2016-10-22 10:33:27 -07:00
YAMADA Tsuyoshi da7d317e46 googlecompute: Run instance with custom scopes 2016-10-22 15:46:14 +09:00
Matthew Hooker e9a5d05a2f builder/vmware: Ignore shutdown errors
Resolves issue where `shutdown_command` would error because the remote
side disconnected, which we should have expected and ignored.
2016-10-21 22:41:45 -07:00
Matthew Hooker f9f47c5bbb fix shell disconnect error when shutting down vmware 2016-10-21 22:26:34 -07:00
Asif R Choudhury 48d94342ba Add us-east-2 (Ohio) AWS region 2016-10-17 21:31:41 -05:00
Arthur Burkart 4a00ffeea2 Closes #3999 - Support VirtualBox Trim/Discard
Adds a couple options to enable/disable the `--nonrotational` and
`--discard` flags during `VBoxManage storageattach` command execution
2016-10-17 08:52:28 -04:00
Sander van Harmelen dbf3bf56d4 Add a CloudStack Builder 2016-10-16 15:29:18 +02:00
Rickard von Essen c273bd6beb Merge pull request #3789 from jeremy-asher/amazon-ebs-delete-volumes-on-failure
clean up EBS volumes on failure
2016-10-15 14:57:15 +02:00
Christopher Boumenot f908e18483 Merge pull request #3996 from boumenot/pr-issue-3968
azure: Allow user to set custom data
2016-10-13 13:39:54 -07:00
Christopher Boumenot 7d07fa7afc azure: Allow user to set custom data 2016-10-13 11:56:23 -07:00
Christopher Boumenot bba643df9a azure: handle os_type errors more gracefully 2016-10-13 09:26:29 -07:00
Christopher Boumenot 249a943a1e azure: Allow user to override size of OS disk 2016-10-12 16:24:04 -07:00
jasminSPC 5da380baed Error handling
Minor typo fixes
2016-10-13 00:41:04 +02:00
Rickard von Essen 3c8dabba9e Fixed formatting 2016-10-11 23:43:50 +02:00
jasminSPC dca286bf38 Merge remote-tracking branch 'upstream/master' into packer-builder-profitbricks 2016-10-10 23:53:52 +02:00
Rickard von Essen ba8cfbd222
builder/openstack: Support using SSH password.
This adds support for using ssh_password to connect to a OpenStack
instance. If ssh_password is defined creation of the temporary keypair
is skipped.
2016-10-08 23:18:19 +02:00
Rickard von Essen 5e96709ee9 Merge pull request #2919 from arizvisa/floppy-recurse
Added an option for copying entire subdirectories via floppy_dirs (supplants floppy_files)
2016-10-08 16:51:16 +02:00
Matthew Hooker badadec140 Merge pull request #3758 from elconas/fix_vmware_builder
Fix License check to work with licensed VMware Workstation
2016-10-07 17:51:32 -07:00
Matthew Hooker b7e14c4354 Merge pull request #3952 from MikeStankavich/f-virtualbox-post-shutdown-delay
Add VirtualBox post shutdown delay to address floppy controller delet…
2016-10-03 16:23:02 -07:00
Matthew Hooker 127142215a Merge pull request #3954 from emo3/f-keepreq-vb
added functionality like vm
2016-10-03 12:55:34 -07:00
Ed Overton 1adb0a5fc3 added functionality like vm 2016-10-03 15:11:21 -04:00
Rickard von Essen 0be3034e82 Merge pull request #3953 from rickard-von-essen/amz-ssh
builder/amazon: Allow using ssh_private_key_file and ssh_password
2016-10-03 10:22:15 +02:00
Rickard von Essen acc2689b49
builder/amazon: Allow using ssh_private_key_file and ssh_password
This adds support for using amazon-ebs/amazon-instance builder without
a keypair. If a ssh_private_key_file is supplied without a
ssh_keypair_name no temporary ssh keypair is created. If ssh_password is
used no temporary ssh keypair is created and the password is used when
trying to connect.

Closes #2301
Closes #3156
2016-10-02 22:20:36 +02:00
Mike Stankavich a187b1cf92 Add VirtualBox post shutdown delay to address floppy controller delete error per issue #2401 2016-10-02 13:12:40 -05:00
Rickard von Essen cffc8e892c Merge pull request #3382 from ahamidi/encrypted-boot-volume
Create AMI with encrypted boot volume
2016-10-02 01:02:43 +02:00
Chris Lundquist 9379cbc5d9 [dynamic-source-ami] support instance builder too 2016-10-01 23:01:01 +00:00
Chris Lundquist ab3500b45c [dynamic-source-ami] support chroot builder 2016-10-01 22:56:09 +00:00
Chris Lundquist 1b4895c684 Merge remote-tracking branch 'origin/master' into dynamic-source-ami 2016-10-01 22:20:17 +00:00
Rickard von Essen 3ad7f067ab Merge pull request #2784 from orivej/fix-ssh-port-max
Make ssh_host_port_max an inclusive bound to agree with documentation
2016-10-02 00:12:37 +02:00
Rickard von Essen 29ae078326 Added missing virtualbox floppy_dirs 2016-10-01 09:05:10 +02:00
Rickard von Essen 924c0bc461 Merge pull request #3932 from pieter-lazzaro/googlecompute-createpassword
GCE Windows instance password creation
2016-09-30 22:01:56 +02:00
Rickard von Essen c0a4067151
builder/googlecompute: Print out WinRM password in debug mode. 2016-09-30 21:17:38 +02:00
Matthew Hooker a8e57d83d2 gofmt project.
noticed many unrelated changes being added to patches because of gofmt.

ran `find . -not -path "./vendor/*" -name "*.go" -exec gofmt -w {} \;`
2016-09-29 14:13:06 -07:00
Matthew Hooker a9abe43325 builder/amazon: add retry login when creating tags.
also move Retry from builder/googlecompute/common to common/retry
2016-09-28 18:22:31 -07:00
jasmin bfb075e505 Renamed artefact -> artifact 2016-09-29 00:04:14 +02:00
jasmin a092701aea Renamed artefact -> artifact 2016-09-28 23:57:37 +02:00
Christopher Boumenot 3dc7873b21 Merge pull request #3936 from boumenot/pr-azure-error-message
azure: fix error message
2016-09-28 14:17:33 -07:00
Christopher Boumenot e42b3d8447 azure: fix error message
There have been two issues raised where the user followed the error
message, but they needed to add yet another config value to disambiguate
the VNET.  It would be better if the message over corrected, and asked the
user to specify everything to ensure complete disambiguation.
2016-09-28 09:50:10 -07:00
Michael Nikitochkin 343c594bb3
Added UserDataFile support for DigitalOcean builder config. 2016-09-28 15:00:57 +02:00
Ali Rizvi-Santiago 86c00490e9 Renamed any and all instances of the word "FloppyContents" to "FloppyDirectories".
Ensure that all builders include FloppyDirectories in the StepCreateFloppy options.
Changed the way the unit-tests in common/step_create_floppy_test work to use the static test-fixtures directory instead of creating the paths dynamically.
Removed a duplicate line of documentation from parallels-pvm.html.md that occurred during rebasing.
2016-09-27 23:31:42 -05:00
Pieter Lazzaro 5db1c1f503 Create a password for windows instances when using WinRM communicator and no password is provided. 2016-09-27 18:42:34 -04:00
Valentin Tjoncke ef1517064c googlecompute/builder: Always show message when checking image exists 2016-09-24 21:12:25 +02:00
nouney 4605f9559f Fix unreachable code 2016-09-23 20:38:28 +02:00
Valentin Tjoncke 6f545af7f7 builder/googlecompute: -force 2016-09-23 14:21:43 +02:00
Rickard von Essen c95aab5e0e Merge pull request #3888 from yoctocloud/qemu
builder/qemu: specify disk format to avoid errors
2016-09-22 08:43:40 +02:00
Rickard von Essen 06e3539ffa Merge pull request #3767 from taliesins/ExtendScanCodes
Add support for ctrl, shift and alt keys and using them as key modifier.
2016-09-20 21:36:46 +02:00
Rickard von Essen 9ce70bfce0 Merge pull request #3338 from yuki-takeichi/gce-nvme 2016-09-20 12:50:43 +02:00
Yuki Takeichi 04078c5c48 Enable to select NVMe images for googlecompute builder 2016-09-20 12:47:14 +02:00
Rickard von Essen 13c9db5dd5 Merge pull request #3885 from orivej/on-error
Add -on-error command line argument to allow preserving artifacts on builder errors
2016-09-20 12:28:14 +02:00
Rickard von Essen 476d981211 Merge pull request #3800 from benwtr/issue-3142-match-dotfiles-docker-communicator
Fix file provisioner dotfile matching w/ docker builder
2016-09-20 12:25:37 +02:00
Matthew Hooker 7cbb98f2d5 QemuArggs -> QemuArgs
Avast ye, resolves #3901
2016-09-19 14:34:13 -07:00
Jeremy Asher aea05f6485 fix nil pointer dereference in amazon-chroot
The refactor of StepRegisterAmi to add the from_scratch parameter
incorrectly replaced iteration over image.BlockDeviceMappings with
iteration over the new mapping array when not using from_scratch.  This
fixes the regression and renames the parameters to newMappings and
mappings for better clarity.
2016-09-16 18:35:36 -07:00
Orivej Desh 6762965696 Add -on-error command line argument to allow preserving artifacts on builder errors
Resolves #409
2016-09-16 12:15:00 +00:00
Vasiliy Tolstov 4986ef39b2 builder/qemu: specify disk format to avoid errors
Image format was not specified for XXX and probing
guessed raw.
Automatically detecting the format is dangerous for raw images, write
operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.

Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
2016-09-15 17:59:05 +03:00
Rickard von Essen 3b47893eb8
builder/amazon: reverted wrongly modified ssh_test's. 2016-09-14 20:26:09 +02:00
jasminSPC 178748bb7c Merge remote-tracking branch 'upstream/master' into packer-builder-profitbricks 2016-09-14 15:32:30 +02:00
jasminSPC 2008c6019a Addressing PR remarks 2016-09-14 15:05:45 +02:00
Rickard von Essen 067155c344
builder/amazon: guard aginst empty/nil PrivateIpAddress and updated tests. 2016-09-14 00:16:09 +02:00
Rickard von Essen 7f4bee737c Merge pull request #3752 from tarrall/issue3751-ssh-privateip-classic
builder/amazon: Honor ssh_private_ip flag in EC2-Classic, not just VPC
2016-09-13 22:54:00 +02:00
Rickard von Essen 3ab49595e6 Merge pull request #3681 from yoctocloud/communicator
builder/qemu: dont fail on communicator set to none
2016-09-13 22:45:12 +02:00
jasminSPC 325401eaf0 Merge remote-tracking branch 'upstream/master' into packer-builder-profitbricks 2016-09-13 12:06:45 +02:00
Ali Rizvi-Santiago a3f0308e92 Re-implemented the support for the floppy_files keyword in order to remain backwards-compatible with templates using the old syntax.
Moved the support for recursive paths from the floppy_files keyword to the new floppy_contents keyword.
Shifted some of the code around to add better logging of what's actually being copied.
Added a couple of unit-tests for the new floppy_contents implementation.
Ensured that all files that were being added were also being included in state.FilesAdded so that the older unit-tests will work.
2016-09-12 11:56:25 -05:00
Rickard von Essen 120b60ae37 Merge pull request #3865 from dpiddy/amazon-ssh-retry-fix
amazon/common/ssh: fix saving of instance to state
2016-09-12 15:04:35 +02:00
Rickard von Essen 10cba9795f Merge pull request #3476 from bhcleek/ansible-with-docker
fix docker builder with ansible provisioner
2016-09-12 13:56:28 +02:00
Rickard von Essen ed1b664d65 Merge pull request #3855 from jeremy-asher/chroot-from-scratch
amazon-chroot build AMI from scratch
2016-09-12 13:48:04 +02:00
Rickard von Essen c3861d1675 Merge pull request #3856 from rickard-von-essen/ecr-support
Improved support for Amazon EC2 Container Registry - ECR
2016-09-12 13:43:41 +02:00
Rickard von Essen 1ca79ad0cb Merge pull request #3873 from crunk1/master
With testify deps fix.
2016-09-12 12:54:25 +02:00
Rickard von Essen 5cbc36103e Merge pull request #3756 from ricardclau/floppy_check
Test floppy disks actually exist
2016-09-12 08:05:16 +02:00
Scott Crunkleton b54b82d3ac Some googlecompute fixes and cleanup. Addresses https://github.com/mitchellh/packer/issues/3829. Changes:
- startup scripts don't run for Windows since it is isn't implemented yet.
- startup scripts use instance metadata instead of serial port output to flag when they are done.
- added licenses to Image data type (to check if an Image is a  Windows Image).
- added GetImage and GetImageFromProject to googlecompute Drivers.
- changed some of the builder/googlecompute tests to use github.com/stretchr/testify/assert.

Tests:
- (in the Packer directory) `go test .`, `go test ./builder/googlecompute`, and `go test ./post-processor/googlecompute-export`
- manual run of `packer build packer_template.json` with the following files

--packer_template.json--
{
  "builders": [
    {
      "type": "googlecompute",
      "account_file": "creds.json",
      "project_id": "google.com:packer-test",
      "source_image": "debian-8-jessie-v20160629",
      "zone": "us-central1-a",
      "startup_script_file": "startup_script.sh",
      "metadata": {
        "startup-script": "#!/bin/sh\necho \"This should be overwritten.\"",
        "startup-script-log-dest": "gs://packer-test.google.com.a.appspot.com/startup-script.log"
      },
      "image_name": "test-packer-modifications",
      "ssh_username": "foo"
    }
  ],
  "post-processors": [
    {
      "type": "googlecompute-export",
      "paths": [
        "gs://packer-test.google.com.a.appspot.com/foo.tar.gz",
        "gs://packer-test.google.com.a.appspot.com/bar.tar.gz"
      ],
      "keep_input_artifact": true
    }
  ]
}

--startup_script.sh--
\#!/bin/sh
echo "Hi, my name is Scott. I'm waiting 60 seconds!" >> /scott
sleep 60
echo "I'm done waiting!" >> /scott
2016-09-09 16:22:50 -07:00
Dan Peterson c7af9a67bc amazon/common/ssh: fix saving of instance to state
Fixes #3496
2016-09-09 10:16:53 -03:00
Rickard von Essen f446c1a1b7
builders/digitalocean: fixes timeout waiting for snapshot #3853 2016-09-08 10:32:43 +02:00
Rickard von Essen b2d9782a9e
Improved support for Amazon EC2 Container Registry - ECR
This adds support for authenticating towards ECR in the docker builder
and docker-push post-processor using them same mechanisms as in the
amazon builders. I.g. access key/secret key, credentials on file,
environment variables, sts tokens or IAM instance roles.
2016-09-04 14:33:47 +02:00
Jeremy Asher 1c392b23f9 generate warnings for unused options with from_scratch 2016-09-03 12:32:43 -07:00
Jeremy Asher ef0c7543e1 add amazon-chroot post mount commands
This adds an additional PostMountCommands step to the amazon-chroot
builder which executes after the volume is mounted, but before the
extra chroot mounts and copy step.
2016-09-03 12:32:41 -07:00