Commit Graph

5710 Commits

Author SHA1 Message Date
upodroid 9b121e85f9 fix dodgy pointers 2020-09-20 15:31:45 +01:00
Upo 3f6230470b
Merge branch 'master' into google-impersonation 2020-09-20 15:23:04 +01:00
upodroid 2faacfd5d7 add service account impersonation 2020-09-20 15:18:37 +01:00
Marco Molteni 4d7e42220b
scaleway: add pre validate step (check image and snapshot names) (#9840)
* Implement Stringer inteface for multistep.StepAction

* scaleway: add pre validate step (check image and snapshot names)

Before, it was possible to create multiple images with the same name,
leading to a confusing and wasteful situation (same for snapshots).

Now, we perform the same kind of checks done by the AWS EC2 builder,
and refuse to proceed if there is an existing image with the same name
(same for snapshots).

As usual, invoking `packer build -force` will bypass the checks.

Closes #9839.
2020-09-18 15:37:48 -04:00
Megan Marsh 77817f80a2
Merge pull request #9956 from hashicorp/do_6734
builder/qemu: Add qemu_img_args option to set special cli flags for our calls to qemu-img
2020-09-18 11:24:34 -07:00
Megan Marsh b7b43a41f3 review cleanup 2020-09-18 10:56:28 -07:00
sylviamoss 778d77e4a2 add floppy and cd to vsphere-clone 2020-09-18 17:09:01 +02:00
Sylvia Moss d347bbec4c
Retry fleet creating only on invalid IAM instance profile (#9946) 2020-09-18 17:05:36 +02:00
Anthony Dong eca9b2f30d
aws ebs_surrogate: fix unhonored kms_key_id (#9959) 2020-09-18 12:39:07 +02:00
Sylvia Moss ce2047f30b
vSphere StepAddCDRom tests (#9955) 2020-09-18 09:57:01 +02:00
Megan Marsh 1d218d9e65 add QemuImgArgs option to set specialized flags in qemu-img
convert, resize, and create commands. Add tests. Refactor steps to
not require config pulled from statebag.
2020-09-17 15:15:04 -07:00
Megan Marsh 1052733fca Skip qemu-img convert if format of output file matches format of input file.
This works around an osx bug in qemu-img convert in osx.
2020-09-17 11:21:04 -07:00
Megan Marsh 83ee4e7d13 miscellaneous qemu cleanup. Fix generator strings; generate code
fix diskimage logic
linting
fix tests
2020-09-17 09:57:45 -07:00
Megan Marsh 3577c4a283 refactor multistep array construction in qemu builder to push majority of conditionals into their steps rather than the multistep constructor code. 2020-09-17 09:57:45 -07:00
Megan Marsh 9ac76be190 clean up step_run 2020-09-17 09:57:45 -07:00
Megan Marsh 5b714497e6 qemu: split config and config tests into their own files to match best practices from other builders 2020-09-17 09:57:45 -07:00
Megan Marsh 6c0c603fbf stub out test 2020-09-17 09:57:45 -07:00
Sylvia Moss 9737b85bbc
Remove usb_keyboard option (#9945) 2020-09-17 10:38:33 +02:00
Wilken Rivera b4be598148
Fix static check issues SA1019 for googlecompute plugins (#9950)
This change replaces the deprecated constructors `New` with
`NewService`.

Static check before the change
```
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | grep SA1019 | grep google
- post-processor/googlecompute-import/post-processor.go:183:18: SA1019:
storage.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- post-processor/googlecompute-import/post-processor.go:219:18: SA1019:
compute.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- post-processor/googlecompute-import/post-processor.go:273:18: SA1019:
storage.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- builder/googlecompute/driver_gce.go:127:18: SA1019: compute.New is
deprecated: please use NewService instead. To provide a custom HTTP
client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- builder/googlecompute/driver_gce.go:132:25: SA1019: oslogin.New is
deprecated: please use NewService instead. To provide a custom HTTP
client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
```

Static check after change
```
[go-1.15.2] [1] wilken@automaton in ~/Development/packer/
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | grep SA1019 | grep google

```
2020-09-17 10:14:41 +02:00
Sylvia Moss 7736ae6c67
Add VMware vnc_over_websocket option (#9938) 2020-09-16 11:02:46 +02:00
Sylvia Moss bdd736b800
Interpolate vagrantfile_template on Vagrant post-processor (#9923) 2020-09-16 10:08:44 +02:00
Megan Marsh bc69db2433
Merge pull request #9941 from hashicorp/cd_files_vsphere
add cd_files to vsphere builder
2020-09-15 11:25:01 -07:00
Megan Marsh ef01bb5ba6 generated code 2020-09-14 15:35:38 -07:00
Megan Marsh e70e8c8651 add cd_files 2020-09-14 14:59:51 -07:00
Megan Marsh f28cabb162
Merge pull request #9940 from hashicorp/fix_9936
skip slot 7 in disk creation so we don't use reserved slot
2020-09-14 13:36:16 -07:00
Megan Marsh b08dd28b85 skip slot 7 in disk creation so we don't use reserved slot 2020-09-14 13:13:59 -07:00
Megan Marsh e00f6afff8
Merge pull request #9918 from hashicorp/fix_7408
skip compaction when backing file is used, to prevent conversion that…
2020-09-14 13:09:13 -07:00
Megan Marsh 6d2df7fb55
Merge pull request #9932 from hashicorp/cd_files_on_virtualbox
Implement cd_files for virtualbox builders.
2020-09-14 13:08:23 -07:00
Megan Marsh eab7672519 regenerate code 2020-09-14 12:16:56 -07:00
Megan Marsh e723609ef1 regenerate hcl config 2020-09-14 12:16:56 -07:00
Megan Marsh 6935f47809 add tests for disk defaulting 2020-09-14 12:16:56 -07:00
Megan Marsh eae3fda901 refactor out disk defaulting so it can be used to default the vmx cdrom too 2020-09-14 12:16:56 -07:00
Megan Marsh 9140985ab0 implement cd_files for qemu builder 2020-09-14 12:16:56 -07:00
Megan Marsh f774983d36 implement on hyperv 2020-09-14 12:16:56 -07:00
Megan Marsh 698bef434f track whether we've detached disks so we don't duplicate work 2020-09-14 11:00:35 -07:00
Megan Marsh a92491d17b capitalization 2020-09-14 10:40:07 -07:00
Andreas Botzner c032d463d3
Bootcommand Fix For Proxmox Builder (#9885) 2020-09-14 17:17:38 +02:00
Sylvia Moss caf65781d7
Add vsphere-clone mac_address option (#9930)
* add vsphere-clone mac_address option

* check on prepare if network is set together with mac_address
2020-09-14 13:18:39 +02:00
Andreas Botzner 3739970b84
Adds ability to specify interfaces for http_directroy and VM for the Proxmox builder (#9874)
Adds two config options to the Proxmox builder:
```
http_interface
vm_interface
```
Both give the user the ability to specify an interface whos IP will be set as the `.HTTPIP` and VMIP respectively.

This is useful when the VM and or the machine executing Packer has multiple interfaces.
Until now Packer would accept the first non-loopback IP as the `.HTTPIP` and VMIP.

I'm open to suggestions and any kind of feedback. 


* Added ability to define the NIC where the HTTPIP is taken from.
* Added VM interface config option
* fmt and documentation
* Moved HTTPInterface into HTTPCommon
* Build Fix
* Documentation Fix
2020-09-14 12:24:01 +02:00
Megan Marsh b73825bb72 Implement cd_files for virtualbox builders.
Since the work to mount isos was duplicated already across the
step_mount_iso and step_mount_guest_additions, this required a
refactor to prevent further code duplication and make the
ports and devices easier to follow
2020-09-11 16:23:29 -07:00
Sylvia Moss 31f08909b4
Add usb_scan_codes option to use USB scancodes instead of VNC for remote builds (#9895) 2020-09-11 16:46:33 +02:00
Megan Marsh 8b9009c571
Merge pull request #9856 from OblateSpheroid/add_vnic_details
Feat (builder/oracle-oci): Add VNIC details to launch details
2020-09-09 16:15:48 -07:00
Megan Marsh 8ffa0d2060
Merge pull request #9842 from jason-azze/patch-1
Builders vsphere-iso and vsphere-clone - CPUs is actually for cores, not sockets.
2020-09-09 16:03:45 -07:00
Megan Marsh ca650c9602 fix struct comment for generation 2020-09-09 16:01:30 -07:00
Megan Marsh 0a02bd5b2a skip compaction when backing file is used, to prevent conversion that would make backing file pointless 2020-09-09 15:26:10 -07:00
js-g 9d5dedc3db tests: add some fields to config_test 2020-09-09 09:25:58 -04:00
Patrik f8e31bff12
feat(builder): bump scaleway to new sdk (#9902) 2020-09-09 10:27:48 +02:00
Wilken Rivera 0719f906d4
Replace if/else with simple if (#9913) 2020-09-09 10:24:17 +02:00
Megan Marsh 8aaa1bd7a6
Merge pull request #9796 from hashicorp/do_8778
create secondary iso containing user's desired files
2020-09-08 15:05:48 -07:00
js-g dd54651125 fix (builder/oracle-oci): add freeform and defined tags to create_vnic_details 2020-09-08 14:40:21 -04:00
Richard Turc b4ff0ea4bc
[builder/qemu] Skip resize step when skip_resize_disk is enable #9860 (#9896)
* [builder/qemu] Skip resize step when skip_resize_disk is enable #9860

* Update builder/qemu/builder_test.go

Improve the code quality

Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>

* Update files for unit tests

Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-09-08 14:32:08 -04:00
js-g f5dc260cf5 refactor (builder/oracle-oci): pass CreateVnicDetails as struct instead of map 2020-09-08 12:03:18 -04:00
Ronald van Zantvoort 7335695c84
qemu vnc: hardcoded lowerbound leaves negative ports
```hcl2
  vnc_port_min = 5000
  vnc_port_max = 5500
```

Build returns faulty:
```
    qemu.one: view the screen of the VM, connect via VNC to vnc://127.0.0.1:5138
(...)
2020/09/06 19:09:52 packer-builder-qemu plugin: Qemu stderr: qemu-system-x86_64: -vnc 127.0.0.1:-762,password: can't convert to a number: -762
```
2020-09-06 21:18:12 +02:00
Megan Marsh d2dd158fe2 Implement cd_files option. The Addfile code is modified from the floppy_files
tooling. Tested on OSX, `Windows, and Linux. Documented command line dependencies.
Added acceptance tests (behind testacc flag, since there are dependencies).
This option is only implemented for the vmware and hyperv iso builders. It will be
implemented for others in later PRs in order to make review easier.
2020-09-04 12:20:19 -07:00
Megan Marsh dfe8aa51b0
Merge pull request #9845 from hashicorp/run_and_stream_helper_func
move runAndStream into a generalized helper function to enable easy l…
2020-09-04 12:19:37 -07:00
Megan Marsh 5120ad9d9a add test file 2020-09-04 11:56:32 -07:00
Megan Marsh 20d62654ae move runAndStream into a generalized helper function to enable easy local exectution in other places than just the docker builder 2020-09-04 11:56:32 -07:00
Lars Lehtonen 03a0beb683
builder/vmware/common: simplify type assertion switch (#9893) 2020-09-04 16:25:19 +02:00
Sylvia Moss ba74602d63
Allows publishing to a Shared Image Gallery with a different subscription id (#9875) 2020-09-03 17:13:32 +02:00
catsby 7a7d3ff350
Remove comments about sharing with catsby 2020-09-03 09:14:40 -05:00
Troy Lindsay d1a61cac33
Add NVMe controller support to vsphere-iso builder (#9880) 2020-09-03 11:14:58 +02:00
Troy Lindsay 602729ac23
Fix vsphere-iso acceptance tests (#9878) (#9881) 2020-09-03 10:17:49 +02:00
Rae Krantz ab23e839c8 update sharing test to req aws account id 2020-09-02 11:44:38 -04:00
Sylvia Moss 903deb9e6a
vSphere StepHardware tests (#9866) 2020-09-02 10:19:57 +02:00
Megan Marsh e96cc07a1a
Merge pull request #9853 from remyleone/boottype_local
change default scaleway boottype to local
2020-09-01 10:44:42 -07:00
Sylvia Moss 24dbd0a28b
Fix iso_path validation regex (#9855) 2020-09-01 10:11:48 +02:00
Wilken Rivera f4556fd3f1
Merge pull request #9831 from hashicorp/remedial_remote_export_config_tests
add tests to make sure prepare for export defaults properly
2020-08-31 15:47:05 -04:00
js-g f74a425661 feat (builder/oracle-oci): pass CreateVnicDetails to instance 2020-08-31 13:13:33 -04:00
Sylvia Moss 58a0bdd780
Look for a default resource pool when root resource pool is not found (#9809) 2020-08-31 16:26:48 +02:00
Wilken Rivera 03a374f4b6
Fix linting issues for SA6005 check (#9854)
Before change
```
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | ack SA6005
builder/profitbricks/step_create_server.go:254:22: SA6005: should use strings.EqualFold(a, b) instead of strings.ToLower(a) == strings.ToLower(b) (staticcheck)
builder/oneandone/config.go:97:7: SA6005: should use strings.EqualFold(a, b) instead of strings.ToLower(a) == strings.ToLower(b) (staticcheck)
builder/vmware/common/driver_parser.go:1199:7: SA6005: should use strings.EqualFold(a, b) instead of strings.ToLower(a) == strings.ToLower(b) (staticcheck)
```

After change
```
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | ack SA6005

```
2020-08-31 15:44:42 +02:00
GennadySpb 804fefef17
yandex-import: allow set custom API endpoint (#9850)
* Separate Access Config from yandex builder Config

* make use of Access Config explicit

* Move `MaxRetries` into AccessConfig

* NewDriverYC use AccessConfig instead Config

* yandex-import PP use common Access Config

Now support set custom API Endpoint

* yandex-export PP use common Access Config

Now support set custom API Endpoint too (as yandex-import)

* fix test

* Tiny doc updates.
2020-08-31 15:29:20 +02:00
OblateSpheroid 0ad26cce01
Feat (oracle-oci): allow freeform and defined tags to be added instance (#9802)
* feat (oracle-oci): allow freeform and defined tags to be added to instance (#6313)

* docs (oracle-oci): add descriptions for instance_tags and instance_defined_tags (#6313)

* fix: fmt

* fix: generate hcl2spec
2020-08-31 14:36:09 +02:00
Rémy Léone ecd1a49a35 change default scaleway boottype to local 2020-08-31 13:15:37 +02:00
Andreas Botzner e4f975fae1
Allows for the mounting of ISOs when a Proxmox VM s created. Same as … (#9653)
Allows the mounting of additional ISOs when the VM is created. The config option was taken from PR #9055 and slightly changed. Users can specify an array of bus names, bus numbers and filenames.

"cd_drive":[
{
"bus": "ide",
"bus_number": 3,
"filename": "isos:iso/virtio-win-0.1.187.iso"
},
{
"bus": "sata",
"bus_number": 3,
"filename": "isos:iso/someother.iso"
}
]

Closes: #7950
Co-authored-by: Calle Pettersson <carlpett@users.noreply.github.com>
2020-08-31 10:48:24 +02:00
Sylvia Moss 942bfbf221
Add driver mocks and write tests to steps Remote Upload and Create VM (#9833) 2020-08-31 10:34:41 +02:00
Megan Marsh 1c5bc41beb refactor ovftool validation into vmware driver 2020-08-27 10:55:57 -07:00
Megan Marsh bbc3a5b0d1 add tests to make sure prepare for export defaults properly 2020-08-27 10:55:57 -07:00
Megan Marsh a6d5106cd7 Allow "export" to ovf/ova for local vmware builds in addition to esx ones.
Refactor step_export and the driver interface to move the ovftool call
into the vmware driver. This refactor allows us to add meaningful tests
to step_export, which I have also added here.
2020-08-26 09:45:12 -07:00
Megan Marsh e9b526ee2d
Move step_create_disk into common folder, and add to vmx builder (#9815)
Pull additional disk related config options into their own file.
2020-08-26 10:13:11 +02:00
Adrien Delorme 0d0bd9ce75
name fields of PolicyDocument correctly in HCL (#9812)
withouth this fix we would have had to do

```hcl
  temporary_iam_instance_profile_policy_document {
    statement {
      action   = ["*"]
      effect   = "Allow"
      resource = ["*"]
    }
    version = "2012-10-17"
  }
```

instead of the same document but with capitalised fields
2020-08-25 10:53:56 +02:00
Adrien Delorme a0c09e85df
retry spot instance creation when an "Invalid IAM Instance Profile name" error pops up (#9810)
PutRolePolicy & AddRoleToInstanceProfile are eventually consistent but it is not possible to wait for them to be done here: 0785c2f6fc/builder/amazon/common/step_iam_instance_profile.go (L117-L134) which was causing the `CreateFleet` to fail (100% for me). So for now we retry a bit later. Waiting 5 seconds after the previously linked code also fixed this.

Test file:

```json
{
	"builders": [
		{
			"type": "amazon-ebs",
			"region": "eu-west-1",
			"ami_name": "ubuntu-16.04 test {{timestamp}}",
			"ami_description": "Ubuntu 16.04 LTS - expand root partition",
			"source_ami_filter": {
				"filters": {
					"virtualization-type": "hvm",
					"name": "ubuntu/images/*/ubuntu-xenial-16.04-amd64-server-*",
					"root-device-type": "ebs"
				},
				"owners": [
					"099720109477"
				],
				"most_recent": true
			},
			"spot_price": "0.03",
			"spot_instance_types": [
				"t2.small"
			],
			"encrypt_boot": true,
			"ssh_username": "ubuntu",
			"ssh_interface": "session_manager",
			"temporary_iam_instance_profile_policy_document": {
				"Version": "2012-10-17",
				"Statement": [
					{
						"Effect": "Allow",
						"Action": [
							"*"
						],
						"Resource": "*"
					}
				]
			},
			"communicator": "ssh"
		}
]}
```
2020-08-25 10:10:32 +02:00
Sylvia Moss 2152fa3313
Accept content library path in iso_paths (#9801) 2020-08-24 16:54:30 +02:00
Sylvia Moss ada2a431ef
Modularise vCenter rest client (#9793) 2020-08-20 17:55:25 +02:00
Sylvia Moss 99800619b7
Add HCL2 examples to the virtualbox builders (#9788) 2020-08-20 15:43:38 +02:00
Sylvia Moss a1524bf96a
Add option to also import OVF templates to the Content Library (#9755) 2020-08-19 13:20:25 +02:00
Megan Marsh a2cfaace59 extract output dir configuration and defaulting into the step_output_dir, using pointers to make sure they get set properly back on the main config 2020-08-18 12:50:06 -07:00
Megan Marsh 3b2bedf794 implement a remote_output_directory option, and heavily comment output code 2020-08-18 12:50:00 -07:00
Megan Marsh b0d2201d55
Merge pull request #9782 from hashicorp/fix_9151
fully destroy vm if it was cancelled or errored
2020-08-18 07:22:18 -07:00
Megan Marsh 0a5cd953fc fully destroy vm if it was cancelled or errored 2020-08-17 14:07:58 -07:00
Megan Marsh aa2418cf01
add progress tracker to vmware-esx iso upload. Add colored prefix to … (#9779)
* add progress tracker to vmware-esx iso upload. Add colored prefix to tracker to make it clear which build a tracker belongs to.

* fix mock; fix tests
2020-08-17 14:35:42 -04:00
Megan Marsh d74b47b734
Merge pull request #9777 from hashicorp/fix_9756
Add aws_polling config option to override env variables
2020-08-17 10:49:38 -07:00
sylviamoss 1a0d5a93ee fix max_attempts typo 2020-08-17 17:58:22 +02:00
sylviamoss 788dc32598 add aws_polling config option to override env variables 2020-08-17 17:15:32 +02:00
Isaac Buckman 46a26a3d9e
Fix Hyper-V boot command refs #5291 (#9765) 2020-08-17 16:00:43 +02:00
Megan Marsh 8b4993e44c
fix docs for kms key ids (#9766) 2020-08-14 11:35:35 +02:00
Megan Marsh acba3ea3e7
Document network path (#9762) 2020-08-13 11:26:40 -04:00
Megan Marsh e97c56dc72
Merge pull request #9559 from hashicorp/fix_9482
builder/azure-arm: Remove duplicate resource deletion steps
2020-08-12 12:43:09 -07:00
Wilken Rivera 55812fe8a5 Handle error of deferred deletion 2020-08-12 11:30:15 -04:00
Wilken Rivera 5700f74c8c Handle lease not present error 2020-08-12 11:27:07 -04:00
Wilken Rivera d91839b93e Update deletion logic for OS disks
* Add an ResourceNotFound error check
* Add logic to break temporary disk lease before deleting it
2020-08-12 11:03:51 -04:00