Commit Graph

5940 Commits

Author SHA1 Message Date
Megan Marsh efc117fb55
Merge pull request #10218 from hashicorp/clean_plugin_sdk
Clean plugin sdk
2020-11-06 12:35:23 -08:00
Simon Gottschlag 65b7d3b604
Add Azure CLI authentication (#10157)
Adds the ability to use an active `az login` session for authenticating the Azure builder
2020-11-06 14:24:16 -05:00
Megan Marsh 2d90ffe7a4 move GeneratedData struct and PlaceholderMsg to same package under common 2020-11-06 10:45:38 -08:00
Megan Marsh f13f3d4d5f fix single tag interpolation to allow golang template engine usage 2020-11-05 16:27:53 -08:00
Megan Marsh 7a1680df97
modify a ton of error messages to make them specifically placable. (#10191)
* modify a ton of error messages to make them specifically placable.
* remove specific loglines but keep the specialized error messages

Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-11-05 11:27:07 +01:00
Megan Marsh 45b5f0c2ee move custom http client creation with env proxy args into same package as other network related helpers 2020-11-04 14:51:14 -08:00
Megan Marsh 7b57e28600 move localexec to common/shell-local dir since it is related to shelling out locally 2020-11-04 11:56:46 -08:00
Aleksandr Serbin a763c8ab02 amazon: validate IOPS only for io volumes 2020-11-04 20:49:04 +01:00
Aleksandr Serbin 09c2620c48 amazon: validate IOPS max and min values 2020-11-04 20:29:09 +01:00
Megan Marsh 23552bfbd7
Merge pull request #10213 from chrisroberts/vagrant-driv-stderr
Return error if ssh-config command fails
2020-11-04 10:39:23 -08:00
Megan Marsh d8fec2e9e5
move the hyperv-only powershell dependency into the hyperv subdirectory (#10198) 2020-11-04 15:15:43 +01:00
Marin Salinas 8b5e2903b9
feat: add hk endpoint support for osc builder (#10207) 2020-11-04 15:13:37 +01:00
Aleksandr Serbin 0cf9b55c5c amazon-ebs: validate IOPS ratio 2020-11-04 14:37:41 +01:00
Chris Roberts 8248f52ff7 Return error if ssh-config command fails
Update error detection to return an error if the process fails instead
of testing for content in stderr.
2020-11-03 16:20:34 -08:00
Aleksandr Serbin 7cb8af1441 amazon: remove region from state for chroot builder 2020-11-03 22:28:19 +01:00
Aleksandr Serbin 8dab31b548 amazon: use spot tags for launch template tags
- use `spot_tags` instead of `run_tags` for launch template
- move region to `StepRunSpotInstance` from state
2020-11-03 22:16:00 +01:00
js-g 749a1372d2 test (builder/oracle-oci): add config test for LaunchMode 2020-11-03 11:36:11 -05:00
js-g ba6173b84c fix (builder/oracle-oci): remove subnet from LaunchInstanceDetails
Deprecated use. Use subnet in CreateVnicDetails instead.
2020-11-03 11:35:34 -05:00
js-g c3ffb3dc22 feat (builder/oracle-oci): add launch mode property to image 2020-11-03 10:33:23 -05:00
Sumit Kalra 2b69a5f496 azure-arm builder: Create keyvaults with SoftDelete enabled 2020-11-02 20:42:55 -08:00
Marin Salinas f2b386a9d0 feat: add x509 certificate support 2020-11-02 10:04:00 -06:00
Aleksandr Serbin d561b404d6 amazon-ebs: add tags to launch template 2020-11-01 16:25:43 +01:00
Aleksandr Serbin 4669c0f852 amazon-ebs: log state details on change 2020-10-31 16:00:19 +01:00
Megan Marsh 845a10867e
Merge pull request #10003 from hashicorp/ssm_session_retry
Add retry mechanism to retry SSM session creation
2020-10-29 16:02:54 -07:00
Megan Marsh 72fd763207 fix crash 2020-10-29 14:29:16 -07:00
Rémy Léone 0cfaf4a620 Fix 2020-10-29 17:45:03 +01:00
Wilken Rivera 01d5e5ca76 test/amazon/ebs: Add acceptance test for Session Manager Interface connectivity
Test Results
```
...

2020/10/29 09:35:39 ui:     test: Starting session with SessionId: wilken-00bcfae4d314f54e7
2020/10/29 09:35:40 [DEBUG] TCP connection to SSH ip/port failed: dial tcp [::1]:8047: connect: connection refused
2020/10/29 09:35:40 ui:     test: Port 8047 opened for sessionId wilken-00bcfae4d314f54e7.
2020/10/29 09:35:45 [INFO] Attempting SSH connection to localhost:8047...
2020/10/29 09:35:45 [DEBUG] reconnecting to TCP connection for SSH
2020/10/29 09:35:45 ui:     test: Connection accepted for session wilken-00bcfae4d314f54e7.
2020/10/29 09:35:45 [DEBUG] handshaking with SSH
2020/10/29 09:35:45 [DEBUG] SSH handshake err: ssh: handshake failed: ssh: invalid packet length, packet too large
2020/10/29 09:35:52 [INFO] Attempting SSH connection to localhost:8047...
2020/10/29 09:35:52 [DEBUG] reconnecting to TCP connection for SSH
2020/10/29 09:35:52 [DEBUG] handshaking with SSH
2020/10/29 09:35:52 [DEBUG] handshake complete!
2020/10/29 09:35:52 [DEBUG] Opening new ssh session
2020/10/29 09:35:53 [INFO] agent forwarding enabled
2020/10/29 09:35:53 ui: ==> test: Connected to SSH!
2020/10/29 09:35:53 Running the provision hook
2020/10/29 09:35:53 ui: ==> test: Stopping the source instance...
2020/10/29 09:35:53 ui:     test: Stopping instance
2020/10/29 09:35:54 ui: ==> test: Waiting for the instance to stop...
2020/10/29 09:36:25 ui: ==> test: Creating AMI packer-ssm-test-1603978447 from instance i-0853cb6186a3406d5
2020/10/29 09:36:25 ui:     test: AMI: ami-0868a41bbb2df77b3
2020/10/29 09:36:25 ui: ==> test: Waiting for AMI to become ready...
2020/10/29 09:37:59 ui: ==> test: Terminating the source AWS instance...
2020/10/29 09:37:59 ui error: ==> test: Bad exit status: -1
2020/10/29 09:38:15 ui: ==> test: Cleaning up any extra volumes...
2020/10/29 09:38:15 ui: ==> test: No volumes to clean up, skipping
2020/10/29 09:38:15 ui: ==> test: Deleting temporary security group...
2020/10/29 09:38:16 ui: ==> test: Deleting temporary keypair...
2020/10/29 09:38:16 Deregistering image ID (ami-0868a41bbb2df77b3) from region (us-east-1)
2020/10/29 09:38:17 Deregistered AMI id: ami-0868a41bbb2df77b3
2020/10/29 09:38:17 Deleted snapshot: snap-09602f15994bc9f51
--- PASS: TestBuilderAcc_SessionManagerInterface (249.87s)
PASS

```
2020-10-29 09:39:19 -04:00
Adrien Delorme aae1992649 remove default PauseBeforeSSM, this will have to be set manually 2020-10-29 13:38:03 +01:00
Adrien Delorme 6c45f04467 Delete ssm_mock_funcs.go 2020-10-29 13:37:44 +01:00
Adrien Delorme 5d06a6e6df rename file correctly 2020-10-29 13:26:09 +01:00
Adrien Delorme c6e2dd5538 remove unit test file for now, I think that an acceptance test will be easier here 2020-10-29 13:22:17 +01:00
Adrien Delorme a4bd744955 simplify things a bit more 2020-10-29 13:11:07 +01:00
Adrien Delorme aef3d24213 Update step_create_ssm_tunnel.go 2020-10-29 12:31:01 +01:00
Adrien Delorme f329cb5b93 simplify code 2020-10-29 12:18:41 +01:00
Adrien Delorme b058de072a move packer/builder/amazon/common.IsAWSErr to builder/amazon/common/awserrors.Matches
to avoid cyclic dependency issues
2020-10-29 12:02:41 +01:00
Adrien Delorme 8e355d0fe7 Move ssm code to its own ssm package and make it singlethreaded 2020-10-29 11:57:29 +01:00
Rémy Léone db96c5c9ae Fix 2020-10-29 11:21:31 +01:00
Rémy Léone 3413fc5c46 Fix 2020-10-29 11:19:18 +01:00
Rémy Léone adb74ba15d Fix 2020-10-29 11:09:04 +01:00
Megan Marsh 69312458c4
Merge pull request #10111 from hashicorp/azr_selectable_temp_keygen_type_gcp
GCP: Allow to select algo when generating temporary SSH keypair
2020-10-28 11:25:50 -07:00
Rémy Léone 288e29b1e0 scaleway: use the SDK functions to load profile from file and env 2020-10-28 17:52:34 +01:00
Wilken Rivera 646b973bd3 Remove logic to retry a connection that reuses an existing SSM Session
After testing it was found that once an session is terminated via an
instance restart, console termination, or SSM agent restart. Any active
session will essentially be terminated and unusable. So knowing that it
is always best to start a new session and let the old one timeout get
terminated.
2020-10-28 06:36:23 -04:00
Wilken Rivera dff9cde775 Remove waitgroups 2020-10-28 06:36:23 -04:00
Wilken Rivera eb11009e2a Check for closed channels as opposed to using a separate closeRetry channel 2020-10-28 06:36:23 -04:00
Megan Marsh fb3d357e84
fix step_shutdown when a null communicator is used (#10178) 2020-10-28 11:14:03 +01:00
sylviamoss aa73cc7d7e add close chan to avoid unwanted retries 2020-10-27 16:32:20 -04:00
sylviamoss 1f62249097 add retry terminated session chan 2020-10-27 16:32:20 -04:00
sylviamoss 8e3f3e514c improve logs 2020-10-27 16:32:20 -04:00
sylviamoss b2c7897f58 add WaitGroup to avoid data race 2020-10-27 16:32:20 -04:00
sylviamoss 1c2b469acd add retry channel to ssm driver 2020-10-27 16:32:19 -04:00
Megan Marsh 8f3a115c5a
Merge pull request #10173 from hashicorp/azr-fix-hcl2-gcp-image_encryption_key
Fix GCP HCL image_encryption_key fields and use the same casing in JSON and HCL2
2020-10-27 10:56:26 -07:00
Megan Marsh dc38fadeeb
Merge pull request #10155 from hashicorp/logging_for_10042
builder/vsphere: Don't try to delete a cd file from the remote datastore if the upload of the cd failed
2020-10-27 10:54:07 -07:00
Megan Marsh 8adbba6904 switch to guard clause format 2020-10-27 10:37:03 -07:00
Adrien Delorme ef32147a4a Fix GCP HCL image_encryption_key fields and use the same casing in JSON and HCL2
this closes #9997
2020-10-27 15:46:45 +01:00
Megan Marsh c4866504e1
respect the destroy flag in content library config (#10165)
* respect the destroy flag in content library config

* Make vsphere-clone respect delete_vm state tag; use a common delete func to prevent future drift
2020-10-27 09:07:08 -04:00
Adrien Delorme bab29060fe regen hcl after master merge 2020-10-26 15:48:10 +01:00
Adrien Delorme f5e037e8b4 Merge remote-tracking branch 'origin/master' into azr_selectable_temp_keygen_type_gcp 2020-10-26 15:47:29 +01:00
blz-ea 73370570f4
Fix Proxmox builder unhandled buildvar type (#10154)
* Fix Proxmox builder unhandled buildvar type #10139

Co-authored-by: Calle Pettersson <carlpett@users.noreply.github.com>
Co-authored-by: Adrien Delorme <adrien.delorme@icloud.com>
2020-10-26 15:11:09 +01:00
Megan Marsh 25f4e24772
Merge pull request #10150 from nshalman/nshalman/tag-snapshots
amazon/ebssurrogate: apply snapshot tags right when taking snapshot
2020-10-23 15:57:18 -07:00
Megan Marsh 7646ecf8c6 fix tests 2020-10-23 15:35:37 -07:00
Megan Marsh 4862b2c0f0
Merge pull request #10163 from hashicorp/fix-azure_arm-manageddisk-deletion-regression
builder/azure_arm: Fix build failures due to the deletion of attached managed disks
2020-10-23 15:19:30 -07:00
Megan Marsh fc619dc977
Merge pull request #9626 from featheredtoast/add-proxmox-vm-clone
builder/proxmox FEATURE: split Proxmox into proxmox-iso and proxmox-clone
2020-10-23 13:36:45 -07:00
Megan Marsh 56c7a9cda5 rebase 2020-10-23 11:47:20 -07:00
netapp-jgriffit 15467aa868 Update builder/vsphere/common/step_shutdown.go
Co-authored-by: Sylvia Moss <sylviamoss.m@gmail.com>
2020-10-23 11:41:03 -07:00
James Griffith 185f3d9d48 vsphere clone config not yet using warnings, leaving code commented out 2020-10-23 11:41:03 -07:00
James Griffith c8ad66f419 update to docs as suggested 2020-10-23 11:41:03 -07:00
James Griffith c86a48fbc9 syntax change 2020-10-23 11:41:03 -07:00
James Griffith f86f7c4464 Another instance using `Prepare()` needed an update 2020-10-23 11:41:03 -07:00
James Griffith 09edada93d moved the warning into Prepare() and updated the invocation 2020-10-23 11:41:03 -07:00
James Griffith 572de129d7 update documentation 2020-10-23 11:41:03 -07:00
James Griffith 1048cdbcc4 check for no communicator and provide messaging to user 2020-10-23 11:41:03 -07:00
James Griffith 2ecf5cc9aa Communicator is not needed unless shutdown_command is populated 2020-10-23 11:41:03 -07:00
James Griffith 6b39b1eed6 StepShutdown should still occur if Comm.Type is none 2020-10-23 11:41:03 -07:00
James Griffith 65ff092fd4 hashicorp/packer#9924 2020-10-23 11:41:03 -07:00
Megan Marsh 19bd997f11 add directory existence check 2020-10-23 11:06:25 -07:00
Wilken Rivera a1d69bfe91 Fix liniting issues 2020-10-23 13:52:05 -04:00
Wilken Rivera afd33679f5 builder/azure_arm: Fix build failures due to the deletion of attached managed disks
Previously (prior to v1.6.2) the Azure ARM builder had two delete functions
one that would run before any of the StepDelete* types, and one on deployment template
cleanup. The refactored coded re-introduces the logic from the previously removed
step in v1.6.1 as the main delete logic for the whole deployment
template. Ensuring that all deployed items are deleted before trying to
remove any managed disks.

This change moves all the deletion logic into the
step_deployment_template#Cleanup function to ensure that dependent steps
are only called once the created deployment items (i.e
StepDelateAdditionalDisks) have been deleted.

Test results before change
```

    compute.DisksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> Code="OperationNotAllowed" Message="Disk pkrdd02e9rzzu5k-1 is attached to VM /subscriptions/1f90521a-24f6-4758-ac3d-88d869fb0bf5/resourceGroups/packer-acceptance-test/providers/Microsoft.Compute/virtualMachines/pkrvm02e9rzzu5k."
--- FAIL: TestBuilderAcc_ManagedDisk_Windows_Build_Resource_Group_Additional_Disk (454.00s)
FAIL
FAIL    github.com/hashicorp/packer/builder/azure/arm   454.008s

```

Test results after change
```
--- PASS: TestBuilderAcc_ManagedDisk_Windows_Build_Resource_Group_Additional_Disk (563.56s)

```

Closes #10070
2020-10-23 13:44:27 -04:00
Megan Marsh 8f67f939f9
Merge pull request #10158 from Direnol/yandex-add-support-iam-auth
Added support for IAM credential in the token field and YC_TOKEN env
2020-10-23 09:39:46 -07:00
Roman Mingazeev 75befba67e add more doc 2020-10-23 18:01:52 +03:00
Roman Mingazeev 60595ad522 remove extra log 2020-10-23 14:09:05 +03:00
Roman Mingazeev 72fe3bbc26
Update builder/yandex/access_config.go
Co-authored-by: GennadySpb <lipenkov@gmail.com>
2020-10-23 14:05:39 +03:00
Roman Mingazeev ae7fc84557 Added support for IAM credential in the token field and YC_TOKEN env 2020-10-23 11:47:28 +03:00
Megan Marsh 7fac596b37 don't try to delete a cd file if the upload of the cd itself failed 2020-10-22 16:53:40 -07:00
Megan Marsh 3be53e10af
Merge pull request #10128 from nayyara-samuel/add-state-timeout-linode
Add documented `state_timeout` parameters to Linode builder
2020-10-22 15:13:59 -07:00
Megan Marsh c4001734d0
Merge pull request #10143 from hashicorp/do_9951
builder/vsphere: skip iso download if hashed file is already present on remote datastore
2020-10-22 15:06:04 -07:00
Megan Marsh 00037d8e69
Merge pull request #10145 from hashicorp/fix_10135
commhostport isn't present if communicator is null; protect casting i…
2020-10-22 14:24:23 -07:00
Megan Marsh 70cb4097e1 review comments, add plugin type for a few missing builders 2020-10-21 16:17:32 -07:00
Megan Marsh c0ce8a9414 rework fixer deprecation code so we know what plugins they relate to 2020-10-21 15:28:59 -07:00
Megan Marsh f1f834b8fe commhostport isn't present if communicator is null; protect casting in a check 2020-10-21 14:30:23 -07:00
Nahum Shalman 83a672f2c9 amazon/ebssurrogate: apply snapshot tags right when taking snapshot 2020-10-21 16:17:37 -04:00
Megan Marsh 796c40f89b builder/vsphere: skip iso download if hashed file is already present in remote packer_cache 2020-10-21 11:33:43 -07:00
Wilken Rivera a2ba700ac0 Remove deprecated object ids from configs 2020-10-21 13:56:22 -04:00
Adrien Delorme 522d122858 regen all code 2020-10-21 12:04:10 +02:00
Adrien Delorme 4aea2efd0f HCL2: fully name SSHTemporaryKeyPair fields so that they don't conflict and can be generated 2020-10-21 11:53:07 +02:00
Wilken Rivera 1621ab59d8
tests/azure/arm: Update test image name (#10133)
I've run into a few cases where running the full test suite for the ARM
builder would fail because of conflicting image names. This is a
workaround for now the bigger fix is to have the acceptance clean up any
created images.

Results before change
```
the managed image named testBuilderAccManagedDiskWindows-1603151855 already exists in the resource group packer-acceptance-test, use the -force option to automatically delete it.
--- FAIL: TestBuilderAcc_ManagedDisk_Windows_Build_Resource_Group (0.66s)
```

Results after change
```
PASS
ok      github.com/hashicorp/packer/builder/azure/arm   2431.115s

```
2020-10-21 11:12:50 +02:00
Megan Marsh 799c548c40
Merge pull request #10116 from OblateSpheroid/GH8915
Feat (builder/oracle-oci): Allow filtering on base image
2020-10-20 14:41:09 -07:00
Adrien Delorme d1e0214bec avoid panic debugging ssh key 2020-10-20 16:59:46 +02:00
Adrien Delorme f51b231c17 Merge remote-tracking branch 'origin/master' into azr_selectable_temp_keygen_type_gcp 2020-10-20 16:39:01 +02:00
Adrien Delorme 536421efaf debug mode: don't dump the ssh key if its from a file 2020-10-20 12:00:49 +02:00
js-g 2f26e6d823 tests (builder/oracle-oci): test base image filter in config 2020-10-19 23:30:46 -04:00