Commit Graph

6110 Commits

Author SHA1 Message Date
Sergei Parshev b3b4559434
Added "attach_snapshot" parameter to vmware vmx builder 2021-02-18 15:16:28 -08:00
Tim Dawson ac2ce0097c Fixed formatting 2021-02-19 11:09:55 +13:00
Tim Dawson ca0b11028e Started instramenting some functions and building out unit tests for snapshot step.
Added some basic config tests.
2021-02-19 11:09:55 +13:00
Tim Dawson b199ce9a22 Added snapshots to artifacts
Increased Snapshot wait timeout
2021-02-19 10:58:03 +13:00
Tim Dawson 3e754c9f3f Using new Polling config for snapshot step 2021-02-19 10:58:03 +13:00
Tim Dawson 7f0b41bb0e Refactored SnapshotConfig
Added Group and user permission to each snapshot
2021-02-19 10:58:03 +13:00
Tim Dawson 482a2c8232 Adding ebs-volume snapshot creation 2021-02-19 10:06:04 +13:00
Sylvia Moss 201869d627
Update HCL2 and JSON example for amazon builders (#10645)
* move hcl2 example up over json to amazon-ebs
* switch hcl and json templates for all amazon builders
* fix json format
2021-02-17 10:32:13 +01:00
Megan Marsh 10d32bda6d
Merge pull request #10591 from neumayer/retry
Add retry strategies to oci calls
2021-02-16 13:59:26 -08:00
Megan Marsh 80ed988ffe
Hcl2 upgrade fix (#10625)
* fix random nil pointer dereference I found while debugging hcl2_upgrade issues

* fix hcl2_upgrade command by creating passthroughs for all text template fields
2021-02-15 10:40:21 +01:00
Trond Isak 2c08479825
add AWS API call retries on AMIGetFilteredImage (#10610) 2021-02-11 11:58:56 +01:00
Robert Neumayer 50a6732859 Use http constants and use switch instead of if 2021-02-10 16:46:00 +01:00
Calle Pettersson 6c77d28537
proxmox: Fix additional_iso_files (#10586) 2021-02-10 10:59:44 +01:00
mflorin 9afaa5a21f
Profitbricks builder fixes (#10549) 2021-02-09 16:56:06 +01:00
Robert Neumayer bcd60c3895 Improve retry policy 2021-02-09 16:16:10 +01:00
Jessi 88c516b2d5
Add IMDSv2 support for AWS EBS builder (#10546) 2021-02-09 11:47:54 +01:00
Robert Neumayer 6b947edd04 Add retry strategies to oci calls
The oci api returns 429, among others in case a per-user rate limit is
hit. Currently our only mechanism to deal with this is to wait 5s
between the requests that poll for image availability (and export).

With a custom retry strategy we can handle more error situations while
putting less load on the api.
2021-02-09 09:33:43 +01:00
Megan Marsh ba1ad9ea15
Vsphere example update (#10574)
* fix ubuntu json template example

* add hcl template example

* add hcl alpine example

* add hcl versions of alpine and macos; update macos json

* add hcl version of windows template, call fix on windows json
2021-02-05 13:21:29 +01:00
Adrien Delorme ed091163be
HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00
Wilken Rivera f36554fa0e
Update redirects and URLs to legacy JSON template docs (#10530) 2021-01-27 09:08:59 -05:00
Miguel Hernández 782cf058b5
Add 'skip_create_ami' option to the amazon-ebs builder (#10531) 2021-01-27 09:00:42 -05:00
Roman Mingazeev e588029d6a
yandex: some fix (#10522) 2021-01-25 12:26:21 +01:00
Megan Marsh c3266cc3b0
remove indexing from cdrom command and let qemu handle it (#10519) 2021-01-25 10:54:41 +01:00
Megan Marsh 37dcf6183c
skip credential validation if we are not exporting an image (#10520) 2021-01-25 10:53:51 +01:00
Megan Marsh 3242b7ee10
read iops and throughput as pointers so we can test for the nil case;… (#10518) 2021-01-25 10:49:37 +01:00
Megan Marsh 3aab42b770 remove confusing from_scratch reference which is not relevant in this config 2021-01-22 09:36:17 -08:00
Sylvia Moss 3c7944624a
(3) Add amazon-ami data source (#10467) 2021-01-20 11:05:03 +01:00
Megan Marsh ab98409069
Merge pull request #10488 from Direnol/yandex/update-dump-method
yandex/update-dump-method
2021-01-19 11:10:33 -08:00
fr123k 056ac4a3ea
Add skip_create_image option to the openstack builder (#10496) 2021-01-19 10:43:18 +01:00
Felix Hillingshaeuser 490616a301
Improve cloud init logging for proxmox builder (#10499) 2021-01-19 10:28:01 +01:00
Megan Marsh 94e22e6664
Merge pull request #10490 from marinsalinas/master
fix: [bsusurrogate] override bsu when omi root device is set
2021-01-15 15:55:23 -08:00
Megan Marsh a9c4c056c9
Merge pull request #10458 from seventieskid/gcp-on-gitlab-using-docker
GCP add timeout to GCP metadata server read
2021-01-15 14:51:28 -08:00
Megan Marsh 1cb9aea752 change markdown references to new pathing 2021-01-15 11:59:36 -08:00
Marin Salinas 170286a427 fix: [bsusurrogate] override bsu when omi root device is set 2021-01-15 09:36:56 -06:00
Roman Mingazeev 379a33de42 yandex/update-dump-method 2021-01-15 12:51:54 +03:00
Megan Marsh bc01ffe2dd
Merge pull request #10456 from triarius/spot-instance-lt-resource-tags
Add tags as resource tags in the launch template used to request spot instances
2021-01-13 12:12:17 -08:00
Daniel Frederick Crisman 1e8b0723f5
minor doc update with missing "use" vsphere config (#10474) 2021-01-13 11:38:45 +01:00
Gareth Rees 2dd2ef3c49 - Isolate OSLogin service account derivation from google metadata server to OSLogin step only 2021-01-12 21:38:41 +00:00
Narthana Epa cf30b49e6c
Use run_tags (resp run_volume_tags) to tags intances (resp volumes) created from the spot instance launch template 2021-01-12 23:37:01 +11:00
seventieskid 707d2d8236 Refined logging and return scope 2021-01-11 23:04:36 +00:00
Gareth Rees f525e884ed Timeout on metadata server 2021-01-10 18:29:37 +00:00
Narthana Epa bb94df2d02
Add mocking of launch template id
It is being printed after it is created so we need to mock it to prevent
a nil pointer dereference when the tests are run with the launch
template create request is mocked.
2021-01-10 00:00:16 +11:00
Narthana Epa f7588a3737
Add printing of launch template id 2021-01-09 23:59:31 +11:00
Narthana Epa f7e68f1d71
Add spot tags as resource tags of instances and volumes in launch template
According to the APIReference:
<https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_LaunchTemplateTagSpecificationRequest.html>,
the resource types `instance` and `volume` support tagging on creation.
It is useful to add the spot tags here as it should be more reliable
than tagging them after the spot request is fulfilled as we currently
do.
2021-01-09 23:58:23 +11:00
Darwin Liu 8ef4cfa070 Add in a sanity check for valid vnc ports
Check to make sure that the max and min VNC ports be below 65535 in
config.
2021-01-08 13:43:02 -07:00
Darwin Liu 2b698478a6 Adjusted QEMU step_run_tests.go to reflect previous changes 2021-01-07 22:23:51 -07:00
Darwin Liu 4f2e9117c3 QEMU: Minor fix to VNC mapping parameters and output
* Set the QEMU builder vnc_port_min to have a minimum value of 5900,
  with updated documentation to explain why.

* Changed output messages so that the correct port is listed in packer's
  messaging to the user.

LONGER DESCRIPTION

If vnc_min_port is set to anything above 5900, then packer will fail to
connect via VNC to the QEMU instance. This is due to how QEMU parses
the port for listening to VNC:

   host:d
               TCP connections will only be allowed from host on
               display d.  By convention the TCP port is 5900+d.

Previously the calculation for the port was vncPort - VNCPortMin,
however this will result in an incorrect port being displayed in
packer's messages and potentially packer being unable to connect via VNC
to the host.

For example if vnc_port_min=5990 nad vnc_port_max=5999:

```
Looking for available port between 5990 and 5999 on 0.0.0.0
Found available port: 5996 on IP: 0.0.0.0
Found available VNC port: 5996 on IP: 0.0.0.0
[....]
==> Starting VM, booting disk image
     view the screen of the VM, connect via VNC without a password to
     vnc://0.0.0.0:6
```

This will cause QEMU to set the listening port to 5906 while packer's
VNC client is attempting to connect to 5996.

I am a touch concerned as this commit undoes pull request #9905
(specfically commit 7335695c), but I am also confused as to how he was
able to get QEMU to get a VNC listening port below 5900, as examining
QEMU's git history has shown that this behavior has been in since at
least 2017, probably older.

Hopefully the more explicit error messaging and documentation can make
it clear why this is being undone.
2021-01-07 22:23:43 -07:00
Wilken Rivera 04dbbe2d8e Fix broken links detected by link checker
- Fix broken example links
- Fix broken provisioner links
- Ignore linode.com because of DDOS protection settings
2021-01-06 16:27:12 -05:00
Ali Rizvi-Santiago 9b1d31cee7 Added a fallback to the vmware builder when trying to determine the networkmapping configuration.
Some instances of VMWare will not generate the networkmapping configuration (netmap.conf)
at the time of installation. To deal with this issue, we check for the networkmapping file
first, and then fallback to the networking file which also might exist and contain the
interface information that we require.

This fixes issue #10009.
2021-01-05 11:24:50 -06:00
Megan Marsh 28245ec143
Merge pull request #10134 from borancar/feat/azure_pfx_support
Add Azure PFX certificate support
2020-12-18 12:09:17 -08:00
Megan Marsh d1d5706048
Merge pull request #10362 from nielspardon/master
OpenStack builder: Make external source image props configurable
2020-12-18 10:55:40 -08:00
Megan Marsh ce99640fe2 fix test fixture path 2020-12-17 14:41:36 -08:00
Megan Marsh 88b7b0e14a remove sdk 2020-12-17 13:29:25 -08:00
Megan Marsh 5f34bbd56e add extra line before package declaration so this line does not become the synopsis of a package for godoc 2020-12-16 15:14:43 -08:00
Niels Pardon 0e56b83196 OpenStack builder: Make external source image props configurable
Signed-off-by: Niels Pardon <par@zurich.ibm.com>
2020-12-16 17:19:52 +00:00
Megan Marsh 75803397cb
Document exception to encryption (#10395)
* Update docs on ebs encrypt_boot to clarify that packer will not override global account settings

* Update struct-markdown generator and regenerate partials with new website location. This overwrites some linting that got automatically applied when the files got moved
2020-12-16 10:35:34 +01:00
Megan Marsh eecac40d77
Merge pull request #10394 from anssijun/aws-ebs-retry-spot-instance-tagging
Fix retry logic in AWS spot instance tagging
2020-12-15 11:07:48 -08:00
Megan Marsh 1113301c1b fix conflicts 2020-12-15 10:53:51 -08:00
Dany Garcia 905d8401ea removing unused import 2020-12-15 10:41:25 -08:00
Dany Garcia 0b13d44a23 exporting reusable validatekmskey function 2020-12-15 10:41:25 -08:00
Dany Garcia 3389fded48 merging latest master 2020-12-15 10:40:46 -08:00
Dany Garcia 39bb3a6c55 incorrect error message regarding parameter names 2020-12-15 10:40:12 -08:00
Dany Garcia 504c3807f7 adding support for root volume encryption for amazon-chroot 2020-12-15 10:40:08 -08:00
Anssi Junnola 82a25ec419 Fix retry logic in AWS spot instance tagging 2020-12-15 17:21:03 +00:00
Megan Marsh 4f419506fd
Merge pull request #10384 from Pyrrvs/vmware-dhcp-leases-bigsur
make packer compatible with MacOS BigSur by using Apple DHCP leases instead of VMWare leases
2020-12-14 16:30:50 -08:00
Pyrrvs 12a6fddbd8 handle apple dhcp lease with missing 'name' and 'lease' informations 2020-12-14 19:16:00 +01:00
Roman Mingazeev 354166350d yandex: do not use cloud-config fileds due to possible collision 2020-12-14 13:04:27 +03:00
Pyrrvs f0612e4688 improve parser by skipping all lines containing a '{' or a '}'. fix unecessary []byte cast (linter issue). 2020-12-14 01:00:24 +01:00
gla 7201ce9248 make packer compatible MacOS BigSur by making vmware fusion drivers able to lookup the VM IP address in apple dhcpd leases instead of vmware leases. 2020-12-13 23:26:53 +01:00
Roman Mingazeev bbd9ee8589 fix export when using empty ubuntu image 2020-12-10 14:52:22 +03:00
Roman Mingazeev aa0efcf73e
Yandex/ssh communicator in export (#10352)
* use ssh for communicate to export
* Update post-processor/yandex-export/step-create-s3-keys.go

Co-authored-by: GennadySpb <lipenkov@gmail.com>
2020-12-09 16:51:34 +01:00
Gareth Rees 355b93730b
Gcp oslogin issue 10170 (#10360) 2020-12-09 15:01:51 +01:00
Roman Mingazeev f8e85f94fd
fix set user-agent (#10361) 2020-12-09 14:48:35 +01:00
Mike Tougeron e0e82e2192
Support AWS gp3 volumes (#10338)
* Support AWS gp3 volumes
* docs
* tests
2020-12-09 13:06:57 +01:00
Megan Marsh 26946f1300
Merge pull request #10320 from seventieskid/gcp-wait-to-add-ssh-keys-10312
Gcp wait to add ssh keys 10312
2020-12-08 15:52:52 -08:00
Megan Marsh 1d2ee0bf81
Merge pull request #10337 from serverwentdown/serverwentdown/duplicate-output-file
Fix duplication of main disk in QEMU
2020-12-08 15:18:38 -08:00
Megan Marsh 3c346867f4 fix nil pointer dereference 2020-12-07 15:45:16 -08:00
Megan Marsh 76177b50ce Refactor provisioner acceptance tests to shell directly out to Packer rather than import the core. Modify test case formatting to more closely reflect the builder test cases. 2020-12-07 15:45:16 -08:00
Roman Mingazeev 9fabad6f1d rebase && some fixes 2020-12-07 18:27:22 +03:00
Roman Mingazeev 1c08463204 rebase 2020-12-07 18:13:12 +03:00
Roman Mingazeev efe6d2f08f add comment and remove overrides 2020-12-07 18:13:12 +03:00
Roman Mingazeev 23118c451c fix delete 2020-12-07 18:13:12 +03:00
Roman Mingazeev 4eb3daec8e lint 2020-12-07 18:13:12 +03:00
Roman Mingazeev 478fda12ba reuse common configs 2020-12-07 18:13:06 +03:00
Roman Mingazeev adb4a14471
do better mount image (#10335)
* do better mount image

* check error
2020-12-07 12:20:06 +01:00
Megan Marsh cda3aa205a linting 2020-12-03 16:11:36 -08:00
Megan Marsh 352f064b55 fix docker acceptance tests. Turns out they were broken on the main branch too; needed to move noop progress bar into the sdk to be used in the builder testui. 2020-12-03 16:04:49 -08:00
Megan Marsh 8f51a8bfae move provisioner acceptance tests into sdk alongside builder acceptance tests. Reorganize slightly to make sure no import cycles of doom get formed 2020-12-03 14:22:18 -08:00
Megan Marsh becf7723e6 move builder/testing to acctest inside sdk 2020-12-03 11:57:06 -08:00
Megan Marsh f3f4fef44a fix import 2020-12-03 11:43:06 -08:00
Megan Marsh 98f5fa453c move aws specific acceptance test helper into amazon builder dir 2020-12-03 11:34:45 -08:00
Megan Marsh 2d04ef9b99 remove commented code 2020-12-03 11:32:05 -08:00
Megan Marsh afc798c30a Modify vmware_vmx acceptance tests to use builderT framework
Modify builderT framework to enable use of shell and file provisioners
2020-12-03 11:25:02 -08:00
Megan Marsh 3681e2a7ee fix pathing 2020-12-03 10:39:21 -08:00
Megan Marsh 5576bb5caa move packer config constants next to the packer config 2020-12-03 10:17:35 -08:00
Megan Marsh 8558d1241f move TestUi func to sdk 2020-12-03 10:15:30 -08:00
Megan Marsh a0e81806d3
Merge pull request #10328 from hashicorp/sdk_testing
move builder/testing framework into sdk; it imports the core but I th…
2020-12-03 10:11:28 -08:00
Ambrose Chua 436ac8ef26 Fix duplication of main disk in QEMU 2020-12-04 01:16:26 +08:00
Ambrose Chua f5d5e28012
Update step_copy_disk.go (#10333) 2020-12-03 15:30:11 +01:00
Gareth Rees 937e39d9f4 Amend commit author to align with Hashicorp license 2020-12-03 11:33:49 +00:00