Commit Graph

15965 Commits

Author SHA1 Message Date
Adrien Delorme 0d0bd9ce75
name fields of PolicyDocument correctly in HCL (#9812)
withouth this fix we would have had to do

```hcl
  temporary_iam_instance_profile_policy_document {
    statement {
      action   = ["*"]
      effect   = "Allow"
      resource = ["*"]
    }
    version = "2012-10-17"
  }
```

instead of the same document but with capitalised fields
2020-08-25 10:53:56 +02:00
Adrien Delorme 5ba134ac5b
JSON to HCL2 (minimal best-effort) transpiler (#9659)
hcl2_upgrade transforms a JSON build-file in a HCL2 build-file.
This starts a validated Packer core and from that core we generate an HCL 'block' per plugin/configuration. So for a builder, a provisioner, a post-processor or a variable. The contents of each block is just transformed as is and basically all fields are HCL2-ified.
A generated field can be valid in JSON but invalid on HCL2; for example JSON templating (in mapstructure) allows to set arrays of strings - like `x = ["a", "b"]` - with single strings - like `x="a"` -, HCL does not allow this.
Since JSON does not make the distinction between variables and locals, everything will be a variable. So variables that use other variables will not work.
hcl2_upgrade tries to transform go templating interpolation calls to HCL2 calls when possible, leaving the go templating calls like they are in case it cannot.

Work:
* transpiler
* tests
* update hcl v2 library so that output looks great.
* update docs
2020-08-25 10:51:43 +02:00
Megan Marsh 9f2cb0d560
make the default target generate dev builds. (#9811) 2020-08-25 10:11:38 +02:00
Adrien Delorme a0c09e85df
retry spot instance creation when an "Invalid IAM Instance Profile name" error pops up (#9810)
PutRolePolicy & AddRoleToInstanceProfile are eventually consistent but it is not possible to wait for them to be done here: 0785c2f6fc/builder/amazon/common/step_iam_instance_profile.go (L117-L134) which was causing the `CreateFleet` to fail (100% for me). So for now we retry a bit later. Waiting 5 seconds after the previously linked code also fixed this.

Test file:

```json
{
	"builders": [
		{
			"type": "amazon-ebs",
			"region": "eu-west-1",
			"ami_name": "ubuntu-16.04 test {{timestamp}}",
			"ami_description": "Ubuntu 16.04 LTS - expand root partition",
			"source_ami_filter": {
				"filters": {
					"virtualization-type": "hvm",
					"name": "ubuntu/images/*/ubuntu-xenial-16.04-amd64-server-*",
					"root-device-type": "ebs"
				},
				"owners": [
					"099720109477"
				],
				"most_recent": true
			},
			"spot_price": "0.03",
			"spot_instance_types": [
				"t2.small"
			],
			"encrypt_boot": true,
			"ssh_username": "ubuntu",
			"ssh_interface": "session_manager",
			"temporary_iam_instance_profile_policy_document": {
				"Version": "2012-10-17",
				"Statement": [
					{
						"Effect": "Allow",
						"Action": [
							"*"
						],
						"Resource": "*"
					}
				]
			},
			"communicator": "ssh"
		}
]}
```
2020-08-25 10:10:32 +02:00
Wilken Rivera 1252658848
Merge pull request #9813 from raygervais/documentation/inspec
adds: note in documentation of inspec on host machine required
2020-08-24 16:38:39 -04:00
raygervais 759ae006df adds: note in documentation of inspec on host machine required 2020-08-24 12:39:57 -04:00
Sylvia Moss 2152fa3313
Accept content library path in iso_paths (#9801) 2020-08-24 16:54:30 +02:00
Adrien Delorme fbfe31ceaf
update flock + go mod tidy (#9805) 2020-08-24 10:49:43 +02:00
Adrien Delorme 7de6c21a8a
HCL: document contextual source variable usage (#9799)
It is possible to use ${source.name} and ${source.type}, this was not documented before
2020-08-21 13:37:08 +02:00
Adrien Delorme f28c06a861
HCL: re-add the option to set a command for inspec (#9800) 2020-08-21 13:32:21 +02:00
Sylvia Moss ada2a431ef
Modularise vCenter rest client (#9793) 2020-08-20 17:55:25 +02:00
Sylvia Moss 99800619b7
Add HCL2 examples to the virtualbox builders (#9788) 2020-08-20 15:43:38 +02:00
Megan Marsh 5b27fc7d61 update changelog 2020-08-19 07:43:03 -07:00
Sylvia Moss a1524bf96a
Add option to also import OVF templates to the Content Library (#9755) 2020-08-19 13:20:25 +02:00
Megan Marsh 01d45d67c8
Merge pull request #9785 from hashicorp/document_7067
document workaround for upstream bug
2020-08-18 14:27:51 -07:00
Megan Marsh 6307710485 document workaround for upstream bug 2020-08-18 14:12:14 -07:00
Megan Marsh 229e80bcba
Merge pull request #9784 from hashicorp/cleanup_local_vs_remote_output_directories
implement a remote_output_directory option, and heavily comment outpu…
2020-08-18 14:11:32 -07:00
Megan Marsh a2cfaace59 extract output dir configuration and defaulting into the step_output_dir, using pointers to make sure they get set properly back on the main config 2020-08-18 12:50:06 -07:00
Megan Marsh 3b2bedf794 implement a remote_output_directory option, and heavily comment output code 2020-08-18 12:50:00 -07:00
Wilken Rivera 20eeffee0d
integration/secretsmanager: Add support for AWS SharedConfig file (#9781)
This changes updates the AWS Secrets manager session authentication
logic to support reading the AWS configuration file for default
credentials and region settings, if they are not provided via
environment variables.

* Modify error output a little to remove stutter "error ... : error ...`

Results before change
```
unset AWS_REGION
⇶  ~/pkg/packer build amazon-ebs_secretsmanager_shell-local.json
template: root:1:3: executing "root" at <aws_secretsmanager `packer/test/keys`
`shell`>: error calling aws_secretsmanager: Error getting secret: MissingRegion:
could not find region configuration
```

Results after change
```
unset AWS_REGION
⇶  ~/pkg/packer build amazon-ebs_secretsmanager_shell-local.json
null: output will be in this color.

==> null: Running local shell script: /tmp/packer-shell721444992
    null: powershell
Build 'null' finished after 4 milliseconds 121 microseconds.

==> Wait completed after 4 milliseconds 192 microseconds

==> Builds finished. The artifacts of successful builds are:
```
2020-08-18 10:51:48 -04:00
Megan Marsh b0d2201d55
Merge pull request #9782 from hashicorp/fix_9151
fully destroy vm if it was cancelled or errored
2020-08-18 07:22:18 -07:00
Megan Marsh 0a5cd953fc fully destroy vm if it was cancelled or errored 2020-08-17 14:07:58 -07:00
Megan Marsh aa2418cf01
add progress tracker to vmware-esx iso upload. Add colored prefix to … (#9779)
* add progress tracker to vmware-esx iso upload. Add colored prefix to tracker to make it clear which build a tracker belongs to.

* fix mock; fix tests
2020-08-17 14:35:42 -04:00
Megan Marsh c88ff4ec45
add colored prefix to progress tracker bar so it's clearer what build an uploading file belongs to (#9780) 2020-08-17 14:32:13 -04:00
Megan Marsh d74b47b734
Merge pull request #9777 from hashicorp/fix_9756
Add aws_polling config option to override env variables
2020-08-17 10:49:38 -07:00
sylviamoss 3cdf1d6234 trigger netlify 2020-08-17 18:09:16 +02:00
sylviamoss 1a0d5a93ee fix max_attempts typo 2020-08-17 17:58:22 +02:00
sylviamoss 788dc32598 add aws_polling config option to override env variables 2020-08-17 17:15:32 +02:00
Megan Marsh d20b34e71a
Merge pull request #9772 from hashicorp/fix_9635
change pause to run connect again afterwards
2020-08-17 07:38:28 -07:00
Isaac Buckman 46a26a3d9e
Fix Hyper-V boot command refs #5291 (#9765) 2020-08-17 16:00:43 +02:00
Wilken Rivera 56f6a976de
integrations/secretsmanager: Add support for plaintext, non key/pair, secrets (#9773) 2020-08-17 13:41:21 +02:00
Wilken Rivera 156b6ac2b4
Update GitHub action (#9774)
Ignore issue events triggered by pull_requests
2020-08-17 13:35:19 +02:00
Sathyajith Bhat 9c8daed65e
Fix typo in error message (#9775)
If a variable has not been passed, the error message shown has a tiny typo (varaibles instead of variables)
2020-08-16 21:39:08 -04:00
Marco Molteni 1a248e4868
Fix: build.sh: give validateToolPresence a chance to inform the user (#9776)
Since the build.sh script runs with `set -e` (exit _immediately_ in case
of error), we cannot first call the `which` command and, on a susequent
line, check its exit status with $?, it would be too late. Instead, we
idiomatically check on the same line of the invocation of `which`.

From the confusing:

    $ make bin
    ==> Checking for necessary tools...
    make: *** [bin] Error 1

To the informative:

    $ make bin
    ==> Checking for necessary tools...
    realpath is not on the path. Exiting...
    make: *** [bin] Error 1
2020-08-16 21:37:12 -04:00
Megan Marsh 2dcf950441 change pause to run connect again afterwards 2020-08-14 11:23:43 -07:00
Megan Marsh d486459a00
Merge pull request #9764 from hashicorp/hcl2_provisioner_override
Add override provisioner's option for HCL2
2020-08-14 09:49:34 -07:00
Wilken Rivera 89d57b8635
post-processor/googlecompute-import: Update documentation (#9767) 2020-08-14 12:04:44 +02:00
Megan Marsh 086141f9a9
add an hcl example to the docs pipeline (#9763) 2020-08-14 11:42:06 +02:00
Megan Marsh 8b4993e44c
fix docs for kms key ids (#9766) 2020-08-14 11:35:35 +02:00
sylviamoss 39a8dee4ea add tests and update override example 2020-08-14 11:22:51 +02:00
Megan Marsh 99f503a22b update changelog 2020-08-13 13:11:50 -07:00
sylviamoss 62c3743890 implemet override provisioner's option for hcl2 2020-08-13 19:27:04 +02:00
Megan Marsh acba3ea3e7
Document network path (#9762) 2020-08-13 11:26:40 -04:00
Sylvia Moss a2277e312f
add inspec provisioner hcl2 examples (#9761) 2020-08-13 11:23:56 -04:00
Megan Marsh 29b08e54bc
Merge pull request #9760 from hashicorp/pipeline_with_hcl_docs
add hcl docs to pipeline build docs
2020-08-13 07:19:14 -07:00
Megan Marsh 321aa090df
Merge pull request #9746 from jsmcnair/f-vault-function
Add vault function for HCL2 and documentation
2020-08-13 07:18:28 -07:00
Megan Marsh 2a80bff8c2
Update website/pages/guides/packer-on-cicd/pipelineing-builds.mdx
Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-08-13 07:05:26 -07:00
Megan Marsh b5b28c55fb
update powershell provisioner docs to include HCL examples (#9759) 2020-08-13 13:45:30 +02:00
Megan Marsh 87b290ae67 add hcl docs to pipeline build docs 2020-08-12 15:24:47 -07:00
jsmcnair 78c280b1b7 Use common funcs / fix panic / clarify docs 2020-08-12 22:34:13 +01:00