hcl2_upgrade transforms a JSON build-file in a HCL2 build-file.
This starts a validated Packer core and from that core we generate an HCL 'block' per plugin/configuration. So for a builder, a provisioner, a post-processor or a variable. The contents of each block is just transformed as is and basically all fields are HCL2-ified.
A generated field can be valid in JSON but invalid on HCL2; for example JSON templating (in mapstructure) allows to set arrays of strings - like `x = ["a", "b"]` - with single strings - like `x="a"` -, HCL does not allow this.
Since JSON does not make the distinction between variables and locals, everything will be a variable. So variables that use other variables will not work.
hcl2_upgrade tries to transform go templating interpolation calls to HCL2 calls when possible, leaving the go templating calls like they are in case it cannot.
Work:
* transpiler
* tests
* update hcl v2 library so that output looks great.
* update docs
added `post-processors` block to run chained post-processors after a build.
Before this, defining multiple `post-processor` blocks after
provisioning steps would run them sequentially, now doing this makes them start
from the build's artifact. To queue post-processors you now have to define them
in a `post-processors` block.
This is a breaking change.
The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost.
This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too.
fix#9478
It would have been a bit nicer if all builder were more specific in order for hcl to tell wether a value will be set during the first prepare time but that also totally works !
* Update validate command to use FixConfig for checking against known
fixers
* Update validation command flag docs
* Add ConfigFixer method to PackerHandler Interface
* Implement ConfigFixer interface in PackerConfig
* Remove all stdout messaging (i.e calls to c.Ui.Say) in the validate
command. The command will only display hcl.Diagnotic messaging when there is an error or warning.
HCL2 Configs
```
⇶ packer validate docker_centos_shell_provisioner.pkr.hcl
```
JSON Configs
```
⇶ packer validate vmware-iso_ubuntu_minimal/vmware-iso_ubuntu_minimal.json
Error: Failed to prepare build: "vmware-iso"
1 error occurred:
* Deprecated configuration key: 'iso_checksum_type'. Please call `packer fix`
against your template to update your template to be compatable with the current
version of Packer. Visit https://www.packer.io/docs/commands/fix/ for more
detail.
Warning: Fixable configuration found.
You may need to run `packer fix` to get your build to run correctly.
See debug log for more information.
map[string]interface{}{
"builders": []interface{}{
map[string]interface{}{
... // 3 identical entries
"guest_os_type": string("ubuntu-64"),
"http_directory": string("http"),
- "iso_checksum":
string("946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
+ "iso_checksum":
string("sha256:946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
- "iso_checksum_type": string("sha256"),
"iso_url":
string("http://old-releases.ubuntu.com/releases/14.04.1/ubuntu-14.04.1-server-amd64.iso"),
"shutdown_command": string("echo 'vagrant' | sudo -S shutdown -P now"),
... // 4 identical entries
},
},
}
```
* refactor so that json and hcl2 templates are both prepared in the same place in the build call, to make code easier to reason about. Remove overly verbose error output which isn't useful in vast majority of cases
* fix tests
* check err msg
* hcl2template.PackerConfig.GetBuilds: raise a diagnostic in case the packer core build perpare call errors
Co-authored-by: Adrien Delorme <adrien.delorme@icloud.com>
to allow just reading the config and to not start anything. This will allow to later on run `validate --syntax-only`.
Note that none of the builder/provisioner/post-processor config will be read but simply ignored. HCL2 still needs the body to be properly formatted and it should detect most syntax errors.
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec
this will alow to do
tags = {
key = "value"
}
instead of
tags {
key = "value"
}
This will also enable using variables directly for those tags
* generate code
* update tests
* hcl2template/types.variables: Update logic for parsing literal value variables
In running tests via the CLI it was determined that when using the variable block with no explicit type assigned the type of the default
value was not being set within the map. This change updates the `decodeConfig` method so that a type is always set for any defined variable if not specified.
The second change is to properly handle the evaluation of basic variable types (e.g String, Number, Bool). Previously variables defined on the
CLI or via PKR_VAR required some additional quoting to for proper evaluation. This change fixes that issue so that it works like it does in Terraform :)
Build results before the change
```
⇶ PKR_VAR_example='["one","two"]' ~/bin/packer build -var 'foo=home' .
Error: Variables not allowed
on <value for var.foo from arguments> line 1:
(source code not available)
Variables may not be used here.
==> Builds finished but no artifacts were created.
```
Build results after the change
```
⇶ PKR_VAR_example='["one","two"]' ~/bin/packer build -var 'foo="home"' .
null: output will be in this color.
==> null: Running local shell script: /tmp/packer-shell885249462
null: two
null: home
Build 'null' finished.
==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder
⇶ ~/bin/packer build -var 'foo=home' -var 'example=["one","another variable"]' .
null: output will be in this color.
==> null: Running local shell script: /tmp/packer-shell123467506
null: another variable
null: home
Build 'null' finished.
==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder
```
* tests/hcl2template/types.variables: Update test to use Bool
Turns out a string value won't actually complain if it's given a non
string looking value. It will just covert the value to a string literal
so using a type Bool which should fail if given anything that is not
true or false.
* tests/hcl2template/types.variables: Update unit tests
During testing it was found that by default the variable stanza were defaulting to a cty.NilType, and not the Type of it's default value.
This change sets the default type of the defined variable to ensure variable evaluation behaviors correctly.
* Add a simple cty.Bool test case
* tests/hcl2template/types.variables: Enable quoted_string test case
* Update hcl2template/types.variables.go
space
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.
All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.
## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files
I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields
## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.
This is a breaking change for packer plugins.
a packer component can be a: builder/provisioner/post-processor
each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.
This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:
Provisioner.Prepare(raws ...interface{}) error
Builder.Prepare(raws ...interface{}) ([]string, error)
PostProcessor.Configure(raws ...interface{}) error
close#1768
Example hcl files:
```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
sources = [
"source.amazon-ebs.first",
]
provisioner "shell" {
inline = [
"sleep 5"
]
}
post-processor "shell-local" {
inline = [
"sleep 5"
]
}
}
// amazon-ebs-kms-key/source.pkr.hcl
source "amazon-ebs" "first" {
ami_name = "hcl2-test"
region = "us-east-1"
instance_type = "t2.micro"
kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
encrypt_boot = true
source_ami_filter {
filters {
virtualization-type = "hvm"
name = "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
root-device-type = "ebs"
}
most_recent = true
owners = ["amazon"]
}
launch_block_device_mappings {
device_name = "/dev/xvda"
volume_size = 20
volume_type = "gp2"
delete_on_termination = "true"
}
launch_block_device_mappings {
device_name = "/dev/xvdf"
volume_size = 500
volume_type = "gp2"
delete_on_termination = true
encrypted = true
}
ami_regions = ["eu-central-1"]
run_tags {
Name = "packer-solr-something"
stack-name = "DevOps Tools"
}
communicator = "ssh"
ssh_pty = true
ssh_username = "ec2-user"
associate_public_ip_address = true
}
```
Before this commit it was possible to set a duration using an integer or a float. Go's time.Duration is an int64 internally an mapstructure will take advantage of this and load the number as a int64 but `1` means one ns which is unexpected/confusing. To avoid confusion and enforce readability this forces users to pass a string with a unit for a duration; ex "56s".
Package hcl2template defines code to parse hcl2 template files correctly.
In order to configure a packer builder,provisioner,communicator and post
processor.
Checkout the files in testdata/complete/ to see what a packer config could
look like.
Adrien Delorme
Adrien Delorme
Megan Marsh
sylviamoss
jsmcnair
Wilken Rivera
Jeremiah Snapp
Jeremiah Snapp
Adrienne Cohea
Tom Dyas
Jeff Escalante
zac