Commit Graph

180 Commits

Author SHA1 Message Date
jsmcnair 33e3f62ff7 Add vault function for HCL2 and document 2020-08-11 09:18:15 +01:00
Sylvia Moss 9181a435aa
Make max_retries a string to allow variable interpolation (#9673) 2020-08-05 17:41:20 +02:00
Megan Marsh 2bd2954412 fix cty bool passing for provisioners, too 2020-07-30 11:25:44 -07:00
Megan Marsh 713e7a2683 fix docker-import postprocessor. 2020-07-29 14:59:50 -07:00
Adrien Delorme c7b35dd6bc
HCL2: add post-processors block to run multiple post-processor after a build (#9638)
added `post-processors` block to run chained post-processors after a build.
Before this, defining multiple `post-processor` blocks after
provisioning steps would run them sequentially, now doing this makes them start
from the build's artifact. To queue post-processors you now have to define them
in a `post-processors` block.

This is a breaking change.
2020-07-28 10:02:37 +02:00
Adrien Delorme 44616d3bff
refactor initialization out from packer configs + tests (#9627)
The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost.

This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too.

fix #9478
2020-07-24 10:58:03 +02:00
Adrien Delorme 125178d943
core: Decode: when in HCL2 decoding mode; reset the whole struct before preparing it. (#9622)
* core: Decode when in HCL2 decoding mode; reset the whole struct before preparing it.
* HCL2: add path variables + docs & tests

Co-authored-by: Megan Marsh <megan@hashicorp.com>
2020-07-23 09:25:07 +02:00
Adrien Delorme f3fa8cd617 post-processor/provisioner: handle uint64 buildvars 2020-07-15 18:22:24 +02:00
Sylvia Moss 9c1409dbba
Add content_library_destination to import VM template to a Content Library (#9551) 2020-07-10 11:01:10 +02:00
Adrien Delorme 03d29386dd
fix error message typo (#9541) 2020-07-07 16:50:15 +02:00
Adrien Delorme 34ff3d0c8f hcl2 GetBuilds: use packer.BuilderDataCommonKeys to avoid forcing all builders to return exactly what they will return
It would have been a bit nicer if all builder were more specific in order for hcl to tell wether a value will be set during the first prepare time but that also totally works !
2020-07-07 11:29:27 +02:00
Adrien Delorme 0293f69fa1 default build values to be unknown 2020-07-06 16:07:46 +02:00
Adrien Delorme eaf63e3bb5 pass source name from local source name too 2020-07-06 16:07:29 +02:00
Adrien Delorme cf514d31f2 avoid passing generatedVars down during first prepare 2020-07-06 15:14:17 +02:00
Adrien Delorme 0ed10f921f add test 2020-07-06 11:48:39 +02:00
Adrien Delorme 90610dcf04 docs 2020-07-06 11:38:12 +02:00
Adrien Delorme 39261f3671 add hcl2 post processor type that reprepares itself 2020-07-06 10:32:39 +02:00
Adrien Delorme db6c3adbba add hcl2 provisioner type that reprepares itself 2020-07-02 18:02:19 +02:00
Adrien Delorme 284d46a1e0 WIP 2020-07-02 15:56:49 +02:00
Adrien Delorme 7969b690de Revert "Share build info with Provisioner and Post-Processor via HCL2 variables (#9444)"
This reverts commit 99046c9178.
2020-07-02 15:07:29 +02:00
Adrien Delorme 3d371a2d5d
Add complete HCL2 examples + allow to name a singular build.source blocks (#9490)
* in the examples/hcl folder
* add possibility to name singular build.source blocks to differentiate their output and to filter on them
2020-07-02 11:07:59 +02:00
Sylvia Moss 99046c9178
Share build info with Provisioner and Post-Processor via HCL2 variables (#9444) 2020-06-29 13:44:57 +02:00
Adrien Delorme c2975140cf
HCL2: allow to use keep_input_artifact in post processors (#9477)
* HCL2: allow to use keep_input_artifact in post processors
* add basic test
* add docs
2020-06-25 09:36:48 +02:00
Adrien Delorme 6f4d79799b add thorough test 2020-06-23 15:40:54 +02:00
Adrien Delorme 51d02f8c2d hcl2: inspect command 2020-06-23 11:58:57 +02:00
Adrien Delorme a96a8f22b9 HCL2: allow to describe a build block 2020-06-23 11:58:34 +02:00
Adrien Delorme fb337f8867
hcl work on only/except (#9454)
* HCL2: allow to skip a named build block too

* test that excepting a build block works

* test only on a named build block

* add/update docs
2020-06-23 10:53:16 +02:00
Moss daaeccd8b0 Skip consecutives post-processors 2020-06-09 17:35:53 +02:00
Moss 6599e7ab05 apply goimports 2020-06-09 15:29:32 +02:00
Moss bfb9df0039 fix except flag for JSON and HCL2 2020-06-09 15:26:45 +02:00
Adrien Delorme cf6eca7e1c Revert "Add pwd function to HCL"
This reverts commit 2800043149.
2020-06-08 14:59:27 +02:00
Adrien Delorme 44dfa221de Revert "Add template_dir function to HCL"
This reverts commit 3b9a0427cd.
2020-06-08 14:59:22 +02:00
Megan Marsh d73f83a9d9
Merge pull request #9367 from jeremiahsnapp/add-template_dir-and-pwd-functions
Add template_dir() and pwd() functions to HCL
2020-06-05 12:57:04 -07:00
Wilken Rivera f672f5bd9b command/validate: Add support for HCL2 configuration files
* Update validate command to use FixConfig for checking against known
fixers
* Update validation command flag docs
* Add ConfigFixer method to PackerHandler Interface
* Implement ConfigFixer interface in PackerConfig
* Remove all stdout messaging (i.e calls to c.Ui.Say) in the validate
command. The command will only display hcl.Diagnotic messaging when there is an error or warning.

HCL2 Configs
```
⇶  packer validate docker_centos_shell_provisioner.pkr.hcl

```

JSON Configs
```
⇶  packer validate vmware-iso_ubuntu_minimal/vmware-iso_ubuntu_minimal.json
Error: Failed to prepare build: "vmware-iso"

1 error occurred:
        * Deprecated configuration key: 'iso_checksum_type'. Please call `packer fix`
against your template to update your template to be compatable with the current
version of Packer. Visit https://www.packer.io/docs/commands/fix/ for more
detail.

Warning: Fixable configuration found.
You may need to run `packer fix` to get your build to run correctly.
See debug log for more information.

  map[string]interface{}{
        "builders": []interface{}{
                map[string]interface{}{
                        ... // 3 identical entries
                        "guest_os_type":     string("ubuntu-64"),
                        "http_directory":    string("http"),
-                       "iso_checksum":
string("946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
+                       "iso_checksum":
string("sha256:946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
-                       "iso_checksum_type": string("sha256"),
                        "iso_url":
string("http://old-releases.ubuntu.com/releases/14.04.1/ubuntu-14.04.1-server-amd64.iso"),
                        "shutdown_command":  string("echo 'vagrant' | sudo -S shutdown -P now"),
                        ... // 4 identical entries
                },
        },
  }
```
2020-06-05 14:24:39 -04:00
Jeremiah Snapp 3b9a0427cd Add template_dir function to HCL
Signed-off-by: Jeremiah Snapp <jeremiah@chef.io>
2020-06-05 13:14:36 -04:00
Jeremiah Snapp 2800043149 Add pwd function to HCL
Signed-off-by: Jeremiah Snapp <jeremiah@chef.io>
2020-06-05 13:14:03 -04:00
Adrien Delorme bac9c74447
packer console for HCL2 (#9359) 2020-06-05 17:23:54 +02:00
Jeremiah Snapp 0d7c5dc670
Enable only/except for provisioners and post-processors (#9357) 2020-06-05 11:51:23 +02:00
Adrien Delorme f09fd790b7
Merge branch 'master' into master 2020-06-02 15:12:44 +02:00
Adrien Delorme 1147ef9a36
Merge pull request #9291 from hashicorp/f-singular-build-source-blocks
Allow to set source fields from a new `build.source` block
2020-06-02 12:38:45 +02:00
Adrien Delorme 6a406ffb1d
Update hcl2template/types.source.go
Co-authored-by: Megan Marsh <megan@hashicorp.com>
2020-06-02 11:52:11 +02:00
Adrien Delorme 225081c3a6
Update hcl2template/types.source.go
Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-06-02 11:51:48 +02:00
Adrien Delorme b4aa71aef2 Merge remote-tracking branch 'origin/master' into pr/AdrienneCohea/9245 2020-06-02 11:36:32 +02:00
Adrien Delorme 059a782b0e remove debug log 2020-05-28 17:07:11 +02:00
Adrien Delorme 8a13dd3073 HCL2: Make the `build`.`sources` argument list optional 2020-05-28 16:38:41 +02:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Megan Marsh 3dc4ba2d68
Json vs HCL2 parity refactor (#9301)
* refactor so that json and hcl2 templates are both prepared in the same place in the build call, to make code easier to reason about. Remove overly verbose error output which isn't useful in vast majority of cases

* fix tests

* check err msg

* hcl2template.PackerConfig.GetBuilds: raise a diagnostic in case the packer core build perpare call errors

Co-authored-by: Adrien Delorme <adrien.delorme@icloud.com>
2020-05-28 10:43:58 +02:00
Adrien Delorme 125ffe36c3 Allow to override some source fields from the build section
Using `hcl.MergeBodies`
2020-05-25 17:09:37 +02:00
Megan Marsh 24dd7e9dbb
Merge pull request #9257 from hashicorp/azr-hcl2-use-source-type-and-name
HCL2: use source type and name as Name of a CoreBuild
2020-05-20 11:36:15 -07:00
Adrien Delorme 1ddf2a7d05 Update types.packer_config_test.go 2020-05-19 17:35:24 +02:00
Adrien Delorme d431db0a38 HCL2: use source type and name as Name of a CoreBuild 2020-05-19 17:22:04 +02:00
Adrienne Cohea 6c6039798c
Add test. 2020-05-17 23:27:06 -07:00
Adrienne Cohea 6aea8ead6a
Support named builds in HCL2 templates. 2020-05-17 22:13:35 -07:00
Megan Marsh ee5635722b enable force, debug, and on-error for hcl2 builds 2020-05-14 16:22:51 -07:00
Adrien Delorme 1f3e85185e Update types.packer_config.go 2020-05-08 17:19:49 +02:00
Adrien Delorme 0062160581 Update common_test.go 2020-05-08 17:01:42 +02:00
Adrien Delorme 42a05e1e80 more refactoring 2020-05-08 16:41:47 +02:00
Adrien Delorme 526d040534 document Parser.Parse better 2020-04-30 11:51:24 +02:00
Adrien Delorme 5c2b8da63b get builds from PackerConfig instead of parser
to allow just reading the config and to not start anything. This will allow to later on run `validate --syntax-only`.

Note that none of the builder/provisioner/post-processor config will be read but simply ignored. HCL2 still needs the body to be properly formatted and it should detect most syntax errors.
2020-04-29 16:36:40 +02:00
Adrien Delorme 2df21496b3 hcl2template: let PackerConfig actually start a builder/provisioner/post-processor 2020-04-29 16:15:42 +02:00
Adrien Delorme a3343c1848 Delete zz_retrocompat.go
it's dead/unused code I previously though I'd need it
2020-04-29 15:33:10 +02:00
Tom Dyas c0a6623ea2
teach HCL mode builds to honor -only and -except options (#8947) 2020-04-28 15:03:24 +02:00
Adrien Delorme 73242e8de4 make sure keyvalue and namevalue filters are using the same naming as the service being used 2020-04-16 17:22:13 +02:00
Sylvia Moss 553b1fb9f8
Add RetriedProvisioner to allow retry provisioners (#9061) 2020-04-16 11:58:54 +02:00
Adrien Delorme 840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Adrien Delorme 2af40c762b
HCL2: pass on builder type and name (#8956)
* sets `packer_build_name` and `packer_builder_type` variables for builder provisioners and post-processors in HCL2
* allows to use the new `${source.type}` and `${source.name}` variables in HCL2
* fixes #8932 

Note that the common.PackerConfig is used everywhere and was not set for HCL2, this had some implications: 

For #8923 you can see the issue here:

dde74232f2/builder/lxd/config.go (L61-L63)

More random examples of where this could cause an issue :

0785c2f6fc/provisioner/ansible-local/provisioner.go (L380-L381)

b4efd13a4d/builder/amazon/ebs/builder.go (L232-L236)



* [All references to PackerConfig.PackerBuildName](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L7:2&tab=references)

* [All references to PackerConfig.PackerBuilderType](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L8:2&tab=references)
2020-04-09 11:14:37 +02:00
Jeff Escalante 9165be87da
packer.io prefix removal, html extension remove for in-code errors 2020-04-07 17:53:22 -04:00
Adrien Delorme d068430abf
make sure locals are evaluated only once variables are + test this (#8918)
fix #8898
2020-03-19 15:30:34 +01:00
Adrien Delorme ad8dafa3bd
HCL: add tests and fixes around var-file and var args (#8914) 2020-03-19 13:57:22 +01:00
Adrien Delorme 88297c796d Merge branch 'master' into hcl2_singular_blocks 2020-03-17 14:47:17 +01:00
Adrien Delorme 3beea4fb94 NameValues.CopyOn: don't instanciate a receiving map if there is nothing to copy over 2020-03-17 12:00:23 +01:00
Adrien Delorme b5e8750f5f Change key/value to name/value to remain consistent with Terraform 2020-03-17 11:23:11 +01:00
Adrien Delorme c845436e32 KeyValues.CopyOn: make sure a receiving nil map is set too 2020-03-16 15:46:08 +01:00
Adrien Delorme 87d6b2433f make generate 2020-03-16 15:21:29 +01:00
Adrien Delorme 6e6490d688 WIP 2020-03-13 18:04:48 +01:00
Adrien Delorme af38430eb8 add kvfilter type that regroups very frequently used filters 2020-03-13 17:17:00 +01:00
Adrien Delorme 9e6d5da277 hcl2: add general kv store that could be used anywhere 2020-03-13 16:04:42 +01:00
Adrien Delorme c9ec967205 better testing 2020-03-12 18:01:25 +01:00
Adrien Delorme 710ebdcef5 add more test for var files 2020-03-12 17:08:53 +01:00
Adrien Delorme c8300b620a allow to use hcl files as var files in HCL mode
fix #8781
2020-03-12 15:41:40 +01:00
zac 79f1fab836
hcl2template/types: add spaces in error messages. (#8881) 2020-03-12 13:40:25 +01:00
Megan Marsh e518a0a8b7
Merge branch 'master' into update_go-cty_regex 2020-03-10 10:52:42 -07:00
Adrien Delorme 0c45bd8b4f import new replace and regex replace funcs from go-cty + documentation 2020-03-10 16:14:25 +01:00
Adrien Delorme 631f402d1e Merge branch 'master' into fix_8730 2020-03-09 17:49:33 +01:00
Adrien Delorme 0ccff0d5b9 all variables must have a value.
A variable's default value can be set to null to force user to set it.
2020-03-09 17:25:56 +01:00
Adrien Delorme 6d8cce501e tweak validation & add tests 2020-03-09 16:16:59 +01:00
Wilken Rivera 9403b48a52
hcl2template/types.variables: Update logic for parsing literal value variables (#8834)
* hcl2template/types.variables: Update logic for parsing literal value variables

In running tests via the CLI it was determined that when using the variable block with no explicit type assigned the type of the default
value was not being set within the map. This change updates the `decodeConfig` method so that a type is always set for any defined variable if not specified.

The second change is to properly handle the evaluation of basic variable types (e.g String, Number, Bool). Previously variables defined on the
CLI or via PKR_VAR required some additional quoting to for proper evaluation. This change fixes that issue so that it works like it does in Terraform :)

Build results before the change
```
⇶  PKR_VAR_example='["one","two"]' ~/bin/packer build -var 'foo=home' .
Error: Variables not allowed

  on <value for var.foo from arguments> line 1:
  (source code not available)

Variables may not be used here.

==> Builds finished but no artifacts were created.

```

Build results after the change
```
⇶  PKR_VAR_example='["one","two"]' ~/bin/packer build -var 'foo="home"' .
null: output will be in this color.

==> null: Running local shell script: /tmp/packer-shell885249462
    null: two
    null: home
Build 'null' finished.

==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder

⇶  ~/bin/packer build -var 'foo=home' -var 'example=["one","another variable"]' .
null: output will be in this color.

==> null: Running local shell script: /tmp/packer-shell123467506
    null: another variable
    null: home
Build 'null' finished.

==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder

```

* tests/hcl2template/types.variables: Update test to use Bool

Turns out a string value won't actually complain if it's given a non
string looking value. It will just covert the value to a string literal
so using a type Bool which should fail if given anything that is not
true or false.

* tests/hcl2template/types.variables: Update unit tests

During testing it was found that by default the variable stanza were defaulting to a cty.NilType, and not the Type of it's default value.
This change sets the default type of the defined variable to ensure variable evaluation behaviors correctly.

* Add a simple cty.Bool test case

* tests/hcl2template/types.variables:  Enable quoted_string test case

* Update hcl2template/types.variables.go

space

Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-03-06 15:12:26 +01:00
Adrien Delorme e9d3826219 variables: use Range instead of Block
because a variable can be defined in the Locals block
2020-03-04 13:19:07 +01:00
Adrien Delorme 9b649594c6 update tests 2020-03-04 13:01:18 +01:00
Adrien Delorme 8482c6c2e6 evaluateLocalVariable: also pass the variable name 2020-03-04 13:01:04 +01:00
Adrien Delorme bd3112fa7c parseLocalVariables: simplify loop 2020-03-04 13:00:51 +01:00
Adrien Delorme 23b940dbd5 fix this ! 2020-03-04 12:42:56 +01:00
Adrien Delorme e851050774 don't force to set unused variables too 2020-03-04 12:32:29 +01:00
Adrien Delorme 382f209b07 Update types.packer_config.go 2020-03-04 12:30:05 +01:00
Adrien Delorme 59cc246942 Merge remote-tracking branch 'origin/master' into fix_8730 2020-03-04 12:30:01 +01:00
Adrien Delorme 50896d4ddf WIP 2020-03-03 11:15:56 +01:00
Sylvia Moss 591b684f08
Fix HCL2 local variables decoding to allow local usage within another local in the same locals block (#8755) 2020-02-21 12:12:30 +01:00
Adrien Delorme a8fcb2d91a
HCL2: add support for dynamic blocks, document for loops and splat expressions (#8720) 2020-02-20 10:51:34 +01:00
Adrien Delorme 52e5b7051e Update mock.go 2020-02-18 11:31:52 +01:00
Adrien Delorme 3c213e6eaf continue and realise and unused undefaulted value triggers an error 2020-02-17 18:05:15 +01:00