Commit Graph

55 Commits

Author SHA1 Message Date
Adrien Delorme 77a29fc2f8
Allow to have `dynamic` blocks in a `build` block + tests (#10825)
This :
* allows to have a `build.dynamic` block
* add tests
* makes sure to show a correct message when a source was not found
  * display only name of source (instead of a weird map printout) 
  * use a "Did you mean %q" feature where possible 


Because dynamic blocks need all variables to be evaluated and available, I moved parsing of everything that is not a variable to "after" variables are extrapolated. Meaning that dynamic block get expanded in the `init` phase and then only we start interpreting HCL2 content.

After #10819 fix #10657
2021-03-30 15:53:04 +02:00
Adrien Delorme a115b428ac simplify fmt test case a little 2021-03-12 11:07:26 +01:00
Sylvia Moss ef4afafde9
Fix force flag for hcl2 provisioners and post-processors (#10571) 2021-02-05 10:57:14 +01:00
Adrien Delorme ed091163be
HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00
Sylvia Moss 291121dd55
(2) Implement datasources (#10440) 2021-01-20 10:37:16 +01:00
Adrien Delorme 52d2d7fe5f
Copy `MapOf...` plugin types back into Packer core (#10466)
* move maps of plugins back in core

* go mod vendor

* more fixes

* fix imports

* Update core_test.go

* fix build

* more fixes

* more fixes

* up vendors after fixing sdk

* Update post_processor_mock.hcl2spec.go

* Leave implementatino of MapOf in the sdk for plugi tests

Other wise use the interface

* go mod tidy

* add MapOfDatasource type too
2021-01-13 12:14:06 +01:00
Megan Marsh 88b7b0e14a remove sdk 2020-12-17 13:29:25 -08:00
Megan Marsh 5576bb5caa move packer config constants next to the packer config 2020-12-03 10:17:35 -08:00
Megan Marsh b69d69095e move postprocessor to sdk, fix generation code 2020-12-01 14:48:55 -08:00
Megan Marsh be5763ec41 move provisioner interface into sdk 2020-12-01 14:25:14 -08:00
Megan Marsh ada91b24e9 extract builder interface to sdk 2020-12-01 13:42:42 -08:00
Megan Marsh d5971ca25e fix tests by moving mock hooks and communicators, and BasicUi definition, into SDK 2020-11-30 14:30:30 -08:00
Adrien Delorme 6dd06fad14 add command/ tests 2020-11-02 15:50:38 +01:00
Megan Marsh 66b81d9bee
add build.name to hcl code (#10114) 2020-10-16 10:57:42 +02:00
Adrien Delorme 3d7ab9ea43 add an hcl2-json test 2020-09-18 16:38:04 +02:00
sylviamoss 39a8dee4ea add tests and update override example 2020-08-14 11:22:51 +02:00
Chris Gilmer 6e918c4bd3
Update test for builder output 2020-08-11 15:25:33 -07:00
Adrien Delorme 125178d943
core: Decode: when in HCL2 decoding mode; reset the whole struct before preparing it. (#9622)
* core: Decode when in HCL2 decoding mode; reset the whole struct before preparing it.
* HCL2: add path variables + docs & tests

Co-authored-by: Megan Marsh <megan@hashicorp.com>
2020-07-23 09:25:07 +02:00
Adrien Delorme a15ad19411 test builds can be 'excepted' 2020-07-07 11:41:24 +02:00
Adrien Delorme c1c8b8d22a complete -> recipes 2020-07-07 11:31:19 +02:00
Adrien Delorme 0deaa5d2a5 add a complete tests for builder variables + only/except 2020-07-06 17:01:55 +02:00
Adrien Delorme 7969b690de Revert "Share build info with Provisioner and Post-Processor via HCL2 variables (#9444)"
This reverts commit 99046c9178.
2020-07-02 15:07:29 +02:00
Sylvia Moss 99046c9178
Share build info with Provisioner and Post-Processor via HCL2 variables (#9444) 2020-06-29 13:44:57 +02:00
Adrien Delorme fb337f8867
hcl work on only/except (#9454)
* HCL2: allow to skip a named build block too

* test that excepting a build block works

* test only on a named build block

* add/update docs
2020-06-23 10:53:16 +02:00
Moss daaeccd8b0 Skip consecutives post-processors 2020-06-09 17:35:53 +02:00
Moss bfb9df0039 fix except flag for JSON and HCL2 2020-06-09 15:26:45 +02:00
Adrien Delorme e6aac8cd6f Update build_test.go 2020-05-08 17:50:48 +02:00
Adrien Delorme efcc6af06c fix tests 2020-05-08 17:46:33 +02:00
Adrien Delorme 42a05e1e80 more refactoring 2020-05-08 16:41:47 +02:00
Tom Dyas c0a6623ea2
teach HCL mode builds to honor -only and -except options (#8947) 2020-04-28 15:03:24 +02:00
Adrien Delorme 2af40c762b
HCL2: pass on builder type and name (#8956)
* sets `packer_build_name` and `packer_builder_type` variables for builder provisioners and post-processors in HCL2
* allows to use the new `${source.type}` and `${source.name}` variables in HCL2
* fixes #8932 

Note that the common.PackerConfig is used everywhere and was not set for HCL2, this had some implications: 

For #8923 you can see the issue here:

dde74232f2/builder/lxd/config.go (L61-L63)

More random examples of where this could cause an issue :

0785c2f6fc/provisioner/ansible-local/provisioner.go (L380-L381)

b4efd13a4d/builder/amazon/ebs/builder.go (L232-L236)



* [All references to PackerConfig.PackerBuildName](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L7:2&tab=references)

* [All references to PackerConfig.PackerBuilderType](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L8:2&tab=references)
2020-04-09 11:14:37 +02:00
Megan Marsh f5c98a7601
make sure cli vars supercede var files (#8964) 2020-03-30 10:31:59 +02:00
Adrien Delorme ad8dafa3bd
HCL: add tests and fixes around var-file and var args (#8914) 2020-03-19 13:57:22 +01:00
Sylvia Moss 23fa3107a3
Render variables one time on prepare method (#8727) 2020-02-13 17:35:23 +01:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Moss 7466c4fdca Return exit code 1 when builder type is not found 2019-12-10 18:55:18 +01:00
Megan Marsh 0683bc409b add test for error-cleanup provisioner, and fix tests by fixing null builder to use an actual none communicator instead of skipping communicator generation altogether 2019-09-25 13:39:09 -07:00
Adrien Delorme cb2d89af6f simplify path parsing by making at string instead of an array + add tests 2019-05-07 11:51:21 +02:00
Adrien Delorme a4b8570991 refactor arg parsing into it's own cfg maker & test it 2019-05-07 11:43:18 +02:00
Adrien Delorme 9281fada2d prevent a breaking change so that we can merge the `-parallel-builds` option first. 2019-05-02 16:24:28 +02:00
Brett Wandel 92e75f838a added the ability to limit number of builds running in parallel 2019-04-13 22:31:54 +10:00
Adrien Delorme d7b1b597a7 test provisionning timeout 2019-04-08 20:09:21 +02:00
Megan Marsh 217a93fc30 fix build_test.go on windows. This code needs to not be run in parallel, or else there can be race conditions with the post-processors all trying to access the same file at once. Additionally, I changed the test template itself so that it is valid code on either bash or cmd. Finally, I found and fixed a small bug in naming the file extensions in shell local inline scripts. 2019-04-04 14:57:56 -07:00
Adrien Delorme 7e2d86731e tests: add a post-processor that creates an unnamed.txt file 2019-02-20 11:03:17 +01:00
Adrien Delorme de336ef15e TestBuildOnlyFileMultipleFlags: check that tomato.txt is not created 2019-02-19 16:40:42 +01:00
Adrien Delorme b04b8b3857 test building with `-except=""` 2019-02-19 15:25:41 +01:00
Adrien Delorme dd3e2c255c respect a 'only' defined in a post-processor 2019-02-01 15:50:06 +01:00
Adrien Delorme 074a74ec38 make sure 'only' completely ignores post-processor
before this commit, if one would put a 'only' inside the post-processor definition, the post process could be skipped
2019-02-01 15:21:41 +01:00
Adrien Delorme 58245f2557 break a chain of post-processors when one is skipped & make `-only`
"blind" to post-processors

* to avoid trouble
* other arrays of post processors might still be there !
* add docs
* update tests
2019-01-11 14:08:30 +01:00
Adrien Delorme 61ade0e127 allow to use --only with post-processors 2019-01-10 15:44:34 +01:00