53 Commits

Author SHA1 Message Date
Sylvia Moss
ef4afafde9
Fix force flag for hcl2 provisioners and post-processors (#10571) 2021-02-05 10:57:14 +01:00
Adrien Delorme
ed091163be
HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00
Sylvia Moss
291121dd55
(2) Implement datasources (#10440) 2021-01-20 10:37:16 +01:00
Adrien Delorme
52d2d7fe5f
Copy MapOf... plugin types back into Packer core (#10466)
* move maps of plugins back in core

* go mod vendor

* more fixes

* fix imports

* Update core_test.go

* fix build

* more fixes

* more fixes

* up vendors after fixing sdk

* Update post_processor_mock.hcl2spec.go

* Leave implementatino of MapOf in the sdk for plugi tests

Other wise use the interface

* go mod tidy

* add MapOfDatasource type too
2021-01-13 12:14:06 +01:00
Megan Marsh
88b7b0e14a remove sdk 2020-12-17 13:29:25 -08:00
Megan Marsh
5576bb5caa move packer config constants next to the packer config 2020-12-03 10:17:35 -08:00
Megan Marsh
b69d69095e move postprocessor to sdk, fix generation code 2020-12-01 14:48:55 -08:00
Megan Marsh
be5763ec41 move provisioner interface into sdk 2020-12-01 14:25:14 -08:00
Megan Marsh
ada91b24e9 extract builder interface to sdk 2020-12-01 13:42:42 -08:00
Megan Marsh
d5971ca25e fix tests by moving mock hooks and communicators, and BasicUi definition, into SDK 2020-11-30 14:30:30 -08:00
Adrien Delorme
6dd06fad14 add command/ tests 2020-11-02 15:50:38 +01:00
Megan Marsh
66b81d9bee
add build.name to hcl code (#10114) 2020-10-16 10:57:42 +02:00
Adrien Delorme
3d7ab9ea43 add an hcl2-json test 2020-09-18 16:38:04 +02:00
sylviamoss
39a8dee4ea add tests and update override example 2020-08-14 11:22:51 +02:00
Chris Gilmer
6e918c4bd3
Update test for builder output 2020-08-11 15:25:33 -07:00
Adrien Delorme
125178d943
core: Decode: when in HCL2 decoding mode; reset the whole struct before preparing it. (#9622)
* core: Decode when in HCL2 decoding mode; reset the whole struct before preparing it.
* HCL2: add path variables + docs & tests

Co-authored-by: Megan Marsh <megan@hashicorp.com>
2020-07-23 09:25:07 +02:00
Adrien Delorme
a15ad19411 test builds can be 'excepted' 2020-07-07 11:41:24 +02:00
Adrien Delorme
c1c8b8d22a complete -> recipes 2020-07-07 11:31:19 +02:00
Adrien Delorme
0deaa5d2a5 add a complete tests for builder variables + only/except 2020-07-06 17:01:55 +02:00
Adrien Delorme
7969b690de Revert "Share build info with Provisioner and Post-Processor via HCL2 variables (#9444)"
This reverts commit 99046c917822ede71b32b46a841fd6b4f2fa5d79.
2020-07-02 15:07:29 +02:00
Sylvia Moss
99046c9178
Share build info with Provisioner and Post-Processor via HCL2 variables (#9444) 2020-06-29 13:44:57 +02:00
Adrien Delorme
fb337f8867
hcl work on only/except (#9454)
* HCL2: allow to skip a named build block too

* test that excepting a build block works

* test only on a named build block

* add/update docs
2020-06-23 10:53:16 +02:00
Moss
daaeccd8b0 Skip consecutives post-processors 2020-06-09 17:35:53 +02:00
Moss
bfb9df0039 fix except flag for JSON and HCL2 2020-06-09 15:26:45 +02:00
Adrien Delorme
e6aac8cd6f Update build_test.go 2020-05-08 17:50:48 +02:00
Adrien Delorme
efcc6af06c fix tests 2020-05-08 17:46:33 +02:00
Adrien Delorme
42a05e1e80 more refactoring 2020-05-08 16:41:47 +02:00
Tom Dyas
c0a6623ea2
teach HCL mode builds to honor -only and -except options (#8947) 2020-04-28 15:03:24 +02:00
Adrien Delorme
2af40c762b
HCL2: pass on builder type and name (#8956)
* sets `packer_build_name` and `packer_builder_type` variables for builder provisioners and post-processors in HCL2
* allows to use the new `${source.type}` and `${source.name}` variables in HCL2
* fixes #8932 

Note that the common.PackerConfig is used everywhere and was not set for HCL2, this had some implications: 

For #8923 you can see the issue here:

dde74232f2/builder/lxd/config.go (L61-L63)

More random examples of where this could cause an issue :

0785c2f6fc/provisioner/ansible-local/provisioner.go (L380-L381)

b4efd13a4d/builder/amazon/ebs/builder.go (L232-L236)



* [All references to PackerConfig.PackerBuildName](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L7:2&tab=references)

* [All references to PackerConfig.PackerBuilderType](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L8:2&tab=references)
2020-04-09 11:14:37 +02:00
Megan Marsh
f5c98a7601
make sure cli vars supercede var files (#8964) 2020-03-30 10:31:59 +02:00
Adrien Delorme
ad8dafa3bd
HCL: add tests and fixes around var-file and var args (#8914) 2020-03-19 13:57:22 +01:00
Sylvia Moss
23fa3107a3
Render variables one time on prepare method (#8727) 2020-02-13 17:35:23 +01:00
Adrien Delorme
0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Moss
7466c4fdca Return exit code 1 when builder type is not found 2019-12-10 18:55:18 +01:00
Megan Marsh
0683bc409b add test for error-cleanup provisioner, and fix tests by fixing null builder to use an actual none communicator instead of skipping communicator generation altogether 2019-09-25 13:39:09 -07:00
Adrien Delorme
cb2d89af6f simplify path parsing by making at string instead of an array + add tests 2019-05-07 11:51:21 +02:00
Adrien Delorme
a4b8570991 refactor arg parsing into it's own cfg maker & test it 2019-05-07 11:43:18 +02:00
Adrien Delorme
9281fada2d prevent a breaking change so that we can merge the -parallel-builds option first. 2019-05-02 16:24:28 +02:00
Brett Wandel
92e75f838a added the ability to limit number of builds running in parallel 2019-04-13 22:31:54 +10:00
Adrien Delorme
d7b1b597a7 test provisionning timeout 2019-04-08 20:09:21 +02:00
Megan Marsh
217a93fc30 fix build_test.go on windows. This code needs to not be run in parallel, or else there can be race conditions with the post-processors all trying to access the same file at once. Additionally, I changed the test template itself so that it is valid code on either bash or cmd. Finally, I found and fixed a small bug in naming the file extensions in shell local inline scripts. 2019-04-04 14:57:56 -07:00
Adrien Delorme
7e2d86731e tests: add a post-processor that creates an unnamed.txt file 2019-02-20 11:03:17 +01:00
Adrien Delorme
de336ef15e TestBuildOnlyFileMultipleFlags: check that tomato.txt is not created 2019-02-19 16:40:42 +01:00
Adrien Delorme
b04b8b3857 test building with -except="" 2019-02-19 15:25:41 +01:00
Adrien Delorme
dd3e2c255c respect a 'only' defined in a post-processor 2019-02-01 15:50:06 +01:00
Adrien Delorme
074a74ec38 make sure 'only' completely ignores post-processor
before this commit, if one would put a 'only' inside the post-processor definition, the post process could be skipped
2019-02-01 15:21:41 +01:00
Adrien Delorme
58245f2557 break a chain of post-processors when one is skipped & make -only
"blind" to post-processors

* to avoid trouble
* other arrays of post processors might still be there !
* add docs
* update tests
2019-01-11 14:08:30 +01:00
Adrien Delorme
61ade0e127 allow to use --only with post-processors 2019-01-10 15:44:34 +01:00
Adrien Delorme
4bf3cd44fc allow to skip a post processor 2019-01-10 15:44:34 +01:00
Adrien Delorme
905db043c4 command.TestBuildOnlyFileCommaFlags: create some files using post processors 2019-01-10 15:44:02 +01:00