Commit Graph

153 Commits

Author SHA1 Message Date
Megan Marsh 76177b50ce Refactor provisioner acceptance tests to shell directly out to Packer rather than import the core. Modify test case formatting to more closely reflect the builder test cases. 2020-12-07 15:45:16 -08:00
Megan Marsh 8f51a8bfae move provisioner acceptance tests into sdk alongside builder acceptance tests. Reorganize slightly to make sure no import cycles of doom get formed 2020-12-03 14:22:18 -08:00
Megan Marsh 5576bb5caa move packer config constants next to the packer config 2020-12-03 10:17:35 -08:00
Megan Marsh be5763ec41 move provisioner interface into sdk 2020-12-01 14:25:14 -08:00
Megan Marsh d5971ca25e fix tests by moving mock hooks and communicators, and BasicUi definition, into SDK 2020-11-30 14:30:30 -08:00
Megan Marsh 56a45b04ab Move hook and communicator definitions to packer-plugin-sdk 2020-11-30 14:30:30 -08:00
Megan Marsh c70870cb83 move multierror and multierrorappend into sdk 2020-11-30 14:29:06 -08:00
Megan Marsh 001886670d move Ui definition into the packer plugin sdk. 2020-11-30 14:26:54 -08:00
Megan Marsh f8f1ebf0c7 move version and useragent definitions into sdk 2020-11-18 11:42:02 -08:00
Megan Marsh 9e47696dff move helper/config to bundle with the other template definitions 2020-11-18 10:34:59 -08:00
Megan Marsh 3f6deb8828 move multistep into the plugin sdk 2020-11-17 16:31:03 -08:00
Megan Marsh 7d658149ea update tests and website and generated partials to reflect new package format 2020-11-17 10:36:01 -08:00
Megan Marsh 1ffc358ada move tmp helper func into packer-plugin-sdk 2020-11-16 15:13:08 -08:00
Megan Marsh c4d0c5505a fix tests
regenerate code
regenerate code, revendor
fix website
2020-11-13 15:00:12 -08:00
Megan Marsh e04eaa89e6 move steps and step configs from packer/common to packer/common/commonsteps 2020-11-13 11:15:04 -08:00
Megan Marsh 6402362018 create packer-plugin-sdk directory and begin moving the relevant folders into it. 2020-11-13 10:47:36 -08:00
Megan Marsh 46cf8a1602 add all provisioner version files 2020-11-10 15:01:07 -08:00
Megan Marsh 8ecd16db81 regenerate code 2020-11-10 08:16:59 -08:00
Megan Marsh c0ce8a9414 rework fixer deprecation code so we know what plugins they relate to 2020-10-21 15:28:59 -07:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Sylvia Moss 1c30a71d09
Replace file shared state by statebag (#9238) 2020-05-19 11:49:48 +02:00
Adrien Delorme 840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Wilken Rivera 1eee0bca5f
Merge pull request #9011 from hashicorp/powershell-acceptance-test
provisioner/powershell: Add basic Powershell provisioner acceptance tests
2020-04-07 05:40:26 -04:00
Lars Lehtonen 78bb742aa5
provisioner/shell: fix dropped test error 2020-04-06 14:30:19 -07:00
Wilken Rivera 360f81b030 tests/provisioners: Add TestProvsionerPreCheck function
This function can be used to check if a Provisioner has been marked for testing within the ACC_TEST_PROVISIONERS environment variable.

While testing I found that the shell acceptance test were also running when trying to run powershell tests.

Before change
```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test -v ./provisioner/shell/... -timeout=1h

=== RUN
TestShellProvisioner/testing_amazon-ebs_builder_against_shell_provisioner
2020/04/06 15:18:12 ui: amazon-ebs: output will be in this color.
2020/04/06 15:18:12 ui:
2020/04/06 15:18:12 Build debug mode: false
2020/04/06 15:18:12 Force build: false
2020/04/06 15:18:12 On error:
2020/04/06 15:18:12 Preparing build: amazon-ebs
2020/04/06 15:18:12 Waiting on builds to complete...
2020/04/06 15:18:12 Starting build run: amazon-ebs
2020/04/06 15:18:12 Running builder: amazon-ebs
```

After changes
```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test -v ./provisioner/shell/... -timeout=1h
--- SKIP: TestShellProvisioner (0.00s)
    provisioners.go:88: Provisioner "shell" not defined in ACC_TEST_PROVISIONERS
```
2020-04-06 16:52:24 -04:00
Moss 185deeeb38 Fix provisioner acc-test builder and provisioner store 2020-04-06 16:52:24 -04:00
Sylvia Moss 665330de92
Write generic Builder-Provisioner acceptance test logic (#8963) 2020-04-03 18:17:09 +02:00
Adrien Delorme b94937c05c
Update provisioner_test.go (#8900) 2020-03-18 11:51:54 +01:00
Adrien Delorme 569c802630 Update provisioner_test.go
fix TestProvisioner_RemoteFileDefaultsToScriptnnnn
2020-03-17 17:10:43 +01:00
Adrien Delorme f1c0a9cdbd see what's wrong with that weird test 2020-03-17 17:02:18 +01:00
Moss 112d4daa3d Fix linter 2020-03-13 17:52:33 +01:00
Moss 5a8c628880 Add acc test validation to avoid running with unit tests 2020-03-13 17:50:05 +01:00
Moss 3f49b7c66e Fix linter 2020-03-13 17:17:42 +01:00
Moss 0cc1092222 Fix format 2020-03-13 17:11:13 +01:00
Moss 2ca6804827 Move shell prov acc test to the same folder 2020-03-13 17:10:51 +01:00
Moss 6c06a2a048 Fix format 2020-03-12 17:59:34 +01:00
Moss a3740bb9be Interpolate shell inline config 2020-03-12 17:54:31 +01:00
Wilken Rivera e1a46ec293
Fix gosimple S1025 linting errors (#8838)
Remove unneeded use of fmt.Sprintf for variables that are already strings.
2020-03-04 15:31:30 -05:00
nywilken b80882f3b8 provisioner/shell: Fix envVarFile clean up issue
This change ensures the deletion of the external envVarFile (use_env_var_file = true) occurs after all script files have been executed and deleted.

Build results before change
```
> packer build build.pkr.hcl                                                                                                                                                             [~0]
docker: output will be in this color.

==> docker: Creating a temporary directory for sharing data...
==> docker: Pulling Docker image: ubuntu:bionic
    docker: bionic: Pulling from library/ubuntu
    docker: Digest: sha256:8d31dad0c58f552e890d68bbfb735588b6b820a46e459672d96e585871acc110
    docker: Status: Image is up to date for ubuntu:bionic
    docker: docker.io/library/ubuntu:bionic
==> docker: Starting docker container...
    docker: Run command: docker run -v /home/wilken/.packer.d/tmp476880774:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:bionic
    docker: Container ID: 812069b4d70746a6d4592a8f75c06867c6774b8b0bd81ade76eae7926a30f64b
==> docker: Using docker communicator to connect: 172.17.0.2
==> docker: Provisioning with shell script: sample.sh
    docker: Sample Script
    docker: wilken is NotForSale!
==> docker: Provisioning with shell script: foobar.sh
==> docker: /bin/sh: 1: .: Can't open /tmp/varfile_2555.sh
==> docker: Provisioning step had errors: Running the cleanup provisioner, if present...
==> docker: Killing the container: 812069b4d70746a6d4592a8f75c06867c6774b8b0bd81ade76eae7926a30f64b
Build 'docker' errored: Script exited with non-zero exit status: 2.Allowed exit codes are: [0]

==> Some builds didn't complete successfully and had errors:
--> docker: Script exited with non-zero exit status: 2.Allowed exit codes are: [0]

==> Builds finished but no artifacts were created.
```

Build results after change
```
> packer build build.pkr.hcl
docker: output will be in this color.

==> docker: Creating a temporary directory for sharing data...
==> docker: Pulling Docker image: ubuntu:bionic
    docker: bionic: Pulling from library/ubuntu
    docker: Digest: sha256:8d31dad0c58f552e890d68bbfb735588b6b820a46e459672d96e585871acc110
    docker: Status: Image is up to date for ubuntu:bionic
    docker: docker.io/library/ubuntu:bionic
==> docker: Starting docker container...
    docker: Run command: docker run -v /home/wilken/.packer.d/tmp819845000:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:bionic
    docker: Container ID: d8ed9100ff5017379bfc0d80703b2b2d10c4104941663aa5ddbbf18f6dcf74a5
==> docker: Using docker communicator to connect: 172.17.0.2
==> docker: Provisioning with shell script: sample.sh
    docker: Sample Script
    docker: wilken is NotForSale!
==> docker: Provisioning with shell script: foobar.sh
    docker: FooBar Script
    docker: wilken is NotForSale!
==> docker: Committing the container
    docker: Image ID: sha256:a412e43f134431f2049a7d06d5d691aad3717d91dd2ee5e2575e05b89384a4b3
==> docker: Killing the container: d8ed9100ff5017379bfc0d80703b2b2d10c4104941663aa5ddbbf18f6dcf74a5
Build 'docker' finished.

==> Builds finished. The artifacts of successful builds are:
--> docker: Imported Docker image: sha256:a412e43f134431f2049a7d06d5d691aad3717d91dd2ee5e2575e05b89384a4b3
```
2020-01-23 14:59:36 -05:00
Adrien Delorme 4b7132c87c Merge remote-tracking branch 'origin/master' into sharing_info 2019-12-17 11:57:09 +01:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Megan Marsh 39fd462b56 change all provision func signatures to use map[string]interface{} 2019-12-12 15:38:32 -08:00
Megan Marsh 6f418d0e54 get data sharing to a working state with the powershell provisioner 2019-12-11 15:43:38 -08:00
Megan Marsh f4c3501af5 pass struct of generated data into provision() call 2019-12-11 15:43:38 -08:00
Megan Marsh 7d4353864f use shell struct in shell-local provisioner and post-processor. 2019-11-21 16:47:29 -08:00
nywilken 3522e13b1e provisioner/shell: Add `env_var_format` support to shell provisioner
* Move setting of default EnvVarFormat to the Prepare function
* Add generated hcl2 structure for the updated Shell provisioner config
* Move `EnvVarFormat` field to shared common shell type
* Add description for the `env_var_format` parameter
* Update `env_var_format` when used with `use_env_var_file`
2019-11-06 16:28:56 -05:00
Adrien Delorme 819329228a Change back to make sure all durations are a time.Duration
It is simply the best/simplest solution and trying to prevent users from passing and integer here would be like opening a can of worms. Because:

* we cannot make mapstructure validate our duration string ( with an UnmarshalJSON func etc.)
* we cannot make mapstructure spit a string instead of a duration and packer will decode-encode-decode config.
* the hcl2 generated code asks for a string, so this will be enforced by default.
2019-10-31 16:12:07 +01:00
Adrien Delorme 078ba7c8c3 commit old code generation tool
for history
2019-10-15 12:56:42 +02:00
Lars Lehtonen 9c19cfdc6f
provisioner/shell: fix dropped error 2019-10-15 03:21:05 -07:00
Adrien Delorme d72040f4fa move retry code into the common/retry pkg and make retry context aware 2019-04-09 17:46:38 +02:00