Commit Graph

152 Commits

Author SHA1 Message Date
Megan Marsh 9e47696dff move helper/config to bundle with the other template definitions 2020-11-18 10:34:59 -08:00
Megan Marsh 3f6deb8828 move multistep into the plugin sdk 2020-11-17 16:31:03 -08:00
Megan Marsh 7d658149ea update tests and website and generated partials to reflect new package format 2020-11-17 10:36:01 -08:00
Megan Marsh 1ffc358ada move tmp helper func into packer-plugin-sdk 2020-11-16 15:13:08 -08:00
Megan Marsh c4d0c5505a fix tests
regenerate code
regenerate code, revendor
fix website
2020-11-13 15:00:12 -08:00
Megan Marsh e04eaa89e6 move steps and step configs from packer/common to packer/common/commonsteps 2020-11-13 11:15:04 -08:00
Megan Marsh 6402362018 create packer-plugin-sdk directory and begin moving the relevant folders into it. 2020-11-13 10:47:36 -08:00
Megan Marsh 46cf8a1602 add all provisioner version files 2020-11-10 15:01:07 -08:00
Megan Marsh 8ecd16db81 regenerate code 2020-11-10 08:16:59 -08:00
Megan Marsh cd74456026
Common provisioner helpers (#10229)
* update docstrings to make it clear that plugins are servers and core is client

* move provisioner guest helper functions into common dir.
2020-11-09 12:16:44 +01:00
Megan Marsh c0ce8a9414 rework fixer deprecation code so we know what plugins they relate to 2020-10-21 15:28:59 -07:00
Wilken Rivera d72173ac1a Revert "Merge pull request #9040 from hashicorp/powershell-exit-code-fix-4916"
This reverts commit 334f399ee3, reversing
changes made to 45a5d28bad.

When testing against Windows SSH the Powershell script fails to parse
the newly added if statement.
2020-06-09 11:41:07 -04:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Wilken Rivera 1e90165a56
Add source URL to test fixture script (#9296) 2020-05-26 10:24:12 -04:00
Megan Marsh fecb040bf4
fix bug where build vars couldn't be properly interpolated into this execute command (#9275) 2020-05-25 11:54:27 +02:00
Wilken Rivera 2e326ef334 Switch back to call operator as opposed to dot sourcing
* Ensure child scope doesn't conflict with parent scope
* Add elevated user options to tests case.
2020-05-21 09:07:55 -04:00
Wilken Rivera 38f799df3e Add conditional to check if LastExitCode is defined
* Add broken requires statement test case
* Add test case to reproduce invalid LastExitCode
2020-05-21 09:07:43 -04:00
Wilken Rivera 35df3914d2 provisioner/powershell: Update default execute command to handle script errors
This change sets the ErrorActionPreference and wraps the script execution in a Try/Catch statement so that the provisioner can capture any errors encountered when running the script. In addition to the try/catch the `&` operator is replaced by the `.` sourcing operator to ensure the script is executed in the same scope as the parent command (so that errors bubble up properly).

Tests after change
```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test ./provisioner/powershell/... -timeout=1h
ok      github.com/hashicorp/packer/provisioner/powershell      915.865s
```
2020-05-21 09:05:43 -04:00
Wilken Rivera 13e0c1a097 tests/provisioner/powershell: Add acceptance test to reproduce issue 4916 2020-05-21 09:05:43 -04:00
Sylvia Moss 1c30a71d09
Replace file shared state by statebag (#9238) 2020-05-19 11:49:48 +02:00
Wilken Rivera c330d2f04c provisioner/powershell: Update cleanup logic
This change reduces the retry timeout from 5m to 1m, and sets the
RetryDelay to 10s for a total of 6 retries. In additional to the retry
time reduction the cleanup script will now check to see if the
provisioner script created by Packer exists before trying to delete to
prevent any file not found issues.

Closes #9181
Closes #9189
2020-05-14 10:21:45 -04:00
Matt Kotsenas fe55494207
Add `debug_mode` to PowerShell provisioner (#8996) 2020-04-17 11:39:39 +02:00
Adrien Delorme 840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Megan Marsh 73c349d09c
Merge pull request #8908 from hashicorp/fix_4795
provisioner/powershell: Add cleanup step to remove any temporarily created scripts
2020-04-09 08:56:19 -07:00
Lars Lehtonen 49f28c9439
provisioner/powershell: fix dropped error
provisioner/powershell: improve error message
2020-04-08 10:04:08 -07:00
Wilken Rivera 4a46d6ff7b docs/provisioner/powershell: Add documentation for `skip_clean` option 2020-04-08 10:23:15 -04:00
Wilken Rivera fe721d8e11 test/provisioner/powershell: Add acceptance test for powershell provisioner cleanup
Passing Tests
```
--- PASS: TestAccPowershellProvisioner_basic (282.02s)
--- PASS: TestAccPowershellProvisioner_basic/testing_amazon-ebs_builder_against_powershell_provisioner (282.01s)                               PASS
ok      github.com/hashicorp/packer/provisioner/powershell 282.046s
```

Failing tests on master
```
2020/04/08 09:59:34 Uploading file to 'c:/Windows/Temp/script.bat'
2020/04/08 09:59:36 [INFO] starting remote command: set "PACKER_BUILDER_TYPE=amazon-ebs" && set "PACKER_BUILD_NAME=amazon-ebs"
&& "c:/Windows/Temp/script.bat"
2020/04/08 09:59:36 ui:     amazon-ebs:
2020/04/08 09:59:36 ui:     amazon-ebs: C:\Users\Administrator>dir C:\Windows\Temp\packer-*.ps1
2020/04/08 09:59:36 ui:     amazon-ebs:  Volume in drive C has no label.
2020/04/08 09:59:36 [INFO] command 'set "PACKER_BUILDER_TYPE=amazon-ebs"
&& set "PACKER_BUILD_NAME=amazon-ebs" && "c:/Windows/Temp/script.bat"' exited with code: 0
2020/04/08 09:59:36 ui:     amazon-ebs:  Volume Serial Number is 46CA-4083
2020/04/08 09:59:36 ui:     amazon-ebs:
2020/04/08 09:59:36 ui:     amazon-ebs:  Directory of C:\Windows\Temp
2020/04/08 09:59:36 ui:     amazon-ebs:
2020/04/08 09:59:36 ui:     amazon-ebs: 04/08/2020  01:59 PM 102 packer-acc-test-script-test.ps1
2020/04/08 09:59:36 ui:     amazon-ebs: 04/08/2020  01:59 PM 76 packer-acc-test-vars.ps1
2020/04/08 09:59:36 ui:     amazon-ebs:                2 File(s) 178 bytes
2020/04/08 09:59:36 ui:     amazon-ebs:                0 Dir(s) 9,735,806,976 bytes free
2020/04/08 09:59:36 ui: ==> amazon-ebs: Provisioning step had errors: Running the cleanup provisioner, if present...
2020/04/08 09:59:36 ui: ==> amazon-ebs: Terminating the source AWS instance...
2020/04/08 10:00:09 ui: ==> amazon-ebs: Cleaning up any extra volumes...
2020/04/08 10:00:09 ui: ==> amazon-ebs: No volumes to clean up, skipping
2020/04/08 10:00:09 ui: ==> amazon-ebs: Deleting temporary security group...
2020/04/08 10:00:10 ui: ==> amazon-ebs: Deleting temporary keypair...
2020/04/08 10:00:11 ui error: Build 'amazon-ebs' errored: Script exited
with non-zero exit status: 0.Allowed exit codes are: [1]
2020/04/08 10:00:11 machine readable: error-count []string{"1"}
2020/04/08 10:00:11 ui error:
==> Some builds didn't complete successfully and had errors:
2020/04/08 10:00:11 machine readable: amazon-ebs,error []string{"Script
exited with non-zero exit status: 0.Allowed exit codes are: [1]"}
2020/04/08 10:00:11 ui error: --> amazon-ebs: Script exited with
non-zero exit status: 0.Allowed exit codes are: [1]
```

test: Fix windows-shell command
2020-04-08 10:05:54 -04:00
Wilken Rivera 11db6014fa provisioner/powershell: Update remote clean up logic
* Add retry logic so that the provisioner will retry if it fails to upload/execute because of some restart provisioner step
* Add a testConfigWithSkipClean for testing that the provisioner executes the correct commands
* Add a test case for toggling the "skip_clean" config option
2020-04-08 00:23:40 -04:00
Wilken Rivera f6a61e2511 provisioner/powershell: Add post clean up step to remove temp script files 2020-04-08 00:23:40 -04:00
Wilken Rivera 1eee0bca5f
Merge pull request #9011 from hashicorp/powershell-acceptance-test
provisioner/powershell: Add basic Powershell provisioner acceptance tests
2020-04-07 05:40:26 -04:00
Megan Marsh bdcc95f989
fix execution policy parser to not interfere with legit integers, and add tests (#8997) 2020-04-07 10:32:58 +02:00
Wilken Rivera 360f81b030 tests/provisioners: Add TestProvsionerPreCheck function
This function can be used to check if a Provisioner has been marked for testing within the ACC_TEST_PROVISIONERS environment variable.

While testing I found that the shell acceptance test were also running when trying to run powershell tests.

Before change
```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test -v ./provisioner/shell/... -timeout=1h

=== RUN
TestShellProvisioner/testing_amazon-ebs_builder_against_shell_provisioner
2020/04/06 15:18:12 ui: amazon-ebs: output will be in this color.
2020/04/06 15:18:12 ui:
2020/04/06 15:18:12 Build debug mode: false
2020/04/06 15:18:12 Force build: false
2020/04/06 15:18:12 On error:
2020/04/06 15:18:12 Preparing build: amazon-ebs
2020/04/06 15:18:12 Waiting on builds to complete...
2020/04/06 15:18:12 Starting build run: amazon-ebs
2020/04/06 15:18:12 Running builder: amazon-ebs
```

After changes
```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test -v ./provisioner/shell/... -timeout=1h
--- SKIP: TestShellProvisioner (0.00s)
    provisioners.go:88: Provisioner "shell" not defined in ACC_TEST_PROVISIONERS
```
2020-04-06 16:52:24 -04:00
Moss 185deeeb38 Fix provisioner acc-test builder and provisioner store 2020-04-06 16:52:24 -04:00
Wilken Rivera b183c5498f provisioner/powershell: Add basic Powershell provisioner acceptance tests
* Modifies the amazon-ebs builder with a windows build configuration

```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test -v ./provisioner/powershell/... -run=TestPowershellProvisioner_Inline
--- PASS: TestPowershellProvisioner_Inline (256.50s)
--- PASS: TestPowershellProvisioner_Inline/testing_amazon-ebs_builder_against_powershell_provisioner (256.50s)
PASS
ok      github.com/hashicorp/packer/provisioner/powershell 256.525s
```
2020-04-06 16:52:05 -04:00
Megan Marsh 9eda2031d2 Remove redundant placeholder (#8503) 2019-12-19 10:39:46 +01:00
Adrien Delorme 4b7132c87c Merge remote-tracking branch 'origin/master' into sharing_info 2019-12-17 11:57:09 +01:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Megan Marsh 82367a88f8 reorganize placeholder data call to live with provisioner implementation; force users to use the generated function, therefore forcing validation, for all variables except winrmpassword, by doing a simple string check against the placeholder data. 2019-12-14 03:32:38 -08:00
Megan Marsh 3389d843f0 update all provisioners that used the winrmpassword tooling to use the new generateddata option 2019-12-12 16:42:53 -08:00
Megan Marsh 39fd462b56 change all provision func signatures to use map[string]interface{} 2019-12-12 15:38:32 -08:00
Megan Marsh 0ca7c9f397 fix tests 2019-12-11 16:29:35 -08:00
Megan Marsh 6f418d0e54 get data sharing to a working state with the powershell provisioner 2019-12-11 15:43:38 -08:00
Megan Marsh f4c3501af5 pass struct of generated data into provision() call 2019-12-11 15:43:38 -08:00
Megan Marsh 7d4353864f use shell struct in shell-local provisioner and post-processor. 2019-11-21 16:47:29 -08:00
nywilken 3522e13b1e provisioner/shell: Add `env_var_format` support to shell provisioner
* Move setting of default EnvVarFormat to the Prepare function
* Add generated hcl2 structure for the updated Shell provisioner config
* Move `EnvVarFormat` field to shared common shell type
* Add description for the `env_var_format` parameter
* Update `env_var_format` when used with `use_env_var_file`
2019-11-06 16:28:56 -05:00
Adrien Delorme 819329228a Change back to make sure all durations are a time.Duration
It is simply the best/simplest solution and trying to prevent users from passing and integer here would be like opening a can of worms. Because:

* we cannot make mapstructure validate our duration string ( with an UnmarshalJSON func etc.)
* we cannot make mapstructure spit a string instead of a duration and packer will decode-encode-decode config.
* the hcl2 generated code asks for a string, so this will be enforced by default.
2019-10-31 16:12:07 +01:00
Adrien Delorme bf3d9841c6 Force durations to be passed a strings
Before this commit it was possible to set a duration using an integer or a float. Go's time.Duration is an int64 internally an mapstructure will take advantage of this and load the number as a int64 but `1` means one ns which is unexpected/confusing. To avoid confusion and enforce readability this forces users to pass a string with a unit for a duration; ex "56s".
2019-10-31 11:47:19 +01:00
Adrien Delorme 078ba7c8c3 commit old code generation tool
for history
2019-10-15 12:56:42 +02:00
Megan Marsh 0f704fb5b2 These cancel functions have been obsolete since merge of c7ce4d598e 2019-07-26 14:12:58 -07:00
Megan Marsh de7106c36b
Merge pull request #7696 from hashicorp/go_generate_from_makefile
run `go generate ./...` from makefile & CI test that it generates no diff
2019-06-03 15:01:21 -07:00