143 Commits

Author SHA1 Message Date
Wilken Rivera
20eeffee0d
integration/secretsmanager: Add support for AWS SharedConfig file (#9781)
This changes updates the AWS Secrets manager session authentication
logic to support reading the AWS configuration file for default
credentials and region settings, if they are not provided via
environment variables.

* Modify error output a little to remove stutter "error ... : error ...`

Results before change
```
unset AWS_REGION
⇶  ~/pkg/packer build amazon-ebs_secretsmanager_shell-local.json
template: root:1:3: executing "root" at <aws_secretsmanager `packer/test/keys`
`shell`>: error calling aws_secretsmanager: Error getting secret: MissingRegion:
could not find region configuration
```

Results after change
```
unset AWS_REGION
⇶  ~/pkg/packer build amazon-ebs_secretsmanager_shell-local.json
null: output will be in this color.

==> null: Running local shell script: /tmp/packer-shell721444992
    null: powershell
Build 'null' finished after 4 milliseconds 121 microseconds.

==> Wait completed after 4 milliseconds 192 microseconds

==> Builds finished. The artifacts of successful builds are:
```
2020-08-18 10:51:48 -04:00
Wilken Rivera
56f6a976de
integrations/secretsmanager: Add support for plaintext, non key/pair, secrets (#9773) 2020-08-17 13:41:21 +02:00
jsmcnair
78c280b1b7 Use common funcs / fix panic / clarify docs 2020-08-12 22:34:13 +01:00
Sylvia Moss
9181a435aa
Make max_retries a string to allow variable interpolation (#9673) 2020-08-05 17:41:20 +02:00
Adrien Delorme
40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Megan Marsh
4a058dfd19 un-bypass secretsmanager tests 2020-05-26 11:50:46 -04:00
Juan Mesa
cc34da365a Add error check
Thrown an error if no key is given when secret has mutliple values
2020-05-23 15:15:06 +02:00
Megan Marsh
8f73a44410 stop failing flaky test until I can fix it 2020-05-21 10:24:42 -07:00
Juan Mesa
b73602eadd Fix corrections
* Fix typos in comments.
* Add documentation of how to use multiple keys with `aws_secretsmanager` function.
* Move `Client` struct to `secretsmanager.go` file.
2020-05-19 13:16:28 +02:00
Juan Mesa
55fa3e1b0b Add support for specifying key to fetch from AWS Secrets Manager 2020-05-15 13:50:33 +02:00
Adrien Delorme
4ebcbad332
update consul and vault dependencies (#9205)
* update consul and vault dependencies

* update triton sign call accorting to https://github.com/joyent/triton-go/pull/135

* Delete readme.md

* put back github.com/mitchellh/reflectwalk to v1.0.0 and explain why

* fix/fixer_pp_docker_tag_tags.go: simplify deduplication loop and keep seen/stable order
2020-05-11 15:26:01 -04:00
Juan Manuel Mesa
75efe3fcd1
Clean linting issues for packages template and interpolate (#9120) 2020-04-27 10:35:47 +02:00
Juan Mesa
e6dbbe454e Fix linting issues 2020-04-21 19:20:45 +02:00
Juan Mesa
0fca4a8ba8 Add template function aws_secretsmanager
Allow users to read secrets from AWS Secrets Manager via
packer template functions
2020-04-21 18:37:29 +02:00
Sylvia Moss
553b1fb9f8
Add RetriedProvisioner to allow retry provisioners (#9061) 2020-04-16 11:58:54 +02:00
Adrien Delorme
840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Jeff Escalante
9165be87da
packer.io prefix removal, html extension remove for in-code errors 2020-04-07 17:53:22 -04:00
Megan Marsh
0cfcbd66ca
execute_command in powershell wasn't interpolating build vars properly (#8771) 2020-03-02 04:20:32 -05:00
Wilken Rivera
9ec8b67392
Add golangci-lint to project (#8686)
* Add golangci-lint as linting tool

* Disable failing staticchecks to start; GitHub issue to handle coming soon

* Run `goimports -w` to repair all source files that have improperly
formatted imports

* makefile: Add ci-lint target to run on travis

This change adds a new make target for running golangci-lint on newly
added Go files only. This target is expected to run during Packer ci builds.

* .github/contributing: Add code linting instructions

* travis: Update job configuration to run parallel builds
2020-02-14 11:42:29 -05:00
Megan Marsh
391cea13ed
Merge pull request #8725 from hashicorp/fix_8655
Check if JSON template doesn't have duplicate configuration
2020-02-13 12:04:26 -08:00
Sylvia Moss
23fa3107a3
Render variables one time on prepare method (#8727) 2020-02-13 17:35:23 +01:00
Moss
dc81720dc9 Fix format 2020-02-12 14:44:28 +01:00
Moss
d654898ebf Add tests for check of json duplicate fields 2020-02-12 14:34:20 +01:00
Moss
a6d90babbf Add check for json duplicate fields 2020-02-12 11:56:18 +01:00
Lars Lehtonen
c6e6220fef
template: replace deprecated os.SEEK_SET with io.SeekStart 2020-02-05 08:56:59 -08:00
Sylvia Moss
328eb8ee96
Interpolate boot_command when defined by user variable (#8640) 2020-01-27 19:10:16 +01:00
Megan Marsh
8490bbc45c add tests for info sharing 2019-12-17 13:41:48 -08:00
Megan Marsh
cd7abf1f9e clean up a few final issues with info sharing 2019-12-17 11:20:57 -08:00
Adrien Delorme
4b7132c87c Merge remote-tracking branch 'origin/master' into sharing_info 2019-12-17 11:57:09 +01:00
Adrien Delorme
0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Megan Marsh
6bc5a95c3a tweak docs; change to 2019-12-16 21:35:42 -08:00
Megan Marsh
0f6d1beccf add an extra string array to the Prepare() return values in the builder interfaces; this sets up the ability for builders to give the provisioners custom user-accessible build-time variables. 2019-12-16 21:23:05 -08:00
Megan Marsh
d47f151bf9 punctuation 2019-12-14 03:38:27 -08:00
Megan Marsh
82367a88f8 reorganize placeholder data call to live with provisioner implementation; force users to use the generated function, therefore forcing validation, for all variables except winrmpassword, by doing a simple string check against the placeholder data. 2019-12-14 03:32:38 -08:00
Megan Marsh
39fd462b56 change all provision func signatures to use map[string]interface{} 2019-12-12 15:38:32 -08:00
Megan Marsh
0ca7c9f397 fix tests 2019-12-11 16:29:35 -08:00
Megan Marsh
5d266b20d0 generated func now validates based on data provided in context, which needs to include the future values if it is to be interpolated properly. 2019-12-11 15:43:38 -08:00
Megan Marsh
528b853178 stub out generated func for validation 2019-12-11 15:43:38 -08:00
Megan Marsh
14baa769ab improve error message 2019-10-23 12:02:31 -07:00
Adrien Delorme
078ba7c8c3 commit old code generation tool
for history
2019-10-15 12:56:42 +02:00
psinghal20
6492995a20
feat: add strftime function in templates
This commit adds strfttime function in templates to provide conventional format for ISO time based on C standard.

Signed-off-by: psinghal20 <psinghal20@gmail.com>
2019-10-11 14:35:39 +05:30
Megan Marsh
4c29f88a0a reuse code 2019-09-27 13:41:22 -07:00
Megan Marsh
a2824a942d remove extraneous import 2019-09-25 14:43:32 -07:00
Megan Marsh
c57e0a9837 remove logline 2019-09-25 13:55:16 -07:00
Megan Marsh
59efa0faee rename option to error-cleanup-privisoner, which I think is clearer 2019-09-25 13:43:29 -07:00
Megan Marsh
0683bc409b add test for error-cleanup provisioner, and fix tests by fixing null builder to use an actual none communicator instead of skipping communicator generation altogether 2019-09-25 13:39:09 -07:00
Megan Marsh
5bd8fee708 Creates a final "cleanup" provisioner to run if an error occurs during a provisioning step, allowing users to perform any custom cleanup tasks that must happen on the VM before the VM is shut down and destroyed. 2019-09-24 16:08:15 -07:00
Adrien Delorme
0f14077664 Merge remote-tracking branch 'origin/master' into switch_sed_because_license 2019-09-23 12:38:52 +02:00
Adrien Delorme
3fe9d52e3f Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-09-20 11:15:44 +02:00
Adrien Delorme
df0b06a87a make sed template call deprecated
and link to docs
2019-09-20 11:08:15 +02:00