Commit Graph

678 Commits

Author SHA1 Message Date
Moss 8ea64c5b35 undocument ssh_keypair_name and ssh_agent_auth 2020-07-02 14:55:58 +02:00
Moss 41edd09d3c undocument temporary_key_pair_name and ssh_private_key_file 2020-07-01 16:33:18 +02:00
Sylvia Moss 99046c9178
Share build info with Provisioner and Post-Processor via HCL2 variables (#9444) 2020-06-29 13:44:57 +02:00
mingsheng.su 8a8abdf615
update ucloud-uhost builder (#9466)
* update ucloud-uhost builder, docs and deps

Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-06-24 14:31:05 +02:00
Megan Marsh 4f77bb0c4a add new ssh_ciphers option 2020-06-17 16:48:23 -07:00
Gennady Lipenkov 56224ae08d Instance SA auth for yandex-export post-processor 2020-06-17 18:15:53 +03:00
Megan Marsh 4178625afb re-add winrm_no_proxy option. Winrm code needs to be modified in order to read no_proxy env var when client is created rather than when net/http library is instantiated. 2020-06-16 10:27:30 -07:00
Megan Marsh a0a3ddbadb
Merge pull request #9379 from GennadySpb/sa-auth-in-yandex-export
Support Authentication by Service Account Key file in Yandex Export post-processor
2020-06-10 09:40:02 -07:00
GennadySpb 79770f919a
Fix error message (#9373) 2020-06-10 13:29:20 +02:00
Gennady Lipenkov 85da85271f Support SA Key authentication in Yandex Export post-processor 2020-06-09 11:06:12 +03:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Megan Marsh 4041cb7775 generated code 2020-05-19 16:10:51 -07:00
Megan Marsh 557eb015ea make artifice post-processor play nicely with vagrant post-processor, including template validation., Update docs for each. 2020-05-19 15:52:12 -07:00
Megan Marsh b982d987a7 Add Artifice postprocessor ID to valid artifacts for postprocessors that perform artifact type validation 2020-05-15 13:13:43 -07:00
Wilken Rivera aa2c069067
Update post-processor/alicloud-import/post-processor.go 2020-05-13 10:19:55 -04:00
Adrien Delorme c2d3e494b6 alicloud: better docs + autogenerated post processor docs 2020-05-13 15:49:34 +02:00
Megan Marsh 9476aa03de
Merge pull request #9105 from hashicorp/google_iap
Implement iap proxy for googlecompute
2020-05-08 12:40:42 -07:00
Megan Marsh f67a8ab431 revert unneeded changes to driver roles 2020-05-08 10:13:42 -07:00
Megan Marsh e6073bcec7 implement iap proxy for googlecompute. ssh-only so far 2020-05-08 10:13:42 -07:00
Megan Marsh 7c13cc5744 swap out tag for tags since it's a list
add fixer for docker tag to tags change
2020-05-07 13:24:25 -07:00
Megan Marsh 26d05abd4f
Merge pull request #9182 from hashicorp/fix_9131
support pushing multiple tags
2020-05-07 12:09:19 -07:00
Megan Marsh 841c23d5f9 fix tests 2020-05-06 17:14:27 -07:00
Megan Marsh 02c1cf5b28 support pushing multiple tags 2020-05-06 16:39:41 -07:00
Wilken Rivera dc6de091e7
Merge pull request #9146 from jhawk28/bug_9141
add vsphere builder's artifact to vsphere-template's supported types
2020-05-05 14:27:52 -04:00
Megan Marsh 6a682aca2e
Merge pull request #9124 from GennadySpb/yandex-export-post-processor
[WIP] Yandex export post processor
2020-05-05 11:01:13 -07:00
Megan Marsh 7e0cfd8b77 add undocumented tag to struct-markdown generator so that we can leave internal-use-only fields out of docs 2020-05-01 10:31:38 -07:00
Joshua Foster 00d0632f4a add vsphere builder's artifact to vsphere-template's supported types 2020-05-01 11:17:31 -04:00
Gennady Lipenkov 2658d58507 Fixes 2020-04-27 10:12:41 +03:00
Gennady Lipenkov d5a6781fb7 Add new 'yandex-export' post-processor 2020-04-27 02:20:30 +03:00
Adrien Delorme 73242e8de4 make sure keyvalue and namevalue filters are using the same naming as the service being used 2020-04-16 17:22:13 +02:00
Adrien Delorme 840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Adrien Delorme 2af40c762b
HCL2: pass on builder type and name (#8956)
* sets `packer_build_name` and `packer_builder_type` variables for builder provisioners and post-processors in HCL2
* allows to use the new `${source.type}` and `${source.name}` variables in HCL2
* fixes #8932 

Note that the common.PackerConfig is used everywhere and was not set for HCL2, this had some implications: 

For #8923 you can see the issue here:

dde74232f2/builder/lxd/config.go (L61-L63)

More random examples of where this could cause an issue :

0785c2f6fc/provisioner/ansible-local/provisioner.go (L380-L381)

b4efd13a4d/builder/amazon/ebs/builder.go (L232-L236)



* [All references to PackerConfig.PackerBuildName](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L7:2&tab=references)

* [All references to PackerConfig.PackerBuilderType](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L8:2&tab=references)
2020-04-09 11:14:37 +02:00
Megan Marsh 0f9415297f
Merge pull request #8679 from kwibus/gcp-fallback-image
[WIP] fix #8589 gcp public fallback image
2020-04-02 16:49:50 -07:00
Adrien Delorme a9255c6ebe Update post-processor.go 2020-03-17 16:37:16 +01:00
Adrien Delorme 88297c796d Merge branch 'master' into hcl2_singular_blocks 2020-03-17 14:47:17 +01:00
Adrien Delorme b5e8750f5f Change key/value to name/value to remain consistent with Terraform 2020-03-17 11:23:11 +01:00
Adrien Delorme 87d6b2433f make generate 2020-03-16 15:21:29 +01:00
r_takaishi f50ff1d270 make generate 2020-03-13 11:29:42 +09:00
Wilken Rivera e1a46ec293
Fix gosimple S1025 linting errors (#8838)
Remove unneeded use of fmt.Sprintf for variables that are already strings.
2020-03-04 15:31:30 -05:00
Lars Lehtonen db7a40d6ea
post-processor/digitalocean-import/post-processor.go: replace deprecated session.New() with session.NewSession() 2020-03-03 11:53:40 -08:00
Wilken Rivera 9ec8b67392
Add golangci-lint to project (#8686)
* Add golangci-lint as linting tool

* Disable failing staticchecks to start; GitHub issue to handle coming soon

* Run `goimports -w` to repair all source files that have improperly
formatted imports

* makefile: Add ci-lint target to run on travis

This change adds a new make target for running golangci-lint on newly
added Go files only. This target is expected to run during Packer ci builds.

* .github/contributing: Add code linting instructions

* travis: Update job configuration to run parallel builds
2020-02-14 11:42:29 -05:00
Adrien Delorme 7d5f0c11af add max_retries option to aws builders 2020-02-07 16:36:14 +01:00
Rens Sikma c8062cc8b3 fix #8679 with source_image_project_id as list 2020-02-06 10:30:04 +01:00
Megan Marsh c6dddb75cf
clarify artifice postprocessor wording to make it clear that it will not clean up old artifacts (#8676)
fix #5451
2020-01-31 16:45:18 +01:00
Sylvia Moss dc31bad539
Sharing info with post-processors via artifact (#8632) 2020-01-30 11:27:58 +01:00
Sylvia Moss 0677b02e18
Share SourceImageName with provisioners and manifest post-processor (#8603) 2020-01-16 12:04:03 +01:00
Sylvia Moss 06a26e74b4
Override post-processor config when processing (#8592) 2020-01-14 11:13:16 +01:00
nywilken 5a54c99e98 post-processor/vagrant-cloud: Update error handling for VagrantCloud API responses
This change updates the underlying type of VagrantCloudErrors.Errors to match the expected errors response type and adds a simple test to verify the parsing of various error responses.
Tests before change

```
--- FAIL: TestVagranCloudErrors (0.00s)
    client_test.go:23: failed to decode error response: json: cannot unmarshal array into Go struct field VagrantCloudErrors.errors of type map[string][]string
    client_test.go:23: failed to decode error response: json: cannot unmarshal array into Go struct field VagrantCloudErrors.errors of type map[string][]string
    client_test.go:26: failed to get expected response; expected "", got "error1. error2"
    client_test.go:23: failed to decode error response: json: cannot unmarshal array into Go struct field VagrantCloudErrors.errors of type map[string][]string
    client_test.go:26: failed to get expected response; expected "", got "message Bad credentials"

```

Test after change
```
--- PASS: TestVagranCloudErrors (0.00s)

```
2020-01-13 11:55:53 -05:00
Pit d7d00d8069 Fix regression in docker-tag post-processor (#8593)
GH-8392 introduced the feature that multiple tags can be specified. This
(presumably inadvertently) broke the option to tag an image without the
actual tag specified -- Docker handles this by assuming `latest` as the
tag.

In addition, use-cases exist for the `repository` field containing the
tag already, which was also broken.
2020-01-13 11:41:52 +01:00
Hanjie Wang d9a128a375 In Vagrant post-processor, check whether the host system is able to create a dummy Vagrant box before processing (#8431) 2020-01-08 11:33:39 -08:00
Megan Marsh 49a33c04cd
Merge pull request #8511 from TJM/vsphere_template_markas
[post-processor/vsphere-template] Simplify method to use vm.MarkAsTemplate (optionally)
2020-01-08 09:41:43 -08:00
Tommy McNeely e469140678 workaround for mapstructure-to-hcl2 and booleans 2020-01-07 16:58:40 -07:00
Megan Marsh 21edd085c7
Merge pull request #8560 from alexyueer/feature/support_aliyun_profile_20200103
support aliyun profile
2020-01-07 12:45:54 -08:00
Tommy McNeely 7f7b54ea31 Use vm.MarkAsTemplate, optionally reregister 2020-01-06 15:34:50 -07:00
Rui Lopes ec17b3d8fa correctly handle the diskSize property as a qemu size string 2020-01-05 15:44:01 +00:00
alexyueer ec4a7b6af0 support aliyun profile 2020-01-03 12:17:34 +08:00
zhuzhih2017 66445ecd2d
Merge pull request #8425 from haoshuwei/master
add show expired arg when describe image
2019-12-20 13:33:55 +08:00
haoshuwei 37dc411f30 add skip_image_validation arg to use an expired image as the soure image
Signed-off-by: haoshuwei <haoshuwei24@gmail.com>
2019-12-20 11:28:31 +08:00
Megan Marsh b23425f637 use pointer to config rather than config 2019-12-19 11:27:21 -08:00
Adrien Delorme 4b7132c87c Merge remote-tracking branch 'origin/master' into sharing_info 2019-12-17 11:57:09 +01:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Megan Marsh 0f6d1beccf add an extra string array to the Prepare() return values in the builder interfaces; this sets up the ability for builders to give the provisioners custom user-accessible build-time variables. 2019-12-16 21:23:05 -08:00
Megan Marsh 39fd462b56 change all provision func signatures to use map[string]interface{} 2019-12-12 15:38:32 -08:00
Lars Lehtonen 20d375f705
post-processor/digitalocean-import: drop unused test functions 2019-12-10 06:59:24 -08:00
Lars Lehtonen d7eee558a6 post-processor/docker-import: drop unused test functions (#8465) 2019-12-09 16:05:03 +01:00
Lars Lehtonen 4fe34a5dc3 post-processor/docker-push: drop unused test helper functions (#8456) 2019-12-06 11:07:51 +01:00
Lars Lehtonen 8146b39986 post-processor/docker-save: remove unused test functions (#8447) 2019-12-05 11:28:38 +01:00
Adrien Delorme b4200c3590 go generate for the google compute export post processor after #8388 2019-12-02 11:32:16 +01:00
Taneli Leppä e1cb0c5f7e Added support for specifying Service Account in Google Compute exporter post-processor. (#8388) 2019-12-02 11:29:34 +01:00
henry-aj-jones d317d7f3d1 post-processor/docker-tag: Allow to tag an artifact with multiple tags (#8392) 2019-11-22 11:48:36 +01:00
Lars Lehtonen 2e7ba19d8c post-processor/vagrant-cloud: fix dropped test errors 2019-11-15 11:44:33 +01:00
Megan Marsh fcb65ee422
Merge pull request #8307 from hashicorp/fix_8116
delete amazon import post-processor intermediary snapshots
2019-11-01 14:55:22 -07:00
Megan Marsh e9e4c102e5 delete amazon import post-processor intermediary snapshots 2019-11-01 14:17:44 -07:00
Adrien Delorme 819329228a Change back to make sure all durations are a time.Duration
It is simply the best/simplest solution and trying to prevent users from passing and integer here would be like opening a can of worms. Because:

* we cannot make mapstructure validate our duration string ( with an UnmarshalJSON func etc.)
* we cannot make mapstructure spit a string instead of a duration and packer will decode-encode-decode config.
* the hcl2 generated code asks for a string, so this will be enforced by default.
2019-10-31 16:12:07 +01:00
Adrien Delorme bf3d9841c6 Force durations to be passed a strings
Before this commit it was possible to set a duration using an integer or a float. Go's time.Duration is an int64 internally an mapstructure will take advantage of this and load the number as a int64 but `1` means one ns which is unexpected/confusing. To avoid confusion and enforce readability this forces users to pass a string with a unit for a duration; ex "56s".
2019-10-31 11:47:19 +01:00
mingsheng.su 4f140d0071 fix docs of ucloud-uhost 2019-10-30 15:05:54 +08:00
mingsheng.su 545575b879 optimize code about ufile config 2019-10-29 16:11:44 -07:00
mingsheng.su 37e898e115 optimize code style 2019-10-29 16:11:44 -07:00
mingsheng.su 1099777978 update bucket about no create 2019-10-29 16:11:43 -07:00
mingsheng.su 083d7896f4 update message of ucloud import 2019-10-29 16:11:43 -07:00
mingsheng.su 2c513fce64 update ucloud packer 2019-10-29 16:11:43 -07:00
mingsheng.su 61f9d3a9fe add ucloud import 2019-10-29 16:11:43 -07:00
Lars Lehtonen 5edabfc849 post-processor/compress: fix dropped errors 2019-10-28 16:19:46 -07:00
Adrien Delorme 078ba7c8c3 commit old code generation tool
for history
2019-10-15 12:56:42 +02:00
Adrien Delorme 771a6e55ef pp.google.[import|export]: unexport Account field
so it doesnt temper with hcl2 cfg generation
2019-10-15 12:56:42 +02:00
Adrien Delorme af08e2a410 pp.alicloud-import: remove duplicate AlicloudImageName field
( already defined in packerecs.Config )
2019-10-15 12:56:42 +02:00
Andrew Starr-Bochicchio ae1bb856a3 digitalocean-import: Fix panic when 'image_regions' not set (Fixes: #7843). 2019-10-01 21:14:15 -04:00
Lars Lehtonen 21f2e67462
post-processor/vagrant-cloud: fix dropped errors 2019-09-26 09:23:09 -07:00
Megan Marsh 3c14c50aba Allow user to real oauth token from properly configured Vault instance 2019-09-23 13:26:03 -07:00
Megan Marsh 61b237b6f7
Merge pull request #8097 from onematchfox/vagrant-cloud
post-proc(vagrant-cloud) allow blank access_token (no auth) for private vagrant box hosting
2019-09-19 11:19:57 -07:00
Megan Marsh 9e28b2f5cf fix post-processor 2019-09-16 13:45:28 -07:00
Brian Fox 331babc2ce allow blank access_token (no auth) for private vagrant box hosting 2019-09-12 12:37:53 +02:00
Robbert Müller 9f972375ed
Removed whitelist check for builders
Fixes https://github.com/hashicorp/packer/issues/8060

Signed-off-by: Robbert Müller <spam.me@grols.ch>
2019-09-04 18:34:39 +02:00
DanHam ca9b500e76
Fix test output: Correct copy/paste errors and make message clearer 2019-08-22 09:48:32 +01:00
DanHam 7770ade74c
Clearer error message for artifice users. OCD nits and logging 2019-08-21 14:32:26 +01:00
DanHam e9ab2203ba
Should return provider correctly with artifacts from other builders or pp's 2019-08-21 14:32:25 +01:00
DanHam e8336039d9
Should return provider correctly with good box and artifice pp 2019-08-21 14:32:25 +01:00
DanHam aee400836f
Should return an error if the provider kv pair is not in the metadata file 2019-08-21 14:32:25 +01:00
DanHam a7603f63c7
Should return an error if the metadata file has badly formatted JSON 2019-08-21 14:32:25 +01:00
DanHam 57137c6e33
Should return an error if the value of the provider key is empty 2019-08-21 14:32:25 +01:00