Commit Graph

727 Commits

Author SHA1 Message Date
GennadySpb 804fefef17
yandex-import: allow set custom API endpoint (#9850)
* Separate Access Config from yandex builder Config

* make use of Access Config explicit

* Move `MaxRetries` into AccessConfig

* NewDriverYC use AccessConfig instead Config

* yandex-import PP use common Access Config

Now support set custom API Endpoint

* yandex-export PP use common Access Config

Now support set custom API Endpoint too (as yandex-import)

* fix test

* Tiny doc updates.
2020-08-31 15:29:20 +02:00
GennadySpb 0df2e15d9f
If proposed exit code not equal 0 set proper metadata key to expected 'cloud-init-error'. (#9849)
Last one checked at `StepWaitCloudInitScript`.
2020-08-31 14:38:22 +02:00
Viktor A. Danilov cd60f32866
fix yandex-export aws: (#9814)
1. move aws validation before disk image creation
2. add `--region` option
2020-08-26 12:41:05 +02:00
sylviamoss 788dc32598 add aws_polling config option to override env variables 2020-08-17 17:15:32 +02:00
Wilken Rivera 89d57b8635
post-processor/googlecompute-import: Update documentation (#9767) 2020-08-14 12:04:44 +02:00
Megan Marsh daccfc42cf
Fix ReregisterVM to default to true instead of false. (#9736) 2020-08-10 13:13:14 +02:00
Wilken Rivera ae23f891e1
post-processor/googlecompute-export: Update documentation (#9727)
* Add markdown generator to post-processor config
* Add HCL2 example
2020-08-07 16:34:38 -04:00
Wilken Rivera 759d2451a4 post-processor/googlecompute-export: Remove IAP configuration from HCL schema
Build results before change
```
[go-1.14.2] [2] wilken@automaton in
~/Development/packer-templates/googlecompute/googlecompute-startup-scripts/
on master
⇶  packer build -var project=$GOOGLE_PROJECT_ID
googlecompute-ubuntu-startup-script-export.pkr.hcl
Error: Failed preparing post-processor-block "googlecompute-export" ""

  on googlecompute-ubuntu-startup-script-export.pkr.hcl line 28:
    (source code not available)

    1 error occurred:
            * unknown configuration key: '"iap"'

==> Builds finished but no artifacts were created.
```

Build results after change
```
⇶  packer build -var project=$GOOGLE_PROJECT_ID
googlecompute-ubuntu-startup-script-export.pkr.hcl
googlecompute.example: output will be in this color.

==> googlecompute.example: Checking image does not exist...
==> googlecompute.example: Creating temporary SSH key for instance...
```
2020-08-07 11:00:07 -04:00
Luke Swithenbank 6d1a146143
Fix Google Compute Export Post-Processor (#9708)
* Fix Google Compute Export Post-Processor

The current Post-Processor hangs on waiting for the Startup Script to finish. The startup script doesn't update the metadata (StartupScriptStatusKey) so this change aims to fix that but adding the `SetMetadata` functionality to this script.

* Update startup.go
2020-08-06 16:18:23 -04:00
sylviamoss fd683daa22 update checksum post-processor docs 2020-08-05 19:46:17 +02:00
Megan Marsh 02d3fb37e3 fix return 2020-07-27 08:53:33 -07:00
Megan Marsh 69b0e66b5d
Update post-processor/vsphere/post-processor.go
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-07-27 08:52:29 -07:00
Adrien Delorme c34e89aec7
getEncodedPassword: simplify encodedPassword return 2020-07-27 10:33:39 +02:00
Megan Marsh 395a0c472e improve postprocessor stdout 2020-07-25 08:24:21 -07:00
Megan Marsh 6383e6cbbf fix vsphere postprocessor password log filtering, write tests 2020-07-25 08:24:21 -07:00
Megan Marsh 1b6b44589d Change to use query syntax instead of manually throwing queries in.
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-07-23 14:36:41 -07:00
Megan Marsh 393b2d16c4 switch to using net/url.URL struct for encoding rather than the incorrect QueryEscape for the whole URL. 2020-07-23 14:33:09 -07:00
Megan Marsh 2e4d0c888a pull ovftool uri generation into separate function, test function 2020-07-23 14:33:09 -07:00
Megan Marsh a687d6fe12
ci/tech-debt: Fix vmware acceptance tests (#9636)
* add kex algorithm option to ssh config

* regenerate code

* This commit fixes old vmware acceptance tests that have not been run in some time. It does this in two parts:
1) It modifies the minimal vmware build configuration to use a custom kex algorithm, which enables the ssh connection to succeed.
2) It modifies logic in reading and defaulting hardware config values, which was crashing.
3) It adds a new acceptance test with a preseed file to test loading from an http directory.
2020-07-23 09:40:57 +02:00
Gennady Lipenkov 46a5561340 go import file 2020-07-22 11:39:28 +03:00
Gennady Lipenkov 53e7f4e71a extend artifact, code refactoring 2020-07-22 01:01:30 +03:00
Gennady Lipenkov e0cc7b9d8c yandex-import: allow create image based on another one
Allow as source for a new Compute Image not only  URL to Storage Object but also a another one Compute Image
Details at official doc about ImageCreateRequest https://cloud.yandex.com/docs/compute/grpc/image_service#CreateImageRequest
2020-07-20 20:26:22 +03:00
Megan Marsh ea4aa67f78
make friendly error message (#9605) 2020-07-20 16:58:38 +02:00
Gennady Lipenkov b16cccde3d Support piping of yandex-import post-processors 2020-07-17 18:36:48 +03:00
Gennady Lipenkov b28be46dc1 Add package alias (fix lint) 2020-07-17 01:22:28 +03:00
Gennady Lipenkov 29a6687475 Separate workflow for file on disk and prepared URL 2020-07-17 01:09:49 +03:00
Gennady Lipenkov 59aaaf7a91 Support provision of token, SA key file and folder ID from OS env vars
As in yandex-export PP and yandex builder.
2020-07-17 01:04:00 +03:00
Gennady Lipenkov 7f9ba42ee2 Add storage util funcs with test 2020-07-17 00:59:39 +03:00
Gennady Lipenkov bd1e4f9d7d yandex-export post-processor artifact id is url.
First storage path used.
2020-07-17 00:45:19 +03:00
Gennady Lipenkov 0092007311 Forms urls and support get url for yandex-export artifact 2020-07-16 18:01:18 +03:00
Gennady Lipenkov fab42a061e Update doc and example. 2020-07-16 18:01:15 +03:00
Gennady Lipenkov f0e1b719d8 Forms urls and support get url for yandex-export artifact 2020-07-16 17:59:21 +03:00
Gennady Lipenkov 8442d570e5 Allow work with yandex-export artifact 2020-07-16 17:57:53 +03:00
zhsj e2cbe0de48
post-processor/vagrant-cloud: missing vagrant_cloud_url in test (#9572) 2020-07-14 10:29:28 +02:00
Lars Lehtonen 1400662db7
post-processor/digitalocean-import: replace deprecated oauth2.NoContext (#9566) 2020-07-13 17:13:35 +02:00
Roger Hu 28f8241a05 update generated files 2020-07-10 16:58:06 -07:00
Gennady Lipenkov d83d2b0273 Update documentation 2020-07-09 17:43:11 +03:00
GennadySpb 446d259459 Update post-processor/yandex-export/post-processor.go
Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-07-09 16:21:21 +03:00
Gennady Lipenkov b02401c41b Update doc and example. 2020-07-09 16:21:21 +03:00
Gennady Lipenkov 5762367de1 Provide 'GeneratedData' value for step 'StepCreateInstance' 2020-07-09 16:21:21 +03:00
Gennady Lipenkov 099dd3e7b3 Validate and render 'paths' config value 2020-07-09 16:21:21 +03:00
GennadySpb 919b54f4f4
yandex-export: tiny improvements (#9554) 2020-07-09 10:46:51 +02:00
Gennady Lipenkov e5a4155378 Update doc 2020-07-09 00:24:53 +03:00
Gennady Lipenkov 64cda51bff Image name not required attr 2020-07-08 23:01:25 +03:00
Gennady Lipenkov 32b77f3b80 Update doc pages 2020-07-08 22:54:20 +03:00
Gennady Lipenkov 409534738a Add error check 2020-07-08 22:22:55 +03:00
Gennady Lipenkov 553d203e68 Add yandex-import post-processor 2020-07-08 22:01:52 +03:00
Megan Marsh 673858a63c
Merge pull request #9513 from hashicorp/undocument_ssh_temp_key
Undocument ssh options not used by all builders
2020-07-07 14:26:21 -07:00
Geoff Schultz 8010a23b34 Adding retry for ImportImage call in amazon-import post-processor. 2020-07-06 09:41:13 -05:00
Moss 8ea64c5b35 undocument ssh_keypair_name and ssh_agent_auth 2020-07-02 14:55:58 +02:00
Moss 41edd09d3c undocument temporary_key_pair_name and ssh_private_key_file 2020-07-01 16:33:18 +02:00
Sylvia Moss 99046c9178
Share build info with Provisioner and Post-Processor via HCL2 variables (#9444) 2020-06-29 13:44:57 +02:00
mingsheng.su 8a8abdf615
update ucloud-uhost builder (#9466)
* update ucloud-uhost builder, docs and deps

Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-06-24 14:31:05 +02:00
Megan Marsh 4f77bb0c4a add new ssh_ciphers option 2020-06-17 16:48:23 -07:00
Gennady Lipenkov 56224ae08d Instance SA auth for yandex-export post-processor 2020-06-17 18:15:53 +03:00
Megan Marsh 4178625afb re-add winrm_no_proxy option. Winrm code needs to be modified in order to read no_proxy env var when client is created rather than when net/http library is instantiated. 2020-06-16 10:27:30 -07:00
Megan Marsh a0a3ddbadb
Merge pull request #9379 from GennadySpb/sa-auth-in-yandex-export
Support Authentication by Service Account Key file in Yandex Export post-processor
2020-06-10 09:40:02 -07:00
GennadySpb 79770f919a
Fix error message (#9373) 2020-06-10 13:29:20 +02:00
Gennady Lipenkov 85da85271f Support SA Key authentication in Yandex Export post-processor 2020-06-09 11:06:12 +03:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Megan Marsh 4041cb7775 generated code 2020-05-19 16:10:51 -07:00
Megan Marsh 557eb015ea make artifice post-processor play nicely with vagrant post-processor, including template validation., Update docs for each. 2020-05-19 15:52:12 -07:00
Megan Marsh b982d987a7 Add Artifice postprocessor ID to valid artifacts for postprocessors that perform artifact type validation 2020-05-15 13:13:43 -07:00
Wilken Rivera aa2c069067
Update post-processor/alicloud-import/post-processor.go 2020-05-13 10:19:55 -04:00
Adrien Delorme c2d3e494b6 alicloud: better docs + autogenerated post processor docs 2020-05-13 15:49:34 +02:00
Megan Marsh 9476aa03de
Merge pull request #9105 from hashicorp/google_iap
Implement iap proxy for googlecompute
2020-05-08 12:40:42 -07:00
Megan Marsh f67a8ab431 revert unneeded changes to driver roles 2020-05-08 10:13:42 -07:00
Megan Marsh e6073bcec7 implement iap proxy for googlecompute. ssh-only so far 2020-05-08 10:13:42 -07:00
Megan Marsh 7c13cc5744 swap out tag for tags since it's a list
add fixer for docker tag to tags change
2020-05-07 13:24:25 -07:00
Megan Marsh 26d05abd4f
Merge pull request #9182 from hashicorp/fix_9131
support pushing multiple tags
2020-05-07 12:09:19 -07:00
Megan Marsh 841c23d5f9 fix tests 2020-05-06 17:14:27 -07:00
Megan Marsh 02c1cf5b28 support pushing multiple tags 2020-05-06 16:39:41 -07:00
Wilken Rivera dc6de091e7
Merge pull request #9146 from jhawk28/bug_9141
add vsphere builder's artifact to vsphere-template's supported types
2020-05-05 14:27:52 -04:00
Megan Marsh 6a682aca2e
Merge pull request #9124 from GennadySpb/yandex-export-post-processor
[WIP] Yandex export post processor
2020-05-05 11:01:13 -07:00
Megan Marsh 7e0cfd8b77 add undocumented tag to struct-markdown generator so that we can leave internal-use-only fields out of docs 2020-05-01 10:31:38 -07:00
Joshua Foster 00d0632f4a add vsphere builder's artifact to vsphere-template's supported types 2020-05-01 11:17:31 -04:00
Gennady Lipenkov 2658d58507 Fixes 2020-04-27 10:12:41 +03:00
Gennady Lipenkov d5a6781fb7 Add new 'yandex-export' post-processor 2020-04-27 02:20:30 +03:00
Adrien Delorme 73242e8de4 make sure keyvalue and namevalue filters are using the same naming as the service being used 2020-04-16 17:22:13 +02:00
Adrien Delorme 840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Adrien Delorme 2af40c762b
HCL2: pass on builder type and name (#8956)
* sets `packer_build_name` and `packer_builder_type` variables for builder provisioners and post-processors in HCL2
* allows to use the new `${source.type}` and `${source.name}` variables in HCL2
* fixes #8932 

Note that the common.PackerConfig is used everywhere and was not set for HCL2, this had some implications: 

For #8923 you can see the issue here:

dde74232f2/builder/lxd/config.go (L61-L63)

More random examples of where this could cause an issue :

0785c2f6fc/provisioner/ansible-local/provisioner.go (L380-L381)

b4efd13a4d/builder/amazon/ebs/builder.go (L232-L236)



* [All references to PackerConfig.PackerBuildName](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L7:2&tab=references)

* [All references to PackerConfig.PackerBuilderType](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L8:2&tab=references)
2020-04-09 11:14:37 +02:00
Megan Marsh 0f9415297f
Merge pull request #8679 from kwibus/gcp-fallback-image
[WIP] fix #8589 gcp public fallback image
2020-04-02 16:49:50 -07:00
Adrien Delorme a9255c6ebe Update post-processor.go 2020-03-17 16:37:16 +01:00
Adrien Delorme 88297c796d Merge branch 'master' into hcl2_singular_blocks 2020-03-17 14:47:17 +01:00
Adrien Delorme b5e8750f5f Change key/value to name/value to remain consistent with Terraform 2020-03-17 11:23:11 +01:00
Adrien Delorme 87d6b2433f make generate 2020-03-16 15:21:29 +01:00
r_takaishi f50ff1d270 make generate 2020-03-13 11:29:42 +09:00
Wilken Rivera e1a46ec293
Fix gosimple S1025 linting errors (#8838)
Remove unneeded use of fmt.Sprintf for variables that are already strings.
2020-03-04 15:31:30 -05:00
Lars Lehtonen db7a40d6ea
post-processor/digitalocean-import/post-processor.go: replace deprecated session.New() with session.NewSession() 2020-03-03 11:53:40 -08:00
Wilken Rivera 9ec8b67392
Add golangci-lint to project (#8686)
* Add golangci-lint as linting tool

* Disable failing staticchecks to start; GitHub issue to handle coming soon

* Run `goimports -w` to repair all source files that have improperly
formatted imports

* makefile: Add ci-lint target to run on travis

This change adds a new make target for running golangci-lint on newly
added Go files only. This target is expected to run during Packer ci builds.

* .github/contributing: Add code linting instructions

* travis: Update job configuration to run parallel builds
2020-02-14 11:42:29 -05:00
Adrien Delorme 7d5f0c11af add max_retries option to aws builders 2020-02-07 16:36:14 +01:00
Rens Sikma c8062cc8b3 fix #8679 with source_image_project_id as list 2020-02-06 10:30:04 +01:00
Megan Marsh c6dddb75cf
clarify artifice postprocessor wording to make it clear that it will not clean up old artifacts (#8676)
fix #5451
2020-01-31 16:45:18 +01:00
Sylvia Moss dc31bad539
Sharing info with post-processors via artifact (#8632) 2020-01-30 11:27:58 +01:00
Sylvia Moss 0677b02e18
Share SourceImageName with provisioners and manifest post-processor (#8603) 2020-01-16 12:04:03 +01:00
Sylvia Moss 06a26e74b4
Override post-processor config when processing (#8592) 2020-01-14 11:13:16 +01:00
nywilken 5a54c99e98 post-processor/vagrant-cloud: Update error handling for VagrantCloud API responses
This change updates the underlying type of VagrantCloudErrors.Errors to match the expected errors response type and adds a simple test to verify the parsing of various error responses.
Tests before change

```
--- FAIL: TestVagranCloudErrors (0.00s)
    client_test.go:23: failed to decode error response: json: cannot unmarshal array into Go struct field VagrantCloudErrors.errors of type map[string][]string
    client_test.go:23: failed to decode error response: json: cannot unmarshal array into Go struct field VagrantCloudErrors.errors of type map[string][]string
    client_test.go:26: failed to get expected response; expected "", got "error1. error2"
    client_test.go:23: failed to decode error response: json: cannot unmarshal array into Go struct field VagrantCloudErrors.errors of type map[string][]string
    client_test.go:26: failed to get expected response; expected "", got "message Bad credentials"

```

Test after change
```
--- PASS: TestVagranCloudErrors (0.00s)

```
2020-01-13 11:55:53 -05:00
Pit d7d00d8069 Fix regression in docker-tag post-processor (#8593)
GH-8392 introduced the feature that multiple tags can be specified. This
(presumably inadvertently) broke the option to tag an image without the
actual tag specified -- Docker handles this by assuming `latest` as the
tag.

In addition, use-cases exist for the `repository` field containing the
tag already, which was also broken.
2020-01-13 11:41:52 +01:00
Hanjie Wang d9a128a375 In Vagrant post-processor, check whether the host system is able to create a dummy Vagrant box before processing (#8431) 2020-01-08 11:33:39 -08:00
Megan Marsh 49a33c04cd
Merge pull request #8511 from TJM/vsphere_template_markas
[post-processor/vsphere-template] Simplify method to use vm.MarkAsTemplate (optionally)
2020-01-08 09:41:43 -08:00