Commit Graph

758 Commits

Author SHA1 Message Date
Chris Roberts d8222b1656
Add support for uploading directly to storage on Vagrant Cloud (#10193)
Vagrant Cloud provides support for uploading directly to the backend
storage instead of streaming through Vagrant Cloud. This adds support
for direct to storage uploads and sets it as the default upload method.
A new option has been added to disable this behavior and revert back
to streaming upload via Vagrant Cloud (`no_direct_upload`).

This default for uploading directly to the backend storage also matches
up with changes being added to Vagrant proper for box upload behavior:
hashicorp/vagrant#11916
2020-11-05 20:01:55 -05:00
Megan Marsh 45b5f0c2ee move custom http client creation with env proxy args into same package as other network related helpers 2020-11-04 14:51:14 -08:00
Megan Marsh 4aa143a429
Merge pull request #10188 from Direnol/yandex/check-access-to-specific-bucket
verifying the access to a specific bucket
2020-11-02 09:21:27 -08:00
Roman Mingazeev 3fc185924d fix delete access key 2020-10-30 12:20:10 +03:00
Roman Mingazeev 7ab2866d5c verifying the access to a specific bucket 2020-10-29 20:53:04 +03:00
Adrien Delorme f5e037e8b4 Merge remote-tracking branch 'origin/master' into azr_selectable_temp_keygen_type_gcp 2020-10-26 15:47:29 +01:00
Megan Marsh 70cb4097e1 review comments, add plugin type for a few missing builders 2020-10-21 16:17:32 -07:00
Megan Marsh c0ce8a9414 rework fixer deprecation code so we know what plugins they relate to 2020-10-21 15:28:59 -07:00
Adrien Delorme 522d122858 regen all code 2020-10-21 12:04:10 +02:00
Adrien Delorme 54032d6c11 gen code and docs 2020-10-15 16:51:23 +02:00
Adrien Delorme f4f731b83c Refactor google compute's temporary ssh keygen genertion to use the sshkey package 2020-10-15 16:51:11 +02:00
Megan Marsh 7ee9a4b638
Merge pull request #9968 from upodroid/google-impersonation
GCP: add service account impersonation
2020-10-06 11:32:03 -07:00
Megan Marsh 721cbac645
Merge pull request #9996 from hashicorp/fix_9995
Make shell-local post-processor return copy of previous artifact
2020-10-05 13:00:51 -07:00
Adrien Delorme 8f9001b531 uncopy 2020-10-02 11:35:20 +02:00
upodroid b1c74f9df0 Merge branch 'master' of github.com:hashicorp/packer into google-impersonation 2020-10-01 23:19:15 +01:00
upodroid f13099edfb rebase the branch 2020-10-01 20:39:06 +01:00
Megan Marsh 4c0ac6917e switch to using aws base config, which means we're using code tested and maitnained by the terraform crowd. 2020-09-29 14:26:38 -07:00
sylviamoss c8874c9382 improve docker_tag cast to avoid failures 2020-09-29 14:41:43 +02:00
sylviamoss 49bc7665c1 fix docker push tags cast 2020-09-28 11:18:24 +02:00
sylviamoss aa9c162c60 improve docker_tags artifact state read 2020-09-28 10:52:28 +02:00
sylviamoss e758891878 make shell-local post-processor return copy of previous artifact 2020-09-25 16:53:59 +02:00
Megan Marsh 429d431365 Implement assume_role for Packer. This functionality is copied shamelessly
from the Terraform AWS provider
2020-09-23 13:18:26 -07:00
upodroid be56632f7e fix vendoring 2020-09-20 16:50:03 +01:00
upodroid 9b121e85f9 fix dodgy pointers 2020-09-20 15:31:45 +01:00
Upo 3f6230470b
Merge branch 'master' into google-impersonation 2020-09-20 15:23:04 +01:00
upodroid 2faacfd5d7 add service account impersonation 2020-09-20 15:18:37 +01:00
Wilken Rivera b4be598148
Fix static check issues SA1019 for googlecompute plugins (#9950)
This change replaces the deprecated constructors `New` with
`NewService`.

Static check before the change
```
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | grep SA1019 | grep google
- post-processor/googlecompute-import/post-processor.go:183:18: SA1019:
storage.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- post-processor/googlecompute-import/post-processor.go:219:18: SA1019:
compute.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- post-processor/googlecompute-import/post-processor.go:273:18: SA1019:
storage.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- builder/googlecompute/driver_gce.go:127:18: SA1019: compute.New is
deprecated: please use NewService instead. To provide a custom HTTP
client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- builder/googlecompute/driver_gce.go:132:25: SA1019: oslogin.New is
deprecated: please use NewService instead. To provide a custom HTTP
client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
```

Static check after change
```
[go-1.15.2] [1] wilken@automaton in ~/Development/packer/
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | grep SA1019 | grep google

```
2020-09-17 10:14:41 +02:00
Sylvia Moss bdd736b800
Interpolate vagrantfile_template on Vagrant post-processor (#9923) 2020-09-16 10:08:44 +02:00
Gennady Lipenkov e078bd6bf2 Fix S3 URL construct process 2020-09-11 16:49:30 +03:00
GennadySpb 2837d56885
Check for error after runner completes (#9925) 2020-09-11 11:14:49 +02:00
Wilken Rivera 10e74961d2
Add check for empty artifact.Files slice (#9857)
* Add check for empty artifact.Files slice

Tests before change
```
⇶  go test ./post-processor/digitalocean-import/... -run=TestPostProcsor_extractImageArtifact
2020/08/31 13:51:25 Looking for image in artifact
--- FAIL: TestPostProcsor_extractImageArtifact (0.00s)
panic: runtime error: index out of range [0] with length 0 [recovered]
        panic: runtime error: index out of range [0] with length 0

goroutine 7 [running]:
testing.tRunner.func1.1(0xfb0300, 0xc000456460)
        /usr/local/go/src/testing/testing.go:940 +0x2f5
testing.tRunner.func1(0xc0003ab560)
        /usr/local/go/src/testing/testing.go:943 +0x3f9
panic(0xfb0300, 0xc000456460)
        /usr/local/go/src/runtime/panic.go:969 +0x166
github.com/hashicorp/packer/post-processor/digitalocean-import.extractImageArtifact(0x0, 0x0, 0x0, 0x24, 0xc000060ea0, 0x453937, 0x1431250)
        /home/wilken/Development/packer/post-processor/digitalocean-import/post-processor.go:262 +0x36d
github.com/hashicorp/packer/post-processor/digitalocean-import.TestPostProcsor_extractImageArtifact(0xc0003ab560)
        /home/wilken/Development/packer/post-processor/digitalocean-import/post-processor_test.go:28 +0x2b0
testing.tRunner(0xc0003ab560, 0x1077208)
        /usr/local/go/src/testing/testing.go:991 +0xdc
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1042 +0x357
FAIL    github.com/hashicorp/packer/post-processor/digitalocean-import  0.009s
FAIL
```

Tests after change
```
[go-1.14.2] [1] wilken@automaton in ~/Development/packer/ on fix_9848 (ahead 1)
⇶  go test ./post-processor/digitalocean-import/... -run=TestPostProcsor_extractImageArtifact
ok      github.com/hashicorp/packer/post-processor/digitalocean-import  0.006s
```

* Update to reflect review feedback
2020-09-01 10:59:01 -04:00
GennadySpb 804fefef17
yandex-import: allow set custom API endpoint (#9850)
* Separate Access Config from yandex builder Config

* make use of Access Config explicit

* Move `MaxRetries` into AccessConfig

* NewDriverYC use AccessConfig instead Config

* yandex-import PP use common Access Config

Now support set custom API Endpoint

* yandex-export PP use common Access Config

Now support set custom API Endpoint too (as yandex-import)

* fix test

* Tiny doc updates.
2020-08-31 15:29:20 +02:00
GennadySpb 0df2e15d9f
If proposed exit code not equal 0 set proper metadata key to expected 'cloud-init-error'. (#9849)
Last one checked at `StepWaitCloudInitScript`.
2020-08-31 14:38:22 +02:00
Viktor A. Danilov cd60f32866
fix yandex-export aws: (#9814)
1. move aws validation before disk image creation
2. add `--region` option
2020-08-26 12:41:05 +02:00
sylviamoss 788dc32598 add aws_polling config option to override env variables 2020-08-17 17:15:32 +02:00
Wilken Rivera 89d57b8635
post-processor/googlecompute-import: Update documentation (#9767) 2020-08-14 12:04:44 +02:00
Megan Marsh daccfc42cf
Fix ReregisterVM to default to true instead of false. (#9736) 2020-08-10 13:13:14 +02:00
Wilken Rivera ae23f891e1
post-processor/googlecompute-export: Update documentation (#9727)
* Add markdown generator to post-processor config
* Add HCL2 example
2020-08-07 16:34:38 -04:00
Wilken Rivera 759d2451a4 post-processor/googlecompute-export: Remove IAP configuration from HCL schema
Build results before change
```
[go-1.14.2] [2] wilken@automaton in
~/Development/packer-templates/googlecompute/googlecompute-startup-scripts/
on master
⇶  packer build -var project=$GOOGLE_PROJECT_ID
googlecompute-ubuntu-startup-script-export.pkr.hcl
Error: Failed preparing post-processor-block "googlecompute-export" ""

  on googlecompute-ubuntu-startup-script-export.pkr.hcl line 28:
    (source code not available)

    1 error occurred:
            * unknown configuration key: '"iap"'

==> Builds finished but no artifacts were created.
```

Build results after change
```
⇶  packer build -var project=$GOOGLE_PROJECT_ID
googlecompute-ubuntu-startup-script-export.pkr.hcl
googlecompute.example: output will be in this color.

==> googlecompute.example: Checking image does not exist...
==> googlecompute.example: Creating temporary SSH key for instance...
```
2020-08-07 11:00:07 -04:00
Luke Swithenbank 6d1a146143
Fix Google Compute Export Post-Processor (#9708)
* Fix Google Compute Export Post-Processor

The current Post-Processor hangs on waiting for the Startup Script to finish. The startup script doesn't update the metadata (StartupScriptStatusKey) so this change aims to fix that but adding the `SetMetadata` functionality to this script.

* Update startup.go
2020-08-06 16:18:23 -04:00
sylviamoss fd683daa22 update checksum post-processor docs 2020-08-05 19:46:17 +02:00
Megan Marsh 02d3fb37e3 fix return 2020-07-27 08:53:33 -07:00
Megan Marsh 69b0e66b5d
Update post-processor/vsphere/post-processor.go
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-07-27 08:52:29 -07:00
Adrien Delorme c34e89aec7
getEncodedPassword: simplify encodedPassword return 2020-07-27 10:33:39 +02:00
Megan Marsh 395a0c472e improve postprocessor stdout 2020-07-25 08:24:21 -07:00
Megan Marsh 6383e6cbbf fix vsphere postprocessor password log filtering, write tests 2020-07-25 08:24:21 -07:00
Megan Marsh 1b6b44589d Change to use query syntax instead of manually throwing queries in.
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-07-23 14:36:41 -07:00
Megan Marsh 393b2d16c4 switch to using net/url.URL struct for encoding rather than the incorrect QueryEscape for the whole URL. 2020-07-23 14:33:09 -07:00
Megan Marsh 2e4d0c888a pull ovftool uri generation into separate function, test function 2020-07-23 14:33:09 -07:00
Megan Marsh a687d6fe12
ci/tech-debt: Fix vmware acceptance tests (#9636)
* add kex algorithm option to ssh config

* regenerate code

* This commit fixes old vmware acceptance tests that have not been run in some time. It does this in two parts:
1) It modifies the minimal vmware build configuration to use a custom kex algorithm, which enables the ssh connection to succeed.
2) It modifies logic in reading and defaulting hardware config values, which was crashing.
3) It adds a new acceptance test with a preseed file to test loading from an http directory.
2020-07-23 09:40:57 +02:00
Gennady Lipenkov 46a5561340 go import file 2020-07-22 11:39:28 +03:00
Gennady Lipenkov 53e7f4e71a extend artifact, code refactoring 2020-07-22 01:01:30 +03:00
Gennady Lipenkov e0cc7b9d8c yandex-import: allow create image based on another one
Allow as source for a new Compute Image not only  URL to Storage Object but also a another one Compute Image
Details at official doc about ImageCreateRequest https://cloud.yandex.com/docs/compute/grpc/image_service#CreateImageRequest
2020-07-20 20:26:22 +03:00
Megan Marsh ea4aa67f78
make friendly error message (#9605) 2020-07-20 16:58:38 +02:00
Gennady Lipenkov b16cccde3d Support piping of yandex-import post-processors 2020-07-17 18:36:48 +03:00
Gennady Lipenkov b28be46dc1 Add package alias (fix lint) 2020-07-17 01:22:28 +03:00
Gennady Lipenkov 29a6687475 Separate workflow for file on disk and prepared URL 2020-07-17 01:09:49 +03:00
Gennady Lipenkov 59aaaf7a91 Support provision of token, SA key file and folder ID from OS env vars
As in yandex-export PP and yandex builder.
2020-07-17 01:04:00 +03:00
Gennady Lipenkov 7f9ba42ee2 Add storage util funcs with test 2020-07-17 00:59:39 +03:00
Gennady Lipenkov bd1e4f9d7d yandex-export post-processor artifact id is url.
First storage path used.
2020-07-17 00:45:19 +03:00
Gennady Lipenkov 0092007311 Forms urls and support get url for yandex-export artifact 2020-07-16 18:01:18 +03:00
Gennady Lipenkov fab42a061e Update doc and example. 2020-07-16 18:01:15 +03:00
Gennady Lipenkov f0e1b719d8 Forms urls and support get url for yandex-export artifact 2020-07-16 17:59:21 +03:00
Gennady Lipenkov 8442d570e5 Allow work with yandex-export artifact 2020-07-16 17:57:53 +03:00
zhsj e2cbe0de48
post-processor/vagrant-cloud: missing vagrant_cloud_url in test (#9572) 2020-07-14 10:29:28 +02:00
Lars Lehtonen 1400662db7
post-processor/digitalocean-import: replace deprecated oauth2.NoContext (#9566) 2020-07-13 17:13:35 +02:00
Roger Hu 28f8241a05 update generated files 2020-07-10 16:58:06 -07:00
Gennady Lipenkov d83d2b0273 Update documentation 2020-07-09 17:43:11 +03:00
GennadySpb 446d259459 Update post-processor/yandex-export/post-processor.go
Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-07-09 16:21:21 +03:00
Gennady Lipenkov b02401c41b Update doc and example. 2020-07-09 16:21:21 +03:00
Gennady Lipenkov 5762367de1 Provide 'GeneratedData' value for step 'StepCreateInstance' 2020-07-09 16:21:21 +03:00
Gennady Lipenkov 099dd3e7b3 Validate and render 'paths' config value 2020-07-09 16:21:21 +03:00
GennadySpb 919b54f4f4
yandex-export: tiny improvements (#9554) 2020-07-09 10:46:51 +02:00
Gennady Lipenkov e5a4155378 Update doc 2020-07-09 00:24:53 +03:00
Gennady Lipenkov 64cda51bff Image name not required attr 2020-07-08 23:01:25 +03:00
Gennady Lipenkov 32b77f3b80 Update doc pages 2020-07-08 22:54:20 +03:00
Gennady Lipenkov 409534738a Add error check 2020-07-08 22:22:55 +03:00
Gennady Lipenkov 553d203e68 Add yandex-import post-processor 2020-07-08 22:01:52 +03:00
Megan Marsh 673858a63c
Merge pull request #9513 from hashicorp/undocument_ssh_temp_key
Undocument ssh options not used by all builders
2020-07-07 14:26:21 -07:00
Geoff Schultz 8010a23b34 Adding retry for ImportImage call in amazon-import post-processor. 2020-07-06 09:41:13 -05:00
Moss 8ea64c5b35 undocument ssh_keypair_name and ssh_agent_auth 2020-07-02 14:55:58 +02:00
Moss 41edd09d3c undocument temporary_key_pair_name and ssh_private_key_file 2020-07-01 16:33:18 +02:00
Sylvia Moss 99046c9178
Share build info with Provisioner and Post-Processor via HCL2 variables (#9444) 2020-06-29 13:44:57 +02:00
mingsheng.su 8a8abdf615
update ucloud-uhost builder (#9466)
* update ucloud-uhost builder, docs and deps

Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-06-24 14:31:05 +02:00
Megan Marsh 4f77bb0c4a add new ssh_ciphers option 2020-06-17 16:48:23 -07:00
Gennady Lipenkov 56224ae08d Instance SA auth for yandex-export post-processor 2020-06-17 18:15:53 +03:00
Megan Marsh 4178625afb re-add winrm_no_proxy option. Winrm code needs to be modified in order to read no_proxy env var when client is created rather than when net/http library is instantiated. 2020-06-16 10:27:30 -07:00
Megan Marsh a0a3ddbadb
Merge pull request #9379 from GennadySpb/sa-auth-in-yandex-export
Support Authentication by Service Account Key file in Yandex Export post-processor
2020-06-10 09:40:02 -07:00
GennadySpb 79770f919a
Fix error message (#9373) 2020-06-10 13:29:20 +02:00
Gennady Lipenkov 85da85271f Support SA Key authentication in Yandex Export post-processor 2020-06-09 11:06:12 +03:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Megan Marsh 4041cb7775 generated code 2020-05-19 16:10:51 -07:00
Megan Marsh 557eb015ea make artifice post-processor play nicely with vagrant post-processor, including template validation., Update docs for each. 2020-05-19 15:52:12 -07:00
Megan Marsh b982d987a7 Add Artifice postprocessor ID to valid artifacts for postprocessors that perform artifact type validation 2020-05-15 13:13:43 -07:00
Wilken Rivera aa2c069067
Update post-processor/alicloud-import/post-processor.go 2020-05-13 10:19:55 -04:00
Adrien Delorme c2d3e494b6 alicloud: better docs + autogenerated post processor docs 2020-05-13 15:49:34 +02:00
Megan Marsh 9476aa03de
Merge pull request #9105 from hashicorp/google_iap
Implement iap proxy for googlecompute
2020-05-08 12:40:42 -07:00
Megan Marsh f67a8ab431 revert unneeded changes to driver roles 2020-05-08 10:13:42 -07:00
Megan Marsh e6073bcec7 implement iap proxy for googlecompute. ssh-only so far 2020-05-08 10:13:42 -07:00
Megan Marsh 7c13cc5744 swap out tag for tags since it's a list
add fixer for docker tag to tags change
2020-05-07 13:24:25 -07:00