Commit Graph

785 Commits

Author SHA1 Message Date
Megan Marsh 14bdb9516c move Artifact and artifact mock to the sdk 2020-11-30 14:29:52 -08:00
Megan Marsh c70870cb83 move multierror and multierrorappend into sdk 2020-11-30 14:29:06 -08:00
Megan Marsh 001886670d move Ui definition into the packer plugin sdk. 2020-11-30 14:26:54 -08:00
Megan Marsh 3bda89467d
Merge pull request #10303 from GennadySpb/yndx-export-wait-and-retries
yandex-export: Add retries and wait after disk attach operation
2020-11-30 10:14:25 -08:00
Pavlos Parissis fac8ba2040
googlecompute-export: Add logging.write to service account scopes 2020-11-30 10:49:34 +01:00
Gennady Lipenkov 4c3e310f57 manually trigger udev 2020-11-26 17:29:33 +03:00
Gennady Lipenkov ef67fb1cc8 Some wait before check 2020-11-25 15:45:49 +03:00
Roman Mingazeev a509781a03 added check of service account id in export 2020-11-24 20:04:37 +03:00
Gennady Lipenkov 138e3a081b Add retries and wait after disk attach operation 2020-11-24 19:15:46 +03:00
Megan Marsh e34248a45d
Merge pull request #10172 from anish/gcpshieldedvms
Add support for creating shielded VMs to GCP
2020-11-20 14:46:51 -08:00
Megan Marsh 030f40a7b6
Merge pull request #10286 from hashicorp/hcl2template_extraction
move hcl2template kv types into the config/custom_types
2020-11-20 12:48:26 -08:00
Megan Marsh ea1ac530b0 move hcl2template kv types into the config/custom_types with trilean definitions 2020-11-18 15:46:59 -08:00
Megan Marsh cbaecf9077 remove unused SkipValidation field which was a duplicate of the SkipRegionValidation field.
remove duplicate temporary key pair name
change communicator code
2020-11-18 14:21:40 -08:00
Megan Marsh f8f1ebf0c7 move version and useragent definitions into sdk 2020-11-18 11:42:02 -08:00
Megan Marsh 9e47696dff move helper/config to bundle with the other template definitions 2020-11-18 10:34:59 -08:00
Anish Bhatt ecb72663f3 Add support for creating shielded VMs to GCP 2020-11-17 18:37:46 -08:00
Megan Marsh 3f6deb8828 move multistep into the plugin sdk 2020-11-17 16:31:03 -08:00
Megan Marsh 7d658149ea update tests and website and generated partials to reflect new package format 2020-11-17 10:36:01 -08:00
Megan Marsh 1ffc358ada move tmp helper func into packer-plugin-sdk 2020-11-16 15:13:08 -08:00
Megan Marsh e04eaa89e6 move steps and step configs from packer/common to packer/common/commonsteps 2020-11-13 11:15:04 -08:00
Megan Marsh 6402362018 create packer-plugin-sdk directory and begin moving the relevant folders into it. 2020-11-13 10:47:36 -08:00
Megan Marsh 2167118126 add version files for all postprocessors 2020-11-10 14:55:45 -08:00
Megan Marsh 3db40194af version files 2020-11-10 14:48:06 -08:00
Megan Marsh 8ecd16db81 regenerate code 2020-11-10 08:16:59 -08:00
Megan Marsh 1ecd715221 refactor code so that packer/version is not a dependency of sdk code or plugin code 2020-11-09 17:20:42 -08:00
Megan Marsh efc117fb55
Merge pull request #10218 from hashicorp/clean_plugin_sdk
Clean plugin sdk
2020-11-06 12:35:23 -08:00
Megan Marsh 2d90ffe7a4 move GeneratedData struct and PlaceholderMsg to same package under common 2020-11-06 10:45:38 -08:00
Chris Roberts d8222b1656
Add support for uploading directly to storage on Vagrant Cloud (#10193)
Vagrant Cloud provides support for uploading directly to the backend
storage instead of streaming through Vagrant Cloud. This adds support
for direct to storage uploads and sets it as the default upload method.
A new option has been added to disable this behavior and revert back
to streaming upload via Vagrant Cloud (`no_direct_upload`).

This default for uploading directly to the backend storage also matches
up with changes being added to Vagrant proper for box upload behavior:
hashicorp/vagrant#11916
2020-11-05 20:01:55 -05:00
Megan Marsh 45b5f0c2ee move custom http client creation with env proxy args into same package as other network related helpers 2020-11-04 14:51:14 -08:00
Megan Marsh 4aa143a429
Merge pull request #10188 from Direnol/yandex/check-access-to-specific-bucket
verifying the access to a specific bucket
2020-11-02 09:21:27 -08:00
Roman Mingazeev 3fc185924d fix delete access key 2020-10-30 12:20:10 +03:00
Roman Mingazeev 7ab2866d5c verifying the access to a specific bucket 2020-10-29 20:53:04 +03:00
Adrien Delorme f5e037e8b4 Merge remote-tracking branch 'origin/master' into azr_selectable_temp_keygen_type_gcp 2020-10-26 15:47:29 +01:00
Megan Marsh 70cb4097e1 review comments, add plugin type for a few missing builders 2020-10-21 16:17:32 -07:00
Megan Marsh c0ce8a9414 rework fixer deprecation code so we know what plugins they relate to 2020-10-21 15:28:59 -07:00
Adrien Delorme 522d122858 regen all code 2020-10-21 12:04:10 +02:00
Adrien Delorme 54032d6c11 gen code and docs 2020-10-15 16:51:23 +02:00
Adrien Delorme f4f731b83c Refactor google compute's temporary ssh keygen genertion to use the sshkey package 2020-10-15 16:51:11 +02:00
Megan Marsh 7ee9a4b638
Merge pull request #9968 from upodroid/google-impersonation
GCP: add service account impersonation
2020-10-06 11:32:03 -07:00
Megan Marsh 721cbac645
Merge pull request #9996 from hashicorp/fix_9995
Make shell-local post-processor return copy of previous artifact
2020-10-05 13:00:51 -07:00
Adrien Delorme 8f9001b531 uncopy 2020-10-02 11:35:20 +02:00
upodroid b1c74f9df0 Merge branch 'master' of github.com:hashicorp/packer into google-impersonation 2020-10-01 23:19:15 +01:00
upodroid f13099edfb rebase the branch 2020-10-01 20:39:06 +01:00
Megan Marsh 4c0ac6917e switch to using aws base config, which means we're using code tested and maitnained by the terraform crowd. 2020-09-29 14:26:38 -07:00
sylviamoss c8874c9382 improve docker_tag cast to avoid failures 2020-09-29 14:41:43 +02:00
sylviamoss 49bc7665c1 fix docker push tags cast 2020-09-28 11:18:24 +02:00
sylviamoss aa9c162c60 improve docker_tags artifact state read 2020-09-28 10:52:28 +02:00
sylviamoss e758891878 make shell-local post-processor return copy of previous artifact 2020-09-25 16:53:59 +02:00
Megan Marsh 429d431365 Implement assume_role for Packer. This functionality is copied shamelessly
from the Terraform AWS provider
2020-09-23 13:18:26 -07:00
upodroid be56632f7e fix vendoring 2020-09-20 16:50:03 +01:00
upodroid 9b121e85f9 fix dodgy pointers 2020-09-20 15:31:45 +01:00
Upo 3f6230470b
Merge branch 'master' into google-impersonation 2020-09-20 15:23:04 +01:00
upodroid 2faacfd5d7 add service account impersonation 2020-09-20 15:18:37 +01:00
Wilken Rivera b4be598148
Fix static check issues SA1019 for googlecompute plugins (#9950)
This change replaces the deprecated constructors `New` with
`NewService`.

Static check before the change
```
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | grep SA1019 | grep google
- post-processor/googlecompute-import/post-processor.go:183:18: SA1019:
storage.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- post-processor/googlecompute-import/post-processor.go:219:18: SA1019:
compute.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- post-processor/googlecompute-import/post-processor.go:273:18: SA1019:
storage.New is deprecated: please use NewService instead. To provide a
custom HTTP client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- builder/googlecompute/driver_gce.go:127:18: SA1019: compute.New is
deprecated: please use NewService instead. To provide a custom HTTP
client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
- builder/googlecompute/driver_gce.go:132:25: SA1019: oslogin.New is
deprecated: please use NewService instead. To provide a custom HTTP
client, use option.WithHTTPClient. If you are using
google.golang.org/api/googleapis/transport.APIKey, use option.WithAPIKey
with NewService instead.  (staticcheck)
```

Static check after change
```
[go-1.15.2] [1] wilken@automaton in ~/Development/packer/
⇶  golangci-lint run --disable-all --no-config --enable=staticcheck | grep SA1019 | grep google

```
2020-09-17 10:14:41 +02:00
Sylvia Moss bdd736b800
Interpolate vagrantfile_template on Vagrant post-processor (#9923) 2020-09-16 10:08:44 +02:00
Gennady Lipenkov e078bd6bf2 Fix S3 URL construct process 2020-09-11 16:49:30 +03:00
GennadySpb 2837d56885
Check for error after runner completes (#9925) 2020-09-11 11:14:49 +02:00
Wilken Rivera 10e74961d2
Add check for empty artifact.Files slice (#9857)
* Add check for empty artifact.Files slice

Tests before change
```
⇶  go test ./post-processor/digitalocean-import/... -run=TestPostProcsor_extractImageArtifact
2020/08/31 13:51:25 Looking for image in artifact
--- FAIL: TestPostProcsor_extractImageArtifact (0.00s)
panic: runtime error: index out of range [0] with length 0 [recovered]
        panic: runtime error: index out of range [0] with length 0

goroutine 7 [running]:
testing.tRunner.func1.1(0xfb0300, 0xc000456460)
        /usr/local/go/src/testing/testing.go:940 +0x2f5
testing.tRunner.func1(0xc0003ab560)
        /usr/local/go/src/testing/testing.go:943 +0x3f9
panic(0xfb0300, 0xc000456460)
        /usr/local/go/src/runtime/panic.go:969 +0x166
github.com/hashicorp/packer/post-processor/digitalocean-import.extractImageArtifact(0x0, 0x0, 0x0, 0x24, 0xc000060ea0, 0x453937, 0x1431250)
        /home/wilken/Development/packer/post-processor/digitalocean-import/post-processor.go:262 +0x36d
github.com/hashicorp/packer/post-processor/digitalocean-import.TestPostProcsor_extractImageArtifact(0xc0003ab560)
        /home/wilken/Development/packer/post-processor/digitalocean-import/post-processor_test.go:28 +0x2b0
testing.tRunner(0xc0003ab560, 0x1077208)
        /usr/local/go/src/testing/testing.go:991 +0xdc
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1042 +0x357
FAIL    github.com/hashicorp/packer/post-processor/digitalocean-import  0.009s
FAIL
```

Tests after change
```
[go-1.14.2] [1] wilken@automaton in ~/Development/packer/ on fix_9848 (ahead 1)
⇶  go test ./post-processor/digitalocean-import/... -run=TestPostProcsor_extractImageArtifact
ok      github.com/hashicorp/packer/post-processor/digitalocean-import  0.006s
```

* Update to reflect review feedback
2020-09-01 10:59:01 -04:00
GennadySpb 804fefef17
yandex-import: allow set custom API endpoint (#9850)
* Separate Access Config from yandex builder Config

* make use of Access Config explicit

* Move `MaxRetries` into AccessConfig

* NewDriverYC use AccessConfig instead Config

* yandex-import PP use common Access Config

Now support set custom API Endpoint

* yandex-export PP use common Access Config

Now support set custom API Endpoint too (as yandex-import)

* fix test

* Tiny doc updates.
2020-08-31 15:29:20 +02:00
GennadySpb 0df2e15d9f
If proposed exit code not equal 0 set proper metadata key to expected 'cloud-init-error'. (#9849)
Last one checked at `StepWaitCloudInitScript`.
2020-08-31 14:38:22 +02:00
Viktor A. Danilov cd60f32866
fix yandex-export aws: (#9814)
1. move aws validation before disk image creation
2. add `--region` option
2020-08-26 12:41:05 +02:00
sylviamoss 788dc32598 add aws_polling config option to override env variables 2020-08-17 17:15:32 +02:00
Wilken Rivera 89d57b8635
post-processor/googlecompute-import: Update documentation (#9767) 2020-08-14 12:04:44 +02:00
Megan Marsh daccfc42cf
Fix ReregisterVM to default to true instead of false. (#9736) 2020-08-10 13:13:14 +02:00
Wilken Rivera ae23f891e1
post-processor/googlecompute-export: Update documentation (#9727)
* Add markdown generator to post-processor config
* Add HCL2 example
2020-08-07 16:34:38 -04:00
Wilken Rivera 759d2451a4 post-processor/googlecompute-export: Remove IAP configuration from HCL schema
Build results before change
```
[go-1.14.2] [2] wilken@automaton in
~/Development/packer-templates/googlecompute/googlecompute-startup-scripts/
on master
⇶  packer build -var project=$GOOGLE_PROJECT_ID
googlecompute-ubuntu-startup-script-export.pkr.hcl
Error: Failed preparing post-processor-block "googlecompute-export" ""

  on googlecompute-ubuntu-startup-script-export.pkr.hcl line 28:
    (source code not available)

    1 error occurred:
            * unknown configuration key: '"iap"'

==> Builds finished but no artifacts were created.
```

Build results after change
```
⇶  packer build -var project=$GOOGLE_PROJECT_ID
googlecompute-ubuntu-startup-script-export.pkr.hcl
googlecompute.example: output will be in this color.

==> googlecompute.example: Checking image does not exist...
==> googlecompute.example: Creating temporary SSH key for instance...
```
2020-08-07 11:00:07 -04:00
Luke Swithenbank 6d1a146143
Fix Google Compute Export Post-Processor (#9708)
* Fix Google Compute Export Post-Processor

The current Post-Processor hangs on waiting for the Startup Script to finish. The startup script doesn't update the metadata (StartupScriptStatusKey) so this change aims to fix that but adding the `SetMetadata` functionality to this script.

* Update startup.go
2020-08-06 16:18:23 -04:00
sylviamoss fd683daa22 update checksum post-processor docs 2020-08-05 19:46:17 +02:00
Megan Marsh 02d3fb37e3 fix return 2020-07-27 08:53:33 -07:00
Megan Marsh 69b0e66b5d
Update post-processor/vsphere/post-processor.go
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-07-27 08:52:29 -07:00
Adrien Delorme c34e89aec7
getEncodedPassword: simplify encodedPassword return 2020-07-27 10:33:39 +02:00
Megan Marsh 395a0c472e improve postprocessor stdout 2020-07-25 08:24:21 -07:00
Megan Marsh 6383e6cbbf fix vsphere postprocessor password log filtering, write tests 2020-07-25 08:24:21 -07:00
Megan Marsh 1b6b44589d Change to use query syntax instead of manually throwing queries in.
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-07-23 14:36:41 -07:00
Megan Marsh 393b2d16c4 switch to using net/url.URL struct for encoding rather than the incorrect QueryEscape for the whole URL. 2020-07-23 14:33:09 -07:00
Megan Marsh 2e4d0c888a pull ovftool uri generation into separate function, test function 2020-07-23 14:33:09 -07:00
Megan Marsh a687d6fe12
ci/tech-debt: Fix vmware acceptance tests (#9636)
* add kex algorithm option to ssh config

* regenerate code

* This commit fixes old vmware acceptance tests that have not been run in some time. It does this in two parts:
1) It modifies the minimal vmware build configuration to use a custom kex algorithm, which enables the ssh connection to succeed.
2) It modifies logic in reading and defaulting hardware config values, which was crashing.
3) It adds a new acceptance test with a preseed file to test loading from an http directory.
2020-07-23 09:40:57 +02:00
Gennady Lipenkov 46a5561340 go import file 2020-07-22 11:39:28 +03:00
Gennady Lipenkov 53e7f4e71a extend artifact, code refactoring 2020-07-22 01:01:30 +03:00
Gennady Lipenkov e0cc7b9d8c yandex-import: allow create image based on another one
Allow as source for a new Compute Image not only  URL to Storage Object but also a another one Compute Image
Details at official doc about ImageCreateRequest https://cloud.yandex.com/docs/compute/grpc/image_service#CreateImageRequest
2020-07-20 20:26:22 +03:00
Megan Marsh ea4aa67f78
make friendly error message (#9605) 2020-07-20 16:58:38 +02:00
Gennady Lipenkov b16cccde3d Support piping of yandex-import post-processors 2020-07-17 18:36:48 +03:00
Gennady Lipenkov b28be46dc1 Add package alias (fix lint) 2020-07-17 01:22:28 +03:00
Gennady Lipenkov 29a6687475 Separate workflow for file on disk and prepared URL 2020-07-17 01:09:49 +03:00
Gennady Lipenkov 59aaaf7a91 Support provision of token, SA key file and folder ID from OS env vars
As in yandex-export PP and yandex builder.
2020-07-17 01:04:00 +03:00
Gennady Lipenkov 7f9ba42ee2 Add storage util funcs with test 2020-07-17 00:59:39 +03:00
Gennady Lipenkov bd1e4f9d7d yandex-export post-processor artifact id is url.
First storage path used.
2020-07-17 00:45:19 +03:00
Gennady Lipenkov 0092007311 Forms urls and support get url for yandex-export artifact 2020-07-16 18:01:18 +03:00
Gennady Lipenkov fab42a061e Update doc and example. 2020-07-16 18:01:15 +03:00
Gennady Lipenkov f0e1b719d8 Forms urls and support get url for yandex-export artifact 2020-07-16 17:59:21 +03:00
Gennady Lipenkov 8442d570e5 Allow work with yandex-export artifact 2020-07-16 17:57:53 +03:00
zhsj e2cbe0de48
post-processor/vagrant-cloud: missing vagrant_cloud_url in test (#9572) 2020-07-14 10:29:28 +02:00
Lars Lehtonen 1400662db7
post-processor/digitalocean-import: replace deprecated oauth2.NoContext (#9566) 2020-07-13 17:13:35 +02:00
Roger Hu 28f8241a05 update generated files 2020-07-10 16:58:06 -07:00
Gennady Lipenkov d83d2b0273 Update documentation 2020-07-09 17:43:11 +03:00
GennadySpb 446d259459 Update post-processor/yandex-export/post-processor.go
Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-07-09 16:21:21 +03:00
Gennady Lipenkov b02401c41b Update doc and example. 2020-07-09 16:21:21 +03:00
Gennady Lipenkov 5762367de1 Provide 'GeneratedData' value for step 'StepCreateInstance' 2020-07-09 16:21:21 +03:00
Gennady Lipenkov 099dd3e7b3 Validate and render 'paths' config value 2020-07-09 16:21:21 +03:00
GennadySpb 919b54f4f4
yandex-export: tiny improvements (#9554) 2020-07-09 10:46:51 +02:00