Commit Graph

373 Commits

Author SHA1 Message Date
Megan Marsh a687d6fe12
ci/tech-debt: Fix vmware acceptance tests (#9636)
* add kex algorithm option to ssh config

* regenerate code

* This commit fixes old vmware acceptance tests that have not been run in some time. It does this in two parts:
1) It modifies the minimal vmware build configuration to use a custom kex algorithm, which enables the ssh connection to succeed.
2) It modifies logic in reading and defaulting hardware config values, which was crashing.
3) It adds a new acceptance test with a preseed file to test loading from an http directory.
2020-07-23 09:40:57 +02:00
Chris Chilvers 63eedf841e Fix failing tests due to IAP communicator host name
Moved setting the host from ApplyIAPTunnel to Config.Prepare but forgot to
update the related tests.
2020-07-22 12:58:05 +01:00
Chris Chilvers 37544f4d5f Support using WinRM over an IAP tunnel
This avoids the need to expose WinRM ports on the internet and allows
using instances with only an internal private IP address.

When using a WinRM tunnel there is a race condition between the tunnel
connection attempt timing out and packer assuming the connection was
successful. To allow for this, when using WinRM the default success
timeout is increased to 40 seconds.
2020-07-19 01:28:31 +01:00
Megan Marsh 8bb2cb44c0 rename launch timeout to launch wait 2020-07-15 10:24:55 -07:00
Megan Marsh 6e9eaa1edd generate code 2020-07-15 10:20:50 -07:00
Megan Marsh af009c2aa0 make tunnel timeout configurable 2020-07-15 10:17:13 -07:00
Roger Hu 28f8241a05 update generated files 2020-07-10 16:58:06 -07:00
Wei Cheng e5606d483d
builder/googlecompute: add support for oslogin (#9339)
* feat: add support for oslogin

Signed-off-by: Wei Cheng <calvinpohwc@gmail.com>

Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-07-08 06:30:25 -04:00
Megan Marsh 673858a63c
Merge pull request #9513 from hashicorp/undocument_ssh_temp_key
Undocument ssh options not used by all builders
2020-07-07 14:26:21 -07:00
Wilken Rivera 4462c0b5ab builder/googlecompute: Add WrapStartupScriptFile configuration option
By default the Google builder will wrap any provided startup script file
in order to track its execution via custom metadata. The wrapper script
can add a bit of complexity to the start script file so a new option is
being added `wrap_startup_script`. This option allows a user to disable
the script wrapping and just let GCE do its own thing when executing a
startup script.
2020-07-07 07:08:09 -04:00
Moss 8ea64c5b35 undocument ssh_keypair_name and ssh_agent_auth 2020-07-02 14:55:58 +02:00
Moss 41edd09d3c undocument temporary_key_pair_name and ssh_private_key_file 2020-07-01 16:33:18 +02:00
Megan Marsh a96194081a
Merge pull request #9453 from hashicorp/fix_ssh
Allow users to provide a list of ciphers that they want Packer to support.
2020-06-18 11:58:53 -07:00
Wilken Rivera ce6284f760 docs/builder/google: Add note about boolean values for compute instance metadata
Closes #9393
2020-06-18 09:50:40 -04:00
Megan Marsh 4f77bb0c4a add new ssh_ciphers option 2020-06-17 16:48:23 -07:00
Megan Marsh 4178625afb re-add winrm_no_proxy option. Winrm code needs to be modified in order to read no_proxy env var when client is created rather than when net/http library is instantiated. 2020-06-16 10:27:30 -07:00
Svetlin 49bf9c9d37
[gcp] Fixup for the deprecated Secure Boot guest OS feature (#9371)
GCP informed us that the GuestOsFeature SECURE_BOOT is deprecated and
that they are using UEFI_COMPATIBLE instead. This means, using an image
with 'UEFI_COMPATIBLE' guest OS feature confirms the 'secureBoot'
feature compatibility.
2020-06-10 16:21:11 +02:00
Megan Marsh 02590f6c44
Merge pull request #9329 from hashicorp/je.website-maintenance-5-29-20
🌷 Website Spring Maintenance
2020-06-03 16:25:23 -07:00
Jeff Escalante f4d7b5f493
fix some stuff 2020-06-02 17:22:40 -04:00
Wei Cheng e0cfb404fb
builder/googlecompute: add support to specify storage location for GCP images (#9326)
* feat: add support to specify storage location for GCP images

Signed-off-by: Wei Cheng <calvinpohwc@gmail.com>
2020-06-02 13:42:33 -04:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Megan Marsh 9476aa03de
Merge pull request #9105 from hashicorp/google_iap
Implement iap proxy for googlecompute
2020-05-08 12:40:42 -07:00
Megan Marsh a55b73473a fix logic 2020-05-08 10:24:28 -07:00
Megan Marsh dde162622d use local port in listener config, when set 2020-05-08 10:20:47 -07:00
Megan Marsh 81e043f2b0 fix windows tests 2020-05-08 10:13:43 -07:00
Megan Marsh 60600e6cf6 make windows work with account files containing spaces 2020-05-08 10:13:43 -07:00
Megan Marsh dae60799bc fix tests 2020-05-08 10:13:43 -07:00
Megan Marsh fda55fe928 deduplicate excess code 2020-05-08 10:13:43 -07:00
Megan Marsh 54b33ad8d1 fix tests 2020-05-08 10:13:43 -07:00
Megan Marsh 850303b8b8 get gcloud integration working on Windows 2020-05-08 10:13:43 -07:00
Megan Marsh c578afc62c working on windows 2020-05-08 10:13:43 -07:00
Megan Marsh 32752d4577 fix linting and tests 2020-05-08 10:13:43 -07:00
Megan Marsh b5744efb42 add config prepare tests 2020-05-08 10:13:43 -07:00
Megan Marsh f583674cd6 linding 2020-05-08 10:13:43 -07:00
Megan Marsh 4fc92d7858 add tests 2020-05-08 10:13:42 -07:00
Megan Marsh 937a4859d4 Make retries a bit smarter, clean up language to be gentler, and give up on parsing stdout for tunnel launch 2020-05-08 10:13:42 -07:00
Megan Marsh 3e1ddad0c7 fix behavior when not using IAP, try to use more sophisticated streaming than buffer.String() 2020-05-08 10:13:42 -07:00
Megan Marsh 9353635b43 send sigint instead of sigkill; we can at least ask the tunnel nicely to shut down. 2020-05-08 10:13:42 -07:00
Megan Marsh f67a8ab431 revert unneeded changes to driver roles 2020-05-08 10:13:42 -07:00
Megan Marsh 80ecd2013d basic untested implementation for Windows. May leave zombie processes lying around 2020-05-08 10:13:42 -07:00
Megan Marsh 3cd28e98a7 linting 2020-05-08 10:13:42 -07:00
Megan Marsh 365eb09842 import context 2020-05-08 10:13:42 -07:00
Megan Marsh eb05f6ff88 fix windows 2020-05-08 10:13:42 -07:00
Megan Marsh d713f7ec64 add conditional building becasue windows support is still forthcoming 2020-05-08 10:13:42 -07:00
Megan Marsh e6073bcec7 implement iap proxy for googlecompute. ssh-only so far 2020-05-08 10:13:42 -07:00
Markus Liljedahl d807e81adb
Changed default disk size for when using googlecompute to match minimum disk sice to the google api. Closes https://github.com/hashicorp/packer/issues/9023 (#9071) 2020-05-08 15:55:38 +02:00
Megan Marsh 7e0cfd8b77 add undocumented tag to struct-markdown generator so that we can leave internal-use-only fields out of docs 2020-05-01 10:31:38 -07:00
Svetlin Zamfirov b706f02408 Fixing auto-generated code errors 2020-04-16 16:22:42 -07:00
Svetlin Zamfirov d827f4f757 Adding custom shielded images support (GCP)
Adds `enable_secure_boot`, `enable_vtpm` and `enable_integrity_monitoring`
config options to enable building of custom Shielded GCP Compute images.

Feedback on this is more than welcome as this is my first attempt in
contributing to anything Packer related.

Packer is great for us to build custom images on top of GCP but we would
like to enhance that to support Shielded VM images. This will allow us
to have more secure and trusted images which our team(s) will be using.
2020-04-16 16:22:41 -07:00
Adrien Delorme 840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Jeff Escalante 4afd107eeb
patch source files for extensions changes 2020-04-07 17:50:51 -04:00
Jeff Escalante bcc848a83d
were rendering yay 2020-04-07 17:50:47 -04:00
Megan Marsh 0f9415297f
Merge pull request #8679 from kwibus/gcp-fallback-image
[WIP] fix #8589 gcp public fallback image
2020-04-02 16:49:50 -07:00
desolatorxxl 87f7160084 [builder/google] Use "ssh-keys" metadata rather than deprecated "sshKeys" 2020-03-24 08:33:55 +01:00
Megan Marsh 6008f911fd
Merge pull request #8847 from takaishi/support-keyboard-interactive
communicator/ssh: support keyboard-interactive auth to connect bastion
2020-03-13 11:48:12 -07:00
r_takaishi f50ff1d270 make generate 2020-03-13 11:29:42 +09:00
Wilken Rivera 7b705545d9
docs: Various doc clean-ups to fix code block formatting and syntax highlighting (#8868) 2020-03-12 15:05:08 +01:00
Rens Sikma 3d8bda31a4 modify docstring 2020-02-20 12:54:35 +01:00
Rens Sikma c8062cc8b3 fix #8679 with source_image_project_id as list 2020-02-06 10:30:04 +01:00
Megan Marsh d252a24b82 allow user to set winrm_host override like we do ssh_host override 2020-01-30 15:22:22 -08:00
Sylvia Moss dc31bad539
Sharing info with post-processors via artifact (#8632) 2020-01-30 11:27:58 +01:00
Adrien Delorme 4b7132c87c Merge remote-tracking branch 'origin/master' into sharing_info 2019-12-17 11:57:09 +01:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Megan Marsh 0f6d1beccf add an extra string array to the Prepare() return values in the builder interfaces; this sets up the ability for builders to give the provisioners custom user-accessible build-time variables. 2019-12-16 21:23:05 -08:00
Megan Marsh 84063d2132 implement ID and Type values in the provisioner template info sharing 2019-12-13 11:57:01 -08:00
Megan Marsh 6f418d0e54 get data sharing to a working state with the powershell provisioner 2019-12-11 15:43:38 -08:00
Megan Marsh 601007e3e2 pas data into provisioners well 2019-12-11 15:43:38 -08:00
Lars Lehtonen 3571c585bb builder/googlecompute: fix dropped error (#8385) 2019-11-19 11:41:39 +01:00
Megan Marsh 6c704f7046 Revert "fix template imports"
This reverts commit 0e70e0e5a4.
2019-11-04 14:29:00 -08:00
Megan Marsh 517834d528 Revert "Revert "remove clean_ami_name and clean_image_name; complete the deprecation for these functions in favor of clean_resource_name""
This reverts commit 44f91c9afc.
2019-11-04 14:28:40 -08:00
Adrien Delorme 819329228a Change back to make sure all durations are a time.Duration
It is simply the best/simplest solution and trying to prevent users from passing and integer here would be like opening a can of worms. Because:

* we cannot make mapstructure validate our duration string ( with an UnmarshalJSON func etc.)
* we cannot make mapstructure spit a string instead of a duration and packer will decode-encode-decode config.
* the hcl2 generated code asks for a string, so this will be enforced by default.
2019-10-31 16:12:07 +01:00
Adrien Delorme bf3d9841c6 Force durations to be passed a strings
Before this commit it was possible to set a duration using an integer or a float. Go's time.Duration is an int64 internally an mapstructure will take advantage of this and load the number as a int64 but `1` means one ns which is unexpected/confusing. To avoid confusion and enforce readability this forces users to pass a string with a unit for a duration; ex "56s".
2019-10-31 11:47:19 +01:00
Adrien Delorme 078ba7c8c3 commit old code generation tool
for history
2019-10-15 12:56:42 +02:00
Adrien Delorme 17c069baa5 builder.google:
* make CustomerEncryptionKey our own type so that it can be hcl2 generated
* make Account setting unexported so that it doesn't temper with HCL2 generation ( the field is set a bit later after processing )
2019-10-15 12:56:41 +02:00
Megan Marsh 0e70e0e5a4 fix template imports 2019-10-14 13:22:06 -07:00
Megan Marsh 44f91c9afc Revert "remove clean_ami_name and clean_image_name; complete the deprecation for these functions in favor of clean_resource_name"
This reverts commit 0026792d2f.
2019-10-14 13:08:15 -07:00
Adrien Delorme d4600a7716 fix go imports 2019-10-04 11:19:45 +02:00
Megan Marsh 0026792d2f remove clean_ami_name and clean_image_name; complete the deprecation for these functions in favor of clean_resource_name 2019-10-03 15:49:01 -07:00
Lars Lehtonen ad860bf1a8
builder: remove deprecated context usage 2019-09-30 11:33:53 -07:00
Megan Marsh 3c14c50aba Allow user to real oauth token from properly configured Vault instance 2019-09-23 13:26:03 -07:00
Adrien Delorme 3fe9d52e3f Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-09-20 11:15:44 +02:00
Megan Marsh 35357616e0
Merge pull request #8112 from milescrabill/packer-googlecompute-validate-imagename
googlecompute: fail fast when image name is invalid, replace unusable characters w/ -'s
2019-09-17 10:00:33 -07:00
Miles Crabill 2bff60bac8
address review feedback 2019-09-16 13:50:43 -07:00
Miles Crabill 328baced05
add some test cases for image names 2019-09-16 10:18:22 -07:00
Miles Crabill c11a444f77
googlecompute: fail fast when image name is invalid, replace unusable characters w/ -'s 2019-09-16 10:10:41 -07:00
Megan Marsh a6e263adb6 don't fall for nil pointer dereference in account file 2019-09-13 16:37:19 -07:00
Adrien Delorme 2db109d55b Merge branch 'master' into scrape_doc_to_builder_struct_config 2019-08-23 17:37:53 +02:00
Megan Marsh fcef7e2d85 this timeout regularly flakes on Travis, I suspect because the instances running the tests are brutally underpowered. Increase the fudge factor. 2019-08-21 16:36:42 -07:00
Adrien Delorme 4cb7c30987 Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-08-21 16:44:23 +02:00
Mason, Elliot ee484f2dd9 Updated public gcp cloud images 2019-07-31 11:38:35 -05:00
Megan Marsh 7c93a5c51e code changed recently so that sometimes the conf is nil. 2019-07-29 13:19:43 -07:00
Megan Marsh a87ce366b3
Merge pull request #7835 from hashicorp/google_oauth
replace some bespoke google auth code with code from golang's oauth2 …
2019-07-08 09:48:27 -07:00
Megan Marsh 617fd18255 supply complete dummy data for googlecompute tests 2019-07-03 15:58:08 -07:00
Megan Marsh 0dfb3cc56f replace some bespoke google auth code with code from golang's oauth2 library 2019-07-03 14:27:33 -07:00
Megan Marsh c03c3f4410 replace duplicate commHost functions with a single communicator helper that checks for ssh_host and does a statebag lookup for a given key 2019-07-03 13:30:29 -07:00
Megan Marsh eb8c74bf85 allow ssh_host config option to override host logic for all builders 2019-07-02 13:56:28 -07:00
Megan Marsh ad8a88e366 allow ssh_host to override the host ip gathered from the instance for GCP builder 2019-07-02 13:01:41 -07:00
Adrien Delorme a3e3c313a9 docs: use partials in googlecompute after master merge 2019-06-12 15:18:39 +02:00
Adrien Delorme 6e3dc1aa9a Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-06-12 14:28:11 +02:00
Pratyush singhal 92af5847a7 refactor: replace *packer.MultiError from type signature of createInstanceMetadata with generic error interface
Signed-off-by: Pratyush singhal <psinghal20@gmail.com>
2019-06-11 21:01:26 +05:30