Commit Graph

261 Commits

Author SHA1 Message Date
Megan Marsh 70cb4097e1 review comments, add plugin type for a few missing builders 2020-10-21 16:17:32 -07:00
Megan Marsh c0ce8a9414 rework fixer deprecation code so we know what plugins they relate to 2020-10-21 15:28:59 -07:00
Adrien Delorme b24911661f
add sshkey package and ssh-keygen comand (#10101)
* add sshkey.Generate function that returns an sshkey.Pair to be used with openssh.
* add cmd/ssh-keygen/main.go for testing purposes
* add a test calling ssh.ParsePrivateKey & ssh.ParseAuthorizedKey (which is very
   similar to what openssh would do to read a keypair)

The wrapping of the keys should be handled by crypto/x509.MarshalPKCS8PrivateKey 
& x/crypto/ssh.NewPublicKey which does not work for ed25519 and dsa. 
x509.MarshalPKCS8PrivateKey marshals ed25519 keys but the keys did not work with openssh. 
x509.MarshalPKCS8PrivateKey does not handle dsa keys.
So I had to 'wrap' those manually by reading the code of the openssh package.
Note that ssh.NewPublicKey works with any keytype. I should probably do a PR to ssh to have a NewPrivateKey & Marshalling funcs
2020-10-19 10:24:34 +02:00
Ricardo Rosales 809f38be3a
[azure-chroot] Updating parameter `exlude_from_latest` to `exclude_from_latest` (#10034) 2020-10-07 11:08:33 +02:00
Marco Molteni 4d7e42220b
scaleway: add pre validate step (check image and snapshot names) (#9840)
* Implement Stringer inteface for multistep.StepAction

* scaleway: add pre validate step (check image and snapshot names)

Before, it was possible to create multiple images with the same name,
leading to a confusing and wasteful situation (same for snapshots).

Now, we perform the same kind of checks done by the AWS EC2 builder,
and refuse to proceed if there is an existing image with the same name
(same for snapshots).

As usual, invoking `packer build -force` will bypass the checks.

Closes #9839.
2020-09-18 15:37:48 -04:00
Megan Marsh 94a9beaac6 fix up documentation for virtualbox key pair ssh options 2020-09-04 16:55:58 -07:00
Megan Marsh 5120ad9d9a add test file 2020-09-04 11:56:32 -07:00
Megan Marsh 20d62654ae move runAndStream into a generalized helper function to enable easy local exectution in other places than just the docker builder 2020-09-04 11:56:32 -07:00
Megan Marsh 2dcf950441 change pause to run connect again afterwards 2020-08-14 11:23:43 -07:00
Adrien Delorme 44616d3bff
refactor initialization out from packer configs + tests (#9627)
The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost.

This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too.

fix #9478
2020-07-24 10:58:03 +02:00
Megan Marsh a687d6fe12
ci/tech-debt: Fix vmware acceptance tests (#9636)
* add kex algorithm option to ssh config

* regenerate code

* This commit fixes old vmware acceptance tests that have not been run in some time. It does this in two parts:
1) It modifies the minimal vmware build configuration to use a custom kex algorithm, which enables the ssh connection to succeed.
2) It modifies logic in reading and defaulting hardware config values, which was crashing.
3) It adds a new acceptance test with a preseed file to test loading from an http directory.
2020-07-23 09:40:57 +02:00
Adrien Delorme 125178d943
core: Decode: when in HCL2 decoding mode; reset the whole struct before preparing it. (#9622)
* core: Decode when in HCL2 decoding mode; reset the whole struct before preparing it.
* HCL2: add path variables + docs & tests

Co-authored-by: Megan Marsh <megan@hashicorp.com>
2020-07-23 09:25:07 +02:00
Adrien Delorme fd6be7f279 better/simpler `pause_before_connecting` example 2020-07-20 16:03:40 +02:00
Megan Marsh 530f5e3f63 more additions to partials 2020-07-17 15:52:11 -07:00
Roger Hu 31a7a1d637 adding basic support for OpenSSH CertificateFile support
small fix
2020-07-10 16:54:08 -07:00
Sylvia Moss 9c1409dbba
Add content_library_destination to import VM template to a Content Library (#9551) 2020-07-10 11:01:10 +02:00
Moss 8ea64c5b35 undocument ssh_keypair_name and ssh_agent_auth 2020-07-02 14:55:58 +02:00
Moss 41edd09d3c undocument temporary_key_pair_name and ssh_private_key_file 2020-07-01 16:33:18 +02:00
Megan Marsh cacdb0ca02 add tests 2020-06-18 11:31:00 -07:00
Megan Marsh 4f77bb0c4a add new ssh_ciphers option 2020-06-17 16:48:23 -07:00
Megan Marsh a07563c64f fix up typos in comments and regenerate docs.
Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-06-16 10:27:30 -07:00
Megan Marsh 4178625afb re-add winrm_no_proxy option. Winrm code needs to be modified in order to read no_proxy env var when client is created rather than when net/http library is instantiated. 2020-06-16 10:27:30 -07:00
Wilken Rivera f672f5bd9b command/validate: Add support for HCL2 configuration files
* Update validate command to use FixConfig for checking against known
fixers
* Update validation command flag docs
* Add ConfigFixer method to PackerHandler Interface
* Implement ConfigFixer interface in PackerConfig
* Remove all stdout messaging (i.e calls to c.Ui.Say) in the validate
command. The command will only display hcl.Diagnotic messaging when there is an error or warning.

HCL2 Configs
```
⇶  packer validate docker_centos_shell_provisioner.pkr.hcl

```

JSON Configs
```
⇶  packer validate vmware-iso_ubuntu_minimal/vmware-iso_ubuntu_minimal.json
Error: Failed to prepare build: "vmware-iso"

1 error occurred:
        * Deprecated configuration key: 'iso_checksum_type'. Please call `packer fix`
against your template to update your template to be compatable with the current
version of Packer. Visit https://www.packer.io/docs/commands/fix/ for more
detail.

Warning: Fixable configuration found.
You may need to run `packer fix` to get your build to run correctly.
See debug log for more information.

  map[string]interface{}{
        "builders": []interface{}{
                map[string]interface{}{
                        ... // 3 identical entries
                        "guest_os_type":     string("ubuntu-64"),
                        "http_directory":    string("http"),
-                       "iso_checksum":
string("946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
+                       "iso_checksum":
string("sha256:946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
-                       "iso_checksum_type": string("sha256"),
                        "iso_url":
string("http://old-releases.ubuntu.com/releases/14.04.1/ubuntu-14.04.1-server-amd64.iso"),
                        "shutdown_command":  string("echo 'vagrant' | sudo -S shutdown -P now"),
                        ... // 4 identical entries
                },
        },
  }
```
2020-06-05 14:24:39 -04:00
Sylvia Moss ce45a1990a
Refactor communicator config for Qemu and add SkipNatMapping option (#9307) 2020-06-02 11:56:36 +02:00
Megan Marsh 4698f90e65 update deprecated options 2020-05-29 10:19:02 -07:00
Megan Marsh a94e147865 fix go fmt 2020-05-29 09:40:31 -07:00
Megan Marsh b28059c0a5 add generator to create a list of all the deprecated options that the fixers fix
fix location of deprecated_options code
2020-05-29 09:40:31 -07:00
Megan Marsh 15b14d4879 add deprecation check to config helper 2020-05-29 09:40:31 -07:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Megan Marsh 3dc4ba2d68
Json vs HCL2 parity refactor (#9301)
* refactor so that json and hcl2 templates are both prepared in the same place in the build call, to make code easier to reason about. Remove overly verbose error output which isn't useful in vast majority of cases

* fix tests

* check err msg

* hcl2template.PackerConfig.GetBuilds: raise a diagnostic in case the packer core build perpare call errors

Co-authored-by: Adrien Delorme <adrien.delorme@icloud.com>
2020-05-28 10:43:58 +02:00
Sylvia Moss 1c30a71d09
Replace file shared state by statebag (#9238) 2020-05-19 11:49:48 +02:00
Megan Marsh 7e0cfd8b77 add undocumented tag to struct-markdown generator so that we can leave internal-use-only fields out of docs 2020-05-01 10:31:38 -07:00
Megan Marsh 217dcbb97f golang 1.14 doesn't like calling NewFile on existing files. Port solution over from Terraform 2020-04-09 14:38:17 -07:00
Jeff Escalante 2999934326
fix generation script, bugfixes in source comments, update website paths in primary codebase 2020-04-07 17:53:22 -04:00
Jeff Escalante 4afd107eeb
patch source files for extensions changes 2020-04-07 17:50:51 -04:00
Wilken Rivera 360f81b030 tests/provisioners: Add TestProvsionerPreCheck function
This function can be used to check if a Provisioner has been marked for testing within the ACC_TEST_PROVISIONERS environment variable.

While testing I found that the shell acceptance test were also running when trying to run powershell tests.

Before change
```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test -v ./provisioner/shell/... -timeout=1h

=== RUN
TestShellProvisioner/testing_amazon-ebs_builder_against_shell_provisioner
2020/04/06 15:18:12 ui: amazon-ebs: output will be in this color.
2020/04/06 15:18:12 ui:
2020/04/06 15:18:12 Build debug mode: false
2020/04/06 15:18:12 Force build: false
2020/04/06 15:18:12 On error:
2020/04/06 15:18:12 Preparing build: amazon-ebs
2020/04/06 15:18:12 Waiting on builds to complete...
2020/04/06 15:18:12 Starting build run: amazon-ebs
2020/04/06 15:18:12 Running builder: amazon-ebs
```

After changes
```
⇶  ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=powershell go test -v ./provisioner/shell/... -timeout=1h
--- SKIP: TestShellProvisioner (0.00s)
    provisioners.go:88: Provisioner "shell" not defined in ACC_TEST_PROVISIONERS
```
2020-04-06 16:52:24 -04:00
Sylvia Moss 665330de92
Write generic Builder-Provisioner acceptance test logic (#8963) 2020-04-03 18:17:09 +02:00
Adrien Delorme 88297c796d Merge branch 'master' into hcl2_singular_blocks 2020-03-17 14:47:17 +01:00
Adrien Delorme 834f44190f go generate 2020-03-17 12:30:36 +01:00
Adrien Delorme 87d6b2433f make generate 2020-03-16 15:21:29 +01:00
Megan Marsh fc35f02004
Merge pull request #8890 from hashicorp/fix_8880
fix googlecompute password interpolation
2020-03-13 12:55:12 -07:00
Megan Marsh e94ff70199
Merge pull request #8883 from hashicorp/fix_8835
Interpolate shell inline config
2020-03-13 12:51:34 -07:00
Megan Marsh 6477d8a0c8
Merge pull request #8882 from hashicorp/fix-var-file-hcl
allow to use hcl files as var files in HCL mode
2020-03-13 12:35:58 -07:00
Megan Marsh 6008f911fd
Merge pull request #8847 from takaishi/support-keyboard-interactive
communicator/ssh: support keyboard-interactive auth to connect bastion
2020-03-13 11:48:12 -07:00
Megan Marsh bcb7b8751c fix googlecompute password interpolation 2020-03-13 11:01:43 -07:00
Moss 5a8c628880 Add acc test validation to avoid running with unit tests 2020-03-13 17:50:05 +01:00
Moss 3f49b7c66e Fix linter 2020-03-13 17:17:42 +01:00
Moss 2ca6804827 Move shell prov acc test to the same folder 2020-03-13 17:10:51 +01:00
Adrien Delorme 60a3105a45 make ssh docs less AWS specific 2020-03-13 11:46:52 +01:00
r_takaishi 70e3f60539 write unit test 2020-03-13 14:01:11 +09:00