Commit Graph

4874 Commits

Author SHA1 Message Date
nywilken 1eb55c2ae3 builder/azure: Remove double override of GalleryImageVersionsClient.PollDuration
This change fixes an issues where custom timeouts set for the
GalleryImagesVersionClient was being over written by the default client
poll duration.
2019-12-17 10:47:32 -05:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Sylvia Moss c57f093efe
Remove unnecessary source_path validation for virtualbox-ovf (#8489) 2019-12-16 16:50:21 +01:00
Sylvia Moss 0a8b172bf9 Remove duplicated retry mechanism when deleting VM ffrom virtual box (#8485)
This retry was also added to fix a locked by session error reported in #5501 and fixed in #5512.
2019-12-16 14:31:15 +01:00
Kris Hicks 40357e2534 docs/builder/azure-arm: Remove errant html comment (#8484)
* Remove errant html comment from source

* make generate
2019-12-13 10:35:24 -05:00
Sylvia Moss 2ee3311082
Retry command execution when VM session is locked (#8483) 2019-12-13 10:47:19 +01:00
Sylvia Moss c6b9efb522 Fix virtualbox-iso TestBuilderAcc_basic (#8481)
make sure the test works for later usage
2019-12-12 10:43:51 +01:00
Oded Priva 5e81c6f44e added the option to skip IAM instance profile validation (#8457) 2019-12-10 09:17:38 +01:00
Sylvia Moss 2f63c581a0
Remove unnecessary 1 sec sleep (#8453) 2019-12-06 11:43:28 +01:00
Sylvia Moss 5ff5623433
Add disable_shutdown option to VirtualBox builder (#8449) 2019-12-05 14:34:56 +01:00
deisenzimmer 2b3935778e amazon-ebssurrogate plugin: StepSnapshotVolumes: Added mutex to synchronise map access across threads. (#8432) 2019-12-04 15:13:07 +01:00
Megan Marsh de9f391a1c
Merge pull request #8430 from hashicorp/fix_8359
Fix request retry mechanism  to launch aws instance
2019-12-03 16:03:03 -08:00
Megan Marsh 05b972e550
Merge pull request #8444 from ad-m/hyperone-builder
Fix tags & modernize config for HyperOne builder
2019-12-03 10:30:26 -08:00
Megan Marsh 9fc37f9456
Merge pull request #8438 from hashicorp/fix_qemu_sizes
make sure we continue to default to Megabytes for qemu disk size to p…
2019-12-03 09:58:21 -08:00
Adam Dobrawy a97ff7ce8b builder/hyperone: Update schema for cli config 2019-12-03 14:57:41 +01:00
Adam Dobrawy 3b091e6046 builder/hyperone: fix & add tests for tags 2019-12-03 14:57:40 +01:00
Megan Marsh 7d8f1c01c4 Revert "fix: persist default drive/device" (#8440)
This reverts commit 30ccab4 which introduced the issue 8436.
2019-12-03 13:42:11 +01:00
Megan Marsh fcda07ee6a clarify docs by removing unclear string (#8439)
the docs generator from code PR #7724  mis scrapped it
2019-12-03 10:07:46 +01:00
Megan Marsh f5806afda8 update docs 2019-12-02 16:04:29 -08:00
Megan Marsh effd330945 make sure we continue to default to Megabytes for qemu disk size to prevent backwards incompatabilities. 2019-12-02 15:13:17 -08:00
Lars Lehtonen df031db9da builder/azure/common/template: remove unused TemplateBuilder.getResourceByType2() (#8419)
builder/azure/common/template: remove unused TemplateBuilder.toKeyVaultID()
2019-12-02 18:09:42 +01:00
Ilhaan Rasheed 2c22b9f721 Update LXD directory upload command (#8416) 2019-12-02 12:01:18 +01:00
Moss 28c720bd3a Organize imports 2019-12-02 10:17:36 +01:00
Moss 250643d54b Remove 5 sec sleep after adding iam instance profile 2019-12-02 10:13:26 +01:00
Moss 4516978555 Fix request retry mechanism to run aws instance 2019-11-29 18:11:10 +01:00
Jon Allie c31bbcf4cf builder/amazon: AmiFilterOptions.GetOwners: avoid taking the address of a loop iterator var (#8417)
GetOwners() always returned a slice of pointers to the last value. Because slice ranging reuses the same local variable.
2019-11-27 12:28:07 +01:00
Mike Dziedziela 7c3e032fb7 amazon docs: update block-device-configuration docs (#8395)
So that it works for all builders as some use it with the following field names: launch_block_device_mappings, ami_block_device_mappings or ebs_volumes. the refactored docs weird being to specific.
2019-11-25 11:58:30 +01:00
henry-aj-jones d317d7f3d1 post-processor/docker-tag: Allow to tag an artifact with multiple tags (#8392) 2019-11-22 11:48:36 +01:00
Megan Marsh 59e6e1d4d5
Merge pull request #8393 from hashicorp/fix_8362
Add no_ephemeral template option to nuke ephemeral drives from launch mappings
2019-11-21 16:08:50 -08:00
Megan Marsh 818d9bb087 send to step run spot instance 2019-11-21 13:21:56 -08:00
Megan Marsh 67c3373107 enable no_ephemeral for spot devices, clean up ranging behavior, update docs 2019-11-21 12:46:31 -08:00
Anthony Dong 2e989088e5 amazon-ebssurrogate: allow enabling ena/sr-iov on spot instances 2019-11-21 11:13:32 +01:00
Megan Marsh 08e97ba4d8 generate code for no_ephemeral option 2019-11-20 13:49:04 -08:00
Megan Marsh 0e2bdc8be6 forcibly prevent ephemeral drives from being created in the amazon-ebs builder, when no_ephemeral is set 2019-11-20 13:42:14 -08:00
DanHam b40e5d90b6
builder/amazon/common/step_pre_validate: Return DescribeVpcs errors 2019-11-19 20:18:53 +00:00
nywilken afb880c134 test/builder/amazon/common/step_pre_validate: Update DescribeVpcs mock
* Update condition to check for a empty VpcId
2019-11-19 14:48:07 -05:00
DanHam 074be9942d
Only validate the user has provided a subnet_id when vpc_id has been set 2019-11-19 14:09:58 +00:00
Lars Lehtonen 3571c585bb builder/googlecompute: fix dropped error (#8385) 2019-11-19 11:41:39 +01:00
Wilken Rivera ad74a87b5c
Merge pull request #8360 from hashicorp/fix-7167
builder/amazon: Add validation for `subnet_id` when specifying `vpc_id`
2019-11-18 16:50:11 -05:00
nywilken f9f4726eff builder/amazon/step_pre_validate: Add check for non-default VPCs
Subnet information is only really needed when the specified `vpc_id` is
not the default VPC for the region where the builder is being executed.
This change uses the AWS API to determine if the VPC provided is a
non-default VPC and only validates the existence of a `subnet_id` if a
user has provided a non-default `vpc_id`.

Tests after change
```
> make test TEST=./builder/amazon/... TESTARGS='-count=1 -v -run=TestStepPreValidate_checkVpc'
...

=== RUN   TestStepPreValidate_checkVpc
=== RUN   TestStepPreValidate_checkVpc/DefaultVpc
=== RUN   TestStepPreValidate_checkVpc/NonDefaultVpcNoSubnet
=== RUN   TestStepPreValidate_checkVpc/NonDefaultVpcWithSubnet
=== RUN   TestStepPreValidate_checkVpc/SubnetWithNoVpc
=== RUN   TestStepPreValidate_checkVpc/NoVpcInformation
--- PASS: TestStepPreValidate_checkVpc (0.00s)
    --- PASS: TestStepPreValidate_checkVpc/DefaultVpc (0.00s)
    --- PASS: TestStepPreValidate_checkVpc/NonDefaultVpcNoSubnet (0.00s)
    --- PASS: TestStepPreValidate_checkVpc/NonDefaultVpcWithSubnet (0.00s)
    --- PASS: TestStepPreValidate_checkVpc/SubnetWithNoVpc (0.00s)
    --- PASS: TestStepPreValidate_checkVpc/NoVpcInformation (0.00s)
PASS

...
```
2019-11-18 16:44:09 -05:00
Megan Marsh ef137a8f08
Merge pull request #8380 from bennu/qemu-args
fix: persist default drive/device
2019-11-18 11:05:45 -08:00
Megan Marsh dfa134cd98 ditch else statement for improved clarity 2019-11-18 10:11:59 +01:00
Megan Marsh ce8bca349f Update builder/amazon/ebs/step_create_ami.go
Co-Authored-By: Adrien Delorme <azr@users.noreply.github.com>
2019-11-18 10:11:59 +01:00
Megan Marsh a35ac0127e fix bug in step_region_copy 2019-11-18 10:11:59 +01:00
bugbuilder 30ccab472f fix: persist default drive/device 2019-11-17 04:38:30 -03:00
Megan Marsh 437983343a
Merge pull request #8371 from stephentawn/master
Fix error message when ovftool is missing
2019-11-14 16:07:12 -08:00
Stephen Tawn c66219084e Fix error message when ovftool is missing 2019-11-14 23:11:16 +00:00
Lars Lehtonen 59472c6aad
builder/lxc: fix dropped error 2019-11-14 13:40:40 -08:00
nywilken 488e539f63 builder/amazon: Add helper function for checking against AWSError
Replace all straight forward occurrences of `if err, ok := err.(awserr.Error)` with the `isAWSErr` helper function
2019-11-12 15:27:47 -05:00
Lars Lehtonen 4c9cfed743
builder/ncloud: fix dropped error 2019-11-11 20:56:55 -08:00