Commit Graph

222 Commits

Author SHA1 Message Date
Megan Marsh 201c8c1332 make sure that you encrypt using the provided kms key id and not the default 2019-04-15 16:44:56 -07:00
Adrien Delorme aee73d745d update code after go mod update 2019-04-11 14:19:24 +02:00
Adrien Delorme f555e7a9f2 allow a provisioner to timeout
* I had to contextualise Communicator.Start and RemoteCmd.StartWithUi
NOTE: Communicator.Start starts a RemoteCmd but RemoteCmd.StartWithUi will run the cmd and wait for a return, so I renamed StartWithUi to RunWithUi so that the intent is clearer.
Ideally in the future RunWithUi will be named back to StartWithUi and the exit status or wait funcs of the command will allow to wait for a return. If you do so please read carrefully https://golang.org/pkg/os/exec/#Cmd.Stdout to avoid a deadlock
* cmd.ExitStatus to cmd.ExitStatus() is now blocking to avoid race conditions
* also had to simplify StartWithUi
2019-04-08 20:09:21 +02:00
Adrien Delorme c4f3dccc14 rename interpolation context from ctx to ictx and contexts to ctx to avoid conflicts 2019-04-03 15:56:15 +02:00
Adrien Delorme a4bf94dd3c change Builder to be passed a context for cancellation
we have to to give it to our hook
2019-04-03 15:55:55 +02:00
Adrien Delorme 9f82b75e57 Use the hashicorp/go-getter to download files
* removed packer.Cache and references since packer.Cache is never used except in the download step. The download step now uses the new func packer.CachePath(targetPath) for this, the behavior is the same.
* removed download code from packer that was reimplemented into the go-getter library: progress bar, http download restart, checksuming from file, skip already downloaded files, symlinking, make a download cancellable by context.
* on windows if packer is running without symlinking rights and we are getting a local file, the file will be copied instead to avoid errors.
* added unit tests for step_download that are now CI tested on windows, mac & linux.
* files are now downloaded under cache dir `sha1(filename + "?checksum=" + checksum) + file_extension`
* since the output dir is based on the source url and the checksum, when the checksum fails, the file is auto deleted.
* a download file is protected and locked by a file lock,
* updated docs
* updated go modules and vendors
2019-03-13 12:11:58 +01:00
Megan Marsh 0196d7f835
Merge pull request #7370 from sargun/allow-users-to-specify-custom-block-device-mapping
amazon-chroot: Allow users to specify custom block device mapping
2019-03-12 14:21:23 -07:00
Megan Marsh 3ed9fe6dee remove unused imports
remove HttpClientWithEnvironmentProxy from amazon builders; let access config handle it
2019-03-04 16:42:33 -08:00
Sargun Dhillon 601e754438 amazon/chroot: Allow creating new block device mappings !not fromScratch
Previously, when you built from an existing image, you were unable
to reconfigure block device mappings, as it just took them and
copied them over. This allows users to specify new, custom
block device mappings, even when building from an existing
image.
2019-03-04 13:11:14 -08:00
Megan Marsh 0abd16527e
Merge pull request #7337 from sargun/fix-chroot-builder
Amazon-chroot: Fix building PV images and where mount_partition is set
2019-03-01 10:25:04 -08:00
Adrien Delorme f03cbd8a10 aws: allow to pass nil bool as encryption parameter
to retain current encryption settings

this changes the fields :
* ami_config.encrypt_boot
* block_device.encrypted

This also removes StepCreateEncryptedAMICopy as this step is now done in StepAMIRegionCopy
2019-02-27 11:17:56 +01:00
Sargun Dhillon 142a84ef43 amazon/chroot: Refactor step_register_ami for testing EBS-related behaviour
There is some logic around how EBS mappings are handled / generated depending
on whether an image if from scratch or not. There was no testing around
this behaviour before. This strips that logic out into a separate function,
to enable testing it independentlhy.
2019-02-25 10:45:23 -08:00
Sargun Dhillon d3c65ee77d builder/amazon/chroot: Fix building PV images with mount_partition
Right now, if we have a source image that's PV, and try to build an
image with mount_partition set to not 0, it does not get picked up.
This is because under PV we only had a filesystem, not partitions,
but you can convert a PV image to an HVM image during build time.
2019-02-22 18:21:01 -08:00
Megan Marsh 3704a053d0 move region validation and credential wait into step pre validate 2019-02-05 14:07:04 -08:00
Megan Marsh 035c3506ba manually set proxyfromenvironment in default http client for ec2 sessions 2019-01-25 12:33:25 -08:00
Adrien Delorme b7222d1f56 Add tmp package that offers Dir & File funcs
this regroups the calls of tmp func calls into one package.
the tmp pkg allows to store tmp files under a common directory for easier managment

Squashed commit of the following:
commit 74b674c01560c6b547e9cab5e8afb486f0fe7d6c
Merge: fc94b5714 66001525d
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 02:02:38 2018 -0400

    Merge branch 'remove_TMPDIR_docs' of github.com:tb3088/packer into remove_TMPDIR_docs

commit fc94b571428447aaf2e5171967c95cdf9e5c73ff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit 5a562359f4937af506607dd96ed1c46426093f52
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 9d5a4a0935bfd049e2f8f50ad73e4455b42c9410
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit fbae4c2e051898aecd8705e9d8a20353430ede23
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 3590fae8bd0afa92a161a49ed794ee32e357d583
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit d5c5306a97b648ded06a8eb76ce728f07c9924f0
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 0a72297da86089fbd07e4f7d9472a92fa4d1c733
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 20f68228b6c372d984ea5b055cfc8bda9b110ac5
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit c73ebe3d8321664a2685a0baa8e441dd895b0db4
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit 63549b8bd745a0090b15ed0b0ebf644162db27db
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit 976101074942db36f10d3a686d6d29ddb7c01926
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit 95159afbc05eac96ed11c3183af5fb0abe2f2d5c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 66001525d72de56a4cf9339b900c46f59bc6e91a
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit e9b6adefeae8c65eb8aa47fef38cbf0aa424338c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 852113ed076e2d14e5dca6815ea680da1e2896bb
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit 54add38d1d1c6e283cd444b367ed8bd49a5f3699
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 6b5b8f6d4edad0e187ca13d64ac4118f34eee643
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit c22092c601c33484327674f322c7379fa41506d7
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 7a730450916bf8e5dbc2a741ec233a49466ab7cc
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 0f2933adb6f6922dfeab78a95371a444ec8918ab
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit d74839ede05dacf712b55a7bb48aec19fe6b007f
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit eb65416619437e4a3dec90277770803dd5b2281c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit acaa2b31ed463219c4ef099f351eec72406e2989
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit e573fde668dcb418561e61535c1d68b2888f5b0f
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 39a9874afc9dea71b36753375fb7feb61ffecf69
Merge: 8a413cfe8 3315812c2
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 18:19:26 2018 -0400

    Merge branch 'master' of https://github.com/hashicorp/packer into prestine

commit 8a413cfe83b0a70dbd99c32d936334ca5788ca9b
Merge: e07491de5 4e14710a6
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Oct 1 20:18:10 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit e07491de59cead0b337a7f57f4a6c625e1f560ab
Merge: 42610a35d a1fa35dff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Aug 21 13:26:19 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 42610a35d5ef65af8844a58ad70e2ec75262da6a
Merge: 5298142da 0d63cf7bc
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue Jun 19 22:45:05 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 5298142da6da95354844f201deeeef3c599e48b2
Merge: 7bb110bc7 9d9736552
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Jun 11 15:10:09 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 7bb110bc744c9797fcaec25a265d9b85a22d6d6b
Merge: a61c869ca 6189d66e7
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Wed May 9 23:41:22 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit a61c869ca7268acf0e3b1e1fa7a8ee7feea65984
Merge: 098101dd9 4be5f072c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri May 4 10:13:04 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 098101dd99f08b0ca110d33eff6904537c6d21de
Merge: fefaf0fa6 554b2b4a5
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue May 1 20:52:30 2018 -0400

    Merge branch 'EOL-handling' into prestine

Co-Authored-By: Matthew Patton <pattonme@yahoo.com>
2018-12-12 16:35:57 +01:00
Matt Dainty feb8067c7d Convert `ena_support` to a pointer
This means it now has three states, `true`, `false`, & `nil`. The
default state is now `nil` which does nothing instead of `false` which
now will explicitly disable ENA support instead of just not enabling it.
2018-10-19 12:40:43 +01:00
Matthew Hooker 9181570bc3
remove stray struct dumps in logs 2018-09-27 00:14:05 -07:00
Megan Marsh 79093da6ad skip region validation in tests that don't care; refactor Prepare func so we can test region validation logic with a mock 2018-09-19 10:30:08 -07:00
Megan Marsh 93f1155a14 remove test-related special casing 2018-09-18 08:08:36 -07:00
Sargun Dhillon 63d784023f Add support to support non-gp2 volume types for building from_scratch 2018-09-06 13:22:09 -07:00
Sargun Dhillon 07ce820c70 Enable setting the volume type when building with the Amazon Chroot builder 2018-09-06 13:22:09 -07:00
Megan Marsh b94e6e913e
Merge pull request #6670 from sargun/fix-ena-on-pv
Allow users to build hvm images from pv images to have ena_support
2018-09-06 12:50:46 -07:00
Sargun Dhillon 902497d6ce Allow users to build hvm images from pv images to have ena_support 2018-09-04 18:16:52 -07:00
Megan Marsh e4f6b6f720 fix chroot communicator to properly escape quotes 2018-08-28 09:16:43 -07:00
Megan Marsh ff6a039d5b replace scrubconfig with packer.LogSecretFilter.Set
filter winrm password from logs
Add new root-level packer template option, sensitive-variables, to tell us what user variables to mark sensitive.
2018-08-20 15:35:55 -07:00
Anshul Sharma 885ecb0790
Issue-6309 Amazon Chroot Provider
-  Add tags on CreateVolume
2018-07-18 13:01:15 +03:00
Megan Marsh f49a2d8aed move UI call to before the wait; add context to these steps 2018-07-11 10:36:21 -07:00
Megan Marsh cf63dd10bf replace AMIStateRefreshFunc, InstanceStateRefreshFunc, our spot instance waiter, our chroot volume waiter, and our snapshot waiters with waiters form AWS's SDK. 2018-07-11 10:34:20 -07:00
Megan Marsh 87a93e84ac sh -c to execute command to accomodate communicator change 2018-06-08 13:14:55 -07:00
M. Marsh 62e1323577
Merge pull request #5956 from hashicorp/dedupe_shell_local
Deduplicate the code for the shell-local post-processor and shell-local provisioner
2018-05-23 15:54:27 -07:00
Megan Marsh e670eed315 Add new option, nvme_device_path, so that we can properly mount nvme block devices. 2018-05-23 09:16:40 -07:00
Megan Marsh 6dc4b1cbdc move all of the run commands for shell-local provisioner and postprocessor into common library too 2018-05-07 15:10:09 -07:00
Megan Marsh 616b41e58f deduplicate the nearly identical communicators for the shell-local provisioner and post-processor, moving single communicator into a new common/shell-local module 2018-05-07 15:10:09 -07:00
Megan Marsh 59e3ec722d allow user to mount entire block device in chroot builder 2018-04-25 10:47:52 -07:00
Matthew Hooker 21812fa17f
Add volume and run tags if in us-gov/china
We can't tag on instance creation when we're in "restricted" regions,
so let's add the tags after the resources have been created.

Adds methods to AccessConfig to detect if we're in China or US Gov
regions (i.e. "restricted").

Also turns tag:tag maps into a type, and moves methods around validating
and converting them to ec2Tags to methods of the type.
2018-02-05 16:39:20 -08:00
Matthew Hooker fad4d5c272
update tests for mapstructure behavior changes 2018-01-26 12:40:34 -08:00
Matthew Hooker 7a189a83a1
fix imports
`find . -type f -name '*.go' -not -path "./vendor/*" -exec goimports -w {} \;`
2018-01-24 17:09:17 -08:00
Matthew Hooker a831d522be
change run signatures
Run now takes a context as well as a statebag. We'll assign the context
to the blank identifier to prevent namespace collisions. We'll let the
step authors opt-in to using the context.

`find . -iname "step_*.go" -exec gsed -i'' 's/func \(.*\)Run(/func \1Run(_ context.Context, /' {} \;`
2018-01-24 17:09:17 -08:00
Matthew Hooker 366dc3da0a
move multistep imports to helper.
gomvpkg -from "github.com/mitchellh/multistep" -to "github.com/hashicorp/packer/helper/multistep"
2018-01-24 17:09:15 -08:00
Matthew Hooker ebe995c0ff
run goimports 2018-01-22 17:21:10 -08:00
Matthew Hooker f3dea272d9
Set session in state bag
We can construct connections to arbitrary regions from it
2017-12-19 11:32:33 -08:00
Matthew Hooker f6c401a143
don't pass in accessconfig 2017-12-19 11:32:32 -08:00
Rong Chen 3e92b1374e issue5606: follow the convention to use AccessConfig to create new aws session for step_create_tags etc. 2017-12-19 10:34:12 -08:00
Krzysztof Wilczynski 05327b7524
amazon: Remove Session Token (STS) from being shown in the log.
This commit adds a change which ensures that the Session Token
config struct item is removed from log output.

Signed-off-by: Krzysztof Wilczynski <kw@linux.com>
2017-12-04 22:14:38 +01:00
Matthew Hooker f681faa296
fix tests 2017-11-07 14:57:10 -08:00
Matthew Hooker c65fa8490d
fix various bugs deregistering AMIs
always deregister ami in session region
validate that session region does not appear in ami_regions
2017-10-30 14:17:19 -07:00
Megan Marsh 85ad1f58fb include ena_support and sriov_support in chroot and instance docs 2017-08-29 09:36:06 -07:00
Megan Marsh 2d4bc70d7b use ami prefix to make it clear that these variables are amazon specific
add fixer, fixer tests
2017-08-28 11:21:27 -07:00
Megan Marsh 9e2e467b31 Allow granular modification of sriov and ena enhanced networking options in amazon amis. Maintain old functionality. 2017-08-25 14:49:53 -07:00