Commit Graph

1256 Commits

Author SHA1 Message Date
Lars Lehtonen c74df8b652
builder/amazon/common: remove dead test function getMockConnSpot() 2019-10-31 12:15:53 -07:00
Lars Lehtonen e5e2bfa83f
builder/amazon/common: remove nil check on slice, len() on nil slices is zero 2019-10-31 12:09:32 -07:00
Adrien Delorme 819329228a Change back to make sure all durations are a time.Duration
It is simply the best/simplest solution and trying to prevent users from passing and integer here would be like opening a can of worms. Because:

* we cannot make mapstructure validate our duration string ( with an UnmarshalJSON func etc.)
* we cannot make mapstructure spit a string instead of a duration and packer will decode-encode-decode config.
* the hcl2 generated code asks for a string, so this will be enforced by default.
2019-10-31 16:12:07 +01:00
Adrien Delorme bf3d9841c6 Force durations to be passed a strings
Before this commit it was possible to set a duration using an integer or a float. Go's time.Duration is an int64 internally an mapstructure will take advantage of this and load the number as a int64 but `1` means one ns which is unexpected/confusing. To avoid confusion and enforce readability this forces users to pass a string with a unit for a duration; ex "56s".
2019-10-31 11:47:19 +01:00
Megan Marsh 6191b9c8c6
Merge pull request #8288 from hashicorp/fix_8271
add some extra layers of validation to make sure that people don't tr…
2019-10-30 10:41:52 -07:00
Megan Marsh d084cd1895 Update builder/amazon/common/ami_config.go 2019-10-30 09:41:23 -07:00
Megan Marsh e9f02e44cb add test to catch this case 2019-10-29 11:28:56 -07:00
Megan Marsh 7cf47fc463 add some extra layers of validation to make sure that people don't trip over magical encrypt_boot settings later in the build 2019-10-29 11:26:22 -07:00
Megan Marsh e2283445f2 need to initialize variables scoped to the individual loop, because the address of k,v stay the same and that means we overwrite previous variables when assigning them to the struct directly. 2019-10-28 14:31:13 -07:00
Megan Marsh a783a09ea3
Merge pull request #8269 from paulmey/paulmey/generalize-chroot
[amazon/chroot] Move common/generic chroot builder steps to common directory
2019-10-24 14:32:01 -07:00
Megan Marsh 7928b5b045 fix transcription error from when we moved to generated docs 2019-10-24 10:19:03 -07:00
Paul Meyer 3be7d81ce2 Move common steps together 2019-10-24 04:21:46 +00:00
Paul Meyer e6dfe301ac Move CommandWrapper & ShellCommand to common 2019-10-24 04:06:29 +00:00
Megan Marsh b9245a402d add structure to its own struct to make generated code stop complaining 2019-10-23 10:46:42 -07:00
Megan Marsh 5614f307fb regerate code
generate flat FlatPolicyDocument
add PolicyDocument to generate command
2019-10-23 10:46:07 -07:00
Megan Marsh d15f43e4eb
Merge pull request #8232 from hashicorp/artifact-tree
HCL2 configs - part one
2019-10-22 11:54:34 -07:00
Bernard Baltrusaitis ec1d70dc44 adding required changes 2019-10-22 21:40:18 +11:00
Bernard Baltrusaitis 65d1447b64 adding formatting 2019-10-19 21:33:58 +11:00
Bernard Baltrusaitis 6e2146324f Merge branch 'master' of github.com:b-b3rn4rd/packer 2019-10-19 20:56:49 +11:00
Bernard Baltrusaitis 8b83cca064 adding temporary instance profile document policy 2019-10-19 20:56:18 +11:00
Adrien Delorme cf1555bf17 Merge remote-tracking branch 'origin/master' into artifact-tree 2019-10-16 10:22:56 +02:00
Paul Meyer 977ca5be1c Add StepCopyFiles and StepChrootProvision 2019-10-15 21:13:37 +00:00
Paul Meyer 9a3e6661b1 Add StepMountExtra 2019-10-15 21:13:37 +00:00
Paul Meyer addbdedea9 Add StepPostMountCommands 2019-10-15 21:13:37 +00:00
Paul Meyer 4d750ddefa Reuse amazon/chroot/step_pre_mount_commands.go 2019-10-15 21:13:37 +00:00
Adrien Delorme 078ba7c8c3 commit old code generation tool
for history
2019-10-15 12:56:42 +02:00
Adrien Delorme 221e72e9c3 builder.amazon.common: remove pointer (*) from []*string and map[*string]*string types
because they make hcl2/cty decoding panic
2019-10-15 12:56:41 +02:00
Adrien Delorme 8b8c2fc138 builder.amazon.chroot: remove specific BlockDevice definition
because the common one already has a KmsKeyId field
2019-10-15 12:56:41 +02:00
Megan Marsh f05fd3969f
Merge pull request #8212 from hashicorp/fix_8204
Fix 8204
2019-10-14 14:40:17 -07:00
Megan Marsh 2e20facc8b fix tests 2019-10-14 14:13:21 -07:00
Megan Marsh 0e70e0e5a4 fix template imports 2019-10-14 13:22:06 -07:00
Megan Marsh 44f91c9afc Revert "remove clean_ami_name and clean_image_name; complete the deprecation for these functions in favor of clean_resource_name"
This reverts commit 0026792d2f.
2019-10-14 13:08:15 -07:00
Megan Marsh 4bc381183d make aws instsance builder work with recent changes for volume copy 2019-10-09 16:59:51 -07:00
Megan Marsh e283023d9b make sure step_register_ami in chroot builder respects encryption and ami skip build retion values once the copy happens 2019-10-09 16:52:41 -07:00
Megan Marsh 469af8e592 ebssurrogate builds didn't keep up with recent optimizations to region encryption in the ebs builder, and that was causing bugs with the common ami region copy step 2019-10-09 16:02:37 -07:00
Adrien Delorme d4600a7716 fix go imports 2019-10-04 11:19:45 +02:00
Megan Marsh 0026792d2f remove clean_ami_name and clean_image_name; complete the deprecation for these functions in favor of clean_resource_name 2019-10-03 15:49:01 -07:00
Megan Marsh e16fb19bfc don't abort on a single error if another availability zone was able to create the instance 2019-10-02 14:17:00 -07:00
Megan Marsh eccf0fb51d update docs 2019-09-27 09:51:18 -07:00
Megan Marsh e047b97bf7
Merge pull request #8162 from hashicorp/fix_8154
make sure amazon builders respect ssh_host option
2019-09-27 09:34:49 -07:00
Megan Marsh ac6e7a4800 updates tests to validate that ssh_host value supplied in the template overrides all else 2019-09-27 09:19:23 -07:00
Adrien Delorme 941dc394f0 Update step_run_spot_instance.go 2019-09-27 13:04:52 +02:00
Adrien Delorme d049284120
Update builder/amazon/common/step_run_spot_instance.go
remove unused code that might induce errors
2019-09-27 13:00:53 +02:00
Adrien Delorme 07e97e26de
step_run_source_instance.go: make sure we use the right ShouldRetry error 2019-09-27 12:57:55 +02:00
Megan Marsh c722009159 we only need one describeInstances call, not 3, and it should be wrapped in a retry 2019-09-26 15:47:01 -07:00
Megan Marsh e47059aaca put retry around describeinstances call in amazon 2019-09-26 14:49:28 -07:00
Megan Marsh 31bb1e818f fix tests 2019-09-26 13:02:11 -07:00
Megan Marsh 0cbd3ff0f3 make sure amazon builders respect ssh_host option 2019-09-26 12:30:04 -07:00
Megan Marsh 72d393c95a update docs to show a use case for block device mappings 2019-09-25 16:48:04 -07:00
Adrien Delorme 4fb05e5bc4 fix test 2019-09-20 15:28:43 +02:00
Adrien Delorme 792f61a4ad fmt 2019-09-20 15:18:48 +02:00
Adrien Delorme 849de792da re-apply doc changes of master 2019-09-20 15:04:01 +02:00
Adrien Delorme 3fe9d52e3f Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-09-20 11:15:44 +02:00
Adrien Delorme 4d4bd38afa
Merge pull request #8087 from hashicorp/fix_8048
add retry wrapper to query so it handles rate limiting
2019-09-16 10:25:12 +02:00
Megan Marsh 9503802814 adding ctx makes the cancel channel redundant 2019-09-13 14:32:13 -07:00
Megan Marsh 4b497006b3 add retry wrapper to query so it handles rate limiting 2019-09-13 14:32:13 -07:00
Adrien Delorme 146b88ba1e Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-09-10 12:44:46 +02:00
Adrien Delorme de9bb26f62
Merge branch 'master' into fix_7959 2019-09-10 12:21:33 +02:00
DanHam bbeb1a3528
Delete any `run_volume_tags` applied to EBS volumes at instance creation
* Allows `run_volume_tags` to be set without requiring a major rewrite
  of the (common) amazon builder code used to start an instance.
  The common start up code tags the instance and *all attached volumes
  at creation*. If `run_volume_tags` are set this means that any volumes
  specified in `ebs_volumes` will *initially* be tagged with the
  `run_volume_tags` rather than the tags set in the `ebs_volumes`
  section
* Once the instance is reported to be 'ready' the step to tag the EBS
  volumes is run. Once complete all volumes should have the tags
  requested by the user:

    * Volumes associated with the source instance should be tagged with
      the tags set in `run_volume_tags` (if any)
    * Each EBS volumes specified in the `ebs_volumes` section of the
      template should only be tagged with its associated tags (if any)
2019-08-30 14:30:25 +01:00
DanHam 61b1605a85
More accurate error description; Add comments; Add output 2019-08-30 14:26:06 +01:00
DanHam 430d41fbf9
Add option to enable tagging of the running instances volumes
* Currently this results in *all* volumes attached to the instance
  being tagged with the `run_volume_tags`. This includes any `ebs_volumes`
  for which the user may have configured other tags.
* This issue will be addressed in a later commit
2019-08-30 14:16:29 +01:00
DanHam 30070ffb30
Sort fields and options alphabetically 2019-08-30 14:16:29 +01:00
Adrien Delorme 9f8e9893b3 Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-08-29 14:22:16 +02:00
Adrien Delorme 86cee5cbef
Merge pull request #8034 from cove/retry-describe-instances
add AWS API call reties on AMI prevalidation
2019-08-29 10:48:54 +02:00
cove 87e0a3b5e7 add prevalidating AMI name retries so the build doesn't fail when throttled 2019-08-27 18:00:08 -07:00
Adrien Delorme 9d0bc476d9 add ommited struct interpolation
after dcc22df609
2019-08-27 15:17:57 +02:00
Adrien Delorme 239f97471c fix ebs volume tags doc field
Co-Authored-By: Megan Marsh <swampdragons@users.noreply.github.com>
2019-08-27 09:21:29 +02:00
Adrien Delorme 5517d41468 make sure all documentation fields are there
* add missing `temporary_key_pair_name` field for alicloud
* add missing description to `vpc_filter` for aws
* add missing ssh communicator fields for aws
* add links for aws
* update vmware-vmx docs
* manually describe AMI Configuration section for ebsvolume
* display missing required ami_name field for aws
* add missing fields for docker
* add missing fields for openstack
2019-08-26 17:16:54 +02:00
Adrien Delorme 2efc94521b use trilean initializers in test where missing 2019-08-23 11:17:45 +02:00
Megan Marsh 6a895db9be converte block device encryption to trilean 2019-08-22 14:24:22 -07:00
Megan Marsh 1d9bba6fe3 change AMIEncryptBootVolume to trilean 2019-08-22 14:04:26 -07:00
Megan Marsh 1a6adadb89 convert EnableAMIENASupport to trilean in code 2019-08-22 13:18:22 -07:00
Megan Marsh 3c3f7f26ce implement custom data type "trilean" (tri-state-boolean) to track booleans which have a "null" or "unset" state. Previously we used *bool for these template options, but it turns out that those won't work because "unset" will evaluate to "false" if a user is using template variables to set the option that maps to a *bool. 2019-08-22 13:18:15 -07:00
Adrien Delorme 4cb7c30987 Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-08-21 16:44:23 +02:00
Megan Marsh 4aa33306cb make sure that fleetId exists before trying to stick it into error message 2019-08-19 13:44:39 -07:00
Megan Marsh 316a30b31d fix spot_tags 2019-08-15 16:06:16 -07:00
Megan Marsh 9f6d14c75a work around spot_tags related regression and bad fleet cleanup in spot instances 2019-08-14 11:08:59 -07:00
Megan Marsh b42701380c add test for nil case 2019-07-29 12:11:59 -07:00
DanHam 088151e51d
Type assertion safety - fix panic when key cannot be retrieved from statebag 2019-07-28 10:44:24 +01:00
Christian Muehlhaeuser 216afbaa8b
Fixed various typos
Just a bunch of nit-picky typo fixes.
2019-07-25 06:49:51 +02:00
Megan Marsh 54d2ad5028 refactored step_ami_region_copy to fix bugs and clarify assumptions; added more tests for that step.
fix race condition caused by variable declaration outside of loop
2019-07-17 11:30:20 -07:00
Megan Marsh 39a4da4d07 add code to deduplicate the region list, and to make sure that the original region's kms key id is in the map. 2019-07-11 16:56:40 -07:00
Megan Marsh 6ad975f296 Revert "move SSHInterface/SSHIPVersion fields to communitator.Config struct"
This reverts commit b0c09087a2.
2019-07-02 10:46:10 -07:00
Megan Marsh 62120c5c0b
Merge pull request #7813 from hashicorp/stop_calculating_spot_bids
stop calculating spot bids
2019-07-02 10:35:45 -07:00
Megan Marsh d4b6a61445 don't store names of volumes we will be deleting when the instance terminates 2019-07-01 14:44:17 -07:00
Megan Marsh 42611e6004 fix build 2019-07-01 13:03:33 -07:00
Megan Marsh a60f7e395e add a test to make sure we don't error becaue of spot price auto product, yet 2019-07-01 13:01:41 -07:00
Megan Marsh c722afe707 stop calculating spot bids; amazon changed the way spot instances are priced to be stable rather than bid-based, so when user sets auto, we can just default to the ondemand price and know that they'll get the same price as everyone else bidding at that time. 2019-07-01 13:01:41 -07:00
Daniel Rodgers-Pryor 8a9753032c Allow EC2 fleet packer instances to run in parallel
Without conflicting over the name of the launch template that they create/delete.

Previously, the launch template name was just hard coded to `packer-fleet-launch-template`, but since AWS enforces unique template names within an account, this caused simultaneously running packer instances to hit template-already-exists errors when creating their templates and race-conditions around deleting the template. Now, the template name is randomly generated on each run, so there should be no conflicts.
2019-06-30 16:19:35 +10:00
Adrien Delorme 02421bad33 simplify BlockDevice prepares 2019-06-19 12:15:18 +02:00
Adrien Delorme 433c40d72d make block device example less specific 2019-06-19 12:10:08 +02:00
Adrien Delorme 8cea5d409a doc ebsvolume builder using partials 2019-06-19 12:02:12 +02:00
Adrien Delorme aeb898d598 document amazon-instance builder with partials 2019-06-19 11:09:00 +02:00
Adrien Delorme c54879717b document ebssurrogate with partials 2019-06-18 18:00:49 +02:00
Adrien Delorme 9f4c8fb321 document ebs builder using partials 2019-06-18 17:41:50 +02:00
Adrien Delorme a476ab79d7 document chroot builder using partials 2019-06-18 17:26:36 +02:00
Adrien Delorme dcc22df609 aws: move the OmitFromArtifact field in ebssurrogat where it's being used
also, simplified a bit the usage of block device
2019-06-18 16:04:12 +02:00
Adrien Delorme a86aae1c7e mode KmsKeyId to chroot block device, as it's only used there 2019-06-18 15:25:17 +02:00
Adrien Delorme 9989845ada add BuildEC2BlockDeviceMapping func to the BlockDevice type 2019-06-18 15:07:03 +02:00
Adrien Delorme c696878d2e aws block devices mapping: BlockDevices.Build -> BuildEC2BlockDeviceMappings() 2019-06-18 12:44:24 +02:00
Adrien Delorme ff2e0298ba simplify block devices utilisation by removing intermediary types 2019-06-18 12:37:47 +02:00
Adrien Delorme 78b29d1a74 up AMIBlockDevices docs 2019-06-18 11:55:12 +02:00
Megan Marsh 305592d8ed fix copy logic and tests 2019-06-17 15:38:28 -07:00
Megan Marsh 8cc82ca8d2 add broken test 2019-06-17 15:04:19 -07:00
Megan Marsh 0d55bc46ee add skip_save_build_region flag to fix naming conflicts when building for multiple regions 2019-06-17 14:39:11 -07:00
Adrien Delorme daa9f9d34e Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-06-14 15:17:56 +02:00
Megan Marsh 1c3b234b55
Merge pull request #7737 from hashicorp/document_retry
retry.Backoff: document
2019-06-13 10:54:08 -07:00
Adrien Delorme 6e3dc1aa9a Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-06-12 14:28:11 +02:00
Adrien Delorme 98206d59d7 aws: step_create_tags make the max waiting time 30s and not 30ns 2019-06-11 12:37:52 +02:00
Pratyush Singhal d92e82c43f refactor: replace the maxBackoff from 30 nanosecond to 30 seconds
Co-Authored-By: Adrien Delorme <azr@users.noreply.github.com>
2019-06-11 16:01:03 +05:30
Pratyush singhal d4b0cb68e3 feat: add retry for temp key-pair generation in amazon-ebs
Signed-off-by: Pratyush singhal <psinghal20@gmail.com>
2019-06-10 22:45:02 +05:30
Adrien Delorme 8e857d64f1 aws: rewrap struct comments for documentation generation 2019-06-06 17:34:17 +02:00
Adrien Delorme 4bf6a56d71 split communitacor.SSH & communitacor.WinRM to facilitate documentation
put doc for communicator into the struct
2019-06-06 16:55:24 +02:00
Adrien Delorme 4399684372 make fmt autogenerated docs 2019-06-06 16:29:25 +02:00
Adrien Delorme e09f3fbd02 amazon: update docs & links 2019-06-05 16:42:18 +02:00
Adrien Delorme e6cbb013ba add // go:generate struct-markdown to all previously edited files 2019-06-05 16:42:18 +02:00
Adrien Delorme f1917edd34 generate the comments for config struct of builders scraping doc website 2019-06-05 16:42:17 +02:00
Megan Marsh 3f036b33c5 test for length of ami tags before tagging 2019-06-03 19:14:06 -07:00
Megan Marsh 078d888dba
Merge pull request #7707 from CARFAX/master
Enable encrypted AMI sharing across accounts
2019-06-03 13:08:06 -07:00
Jason McIntosh abef9820c7 Fixed spacing on comment 2019-05-30 17:25:36 -05:00
Jason McIntosh 24a4e2f23e Updated comments 2019-05-30 17:18:33 -05:00
Jason McIntosh 24bc50ce1b Enable encrypted AMIs with custom KMS Keys 2019-05-30 17:16:36 -05:00
Megan Marsh 6786c0d641 update tests to reflect new behavior 2019-05-30 16:17:23 -05:00
Megan Marsh bda09bf7d5 the build ami is made with an intermediary name, which means that we can't skip copying even from the original build region when unencrypted 2019-05-24 15:08:20 -07:00
Megan Marsh 708165eaa8 refactoring step_run_spot_instance and swapping out the calls to create a spot instance with a call to create a spot fleet of one, so that we can be more flexible with the instance type when desired 2019-05-24 09:32:54 -07:00
Megan Marsh 1062295ecd add omit_from_artifact option for ebssurragote launch_block_devices 2019-05-10 11:12:11 -07:00
Megan Marsh 2cfba56265 fix typos 2019-05-07 16:44:29 -07:00
Megan Marsh 02c9f24531 allow user to set architecture 2019-05-07 16:38:55 -07:00
Megan Marsh 2c352cc681 add tests for step_ami_region_copy 2019-05-03 14:47:09 -07:00
Megan Marsh ae3cf48f71 in ebs, don't clean up temporary amis from encryption step. that's handled in step_copy_ami now 2019-05-02 14:38:56 -07:00
Megan Marsh 8c1d0f9686 clean up temporary unencrypted AMI and associated snapshots 2019-05-02 14:20:26 -07:00
Megan Marsh c0e69a9da0 if encrypt_boot is explicitly set to false, don't copy. Also, make sure we don't cause a wait group block when we aren't performing a copy. 2019-05-02 13:28:43 -07:00
Megan Marsh 105d3f8853 delete unencrypted temporary snapshot when encrypt_boot is true. Also, fix nil map error in region copy 2019-04-17 12:55:30 -07:00
Megan Marsh 201c8c1332 make sure that you encrypt using the provided kms key id and not the default 2019-04-15 16:44:56 -07:00
Adrien Delorme aee73d745d update code after go mod update 2019-04-11 14:19:24 +02:00
Adrien Delorme d72040f4fa move retry code into the common/retry pkg and make retry context aware 2019-04-09 17:46:38 +02:00
Adrien Delorme f555e7a9f2 allow a provisioner to timeout
* I had to contextualise Communicator.Start and RemoteCmd.StartWithUi
NOTE: Communicator.Start starts a RemoteCmd but RemoteCmd.StartWithUi will run the cmd and wait for a return, so I renamed StartWithUi to RunWithUi so that the intent is clearer.
Ideally in the future RunWithUi will be named back to StartWithUi and the exit status or wait funcs of the command will allow to wait for a return. If you do so please read carrefully https://golang.org/pkg/os/exec/#Cmd.Stdout to avoid a deadlock
* cmd.ExitStatus to cmd.ExitStatus() is now blocking to avoid race conditions
* also had to simplify StartWithUi
2019-04-08 20:09:21 +02:00
Adrien Delorme a81abd297b Merge remote-tracking branch 'origin/master' into context_provisioner 2019-04-08 20:09:01 +02:00
Megan Marsh 3e72e65a6c
Merge pull request #7456 from hashicorp/do_5770
introduce the clean_resource_name to clean image/var names
2019-04-05 09:12:20 -07:00
Adrien Delorme c4f3dccc14 rename interpolation context from ctx to ictx and contexts to ctx to avoid conflicts 2019-04-03 15:56:15 +02:00
Adrien Delorme a4bf94dd3c change Builder to be passed a context for cancellation
we have to to give it to our hook
2019-04-03 15:55:55 +02:00
Adrien Delorme 9836dad0c9 introduce the clean_resource_name to clean image names and deprecate the old ones 2019-04-03 11:25:51 +02:00
Akshat Mahajan f07e4214cc Addresses issues #5384, #5494: Rename and change `temporary_security_group_source_cidr`
to accept a list of strings (for Amazon builders).

Per this change, `temporary_security_group_source_cidr` in the configuration:

1. Will be renamed to `temporary_security_group_source_cidrs`.
2. Will accept a list of CIDRs.
3. Will have its documentation updated to reflect this change.
4. Will have a fixer attached for newer templates to avail of.
2019-03-30 15:47:03 -07:00
programmer04 596a9cc7ff Improve error message for AWS and Alibaba 2019-03-15 23:38:05 +01:00
Adrien Delorme 9f82b75e57 Use the hashicorp/go-getter to download files
* removed packer.Cache and references since packer.Cache is never used except in the download step. The download step now uses the new func packer.CachePath(targetPath) for this, the behavior is the same.
* removed download code from packer that was reimplemented into the go-getter library: progress bar, http download restart, checksuming from file, skip already downloaded files, symlinking, make a download cancellable by context.
* on windows if packer is running without symlinking rights and we are getting a local file, the file will be copied instead to avoid errors.
* added unit tests for step_download that are now CI tested on windows, mac & linux.
* files are now downloaded under cache dir `sha1(filename + "?checksum=" + checksum) + file_extension`
* since the output dir is based on the source url and the checksum, when the checksum fails, the file is auto deleted.
* a download file is protected and locked by a file lock,
* updated docs
* updated go modules and vendors
2019-03-13 12:11:58 +01:00
Megan Marsh 0196d7f835
Merge pull request #7370 from sargun/allow-users-to-specify-custom-block-device-mapping
amazon-chroot: Allow users to specify custom block device mapping
2019-03-12 14:21:23 -07:00
Megan Marsh 3ed9fe6dee remove unused imports
remove HttpClientWithEnvironmentProxy from amazon builders; let access config handle it
2019-03-04 16:42:33 -08:00
Megan Marsh 0ed57a37a3 dont overwrite session config in ebs builder 2019-03-04 16:41:31 -08:00
Megan Marsh ca9a5daf29 fix proxy and tls 2019-03-04 16:41:31 -08:00
Sargun Dhillon 601e754438 amazon/chroot: Allow creating new block device mappings !not fromScratch
Previously, when you built from an existing image, you were unable
to reconfigure block device mappings, as it just took them and
copied them over. This allows users to specify new, custom
block device mappings, even when building from an existing
image.
2019-03-04 13:11:14 -08:00
Megan Marsh ec3b0a5138 fix conflict from mergeing too many things to master at once 2019-03-01 11:13:32 -08:00
Megan Marsh 9d2b06bd61
Merge pull request #7342 from hashicorp/fix_7316
swap region copy and encryption steps
2019-03-01 10:26:43 -08:00
Megan Marsh 0abd16527e
Merge pull request #7337 from sargun/fix-chroot-builder
Amazon-chroot: Fix building PV images and where mount_partition is set
2019-03-01 10:25:04 -08:00
Adrien Delorme f535464097 up a test 2019-02-27 11:17:56 +01:00
Adrien Delorme f03cbd8a10 aws: allow to pass nil bool as encryption parameter
to retain current encryption settings

this changes the fields :
* ami_config.encrypt_boot
* block_device.encrypted

This also removes StepCreateEncryptedAMICopy as this step is now done in StepAMIRegionCopy
2019-02-27 11:17:56 +01:00
Adrien Delorme ce8532e94b aws: document ValidateRegion better 2019-02-27 11:17:08 +01:00
Adrien Delorme 6af621c37a aws: StepAMIRegionCopy call waitgroup.add only once 2019-02-27 11:17:08 +01:00
Michael F 01a0b0c766 kmsArnStartPattern - fix support for aws-us-gov
While the `kmsArnStartPattern` regexp supports `us-gov` as a region, it does not take the AWS partition into consideration.
For more info see https://docs.aws.amazon.com/govcloud-us/latest/ug-east/using-govcloud-arns.html

Added test case for AWS partition us-gov as well as a negative test case for a non-existing AWS partition.
2019-02-26 16:54:54 -08:00
Megan Marsh 026e543b24 move snapshot deletion to cleanup; this allows people with tag-specific permissions to perform snapshot deletion operation 2019-02-25 17:01:19 -08:00
Megan Marsh 3b726e66c1 swap region copy and encryption steps 2019-02-25 11:03:03 -08:00
Sargun Dhillon 142a84ef43 amazon/chroot: Refactor step_register_ami for testing EBS-related behaviour
There is some logic around how EBS mappings are handled / generated depending
on whether an image if from scratch or not. There was no testing around
this behaviour before. This strips that logic out into a separate function,
to enable testing it independentlhy.
2019-02-25 10:45:23 -08:00
Sargun Dhillon d3c65ee77d builder/amazon/chroot: Fix building PV images with mount_partition
Right now, if we have a source image that's PV, and try to build an
image with mount_partition set to not 0, it does not get picked up.
This is because under PV we only had a filesystem, not partitions,
but you can convert a PV image to an HVM image during build time.
2019-02-22 18:21:01 -08:00
Megan Marsh 2a613dd6e9 fix up the github suggestion 2019-02-11 11:49:17 -08:00
Adrien Delorme 3ae5a912d4
Update builder/amazon/common/step_pre_validate.go
Co-Authored-By: SwampDragons <megan@hashicorp.com>
2019-02-11 11:41:58 -08:00
Megan Marsh 3704a053d0 move region validation and credential wait into step pre validate 2019-02-05 14:07:04 -08:00
Megan Marsh 8add176ab7 finish first pass at vault code; needs testing and cleanup of error messages 2019-02-04 15:54:14 -08:00
Megan Marsh 7a78b47e83 make vault engine stuff into a little struct for easier management 2019-02-04 15:29:45 -08:00
Megan Marsh 035c3506ba manually set proxyfromenvironment in default http client for ec2 sessions 2019-01-25 12:33:25 -08:00
Adrien Delorme 4a2773f8fa remove default max retries config of 8, it will now be used from env
removing the code that wants to guess region from metadata allows us to change that setting, which can now be taken from environment.
2019-01-25 13:51:02 +01:00
Adrien Delorme 964b149df2 aws: remove now unused AccessConfig.metadataRegion 2019-01-25 13:03:14 +01:00
Adrien Delorme 335f442b51 aws: unwrap AccessConfig.region func
* it was used only in the Session() call.
* default region guessing from metadata should happen in the SDK, not 'manually'
2019-01-25 13:02:42 +01:00
Adrien Delorme 54af9951a8 make guard clauses to clearly see success pass 2019-01-25 13:00:21 +01:00
Adrien Delorme 0864b4d07b name unclear import 2019-01-25 12:59:43 +01:00
Megan Marsh e0543246fa remove commented code 2019-01-16 11:05:11 -08:00
Megan Marsh 783ed32e7e fix region stuff 2019-01-16 11:02:58 -08:00
Adrien Delorme 9f7b4ffc17
Merge branch 'master' into pr/6950 2019-01-09 10:11:18 +01:00
Megan Marsh ee2e5cc874
Merge pull request #7080 from hashicorp/do_6508
Move logging about aws waiters to the access config prepare() so that…
2018-12-12 15:42:37 -08:00
Adrien Delorme b7222d1f56 Add tmp package that offers Dir & File funcs
this regroups the calls of tmp func calls into one package.
the tmp pkg allows to store tmp files under a common directory for easier managment

Squashed commit of the following:
commit 74b674c01560c6b547e9cab5e8afb486f0fe7d6c
Merge: fc94b5714 66001525d
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 02:02:38 2018 -0400

    Merge branch 'remove_TMPDIR_docs' of github.com:tb3088/packer into remove_TMPDIR_docs

commit fc94b571428447aaf2e5171967c95cdf9e5c73ff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit 5a562359f4937af506607dd96ed1c46426093f52
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 9d5a4a0935bfd049e2f8f50ad73e4455b42c9410
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit fbae4c2e051898aecd8705e9d8a20353430ede23
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 3590fae8bd0afa92a161a49ed794ee32e357d583
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit d5c5306a97b648ded06a8eb76ce728f07c9924f0
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 0a72297da86089fbd07e4f7d9472a92fa4d1c733
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 20f68228b6c372d984ea5b055cfc8bda9b110ac5
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit c73ebe3d8321664a2685a0baa8e441dd895b0db4
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit 63549b8bd745a0090b15ed0b0ebf644162db27db
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit 976101074942db36f10d3a686d6d29ddb7c01926
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit 95159afbc05eac96ed11c3183af5fb0abe2f2d5c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 66001525d72de56a4cf9339b900c46f59bc6e91a
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit e9b6adefeae8c65eb8aa47fef38cbf0aa424338c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 852113ed076e2d14e5dca6815ea680da1e2896bb
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit 54add38d1d1c6e283cd444b367ed8bd49a5f3699
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 6b5b8f6d4edad0e187ca13d64ac4118f34eee643
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit c22092c601c33484327674f322c7379fa41506d7
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 7a730450916bf8e5dbc2a741ec233a49466ab7cc
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 0f2933adb6f6922dfeab78a95371a444ec8918ab
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit d74839ede05dacf712b55a7bb48aec19fe6b007f
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit eb65416619437e4a3dec90277770803dd5b2281c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit acaa2b31ed463219c4ef099f351eec72406e2989
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit e573fde668dcb418561e61535c1d68b2888f5b0f
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 39a9874afc9dea71b36753375fb7feb61ffecf69
Merge: 8a413cfe8 3315812c2
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 18:19:26 2018 -0400

    Merge branch 'master' of https://github.com/hashicorp/packer into prestine

commit 8a413cfe83b0a70dbd99c32d936334ca5788ca9b
Merge: e07491de5 4e14710a6
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Oct 1 20:18:10 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit e07491de59cead0b337a7f57f4a6c625e1f560ab
Merge: 42610a35d a1fa35dff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Aug 21 13:26:19 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 42610a35d5ef65af8844a58ad70e2ec75262da6a
Merge: 5298142da 0d63cf7bc
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue Jun 19 22:45:05 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 5298142da6da95354844f201deeeef3c599e48b2
Merge: 7bb110bc7 9d9736552
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Jun 11 15:10:09 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 7bb110bc744c9797fcaec25a265d9b85a22d6d6b
Merge: a61c869ca 6189d66e7
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Wed May 9 23:41:22 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit a61c869ca7268acf0e3b1e1fa7a8ee7feea65984
Merge: 098101dd9 4be5f072c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri May 4 10:13:04 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 098101dd99f08b0ca110d33eff6904537c6d21de
Merge: fefaf0fa6 554b2b4a5
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue May 1 20:52:30 2018 -0400

    Merge branch 'EOL-handling' into prestine

Co-Authored-By: Matthew Patton <pattonme@yahoo.com>
2018-12-12 16:35:57 +01:00
Tom Elliff 12bf7fc4d6 Check that the KMS key ID is valid
KMS key IDs can be given as either a raw key ID, an alias (always prefixed with 'alias/'), the full ARN to the KMS key ID or the full ARN to the KMS key alias.

Should help with https://github.com/hashicorp/packer/issues/6599 but probably isn't enough to fully close it.
2018-12-12 11:41:58 +00:00
Megan Marsh 042e9ad14b Move logging about aws waiters to the access config prepare() so that it's only spit out once per builder. 2018-12-06 14:37:31 -08:00
Megan Marsh c03894c05d fix bad interpolation exemption for volume_size 2018-11-29 16:39:43 -08:00
Megan Marsh da6e350949 fix tests 2018-11-20 15:20:01 -08:00
Megan Marsh fbb3d526e3 move region validation to run so that we don't break validation when we don't have aws creds set 2018-11-20 15:01:28 -08:00
Mikhail Ushanov 536252683e builders: reuse private key file reading function
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-11-08 14:25:50 +03:00
Megan Marsh 79862f1bbb
Merge pull request #6931 from hashicorp/fix_6919
aws: better error handling of region/credential guessing from metadata
2018-11-05 11:24:09 -08:00
Adrien Delorme 3f288fc6ca aws client: config.WithMaxRetries(8) 2018-11-01 15:09:07 +01:00
Mikhail Ushanov 4608b5d39d amazon: add option for skipping TLS verification
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-29 21:35:29 +03:00
Adrien Delorme f4fbf45349 remove duplicate region check error 2018-10-26 17:44:56 +02:00
Adrien Delorme 16aa21dffe aws: better error handling of region guessing from metadata
* this will fix #6919
2018-10-26 12:24:10 +02:00
Adrien Delorme ed793a8fb8 make ValidateRegion a member of *AccessConfig and make it variadic 2018-10-24 11:26:53 +02:00
Megan Marsh fe044d8ff7 fix tests 2018-10-23 13:31:56 -07:00
Megan Marsh 340009009d Revert "Revert "Merge pull request #6892 from hashicorp/fix_6890""
This reverts commit 0cd3f36d50.
2018-10-23 09:58:27 -07:00
Megan Marsh 0cd3f36d50 Revert "Merge pull request #6892 from hashicorp/fix_6890"
This reverts commit 1c4062c316, reversing
changes made to 9bc41a23b9.
2018-10-23 09:34:05 -07:00
Megan Marsh 0924a316b7 fix template-set credentials
update tests with mocked session
2018-10-22 13:53:40 -07:00
Megan Marsh 9573013d3a
Merge pull request #6872 from aspectcapital/disable-ena-support
Add support to explicitly disable ENA support
2018-10-19 10:54:38 -07:00
Matt Dainty feb8067c7d Convert `ena_support` to a pointer
This means it now has three states, `true`, `false`, & `nil`. The
default state is now `nil` which does nothing instead of `false` which
now will explicitly disable ENA support instead of just not enabling it.
2018-10-19 12:40:43 +01:00
Megan Marsh 57f87f58b9
Merge pull request #6374 from rickard-von-essen/aws-filters
builder/amazon: Add suppport for vpc_filter, subnet_filter, and security_group_filter
2018-10-18 14:34:02 -07:00
Megan Marsh 2266514408
Merge pull request #6843 from gmmephisto/pr-master-aws-security-group
aws: wait for temporary security group before authorize rules
2018-10-17 14:08:49 -07:00
Megan Marsh e7d3a30c0e
Merge pull request #6865 from hashicorp/fix_6862
Fix 6862
2018-10-17 13:07:29 -07:00
Mikhail Ushanov 00c96fc400 aws: use IpPermissions for authorizing temporary security group
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-17 22:00:25 +03:00