8a9753032c
Allow EC2 fleet packer instances to run in parallel
...
Without conflicting over the name of the launch template that they create/delete.
Previously, the launch template name was just hard coded to `packer-fleet-launch-template`, but since AWS enforces unique template names within an account, this caused simultaneously running packer instances to hit template-already-exists errors when creating their templates and race-conditions around deleting the template. Now, the template name is randomly generated on each run, so there should be no conflicts.
2019-06-30 16:19:35 +10:00
433c40d72d
make block device example less specific
2019-06-19 12:10:08 +02:00
9f4c8fb321
document ebs builder using partials
2019-06-18 17:41:50 +02:00
a476ab79d7
document chroot builder using partials
2019-06-18 17:26:36 +02:00
dcc22df609
aws: move the OmitFromArtifact field in ebssurrogat where it's being used
...
also, simplified a bit the usage of block device
2019-06-18 16:04:12 +02:00
a86aae1c7e
mode KmsKeyId to chroot block device, as it's only used there
2019-06-18 15:25:17 +02:00
9989845ada
add BuildEC2BlockDeviceMapping func to the BlockDevice type
2019-06-18 15:07:03 +02:00
c696878d2e
aws block devices mapping: BlockDevices.Build -> BuildEC2BlockDeviceMappings()
2019-06-18 12:44:24 +02:00
ff2e0298ba
simplify block devices utilisation by removing intermediary types
2019-06-18 12:37:47 +02:00
78b29d1a74
up AMIBlockDevices docs
2019-06-18 11:55:12 +02:00
305592d8ed
fix copy logic and tests
2019-06-17 15:38:28 -07:00
8cc82ca8d2
add broken test
2019-06-17 15:04:19 -07:00
0d55bc46ee
add skip_save_build_region flag to fix naming conflicts when building for multiple regions
2019-06-17 14:39:11 -07:00
daa9f9d34e
Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config
2019-06-14 15:17:56 +02:00
1c3b234b55
Merge pull request #7737 from hashicorp/document_retry
...
retry.Backoff: document
2019-06-13 10:54:08 -07:00
6e3dc1aa9a
Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config
2019-06-12 14:28:11 +02:00
98206d59d7
aws: step_create_tags make the max waiting time 30s and not 30ns
2019-06-11 12:37:52 +02:00
d92e82c43f
refactor: replace the maxBackoff from 30 nanosecond to 30 seconds
...
Co-Authored-By: Adrien Delorme <azr@users.noreply.github.com>
2019-06-11 16:01:03 +05:30
d4b0cb68e3
feat: add retry for temp key-pair generation in amazon-ebs
...
Signed-off-by: Pratyush singhal <psinghal20@gmail.com>
2019-06-10 22:45:02 +05:30
8e857d64f1
aws: rewrap struct comments for documentation generation
2019-06-06 17:34:17 +02:00
4bf6a56d71
split communitacor.SSH & communitacor.WinRM to facilitate documentation
...
put doc for communicator into the struct
2019-06-06 16:55:24 +02:00
4399684372
make fmt autogenerated docs
2019-06-06 16:29:25 +02:00
e09f3fbd02
amazon: update docs & links
2019-06-05 16:42:18 +02:00
e6cbb013ba
add // go:generate struct-markdown to all previously edited files
2019-06-05 16:42:18 +02:00
f1917edd34
generate the comments for config struct of builders scraping doc website
2019-06-05 16:42:17 +02:00
3f036b33c5
test for length of ami tags before tagging
2019-06-03 19:14:06 -07:00
078d888dba
Merge pull request #7707 from CARFAX/master
...
Enable encrypted AMI sharing across accounts
2019-06-03 13:08:06 -07:00
abef9820c7
Fixed spacing on comment
2019-05-30 17:25:36 -05:00
24a4e2f23e
Updated comments
2019-05-30 17:18:33 -05:00
24bc50ce1b
Enable encrypted AMIs with custom KMS Keys
2019-05-30 17:16:36 -05:00
6786c0d641
update tests to reflect new behavior
2019-05-30 16:17:23 -05:00
bda09bf7d5
the build ami is made with an intermediary name, which means that we can't skip copying even from the original build region when unencrypted
2019-05-24 15:08:20 -07:00
708165eaa8
refactoring step_run_spot_instance and swapping out the calls to create a spot instance with a call to create a spot fleet of one, so that we can be more flexible with the instance type when desired
2019-05-24 09:32:54 -07:00
1062295ecd
add omit_from_artifact option for ebssurragote launch_block_devices
2019-05-10 11:12:11 -07:00
2c352cc681
add tests for step_ami_region_copy
2019-05-03 14:47:09 -07:00
8c1d0f9686
clean up temporary unencrypted AMI and associated snapshots
2019-05-02 14:20:26 -07:00
c0e69a9da0
if encrypt_boot is explicitly set to false, don't copy. Also, make sure we don't cause a wait group block when we aren't performing a copy.
2019-05-02 13:28:43 -07:00
105d3f8853
delete unencrypted temporary snapshot when encrypt_boot is true. Also, fix nil map error in region copy
2019-04-17 12:55:30 -07:00
201c8c1332
make sure that you encrypt using the provided kms key id and not the default
2019-04-15 16:44:56 -07:00
d72040f4fa
move retry code into the common/retry pkg and make retry context aware
2019-04-09 17:46:38 +02:00
a81abd297b
Merge remote-tracking branch 'origin/master' into context_provisioner
2019-04-08 20:09:01 +02:00
3e72e65a6c
Merge pull request #7456 from hashicorp/do_5770
...
introduce the clean_resource_name to clean image/var names
2019-04-05 09:12:20 -07:00
c4f3dccc14
rename interpolation context from ctx to ictx and contexts to ctx to avoid conflicts
2019-04-03 15:56:15 +02:00
a4bf94dd3c
change Builder to be passed a context for cancellation
...
we have to to give it to our hook
2019-04-03 15:55:55 +02:00
9836dad0c9
introduce the clean_resource_name to clean image names and deprecate the old ones
2019-04-03 11:25:51 +02:00
f07e4214cc
Addresses issues #5384 , #5494 : Rename and change `temporary_security_group_source_cidr`
...
to accept a list of strings (for Amazon builders).
Per this change, `temporary_security_group_source_cidr` in the configuration:
1. Will be renamed to `temporary_security_group_source_cidrs`.
2. Will accept a list of CIDRs.
3. Will have its documentation updated to reflect this change.
4. Will have a fixer attached for newer templates to avail of.
2019-03-30 15:47:03 -07:00
596a9cc7ff
Improve error message for AWS and Alibaba
2019-03-15 23:38:05 +01:00
3ed9fe6dee
remove unused imports
...
remove HttpClientWithEnvironmentProxy from amazon builders; let access config handle it
2019-03-04 16:42:33 -08:00
ca9a5daf29
fix proxy and tls
2019-03-04 16:41:31 -08:00
f535464097
up a test
2019-02-27 11:17:56 +01:00
f03cbd8a10
aws: allow to pass nil bool as encryption parameter
...
to retain current encryption settings
this changes the fields :
* ami_config.encrypt_boot
* block_device.encrypted
This also removes StepCreateEncryptedAMICopy as this step is now done in StepAMIRegionCopy
2019-02-27 11:17:56 +01:00
ce8532e94b
aws: document ValidateRegion better
2019-02-27 11:17:08 +01:00
6af621c37a
aws: StepAMIRegionCopy call waitgroup.add only once
2019-02-27 11:17:08 +01:00
01a0b0c766
kmsArnStartPattern - fix support for aws-us-gov
...
While the `kmsArnStartPattern` regexp supports `us-gov` as a region, it does not take the AWS partition into consideration.
For more info see https://docs.aws.amazon.com/govcloud-us/latest/ug-east/using-govcloud-arns.html
Added test case for AWS partition us-gov as well as a negative test case for a non-existing AWS partition.
2019-02-26 16:54:54 -08:00
026e543b24
move snapshot deletion to cleanup; this allows people with tag-specific permissions to perform snapshot deletion operation
2019-02-25 17:01:19 -08:00
2a613dd6e9
fix up the github suggestion
2019-02-11 11:49:17 -08:00
3ae5a912d4
Update builder/amazon/common/step_pre_validate.go
...
Co-Authored-By: SwampDragons <megan@hashicorp.com>
2019-02-11 11:41:58 -08:00
3704a053d0
move region validation and credential wait into step pre validate
2019-02-05 14:07:04 -08:00
8add176ab7
finish first pass at vault code; needs testing and cleanup of error messages
2019-02-04 15:54:14 -08:00
7a78b47e83
make vault engine stuff into a little struct for easier management
2019-02-04 15:29:45 -08:00
035c3506ba
manually set proxyfromenvironment in default http client for ec2 sessions
2019-01-25 12:33:25 -08:00
4a2773f8fa
remove default max retries config of 8, it will now be used from env
...
removing the code that wants to guess region from metadata allows us to change that setting, which can now be taken from environment.
2019-01-25 13:51:02 +01:00
964b149df2
aws: remove now unused AccessConfig.metadataRegion
2019-01-25 13:03:14 +01:00
335f442b51
aws: unwrap AccessConfig.region func
...
* it was used only in the Session() call.
* default region guessing from metadata should happen in the SDK, not 'manually'
2019-01-25 13:02:42 +01:00
54af9951a8
make guard clauses to clearly see success pass
2019-01-25 13:00:21 +01:00
0864b4d07b
name unclear import
2019-01-25 12:59:43 +01:00
e0543246fa
remove commented code
2019-01-16 11:05:11 -08:00
783ed32e7e
fix region stuff
2019-01-16 11:02:58 -08:00
ee2e5cc874
Merge pull request #7080 from hashicorp/do_6508
...
Move logging about aws waiters to the access config prepare() so that…
2018-12-12 15:42:37 -08:00
12bf7fc4d6
Check that the KMS key ID is valid
...
KMS key IDs can be given as either a raw key ID, an alias (always prefixed with 'alias/'), the full ARN to the KMS key ID or the full ARN to the KMS key alias.
Should help with https://github.com/hashicorp/packer/issues/6599 but probably isn't enough to fully close it.
2018-12-12 11:41:58 +00:00
042e9ad14b
Move logging about aws waiters to the access config prepare() so that it's only spit out once per builder.
2018-12-06 14:37:31 -08:00
da6e350949
fix tests
2018-11-20 15:20:01 -08:00
fbb3d526e3
move region validation to run so that we don't break validation when we don't have aws creds set
2018-11-20 15:01:28 -08:00
536252683e
builders: reuse private key file reading function
...
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-11-08 14:25:50 +03:00
79862f1bbb
Merge pull request #6931 from hashicorp/fix_6919
...
aws: better error handling of region/credential guessing from metadata
2018-11-05 11:24:09 -08:00
3f288fc6ca
aws client: config.WithMaxRetries(8)
2018-11-01 15:09:07 +01:00
4608b5d39d
amazon: add option for skipping TLS verification
...
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-29 21:35:29 +03:00
f4fbf45349
remove duplicate region check error
2018-10-26 17:44:56 +02:00
16aa21dffe
aws: better error handling of region guessing from metadata
...
* this will fix #6919
2018-10-26 12:24:10 +02:00
ed793a8fb8
make ValidateRegion a member of *AccessConfig and make it variadic
2018-10-24 11:26:53 +02:00
fe044d8ff7
fix tests
2018-10-23 13:31:56 -07:00
340009009d
Revert "Revert "Merge pull request #6892 from hashicorp/fix_6890""
...
This reverts commit 0cd3f36d50
2018-10-23 09:58:27 -07:00
0cd3f36d50
Revert "Merge pull request #6892 from hashicorp/fix_6890"
...
This reverts commit 1c4062c3169bc41a23b9
2018-10-23 09:34:05 -07:00
0924a316b7
fix template-set credentials
...
update tests with mocked session
2018-10-22 13:53:40 -07:00
9573013d3a
Merge pull request #6872 from aspectcapital/disable-ena-support
...
Add support to explicitly disable ENA support
2018-10-19 10:54:38 -07:00
feb8067c7d
Convert `ena_support` to a pointer
...
This means it now has three states, `true`, `false`, & `nil`. The
default state is now `nil` which does nothing instead of `false` which
now will explicitly disable ENA support instead of just not enabling it.
2018-10-19 12:40:43 +01:00
57f87f58b9
Merge pull request #6374 from rickard-von-essen/aws-filters
...
builder/amazon: Add suppport for vpc_filter, subnet_filter, and security_group_filter
2018-10-18 14:34:02 -07:00
2266514408
Merge pull request #6843 from gmmephisto/pr-master-aws-security-group
...
aws: wait for temporary security group before authorize rules
2018-10-17 14:08:49 -07:00
e7d3a30c0e
Merge pull request #6865 from hashicorp/fix_6862
...
Fix 6862
2018-10-17 13:07:29 -07:00
00c96fc400
aws: use IpPermissions for authorizing temporary security group
...
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-17 22:00:25 +03:00
b316be51cd
aws: wait for temporary security group before authorizing
...
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-17 22:00:24 +03:00
b0cfecf314
remove unnecessary valid flag
2018-10-17 11:03:31 -07:00
9e0fae9db3
Add support to explicitly disable ENA support
...
If `ena_support` is set to false then it previously didn't do anything
whereas now it will explicitly disable ENA support.
Fixes #6852
2018-10-17 14:58:07 +01:00
d4767845f0
fix tests
2018-10-16 15:01:13 -07:00
7c3f0aa3b2
make sure region validation catches authentication errors
2018-10-16 14:17:30 -07:00
ee133482b4
aws: add validation for device_name parameter
...
cause it's required in block-device-mapping.
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-16 21:37:48 +03:00
02bfac118f
default wait time for amazon import to one hour.
2018-10-09 13:08:42 -07:00
f494d87dc5
Merge pull request #6747 from hashicorp/fix_6743
...
abort if image import fails
2018-10-09 12:56:28 -07:00
f44dfcc62f
Merge pull request #6727 from hashicorp/fix_6723
...
reset poll delay seconds default to 2 because it can actuall be used …
2018-10-03 11:15:42 +02:00
d616f0ba71
abort if image import fails
2018-09-20 13:18:59 -07:00
79093da6ad
skip region validation in tests that don't care; refactor Prepare func so we can test region validation logic with a mock
2018-09-19 10:30:08 -07:00
93f1155a14
remove test-related special casing
2018-09-18 08:08:36 -07:00
ef3c10c403
Modify Validate Region for test cases
2018-09-17 14:45:55 -07:00
98f13eaf29
Use DescribeRegions for aws region validation
2018-09-17 14:45:55 -07:00
f4e6d06e68
reset poll delay seconds default to 2 because it can actuall be used in one part of the conditional tree for applyEnvOverrides
2018-09-17 11:10:50 -07:00
770f61e682
spot instance creation, only set blockDurationMinutes when > 0
2018-09-12 10:21:39 -07:00
09797df958
use a UI wrapper to auto-decode error messages
...
update docs to reflect optional config
2018-09-11 16:05:43 -07:00
a0edaf6c46
Going to revert this change for now, becuase of potential issues that arise from calling Prepare() twice
...
Revert "use statebag instead of SetSharedState for winRM password"
This reverts commit b35acbd879
2018-09-10 16:48:42 -07:00
e02d0dacc4
Merge branch 'master' into aws-filters
2018-09-07 16:14:00 +02:00
b94e6e913e
Merge pull request #6670 from sargun/fix-ena-on-pv
...
Allow users to build hvm images from pv images to have ena_support
2018-09-06 12:50:46 -07:00
902497d6ce
Allow users to build hvm images from pv images to have ena_support
2018-09-04 18:16:52 -07:00
349f9ae62b
Merge pull request #6641 from hashicorp/increase_retries
...
Increase retries
2018-09-04 11:39:28 -07:00
2a2734505b
Merge pull request #6649 from hashicorp/fix_6569
...
Fix 6569
2018-09-03 08:44:07 +02:00
26aab49aaf
aws spot instances: add block_duration_minutes option ( #6638 )
...
* Add block_duration_minutes option
* int64 cannot be nil
* Update doc
* Fix formating
2018-09-03 08:33:58 +02:00
4472ecccb0
fix waiters that weren't applying env defaults
2018-08-31 13:30:41 -07:00
2b9f937c23
increase max retries from 3 to 20
2018-08-30 09:35:06 -07:00
b0c09087a2
move SSHInterface/SSHIPVersion fields to communitator.Config struct
2018-08-29 14:51:28 +02:00
0938b640cc
use key pair name from config
2018-08-29 14:40:33 +02:00
ef4ca9c48e
builder.amazon: use c.Comm for ssh
2018-08-29 14:40:33 +02:00
9a9b82715b
config.Comm.SSHKeyPair => SSHKeyPairName
2018-08-29 14:40:33 +02:00
3a4bb26150
builder.aws: use key pair from comm struct
2018-08-29 14:40:33 +02:00
293c485372
builder.aws: use SSHAgentAuth from Comm
2018-08-29 14:40:33 +02:00
ef08c441c9
builder.aws: use Comm field for StepKeyPair & StepGetPassword
2018-08-29 14:40:33 +02:00
51d2aac9f6
SSHPrivateKey => SSHPrivateKeyFile
2018-08-29 14:40:32 +02:00
b35acbd879
use statebag instead of SetSharedState for winRM password
2018-08-24 13:51:16 -07:00
23f62f221e
Only filter SG's on VPC if it's set. Only use VPC's and Subnets that are available
2018-08-24 20:24:48 +02:00
fa99f931b7
Merge pull request #6601 from hashicorp/fix_6536
...
Increase waiter timeout for AWS WaitUntilImageAvailable command
2018-08-24 09:35:58 -07:00
6bbcd3e914
Remove remaining unused ssh config funcs and add missing calls ( #6622 )
...
* Remove duplicate unused SSHConfig funcs
* call to SSHConfigFunc() for azure/arm builder
* ncloud too
* remove empty ncloud file
2018-08-24 16:00:40 +02:00
533967cb66
builder/amazon: Add suppport for security_group_filter
...
Allow selecting SG's using a filter. Also some cleanup of vpc_filter and
subnet_filter.
2018-08-24 12:42:38 +02:00
9840862757
builder/amazon: Add suppport for vpc_filter and subnet_filter
...
First step of adding support for discovering VPC's and Subnets using
filters.
2018-08-23 12:52:03 +02:00
ff6a039d5b
replace scrubconfig with packer.LogSecretFilter.Set
...
filter winrm password from logs
Add new root-level packer template option, sensitive-variables, to tell us what user variables to mark sensitive.
2018-08-20 15:35:55 -07:00
00c6ec979f
do 30 minutes not 25
2018-08-17 13:30:02 -07:00
00de0f4996
increase the timeout for waiting for an ami to become available
2018-08-17 13:27:19 -07:00
71cad4f2a9
fix tests
2018-08-14 12:30:05 -07:00
250da0ab49
fix security hole with ami filter
2018-08-13 17:01:13 -07:00
c1edcd3774
amazon-ebssurrogate clean up volumes
2018-07-26 09:38:59 +03:00
814c1cf2b2
spellfix
2018-07-18 13:00:45 -07:00
9fffe4056e
Merge pull request #5452 from smaato/tag-spot-requests
...
Enable tagging of spot requests
2018-07-16 13:00:07 -05:00
2fec76ea87
Check if spot price is empty, when spot_tags is set
2018-07-13 22:42:42 +02:00
b6b907f523
read in the environment variables that government wait time and polling time for the AWS wait in step_stop_ebs_volume
2018-07-13 09:43:28 -07:00
8e22803099
fix loading of environment variables for managing aws waiters
2018-07-11 13:10:38 -07:00
f657ca39c9
refactored env var code and tests so that I don't have to set env vars during tests to check logic
2018-07-11 10:36:21 -07:00
14166fdd99
update amazon import docs to include the env vars for setting aws waiter delays and timeouts
2018-07-11 10:36:21 -07:00
bfbe318727
fix the homegrown waiters
...
fix image import; issue was with wait options not being evaluated
2018-07-11 10:36:21 -07:00
f49a2d8aed
move UI call to before the wait; add context to these steps
2018-07-11 10:36:21 -07:00
cf63dd10bf
replace AMIStateRefreshFunc, InstanceStateRefreshFunc, our spot instance waiter, our chroot volume waiter, and our snapshot waiters with waiters form AWS's SDK.
2018-07-11 10:34:20 -07:00
3dbf1cb371
Enable tagging of spot requests
...
This adds a new parameter to the EBS builders named `spot_tags'. This
parameter accepts a map of tags, much like `tags'. These tags will be
applied to a spot request that is created.
Improve visibility.
2018-07-07 16:06:51 +02:00
58f73aabbf
Add ap-northeast-3 region
2018-06-15 07:15:41 +00:00
af7f7f2ce9
Fix redundant Sprintf
2018-05-24 21:05:03 +01:00
ec8b70721c
Use an explicit error message when an error is expected and we don't get one
...
Previously, if the validation check generating the error in the main code
is removed, the 'should error' tests would just return an empty message
2018-05-15 12:25:06 +01:00
Daniel Rodgers-Pryor
Adrien Delorme
Megan Marsh
Pratyush Singhal
Jason McIntosh
Akshat Mahajan
programmer04
Michael F
Adrien Delorme
Tom Elliff
Mikhail Ushanov
Matt Dainty
Anshul Sharma
Matt DeBoer
Sargun Dhillon
RenaudS
Rickard von Essen
Matthew Hooker
James Nugent
Mark Meyer
EC2 Default User
DanHam