Commit Graph

49 Commits

Author SHA1 Message Date
Adrien Delorme a0c09e85df
retry spot instance creation when an "Invalid IAM Instance Profile name" error pops up (#9810)
PutRolePolicy & AddRoleToInstanceProfile are eventually consistent but it is not possible to wait for them to be done here: 0785c2f6fc/builder/amazon/common/step_iam_instance_profile.go (L117-L134) which was causing the `CreateFleet` to fail (100% for me). So for now we retry a bit later. Waiting 5 seconds after the previously linked code also fixed this.

Test file:

```json
{
	"builders": [
		{
			"type": "amazon-ebs",
			"region": "eu-west-1",
			"ami_name": "ubuntu-16.04 test {{timestamp}}",
			"ami_description": "Ubuntu 16.04 LTS - expand root partition",
			"source_ami_filter": {
				"filters": {
					"virtualization-type": "hvm",
					"name": "ubuntu/images/*/ubuntu-xenial-16.04-amd64-server-*",
					"root-device-type": "ebs"
				},
				"owners": [
					"099720109477"
				],
				"most_recent": true
			},
			"spot_price": "0.03",
			"spot_instance_types": [
				"t2.small"
			],
			"encrypt_boot": true,
			"ssh_username": "ubuntu",
			"ssh_interface": "session_manager",
			"temporary_iam_instance_profile_policy_document": {
				"Version": "2012-10-17",
				"Statement": [
					{
						"Effect": "Allow",
						"Action": [
							"*"
						],
						"Resource": "*"
					}
				]
			},
			"communicator": "ssh"
		}
]}
```
2020-08-25 10:10:32 +02:00
sylviamoss 788dc32598 add aws_polling config option to override env variables 2020-08-17 17:15:32 +02:00
Megan Marsh a56942d3c7 change isAwsErr to an exported func so I can use it in other aws modules. 2020-07-21 15:30:45 -07:00
Megan Marsh a885e620ee don't error for no mappings if no mappings were set. 2020-07-08 11:55:56 -07:00
Anthony Dong 2322d3ae6b aws: retry fetching blockdevicemapping if empty 2020-06-25 10:48:24 +02:00
Megan Marsh fc36a37db8 only set NoDevice if NoEphemeral is set; otherwise, legit block device mappings get destroyed 2020-03-24 08:51:32 -07:00
Adrien Delorme d532bb376c wip 2020-03-16 17:47:44 +01:00
Wilken Rivera e1a46ec293
Fix gosimple S1025 linting errors (#8838)
Remove unneeded use of fmt.Sprintf for variables that are already strings.
2020-03-04 15:31:30 -05:00
Megan Marsh 84063d2132 implement ID and Type values in the provisioner template info sharing 2019-12-13 11:57:01 -08:00
Megan Marsh 67c3373107 enable no_ephemeral for spot devices, clean up ranging behavior, update docs 2019-11-21 12:46:31 -08:00
nywilken 488e539f63 builder/amazon: Add helper function for checking against AWSError
Replace all straight forward occurrences of `if err, ok := err.(awserr.Error)` with the `isAWSErr` helper function
2019-11-12 15:27:47 -05:00
Bernard Baltrusaitis 8b83cca064 adding temporary instance profile document policy 2019-10-19 20:56:18 +11:00
Megan Marsh e16fb19bfc don't abort on a single error if another availability zone was able to create the instance 2019-10-02 14:17:00 -07:00
Adrien Delorme 941dc394f0 Update step_run_spot_instance.go 2019-09-27 13:04:52 +02:00
Adrien Delorme d049284120
Update builder/amazon/common/step_run_spot_instance.go
remove unused code that might induce errors
2019-09-27 13:00:53 +02:00
Megan Marsh c722009159 we only need one describeInstances call, not 3, and it should be wrapped in a retry 2019-09-26 15:47:01 -07:00
Adrien Delorme 4cb7c30987 Merge remote-tracking branch 'origin/master' into scrape_doc_to_builder_struct_config 2019-08-21 16:44:23 +02:00
Megan Marsh 4aa33306cb make sure that fleetId exists before trying to stick it into error message 2019-08-19 13:44:39 -07:00
Megan Marsh 316a30b31d fix spot_tags 2019-08-15 16:06:16 -07:00
Megan Marsh 9f6d14c75a work around spot_tags related regression and bad fleet cleanup in spot instances 2019-08-14 11:08:59 -07:00
Megan Marsh 42611e6004 fix build 2019-07-01 13:03:33 -07:00
Megan Marsh c722afe707 stop calculating spot bids; amazon changed the way spot instances are priced to be stable rather than bid-based, so when user sets auto, we can just default to the ondemand price and know that they'll get the same price as everyone else bidding at that time. 2019-07-01 13:01:41 -07:00
Daniel Rodgers-Pryor 8a9753032c Allow EC2 fleet packer instances to run in parallel
Without conflicting over the name of the launch template that they create/delete.

Previously, the launch template name was just hard coded to `packer-fleet-launch-template`, but since AWS enforces unique template names within an account, this caused simultaneously running packer instances to hit template-already-exists errors when creating their templates and race-conditions around deleting the template. Now, the template name is randomly generated on each run, so there should be no conflicts.
2019-06-30 16:19:35 +10:00
Adrien Delorme dcc22df609 aws: move the OmitFromArtifact field in ebssurrogat where it's being used
also, simplified a bit the usage of block device
2019-06-18 16:04:12 +02:00
Adrien Delorme c696878d2e aws block devices mapping: BlockDevices.Build -> BuildEC2BlockDeviceMappings() 2019-06-18 12:44:24 +02:00
Adrien Delorme ff2e0298ba simplify block devices utilisation by removing intermediary types 2019-06-18 12:37:47 +02:00
Megan Marsh 708165eaa8 refactoring step_run_spot_instance and swapping out the calls to create a spot instance with a call to create a spot fleet of one, so that we can be more flexible with the instance type when desired 2019-05-24 09:32:54 -07:00
Adrien Delorme d72040f4fa move retry code into the common/retry pkg and make retry context aware 2019-04-09 17:46:38 +02:00
Megan Marsh 57f87f58b9
Merge pull request #6374 from rickard-von-essen/aws-filters
builder/amazon: Add suppport for vpc_filter, subnet_filter, and security_group_filter
2018-10-18 14:34:02 -07:00
Adrien Delorme 770f61e682 spot instance creation, only set blockDurationMinutes when > 0 2018-09-12 10:21:39 -07:00
Adrien Delorme e02d0dacc4
Merge branch 'master' into aws-filters 2018-09-07 16:14:00 +02:00
RenaudS 26aab49aaf aws spot instances: add block_duration_minutes option (#6638)
* Add block_duration_minutes option

* int64 cannot be nil

* Update doc

* Fix formating
2018-09-03 08:33:58 +02:00
Adrien Delorme 0938b640cc use key pair name from config 2018-08-29 14:40:33 +02:00
Rickard von Essen 533967cb66 builder/amazon: Add suppport for security_group_filter
Allow selecting SG's using a filter. Also some cleanup of vpc_filter and
subnet_filter.
2018-08-24 12:42:38 +02:00
James Nugent 9fffe4056e
Merge pull request #5452 from smaato/tag-spot-requests
Enable tagging of spot requests
2018-07-16 13:00:07 -05:00
Megan Marsh f49a2d8aed move UI call to before the wait; add context to these steps 2018-07-11 10:36:21 -07:00
Megan Marsh cf63dd10bf replace AMIStateRefreshFunc, InstanceStateRefreshFunc, our spot instance waiter, our chroot volume waiter, and our snapshot waiters with waiters form AWS's SDK. 2018-07-11 10:34:20 -07:00
Mark Meyer 3dbf1cb371 Enable tagging of spot requests
This adds a new parameter to the EBS builders named `spot_tags'. This
parameter accepts a map of tags, much like `tags'. These tags will be
applied to a spot request that is created.

Improve visibility.
2018-07-07 16:06:51 +02:00
Matthew Hooker cbdf9db832
Use contexts with amazon waiters. 2018-04-23 12:57:15 -07:00
ADO 507b3619e7 Expose more source ami data in the template 2018-03-30 10:47:11 +02:00
Matthew Hooker 21812fa17f
Add volume and run tags if in us-gov/china
We can't tag on instance creation when we're in "restricted" regions,
so let's add the tags after the resources have been created.

Adds methods to AccessConfig to detect if we're in China or US Gov
regions (i.e. "restricted").

Also turns tag:tag maps into a type, and moves methods around validating
and converting them to ec2Tags to methods of the type.
2018-02-05 16:39:20 -08:00
Matthew Hooker 7a189a83a1
fix imports
`find . -type f -name '*.go' -not -path "./vendor/*" -exec goimports -w {} \;`
2018-01-24 17:09:17 -08:00
Matthew Hooker a831d522be
change run signatures
Run now takes a context as well as a statebag. We'll assign the context
to the blank identifier to prevent namespace collisions. We'll let the
step authors opt-in to using the context.

`find . -iname "step_*.go" -exec gsed -i'' 's/func \(.*\)Run(/func \1Run(_ context.Context, /' {} \;`
2018-01-24 17:09:17 -08:00
Matthew Hooker 366dc3da0a
move multistep imports to helper.
gomvpkg -from "github.com/mitchellh/multistep" -to "github.com/hashicorp/packer/helper/multistep"
2018-01-24 17:09:15 -08:00
Matthew Hooker f5ea1e8312
Use WaitUntilInstanceReady waiter 2018-01-10 15:57:31 -08:00
Matthew Hooker dada63801b
also use waiter code for spot instances 2017-12-08 13:38:48 -08:00
Mark Meyer 7e1646826d Check if VolumeTags is empty before tagging volumes
Related to #5486
2017-10-23 21:10:40 +02:00
Mark Meyer dbb3c76032 Remove redundant step and its usage
StepTagEBSVolumes is no longer needed, since this functionality is now
taken over by StepRunSourceInstance and StepRunSpotInstance. So remove
this functionality from the codebase.
2017-10-12 21:40:21 +02:00
Mark Meyer 2661fd7869 Move building of spot instances into its own step 2017-10-04 12:38:21 +02:00