0cf9b55c5c
amazon-ebs: validate IOPS ratio
2020-11-04 14:37:41 +01:00
7cb8af1441
amazon: remove region from state for chroot builder
2020-11-03 22:28:19 +01:00
8dab31b548
amazon: use spot tags for launch template tags
...
- use `spot_tags` instead of `run_tags` for launch template
- move region to `StepRunSpotInstance` from state
2020-11-03 22:16:00 +01:00
d561b404d6
amazon-ebs: add tags to launch template
2020-11-01 16:25:43 +01:00
4669c0f852
amazon-ebs: log state details on change
2020-10-31 16:00:19 +01:00
845a10867e
Merge pull request #10003 from hashicorp/ssm_session_retry
...
Add retry mechanism to retry SSM session creation
2020-10-29 16:02:54 -07:00
01d5e5ca76
test/amazon/ebs: Add acceptance test for Session Manager Interface connectivity
...
Test Results
```
...
2020/10/29 09:35:39 ui: test: Starting session with SessionId: wilken-00bcfae4d314f54e7
2020/10/29 09:35:40 [DEBUG] TCP connection to SSH ip/port failed: dial tcp [::1]:8047: connect: connection refused
2020/10/29 09:35:40 ui: test: Port 8047 opened for sessionId wilken-00bcfae4d314f54e7.
2020/10/29 09:35:45 [INFO] Attempting SSH connection to localhost:8047...
2020/10/29 09:35:45 [DEBUG] reconnecting to TCP connection for SSH
2020/10/29 09:35:45 ui: test: Connection accepted for session wilken-00bcfae4d314f54e7.
2020/10/29 09:35:45 [DEBUG] handshaking with SSH
2020/10/29 09:35:45 [DEBUG] SSH handshake err: ssh: handshake failed: ssh: invalid packet length, packet too large
2020/10/29 09:35:52 [INFO] Attempting SSH connection to localhost:8047...
2020/10/29 09:35:52 [DEBUG] reconnecting to TCP connection for SSH
2020/10/29 09:35:52 [DEBUG] handshaking with SSH
2020/10/29 09:35:52 [DEBUG] handshake complete!
2020/10/29 09:35:52 [DEBUG] Opening new ssh session
2020/10/29 09:35:53 [INFO] agent forwarding enabled
2020/10/29 09:35:53 ui: ==> test: Connected to SSH!
2020/10/29 09:35:53 Running the provision hook
2020/10/29 09:35:53 ui: ==> test: Stopping the source instance...
2020/10/29 09:35:53 ui: test: Stopping instance
2020/10/29 09:35:54 ui: ==> test: Waiting for the instance to stop...
2020/10/29 09:36:25 ui: ==> test: Creating AMI packer-ssm-test-1603978447 from instance i-0853cb6186a3406d5
2020/10/29 09:36:25 ui: test: AMI: ami-0868a41bbb2df77b3
2020/10/29 09:36:25 ui: ==> test: Waiting for AMI to become ready...
2020/10/29 09:37:59 ui: ==> test: Terminating the source AWS instance...
2020/10/29 09:37:59 ui error: ==> test: Bad exit status: -1
2020/10/29 09:38:15 ui: ==> test: Cleaning up any extra volumes...
2020/10/29 09:38:15 ui: ==> test: No volumes to clean up, skipping
2020/10/29 09:38:15 ui: ==> test: Deleting temporary security group...
2020/10/29 09:38:16 ui: ==> test: Deleting temporary keypair...
2020/10/29 09:38:16 Deregistering image ID (ami-0868a41bbb2df77b3) from region (us-east-1)
2020/10/29 09:38:17 Deregistered AMI id: ami-0868a41bbb2df77b3
2020/10/29 09:38:17 Deleted snapshot: snap-09602f15994bc9f51
--- PASS: TestBuilderAcc_SessionManagerInterface (249.87s)
PASS
```
2020-10-29 09:39:19 -04:00
aae1992649
remove default PauseBeforeSSM, this will have to be set manually
2020-10-29 13:38:03 +01:00
6c45f04467
Delete ssm_mock_funcs.go
2020-10-29 13:37:44 +01:00
5d06a6e6df
rename file correctly
2020-10-29 13:26:09 +01:00
c6e2dd5538
remove unit test file for now, I think that an acceptance test will be easier here
2020-10-29 13:22:17 +01:00
a4bd744955
simplify things a bit more
2020-10-29 13:11:07 +01:00
aef3d24213
Update step_create_ssm_tunnel.go
2020-10-29 12:31:01 +01:00
f329cb5b93
simplify code
2020-10-29 12:18:41 +01:00
b058de072a
move packer/builder/amazon/common.IsAWSErr to builder/amazon/common/awserrors.Matches
...
to avoid cyclic dependency issues
2020-10-29 12:02:41 +01:00
8e355d0fe7
Move ssm code to its own ssm package and make it singlethreaded
2020-10-29 11:57:29 +01:00
646b973bd3
Remove logic to retry a connection that reuses an existing SSM Session
...
After testing it was found that once an session is terminated via an
instance restart, console termination, or SSM agent restart. Any active
session will essentially be terminated and unusable. So knowing that it
is always best to start a new session and let the old one timeout get
terminated.
2020-10-28 06:36:23 -04:00
dff9cde775
Remove waitgroups
2020-10-28 06:36:23 -04:00
eb11009e2a
Check for closed channels as opposed to using a separate closeRetry channel
2020-10-28 06:36:23 -04:00
aa73cc7d7e
add close chan to avoid unwanted retries
2020-10-27 16:32:20 -04:00
1f62249097
add retry terminated session chan
2020-10-27 16:32:20 -04:00
8e3f3e514c
improve logs
2020-10-27 16:32:20 -04:00
b2c7897f58
add WaitGroup to avoid data race
2020-10-27 16:32:20 -04:00
1c2b469acd
add retry channel to ssm driver
2020-10-27 16:32:19 -04:00
f5e037e8b4
Merge remote-tracking branch 'origin/master' into azr_selectable_temp_keygen_type_gcp
2020-10-26 15:47:29 +01:00
25f4e24772
Merge pull request #10150 from nshalman/nshalman/tag-snapshots
...
amazon/ebssurrogate: apply snapshot tags right when taking snapshot
2020-10-23 15:57:18 -07:00
c0ce8a9414
rework fixer deprecation code so we know what plugins they relate to
2020-10-21 15:28:59 -07:00
83a672f2c9
amazon/ebssurrogate: apply snapshot tags right when taking snapshot
2020-10-21 16:17:37 -04:00
522d122858
regen all code
2020-10-21 12:04:10 +02:00
f51b231c17
Merge remote-tracking branch 'origin/master' into azr_selectable_temp_keygen_type_gcp
2020-10-20 16:39:01 +02:00
b8e6e2c081
Merge pull request #10085 from 10000coins/f-aws-tenancy
...
builder/amazon: Support for source instance tenancy
2020-10-19 12:49:07 -07:00
608307cd1e
Re-allow spot + tenancy. Validate tenancy is set to a usable value
2020-10-16 21:43:22 +00:00
d5d1a8708e
Add tests for Tenancy vs Spot Price
2020-10-16 21:43:22 +00:00
6967e02103
Add support for source instance tenancy to amazon builders
2020-10-16 21:43:22 +00:00
54032d6c11
gen code and docs
2020-10-15 16:51:23 +02:00
15f97421e2
builder/amazon: Add io2 as a supported volume type
...
io2 volumes are new as of 2020-08-24. This adds support for specifying
them in a packer template and having the iops value specified take
effect.
2020-10-14 09:01:17 -05:00
4c0ac6917e
switch to using aws base config, which means we're using code tested and maitnained by the terraform crowd.
2020-09-29 14:26:38 -07:00
96e3def294
Add pause_before_ssm config to pause before establishing the Session Manager session ( #9988 )
2020-09-25 10:19:19 +02:00
d66b358100
Merge pull request #9991 from hashicorp/b-amazon-ssm-infinite-loop
...
amazon/ssm_driver: Update log polling logic
2020-09-24 09:55:39 -07:00
2f2b5683a2
amazon/ssm_driver: Update log polling logic
...
This change checks for closed iochans related to the log polling
function and will break out of the loop when both channels are closed.
Which is an indicator that the SSM session has been terminated by some
external process. This fixes an issue where Packer hangs, due to the
infinite loop, when an SSM session is killed outside of Packer.
Related to #9442
2020-09-24 10:19:37 -04:00
429d431365
Implement assume_role for Packer. This functionality is copied shamelessly
...
from the Terraform AWS provider
2020-09-23 13:18:26 -07:00
d347bbec4c
Retry fleet creating only on invalid IAM instance profile ( #9946 )
2020-09-18 17:05:36 +02:00
eca9b2f30d
aws ebs_surrogate: fix unhonored kms_key_id ( #9959 )
2020-09-18 12:39:07 +02:00
7a7d3ff350
Remove comments about sharing with catsby
2020-09-03 09:14:40 -05:00
ab23e839c8
update sharing test to req aws account id
2020-09-02 11:44:38 -04:00
0d0bd9ce75
name fields of PolicyDocument correctly in HCL ( #9812 )
...
withouth this fix we would have had to do
```hcl
temporary_iam_instance_profile_policy_document {
statement {
action = ["*"]
effect = "Allow"
resource = ["*"]
}
version = "2012-10-17"
}
```
instead of the same document but with capitalised fields
2020-08-25 10:53:56 +02:00
a0c09e85df
retry spot instance creation when an "Invalid IAM Instance Profile name" error pops up ( #9810 )
...
PutRolePolicy & AddRoleToInstanceProfile are eventually consistent but it is not possible to wait for them to be done here: 0785c2f6fc/builder/amazon/common/step_iam_instance_profile.go (L117-L134)
2020-08-25 10:10:32 +02:00
1a0d5a93ee
fix max_attempts typo
2020-08-17 17:58:22 +02:00
788dc32598
add aws_polling config option to override env variables
2020-08-17 17:15:32 +02:00
8b4993e44c
fix docs for kms key ids ( #9766 )
2020-08-14 11:35:35 +02:00
54469c4728
Add all of the custom AWS template engines to GeneratedData for use b… ( #9751 )
2020-08-12 10:43:15 +02:00
8f362656ee
Update builder/amazon/ebs/step_create_ami.go
...
Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-08-07 08:02:57 -07:00
484dbfe6ad
improve error handling when waiting for image so that the resourcenotready error is bubbled up more clearly without verbose logging turned on
2020-08-06 14:58:05 -07:00
a28edbaa0b
Fix skip_save_build_region in the ebssurrogate builder. ( #9666 )
2020-07-31 16:01:46 -04:00
a687d6fe12
ci/tech-debt: Fix vmware acceptance tests ( #9636 )
...
* add kex algorithm option to ssh config
* regenerate code
* This commit fixes old vmware acceptance tests that have not been run in some time. It does this in two parts:
1) It modifies the minimal vmware build configuration to use a custom kex algorithm, which enables the ssh connection to succeed.
2) It modifies logic in reading and defaulting hardware config values, which was crashing.
3) It adds a new acceptance test with a preseed file to test loading from an http directory.
2020-07-23 09:40:57 +02:00
f9df185ef1
Merge pull request #9579 from hashicorp/fix_9562
...
wrap CreateImage call in a retry to account for eventual consistencey…
2020-07-22 08:43:10 -07:00
1f3b3f8fd9
change retry func to a 15 min timeout
2020-07-21 15:55:39 -07:00
a56942d3c7
change isAwsErr to an exported func so I can use it in other aws modules.
2020-07-21 15:30:45 -07:00
036ea238bf
wrap CreateImage call in a retry to account for eventual consistencey issues with image state
2020-07-21 15:30:45 -07:00
810f161d22
builder/amazon/common/block_device.go: replace tabs with words
2020-07-20 16:34:13 +02:00
966b17d2b5
builder/amazon/common/access_config.go: replace tabs with words
2020-07-20 16:31:23 +02:00
5c4901c2c2
builder/amazon/common/run_config.go: replace tabs with words
2020-07-20 16:28:14 +02:00
1455a29f2b
remove tabs for security_group_filter too
2020-07-20 16:24:47 +02:00
191094edcc
better `disable_stop_instance` docs
2020-07-20 16:21:43 +02:00
530f5e3f63
more additions to partials
2020-07-17 15:52:11 -07:00
a1651977b3
add tabs to generated docs from access_config and run_config structs
2020-07-17 15:10:55 -07:00
28f8241a05
update generated files
2020-07-10 16:58:06 -07:00
4d3b2cea96
Merge pull request #9480 from adongy/master
...
aws: retry fetching blockdevicemapping if empty
2020-07-08 12:32:12 -07:00
a885e620ee
don't error for no mappings if no mappings were set.
2020-07-08 11:55:56 -07:00
8ea64c5b35
undocument ssh_keypair_name and ssh_agent_auth
2020-07-02 14:55:58 +02:00
41edd09d3c
undocument temporary_key_pair_name and ssh_private_key_file
2020-07-01 16:33:18 +02:00
122e80f591
generated code
2020-06-30 10:12:50 -07:00
0122530a07
make statement an array of strings
2020-06-30 10:01:49 -07:00
2322d3ae6b
aws: retry fetching blockdevicemapping if empty
2020-06-25 10:48:24 +02:00
a8dfb66d6c
docs/builder/amazon: Add WinRM unsupported note to Session Manager docs
...
Closes #9451
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
2020-06-23 07:41:21 -04:00
b2320ca911
HCL2: add singular run_volume_tag block to ebs & ebssurrogate builders ( #9457 )
...
close #9280
related to #8889
2020-06-23 10:49:50 +02:00
4f77bb0c4a
add new ssh_ciphers option
2020-06-17 16:48:23 -07:00
4178625afb
re-add winrm_no_proxy option. Winrm code needs to be modified in order to read no_proxy env var when client is created rather than when net/http library is instantiated.
2020-06-16 10:27:30 -07:00
40947c2bf6
HCL2: generate hcl tags with go-cty tags too ( #9306 )
...
This will allow to generate the config files:
```go
package main
import (
"fmt"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclwrite"
"github.com/hashicorp/packer/builder/alicloud/ecs"
)
func main() {
name := "name"
app := ecs.FlatConfig{
AlicloudImageName: &name,
ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
DiskName: &name,
},
}
f := hclwrite.NewEmptyFile()
block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
f.Body().AppendBlock(block)
fmt.Printf("%s", f.Bytes())
}
```
Will output:
```
source "something" "something" {
packer_user_variables = null
packer_sensitive_variables = null
image_name = "name"
image_share_account = null
image_unshare_account = null
image_copy_regions = null
image_copy_names = null
tags = null
tag = null
system_disk_mapping = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
image_disk_mappings = null
ssh_remote_tunnels = null
ssh_local_tunnels = null
ssh_public_key = null
ssh_private_key = null
}
```
This is a good first step for #9015 and #9282
fix #9304
2020-05-28 11:19:00 +02:00
6d29ec956b
Added SourceAMICreationDate
2020-05-26 15:04:03 -04:00
b5639d4697
tests/builder/amazon: Enhance the test coverage for SSM Tunnel related steps ( #9213 )
...
* test/builder/amazon: Update mocks to properly test SSM tunnel and driver code base
* Update mod files
2020-05-13 10:10:55 -04:00
6f5f6bf999
builder/amazon Fix invalid pointer issue for non SSMAgengtEnabled builds
...
Tests before change
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1392ca2]
goroutine 299 [running]:
github.com/hashicorp/packer/builder/amazon/common.(*StepCreateSSMTunnel).Cleanup(0xc0003dc460, 0x4d1a4c0, 0xc0006e9800)
/home/wilken/Development/packer/builder/amazon/common/step_create_ssm_tunnel.go:95 +0xf2
github.com/hashicorp/packer/helper/multistep.(*BasicRunner).Run(0xc0006e98f0, 0x4d408c0, 0xc00065fcc0, 0x4d1a4c0, 0xc0006e9800)
/home/wilken/Development/packer/helper/multistep/basic_runner.go:79 +0x2c6
github.com/hashicorp/packer/builder/amazon/ebs.(*Builder).Run(0xc000726800, 0x4d408c0, 0xc00065fcc0, 0x4d5e300, 0xc0006e8d80, 0x4cc7220, 0xc000434120, 0x0, 0x0, 0x0, ...)
/home/wilken/Development/packer/builder/amazon/ebs/builder.go:330 +0x17e2
github.com/hashicorp/packer/packer.(*CoreBuild).Run(0xc000720500, 0x4d408c0, 0xc00065fcc0, 0x4d5e180, 0xc0006fe510, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/wilken/Development/packer/packer/build.go:287 +0x7ef
github.com/hashicorp/packer/command.(*BuildCommand).RunContext.func1(0xc0004d14d0, 0xc0003dc3c0, 0xc000441500, 0xa, 0x4d5e1e0, 0xc000720500, 0x4d408c0, 0xc00065fcc0, 0x4d5e180, 0xc0006fe510, ...)
/home/wilken/Development/packer/command/build.go:290 +0x189
created by github.com/hashicorp/packer/command.(*BuildCommand).RunContext
/home/wilken/Development/packer/command/build.go:284 +0xd5a
FAIL github.com/hashicorp/packer/provisioner/shell 188.335s
FAIL
```
Test After change
```
--- PASS: TestShellProvisioner (212.39s)
--- PASS: TestShellProvisioner/testing_amazon-ebs_builder_against_shell_provisioner (212.39s)
PASS
```
2020-05-12 14:49:57 -04:00
89fb7bb080
Apply suggestions from code review
...
Co-authored-by: Megan Marsh <megan@hashicorp.com>
2020-05-08 09:36:43 -04:00
056f1f6e76
docs/amazon/builder: Add not about using a temporary_iam_instance_profile_policy_document
2020-05-08 09:32:54 -04:00
bf1da57a05
Update address to localhost to be consistent with other proxies
2020-05-08 09:32:54 -04:00
7cea7466ba
buider/amazon: Add `session_manager_port` option for setting a custom local port
2020-05-08 09:32:54 -04:00
0d13c634b7
Update logging constructs for Session Manger driver
2020-05-08 09:32:49 -04:00
81c40b8d08
Add section to docs about the session-manager-plugin tunnel
2020-05-08 09:32:49 -04:00
5f68303504
Remove no longer needed port check
...
The communicator code base will default to the correct port settings for
valid communicators and fail the build for non valid communicators.
2020-05-08 09:32:49 -04:00
3ae7ab994b
step_create_ssm_tunnel: Add port availability check for LocalPortNumber
2020-05-08 09:32:49 -04:00
08dc2cb847
Remove call to unstubbed session start
2020-05-08 09:32:49 -04:00
f498866c3a
Update validation for session_manager connectivity
...
Validation will now check that either a IamInstanceProfile or a TemporaryIamInstanceProfilePolicyDocument is set
2020-05-08 09:32:49 -04:00
2f09b74587
amazon/builder: Move port validation for security group into RunConfig.Prepare
2020-05-08 09:32:48 -04:00
3f6b28b70c
Add SSMTunnel step to other amazon builders, minus chroot
2020-05-08 09:32:48 -04:00
743df19af2
Add tests for driver and ssm tunnel step
2020-05-08 09:32:48 -04:00
e53d6aea66
amazon/ssm_driver: Update to support command context
...
* Defer the release of the found port to prevent possible Packer build collisions.
2020-05-08 09:32:48 -04:00
5e14089851
tests/builder/amazon: Add basic testing for session_manager
...
* Add a test to ensure the proper host is returned when using session_manager
* Add a basic driver test for the validating that the SSMDriver will fail if the plugin is not in the path.
2020-05-08 09:32:48 -04:00
60e0ba7e68
Update iam instance profile name
2020-05-08 09:32:48 -04:00
d5a0262d6c
builder/amazon/instance: Add session manager support
2020-05-08 09:32:48 -04:00
e515aefee3
Add validation checks for session_manager
...
* Session manager connectivity only supported via SSH
* Session manager requires the use of an iam instance profile with system manager permissions
2020-05-08 09:32:48 -04:00
Aleksandr Serbin
Megan Marsh
Wilken Rivera
Adrien Delorme
sylviamoss
Nahum Shalman
Evan Pipho
Matt Rogers
Anthony Dong
catsby
Rae Krantz
Adrien Delorme
Etan Reisner
Roger Hu
nitrocode