Commit Graph

641 Commits

Author SHA1 Message Date
Megan Marsh 6986fb0e81 make upgrade set isotime func properly 2021-03-17 15:52:22 -07:00
Adrien Delorme fe21ff905d test that folders containing a - file won't hang forever 2021-03-15 11:51:21 +01:00
Adrien Delorme 832c0f2b2a actually run fmt tests and then remove debug statements 2021-03-15 11:41:14 +01:00
teddylear 22d373c279 Adding more debug logic 2021-03-15 11:36:10 +01:00
Adrien Delorme be7d7313c5 add tests for piping fmt 2021-03-12 11:25:10 +01:00
Adrien Delorme a115b428ac simplify fmt test case a little 2021-03-12 11:07:26 +01:00
Adrien Delorme 4e22147909 Merge remote-tracking branch 'origin/master' into pr/teddylear/10457 2021-03-12 10:38:04 +01:00
Wilken Rivera 3c282de6c3 Add maps for statically vendored components
This change adds a new set of maps for builders, provisioners, and
post-processors that store reference to components that were once part
of Packer and are now vendored. This file acts as a single place for
defining this vendored components, which are then merged into the main
components maps to be used in Packer.

Quick test to ensure the exoscale-import post-processor is still loaded
```
// Validate that exoscale-import is in the know post-procoessors list
~>  packer.test build docker_centos_shell_provisioner.pkr.hcl
Error: Unknown post-processor type "badlynamed-import"

  on docker_centos_shell_provisioner.pkr.hcl line 18:
  (source code not available)

known post-processors: [ucloud-import digitalocean-import docker-push
googlecompute-export manifest vsphere-template docker-tag vsphere checksum
docker-import exoscale-import yandex-export compress googlecompute-import
yandex-import vagrant-cloud alicloud-import amazon-import artifice shell-local
docker-save vagrant]

// Validate that exoscale-import get loaded
~>  packer.test build docker_centos_shell_provisioner.pkr.hcl
Error: Failed preparing post-processor-block "exoscale-import" ""

  on docker_centos_shell_provisioner.pkr.hcl line 18:
  (source code not available)

4 error(s) occurred:

* api_key must be set
* api_secret must be set
* image_bucket must be set
* template_zone must be set

==> Wait completed after 2 microseconds

==> Builds finished but no artifacts were created.
```
2021-03-09 10:09:13 -05:00
Wilken Rivera edd4567096 Remove plugin registration generation step
This change removes the generation of command/plugin.go so that plugin
registration can be managed manually. As we begin to break out plugins
we will need to begin modifying this file, possibly with stubs to allow
for the removal of plugins without breaking Packer's backwards
  compatibility with 1.7.0. After 1.8.0 is released we should be able to
  revet these changes so that we can continue to generate the plugin
  file for those plugins core to Packer.
2021-03-09 10:02:02 -05:00
Marc Falzon 125a2f1f76 Remove "exoscale-import" post-processor
This change removes the `exoscale-import` post-processor from the
upstream Packer repository, following extraction as a standalone plugin
in a dedicated repository (https://github.com/exoscale/packer-post-processor-exoscale-import)
2021-03-09 10:01:05 -05:00
Megan Marsh 71b815a5fd reword commits 2021-03-05 16:07:32 -08:00
Wilken Rivera 79481ed7ec Add vendored components map as opposed to removing code generation for command/plugin.go 2021-03-05 15:44:04 -05:00
Megan Marsh 1c3c1f17d9 stop generating plugins file to make it possible to vendor plugins 2021-03-05 15:33:34 -05:00
Megan Marsh a5b0e37d7e docker extraction POC 2021-03-05 15:33:34 -05:00
Sylvia Moss 610dde7f02
check for nil config map (#10730) 2021-03-05 11:10:00 +01:00
Megan Marsh 96b753f3b0
pin packer to golang 1.16 (#10702)
* pin packer to golang 1.16
* vet command/build_cancellation_test.go
2021-03-02 11:43:58 +01:00
sylviamoss 5ccbd27b72 Improve upgrade error and transform all variables with template eng to locals 2021-02-23 16:51:13 +01:00
sylviamoss 20e8f666d9 make locals out of variables with template engines 2021-02-23 14:18:43 +01:00
sylviamoss a1a5cf0113 upgrade variables with other variables 2021-02-22 17:16:19 +01:00
sylviamoss 2016d6baec Fix panic on upgrading variables json file 2021-02-22 16:07:37 +01:00
Megan Marsh dd167925d7
add machine readable to the cli help func and docs (#10658) 2021-02-19 11:17:47 +01:00
Sylvia Moss 37d05c85ff
upgrade pause_before (#10654) 2021-02-18 15:05:12 +01:00
Megan Marsh 00e503388e
change template parsing error to include warning about file extensions (#10652) 2021-02-18 09:59:30 +01:00
Megan Marsh 1710590418
Merge pull request #10633 from hashicorp/azr_init_no_magic_host
Packer init: remove host and namespace guessing
2021-02-16 14:32:20 -08:00
Wilken Rivera 728c5a217d Add test case for non-github hostname
Tests results on current branch; install succeeded which was not expected
```
2021/02/16 14:02:24 ui: Installed plugin example.com/sylviamoss/comment v0.2.19 in "/tmp/pkr-test-cfg-dir-6_pkr_config458005728/example.com/sylviamoss/comment/packer-plugin-comment_v0.2.19_x5.0_linux_amd64"
    init_test.go:361: InitCommand.Run() = 0, want 1
    init_test.go:381: unexpected dir hash after init:   string(
        -       "h1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
        +       "h1:iVtzkl/nVm2KiLvlz8rH56ME8QEqRxq8+XT2Lo6bzGU=",
          )
--- FAIL: TestInitCommand_Run (6.39s)
    --- PASS: TestInitCommand_Run/already-installed-no-op (0.01s)
        --- PASS: TestInitCommand_Run/already-installed-no-op/-subtest-0 (0.00s)
    --- PASS: TestInitCommand_Run/already-installed-upgrade (2.30s)
        --- PASS: TestInitCommand_Run/already-installed-upgrade/-subtest-0 (0.06s)
    --- PASS: TestInitCommand_Run/release-with-no-binary (0.17s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works (1.32s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works/-subtest-0 (0.01s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails (1.42s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails/-subtest-0 (0.01s)
    --- FAIL: TestInitCommand_Run/unsupported-non-github-source-address (1.18s)
```

Tests results after change with change in this branch
```
2021/02/16 14:03:14 [TRACE] getting available versions for the example.com/sylviamoss/comment plugin
2021/02/16 14:03:14 [TRACE] &{%!q(*github.Client=<nil>) "packer-getter-github-1.7.0-dev"} getter could not get release: example.com/sylviamoss/comment doesn't appear to be a valid github.com source address; check source and try again.
2021/02/16 14:03:14 [DEBUG] will try to install: []
2021/02/16 14:03:14 ui error: no release version found for the example.com/sylviamoss/comment plugin matching the constraint(s): "v0.2.19"
--- PASS: TestInitCommand_Run (5.38s)
    --- PASS: TestInitCommand_Run/already-installed-no-op (0.01s)
        --- PASS: TestInitCommand_Run/already-installed-no-op/-subtest-0 (0.00s)
    --- PASS: TestInitCommand_Run/already-installed-upgrade (2.08s)
        --- PASS: TestInitCommand_Run/already-installed-upgrade/-subtest-0 (0.07s)
    --- PASS: TestInitCommand_Run/release-with-no-binary (0.21s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works (1.20s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works/-subtest-0 (0.01s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails (1.88s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails/-subtest-0 (0.01s)
    --- PASS: TestInitCommand_Run/unsupported-non-github-source-address (0.00s)
```
2021-02-16 14:03:29 -05:00
Sylvia Moss e0557f84e9
Add access config to generated amazon ami data source (#10634) 2021-02-15 17:35:03 +01:00
Adrien Delorme 8208f425c8 addrs: remove Plugin.ForDisplay func, the String one does the job
to make things less confusing
2021-02-15 15:32:42 +01:00
Megan Marsh cac6b04546
update help text for hcl2_upgrade (#10624) 2021-02-15 11:03:44 +01:00
Megan Marsh 80ed988ffe
Hcl2 upgrade fix (#10625)
* fix random nil pointer dereference I found while debugging hcl2_upgrade issues

* fix hcl2_upgrade command by creating passthroughs for all text template fields
2021-02-15 10:40:21 +01:00
Megan Marsh b20dea6aec
Merge pull request #10614 from hashicorp/azr_acc_test_old_single_component_plugins
Acc test "old style" single component plugins
2021-02-12 14:22:29 -08:00
sylviamoss e69410b633 fix lint 2021-02-12 15:38:44 +01:00
sylviamoss 855ba9775a add docs 2021-02-12 15:31:13 +01:00
sylviamoss 76e3d57cae add with-annotation flag to hcl2_upgrade 2021-02-12 15:18:53 +01:00
teddylear 0637601eda Fixing recursive formatting tests to work on all platforms 2021-02-11 22:08:36 -05:00
Adrien Delorme 7089e0854a rename setup func to avoid name conflict 2021-02-11 18:00:18 +01:00
Adrien Delorme 40c2b2a153 add manually-installed-single-component-plugin-old-api-fails 2021-02-11 16:55:30 +01:00
Adrien Delorme 3677069010 test: manually-installed-single-component-plugin-works 2021-02-11 16:35:08 +01:00
Adrien Delorme b321c64c1e Update init_test.go 2021-02-11 15:26:33 +01:00
Adrien Delorme ad0ce49cec fix tmpdir creation 2021-02-11 15:17:05 +01:00
Adrien Delorme 021044e963 put name first 2021-02-11 14:56:42 +01:00
Adrien Delorme ba8484ee6b tests: pre make dir in case nothing happens 2021-02-11 14:49:52 +01:00
Adrien Delorme 3052e3c5d5 tests 2021-02-11 14:46:53 +01:00
Sylvia Moss 774c5903f6
Add error-cleanup-provisioner to HCL2 (#10604) 2021-02-11 10:23:15 +01:00
teddylear d85286e228 Merging in master 2021-02-10 17:44:11 -05:00
Sylvia Moss d28e6fe009
write timestamp local only when necessary (#10602) 2021-02-10 14:54:19 +01:00
Adrien Delorme ffbb110167
Test running plugins with fake and fresh plugins (#10595)
* show error as to why plugin discovery failed
* allow to run more manual tests after a plugin installation
* test that a freshly installed external plugin can run
2021-02-09 17:07:59 +01:00
Sylvia Moss f5006d0842
fix empty locals and source name with blank space (#10593) 2021-02-09 14:57:42 +01:00
Sylvia Moss d53488db68
Add aws_secretsmanager transformation to hcl2_upgrade (#10553) 2021-02-08 11:28:26 +01:00
teddylear d3754e3021 Updating recursive formatter tests to be cleaner and table driven 2021-02-06 18:02:26 -05:00
teddylear 40a97e29db Clean up recursive format tests to be more accurate 2021-02-06 18:02:26 -05:00
teddylear 93df53a275 Refactor recursive formatting test cases to be table driven 2021-02-06 18:02:26 -05:00
teddylear 6adf1f6659 Fixing recursive fmt tests syntax and adding test case when recursive
option is off
2021-02-06 18:02:26 -05:00
teddylear 261abe0cae Setting recursive fmt to false, updatting recursive fmt test to validate
formatted files
2021-02-06 18:02:26 -05:00
teddylear ab4b3a8465 Adding recursive flag to formatter to format subdirectories 2021-02-06 18:02:23 -05:00
Adrien Delorme 692433721d
Add some acceptance testing for Packer init commands (#10566) 2021-02-05 14:06:28 +01:00
Sylvia Moss ef4afafde9
Fix force flag for hcl2 provisioners and post-processors (#10571) 2021-02-05 10:57:14 +01:00
Adrien Delorme 92ccd5fa1d
init: show successful installs in cyan (#10557) 2021-02-04 11:11:42 +01:00
Adrien Delorme ed091163be
HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00
Sylvia Moss d1ada744e1
Aws Secrets Manager data sources (#10505) 2021-01-22 14:49:45 +01:00
Megan Marsh 93008045cd
Merge pull request #10500 from hashicorp/format_stdin
enable Packer fmt to read from stdin
2021-01-20 11:47:55 -08:00
Sylvia Moss 3c7944624a
(3) Add amazon-ami data source (#10467) 2021-01-20 11:05:03 +01:00
Sylvia Moss 291121dd55
(2) Implement datasources (#10440) 2021-01-20 10:37:16 +01:00
Megan Marsh 4323b49130 enable Packer fmt to read from stdin 2021-01-19 15:30:34 -08:00
Sylvia Moss e44cb9d7e5
Add Amazon AMI data source to hcl_upgrade command (#10491) 2021-01-19 10:21:39 +01:00
Megan Marsh 4cb6e07900
Hcl beta tag (#10493)
* change beta tag

* remove beta tag from hcl2_upgrade comment

* add a partial to describe the legacy json limitations

* exclude legacy category from templates until it is deployed to the website
2021-01-18 15:08:04 +01:00
Megan Marsh 54a75421ea regenerate 2021-01-15 11:59:36 -08:00
Megan Marsh 1cb9aea752 change markdown references to new pathing 2021-01-15 11:59:36 -08:00
Adrien Delorme 52d2d7fe5f
Copy `MapOf...` plugin types back into Packer core (#10466)
* move maps of plugins back in core

* go mod vendor

* more fixes

* fix imports

* Update core_test.go

* fix build

* more fixes

* more fixes

* up vendors after fixing sdk

* Update post_processor_mock.hcl2spec.go

* Leave implementatino of MapOf in the sdk for plugi tests

Other wise use the interface

* go mod tidy

* add MapOfDatasource type too
2021-01-13 12:14:06 +01:00
Tristan Watson 17c806fbf5
Removing obsolete packer HCL warnings from CLI for validate and command (#10461) 2021-01-11 11:50:23 +01:00
Megan Marsh 88b7b0e14a remove sdk 2020-12-17 13:29:25 -08:00
Wilken Rivera 4e58987026
command/fmt: Ensure all variable files ending in `.pkrvars.hcl` get formatted (#10377)
Before change
```
⇶  packer fmt -check /tmp/unformatted.pkrvars.hcl
Error: Cannot tell whether /tmp/unformatted.pkrvars.hcl contains HCL2 configuration data

⇶  echo $?
1
```

After fix
```
⇶  packer fmt -check /tmp/unformatted.pkrvars.hcl
/tmp/unformatted.pkrvars.hcl

⇶  echo $?
3

⇶  packer fmt -check command/test-fixtures/fmt
command/test-fixtures/fmt/unformatted.pkr.hcl
command/test-fixtures/fmt/unformatted.auto.pkrvars.hcl
command/test-fixtures/fmt/unformatted.pkrvars.hcl

```
2020-12-14 10:29:58 -05:00
Megan Marsh 39ab646236
move plugin and rpc code into sdk; other minor tweaks (#10359) 2020-12-09 12:39:54 +01:00
Megan Marsh 5576bb5caa move packer config constants next to the packer config 2020-12-03 10:17:35 -08:00
Megan Marsh b69d69095e move postprocessor to sdk, fix generation code 2020-12-01 14:48:55 -08:00
Megan Marsh be5763ec41 move provisioner interface into sdk 2020-12-01 14:25:14 -08:00
Megan Marsh ada91b24e9 extract builder interface to sdk 2020-12-01 13:42:42 -08:00
Megan Marsh d5971ca25e fix tests by moving mock hooks and communicators, and BasicUi definition, into SDK 2020-11-30 14:30:30 -08:00
Megan Marsh 56a45b04ab Move hook and communicator definitions to packer-plugin-sdk 2020-11-30 14:30:30 -08:00
Megan Marsh 14bdb9516c move Artifact and artifact mock to the sdk 2020-11-30 14:29:52 -08:00
Megan Marsh 001886670d move Ui definition into the packer plugin sdk. 2020-11-30 14:26:54 -08:00
Adrien Delorme 73b7499811
HCL2: version block: test validation & document version/availability (#10298)
* HCL2: Test that the packer block passes in packer validate

* HCL2: Test invalid packer blocks are invalid

* docs: state from which version the packer block is available
2020-11-23 10:27:26 -05:00
Megan Marsh 6a1a22151c move the various flag packages from helper to command, since command is the only package that uses them. 2020-11-16 14:23:35 -08:00
Megan Marsh efe11da4a3
Merge pull request #10254 from hashicorp/plugin-sdk-2
Plugin sdk 2
2020-11-16 11:29:20 -08:00
Megan Marsh 3e54e9ea80
Merge pull request #10235 from hashicorp/refactor_version_code
Refactor version code
2020-11-16 11:12:59 -08:00
Megan Marsh 6402362018 create packer-plugin-sdk directory and begin moving the relevant folders into it. 2020-11-13 10:47:36 -08:00
Adrien Delorme f44e912072
Update hcl2upgrade command to update env calls + tests (#10244) 2020-11-11 14:54:22 -05:00
Wilken Rivera acabc1c1aa
Add packer fmt command (#10225)
* Add packer fmt command

This change adds a new command that allows users to format one or more
HCL2 Packer configuration template files.

Related to: #9174

* command/fmt: Add check flag

Packer's fmt command now supports a check flag that will output the name
of any file that would be changed by the HCL2 formatting engine. The
check flag is mutually exclusive with the write flag and will only check
if formatting is needed.

The update write flag will now overwrite the source files with the newly
formatted HCL2 source unless the `-write=false` or `-check` is passed at
the command line.

* Returns a diagnostic error if Format is unable to show a diff - equivalent to `terraform fmt`
* Updates testing to run against #Format and not the private methods of the HCL2Formatter; fixes ShowDiff test failure on Windows
* Updates comments for exported functions

* Add docs for fmt command
2020-11-11 11:49:39 -05:00
Adrien Delorme deba1484ff
HCL2: allow calling env as input var default value (#10240)
* HCL2: allow to use env in default value of input variables
2020-11-11 11:27:32 +01:00
Adrien Delorme 17ec88246f
hcl2_upgrade update (#10221)
* handle template_dir, pwd funcs, handle min_packer_version
* gotpl packer_version becomes hcl2 packer.version
* uuid becomes hcl2 uuidv4
* test code with emojis
* handle all cases were we cannot guess what the argument was
* handle clean_resource_name cases
* up docs
2020-11-10 10:46:20 +01:00
Megan Marsh bc85854a53 refactor packer version out of hcltemplate code. 2020-11-09 12:29:53 -08:00
Adrien Delorme 6dd06fad14 add command/ tests 2020-11-02 15:50:38 +01:00
Adrien Delorme 4bc16455b4
HCL2: add a packer block with a required_version input setting (#10149)
* add the possibility to set the packer.required_version field; to make sure the template file works with that version of Packer
* add tests
* add documentation on packer.required_version

Example:

packer {
  required_version = ">= 1.2.0, < 2.0.0"
}
2020-10-27 10:03:36 +01:00
Megan Marsh fc619dc977
Merge pull request #9626 from featheredtoast/add-proxmox-vm-clone
builder/proxmox FEATURE: split Proxmox into proxmox-iso and proxmox-clone
2020-10-23 13:36:45 -07:00
Megan Marsh 66b81d9bee
add build.name to hcl code (#10114) 2020-10-16 10:57:42 +02:00
Megan Marsh 2cdc052d7c
Merge pull request #10097 from hashicorp/more_hcl_docs
Update HCL docs with a local resource-free example to empower users t…
2020-10-14 13:34:54 -07:00
Megan Marsh 00cc425b84 docs tweaks 2020-10-14 12:58:04 -07:00
Peter Willis f1b57b1e66
Fix typos 2020-10-14 15:04:18 -04:00
Peter Willis a81811663a
Fix typos 2020-10-14 14:52:17 -04:00
Jeff Wong 0f12414126
Merge branch 'master' into add-proxmox-vm-clone 2020-10-12 10:48:15 -07:00
James Cradock 153d4111fa
Fix typo in build argument description (#10066) 2020-10-08 14:58:22 +02:00