Commit Graph

58 Commits

Author SHA1 Message Date
Megan Marsh a805bc538e fix tests 2020-09-21 16:23:00 -07:00
Megan Marsh d826711e7a
File provisioner docs (#9735) 2020-08-10 13:15:27 +02:00
Maxim Lobanov dd2927f871 fix bug with copying a few files in order 2020-08-07 09:38:30 +03:00
Maxim Lobanov cf999e07db fix formatting 2020-08-06 21:14:53 +03:00
Maxim Lobanov cdd33d3ff2 proceed with next files when copy directory 2020-08-06 21:04:00 +03:00
Vladislav Rassokhin 2ac9f5a276 Fixup for #8883: interpolate in file provisioner
* `UploadDir` should receive interpolated `Destination`
* Render `Destination` only once
2020-08-02 21:09:47 +03:00
Adrien Delorme 40947c2bf6
HCL2: generate hcl tags with go-cty tags too (#9306)
This will allow to generate the config files:

```go
package main

import (
	"fmt"

	"github.com/hashicorp/hcl/v2/gohcl"
	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/hashicorp/packer/builder/alicloud/ecs"
)

func main() {
	name := "name"
	app := ecs.FlatConfig{
		AlicloudImageName: &name,
		ECSSystemDiskMapping: &ecs.FlatAlicloudDiskDevice{
			DiskName: &name,
		},
	}

	f := hclwrite.NewEmptyFile()
	block := gohcl.EncodeAsBlock(&app, `source "something" "something"`)
	f.Body().AppendBlock(block)
	fmt.Printf("%s", f.Bytes())
}

```

Will output:

```
source "something" "something" {
  packer_user_variables      = null
  packer_sensitive_variables = null
  image_name                 = "name"
  image_share_account        = null
  image_unshare_account      = null
  image_copy_regions         = null
  image_copy_names           = null
  tags                       = null
  tag                        = null
  system_disk_mapping        = { disk_category = null, disk_delete_with_instance = null, disk_descri
ption = null, disk_device = null, disk_encrypted = null, disk_name = "name", disk_size = null, disk_
snapshot_id = null }
  image_disk_mappings        = null
  ssh_remote_tunnels         = null
  ssh_local_tunnels          = null
  ssh_public_key             = null
  ssh_private_key            = null
}
```

This is a good first step for #9015 and #9282 

fix #9304
2020-05-28 11:19:00 +02:00
Adrien Delorme 840e67f775
HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Moss 7fbbbffd5c Interpolate file provisioner and add integration tests 2020-03-13 16:17:40 +01:00
Adrien Delorme 4b7132c87c Merge remote-tracking branch 'origin/master' into sharing_info 2019-12-17 11:57:09 +01:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Megan Marsh 39fd462b56 change all provision func signatures to use map[string]interface{} 2019-12-12 15:38:32 -08:00
Megan Marsh 0ca7c9f397 fix tests 2019-12-11 16:29:35 -08:00
Megan Marsh 6f418d0e54 get data sharing to a working state with the powershell provisioner 2019-12-11 15:43:38 -08:00
Megan Marsh f4c3501af5 pass struct of generated data into provision() call 2019-12-11 15:43:38 -08:00
Adrien Delorme 078ba7c8c3 commit old code generation tool
for history
2019-10-15 12:56:42 +02:00
Adrien Delorme c7ce4d598e change Provisioner to be passed a context for cancellation 2019-04-03 15:55:54 +02:00
Adrien Delorme 9f82b75e57 Use the hashicorp/go-getter to download files
* removed packer.Cache and references since packer.Cache is never used except in the download step. The download step now uses the new func packer.CachePath(targetPath) for this, the behavior is the same.
* removed download code from packer that was reimplemented into the go-getter library: progress bar, http download restart, checksuming from file, skip already downloaded files, symlinking, make a download cancellable by context.
* on windows if packer is running without symlinking rights and we are getting a local file, the file will be copied instead to avoid errors.
* added unit tests for step_download that are now CI tested on windows, mac & linux.
* files are now downloaded under cache dir `sha1(filename + "?checksum=" + checksum) + file_extension`
* since the output dir is based on the source url and the checksum, when the checksum fails, the file is auto deleted.
* a download file is protected and locked by a file lock,
* updated docs
* updated go modules and vendors
2019-03-13 12:11:58 +01:00
Megan Marsh 5c2da6b816 fix destination pathing so that it doesnt break on windows 2019-01-03 16:03:42 -08:00
Megan Marsh 7597f609b1 update winrmcp vendor and update error message in file provisioner to aid user who messes up file path name. update file provisioner docs 2018-09-21 14:51:32 -07:00
Megan Marsh 3f8407d95f Revert "Merge pull request #6700 from hashicorp/multi_progressbar"
This reverts commit 5470c08345, reversing
changes made to c427486bca.
2018-09-20 13:14:17 -07:00
Adrien Delorme dfeca9f0df split progressbars per object 2018-09-12 16:21:58 -07:00
Adrien Delorme bb59a70e8f progressbar: use int64 instead of uint64
* it's what's used for file sizes and used lib
2018-09-06 16:49:15 +02:00
Adrien Delorme 5b66069da0 tests: remove Ui stubs to use packer.NoopUi to deduplicate code 2018-09-06 15:53:59 +02:00
Adrien Delorme 7efe3cac3d todos 2018-09-06 15:53:59 +02:00
Adrien Delorme fd7cb47adc use proxy reader for download progress & stop storing total/current in downloaders 2018-09-06 15:53:59 +02:00
Adrien Delorme ddd96c513b first draft at self refreshing loading bar centralized/controlled by Ui 2018-09-06 15:53:59 +02:00
Ali Rizvi-Santiago cf9bbe3ecf Incorporated the hack demonstrated by @SwampDragons in order to deal with packer.rpc.Ui not exporting any information about what it's doing to anybody. 2018-08-20 21:55:18 -05:00
Ali Rizvi-Santiago 5726927cba Moved the progress bar out of packer.Ui and unlinked it out of all the packer.Ui implementations. Split up the terminal-related functions into a separate terminal.go and calculate the progress bar width by traversing through packer.Ui to avoid the issue with github.com/ugorji/go/codec serializing private members (or unsafe pointers) of structs. Shuffled some arguments around in getConsoleScreenBufferInfo in common/terminal_windows.go so that the interface forces the user to correctly declare a _CONSOLE_SCREEN_BUFFER_INFO type. 2018-08-20 21:55:18 -05:00
Ali Rizvi-Santiago 0f10032b3d Moved the progress bar from common to the packer.UI interface and refactored it so that the terminal width is calculated based on each interface which returns a custom progressbar specific to its ui. 2018-08-20 21:55:18 -05:00
Ali Rizvi-Santiago fd2fef8738 Added support for the progress bar to automatically determine its width using the minimum length from a packer.UI and the terminal dimensions via kernel32.GetConsoleScreenBufferInfo or an ioctl (TIOCGWINSZ) to "/dev/tty". 2018-08-20 21:55:18 -05:00
M. Marsh 56dd8cdb63
Merge pull request #3891 from nouney/f-generated-files
Add "generated" file tag to allow users to upload files created on the fly.
2018-04-05 16:08:38 -07:00
Matthew Hooker 81522dced0
move packer to hashicorp 2017-04-04 13:39:01 -07:00
Matthew Hooker 79287d7e47
simplify some code 2017-03-28 20:36:20 -07:00
Vasiliy Tolstov 00ebe9e1d3 various fixes
Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
2016-11-29 14:55:44 +03:00
开恒 96a637684b fix #3897
bug fix for file-provisioner download file, including unit-test code.
2016-09-19 10:36:18 +08:00
nouney 59dac451a5 provisioner/file: add "generated" key to allow files created on-the-fly. 2016-09-15 22:15:56 +02:00
Joel Scoble 94d158b0b7 remove debug fmt.Print from test 2016-07-07 18:12:54 -05:00
Joel Scoble 999b8b2ed0 file provisioner: download should create all dirs in the destination path 2016-07-06 15:42:26 -05:00
Vasiliy Tolstov feee19e4ed file provisioner improvements
* allow specify source/destination as dir
* allow specify many files as source

Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
2016-02-12 11:48:28 -08:00
Vasiliy Tolstov 036c9ddf7c fix file download provisioner
when file used with download direction we don't need
to check source on builder because it on machine.

Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
2015-06-23 12:27:27 +03:00
Mitchell Hashimoto 7c8e86c9c5 provisioner/*: fix interpolation context 2015-06-22 12:26:54 -07:00
Alexander Golovko 500d83b673 add download support to file provisioner 2015-06-15 15:08:56 -07:00
Mitchell Hashimoto 2b4df93f2f provisioner/*: interpolation 2015-05-27 14:50:20 -07:00
Ross Smith II d92179847d provisioner/file: set file/directory permissions
fixes #1064
2014-05-09 21:03:35 -07:00
Mitchell Hashimoto adbb490189 provisioner/file: support uploading directories [GH-251] 2013-09-09 14:00:06 -07:00
Mitchell Hashimoto 28bf6122bd provisioner/*: implement Cancel 2013-08-30 23:23:36 -07:00
Mitchell Hashimoto 095631107a communicator/ssh: UploadDir works properly 2013-08-25 20:30:56 -07:00
Mitchell Hashimoto 64d87bfaf3 *: switch to packer ConfigTemplate 2013-08-15 19:17:23 -07:00
Mitchell Hashimoto fb6d2754da packer: Add Machine func to Ui 2013-08-11 18:16:00 -07:00