Commit Graph

54 Commits

Author SHA1 Message Date
Megan Marsh 7d658149ea update tests and website and generated partials to reflect new package format 2020-11-17 10:36:01 -08:00
Megan Marsh 1ffc358ada move tmp helper func into packer-plugin-sdk 2020-11-16 15:13:08 -08:00
Dmitry Borodaenko d18b7839b9 no panicwrap in plugins
As of mitchellh/panicwrap#25, a call to panicwrap.Wrapped() unsets the
cookie in the env, that makes packer plugin child process inherit an env
without the panicwrap cookie and panicwrap itself.

This trips up CleanupClients() in Packer's plugin client: instead of the
real plugin server it now kills its panicwrap parent -- which doesn't
forward SIGKILL to its child because it's not a signal that can be
caught -- and ends up indefinitely waiting in client.Kill() for an EOF
that will never come.

This workaround is to not even try to panicwrap in a plugin server.
2020-10-01 13:32:00 -07:00
Megan Marsh 15ee44e857 review comments 2020-09-23 11:33:51 -07:00
Megan Marsh ae8d359bf1 Switch to explicitly declaring TrackProgress on UI, using an underlying progress bar specifically stored on the struct. Export UiProgressTracker. These changes allow us to optionaly switch to the NoopProgressTracker when no TTY is set. 2020-09-21 16:05:14 -07:00
Megan Marsh ee5635722b enable force, debug, and on-error for hcl2 builds 2020-05-14 16:22:51 -07:00
nywilken fb76323c4d config: Fix loading external plugins from a packerconfig
This change introduces a loadExternalComponent which can be used for
loading a single plugin path. The function is a combination of
the discoverSingle and discoverExternalComponents functions.
2020-01-15 13:08:30 -05:00
Adrien Delorme 0785c2f6fc
build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Megan Marsh a14498ff00 remove timestamps from logger inside of subprocesses to prevent double logging; also remove some cruft and fix spelling in some logs 2019-12-09 12:13:18 -08:00
Megan Marsh e59c122dff build differently for openbsd because we can't cross compile the process library 2019-09-18 10:51:46 -07:00
Megan Marsh 09f63cea0b processinfo 2019-09-18 10:32:50 -07:00
Adrien Delorme 5f076d4328 allow building packer on solaris by removing progress bar and tty imports
fix #7586
2019-05-07 15:58:49 +02:00
Adrien Delorme fa7b922e7a main.go: don't set TTY interface from tty.Open() in case open fails
If tty.Open fails the return tty is still nil, but the interface type will be set; meaning in go that `TTY == nil` will always be false.

fix #7506
2019-04-25 10:48:40 +02:00
Adrien Delorme 513999b8e7
Close tty on exit (#7411) 2019-03-15 11:10:30 +01:00
Adrien Delorme 9f82b75e57 Use the hashicorp/go-getter to download files
* removed packer.Cache and references since packer.Cache is never used except in the download step. The download step now uses the new func packer.CachePath(targetPath) for this, the behavior is the same.
* removed download code from packer that was reimplemented into the go-getter library: progress bar, http download restart, checksuming from file, skip already downloaded files, symlinking, make a download cancellable by context.
* on windows if packer is running without symlinking rights and we are getting a local file, the file will be copied instead to avoid errors.
* added unit tests for step_download that are now CI tested on windows, mac & linux.
* files are now downloaded under cache dir `sha1(filename + "?checksum=" + checksum) + file_extension`
* since the output dir is based on the source url and the checksum, when the checksum fails, the file is auto deleted.
* a download file is protected and locked by a file lock,
* updated docs
* updated go modules and vendors
2019-03-13 12:11:58 +01:00
Adrien Delorme cda87d777f Update main.go
move machine readable comment to a more logical place.
2019-03-07 09:44:32 +01:00
Adrien Delorme d177a2647a stop piping stdin
setupStdin switched out stdin for a pipe so that we could close the writer end of the pipe when we receive an interrupt so plugins blocked on reading from stdin are unblocked.

But this is now handled using contexts.
2019-03-06 16:47:26 +01:00
Adrien Delorme 94d6fc10f5 init tty in main 2019-03-06 15:52:59 +01:00
Adrien Delorme b7222d1f56 Add tmp package that offers Dir & File funcs
this regroups the calls of tmp func calls into one package.
the tmp pkg allows to store tmp files under a common directory for easier managment

Squashed commit of the following:
commit 74b674c01560c6b547e9cab5e8afb486f0fe7d6c
Merge: fc94b5714 66001525d
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 02:02:38 2018 -0400

    Merge branch 'remove_TMPDIR_docs' of github.com:tb3088/packer into remove_TMPDIR_docs

commit fc94b571428447aaf2e5171967c95cdf9e5c73ff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit 5a562359f4937af506607dd96ed1c46426093f52
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 9d5a4a0935bfd049e2f8f50ad73e4455b42c9410
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit fbae4c2e051898aecd8705e9d8a20353430ede23
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 3590fae8bd0afa92a161a49ed794ee32e357d583
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit d5c5306a97b648ded06a8eb76ce728f07c9924f0
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 0a72297da86089fbd07e4f7d9472a92fa4d1c733
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 20f68228b6c372d984ea5b055cfc8bda9b110ac5
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit c73ebe3d8321664a2685a0baa8e441dd895b0db4
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit 63549b8bd745a0090b15ed0b0ebf644162db27db
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit 976101074942db36f10d3a686d6d29ddb7c01926
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit 95159afbc05eac96ed11c3183af5fb0abe2f2d5c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 66001525d72de56a4cf9339b900c46f59bc6e91a
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit e9b6adefeae8c65eb8aa47fef38cbf0aa424338c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 852113ed076e2d14e5dca6815ea680da1e2896bb
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit 54add38d1d1c6e283cd444b367ed8bd49a5f3699
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 6b5b8f6d4edad0e187ca13d64ac4118f34eee643
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit c22092c601c33484327674f322c7379fa41506d7
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 7a730450916bf8e5dbc2a741ec233a49466ab7cc
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 0f2933adb6f6922dfeab78a95371a444ec8918ab
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit d74839ede05dacf712b55a7bb48aec19fe6b007f
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit eb65416619437e4a3dec90277770803dd5b2281c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit acaa2b31ed463219c4ef099f351eec72406e2989
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit e573fde668dcb418561e61535c1d68b2888f5b0f
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 39a9874afc9dea71b36753375fb7feb61ffecf69
Merge: 8a413cfe8 3315812c2
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 18:19:26 2018 -0400

    Merge branch 'master' of https://github.com/hashicorp/packer into prestine

commit 8a413cfe83b0a70dbd99c32d936334ca5788ca9b
Merge: e07491de5 4e14710a6
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Oct 1 20:18:10 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit e07491de59cead0b337a7f57f4a6c625e1f560ab
Merge: 42610a35d a1fa35dff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Aug 21 13:26:19 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 42610a35d5ef65af8844a58ad70e2ec75262da6a
Merge: 5298142da 0d63cf7bc
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue Jun 19 22:45:05 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 5298142da6da95354844f201deeeef3c599e48b2
Merge: 7bb110bc7 9d9736552
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Jun 11 15:10:09 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 7bb110bc744c9797fcaec25a265d9b85a22d6d6b
Merge: a61c869ca 6189d66e7
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Wed May 9 23:41:22 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit a61c869ca7268acf0e3b1e1fa7a8ee7feea65984
Merge: 098101dd9 4be5f072c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri May 4 10:13:04 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 098101dd99f08b0ca110d33eff6904537c6d21de
Merge: fefaf0fa6 554b2b4a5
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue May 1 20:52:30 2018 -0400

    Merge branch 'EOL-handling' into prestine

Co-Authored-By: Matthew Patton <pattonme@yahoo.com>
2018-12-12 16:35:57 +01:00
Adrien Delorme 25bfd4aa43 document wrapConfig a little 2018-09-24 16:24:43 +02:00
Megan Marsh 76c6d871a9 don't use default panic wrap key 2018-09-21 17:01:54 -07:00
Matthew Hooker 31fcfe4bc2 PoC for filtering logs 2018-08-20 15:20:21 -07:00
M. Marsh eb557af8bf
Merge branch 'master' into f-autocomplete 2018-04-26 11:18:37 -07:00
Matthew Hooker c6299972b9
s/boot_command/bootcommand/ 2018-04-19 14:26:31 -07:00
Matthew Hooker 673245afcf
Replace boot command parser with PEG parser. 2018-04-19 14:26:28 -07:00
SwampDragons 997f8e4a2a
Merge pull request #5318 from hashicorp/sigtermcleanup
Gracefully clean up on SIGTERM
2018-02-02 11:56:56 -08:00
Matthew Hooker c3eb54f183
log when loading config from environment 2017-12-14 11:26:09 -08:00
Jeremy Voorhis 40625777c1 Enable autocomplete for top-level commands 2017-10-13 11:57:44 -07:00
Matthew Hooker afd394e0bd
only output telemetry logs when enabled. 2017-10-05 14:31:24 -07:00
Matthew Hooker fdaf4ed8d3
Gracefully clean up on SIGTERM 2017-09-08 11:42:32 -07:00
Matthew Hooker 0af5b4d1ad
don't do any logging in realMain.
We should write to os.Stderr explicitly, like we already do.

don't warn if config lookup fails in main
2017-06-21 15:23:31 -07:00
Matthew Hooker f534b85ee9
get chris's nits. 2017-06-16 17:23:41 -07:00
Matthew Hooker 3898556caa
log per eng-003 2017-06-16 17:09:42 -07:00
Matthew Hooker 35332e4498
fix panic with deadlines. update panicwrap dep 2017-06-15 13:23:05 -07:00
Matthew Hooker 7382382727
Add telemetry reporting through checkpoint
Will report builders/provisioner/post-processor types used per build,
and whether or not the build passed.

Will also report any panics we see.

You may opt out of this reporting by setting the environment variable
`CHECKPOINT_DISABLE`.
2017-06-15 13:21:11 -07:00
Matthew Hooker 81522dced0
move packer to hashicorp 2017-04-04 13:39:01 -07:00
Matthew Hooker d1b20b3d9c
remove a bunch of dead code.
https://github.com/dominikh/go-tools/tree/master/cmd/unused
2017-03-28 20:36:20 -07:00
Chris Bednarski 7a9c3bc2a0 Added -force truncation behavior for manifest, and added docs 2016-06-10 15:57:01 -07:00
Christopher Boumenot 728c496370 Move version to its own package. (#3460) 2016-04-21 13:19:43 -07:00
wrc 9674a30227 Fix -debug command line option misbehaviour: forbid stdin redirection to a pipe when run as plugin server 2016-03-13 13:14:44 +03:00
Chris Bednarski 33d1c17d29 Generate plugins.go from main.go 2016-02-05 11:17:05 -08:00
Mark Peek ba7814b0ed Merge pull request #2846 from markpeek/packer-tmp
Create docker temp files under packer.d when TMPDIR is not set
2015-10-26 17:09:22 -07:00
Chris Bednarski 7acdc1b6af Hide the plugin command from help output 2015-10-21 16:57:38 -07:00
Mark Peek 38c81cf3e3 Move ConfigFile() and ConfigDir() from package main to packer 2015-10-16 17:32:36 -07:00
Mark Peek e5a713ff01 Alternative fix for #2641: make random script name actually random 2015-10-11 13:31:09 -07:00
Mitchell Hashimoto 2498ad02c8 packer: validate minimum version [GH-2310] 2015-06-29 11:49:45 -07:00
Mitchell Hashimoto 511013dbe4 fix go vet warning 2015-06-13 17:05:38 -04:00
Mitchell Hashimoto 0dd80c0eca config file doesn't need to exist if set [GH-2225] 2015-06-13 10:17:42 -04:00
Mitchell Hashimoto 1ee2b014a6 packer: remove Ui/Cache from CoreConfig 2015-05-27 20:09:52 -07:00
Mitchell Hashimoto dc74ec5612 packer: remove Environment 2015-05-25 17:29:10 -07:00