Commit Graph

874 Commits

Author SHA1 Message Date
Paul Meyer ec14ab4875 Add unit test to find issue #7655 2019-05-17 22:27:19 +00:00
Megan Marsh 70c223045e fix race condition in hooks 2019-05-10 10:25:54 -07:00
Adrien Delorme 5f076d4328 allow building packer on solaris by removing progress bar and tty imports
fix #7586
2019-05-07 15:58:49 +02:00
Adrien Delorme 051f39e21d
Merge pull request #7583 from hashicorp/update_custom_plugin_docs
update docs defining Builders and PostProcessors
2019-04-30 11:58:53 +02:00
Megan Marsh 2e0f9223c8 remove all the extra debug lines 2019-04-29 10:33:41 -07:00
Adrien Delorme 1be337b383 update docs for definig Builders and PostProcessors 2019-04-29 13:16:46 +02:00
Megan Marsh b84b665ba3 fix race 2019-04-26 16:05:14 -07:00
Adrien Delorme 63f5bbbc73 use our own copy of iochan.LineReader 2019-04-11 14:25:24 +02:00
Adrien Delorme 8565a30c69 TimeoutProvisioner: also display an error log when the context times out 2019-04-09 17:46:38 +02:00
Adrien Delorme eadb40da91 Update communicator_test.go
fix tess
2019-04-08 20:09:21 +02:00
Adrien Delorme 2a90ce6178 packer communicator: use iochan.LineReader instead of iochan.LineReader(in)
* as it's the recommended way
2019-04-08 20:09:21 +02:00
Adrien Delorme d8d5631dc2 allow to set provisioner timeout from buildfile 2019-04-08 20:09:21 +02:00
Adrien Delorme f555e7a9f2 allow a provisioner to timeout
* I had to contextualise Communicator.Start and RemoteCmd.StartWithUi
NOTE: Communicator.Start starts a RemoteCmd but RemoteCmd.StartWithUi will run the cmd and wait for a return, so I renamed StartWithUi to RunWithUi so that the intent is clearer.
Ideally in the future RunWithUi will be named back to StartWithUi and the exit status or wait funcs of the command will allow to wait for a return. If you do so please read carrefully https://golang.org/pkg/os/exec/#Cmd.Stdout to avoid a deadlock
* cmd.ExitStatus to cmd.ExitStatus() is now blocking to avoid race conditions
* also had to simplify StartWithUi
2019-04-08 20:09:21 +02:00
Adrien Delorme f7cd2b9334 add a 5 seconds timeout to provisioner hook 2019-04-08 20:09:21 +02:00
Adrien Delorme a81abd297b Merge remote-tracking branch 'origin/master' into context_provisioner 2019-04-08 20:09:01 +02:00
Megan Marsh 1f7150e2ee
Merge pull request #7463 from hashicorp/default_keep_input_artifact
Default keep input artifact
2019-04-05 08:10:40 -07:00
Megan Marsh 96c94d2fa0 clean up code comments 2019-04-03 14:04:03 -07:00
Megan Marsh 10f47b5158 document clearly what keep_input_artifact does for each post-processor 2019-04-03 13:57:22 -07:00
Megan Marsh 056fcb7cea fix tests and add a few new ones 2019-04-03 11:32:49 -07:00
Megan Marsh c1960840e3 fix tests 2019-04-03 11:01:52 -07:00
Megan Marsh a358b174a4 make sure we filter the ui as well as the logs 2019-04-03 10:43:41 -07:00
Megan Marsh 365b32eb9c goofing 2019-04-03 09:43:39 -07:00
Adrien Delorme 82c4b76639 [Builder|Build|PostProcessor|Provisioner|Hook]Server: context cancel using a RPC cancel method 2019-04-03 15:55:55 +02:00
Adrien Delorme 14048b1e11 Revert "WIP: go towards context cancelled rpc pkg"
This reverts commit e4609b4a6a4ba2ec76a45ad67d50d180f1a914f9.
2019-04-03 15:55:55 +02:00
Adrien Delorme 9e3a179c0d WIP: go towards context cancelled rpc pkg 2019-04-03 15:55:55 +02:00
Adrien Delorme 39a5e4e612 rpc server register funcs: return errors 2019-04-03 15:55:55 +02:00
Adrien Delorme e65115a7a0 contextualize post-processor 2019-04-03 15:55:55 +02:00
Adrien Delorme a4bf94dd3c change Builder to be passed a context for cancellation
we have to to give it to our hook
2019-04-03 15:55:55 +02:00
Adrien Delorme 829851fc8a change hooks to be passed a context for cancellation
we have to as it is what calls our provisioners
2019-04-03 15:55:54 +02:00
Adrien Delorme c7ce4d598e change Provisioner to be passed a context for cancellation 2019-04-03 15:55:54 +02:00
Adrien Delorme dae02c0877 add context to Provisioner.Provision interface signature 2019-04-03 15:55:54 +02:00
Megan Marsh 12fc1fa751 default_keep_input_artifact 2019-04-02 16:51:58 -07:00
Megan Marsh 3aa0e51104 I use Dir instead of Base for mkdirall call 2019-03-28 13:17:15 -07:00
Adrien Delorme 42107c2b8b CachePath: only create base path 2019-03-19 15:30:01 +01:00
Adrien Delorme 5a6dffde9a use port as ints 2019-03-19 15:01:12 +01:00
Adrien Delorme 4cb58446f7 packer.CachePath: try to create dir if it doesn't exist 2019-03-19 12:54:15 +01:00
Adrien Delorme 513999b8e7
Close tty on exit (#7411) 2019-03-15 11:10:30 +01:00
Megan Marsh 4e76f51462
Merge pull request #7390 from hashicorp/do_4837
Allow user variables to be interpreted within the variables section o…
2019-03-14 11:22:23 -07:00
Megan Marsh b52ee14776 remove unreachable code 2019-03-14 10:05:10 -07:00
Megan Marsh 86c3c44afe switch on err type not string 2019-03-13 14:59:05 -07:00
Megan Marsh a62db1a0c6 add tests fixtures 2019-03-13 10:49:13 -07:00
Megan Marsh bf0d7b3620 make it work and add tests 2019-03-13 10:49:13 -07:00
Megan Marsh 7cb31714ad Allow user variables to be interpreted within the variables section of the template. 2019-03-13 10:49:13 -07:00
Adrien Delorme 9f82b75e57 Use the hashicorp/go-getter to download files
* removed packer.Cache and references since packer.Cache is never used except in the download step. The download step now uses the new func packer.CachePath(targetPath) for this, the behavior is the same.
* removed download code from packer that was reimplemented into the go-getter library: progress bar, http download restart, checksuming from file, skip already downloaded files, symlinking, make a download cancellable by context.
* on windows if packer is running without symlinking rights and we are getting a local file, the file will be copied instead to avoid errors.
* added unit tests for step_download that are now CI tested on windows, mac & linux.
* files are now downloaded under cache dir `sha1(filename + "?checksum=" + checksum) + file_extension`
* since the output dir is based on the source url and the checksum, when the checksum fails, the file is auto deleted.
* a download file is protected and locked by a file lock,
* updated docs
* updated go modules and vendors
2019-03-13 12:11:58 +01:00
Megan Marsh 6e442e616c
Merge pull request #7352 from hashicorp/research_7299
use a tty to ui.Ask
2019-03-07 09:47:01 -08:00
Adrien Delorme 55261f1bf3 Update ui_test.go
fix typo and tty usage
2019-03-06 16:09:08 +01:00
Adrien Delorme 94d6fc10f5 init tty in main 2019-03-06 15:52:59 +01:00
Adrien Delorme 22df491c67 mock tty using interfaces to test more easily 2019-02-28 14:40:55 +01:00
Adrien Delorme 239a0c633f use go-tty in ui.Ask to fix #7299 2019-02-27 16:59:59 +01:00
Adrien Delorme 0f7065f6b1 post-processor except: don't match empty names 2019-02-20 12:38:03 +01:00
xinau b449af84ee moved concurrency-safe ui code to packer/ui.go 2019-02-13 21:01:50 +00:00
Adrien Delorme dd3e2c255c respect a 'only' defined in a post-processor 2019-02-01 15:50:06 +01:00
Adrien Delorme 074a74ec38 make sure 'only' completely ignores post-processor
before this commit, if one would put a 'only' inside the post-processor definition, the post process could be skipped
2019-02-01 15:21:41 +01:00
Adrien Delorme 58245f2557 break a chain of post-processors when one is skipped & make `-only`
"blind" to post-processors

* to avoid trouble
* other arrays of post processors might still be there !
* add docs
* update tests
2019-01-11 14:08:30 +01:00
Adrien Delorme 61ade0e127 allow to use --only with post-processors 2019-01-10 15:44:34 +01:00
Adrien Delorme 4bf3cd44fc allow to skip a post processor 2019-01-10 15:44:34 +01:00
Adrien Delorme 848d4e981b clarify comments 2019-01-09 16:30:43 +01:00
Adrien Delorme 9f7b4ffc17
Merge branch 'master' into pr/6950 2019-01-09 10:11:18 +01:00
Adrien Delorme 7384b86e95 windows plugin: prioritize AppData over default user directory ( UserProfile )
It was the default behaviour before.
It is also standard for a binary to use files from the AppData directory. ex: c453e0c757/google/default.go (L104)

More on home: https://en.wikipedia.org/wiki/Home_directory
2019-01-04 14:14:38 +01:00
Megan Marsh a734aa2255 Revert "mux_broker_test.go: make non blocking errChan"
This reverts commit d9e32d437c. It turns out that it was already non-blocking with a length of one.
2018-12-17 13:49:47 -08:00
Adrien Delorme 895d49ca10
Merge pull request #7119 from hashicorp/fix_race
try to remove race condition in mux test
2018-12-17 11:42:55 +01:00
Adrien Delorme d9e32d437c
mux_broker_test.go: make non blocking errChan
* in case of Dial errors to avoid a deadlocks
2018-12-17 11:09:56 +01:00
Adrien Delorme 2944030daf
mux_broker_test.go: range over chan for simplicity 2018-12-17 11:01:46 +01:00
Adrien Delorme 9a31d0de58
mux_broker_test.go: defer close chan 2018-12-17 10:58:32 +01:00
Megan Marsh 70c6fcb824
Merge pull request #7078 from aspectcapital/issue-5478
Elevated support for puppet-* & chef-client provisioners
2018-12-14 19:06:46 -08:00
Megan Marsh f6070de070 fix test 2018-12-14 13:59:26 -08:00
Megan Marsh 52176ecf2d try to remove race condition in mux test 2018-12-14 13:53:39 -08:00
Adrien Delorme cecf352aa7 remove any reference to TMPDIR_SUFFIX. Using system tempdir is enough 2018-12-12 18:52:25 +01:00
Adrien Delorme b7222d1f56 Add tmp package that offers Dir & File funcs
this regroups the calls of tmp func calls into one package.
the tmp pkg allows to store tmp files under a common directory for easier managment

Squashed commit of the following:
commit 74b674c01560c6b547e9cab5e8afb486f0fe7d6c
Merge: fc94b5714 66001525d
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 02:02:38 2018 -0400

    Merge branch 'remove_TMPDIR_docs' of github.com:tb3088/packer into remove_TMPDIR_docs

commit fc94b571428447aaf2e5171967c95cdf9e5c73ff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit 5a562359f4937af506607dd96ed1c46426093f52
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 9d5a4a0935bfd049e2f8f50ad73e4455b42c9410
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit fbae4c2e051898aecd8705e9d8a20353430ede23
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 3590fae8bd0afa92a161a49ed794ee32e357d583
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit d5c5306a97b648ded06a8eb76ce728f07c9924f0
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 0a72297da86089fbd07e4f7d9472a92fa4d1c733
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 20f68228b6c372d984ea5b055cfc8bda9b110ac5
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit c73ebe3d8321664a2685a0baa8e441dd895b0db4
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit 63549b8bd745a0090b15ed0b0ebf644162db27db
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit 976101074942db36f10d3a686d6d29ddb7c01926
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit 95159afbc05eac96ed11c3183af5fb0abe2f2d5c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 66001525d72de56a4cf9339b900c46f59bc6e91a
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit e9b6adefeae8c65eb8aa47fef38cbf0aa424338c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 852113ed076e2d14e5dca6815ea680da1e2896bb
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit 54add38d1d1c6e283cd444b367ed8bd49a5f3699
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 6b5b8f6d4edad0e187ca13d64ac4118f34eee643
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit c22092c601c33484327674f322c7379fa41506d7
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 7a730450916bf8e5dbc2a741ec233a49466ab7cc
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 0f2933adb6f6922dfeab78a95371a444ec8918ab
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit d74839ede05dacf712b55a7bb48aec19fe6b007f
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit eb65416619437e4a3dec90277770803dd5b2281c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit acaa2b31ed463219c4ef099f351eec72406e2989
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit e573fde668dcb418561e61535c1d68b2888f5b0f
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 39a9874afc9dea71b36753375fb7feb61ffecf69
Merge: 8a413cfe8 3315812c2
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 18:19:26 2018 -0400

    Merge branch 'master' of https://github.com/hashicorp/packer into prestine

commit 8a413cfe83b0a70dbd99c32d936334ca5788ca9b
Merge: e07491de5 4e14710a6
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Oct 1 20:18:10 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit e07491de59cead0b337a7f57f4a6c625e1f560ab
Merge: 42610a35d a1fa35dff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Aug 21 13:26:19 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 42610a35d5ef65af8844a58ad70e2ec75262da6a
Merge: 5298142da 0d63cf7bc
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue Jun 19 22:45:05 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 5298142da6da95354844f201deeeef3c599e48b2
Merge: 7bb110bc7 9d9736552
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Jun 11 15:10:09 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 7bb110bc744c9797fcaec25a265d9b85a22d6d6b
Merge: a61c869ca 6189d66e7
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Wed May 9 23:41:22 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit a61c869ca7268acf0e3b1e1fa7a8ee7feea65984
Merge: 098101dd9 4be5f072c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri May 4 10:13:04 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 098101dd99f08b0ca110d33eff6904537c6d21de
Merge: fefaf0fa6 554b2b4a5
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue May 1 20:52:30 2018 -0400

    Merge branch 'EOL-handling' into prestine

Co-Authored-By: Matthew Patton <pattonme@yahoo.com>
2018-12-12 16:35:57 +01:00
Matt Dainty 19bd28cd72 Fix/add tests 2018-12-07 16:23:03 +00:00
Adrien Delorme d876978337
Fixed misspelling in a comment residing in packer/config_file.go as suggested by @azr.
Co-Authored-By: arizvisa <arizvisa@users.noreply.github.com>
2018-12-05 17:34:09 -06:00
Ali Rizvi-Santiago ecf45d4740 Added tests for packer.ExpandUser in packer/config_file_test.go. 2018-12-04 16:54:49 -06:00
Ali Rizvi-Santiago 040a33567e Trying again one more time to prevent import cycles.. Moved common.ExpandUser into packer.ExpandUser.. 2018-12-04 16:54:47 -06:00
Adrien Delorme 1055c119bc make code of homeDir, configFile & configDir not os specific anymore 2018-11-30 15:47:43 +01:00
Adrien Delorme 5147ac0364 use Getenv instead of LookupEnv so users can set USER="" and HOME="" + lookup different username only if it is different thant the current username 2018-11-30 14:56:53 +01:00
Ali Rizvi-Santiago eec68e319e Replaced unsafe method of determining a user's home directory with an implementation based on os/user. 2018-11-24 00:56:53 -06:00
Rémi Jouannet 118a9cb8c6 more explicit message 2018-11-09 14:44:34 +01:00
Matthew Hooker 098764d8ac
WIP 2018-10-26 16:59:20 -07:00
Adrien Delorme c393ccb7ca fix progress bar termination typo 2018-10-26 11:29:15 +02:00
Adrien Delorme 25775cd266 fix more race conditions 2018-10-15 16:31:59 +02:00
Adrien Delorme 27f4b9e4a1 pb/testing: triger more race condition by having the progress bar refresh more often 2018-10-15 15:28:46 +02:00
Adrien Delorme 4cae413a29 remove unedded atomic call ( protected by a lock ) 2018-10-15 15:28:08 +02:00
Adrien Delorme 56ccba86c7 fix some race conditions 2018-10-15 13:53:59 +02:00
Adrien Delorme cf2c309c5e add race conditions triggers for progress bar 2018-10-15 13:53:51 +02:00
Adrien Delorme 611610c89d BasicProgresssBar: protect add with a lock
to avoid weird race conditions.
2018-10-02 16:57:16 +02:00
Ashley Lowde 2884f6fab6 improve formatting and documentation for PR#6784 2018-10-02 20:00:45 +09:30
Ashley Lowde 12496e3702 add optional timestamps to build log 2018-09-29 22:39:24 +09:30
Megan Marsh 3f8407d95f Revert "Merge pull request #6700 from hashicorp/multi_progressbar"
This reverts commit 5470c08345, reversing
changes made to c427486bca.
2018-09-20 13:14:17 -07:00
Adrien Delorme be27775a3c start the pool only once 2018-09-12 17:04:10 -07:00
Adrien Delorme dfeca9f0df split progressbars per object 2018-09-12 16:21:58 -07:00
Megan Marsh 77980a7b53 fix progress bar mutex 2018-09-12 10:53:15 -07:00
Megan Marsh a0edaf6c46 Going to revert this change for now, becuase of potential issues that arise from calling Prepare() twice
Revert "use statebag instead of SetSharedState for winRM password"

This reverts commit b35acbd879.
2018-09-10 16:48:42 -07:00
Adrien Delorme 0ac7b64364 fix panic of the future using a mutex instead of some atomic calls + more docs 2018-09-06 18:12:15 +02:00
Adrien Delorme 8a851efcc8 progressbar: more godocs 2018-09-06 17:01:14 +02:00
Adrien Delorme bb59a70e8f progressbar: use int64 instead of uint64
* it's what's used for file sizes and used lib
2018-09-06 16:49:15 +02:00
Adrien Delorme f3c923c47d add tests for progress bar rpc calls 2018-09-06 16:15:09 +02:00
Adrien Delorme 6d3e36e6ea simplify remote progress bar as we are using a single instance 2018-09-06 16:14:59 +02:00
Adrien Delorme d89e1133c3 use freshly merged random.AlphaNum instead of our own random 2018-09-06 15:55:19 +02:00
Adrien Delorme 42561cf777 packer/rpc/ui_test.go: test progress bar too 2018-09-06 15:53:59 +02:00
Adrien Delorme 7efe3cac3d todos 2018-09-06 15:53:59 +02:00