Commit Graph

855 Commits

Author SHA1 Message Date
Megan Marsh c1960840e3 fix tests 2019-04-03 11:01:52 -07:00
Megan Marsh a358b174a4 make sure we filter the ui as well as the logs 2019-04-03 10:43:41 -07:00
Megan Marsh 365b32eb9c goofing 2019-04-03 09:43:39 -07:00
Adrien Delorme 82c4b76639 [Builder|Build|PostProcessor|Provisioner|Hook]Server: context cancel using a RPC cancel method 2019-04-03 15:55:55 +02:00
Adrien Delorme 14048b1e11 Revert "WIP: go towards context cancelled rpc pkg"
This reverts commit e4609b4a6a4ba2ec76a45ad67d50d180f1a914f9.
2019-04-03 15:55:55 +02:00
Adrien Delorme 9e3a179c0d WIP: go towards context cancelled rpc pkg 2019-04-03 15:55:55 +02:00
Adrien Delorme 39a5e4e612 rpc server register funcs: return errors 2019-04-03 15:55:55 +02:00
Adrien Delorme e65115a7a0 contextualize post-processor 2019-04-03 15:55:55 +02:00
Adrien Delorme a4bf94dd3c change Builder to be passed a context for cancellation
we have to to give it to our hook
2019-04-03 15:55:55 +02:00
Adrien Delorme 829851fc8a change hooks to be passed a context for cancellation
we have to as it is what calls our provisioners
2019-04-03 15:55:54 +02:00
Adrien Delorme c7ce4d598e change Provisioner to be passed a context for cancellation 2019-04-03 15:55:54 +02:00
Adrien Delorme dae02c0877 add context to Provisioner.Provision interface signature 2019-04-03 15:55:54 +02:00
Megan Marsh 12fc1fa751 default_keep_input_artifact 2019-04-02 16:51:58 -07:00
Megan Marsh 3aa0e51104 I use Dir instead of Base for mkdirall call 2019-03-28 13:17:15 -07:00
Adrien Delorme 42107c2b8b CachePath: only create base path 2019-03-19 15:30:01 +01:00
Adrien Delorme 5a6dffde9a use port as ints 2019-03-19 15:01:12 +01:00
Adrien Delorme 4cb58446f7 packer.CachePath: try to create dir if it doesn't exist 2019-03-19 12:54:15 +01:00
Adrien Delorme 513999b8e7
Close tty on exit (#7411) 2019-03-15 11:10:30 +01:00
Megan Marsh 4e76f51462
Merge pull request #7390 from hashicorp/do_4837
Allow user variables to be interpreted within the variables section o…
2019-03-14 11:22:23 -07:00
Megan Marsh b52ee14776 remove unreachable code 2019-03-14 10:05:10 -07:00
Megan Marsh 86c3c44afe switch on err type not string 2019-03-13 14:59:05 -07:00
Megan Marsh a62db1a0c6 add tests fixtures 2019-03-13 10:49:13 -07:00
Megan Marsh bf0d7b3620 make it work and add tests 2019-03-13 10:49:13 -07:00
Megan Marsh 7cb31714ad Allow user variables to be interpreted within the variables section of the template. 2019-03-13 10:49:13 -07:00
Adrien Delorme 9f82b75e57 Use the hashicorp/go-getter to download files
* removed packer.Cache and references since packer.Cache is never used except in the download step. The download step now uses the new func packer.CachePath(targetPath) for this, the behavior is the same.
* removed download code from packer that was reimplemented into the go-getter library: progress bar, http download restart, checksuming from file, skip already downloaded files, symlinking, make a download cancellable by context.
* on windows if packer is running without symlinking rights and we are getting a local file, the file will be copied instead to avoid errors.
* added unit tests for step_download that are now CI tested on windows, mac & linux.
* files are now downloaded under cache dir `sha1(filename + "?checksum=" + checksum) + file_extension`
* since the output dir is based on the source url and the checksum, when the checksum fails, the file is auto deleted.
* a download file is protected and locked by a file lock,
* updated docs
* updated go modules and vendors
2019-03-13 12:11:58 +01:00
Megan Marsh 6e442e616c
Merge pull request #7352 from hashicorp/research_7299
use a tty to ui.Ask
2019-03-07 09:47:01 -08:00
Adrien Delorme 55261f1bf3 Update ui_test.go
fix typo and tty usage
2019-03-06 16:09:08 +01:00
Adrien Delorme 94d6fc10f5 init tty in main 2019-03-06 15:52:59 +01:00
Adrien Delorme 22df491c67 mock tty using interfaces to test more easily 2019-02-28 14:40:55 +01:00
Adrien Delorme 239a0c633f use go-tty in ui.Ask to fix #7299 2019-02-27 16:59:59 +01:00
Adrien Delorme 0f7065f6b1 post-processor except: don't match empty names 2019-02-20 12:38:03 +01:00
xinau b449af84ee moved concurrency-safe ui code to packer/ui.go 2019-02-13 21:01:50 +00:00
Adrien Delorme dd3e2c255c respect a 'only' defined in a post-processor 2019-02-01 15:50:06 +01:00
Adrien Delorme 074a74ec38 make sure 'only' completely ignores post-processor
before this commit, if one would put a 'only' inside the post-processor definition, the post process could be skipped
2019-02-01 15:21:41 +01:00
Adrien Delorme 58245f2557 break a chain of post-processors when one is skipped & make `-only`
"blind" to post-processors

* to avoid trouble
* other arrays of post processors might still be there !
* add docs
* update tests
2019-01-11 14:08:30 +01:00
Adrien Delorme 61ade0e127 allow to use --only with post-processors 2019-01-10 15:44:34 +01:00
Adrien Delorme 4bf3cd44fc allow to skip a post processor 2019-01-10 15:44:34 +01:00
Adrien Delorme 848d4e981b clarify comments 2019-01-09 16:30:43 +01:00
Adrien Delorme 9f7b4ffc17
Merge branch 'master' into pr/6950 2019-01-09 10:11:18 +01:00
Adrien Delorme 7384b86e95 windows plugin: prioritize AppData over default user directory ( UserProfile )
It was the default behaviour before.
It is also standard for a binary to use files from the AppData directory. ex: c453e0c757/google/default.go (L104)

More on home: https://en.wikipedia.org/wiki/Home_directory
2019-01-04 14:14:38 +01:00
Megan Marsh a734aa2255 Revert "mux_broker_test.go: make non blocking errChan"
This reverts commit d9e32d437c. It turns out that it was already non-blocking with a length of one.
2018-12-17 13:49:47 -08:00
Adrien Delorme 895d49ca10
Merge pull request #7119 from hashicorp/fix_race
try to remove race condition in mux test
2018-12-17 11:42:55 +01:00
Adrien Delorme d9e32d437c
mux_broker_test.go: make non blocking errChan
* in case of Dial errors to avoid a deadlocks
2018-12-17 11:09:56 +01:00
Adrien Delorme 2944030daf
mux_broker_test.go: range over chan for simplicity 2018-12-17 11:01:46 +01:00
Adrien Delorme 9a31d0de58
mux_broker_test.go: defer close chan 2018-12-17 10:58:32 +01:00
Megan Marsh 70c6fcb824
Merge pull request #7078 from aspectcapital/issue-5478
Elevated support for puppet-* & chef-client provisioners
2018-12-14 19:06:46 -08:00
Megan Marsh f6070de070 fix test 2018-12-14 13:59:26 -08:00
Megan Marsh 52176ecf2d try to remove race condition in mux test 2018-12-14 13:53:39 -08:00
Adrien Delorme cecf352aa7 remove any reference to TMPDIR_SUFFIX. Using system tempdir is enough 2018-12-12 18:52:25 +01:00
Adrien Delorme b7222d1f56 Add tmp package that offers Dir & File funcs
this regroups the calls of tmp func calls into one package.
the tmp pkg allows to store tmp files under a common directory for easier managment

Squashed commit of the following:
commit 74b674c01560c6b547e9cab5e8afb486f0fe7d6c
Merge: fc94b5714 66001525d
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 02:02:38 2018 -0400

    Merge branch 'remove_TMPDIR_docs' of github.com:tb3088/packer into remove_TMPDIR_docs

commit fc94b571428447aaf2e5171967c95cdf9e5c73ff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit 5a562359f4937af506607dd96ed1c46426093f52
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 9d5a4a0935bfd049e2f8f50ad73e4455b42c9410
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit fbae4c2e051898aecd8705e9d8a20353430ede23
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 3590fae8bd0afa92a161a49ed794ee32e357d583
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit d5c5306a97b648ded06a8eb76ce728f07c9924f0
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 0a72297da86089fbd07e4f7d9472a92fa4d1c733
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 20f68228b6c372d984ea5b055cfc8bda9b110ac5
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit c73ebe3d8321664a2685a0baa8e441dd895b0db4
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit 63549b8bd745a0090b15ed0b0ebf644162db27db
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit 976101074942db36f10d3a686d6d29ddb7c01926
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit 95159afbc05eac96ed11c3183af5fb0abe2f2d5c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 66001525d72de56a4cf9339b900c46f59bc6e91a
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Fri Nov 2 01:42:38 2018 -0400

    update tests to honor configured temporary directory

commit e9b6adefeae8c65eb8aa47fef38cbf0aa424338c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 18:52:38 2018 -0400

    sync straggler to refactor. implement tests at packer/configfile instead of buried in docker.

commit 852113ed076e2d14e5dca6815ea680da1e2896bb
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 15:39:46 2018 -0400

    insert URL to GoLang API for os.TempDir()

commit 54add38d1d1c6e283cd444b367ed8bd49a5f3699
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:41:29 2018 -0400

    revise documentation to match code

commit 6b5b8f6d4edad0e187ca13d64ac4118f34eee643
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Thu Nov 1 07:21:59 2018 -0400

    refacter config_file. replace all hard-coded os.TempDir with wrapper

commit c22092c601c33484327674f322c7379fa41506d7
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 20:11:36 2018 -0400

    close massive file overwrite hole with TempDir

commit 7a730450916bf8e5dbc2a741ec233a49466ab7cc
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 01:06:00 2018 -0400

    adjust var declaration scope

commit 0f2933adb6f6922dfeab78a95371a444ec8918ab
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Wed Oct 31 00:54:35 2018 -0400

    use mktemp() equivalent to create temporary directory

commit d74839ede05dacf712b55a7bb48aec19fe6b007f
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 20:40:58 2018 -0400

    remove extraneous variable declaration, fix FOR loop

commit eb65416619437e4a3dec90277770803dd5b2281c
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:30:44 2018 -0400

    match styistic convension with rest of docs

commit acaa2b31ed463219c4ef099f351eec72406e2989
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 19:08:39 2018 -0400

    revert dangling config file change

commit e573fde668dcb418561e61535c1d68b2888f5b0f
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri Dec 29 23:53:43 2017 -0500

    replace invalid TMPDIR variable wth PACKER_TMP_DIR.
    update ConfigTmpDir() to try common temporary paths first and
    only write to configDir() as a last resort.

commit 39a9874afc9dea71b36753375fb7feb61ffecf69
Merge: 8a413cfe8 3315812c2
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Oct 30 18:19:26 2018 -0400

    Merge branch 'master' of https://github.com/hashicorp/packer into prestine

commit 8a413cfe83b0a70dbd99c32d936334ca5788ca9b
Merge: e07491de5 4e14710a6
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Oct 1 20:18:10 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit e07491de59cead0b337a7f57f4a6c625e1f560ab
Merge: 42610a35d a1fa35dff
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Tue Aug 21 13:26:19 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 42610a35d5ef65af8844a58ad70e2ec75262da6a
Merge: 5298142da 0d63cf7bc
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue Jun 19 22:45:05 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 5298142da6da95354844f201deeeef3c599e48b2
Merge: 7bb110bc7 9d9736552
Author: Matthew Patton <mpatton@enquizit.com>
Date:   Mon Jun 11 15:10:09 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 7bb110bc744c9797fcaec25a265d9b85a22d6d6b
Merge: a61c869ca 6189d66e7
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Wed May 9 23:41:22 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit a61c869ca7268acf0e3b1e1fa7a8ee7feea65984
Merge: 098101dd9 4be5f072c
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Fri May 4 10:13:04 2018 -0400

    Merge remote-tracking branch 'upstream/master' into prestine

commit 098101dd99f08b0ca110d33eff6904537c6d21de
Merge: fefaf0fa6 554b2b4a5
Author: Matthew Patton <matthew.patton@itgfirm.com>
Date:   Tue May 1 20:52:30 2018 -0400

    Merge branch 'EOL-handling' into prestine

Co-Authored-By: Matthew Patton <pattonme@yahoo.com>
2018-12-12 16:35:57 +01:00
Matt Dainty 19bd28cd72 Fix/add tests 2018-12-07 16:23:03 +00:00
Adrien Delorme d876978337
Fixed misspelling in a comment residing in packer/config_file.go as suggested by @azr.
Co-Authored-By: arizvisa <arizvisa@users.noreply.github.com>
2018-12-05 17:34:09 -06:00
Ali Rizvi-Santiago ecf45d4740 Added tests for packer.ExpandUser in packer/config_file_test.go. 2018-12-04 16:54:49 -06:00
Ali Rizvi-Santiago 040a33567e Trying again one more time to prevent import cycles.. Moved common.ExpandUser into packer.ExpandUser.. 2018-12-04 16:54:47 -06:00
Adrien Delorme 1055c119bc make code of homeDir, configFile & configDir not os specific anymore 2018-11-30 15:47:43 +01:00
Adrien Delorme 5147ac0364 use Getenv instead of LookupEnv so users can set USER="" and HOME="" + lookup different username only if it is different thant the current username 2018-11-30 14:56:53 +01:00
Ali Rizvi-Santiago eec68e319e Replaced unsafe method of determining a user's home directory with an implementation based on os/user. 2018-11-24 00:56:53 -06:00
Rémi Jouannet 118a9cb8c6 more explicit message 2018-11-09 14:44:34 +01:00
Matthew Hooker 098764d8ac
WIP 2018-10-26 16:59:20 -07:00
Adrien Delorme c393ccb7ca fix progress bar termination typo 2018-10-26 11:29:15 +02:00
Adrien Delorme 25775cd266 fix more race conditions 2018-10-15 16:31:59 +02:00
Adrien Delorme 27f4b9e4a1 pb/testing: triger more race condition by having the progress bar refresh more often 2018-10-15 15:28:46 +02:00
Adrien Delorme 4cae413a29 remove unedded atomic call ( protected by a lock ) 2018-10-15 15:28:08 +02:00
Adrien Delorme 56ccba86c7 fix some race conditions 2018-10-15 13:53:59 +02:00
Adrien Delorme cf2c309c5e add race conditions triggers for progress bar 2018-10-15 13:53:51 +02:00
Adrien Delorme 611610c89d BasicProgresssBar: protect add with a lock
to avoid weird race conditions.
2018-10-02 16:57:16 +02:00
Ashley Lowde 2884f6fab6 improve formatting and documentation for PR#6784 2018-10-02 20:00:45 +09:30
Ashley Lowde 12496e3702 add optional timestamps to build log 2018-09-29 22:39:24 +09:30
Megan Marsh 3f8407d95f Revert "Merge pull request #6700 from hashicorp/multi_progressbar"
This reverts commit 5470c08345, reversing
changes made to c427486bca.
2018-09-20 13:14:17 -07:00
Adrien Delorme be27775a3c start the pool only once 2018-09-12 17:04:10 -07:00
Adrien Delorme dfeca9f0df split progressbars per object 2018-09-12 16:21:58 -07:00
Megan Marsh 77980a7b53 fix progress bar mutex 2018-09-12 10:53:15 -07:00
Megan Marsh a0edaf6c46 Going to revert this change for now, becuase of potential issues that arise from calling Prepare() twice
Revert "use statebag instead of SetSharedState for winRM password"

This reverts commit b35acbd879.
2018-09-10 16:48:42 -07:00
Adrien Delorme 0ac7b64364 fix panic of the future using a mutex instead of some atomic calls + more docs 2018-09-06 18:12:15 +02:00
Adrien Delorme 8a851efcc8 progressbar: more godocs 2018-09-06 17:01:14 +02:00
Adrien Delorme bb59a70e8f progressbar: use int64 instead of uint64
* it's what's used for file sizes and used lib
2018-09-06 16:49:15 +02:00
Adrien Delorme f3c923c47d add tests for progress bar rpc calls 2018-09-06 16:15:09 +02:00
Adrien Delorme 6d3e36e6ea simplify remote progress bar as we are using a single instance 2018-09-06 16:14:59 +02:00
Adrien Delorme d89e1133c3 use freshly merged random.AlphaNum instead of our own random 2018-09-06 15:55:19 +02:00
Adrien Delorme 42561cf777 packer/rpc/ui_test.go: test progress bar too 2018-09-06 15:53:59 +02:00
Adrien Delorme 7efe3cac3d todos 2018-09-06 15:53:59 +02:00
Adrien Delorme 408578507f also prefix bar with number of items processing 2018-09-06 15:53:59 +02:00
Adrien Delorme 541c68aed5 add StackableProgressBar struct that will refresh/show dl status for multiple files
* simplified the downloader interface, and removed the total/current values from them
* downloaders use a proxy reader that will add all read bytes to progress
* removed unused const mtu
* DownloadClient doesn't need a downloader, so I removed it too
2018-09-06 15:53:59 +02:00
Adrien Delorme 9b07d7670e use no ops for ansible ui & MachineReadableUi 2018-09-06 15:53:59 +02:00
Adrien Delorme fd7cb47adc use proxy reader for download progress & stop storing total/current in downloaders 2018-09-06 15:53:59 +02:00
Adrien Delorme ddd96c513b first draft at self refreshing loading bar centralized/controlled by Ui 2018-09-06 15:53:59 +02:00
Megan Marsh 8fbce8c40c
Merge pull request #6357 from ladar/master
Added a 'split' function to parse template variables.
2018-08-24 15:18:36 -07:00
Megan Marsh b35acbd879 use statebag instead of SetSharedState for winRM password 2018-08-24 13:51:16 -07:00
Megan Marsh 8a6441a7a5 add tests for SensitiveVars 2018-08-21 10:28:22 -07:00
Megan Marsh ff6a039d5b replace scrubconfig with packer.LogSecretFilter.Set
filter winrm password from logs
Add new root-level packer template option, sensitive-variables, to tell us what user variables to mark sensitive.
2018-08-20 15:35:55 -07:00
Matthew Hooker 31fcfe4bc2 PoC for filtering logs 2018-08-20 15:20:21 -07:00
Patrick Double d796edc783 Add to vagrant post-processor support for Azure 2018-08-08 10:04:28 -05:00
Ladar Levison 8fee49ab9f Added a 'split' function to parse template variables. 2018-06-08 00:41:57 -05:00
Megan Marsh 91aa5f8bbb resolve conflicts; fix to work with hookedprovisioner which has been added since PR was made 2018-06-07 15:53:18 -07:00
Marc Carmier a5f802ec01 Add unit tests for DebuggedProvisionner 2018-06-07 14:09:26 -07:00
Marc Carmier 93f5bbbf45 Add waiting for key between provisioner 2018-06-07 14:09:26 -07:00
Josh Soref c5482b3e7e spelling: warnings 2018-03-14 03:29:50 +00:00
Josh Soref c49c40ac41 spelling: targeted 2018-03-14 03:24:39 +00:00
Josh Soref 5dd9be653a spelling: should 2018-03-14 03:22:32 +00:00
Josh Soref d3dc5ba6d3 spelling: provisioner 2018-03-14 02:15:21 +00:00