Commit Graph

54 Commits

Author SHA1 Message Date
Megan Marsh 796c40f89b builder/vsphere: skip iso download if hashed file is already present in remote packer_cache 2020-10-21 11:33:43 -07:00
Sylvia Moss aa52e690ee
Update go-getter and add samba shared files tests to step_download (#8954) 2020-06-04 15:00:18 +02:00
Adrien Delorme 0fa60c68fb
Drop the iso_checksum_type & iso_checksum_url fields (#8437)
* Drop the iso_checksum_type & iso_checksum_url fields

In favor of simply using iso_checksum that will know what to do.

* fix after master merge

* Update builder_test.go

* Update builder_test.go

* Update builder_test.go

* Update builder_test.go

* Update builder_test.go

* remove checksum lowercasing tests

* Update builder_test.go

* Update builder_test.go

* better docs

* Update builder_test.go

* even better docs

* Update config.go

* Update builder_test.go

* Update step_create_vmx_test.go

* make generate

* better docs

* fix imports

* up tests

* Update _ISOConfig-required.html.md

* Update builder_test.go

* don't use sha1.Sum("none") as a caching path

* Update builder_test.go

* better docs

* Update iso_config_test.go

remove ISOChecksumType/ISOChecksumURL references

* Update step_download_test.go

* add iso_checksum_url and iso_checksum_type fixers + tests

* add concrete examples of checksum values

* add examples of checksumming from local file

* update go-getter dep

* up deps

* use new go-getter version

* up ESX5Driver.VerifyChecksum: use go-getter's checksumming

* ISOConfig.Prepare: get checksum there in case we need it as a string in ESX5Driver.VerifyChecksum

* Update iso_config.go

* get go-getter from v2 branch

* Update driver_esx5.go

add more comments

* Update driver_esx5.go

* show better error message when the checksum is invalid

* Update builder_test.go

put in a valid checksum to fix tests, checksum is md5("packer")

* Update builder_test.go

test invalid and valid checksum

* more test updating

* fix default md5 string to be a valid md5

* TestChecksumFileNameMixedCaseBug: use 'file:' prefix for file checksumming

* Update iso_config_test.go

* Update iso_config_test.go

* Update builder_test.go

* Update builder_test.go

* Update builder_test.go

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update go.mod

* Update go.mod

* Update CHANGELOG.md
2020-05-28 11:02:09 +02:00
Adrien Delorme 8e6bad209b try to use isos inplace whatever the os 2020-02-06 17:50:29 +01:00
Adrien Delorme a19214afeb Allow to use isos in place 2020-02-06 17:07:58 +01:00
Moss e020da49f3 Add iso file name when targetPath is absolute 2020-02-04 18:06:47 +01:00
Megan Marsh 90bdcf58bd update step_download to return an ActionContinue if the URls field is empty. this allows us to simplify the hyperv builder, and is still safe because all other builders and uses of step_download already validate that the iso url is not empty if that's what they need, most of them inside of the IsoConfig prepare function. 2020-01-07 10:45:24 -08:00
Megan Marsh cf0130b9ed pick up dropped error 2019-11-20 16:16:33 -08:00
Megan Marsh 9fc8ed02ae Don't cache when TargetPath is non-nil; this messes up use of the iso_target_path iso template option. 2019-11-20 16:05:24 -08:00
Stephen Fox a6830f8348 Move 'ui' variable closer to invocation. 2019-07-31 14:08:40 -04:00
Stephen Fox e362d2f3ee Use 'log.Printf()' for StepDownload debug instead of 'ui.Say()'. 2019-07-31 14:02:57 -04:00
Megan Marsh e9549d2688 add test for local fs download 2019-07-22 09:17:48 -07:00
Megan Marsh 197a283b5e actually check for filepath being present 2019-07-19 10:35:50 -07:00
Megan Marsh 64f16474f8 fix double slash conundrum 2019-07-19 10:16:40 -07:00
Megan Marsh 88d589cc88 enable smb share double backslash pathing too 2019-06-26 13:43:05 -07:00
Megan Marsh 12471a8b06 use file getter for smb shares 2019-06-26 13:21:49 -07:00
Adrien Delorme d9e5145de9 Shadow the flock pkg & add a noop filelock so that at least the solaris binary can be built
Without this the following error occurs:

$ GOOS=solaris go build .
# github.com/hashicorp/packer/vendor/github.com/gofrs/flock
vendor/github.com/gofrs/flock/flock_unix.go:28:22: undefined: syscall.LOCK_EX
vendor/github.com/gofrs/flock/flock_unix.go:39:22: undefined: syscall.LOCK_SH
vendor/github.com/gofrs/flock/flock_unix.go:56:12: undefined: syscall.Flock
vendor/github.com/gofrs/flock/flock_unix.go:66:12: undefined: syscall.Flock
vendor/github.com/gofrs/flock/flock_unix.go:96:12: undefined: syscall.Flock
vendor/github.com/gofrs/flock/flock_unix.go:96:42: undefined: syscall.LOCK_UN
vendor/github.com/gofrs/flock/flock_unix.go:118:21: undefined: syscall.LOCK_EX
vendor/github.com/gofrs/flock/flock_unix.go:130:21: undefined: syscall.LOCK_SH
vendor/github.com/gofrs/flock/flock_unix.go:149:9: undefined: syscall.Flock
vendor/github.com/gofrs/flock/flock_unix.go:149:44: undefined: syscall.LOCK_NB
vendor/github.com/gofrs/flock/flock_unix.go:149:44: too many errors
2019-04-30 16:01:23 +02:00
Adrien Delorme 65be2be38b step_download: force copy of local files only on windows 2019-04-29 09:39:21 +02:00
Adrien Delorme 200e26ea85 Step download: always copy local files
instead of symlinking to fix #7534. The longer term fix for this would be to change the go-getter so that it can leave the source file where it is & tell us where it is.

We will do this when the right time comes.
2019-04-26 10:35:18 +02:00
Adrien Delorme 5f0259fe89 step download: don't fail when checksum_type is not set 2019-04-16 18:09:54 +02:00
Adrien Delorme 4dc12701d3 step_download: display clearer error messages, also when the checksum is not set
Usually the builder validation should error when the checksum is empty, but for the virtualbox-ovf builder this is not validated. Which is something I didn't see when refactoring for go-getter incorporation/refactor.
2019-04-15 14:34:09 +02:00
Adrien Delorme 9ac8d8aed8 step download: ovf files usually point to a file in the same directory, using them in place is the only way 2019-04-12 16:51:57 +02:00
Adrien Delorme 9f82b75e57 Use the hashicorp/go-getter to download files
* removed packer.Cache and references since packer.Cache is never used except in the download step. The download step now uses the new func packer.CachePath(targetPath) for this, the behavior is the same.
* removed download code from packer that was reimplemented into the go-getter library: progress bar, http download restart, checksuming from file, skip already downloaded files, symlinking, make a download cancellable by context.
* on windows if packer is running without symlinking rights and we are getting a local file, the file will be copied instead to avoid errors.
* added unit tests for step_download that are now CI tested on windows, mac & linux.
* files are now downloaded under cache dir `sha1(filename + "?checksum=" + checksum) + file_extension`
* since the output dir is based on the source url and the checksum, when the checksum fails, the file is auto deleted.
* a download file is protected and locked by a file lock,
* updated docs
* updated go modules and vendors
2019-03-13 12:11:58 +01:00
Vadym Haidamaka f736e2aba5 Print VerifyChecksum log for the download as ui.Message output 2019-03-11 22:33:46 +02:00
Adrien Delorme a11f985e3c revert unecessary check 2018-09-06 15:53:59 +02:00
Adrien Delorme 541c68aed5 add StackableProgressBar struct that will refresh/show dl status for multiple files
* simplified the downloader interface, and removed the total/current values from them
* downloaders use a proxy reader that will add all read bytes to progress
* removed unused const mtu
* DownloadClient doesn't need a downloader, so I removed it too
2018-09-06 15:53:59 +02:00
Adrien Delorme ddd96c513b first draft at self refreshing loading bar centralized/controlled by Ui 2018-09-06 15:53:59 +02:00
Ali Rizvi-Santiago cf9bbe3ecf Incorporated the hack demonstrated by @SwampDragons in order to deal with packer.rpc.Ui not exporting any information about what it's doing to anybody. 2018-08-20 21:55:18 -05:00
Ali Rizvi-Santiago 5726927cba Moved the progress bar out of packer.Ui and unlinked it out of all the packer.Ui implementations. Split up the terminal-related functions into a separate terminal.go and calculate the progress bar width by traversing through packer.Ui to avoid the issue with github.com/ugorji/go/codec serializing private members (or unsafe pointers) of structs. Shuffled some arguments around in getConsoleScreenBufferInfo in common/terminal_windows.go so that the interface forces the user to correctly declare a _CONSOLE_SCREEN_BUFFER_INFO type. 2018-08-20 21:55:18 -05:00
Ali Rizvi-Santiago 0f10032b3d Moved the progress bar from common to the packer.UI interface and refactored it so that the terminal width is calculated based on each interface which returns a custom progressbar specific to its ui. 2018-08-20 21:55:18 -05:00
Ali Rizvi-Santiago fd2fef8738 Added support for the progress bar to automatically determine its width using the minimum length from a packer.UI and the terminal dimensions via kernel32.GetConsoleScreenBufferInfo or an ioctl (TIOCGWINSZ) to "/dev/tty". 2018-08-20 21:55:18 -05:00
Ali Rizvi-Santiago 16ecb3ad9a Reverted removal of progress-bar that was done by commit 5d97b105a8 and added some missing arguments that were missed during the rebase. Modified the default progress bar's width to 80 as a result of the conversation on PR #5851. 2018-08-20 21:55:18 -05:00
Adrien Delorme a5587e30ec log wether the file was transfered or is just being inplace referenced 2018-08-20 11:45:57 +02:00
Adrien Delorme 6b3844a64f
Revert "allow to use ISO images in-place v.s. copying them" 2018-08-20 10:48:06 +02:00
Adrien Delorme c744e8b2bb make download messages less redudant and more simple 2018-08-17 09:29:39 +02:00
Adrien Delorme d7d4aed51c be even more precise 2018-08-16 18:56:28 +02:00
Adrien Delorme 2b3ea29970 show more precise error download/copy/referencing messages 2018-08-16 18:41:44 +02:00
Adrien Delorme 863222b1e2 Also use the terminology Inplace in DownloadConfig for clarity/consistency
* swapped boolean checks
* swapped in tests too
2018-08-15 15:26:31 +02:00
Adrien Delorme 82e480a285 allow to use ISO images inplace v.s. copying them 2018-08-15 14:37:38 +02:00
Seth Vargo b17b333e29
Add a common package for specifying useragent and adopt that everywhere
There were 5 different formats for the Packer useragent string. This
fixes that and unifies it into a helper package.

I did not touch oracle's user-agent, because it looked kinda special.
2018-04-05 14:28:50 -04:00
Josh Soref 5178dd36e8 spelling: regular 2018-03-14 02:19:10 +00:00
Matthew Hooker 7a189a83a1
fix imports
`find . -type f -name '*.go' -not -path "./vendor/*" -exec goimports -w {} \;`
2018-01-24 17:09:17 -08:00
Matthew Hooker a831d522be
change run signatures
Run now takes a context as well as a statebag. We'll assign the context
to the blank identifier to prevent namespace collisions. We'll let the
step authors opt-in to using the context.

`find . -iname "step_*.go" -exec gsed -i'' 's/func \(.*\)Run(/func \1Run(_ context.Context, /' {} \;`
2018-01-24 17:09:17 -08:00
Matthew Hooker 366dc3da0a
move multistep imports to helper.
gomvpkg -from "github.com/mitchellh/multistep" -to "github.com/hashicorp/packer/helper/multistep"
2018-01-24 17:09:15 -08:00
Vladislav Rassokhin af8a0c46c5 Do not re-download iso multiple times from different urls
In case of two or more iso_urls checks for downloaded files prior to downloading them.
Speedups case when some iso already downloaded and another url prepended to iso_urls list.
2017-11-26 00:10:34 +03:00
Matthew Hooker 81522dced0
move packer to hashicorp 2017-04-04 13:39:01 -07:00
Mitchell Hashimoto e65e2d104a common: StepDownload can force an extension 2015-06-08 20:41:39 -07:00
Mitchell Hashimoto 8dbe0f065c Remove version from "packer" package 2014-10-27 20:51:34 -07:00
Mark Rushakoff 4548495116 StepDownload uses packer version as user agent 2014-01-09 13:20:30 -08:00
Mitchell Hashimoto 483cda18c1 builder/vmware: make things more Go-like
This commit currently breaks the builder though, since the ISo is now
uploaded back into ESX.
2013-11-07 12:01:18 -08:00