Commit Graph

126 Commits

Author SHA1 Message Date
Megan Marsh 949908e48b
slow down the changelog script to prevent rate limiting (#9172)
* slow down the changelog script to prevent rate limiting
2020-05-05 20:22:25 -04:00
Wilken Rivera 951cd0f55e scripts/build: Remove unsupported Freebsd/arm builds from build chain
Builds before change
```
⇶  make bin
WARN: 'make bin' is for debug / test builds only. Use 'make release' for
release builds.
==> Checking for necessary tools...
==> Entering Packer source dir...
==> Ensuring output directories are present...
==> Removing old builds...
==> Building...
Number of parallel builds: 3

-->     linux/s390x: github.com/hashicorp/packer
-->     linux/amd64: github.com/hashicorp/packer
-->       linux/arm: github.com/hashicorp/packer
-->   solaris/amd64: github.com/hashicorp/packer
-->      linux/mips: github.com/hashicorp/packer
-->     freebsd/386: github.com/hashicorp/packer
-->     linux/arm64: github.com/hashicorp/packer
-->    linux/mips64: github.com/hashicorp/packer
-->    linux/mipsle: github.com/hashicorp/packer
-->    darwin/amd64: github.com/hashicorp/packer
-->       linux/386: github.com/hashicorp/packer
-->      darwin/386: github.com/hashicorp/packer
-->     windows/386: github.com/hashicorp/packer
-->   windows/amd64: github.com/hashicorp/packer
-->   linux/ppc64le: github.com/hashicorp/packer
-->     openbsd/386: github.com/hashicorp/packer
-->     freebsd/arm: github.com/hashicorp/packer
-->   openbsd/amd64: github.com/hashicorp/packer
-->   freebsd/amd64: github.com/hashicorp/packer

1 errors occurred:
--> freebsd/arm error: exit status 2
Stderr: # github.com/shirou/gopsutil/cpu
../go/pkg/mod/github.com/shirou/gopsutil@v2.18.12+incompatible/cpu/cpu_freebsd.go:25:16:
undefined: cpuTimes
../go/pkg/mod/github.com/shirou/gopsutil@v2.18.12+incompatible/cpu/cpu_freebsd.go:42:31:
undefined: cpuTimes
../go/pkg/mod/github.com/shirou/gopsutil@v2.18.12+incompatible/cpu/cpu_freebsd.go:66:38:
undefined: cpuTimes
../go/pkg/mod/github.com/shirou/gopsutil@v2.18.12+incompatible/cpu/cpu_freebsd.go:72:15:
undefined: cpuTimes
../go/pkg/mod/github.com/shirou/gopsutil@v2.18.12+incompatible/cpu/cpu_freebsd.go:87:13:
undefined: cpuTimes

==> Copying binaries for this platform...
'./pkg/linux_amd64/packer' -> 'bin/packer'
'./pkg/linux_amd64/packer' -> '/home/wilken/Development/go/bin/packer'

==> Results:
total 111M
-rwxr-xr-x 1 wilken wilken 111M Apr 14 22:02 packer
```

Builds after change
```
⇶  make bin
WARN: 'make bin' is for debug / test builds only. Use 'make release' for
release builds.
==> Checking for necessary tools...
==> Entering Packer source dir...
==> Ensuring output directories are present...
==> Removing old builds...
==> Building...
Number of parallel builds: 3

-->   solaris/amd64: github.com/hashicorp/packer
-->   windows/amd64: github.com/hashicorp/packer
-->     linux/s390x: github.com/hashicorp/packer
-->    darwin/amd64: github.com/hashicorp/packer
-->      darwin/386: github.com/hashicorp/packer
-->     windows/386: github.com/hashicorp/packer
-->   freebsd/amd64: github.com/hashicorp/packer
-->     freebsd/386: github.com/hashicorp/packer
-->     openbsd/386: github.com/hashicorp/packer
-->   openbsd/amd64: github.com/hashicorp/packer
-->     linux/arm64: github.com/hashicorp/packer
-->       linux/386: github.com/hashicorp/packer
-->     linux/amd64: github.com/hashicorp/packer
-->       linux/arm: github.com/hashicorp/packer
-->      linux/mips: github.com/hashicorp/packer
-->   linux/ppc64le: github.com/hashicorp/packer
-->    linux/mips64: github.com/hashicorp/packer
-->    linux/mipsle: github.com/hashicorp/packer
==> Copying binaries for this platform...
'./pkg/linux_amd64/packer' -> 'bin/packer'
'./pkg/linux_amd64/packer' ->
'/home/wilken/Development/golang/bin/packer'

==> Results:
total 111M
-rwxr-xr-x 1 wilken wilken 111M Apr 15 20:52 packer
```
2020-04-16 06:05:34 -04:00
Wilken Rivera 7ec55860e5
Revert golangci-lint to use new-from-rev pinned at 1.23.8 (#9072)
Turns out linting each file individually causes issues with the linter not being able to find the import types.
2020-04-15 06:53:51 -04:00
Megan Marsh f12269f124
fix builds on linux (#9031)
* fix builds on linux

* Build: Move to CGO_ENABLED=0 (#9057)

After further investigation on cross-compiling Go bins on Linux. I found
that statically linking against GCC (for libc) failed to build for ARM
and introduced a possible licensing issue as our bins would essentially
be bundling libc into the bin. Diving further into cross compiling on Linux
I found that the defacto solution is to compile with CGO disabled - this
was also found to be the case for other HashiCorp products.

Disabling CGO has the limitation of not allowing the use of any pkg that
calls out to C (net, os), but in looking into the Packer code base and
the relevant Go code base it appears that the latest versions of Go have
pure Go implementations of the said packages so I believe we are good to
go. I should also point out that CGO is disabled by default when cross
compiling via `go build`. However, the GOX tool will enable it if it is
not explicitly disabled.

Below are three test cases executed to validate the compile bins work as
expected.

Build results after change
```
⇶  make bin
WARN: 'make bin' is for debug / test builds only. Use 'make release' for
release builds.
==> Checking for necessary tools...
==> Entering Packer source dir...
==> Ensuring output directories are present...
==> Removing old builds...
==> Building...
Number of parallel builds: 7

-->   windows/amd64: github.com/hashicorp/packer
-->     linux/arm64: github.com/hashicorp/packer
-->       linux/386: github.com/hashicorp/packer
-->       linux/arm: github.com/hashicorp/packer
-->    darwin/amd64: github.com/hashicorp/packer
-->     windows/386: github.com/hashicorp/packer
-->     linux/amd64: github.com/hashicorp/packer
-->      darwin/386: github.com/hashicorp/packer
==> Copying binaries for this platform...
'./pkg/linux_amd64/packer' -> 'bin/packer'
'./pkg/linux_amd64/packer' -> '/home/wilken/Development/go/bin/packer'

==> Results:
total 111M
-rwxr-xr-x 1 wilken wilken 111M Apr 13 12:29 packer
```

Packer executed on ARM based machine
```
ubuntu@ip-172-31-10-18:~$ ./packer version
Packer v1.5.6-dev (314ac5b65+CHANGES

ubuntu@ip-172-31-10-18:~$ uname -a
Linux ip-172-31-10-18 4.15.0-1054-aws #56-Ubuntu SMP Thu Nov 7 16:18:50 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux

ubuntu@ip-172-31-10-18:~$ ./packer build build.json
null: output will be in this color.

==> null: Running local shell script: /tmp/packer-shell170248556
    null: UUID from Packer: 79cc8532-6114-925d-2a79-33ef6ce281cd
Build 'null' finished.

==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder
```

Custom Docker image with updated bin
```
⇶  docker run packertest:latest version
Packer v1.5.6-dev (314ac5b65+CHANGES)

⇶  docker run packertest:latest build build.json
null: output will be in this color.

==> null: Running local shell script: /tmp/packer-shell065599452
    null: UUID from Packer: 852f0604-2be4-9e16-99af-6d7df972ac2e
Build 'null' finished.

==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder
```

Windows AMI
```
[...]
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
    amazon-ebs: Adding tag: "Name": "Packer Builder"
    amazon-ebs: Instance ID: i-04387545cf3e2acd3
==> amazon-ebs: Waiting for instance (i-04387545cf3e2acd3) to become ready...
==> amazon-ebs: Skipping waiting for password since WinRM password set...
==> amazon-ebs: Using winrm communicator to connect: 18.206.100.104
==> amazon-ebs: Waiting for WinRM to become available...
    amazon-ebs: WinRM connected.
==> amazon-ebs: Connected to WinRM!
==> amazon-ebs: Uploading packertest => c:/Windows/Temp
==> amazon-ebs: Provisioning with Powershell...
==> amazon-ebs: Provisioning with powershell script: /tmp/powershell-provisioner173180945
    amazon-ebs: Packer v1.5.6-dev (314ac5b65+CHANGES)
    amazon-ebs: null: output will be in this color.
    amazon-ebs:
```

Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>
2020-04-14 14:48:50 -04:00
Wilken Rivera c9c0ee65d3
circle-ci: update ci-lint steps (#9043)
* new-from-rev option is showing inconsistent results on circle and
locally. This change moves to a custom command `script/lint.sh` that gets a list of added
go files and pipes them to golangci-lint for testing.

* Add a git fetch as a step before retrieving merge-base changes to fix
the issue described at https://discuss.circleci.com/t/checkout-script-adds-commits-to-master-from-circle-branch/14194/2

* Moved source code out of GOPATH to ensure go mod support and reduce
the risk of having golangci-lint trying to scan all of the files within
GOPATH. This was an issue in the past, in changing it I found less OOM
issues on circle.
2020-04-14 12:03:22 +02:00
Wilken Rivera ff2289a64e scripts/prepare_changelog: Update unescape parenthesis for improved matching
Before change
```
413e19b84 Merge pull request #8942 from desolatorxxl/google-fix-ssh-keys-metadata
b81800db2 Merge pull request #8935 from zaventh/feature/start-on-boot
94863168b Merge pull request #8922 from hashicorp/f-vsphere_iso-export-ovf-options
56aebbeda Merge pull request #8920 from rhencke/patch-1
d068430ab make sure locals are evaluated only once variables are + test this (#8918)
3dae5df6e Merge pull request #8905 from hashicorp/fix_8493
811a7304a Merge pull request #8907 from hashicorp/fix_8428
fa49d2145 Merge pull request #8906 from hashicorp/fix_8904
23f56036a Merge pull request #8889 from hashicorp/hcl2_singular_blocks
dc9259f73 Merge pull request #8892 from zaventh/feature/vga-adapter
fc35f0200 Merge pull request #8890 from hashicorp/fix_8880
7972ab723 Merge pull request #8735 from hashicorp/fix_plugin_loading
890d7b2ec Merge pull request #8875 from hashicorp/fix_8812
e94ff7019 Merge pull request #8883 from hashicorp/fix_8835
9075b807d Merge pull request #8891 from rhencke/patch-1
6477d8a0c Merge pull request #8882 from hashicorp/fix-var-file-hcl
6008f911f Merge pull request #8847 from takaishi/support-keyboard-interactive
56045619d Merge pull request #8877 from paulcichonski/remote-esxi-bastion
698f74478 Merge pull request #8887 from hashicorp/untangle_ssh_docs_from_aws
aeedc9af7 Merge pull request #8879 from mbrancato/specify_keyvault_sku
5365fda5f Merge pull request #8884 from hashicorp/fix_codecov_config
4bd7b1409 Merge pull request #8732 from jhawk28/reorder_cdrom_drive
072a71b41 Merge pull request #8863 from hashicorp/update_go-cty_regex
8a1caaa80 Merge pull request #8837 from hashicorp/fix_8730
7873cabf6 Merge pull request #8858 from hashicorp/fix_8791
7e382d0df Merge pull request #8828 from mvitaly/fix_8816
8832b3e2c Merge pull request #8787 from jhawk28/vsphere_iso_multiple_disks
528174027 Merge pull request #8831 from rjhornsby/master
e35a87241 Merge pull request #8830 from hashicorp/d-var-file-hcl2-not-yet
```

After change
```
⇶  git log v1.5.4...v1.5.5 --first-parent --oneline --grep="Merge pull request #[0-9]\+" --grep="(#[0-9]\+)$"
413e19b84 Merge pull request #8942 from desolatorxxl/google-fix-ssh-keys-metadata
c387dc2c5 builder/vsphere-clone: Find the vm within the folder (#8938)
b17b211aa Add cleanup_remote_cache config option to vmware-iso (#8917)
e6368b924 Fix azure winrm_password attribution and allow to set winrm_username (#8928)
fcf10e9b7 Replace Amazon with Outscale for OSC BSU doc (#8944)
9240fb7f0 Fix typo in title (#8943)
2c6f0968b Allow accepting image for the members in OpenStack builder (#8931)
b81800db2 Merge pull request #8935 from zaventh/feature/start-on-boot
daffd9c31 CONTRIBUTING: Update documentation for linting on Travis (#8933)
3a9d356c9 golangci-lint: Update --new-from-rev option to check only newly added commits (#8923)
97d797d2f Fix small typos in osc-bsuvolume.html.md (#8926)
94863168b Merge pull request #8922 from hashicorp/f-vsphere_iso-export-ovf-options
56aebbeda Merge pull request #8920 from rhencke/patch-1
99b0b9831 Add ovf export capability to vsphere builders (#8764)
d068430ab make sure locals are evaluated only once variables are + test this (#8918)
ad8dafa3b HCL: add tests and fixes around var-file and var args  (#8914)
7979ab054 Add after_n_builds to codecov.yml (#8913)
3dae5df6e Merge pull request #8905 from hashicorp/fix_8493
811a7304a Merge pull request #8907 from hashicorp/fix_8428
fa49d2145 Merge pull request #8906 from hashicorp/fix_8904
b94937c05 Update provisioner_test.go (#8900)
2319521aa Add iso config test for checksum from file specific case (#8897)
23f56036a Merge pull request #8889 from hashicorp/hcl2_singular_blocks
dc9259f73 Merge pull request #8892 from zaventh/feature/vga-adapter
690bf714c Add Codecov badge and remove report style (#8896)
fc35f0200 Merge pull request #8890 from hashicorp/fix_8880
7972ab723 Merge pull request #8735 from hashicorp/fix_plugin_loading
890d7b2ec Merge pull request #8875 from hashicorp/fix_8812
e94ff7019 Merge pull request #8883 from hashicorp/fix_8835
```
2020-03-27 07:19:49 -04:00
Wilken Rivera 4b90144537 scripts/prepare_changelog: Update git log to only display PR merged commits
This change uses git flags to shorten the log messages to titles only, and uses the grep pattern on git to filter only commits that match merged commits, including squashed and merged commits.
2020-03-26 21:56:43 -04:00
Adrien Delorme b15c92bc1e scripts/generate-plugins.go: programatically gofmt the file before outputing it to avoid having to do that later on 2020-03-17 11:56:32 +01:00
Megan Marsh 9a85fdd0a5 Cut version 1.5.4 2020-02-14 16:14:01 -08:00
Megan Marsh ac239d1188 fix checking for codesign 2020-02-14 14:50:21 -08:00
Wilken Rivera 9c171c1f13
scripts: Update code signing scripts (#8746)
* scripts/codesign_example: Fix reference to SHASUM_PROG variable

* scripts/sign: Add check for required Artifactory token
2020-02-14 17:38:56 -05:00
Wilken Rivera 7254b04129
script/prepare_changelog: Update to show squashed merge commits (#8744)
* script/prepare_changelog: Update regex to include squashed PRs

* scripts/prepare_changelog: Update to show all commits not just merged commits
2020-02-14 11:33:33 -05:00
Wilken Rivera 2bdca997ac Update function name 2020-02-14 09:44:45 -05:00
Wilken Rivera 19e7114301 scripts/prepare_changelog: Update jq filter to ignore tech-debt labelled pull-requests 2020-02-14 09:44:45 -05:00
Wilken Rivera c121aa9104 scripts/codesign_example: Add check for sha256sum program 2020-02-12 16:54:04 -05:00
nywilken 8a36ddd69d scripts/vagrant: Update Vagrant bootstrapping scripts 2020-01-15 12:42:15 -05:00
Megan Marsh 7ce9992357 move osx code signing to before checksums are calculated 2019-12-19 14:07:10 -08:00
Megan Marsh e03342cd32
Cut version 1.5.0 2019-12-18 15:54:41 -08:00
Megan Marsh e3c0022a8e update release process to incorporate binary signing 2019-12-18 15:44:49 -08:00
Megan Marsh ee5e0315d2
Cut version pipefail 2019-12-18 15:17:00 -08:00
Megan Marsh b70ab5c8d4
Cut version 1.5.0 2019-12-18 14:55:07 -08:00
nywilken 89ad8ae4fa prepare_changelog: Update regex to work with GNU grep 2019-12-09 15:05:39 -05:00
Lars Lehtonen 3c170522f3
scripts: close file after error handling 2019-11-12 09:52:40 -08:00
Megan Marsh b82ab2cc6a change installation of pigeon if off gopath 2019-07-01 12:16:50 -07:00
bzhaoopenstack f7793649ec
Fix failed to copy binary when using make bin
If we use make bin, XC_OS and XC_ARCH will be None, the binary will failed to be copied.
2019-06-15 00:34:33 +08:00
Megan Marsh 3e73abc93a refresh line endings 2019-05-17 14:58:20 -07:00
Megan Marsh 5e23083df1
Merge pull request #7605 from hashicorp/verbose-packer-binary-copy
scripts/build.sh: don't discard stderr when copying packer
2019-05-07 10:23:20 -07:00
Adrien Delorme 5f076d4328 allow building packer on solaris by removing progress bar and tty imports
fix #7586
2019-05-07 15:58:49 +02:00
Adrien Delorme e8ac0e6c75
scripts/build.sh: don't discard stderr when copying packer
fix #7604
2019-05-06 11:06:38 +02:00
Guillaume J. Charmes 36641e66a7 Update build.sh. Remove solaris, add mipsXX and s390x
Signed-off-by: Guillaume J. Charmes <gcharmes@magicleap.com>
2019-04-25 16:29:25 -04:00
Adrien Delorme 413e242a14 Revert "scripts/build.sh: allow to set build settings when building"
This reverts commit f6be550f1a.
2019-04-15 16:35:32 +02:00
Adrien Delorme f6be550f1a scripts/build.sh: allow to set build settings when building
this will for example allow me to have the following alias:
alias buildmain='export T=$(mktemp -d) && ALL_XC_OS="linux darwin windows" ALL_XC_ARCH="amd64" GOLDFLAGS="-s -w" ./scripts/build.sh && cd pkg/ && for dir in *; do zip -r "$dir.zip" $dir & ; done ;  wait && mv *.zip $T/. && open $T'

that build only on 'main' platforms, so that I can share binaries easily.
2019-02-19 15:10:30 +01:00
Daniel Poggenpohl 6857e937e2 - FIX: That's what I get for wanting to use fancy or/and piping, back to basics (Now using explicit rc test to decide whether to quit) 2019-02-01 23:11:22 +01:00
Daniel Poggenpohl b0abd1fbf3 - FIX: I should call my defined functions, else they are useless 2019-02-01 20:48:42 +01:00
Daniel Poggenpohl ce760eec09 - More comments
- encapsulated code in functions to make it more readable
- validate presence of used tools/binaries (check if they're in PATH)
- more output
- FIX: make all uname/OSTYPE output to lowercase and check only for that
- refactored method name convert_path -> convertPathOnCygwin
- gave convert_path two arguments to make it more readable (Readability over tight code? I don't know what is required)
- some variable expansion now uses braces
2019-01-31 15:50:10 +01:00
Mikhail Ushanov 4be83e20b1 scripts: add gcc package for using gco on build
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-31 16:58:07 +03:00
Mikhail Ushanov 15235ce315 scripts: update go version to 1.11.1
Signed-off-by: Mikhail Ushanov <gm.mephisto@gmail.com>
2018-10-31 16:58:07 +03:00
Matthew Hooker c163fbed35
use xargs to check for formatting 2018-05-01 20:39:48 -07:00
Matthew Hooker 2fc67f2210
Revert "Merge pull request #4494 from tb3088/EOL-handling"
This reverts commit dfe4f56c75, reversing
changes made to e90c87f885.

Revert changes to makefile and format checker
2018-05-01 13:09:55 -07:00
Matthew Patton 554b2b4a5d ignore errors during Find 2018-04-09 19:47:41 -04:00
Matthew Patton d5bf9277ce remove rebase duplicate 2018-04-07 05:33:46 -04:00
Matthew Patton 6a85f5aed7 handle missing GOPATH and cygwin considerations 2018-04-07 05:29:43 -04:00
Matthew Patton 1d0cf3d909 handle missing GOPATH and cygwin considerations 2018-04-07 05:24:31 -04:00
Matthew Patton 049e1bbf73 too many files for shell during Make, convert .go and .sh to EOL=lf 2018-04-07 05:22:39 -04:00
Christopher Boumenot 676fb59090 Better override support for PS build script
Developers can now independently controls XC_OS and XC_ARCH.
2018-03-05 01:58:30 -08:00
James Nugent c132bd867e vagrant: Correct name of vim package
The non-X11-linked version of vim recently changed name from vim-lite to
vim-console, which was preventing bootstrap.
2018-02-02 13:14:13 -06:00
James Nugent 4fc0a1ea0a build: Allow multi-platform dev with Vagrantfile
This commit rewrites the Vagrantfile for Packer in a similar manner to
the work done for Nomad (hashicorp/nomad#3175) in order to make
cross-platform development easier. It also adds support for a FreeBSD
base box.

Provisioning scripts are separated out in order that they can be
correctly linted. Each script is prefixed `vagrant`, then the operating
system, then whether or not it expects to be run in a privileged shell.

Finally, dependencies have been bumped - Go 1.6 is switched out for the
latest (1.9.2).
2017-11-02 11:00:19 -07:00
Matthew Hooker 1e162ffd26
skip doc PRs in changelog helper 2017-10-10 17:11:28 -07:00
Matthew Hooker 339d768420
add ppc64le as a build target 2017-09-25 15:38:32 -07:00
Matthew Hooker b7ddf80155
build solaris binary 2017-08-21 12:32:16 -07:00