iSharkFly-Docs
/
packer-cn
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix documentation for acceptance tests

This commit is contained in:
Megan Marsh 2020-12-07 15:30:59 -08:00
parent 6ecdd3ca16
commit 00c61f9987
1 changed files with 223 additions and 146 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

369
.github/CONTRIBUTING.md vendored
View File

@ -301,15 +301,15 @@ You can run tests for individual packages using commands like this:
make test TEST=./builder/amazon/... make test TEST=./builder/amazon/...
``` ```
#### Running Acceptance Tests #### Running Builder Acceptance Tests
Packer has [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing) Packer has [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing)
for various builders. These typically require an API key (AWS, GCE), or for various builders. These typically require an API key (AWS, GCE), or
additional software to be installed on your computer (VirtualBox, VMware). additional software to be installed on your computer (VirtualBox, VMware).
If you're working on a new builder or builder feature and want to verify it is If you're working on a new builder or builder feature and want to verify it is
functioning (and also hasn't broken anything else), we recommend running the functioning (and also hasn't broken anything else), we recommend creating or
acceptance tests. running the acceptance tests.
**Warning:** The acceptance tests create/destroy/modify _real resources_, which **Warning:** The acceptance tests create/destroy/modify _real resources_, which
may incur costs for real money. In the presence of a bug, it is possible that may incur costs for real money. In the presence of a bug, it is possible that
@ -340,7 +340,7 @@ Acceptance tests typically require other environment variables to be set for
things such as API tokens and keys. Each test should error and tell you which things such as API tokens and keys. Each test should error and tell you which
credentials are missing, so those are not documented here. credentials are missing, so those are not documented here.
#### Running Provisioners Acceptance Tests #### Running Provisioner Acceptance Tests
**Warning:** The acceptance tests create/destroy/modify _real resources_, which **Warning:** The acceptance tests create/destroy/modify _real resources_, which
may incur costs for real money. In the presence of a bug, it is possible that may incur costs for real money. In the presence of a bug, it is possible that
@ -351,124 +351,164 @@ resources are not accidentally destroyed or overwritten during testing.
Also, these typically require an API key (AWS, GCE), or additional software Also, these typically require an API key (AWS, GCE), or additional software
to be installed on your computer (VirtualBox, VMware). to be installed on your computer (VirtualBox, VMware).
To run the Provisioners Acceptance Tests you should use both **ACC_TEST_BUILDERS** and **ACC_TEST_PROVISIONERS** variables to To run the Provisioners Acceptance Tests you should use the
tell which provisioner and builder the test should be run against. **ACC_TEST_BUILDERS** environment variable to tell the tests which builder the
test should be run against.
Examples of usage: Examples of usage:
- Run the Shell provisioner acceptance tests against the Amazon EBS builder. - Run the Shell provisioner acceptance tests against the Amazon EBS builder.
``` ```
ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell go test ./provisioner/shell/... -v -timeout=1h ACC_TEST_BUILDERS=amazon-ebs go test ./provisioner/shell/... -v -timeout=1h
``` ```
- Do the same but using the Makefile - Do the same but using the Makefile
``` ```
ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell make provisioners-acctest ACC_TEST_BUILDERS=amazon-ebs make provisioners-acctest TEST=./provisioner/shell
``` ```
- Run the all Shell and Powershell provisioners acceptance tests against the Amazon EBS builder. - Run all provisioner acceptance tests against the Amazon EBS builder.
``` ```
ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell,powershell make provisioners-acctest ACC_TEST_BUILDERS=amazon-ebs make provisioners-acctest TEST=./...
``` ```
- Run the all provisioners acceptance tests against the Amazon EBS builder. - Run all provisioner acceptance tests against all builders whenever they are compatible.
``` ```
ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=all make provisioners-acctest ACC_TEST_BUILDERS=all make provisioners-acctest TEST=./...
```
- Run the all provisioners acceptance tests against all builders whenever they are compatible.
```
ACC_TEST_BUILDERS=all ACC_TEST_PROVISIONERS=all make provisioners-acctest
``` ```
Both **ACC_TEST_BUILDERS** and **ACC_TEST_PROVISIONERS** allows defining a list of builders and provisioners separated by comma The **ACC_TEST_BUILDERS** env variable accepts a list of builders separated by
(e.g. `ACC_TEST_BUILDERS=amazon-ebs,virtualbox-iso`) commas. (e.g. `ACC_TEST_BUILDERS=amazon-ebs,virtualbox-iso`)
#### Writing Provisioner Acceptance Tests #### Writing Provisioner Acceptance Tests
Packer has an already implemented structure that will run the provisioner against builders and you can find it in `helper/tests/acc/provisioners.go`. Packer has implemented a `ProvisionerTestCase` structure to help write
All provisioners should use this structure in their acceptance tests. provisioner acceptance tests.
To start writing a new provisioner acceptance test, you should add a test file named as `provisioner_acc_test.go` in the provisioner folder
and the package should be `<provisioner>_test`. This file should have a struct that will implement the ProvisionerAcceptance interface.
```go ```go
type ProvisionerAcceptance interface { type ProvisionerTestCase struct {
GetName() string // Check is called after this step is executed in order to test that
GetConfig() (string, error) // the step executed successfully. If this is not set, then the next
GetProvisionerStore() packer.MapOfProvisioner // step will be called
IsCompatible(builder string, vmOS string) bool Check func(*exec.Cmd, string) error
RunTest(c *command.BuildCommand, args []string) error // IsCompatible checks whether a provisioner is able to run against a
// given builder type and guest operating system, and returns a boolean.
// if it returns true, the test combination is okay to run. If false, the
// test combination is not okay to run.
IsCompatible func(builderType string, BuilderGuestOS string) bool
// Name is the name of the test case. Be simple but unique and descriptive.
Name string
// Setup, if non-nil, will be called once before the test case
// runs. This can be used for some setup like setting environment
// variables, or for validation prior to the
// test running. For example, you can use this to make sure certain
// binaries are installed, or text fixtures are in place.
Setup func() error
// Teardown will be called before the test case is over regardless
// of if the test succeeded or failed. This should return an error
// in the case that the test can't guarantee all resources were
// properly cleaned up.
Teardown builderT.TestTeardownFunc
// Template is the provisioner template to use.
// The provisioner template fragment must be a json-formatted string
// containing the provisioner definition but no other portions of a packer
// template. For
// example:
//
// ```json
// {
// "type": "shell-local",
// "inline", ["echo hello world"]
// }
//```
//
// is a valid entry for "template" here, but the complete Packer template:
//
// ```json
// {
// "provisioners": [
// {
// "type": "shell-local",
// "inline", ["echo hello world"]
// }
// ]
// }
// ```
//
// is invalid as input.
//
// You may provide multiple provisioners in the same template. For example:
// ```json
// {
// "type": "shell-local",
// "inline", ["echo hello world"]
// },
// {
// "type": "shell-local",
// "inline", ["echo hello world 2"]
// }
// ```
Template string
// Type is the type of provisioner.
Type string
} }
``` ```
- **GetName()** should return the provisioner type. For example for the Shell provisioner the method returns "shell". To start writing a new provisioner acceptance test, you should add a test file
named `provisioner_acc_test.go` in the same folder as your provisioner is
defined. Create a test case by implementing the above struct, and run it
by calling `provisioneracc.TestProvisionersAgainstBuilders(testCase, t)`
- **GetConfig()** should read a text file with the json configuration block for the provisioner and any other necessary provisioner. The following example has been adapted from a shell-local provisioner test:
For the Shell one the file contains:
``` ```
{ import (
"type": "shell", "github.com/hashicorp/packer/packer-plugin-sdk/acctest/provisioneracc"
"inline": [ "github.com/hashicorp/packer/packer-plugin-sdk/acctest/testutils"
"echo {{ build `ID`}} > provisioner.{{ build `PackerRunUUID`}}.txt" )
]
// ...
func TestAccShellProvisioner_basic(t *testing.T) {
// Create a json template fragment containing just the provisioners you want
// to run.
templateString := `{
"type": "shell-local",
"script": "test-fixtures/script.sh",
"max_retries" : 5
}`
// instantiate a test case.
testCase := &provisioneracc.ProvisionerTestCase{
IsCompatible: func() bool {return true},
Name: "shell-local-provisioner-basic",
Teardown: func() error {
testutils.CleanupFiles("test-fixtures/file.txt")
return nil
}, },
{ Template: templateString,
"type": "file", Type: "shell-local",
"source": "provisioner.{{ build `PackerRunUUID`}}.txt", Check: func(buildcommand *exec.Cmd, logfile string) error {
"destination": "provisioner.shell.{{ build `PackerRunUUID`}}.txt", if buildcommand.ProcessState != nil {
"direction": "download" if buildcommand.ProcessState.ExitCode() != 0 {
} return fmt.Errorf("Bad exit code. Logfile: %s", logfile)
``` }
The file should be placed under the `test-fixtures` folder.
In this case, it's necessary to use the File provisioner to validate if the Shell provisioner test is successful or not.
This config should be returned as string that will be later merged with the builder config into a full template.
- **GetProvisionerStore()** this returns the provisioner store where we declare the available provisioners for running the build.
For the Shell provisioners this is:
```go
func (s *ShellProvisionerAccTest) GetProvisionerStore() packer.MapOfProvisioner {
return packer.MapOfProvisioner{
"shell": func() (packer.Provisioner, error) { return &shell.Provisioner{}, nil },
"file": func() (packer.Provisioner, error) { return &file.Provisioner{}, nil },
}
} }
``` filecontents, err := loadFile("file.txt")
if err != nil {
- **IsCompatible(builder string, vmOS string)** returns true or false whether the provisioner should run against a return err
specific builder or/and specific OS.
- **RunTest(c \*command.BuildCommand, args []string)** it will actually run the build and return any error if it fails the validations.
For the Shell provisioner this is:
```go
func (s *ShellProvisionerAccTest) RunTest(c *command.BuildCommand, args []string) error {
// Provisioner specific setup
UUID := os.Getenv("PACKER_RUN_UUID")
if UUID == "" {
UUID, _ = uuid.GenerateUUID()
os.Setenv("PACKER_RUN_UUID", UUID)
}
file := "provisioner.shell." + UUID + ".txt"
defer testutils.CleanupFiles(file)
// Run build
// All provisioner acc tests should contain this code and validation
if code := c.Run(args); code != 0 {
ui := c.Meta.Ui.(*packer.BasicUi)
out := ui.Writer.(*bytes.Buffer)
err := ui.ErrorWriter.(*bytes.Buffer)
return fmt.Errorf(
"Bad exit code.\n\nStdout:\n\n%s\n\nStderr:\n\n%s",
out.String(),
err.String())
}
// Any other extra specific validation
if !testutils.FileExists(file) {
return fmt.Errorf("Expected to find %s", file)
}
return nil
} }
if !strings.Contains(filecontents, "hello") {
return fmt.Errorf("file contents were wrong: %s", filecontents)
}
return nil
},
}
provisioneracc.TestProvisionersAgainstBuilders(testCase, t)
}
```
```
After writing the struct and implementing the interface, now is time to write the test that will run all After writing the struct and implementing the interface, now is time to write the test that will run all
of this code you wrote. Your test should be like: of this code you wrote. Your test should be like:
@ -480,64 +520,101 @@ func TestShellProvisioner(t *testing.T) {
} }
``` ```
If the environment variable **ACC_TEST_PROVISIONERS** is set as `all` or contains the provisioner type, then the test should run, otherwise the test should skip. The method `TestProvisionersAgainstBuilders` will run the provisioner against
In case of running it, you'll need to call the helper function `acc.TestProvisionersAgainstBuilders` passing a pointer to the test struct created above and the test testing pointer. all available and compatible builders. If there are not builders compatible with
the test you want to run, you can add a builder using the following steps:
The method `TestProvisionersAgainstBuilders` will run the provisioner against all available and compatible builders. An available builder Create a subdirectory in provisioneracc/test-fixtures for the type of builder
is the one that has the necessary code for running this type of test. In case the builder you want to run against is not available for testing, you can write it following the next steps. you are adding. In this subdirectory, add one json file containing a single
builder fragment. For example, one of our amazon-ebs builders is defined in
provisioneracc/test-fixtures/amazon-ebs/amazon-ebs.txt and contains:
To add a new builder to the available builders for provisioners acc testing, you'll need to create a new folder under the builder folder ```json
called `acceptance` and inside you create the `builder_acceptance.go` file and the package should be `<builder>_acc`. Like the provisioners, you'll need to create a struct that will {
implement the BuilderAcceptance interface. "type": "amazon-ebs",
```go "ami_name": "packer-acc-test",
type BuilderAcceptance interface { "instance_type": "t2.micro",
GetConfigs() (map[string]string, error) "region": "us-east-1",
GetBuilderStore() packer.MapOfBuilder "ssh_username": "ubuntu",
CleanUp() error "source_ami_filter": {
} "filters": {
``` "virtualization-type": "hvm",
- **GetConfigs()** should read a text file with the json configuration block for the builder and return a map of configs by OS type. "name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*",
For the Amazon EBS builder the file contains: "root-device-type": "ebs"
``` },
{ "owners": ["099720109477"],
"type": "amazon-ebs", "most_recent": true
"ami_name": "packer-acc-test", },
"instance_type": "m1.small", "force_deregister" : true,
"region": "us-east-1", "tags": {
"ssh_username": "ubuntu", "packer-test": "true"
"source_ami": "ami-0568456c", }
"force_deregister" : true,
"tags": {
"packer-test": "true"
}
}
```
The file should be placed under the `test-fixtures` folder.
In case you need to make references to another file, you'll need to add the relative path to provisioners folder like:
`../../builder/amazon/ebs/acceptance/test-fixtures/file.txt`.
- **GetBuilderStore()** this returns the builder store where we declare the available builders for running the build.
For the Amazon EBS builder this is:
```go
func (s *AmazonEBSAccTest) GetBuilderStore() packer.MapOfBuilder {
return packer.MapOfBuilder{
"amazon-ebs": func() (packer.Builder, error) { return &amazonebsbuilder.Builder{}, nil },
}
}
```
- **CleanUp()** cleans any resource created by the builder whether local or remote.
Once you created the builder necessary code, the last step is adding it to the `BuildersAccTest` map in `helper/tests/acc/provisioners.go`.
```go
var BuildersAccTest = map[string]BuilderAcceptance{
...
"amazon-ebs": new(amazonEBS.AmazonEBSAccTest),
...
} }
``` ```
Once you finish the steps, you should be ready to run your new provisioner acceptance test. note that this fragment does not contain anything other than a single builder
definition. The testing framework will combine this with the provisioner
fragment to create a working json template.
In order to tell the testing framework how to use this builder fragment, you
need to implement a `BuilderFixture` struct:
```go
type BuilderFixture struct {
// Name is the name of the builder fixture.
// Be simple and descriptive.
Name string
// Setup creates necessary extra test fixtures, and renders their values
// into the BuilderFixture.Template.
Setup func()
// Template is the path to a builder template fragment.
// The builder template fragment must be a json-formatted file containing
// the builder definition but no other portions of a packer template. For
// example:
//
// ```json
// {
// "type": "null",
// "communicator", "none"
// }
//```
//
// is a valid entry for "template" here, but the complete Packer template:
//
// ```json
// {
// "builders": [
// "type": "null",
// "communicator": "none"
// ]
// }
// ```
//
// is invalid as input.
//
// Only provide one builder template fragment per file.
TemplatePath string
// GuestOS says what guest os type the builder template fragment creates.
// Valid values are "windows", "linux" or "darwin" guests.
GuestOS string
// HostOS says what host os type the builder is capable of running on.
// Valid values are "any", windows", or "posix". If you set "posix", then
// this builder can run on a "linux" or "darwin" platform. If you set
// "any", then this builder can be used on any platform.
HostOS string
Teardown builderT.TestTeardownFunc
}
```
Implement this struct to the file "provisioneracc/builders.go", then add
the new implementation to the `BuildersAccTest` map in
`provisioneracc/provisioners.go`
Once you finish these steps, you should be ready to run your new provisioner
acceptance test by setting the name used in the BuildersAccTest map as your
`ACC_TEST_BUILDERS` environment variable.
#### Debugging Plugins #### Debugging Plugins