diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index ae1f19343..96e5c6c6e 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -330,40 +330,49 @@ credentials are missing, so those are not documented here. #### Running Provisioners Acceptance Tests -To run the Provisioners Acceptance Tests you should set both ACC_TEST_BUILDERS and ACC_TEST_PROVISIONERS environment variables. -These will tell which provisioner and builder should run the test against with. +**Warning:** The acceptance tests create/destroy/modify _real resources_, which +may incur costs for real money. In the presence of a bug, it is possible that +resources may be left behind, which can cost money even though you were not +using them. We recommend running tests in an account used only for that purpose +so it is easy to see if there are any dangling resources, and so production +resources are not accidentally destroyed or overwritten during testing. +Also, these typically require an API key (AWS, GCE), or additional software +to be installed on your computer (VirtualBox, VMware). + +To run the Provisioners Acceptance Tests you should use both ACC_TEST_BUILDERS and ACC_TEST_PROVISIONERS variables to +tell which provisioner and builder the test should be run against. Examples of usage: -``` -ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell go test ./provisioner/shell/... -v -timeout=1 -``` -Will run the Shell provisioner acceptance tests against the Amazon EBS builder. -``` -ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell make provisioners-acctest -``` -Will do the same but using the Makefile -``` -ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell,powershell make provisioners-acctest -``` -Will run the all Shell and Powershell provisioners acceptance tests against the Amazon EBS builder. -``` -ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=all make provisioners-acctest -``` -Will run the all provisioners acceptance tests against the Amazon EBS builder. -``` -ACC_TEST_BUILDERS=all ACC_TEST_PROVISIONERS=all make provisioners-acctest -``` -Will run the all provisioners acceptance tests against all builders whenever they are compatible. +- Run the Shell provisioner acceptance tests against the Amazon EBS builder. + ``` + ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell go test ./provisioner/shell/... -v -timeout=1 + ``` +- Do the same but using the Makefile + ``` + ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell make provisioners-acctest + ``` +- Run the all Shell and Powershell provisioners acceptance tests against the Amazon EBS builder. + ``` + ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=shell,powershell make provisioners-acctest + ``` +- Run the all provisioners acceptance tests against the Amazon EBS builder. + ``` + ACC_TEST_BUILDERS=amazon-ebs ACC_TEST_PROVISIONERS=all make provisioners-acctest + ``` +- Run the all provisioners acceptance tests against all builders whenever they are compatible. + ``` + ACC_TEST_BUILDERS=all ACC_TEST_PROVISIONERS=all make provisioners-acctest + ``` Both ACC_TEST_BUILDERS and ACC_TEST_PROVISIONERS allows defining a list of builders and provisioners separated by comma -(e.g. ACC_TEST_BUILDERS=amazon-ebs,virtualbox-iso) +(e.g. `ACC_TEST_BUILDERS=amazon-ebs,virtualbox-iso`) #### Writing Provisioner Acceptance Tests Packer has an already implemented structure that will run the provisioner against builders and you can find it in `helper/tests/acc/provisioners.go`. -All provisioner's acceptance test should use this structure in their acceptance tests. +All provisioner's should use this structure in their acceptance tests. To start writing a new provisioner acceptance test, you should add a test file named as `provisioner_acc_test.go` in the provisioner folder and the package should be `_test`. This file should have a struct that will implement the ProvisionerAcceptance interface. @@ -380,8 +389,8 @@ type ProvisionerAcceptance interface { - **GetName()** should return the provisioner type. For example for the Shell provisioner the method returns "shell". -- **GetConfig()** should read a text file with the json configuration block for the provisioner and any other necessary. -For the Shell provisioner the file contains: +- **GetConfig()** should read a text file with the json configuration block for the provisioner and any other necessary provisioner. +For the Shell one the file contains: ``` { @@ -402,7 +411,7 @@ For the Shell provisioner the file contains: This config should be returned as string that will be later merged with the builder config into a full template. - **GetProvisionerStore()** this returns the provisioner store where we declare the available provisioners for running the build. -For Shell provisioners this is: +For the Shell provisioners this is: ```go func (s *ShellProvisionerAccTest) GetProvisionerStore() packer.MapOfProvisioner { return packer.MapOfProvisioner{ @@ -415,8 +424,8 @@ For Shell provisioners this is: - **IsCompatible(builder string, vmOS string)** returns true or false whether the provisioner should run against a specific builder or/and specific OS. -- **RunTest(c \*command.BuildCommand, args []string)** it will actually run the build and return any error if it fails any necessary validation. -For Shell provisioner this is: +- **RunTest(c \*command.BuildCommand, args []string)** it will actually run the build and return any error if it fails the validations. +For the Shell provisioner this is: ```go func (s *ShellProvisionerAccTest) RunTest(c *command.BuildCommand, args []string) error { // Provisioner specific setup @@ -449,8 +458,8 @@ For Shell provisioner this is: ``` -After writing the struct and implementing the interface, now is time to actually write the test that will run all -of this code. Your test should look like: +After writing the struct and implementing the interface, now is time to write the test that will run all +of this code you wrote. Your test should be like: ```go func TestShellProvisioner(t *testing.T) { @@ -462,13 +471,13 @@ func TestShellProvisioner(t *testing.T) { } ``` -If the environment variable **ACC_TEST_PROVISIONERS** is set as `all` or contains the provisioner type the test should run, otherwise the test should skip. +If the environment variable **ACC_TEST_PROVISIONERS** is set as `all` or contains the provisioner type, then the test should run, otherwise the test should skip. In case of running it, you'll need to call the helper function `acc.TestProvisionersAgainstBuilders` passing a pointer to the test struct created above and the test testing pointer. -The method `TestProvisionersAgainstBuilders` will run the provisioner against all available and compatible builder. An available builder +The method `TestProvisionersAgainstBuilders` will run the provisioner against all available and compatible builders. An available builder is the one that has the necessary code for running this type of test. In case the builder you want to run against is not available for testing, you can write it following the next steps. -To add a new builder to the available builders for acc testing, you'll need to create a new folder under the builder folder +To add a new builder to the available builders for provisioners acc testing, you'll need to create a new folder under the builder folder called `acceptace` and inside you create the `builder_acceptance.go` file and the package should be `_acc`. Like the provisioners, you'll need to create a struct that will implement the BuilderAcceptance interface. ```go