iSharkFly-Docs
/
packer-cn
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #10478 from hashicorp/builder_revamp 

refresh builder docs
This commit is contained in:
Megan Marsh 2021-01-14 11:21:22 -08:00 committed by GitHub
parent b6fb3e8bc9 c64aa83c56
commit f5cbb56c14
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 143 additions and 90 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

233
website/content/docs/extending/custom-builders.mdx
View File

@ -10,23 +10,27 @@ sidebar_title: Custom Builders
Packer Builders are the components of Packer responsible for creating a
machine, bringing it to a point where it can be provisioned, and then turning
that provisioned machine into some sort of machine image. Several builders are
officially distributed with Packer itself, such as the AMI builder, the VMware
builder, etc. However, it is possible to write custom builders using the Packer
plugin interface, and this page documents how to do that.
that provisioned machine into a machine image. Several builders are
officially maintained and distributed by the HashiCorp Packer team -- these
builders include builders for creating images on Amazon EC2, VMWare, Google
Compute Engine, and many more. It is also possible to write custom builders
using the Packer plugin interface, and this page documents how to do that.
Prior to reading this page, it is assumed you have read the page on [plugin
development basics](/docs/extending/plugins).
Prior to reading this page, you should read the page on [plugin development
basics](/docs/extending/plugins).
~> **Warning!** This is an advanced topic. If you're new to Packer, we
recommend getting a bit more comfortable before you dive into writing plugins.
recommend getting comfortable with using Packer and its officially maintained
plugins before you dive into writing plugins of your own.
Custom plugins are written in [golang](https://golang.org/), so this guide
assumes that you have some familiarity with that programming language.
## The Interface
The interface that must be implemented for a builder is the `packer.Builder`
interface. It is reproduced below for reference. The actual interface in the
source code contains some basic documentation as well explaining what each
method should do.
To create your own builder, you must create a struct that implements the
`packer.Builder` interface. It is reproduced below for reference.
```go
type Builder interface {
@ -45,72 +49,101 @@ function, check our
### The "Prepare" Method
The `Prepare` method for each builder is called prior to any runs with the
configuration that was given in the template. This is passed in as an array of
`interface{}` types, but is generally `map[string]interface{}`. The prepare
method is responsible for translating this configuration into an internal
structure, validating it, and returning any errors.
The `Prepare` method for each builder will be called by the Packer core
at the beginning of the build. Its purpose is to parse and validate the
configuration template provided to Packer with `packer build
your_packer_template.json`, but not to execute API calls or begin creating any
resources or artifacts.
For multiple parameters, they should be merged together into the final
configuration, with later parameters overwriting any previous configuration.
The exact semantics of the merge are left to the builder author.
The configuration from your Packer template is passed into the Prepare() method
as an array of `interface{}` types, but is generally `map[string]interface{}`.
The Prepare method is responsible for translating this configuration into an
internal structure, validating it, and returning any errors.
For decoding the `interface{}` into a meaningful structure, the
[mapstructure](https://godoc.org/github.com/mitchellh/mapstructure) library is
recommended. Mapstructure will take an `interface{}` and decode it into an
arbitrarily complex struct. If there are any errors, it generates very human
friendly errors that can be returned directly from the prepare method.
If multiple parameters are passed into Prepare(), they should be merged together
into the final configuration, with later parameters overwriting any previous
configuration. The exact semantics of the merge are left to the builder author.
While it is not actively enforced, **no side effects** should occur from
running the `Prepare` method. Specifically, don't create files, don't launch
virtual machines, etc. Prepare's purpose is solely to configure the builder and
validate the configuration.
We recommend that you use the
[mapstructure](https://godoc.org/github.com/mitchellh/mapstructure) library to
decode the interface{} into a meaningful structure. Mapstructure will take an
`interface{}` and decode it into an arbitrarily complex struct. If there are any
errors, it generates very human friendly errors that can be returned directly
from the prepare method. You can find many usage examples of this library within
the Prepare() methods of HashiCorp-maintained Packer plugins.
In addition to normal configuration, Packer will inject a
`map[string]interface{}` with a key of `packer.DebugConfigKey` set to boolean
While Packer does not actively enforce this, **no side effects** should occur
from running the `Prepare` method. Specifically: don't create files, don't
launch virtual machines, etc. Prepare's purpose is solely to load the
configuration from the template into a format usable by your builder, to
validate that configuration, and to apply necessary defaults to that
configuration.
In addition to the configuration provided in the Packer template, Packer will
also supply a [common.PackerConfig](https://github.com/hashicorp/packer-plugin-sdk/blob/8a28198491f70deca3824ce452adf6f9bd507880/common/packer_config.go#L44)
containing meta-information such as the build name, builder type, core version,
etc, and coded into a `map[string]interface{}`. One important piece of
meta information in this map is the `packer.DebugConfigKey` set to boolean
`true` if debug mode is enabled for the build. If this is set to true, then the
builder should enable a debug mode which assists builder developers and
advanced users to introspect what is going on during a build. During debug
builds, parallelism is strictly disabled, so it is safe to request input from
stdin and so on.
Prepare() returns an array of strings and an error. The array of strings is a
special list of keys for variables created at runtime which your builder will
make accessible to provisioners using the generatedData mechanism (see below for
more details) An example could be an instance ID for the cloud instance created
by your builder. If you do not plan to make use of the generatedData feature,
just return an empty list. The error should be used if there is something wrong
with the user-provided configuration and the build should not proceed.
### The "Run" Method
`Run` is where all the interesting stuff happens. Run is executed, often in
parallel for multiple builders, to actually build the machine, provision it,
and create the resulting machine image, which is returned as an implementation
of the `packer.Artifact` interface.
`Run` is executed, often in parallel for multiple builders, to actually build
the machine, provision it, and create the resulting machine image, which is
returned as an implementation of the `packer.Artifact` interface.
The `Run` method takes three parameters. These are all very useful. The
`packer.Ui` object is used to send output to the console. `packer.Hook` is used
to execute hooks, which are covered in more detail in the hook section below.
And `packer.Cache` is used to store files between multiple Packer runs, and is
covered in more detail in the cache section below.
The `Run` method takes three parameters. The context.Context used to cancel the
build. The `packer.Ui` object is used to send output to the console.
`packer.Hook` is used to execute hooks, which are covered in more detail in the
Provisioning section below.
Because builder runs are typically a complex set of many steps, the
packer-plugin-sdk contains a
[multistep](https://godoc.org/github.com/hashicorp/packer-plugin-sdk/multistep)
package is recommended to bring order to the complexity. Multistep is a library
which allows you to separate your logic into multiple distinct "steps" and
string them together. It fully supports cancellation mid-step and so on. Please
check it out, it is how the built-in builders are all implemented.
module. Multistep allows you to separate your build logic into multiple distinct
"steps" with separate run and cleanup phases, and run them in order. It
supports cancellation mid-step, pausing between steps when debugging, the CLI's
on-error flag, and more. All of the HashiCorp maintained builders make use of
this module, and while it is not required for builder implementation, it will
help you create your builder in a way that matches user and Packer Core
assumptions. The SDK also provides a number of "helper" generic steps that may
prevent you from having to re-implement work that has already been done by the
HashiCorp maintainers. Examples include sending boot commands, connecting to
SSH, and creating virtual CDs to mount on your VM. Take a look at the
[communicator](https://github.com/hashicorp/packer-plugin-sdk/tree/main/communicator)
and
[multistep/commonsteps](https://github.com/hashicorp/packer-plugin-sdk/tree/main/multistep/commonsteps)
modules in the SDK to see what tools are available to you.
Finally, as a result of `Run`, an implementation of `packer.Artifact` should be
returned. More details on creating a `packer.Artifact` are covered in the
artifact section below. If something goes wrong during the build, an error can
be returned, as well. Note that it is perfectly fine to produce no artifact and
no error, although this is rare.
Finally, `Run` should return an implementation of `packer.Artifact`. More
details on creating a `packer.Artifact` are covered in the artifact section
below. If something goes wrong during the build that prevents an artifact from
being correctly created, `Run` should return an error and a nil artifact. Note
that your builder is allowed to produce no artifact and no error, although this
is a rare use case.
### Cancellation
The `Run` method is often run in parallel.
#### With the "Cancel" Method ( up until packer 1.3 )
#### With the "Cancel" Method ( for plugins for Packer < v1.3 )
The `Cancel` method can be called at any time and requests cancellation of any
builder run in progress. This method should block until the run actually stops.
Not that the Cancel method will no longer be called since packer 1.4.0.
Note that the Cancel method will not be called by Packer versions >= 1.4.0.
#### Context cancellation ( from packer 1.4 )
#### Context cancellation ( from Packer v1.4 )
The `<-ctx.Done()` can unblock at any time and signifies request for
cancellation of any builder run in progress.
@ -118,32 +151,36 @@ cancellation of any builder run in progress.
Cancels are most commonly triggered by external interrupts, such as the user
pressing `Ctrl-C`. Packer will only exit once all the builders clean up, so it
is important that you architect your builder in a way that it is quick to
respond to these cancellations and clean up after itself.
respond to these cancellations and clean up after itself. If your builder makes
a long-running call, you should consider the possibility that a user may cancel
the build during that call, and make sure that such a cancellation is not
blocked.
## Creating an Artifact
The `Run` method is expected to return an implementation of the
`packer.Artifact` interface. Each builder must create their own implementation.
The interface has ample documentation to help you get started.
`packer.Artifact` interface. Each builder must create its own implementation of
this interface.
The only part of an artifact that may be confusing is the `BuilderId` method.
This method must return an absolutely unique ID for the builder. In general, I
follow the practice of making the ID contain my GitHub username and then the
platform it is building for. For example, the builder ID of the VMware builder
is "hashicorp.vmware" or something similar.
Most of the pieces of an artifact should be fairly self-explanatory by reading
the [packer.Artifact interface
documentation](https://godoc.org/github.com/hashicorp/packer-plugin-sdk/packer#Artifact).
However one part of an artifact that may be confusing is the `BuilderId` method.
This method must return an absolutely unique ID for the builder. In general, a
reasonable ID would be the github username or organization that created the
builder, followed by the platform it is building for. For example, the builder
ID of the VMware builder is "hashicorp.vmware".
Post-processors use the builder ID value in order to make some assumptions
about the artifact results, so it is important it never changes.
Other than the builder ID, the rest should be self-explanatory by reading the
[packer.Artifact interface
documentation](https://godoc.org/github.com/hashicorp/packer-plugin-sdk/packer#Artifact).
## Provisioning
Packer has built-in support for provisioning, but the moment when provisioning
runs must be invoked by the builder itself, since only the builder knows when
the machine is running and ready for communication.
Packer has built-in support for provisioning using the Provisioner plugins. But
builders themselves, rather than the Packer core, must determine when to invoke
the provisioners since only the builder knows when the machine is running and
ready for communication.
When the machine is ready to be provisioned, run the `packer.HookProvision`
hook, making sure the communicator is not nil, since this is required for
@ -156,22 +193,25 @@ hook.Run(context.Context, packer.HookProvision, ui, comm, nil)
At this point, Packer will run the provisioners and no additional work is
necessary.
-> **Note:** Hooks are still undergoing thought around their general design
and will likely change in a future version. They aren't fully "baked" yet, so
they aren't documented here other than to tell you how to hook in provisioners.
If you are using the multistep tooling, the Packer plugin SDK contains a
generic StepProvision which handles execution the provision hook for you and
automatically supplies any custom builder generatedData you would like to
provide to procisioners (see below for more details on generatedData.)
## Template Engine
### Build variables
Packer makes it possible to provide custom template engine variables to be shared with
provisioners and post-processors using the `build` function.
Packer makes it possible to provide custom template engine variables to be
shared with provisioners and post-processors using the `build` function.
Json template `build` docs are [here](https://www.packer.io/docs/templates/engine#build)
and HCL template build docs are [here](https://www.packer.io/docs/from-1.5/contextual-variables#build-variables).
Part of the builder interface changes made in 1.5.0 was to make builder Prepare() methods
return a list of custom variables which we call `generated data`.
We use that list of variables to generate a custom placeholder map per builder that
combines custom variables with the placeholder map of default build variables created by Packer.
Here's an example snippet telling packer what will be made available by the builder:
As of Packer v1.5.0, builder Prepare() methods return a list of custom variables
which we call `generated data`. We use that list of variables to generate a
custom placeholder map per builder that combines custom variables with the
placeholder map of default build variables created by Packer. Here's an example
snippet telling packer what will be made available by the builder:
```go
func (b *Builder) Prepare(raws ...interface{}) ([]string, []string, error) {
@ -182,11 +222,13 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, []string, error) {
}
```
Returning the custom variable name(s) within the `generated_data` placeholder is necessary
for the template containing the build variable(s) to validate.
When a user provides a Packer template that uses a `build` function, Packer
validates that the key in the `build` function exists for a given builder using
this generatedData array. If it does not exist, then Packer validation will
fail.
Once the placeholder is set, it's necessary to pass the variables' real values when calling
the provisioner. This can be done as the example below:
Once the placeholder is set, it's necessary to pass the variables' real values
when calling the provisioner. This can be done as the example below:
```go
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
@ -201,10 +243,11 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
}
```
In order to make these same variables and the Packer default ones also available to post-processor,
it is necessary to add them to the Artifact returned by the builder. This can be done by adding an attribute of type
`map[string]interface{}` to the Artifact and putting the generated data in it. The post-processor
will access this data later via the Artifact's `State` method.
In order to make these same variables and the Packer default ones also available
to post-processors, your builder will need to add them to its Artifact.
This can be done by adding an attribute of type `map[string]interface{}` to the
Artifact and putting the generated data in it. The post-processor will access
this data later via the Artifact's `State` method.
The Artifact code should be implemented similar to the below:
@ -226,8 +269,8 @@ func (a *Artifact) State(name string) interface{} {
// ...
```
The builder should return the above Artifact containing the generated data and the code should be similar
to the example snippet below:
The builder should return the above Artifact containing the generated data and
the code should be similar to the example snippet below:
```go
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
@ -240,7 +283,8 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
}
```
The code above assigns the `generated_data` state to the `StateData` map with the key `generated_data`.
The code above assigns the `generated_data` state to the `StateData` map with
the key `generated_data`.
Here some example of how this data will be used by post-processors:
@ -254,4 +298,13 @@ func (p *PostProcessor) PostProcess(ctx context.Context, ui packer.Ui, source pa
}
```
To know more about the template engine build function, please refer to the [template engine docs](/docs/templates/engine).
## Putting it all together
This page has focused up until now on the implementation details for the Builder
interface. You will need to create a server and store the builder in a binary in
order to make it available to the Packer core as a plugin. We have created a
[scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/builder/scaffolding/builder.go)
repo to give you an idea of the relationship between the builder
implementation and the server implementation within a repository, and then read
[basics of how Plugins work](/docs/extending/plugins), which breaks down all the
server details.