Adrien Delorme ed091163be
HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00

245 lines
10 KiB
Plaintext

---
description: |
Packer is designed to be extensible. Because the surface area for workloads is
infinite, Packer supports plugins for builders, provisioners, and
post-processors.
page_title: Extending
sidebar_title: Extending Packer
---
# Extending Packer
Packer is designed to be extensible. Because the surface area for workloads is
infinite, Packer supports plugins for builders, provisioners, and
post-processors. To learn more about the different customizations, please
choose a link from the sidebar.
## Developing Plugins
This page will document how you can develop your own Packer plugins. Prior to
reading this, it is assumed that you're comfortable with Packer and also know
the [basics of how Plugins work](/docs/plugins), from a user
standpoint.
Packer plugins must be written in [Go](https://golang.org/), so it is also
assumed that you're familiar with the language. This page will not be a Go
language tutorial. Thankfully, if you are familiar with Go, the Go toolchain
provides many conveniences to help to develop Packer plugins.
~> **Warning!** This is an advanced topic. If you're new to Packer, we
recommend getting a bit more comfortable before you dive into writing plugins.
### Plugin System Architecture
Packer has a fairly unique plugin architecture. Instead of loading plugins
directly into a running application, Packer runs each plugin as a _separate
application_. Inter-process communication and RPC is then used to communicate
between the many running Packer processes. Packer core itself is responsible
for orchestrating the processes and handles cleanup.
The beauty of this is that your plugin can have any dependencies it wants.
Dependencies don't need to line up with what Packer core or any other plugin
uses, because they're completely isolated into the process space of the plugin
itself.
And, thanks to Go's
[interfaces](https://golang.org/doc/effective_go.html#interfaces_and_types), it
doesn't even look like inter-process communication is occurring. You just use
the interfaces like normal, but in fact they're being executed in a remote
process. Pretty cool.
### Plugin Development Basics
Developing a plugin allows you to create additional functionality for Packer.
All the various kinds of plugins have a corresponding interface. The plugin
needs to implement this interface and expose it using the Packer plugin package
(covered here shortly), and that's it!
There are two packages that really matter that every plugin must use. Other
than the following two packages, you're encouraged to use whatever packages you
want. Because plugins are their own processes, there is no danger of colliding
dependencies.
- [`github.com/hashicorp/packer-plugin-sdk/packer`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer) - Contains all the interfaces that you have
to implement for any given plugin.
- [`github.com/hashicorp/packer-plugin-sdk/plugin`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin) - Contains the code to serve
the plugin. This handles all the inter-process communication stuff.
There are two steps involved in creating a plugin:
1. Implement the desired interface. For example, if you're building a builder
plugin, implement the [packer.Builder](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer#Builder) interface.
2. Serve the interface by calling the appropriate [plugin](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin#Set.RegisterBuilder) serving method in
your main method.
Basic examples are shown below. Note that if you can define a multi-plugin
binary as it will allow to add more that one plugin per binary.
<Tabs>
<Tab heading="Multi-plugin Binary">
```go
import (
"github.com/hashicorp/packer-plugin-sdk/plugin"
)
// Assume this implements packer.Builder
type ExampleBuilder struct{}
// Assume this implements packer.PostProcessor
type FooPostProcessor struct{}
// Assume this implements packer.Provisioner
type BarProvisioner struct{}
func main() {
pps := plugin.NewSet()
pps.RegisterBuilder("example", new(ExampleBuilder))
pps.RegisterPostProcessor("foo", new(FooPostProcessor))
pps.RegisterProvisioner("bar", new(BarProvisioner))
err := pps.Run()
if err != nil {
fmt.Fprintln(os.Stderr, err.Error())
os.Exit(1)
}
}
```
**That's it!** `plugin.NewSet` handles all the nitty gritty of
communicating with Packer core and serving your builder over RPC. It can't get
much easier than that.
Here the name of the plugin will be used to use each plugin, so if your plugin
is named `packer-plugin-my`, this would make the following parts available:
* the `my-example` builder
* the `my-foo` post-processor
* the `my-bar` provisioner
</Tab>
<Tab heading="Single Plugin Binary">
```go
import (
"github.com/hashicorp/packer-plugin-sdk/plugin"
)
// Assume this implements packer.Builder
type Builder struct{}
func main() {
server, err := plugin.Server()
if err != nil {
panic(err)
}
server.RegisterBuilder(new(Builder))
server.Serve()
}
```
**That's it!** `plugin.Server` handles all the nitty gritty of communicating with
Packer core and serving your builder over RPC. It can't get much easier than
that.
</Tab>
</Tabs>
Next, just build your plugin like a normal Go application, using `go build` or
however you please. The resulting binary is the plugin that can be installed
using standard installation procedures.
The specifics of how to implement each type of interface are covered in the
relevant subsections available in the navigation to the left.
~> **Lock your dependencies!** Using `go mod` is highly recommended since
the Packer codebase will continue to improve, potentially breaking APIs along
the way until there is a stable release. By locking your dependencies, your
plugins will continue to work with the version of Packer you lock to.
### Logging and Debugging
Plugins can use the standard Go `log` package to log. Anything logged using
this will be available in the Packer log files automatically. The Packer log is
visible on stderr when the `PACKER_LOG` environment var is set.
Packer will prefix any logs from plugins with the path to that plugin to make
it identifiable where the logs come from. Some example logs are shown below:
```text
2013/06/10 21:44:43 Loading builder: custom
2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin minimum port: 10000
2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin maximum port: 25000
2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin address: :10000
```
As you can see, the log messages from the custom builder plugin are prefixed
with "packer-builder-custom". Log output is _extremely_ helpful in debugging
issues and you're encouraged to be as verbose as you need to be in order for
the logs to be helpful.
### Creating a GitHub Release
To create a GitHub release with the assets that will be consumed by Packer, we provide a pre-defined release workflow configuration
using [GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions). We strongly encourage maintainers to use this configuration
to make sure the release contains the assets expected by Packer to load and use a plugin.
[GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions) allow you to execute workflows when events on your repository occur.
You can use this to create releases whenever a new version tag is created on your repository.
Here's what you need to create releases using GitHub Actions:
1. Generate a GPG key to be used when signing releases (See [GitHub's detailed instructions](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/generating-a-new-gpg-key)
for help with this step)
2. Copy the [GoReleaser configuration from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.goreleaser.yml) to the root of your repository.
3. Copy the [GitHub Actions workflow from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.github/workflows/release.yml) to `.github/workflows/release.yml` in your repository.
An easier way to execute steps 1 and 2, is by running the following command in the root folder of your plugin to download
and place both files in their expected place:
```shell
curl -L -o .goreleaser.yml https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.goreleaser.yml &&
mkdir -p .github/workflows &&
curl -L -o .github/workflows/release.yml https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.github/workflows/release.yml
```
4. Go to Settings > Secrets in your repository, and add the following secrets:
- `GPG_PRIVATE_KEY` - Your ASCII-armored GPG private key. You can export this with `gpg --armor --export-secret-keys [key ID or email]`.
- `PASSPHRASE` - The passphrase for your GPG private key.
5. Push a new valid version tag (e.g. `v1.2.3`) to test that the GitHub Actions releaser is working. The tag must be a valid [Semantic Version](https://semver.org/) preceded with a `v`.
### Plugin Development Tips
Here are some tips for developing plugins, often answering common questions or
concerns.
#### Naming Conventions
It is standard practice to name the resulting plugin application in the format
of `packer-plugin-NAME`. For example, if you're building a new builder for
CustomCloud, it would be standard practice to name the resulting plugin
`packer-plugin-custom-cloud`. This naming convention helps users identify the
scope of a plugin.
#### Testing Plugins
Making your unpublished plugin available to Packer is possible by either :
* Starting Packer from the directory where the plugin binary is located.
* Putting the plugin binary in the same directory as Packer.
In both these cases, if the binary is called `packer-plugin-myawesomecloud` and
defines an `ebs` builder then you will be able to use an `myawesomecloud-ebs`
builder or source without needing to have a `required_plugin` block.
This is extremely useful during development.
#### Distributing Plugins
It is recommended you use a tool like the GoReleaser in order to cross-compile
your plugin for every platform that Packer supports, since Go applications are
platform-specific. If you have created your plugin from the
[packer-plugin-scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding)
repo, simply tagging a commit will correctly build and release the binaries using GoReleaser.