2013-06-11 00:47:16 -04:00
---
2017-06-14 21:04:16 -04:00
description: |
2021-02-02 12:05:04 -05:00
Packer is designed to be extensible. Because the surface area for workloads is
infinite, Packer supports plugins for builders, provisioners, and
post-processors.
page_title: Extending
2017-03-25 18:13:52 -04:00
---
2021-02-02 12:05:04 -05:00
# Extending Packer
2017-03-25 18:13:52 -04:00
2021-02-15 05:10:43 -05:00
Packer is designed to be extensible, and supports plugins that allow you to
create and use custom builders, provisioners, post-processors, and data sources.
To learn more about developing these different types of components, please
choose a link from the sidebar. To learn more about the general plugin
architecture, stay on this page.
2020-12-10 03:22:09 -05:00
2017-03-25 18:13:52 -04:00
## Developing Plugins
2013-06-11 00:47:16 -04:00
2015-07-22 22:31:00 -04:00
This page will document how you can develop your own Packer plugins. Prior to
2021-02-15 05:10:43 -05:00
reading this, you should be comfortable with Packer and know the
basics of [how plugins work from a user standpoint](/docs/plugins).
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
Packer plugins must be written in [Go](https://golang.org/), so you should also
be familiar with the language.
2013-06-11 00:47:16 -04:00
2020-03-23 20:02:12 -04:00
~> **Warning!** This is an advanced topic. If you're new to Packer, we
2014-10-20 13:55:16 -04:00
recommend getting a bit more comfortable before you dive into writing plugins.
2013-06-11 00:47:16 -04:00
2017-03-25 18:13:52 -04:00
### Plugin System Architecture
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
A Packer plugin is just a go binary. Instead of loading plugins directly into a
running application, Packer runs each plugin as a _separate application_.
The multiple separate Packer plugin processes communicate with the Core using
an RPC defined in the packer-plugin SDK. The Packer core itself is responsible
launching and cleaning up the plugin processes.
2013-06-11 00:47:16 -04:00
2017-03-25 18:13:52 -04:00
### Plugin Development Basics
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
The components that can be created and used in a Packer plugin are builders,
provisioners, post-processors, and data sources.
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
Each of these components has a corresponding [interface](https://golang.org/doc/effective_go.html#interfaces_and_types).
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
All you need to do to create a plugin is:
2021-03-31 15:07:00 -04:00
1. create an implementation of the desired interface, and
2. serve it using the server provided in the [packer-plugin-sdk](https://github.com/hashicorp/packer-plugin-sdk).
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
The core and the SDK handle all of the communication details inside the server.
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
Your plugin must use two packages from the SDK to implement the server and
interfaces. You're encouraged to use whatever other packages you want in your
plugin implementation. Because plugins are their own processes, there is no
danger of colliding dependencies.
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
- [`github.com/hashicorp/packer-plugin-sdk/packer`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer) - Contains all the interfaces that you have to implement for any given plugin.
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
- [`github.com/hashicorp/packer-plugin-sdk/plugin`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin) - Contains the code to serve the plugin. This handles all the inter-process communication.
2020-12-10 03:22:09 -05:00
2021-02-15 05:10:43 -05:00
Basic examples of serving your component are shown below. Note that if you
define a multi-component plugin, you can (but do not need to) add more than one
component per plugin binary. The multi-component plugin is also compatible with
download and installation via `packer init`, whereas the single-component plugin
is not.
2020-12-10 03:22:09 -05:00
<Tabs>
2021-03-09 13:06:11 -05:00
2021-02-15 05:10:43 -05:00
<Tab heading="Multi-component Plugin (recommended) ">
2013-06-11 00:47:16 -04:00
2020-03-18 18:46:47 -04:00
```go
2021-02-15 05:10:43 -05:00
// main.go
2013-06-11 00:47:16 -04:00
import (
2020-12-17 16:29:25 -05:00
"github.com/hashicorp/packer-plugin-sdk/plugin"
2013-06-11 00:47:16 -04:00
)
2021-02-15 05:10:43 -05:00
// Assume this implements the packer.Builder interface
2020-12-10 03:22:09 -05:00
type ExampleBuilder struct{}
2021-02-15 05:10:43 -05:00
// Assume this implements the packer.PostProcessor interface
2020-12-10 03:22:09 -05:00
type FooPostProcessor struct{}
2021-02-15 05:10:43 -05:00
// Assume this implements the packer.Provisioner interface
2020-12-10 03:22:09 -05:00
type BarProvisioner struct{}
2013-06-11 00:47:16 -04:00
func main() {
2020-12-10 03:22:09 -05:00
pps := plugin.NewSet()
pps.RegisterBuilder("example", new(ExampleBuilder))
2021-02-15 05:10:43 -05:00
pps.RegisterBuilder(plugin.DEFAULT_NAME, new(AnotherBuilder))
2020-12-10 03:22:09 -05:00
pps.RegisterPostProcessor("foo", new(FooPostProcessor))
pps.RegisterProvisioner("bar", new(BarProvisioner))
err := pps.Run()
2019-07-24 23:46:37 -04:00
if err != nil {
2020-12-10 03:22:09 -05:00
fmt.Fprintln(os.Stderr, err.Error())
os.Exit(1)
2019-07-24 23:46:37 -04:00
}
2013-06-11 00:47:16 -04:00
}
2014-10-20 13:55:16 -04:00
```
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
This `plugin.NewSet` invocation handles all the details of communicating with
Packer core and serving your component over RPC. As long as your struct being
registered implements one of the component interfaces, Packer will now be able
to launch your plugin and use it.
If you register a component with its own name, the component name will be
appended to the plugin name to create a unique name. If you register a component
using the special string constant `plugin.DEFAULT_NAME`, then the component will
be referenced by using only the plugin name. For example:
2013-06-11 00:47:16 -04:00
2021-02-15 05:10:43 -05:00
If your plugin is named `packer-plugin-my`, the above set definition would make
the following components available:
2020-12-10 03:22:09 -05:00
2021-03-31 15:07:00 -04:00
- the `my-example` builder
- the `my` builder
- the `my-foo` post-processor
- the `my-bar` provisioner
2020-12-10 03:22:09 -05:00
</Tab>
2021-03-09 13:06:11 -05:00
2021-02-15 05:10:43 -05:00
<Tab heading="Single Component Plugin (deprecated)">
2020-12-10 03:22:09 -05:00
```go
2021-02-15 05:10:43 -05:00
// main.go
2020-12-10 03:22:09 -05:00
import (
2021-02-02 12:05:04 -05:00
"github.com/hashicorp/packer-plugin-sdk/plugin"
2020-12-10 03:22:09 -05:00
)
2021-02-15 05:10:43 -05:00
// Assume this implements the packer.Builder interface
2020-12-10 03:22:09 -05:00
type Builder struct{}
func main() {
server, err := plugin.Server()
if err != nil {
panic(err)
}
server.RegisterBuilder(new(Builder))
server.Serve()
}
```
2021-02-15 05:10:43 -05:00
This `server.Serve()` invocation handles all the details of communicating with
Packer core and serving your component over RPC. As long as your struct being
registered implements one of the component interfaces, Packer will now be able
to launch your plugin and use it.
Please note that single-component plugins exist for backwards-compatability. We
would rather you register your component using the multi-component method shown
in the other tab, even if you only have one component in your binary. This is
because the `packer init` command only supports multi-component plugins.
2020-12-10 03:22:09 -05:00
</Tab>
</Tabs>
2021-02-15 05:10:43 -05:00
Next, build your plugin as you would any other Go application. The resulting
binary is the plugin that can be installed using
[standard installation procedures](/docs/plugins#installing-plugins).
2013-06-11 00:47:16 -04:00
2015-07-22 22:31:00 -04:00
The specifics of how to implement each type of interface are covered in the
relevant subsections available in the navigation to the left.
2013-06-11 00:47:16 -04:00
2020-03-23 20:02:12 -04:00
~> **Lock your dependencies!** Using `go mod` is highly recommended since
2018-01-24 19:59:32 -05:00
the Packer codebase will continue to improve, potentially breaking APIs along
the way until there is a stable release. By locking your dependencies, your
plugins will continue to work with the version of Packer you lock to.
2013-06-27 13:43:19 -04:00
2017-03-25 18:13:52 -04:00
### Logging and Debugging
2013-06-11 00:47:16 -04:00
2018-10-26 20:02:51 -04:00
Plugins can use the standard Go `log` package to log. Anything logged using
this will be available in the Packer log files automatically. The Packer log is
2021-02-02 12:05:04 -05:00
visible on stderr when the `PACKER_LOG` environment var is set.
2013-06-11 00:47:16 -04:00
2018-10-26 20:02:51 -04:00
Packer will prefix any logs from plugins with the path to that plugin to make
it identifiable where the logs come from. Some example logs are shown below:
2013-06-11 00:47:16 -04:00
2020-03-18 18:46:47 -04:00
```text
2017-07-04 10:39:41 -04:00
2013/06/10 21:44:43 Loading builder: custom
2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin minimum port: 10000
2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin maximum port: 25000
2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin address: :10000
2013-06-11 00:47:16 -04:00
```
2017-07-04 10:41:40 -04:00
As you can see, the log messages from the custom builder plugin are prefixed
2020-03-18 18:46:47 -04:00
with "packer-builder-custom". Log output is _extremely_ helpful in debugging
2018-10-26 20:02:51 -04:00
issues and you're encouraged to be as verbose as you need to be in order for
the logs to be helpful.
2013-06-11 00:47:16 -04:00
2020-12-15 04:28:17 -05:00
### Creating a GitHub Release
2021-02-15 05:10:43 -05:00
`packer init` does not work using a centralized registry. Instead, it requires
you to publish your plugin in a GitHub repo with the name
2021-03-31 15:07:00 -04:00
`packer-plugin-*` where \* represents the name of your plugin. You also need to
2021-02-15 05:10:43 -05:00
create a GitHub release of your plugin with specific assets for the
`packer init` download to work. We provide a pre-defined release workflow
configuration using
[GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions). We
strongly encourage maintainers to use this configuration to make sure the
release contains the right assets with the right names for Packer to leverage
`packer init` installation.
2020-12-15 04:28:17 -05:00
2020-12-15 06:38:03 -05:00
Here's what you need to create releases using GitHub Actions:
2021-03-31 15:07:00 -04:00
2020-12-15 04:28:17 -05:00
1. Generate a GPG key to be used when signing releases (See [GitHub's detailed instructions](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/generating-a-new-gpg-key)
2021-03-31 15:07:00 -04:00
for help with this step)
2021-02-02 12:05:04 -05:00
2. Copy the [GoReleaser configuration from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.goreleaser.yml) to the root of your repository.
2021-03-31 15:07:00 -04:00
```sh
curl -L -o .goreleaser.yml \
https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.goreleaser.yml
```
2021-02-02 12:05:04 -05:00
3. Copy the [GitHub Actions workflow from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.github/workflows/release.yml) to `.github/workflows/release.yml` in your repository.
2021-03-31 15:07:00 -04:00
```sh
mkdir -p .github/workflows &&
curl -L -o .github/workflows/release.yml \
https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.github/workflows/release.yml
```
2021-02-15 05:10:43 -05:00
4. Go to your repository page on GitHub and navigate to Settings > Secrets. Add
2021-03-31 15:07:00 -04:00
the following secrets:
- `GPG_PRIVATE_KEY` - Your ASCII-armored GPG private key. You can export this with `gpg --armor --export-secret-keys [key ID or email]`.
- `PASSPHRASE` - The passphrase for your GPG private key.
2021-02-15 05:10:43 -05:00
5. Push a new valid version tag (e.g. `v1.2.3`) to test that the GitHub Actions
2021-03-31 15:07:00 -04:00
releaser is working. The tag must be a valid
[Semantic Version](https://semver.org/) preceded with a `v`. Once the tag is pushed, the github actions you just configured will automatically build release binaries that Packer can download using `packer init`. For more details on how
to install a plugin using `packer init`, see the
[init docs](/docs/commands/init).
2013-06-11 00:47:16 -04:00
2021-03-09 13:06:11 -05:00
### Registering Plugin Documentation
~> Note: Registering a remote plugin's plugin documentation requires the use of [Packer's plugin docs configuration](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/docs).
`packer init` allows users to require and install remote Packer plugins, those not bundled with Packer core, that have been published to GitHub automatically.
To help with the discovery of remote Packer plugins on GitHub, plugins maintainers can choose to register plugin documentation for each component directly on the [Packer Documentation Page](https://packer.io/docs).
2021-03-30 17:12:28 -04:00
The registration process requires the creation of a `docs.zip` file archive containing the `.mdx` files for each of the plugin components in the remote plugin's repository. A working example can be seen at the [packer-plugin-docker repository](https://github.com/hashicorp/packer-plugin-docker/releases/latest).
2021-03-09 13:06:11 -05:00
2021-03-30 17:12:28 -04:00
Once in place the remote plugin can be added to Packer's website builds by opening a pull-request against [hashicorp/packer](https://github.com/packer), with the needed configuration for pulling in the remote documentation.
2021-03-09 13:06:11 -05:00
Remote plugins will have their components listed under the respected types (i.e builders, provisioners, etc) using the names specified in the remote block configuration, and labeled with their respective [tier and namespace](/docs/plugins#tiers-and-namespaces).
To register a plugin follow one of the following setups
<Tabs>
2021-03-30 17:12:28 -04:00
<Tab heading="Generating docs.zip on plugin release">
2021-03-09 13:06:11 -05:00
Documentation for a plugin is maintained within the `docs` directory and served on GitHub.
2021-03-30 17:12:28 -04:00
To include plugin docs on Packer.io a global pre-hook has been added to the main scaffolding [.goreleaser.yml](https://github.com/hashicorp/packer-plugin-scaffolding/blob/42e5b0b1e575879b0477cb6d4291e027f4d92f85/.goreleaser.yml#L10) file, that if uncommented will generate and include a docs.zip file as part of the plugin release.
2021-03-09 13:06:11 -05:00
2021-03-30 17:12:28 -04:00
The `docs.zip` file will contain all of the `.mdx` files under the plugins root `docs/` directory that can be consumed remotely by Packer.io.
2021-03-09 13:06:11 -05:00
2021-03-30 17:12:28 -04:00
Once the first `docs.zip` file has been included into a release you will need to open a one time pull-request against [hashicorp/packer](https://github.com/hashicorp/packer) to register the plugin docs.
2021-03-09 13:06:11 -05:00
2021-03-30 17:12:28 -04:00
This is done by adding the block below for the respective plugin to the file [website/data/docs-remote-navigation.js](https://github.com/hashicorp/packer/blob/master/website/data/docs-remote-plugins.json).
2021-03-09 13:06:11 -05:00
```json
{
2021-03-30 17:12:28 -04:00
"title": "Scaffolding",
"path": "scaffolding",
"repo": "hashicorp/packer-plugin-scaffolding",
"version": "latest"
2021-03-09 13:06:11 -05:00
}
```
2021-03-30 17:12:28 -04:00
If a plugin maintainer wishes to only include a specific version of released docs then the `"version"` key in the above configuration should be set to a released version of the plugin. Otherwise it should be set to "latest".
2021-03-09 13:06:11 -05:00
</Tab>
2021-03-30 17:12:28 -04:00
<Tab heading="Manually generating docs.zip">
2021-03-09 13:06:11 -05:00
2021-03-30 17:12:28 -04:00
The documentation structure needed for Packer.io can be generated manually, by creating a zip file called `docs.zip` of the docs directory and included in the plugin release.
2021-03-09 13:06:11 -05:00
2021-03-30 17:12:28 -04:00
```/bin/bash
[[ -d docs/ ]] && zip -r docs.zip docs/
2021-03-09 13:06:11 -05:00
```
2021-03-30 17:12:28 -04:00
Once the first `docs.zip` file has been included into a release you will need to open a one time pull-request against [hashicorp/packer](https://github.com/hashicorp/packer) to register the plugin docs.
2021-03-09 13:06:11 -05:00
2021-03-30 17:12:28 -04:00
This is done by adding the block below for the respective plugin to the file [website/data/docs-remote-navigation.js](https://github.com/hashicorp/packer/blob/master/website/data/docs-remote-plugins.json).
2021-03-09 13:06:11 -05:00
```json
{
2021-03-30 17:12:28 -04:00
"title": "Scaffolding",
"path": "scaffolding",
"repo": "hashicorp/packer-plugin-scaffolding",
"version": "latest"
2021-03-09 13:06:11 -05:00
}
```
2021-03-30 17:12:28 -04:00
If a plugin maintainer wishes to only include a specific version of released docs then the `"version"` key in the above configuration should be set to a released version of the plugin. Otherwise it should be set to "latest".
2021-03-09 13:06:11 -05:00
</Tab>
</Tabs>
2021-02-15 05:10:43 -05:00
### Plugin Development Tips and FAQs
2013-06-11 00:47:16 -04:00
2021-03-11 05:48:23 -05:00
#### Working Examples
2021-03-11 19:42:55 -05:00
Here's a non exaustive list of Packer plugins that you can check out:
2021-03-11 05:48:23 -05:00
2021-03-31 15:07:00 -04:00
- [github.com/hashicorp/packer-plugin-docker](https://github.com/hashicorp/packer-plugin-docker)
- [github.com/exoscale/packer-plugin-exoscale](https://github.com/exoscale/packer-plugin-exoscale)
- [github.com/sylviamoss/packer-plugin-comment](https://github.com/sylviamoss/packer-plugin-comment)
2021-03-11 05:48:23 -05:00
Looking at their code will give you good examples.
2017-03-25 18:13:52 -04:00
#### Naming Conventions
2013-06-20 17:30:02 -04:00
2015-07-22 22:31:00 -04:00
It is standard practice to name the resulting plugin application in the format
2020-12-10 03:22:09 -05:00
of `packer-plugin-NAME`. For example, if you're building a new builder for
2015-07-22 22:31:00 -04:00
CustomCloud, it would be standard practice to name the resulting plugin
2021-02-15 05:10:43 -05:00
`packer-plugin-customcloud`. This naming convention helps users identify the
2020-12-10 03:22:09 -05:00
scope of a plugin.
2013-06-11 00:47:16 -04:00
2017-03-25 18:13:52 -04:00
#### Testing Plugins
2013-06-20 17:30:02 -04:00
2021-02-15 05:10:43 -05:00
Making your unpublished plugin available to Packer is possible by either:
2013-06-11 00:47:16 -04:00
2021-03-31 15:07:00 -04:00
- Starting Packer from the directory where the plugin binary is located.
- Putting the plugin binary in the same directory as Packer.
2021-02-02 12:05:04 -05:00
In both these cases, if the binary is called `packer-plugin-myawesomecloud` and
defines an `ebs` builder then you will be able to use an `myawesomecloud-ebs`
builder or source without needing to have a `required_plugin` block.
2013-06-11 00:47:16 -04:00
2021-02-02 12:05:04 -05:00
This is extremely useful during development.
2013-06-11 00:47:16 -04:00
2017-03-25 18:13:52 -04:00
#### Distributing Plugins
2013-06-20 17:30:02 -04:00
2021-02-15 05:10:43 -05:00
We recommend that you use a tool like the GoReleaser in order to cross-compile
2021-02-02 12:05:04 -05:00
your plugin for every platform that Packer supports, since Go applications are
platform-specific. If you have created your plugin from the
[packer-plugin-scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding)
2021-02-15 05:10:43 -05:00
repo, simply tagging a commit and pushing the tag to GitHub will correctly build
and release the binaries using GoReleaser.