Multi plugin naming (#10608)

This commit is contained in:
Megan Marsh 2021-02-15 02:10:43 -08:00 committed by GitHub
parent cac6b04546
commit bd7b31853e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 257 additions and 271 deletions

View File

@ -109,7 +109,7 @@ func checkPluginName(name string) error {
strings.HasPrefix(name, "packer-post-processor-") {
fmt.Printf("\n[WARNING] Plugin is named with old prefix `packer-[builder|provisioner|post-processor]-{name})`. " +
"These will be detected but Packer cannot install them automatically. " +
"The plugin must be a multi-plugin named packer-plugin-{name} to be installable through the `packer init` command.\n")
"The plugin must be a multi-component plugin named packer-plugin-{name} to be installable through the `packer init` command.\n")
return nil
}
return fmt.Errorf("plugin's name is not valid")

View File

@ -238,8 +238,8 @@ func (c *PluginConfig) discoverSingle(glob string) (map[string]string, error) {
return res, nil
}
// DiscoverMultiPlugin takes the description from a multiplugin binary and
// makes the plugins available to use in Packer. Each plugin found in the
// DiscoverMultiPlugin takes the description from a multi-component plugin
// binary and makes the plugins available to use in Packer. Each plugin found in the
// binary will be addressable using `${pluginName}-${builderName}` for example.
// pluginName could be manually set. It usually is a cloud name like amazon.
// pluginName can be extrapolated from the filename of the binary; so

View File

@ -184,7 +184,7 @@ func generateFakePlugins(dirname string, pluginNames []string) (string, []string
}
// TestHelperProcess isn't a real test. It's used as a helper process
// for multiplugin-binary tests.
// for multi-component plugin tests.
func TestHelperPlugins(t *testing.T) {
if os.Getenv("PKR_WANT_TEST_PLUGINS") != "1" {
return
@ -265,7 +265,7 @@ func helperCommand(t *testing.T, s ...string) []string {
}
func createMockPlugins(t *testing.T, plugins map[string]pluginsdk.Set) {
pluginDir, err := tmp.Dir("pkr-multiplugin-test-*")
pluginDir, err := tmp.Dir("pkr-multi-component-plugin-test-*")
{
// create an exectutable file with a `sh` sheebang
// this file will look like:
@ -275,7 +275,7 @@ func createMockPlugins(t *testing.T, plugins map[string]pluginsdk.Set) {
// $@ just passes all passed arguments
// This will allow to run the fake plugin from go tests which in turn
// will run go tests callback to `TestHelperPlugins`, this one will be
// transparently calling our mock multiplugins `mockPlugins`.
// transparently calling our mock multi-component plugins `mockPlugins`.
if err != nil {
t.Fatal(err)
}

View File

@ -10,9 +10,9 @@ sidebar_title: <tt>init</tt>
-> **Note:** Packer init does not work with legacy JSON templates. You can
upgrade your JSON config files to HCL using the [hcl2_upgrade](/docs/commands/hcl2_upgrade) command.
-> **Note:** Packer init will only work with multiplugins -- that is plugins
that are named `packer-plugin-*`. To install a single plugin binary -- that is
`packer-provisioner-*`, `packer-builder-*`, etc. -- nothing changes, you will
-> **Note:** Packer init will only work with multi-component plugins -- that is
plugins that are named `packer-plugin-*`. To install a single-component plugin --
that is `packer-provisioner-*`, `packer-builder-*`, etc. -- nothing changes, you will
have to [install the plugin manually](/docs/plugins#installing-plugins).
The `packer init` command is used to download Packer plugin binaries. This is

View File

@ -9,12 +9,14 @@ sidebar_title: Custom Builders
# Custom Builders
Packer Builders are the components of Packer responsible for creating a
machine, bringing it to a point where it can be provisioned, and then turning
that provisioned machine into a machine image. Several builders are
officially maintained and distributed by the HashiCorp Packer team -- these
builders include builders for creating images on Amazon EC2, VMWare, Google
Compute Engine, and many more. It is also possible to write custom builders
using the Packer plugin interface, and this page documents how to do that.
virtual machine, bringing it to a point where it can be provisioned, and then
turning that provisioned virtual machine into a machine image. Several builders
are officially maintained and distributed by the HashiCorp Packer team -- among
these are builders for creating images on Amazon EC2, VMWare, Google
Compute Engine, and many more. You can find documentation for how to use these
official builders [here](/docs/builders). It is also possible to write custom
builders using the Packer plugin interface, and this page documents how to do
that.
Prior to reading this page, you should read the page on [plugin development
basics](/docs/extending/plugins).

View File

@ -9,96 +9,83 @@ sidebar_title: Extending Packer
# Extending Packer
Packer is designed to be extensible. Because the surface area for workloads is
infinite, Packer supports plugins for builders, provisioners, and
post-processors. To learn more about the different customizations, please
choose a link from the sidebar.
Packer is designed to be extensible, and supports plugins that allow you to
create and use custom builders, provisioners, post-processors, and data sources.
To learn more about developing these different types of components, please
choose a link from the sidebar. To learn more about the general plugin
architecture, stay on this page.
## Developing Plugins
This page will document how you can develop your own Packer plugins. Prior to
reading this, it is assumed that you're comfortable with Packer and also know
the [basics of how Plugins work](/docs/plugins), from a user
standpoint.
reading this, you should be comfortable with Packer and know the
basics of [how plugins work from a user standpoint](/docs/plugins).
Packer plugins must be written in [Go](https://golang.org/), so it is also
assumed that you're familiar with the language. This page will not be a Go
language tutorial. Thankfully, if you are familiar with Go, the Go toolchain
provides many conveniences to help to develop Packer plugins.
Packer plugins must be written in [Go](https://golang.org/), so you should also
be familiar with the language.
~> **Warning!** This is an advanced topic. If you're new to Packer, we
recommend getting a bit more comfortable before you dive into writing plugins.
### Plugin System Architecture
Packer has a fairly unique plugin architecture. Instead of loading plugins
directly into a running application, Packer runs each plugin as a _separate
application_. Inter-process communication and RPC is then used to communicate
between the many running Packer processes. Packer core itself is responsible
for orchestrating the processes and handles cleanup.
The beauty of this is that your plugin can have any dependencies it wants.
Dependencies don't need to line up with what Packer core or any other plugin
uses, because they're completely isolated into the process space of the plugin
itself.
And, thanks to Go's
[interfaces](https://golang.org/doc/effective_go.html#interfaces_and_types), it
doesn't even look like inter-process communication is occurring. You just use
the interfaces like normal, but in fact they're being executed in a remote
process. Pretty cool.
A Packer plugin is just a go binary. Instead of loading plugins directly into a
running application, Packer runs each plugin as a _separate application_.
The multiple separate Packer plugin processes communicate with the Core using
an RPC defined in the packer-plugin SDK. The Packer core itself is responsible
launching and cleaning up the plugin processes.
### Plugin Development Basics
Developing a plugin allows you to create additional functionality for Packer.
All the various kinds of plugins have a corresponding interface. The plugin
needs to implement this interface and expose it using the Packer plugin package
(covered here shortly), and that's it!
The components that can be created and used in a Packer plugin are builders,
provisioners, post-processors, and data sources.
There are two packages that really matter that every plugin must use. Other
than the following two packages, you're encouraged to use whatever packages you
want. Because plugins are their own processes, there is no danger of colliding
dependencies.
Each of these components has a corresponding [interface](https://golang.org/doc/effective_go.html#interfaces_and_types).
- [`github.com/hashicorp/packer-plugin-sdk/packer`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer) - Contains all the interfaces that you have
to implement for any given plugin.
All you need to do to create a plugin is:
1. create an implementation of the desired interface, and
2. serve it using the server provided in the [packer-plugin-sdk](https://github.com/hashicorp/packer-plugin-sdk).
- [`github.com/hashicorp/packer-plugin-sdk/plugin`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin) - Contains the code to serve
the plugin. This handles all the inter-process communication stuff.
The core and the SDK handle all of the communication details inside the server.
There are two steps involved in creating a plugin:
Your plugin must use two packages from the SDK to implement the server and
interfaces. You're encouraged to use whatever other packages you want in your
plugin implementation. Because plugins are their own processes, there is no
danger of colliding dependencies.
1. Implement the desired interface. For example, if you're building a builder
plugin, implement the [packer.Builder](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer#Builder) interface.
- [`github.com/hashicorp/packer-plugin-sdk/packer`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer) - Contains all the interfaces that you have to implement for any given plugin.
2. Serve the interface by calling the appropriate [plugin](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin#Set.RegisterBuilder) serving method in
your main method.
Basic examples are shown below. Note that if you can define a multi-plugin
binary as it will allow to add more that one plugin per binary.
- [`github.com/hashicorp/packer-plugin-sdk/plugin`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin) - Contains the code to serve the plugin. This handles all the inter-process communication.
Basic examples of serving your component are shown below. Note that if you
define a multi-component plugin, you can (but do not need to) add more than one
component per plugin binary. The multi-component plugin is also compatible with
download and installation via `packer init`, whereas the single-component plugin
is not.
<Tabs>
<Tab heading="Multi-plugin Binary">
<Tab heading="Multi-component Plugin (recommended) ">
```go
// main.go
import (
"github.com/hashicorp/packer-plugin-sdk/plugin"
)
// Assume this implements packer.Builder
// Assume this implements the packer.Builder interface
type ExampleBuilder struct{}
// Assume this implements packer.PostProcessor
// Assume this implements the packer.PostProcessor interface
type FooPostProcessor struct{}
// Assume this implements packer.Provisioner
// Assume this implements the packer.Provisioner interface
type BarProvisioner struct{}
func main() {
pps := plugin.NewSet()
pps.RegisterBuilder("example", new(ExampleBuilder))
pps.RegisterBuilder(plugin.DEFAULT_NAME, new(AnotherBuilder))
pps.RegisterPostProcessor("foo", new(FooPostProcessor))
pps.RegisterProvisioner("bar", new(BarProvisioner))
err := pps.Run()
@ -109,26 +96,35 @@ func main() {
}
```
**That's it!** `plugin.NewSet` handles all the nitty gritty of
communicating with Packer core and serving your builder over RPC. It can't get
much easier than that.
This `plugin.NewSet` invocation handles all the details of communicating with
Packer core and serving your component over RPC. As long as your struct being
registered implements one of the component interfaces, Packer will now be able
to launch your plugin and use it.
Here the name of the plugin will be used to use each plugin, so if your plugin
is named `packer-plugin-my`, this would make the following parts available:
If you register a component with its own name, the component name will be
appended to the plugin name to create a unique name. If you register a component
using the special string constant `plugin.DEFAULT_NAME`, then the component will
be referenced by using only the plugin name. For example:
If your plugin is named `packer-plugin-my`, the above set definition would make
the following components available:
* the `my-example` builder
* the `my` builder
* the `my-foo` post-processor
* the `my-bar` provisioner
</Tab>
<Tab heading="Single Plugin Binary">
<Tab heading="Single Component Plugin (deprecated)">
```go
// main.go
import (
"github.com/hashicorp/packer-plugin-sdk/plugin"
)
// Assume this implements packer.Builder
// Assume this implements the packer.Builder interface
type Builder struct{}
func main() {
@ -141,17 +137,23 @@ func main() {
}
```
**That's it!** `plugin.Server` handles all the nitty gritty of communicating with
Packer core and serving your builder over RPC. It can't get much easier than
that.
This `server.Serve()` invocation handles all the details of communicating with
Packer core and serving your component over RPC. As long as your struct being
registered implements one of the component interfaces, Packer will now be able
to launch your plugin and use it.
Please note that single-component plugins exist for backwards-compatability. We
would rather you register your component using the multi-component method shown
in the other tab, even if you only have one component in your binary. This is
because the `packer init` command only supports multi-component plugins.
</Tab>
</Tabs>
Next, just build your plugin like a normal Go application, using `go build` or
however you please. The resulting binary is the plugin that can be installed
using standard installation procedures.
Next, build your plugin as you would any other Go application. The resulting
binary is the plugin that can be installed using
[standard installation procedures](/docs/plugins#installing-plugins).
The specifics of how to implement each type of interface are covered in the
relevant subsections available in the navigation to the left.
@ -184,47 +186,56 @@ the logs to be helpful.
### Creating a GitHub Release
To create a GitHub release with the assets that will be consumed by Packer, we provide a pre-defined release workflow configuration
using [GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions). We strongly encourage maintainers to use this configuration
to make sure the release contains the assets expected by Packer to load and use a plugin.
[GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions) allow you to execute workflows when events on your repository occur.
You can use this to create releases whenever a new version tag is created on your repository.
`packer init` does not work using a centralized registry. Instead, it requires
you to publish your plugin in a GitHub repo with the name
`packer-plugin-*` where * represents the name of your plugin. You also need to
create a GitHub release of your plugin with specific assets for the
`packer init` download to work. We provide a pre-defined release workflow
configuration using
[GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions). We
strongly encourage maintainers to use this configuration to make sure the
release contains the right assets with the right names for Packer to leverage
`packer init` installation.
Here's what you need to create releases using GitHub Actions:
1. Generate a GPG key to be used when signing releases (See [GitHub's detailed instructions](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/generating-a-new-gpg-key)
for help with this step)
2. Copy the [GoReleaser configuration from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.goreleaser.yml) to the root of your repository.
```sh
curl -L -o .goreleaser.yml \
https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.goreleaser.yml
```
3. Copy the [GitHub Actions workflow from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.github/workflows/release.yml) to `.github/workflows/release.yml` in your repository.
An easier way to execute steps 1 and 2, is by running the following command in the root folder of your plugin to download
and place both files in their expected place:
```shell
curl -L -o .goreleaser.yml https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.goreleaser.yml &&
mkdir -p .github/workflows &&
curl -L -o .github/workflows/release.yml https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.github/workflows/release.yml
```
4. Go to Settings > Secrets in your repository, and add the following secrets:
```sh
mkdir -p .github/workflows &&
curl -L -o .github/workflows/release.yml \
https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.github/workflows/release.yml
```
4. Go to your repository page on GitHub and navigate to Settings > Secrets. Add
the following secrets:
- `GPG_PRIVATE_KEY` - Your ASCII-armored GPG private key. You can export this with `gpg --armor --export-secret-keys [key ID or email]`.
- `PASSPHRASE` - The passphrase for your GPG private key.
5. Push a new valid version tag (e.g. `v1.2.3`) to test that the GitHub Actions releaser is working. The tag must be a valid [Semantic Version](https://semver.org/) preceded with a `v`.
5. Push a new valid version tag (e.g. `v1.2.3`) to test that the GitHub Actions
releaser is working. The tag must be a valid
[Semantic Version](https://semver.org/) preceded with a `v`. Once the tag is
pushed, the github actions you just configured will automatically build release
binaries that Packer can download using `packer init`. For more details on how
to install a plugin using `packer init`, see the
[init docs](/docs/commands/init).
### Plugin Development Tips
Here are some tips for developing plugins, often answering common questions or
concerns.
### Plugin Development Tips and FAQs
#### Naming Conventions
It is standard practice to name the resulting plugin application in the format
of `packer-plugin-NAME`. For example, if you're building a new builder for
CustomCloud, it would be standard practice to name the resulting plugin
`packer-plugin-custom-cloud`. This naming convention helps users identify the
`packer-plugin-customcloud`. This naming convention helps users identify the
scope of a plugin.
#### Testing Plugins
Making your unpublished plugin available to Packer is possible by either :
Making your unpublished plugin available to Packer is possible by either:
* Starting Packer from the directory where the plugin binary is located.
* Putting the plugin binary in the same directory as Packer.
@ -237,8 +248,9 @@ This is extremely useful during development.
#### Distributing Plugins
It is recommended you use a tool like the GoReleaser in order to cross-compile
We recommend that you use a tool like the GoReleaser in order to cross-compile
your plugin for every platform that Packer supports, since Go applications are
platform-specific. If you have created your plugin from the
[packer-plugin-scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding)
repo, simply tagging a commit will correctly build and release the binaries using GoReleaser.
repo, simply tagging a commit and pushing the tag to GitHub will correctly build
and release the binaries using GoReleaser.

View File

@ -10,54 +10,66 @@ sidebar_title: Packer Plugins
# Packer Plugins
Packer Plugins allow new functionality to be added to Packer without modifying
the core source code. Packer plugins are able to add new builders, provisioners,
hooks, and more. In fact, much of Packer itself is implemented by writing
plugins that are simply distributed with Packer. For example, many of the
builders, provisioners, and more that ship with Packer are implemented as
Plugins that are simply hardcoded to load with Packer.
the core source code. Packer plugins are able to add new components to Packer,
such as builders, provisioners, post-processors, and data sources.
* This page documents how to install plugins.
* Plugins cannot be configured yet.
* If you're interested in developing plugins, see the [developing
This page documents how to install plugins.
If you're interested in developing plugins, see the [developing
plugins](/docs/plugins/creation#developing-plugins) page.
The current official listing of available Packer plugins can be found
[here](/community-tools#third-party-plugins). These plugins will need to be
manually installed. This is an incomplete list, and more plugins can be found by
searching. Typically, searching "packer plugin _x_" will find what you're
looking for if it exists. We hope to create an offical registry for third party
plugins in the future.
The current official listing of community-written plugins can ve found
[here](/community-tools#third-party-plugins); if you have written a Packer
plugin, please reach out to us so we can add your plugin to the website.
## How Plugins Work
Packer plugins are completely separate, standalone applications that the core
of Packer starts and communicates with.
of Packer starts and communicates with. Even the components that ship with the
Packer core (core builders, provisioners, and post-processors) are implemented
in a similar way and run as though they are standalone plugins.
These plugin applications aren't meant to be run manually. Instead, Packer core
executes them as a sub-process, run as a sub-command (`packer plugin`) and
communicates with them. For example, the Shell provisioner is actually run as
`packer plugin packer-provisioner-shell`. The next time you run a Packer build,
launches and communicates with them. The next time you run a Packer build,
look at your process list and you should see a handful of `packer-` prefixed
applications running.
applications running. One of those applications is the core; the rest are
plugins -- one plugin process is launched for each component used in a Packer
build.
## Installing Plugins
<Tabs>
<Tab heading="with Packer init (from Packer v1.7.0)">
Currently, you do not need to install plugins for builder, provisioner, or
post-processor components documented on the Packer website; these components
ship with the Packer core and Packer automatically knows how to find and launch
them. These instructions are for installing custom components that are not
bundled with the Packer core.
~> **Note**: Only _multi-plugin binaries_ -- that is plugins named
The below tabs reference "multi-component" and "single-component" plugins. If
you are not sure what kind of plugin you are trying to install, the easiest way
to find out is to check the name. If the name starts with `packer-plugin-`, then
it is a multi-component plugin. If the name starts with a prefix that actually
says the component type (e.g. `packer-provisioner-` or `packer-builder`), then
it is a single-component plugin.
<Tabs>
<Tab heading="Packer init (from Packer v1.7.0)">
~> **Note**: Only _multi-component plugin binaries_ -- that is plugins named
packer-plugin-*, like the `packer-plugin-amazon` -- are expected to work with
Packer init. The legacy `builder`, `post-processor` and `provisioner` plugin
types will keep on being detected but Packer cannot install them automatically.
If a plugin you use has not been upgraded to use the multi-component plugin
architecture, contact your maintainer to request an upgrade.
## Installing Plugins
### Create a required_plugins block
1. Add a
[`required_plugin`](/docs/templates/hcl_templates/blocks/packer#specifying-plugin-requirements)
block to your `packer` block, each block will tell Packer what version(s) can be
installed, make sure to set a correct [version constraint
string](/docs/templates/hcl_templates/blocks/packer#version-constraints).
Example `required_plugins` block :
[`required_plugins`](/docs/templates/hcl_templates/blocks/packer#specifying-plugin-requirements)
block to your [packer block](/docs/templates/hcl_templates/blocks/packer). Each block will tell Packer what version(s) of a
particular plugin can be installed. Make sure to set a valid [version
constraint string](/docs/templates/hcl_templates/blocks/packer#version-constraints).
Here is an example `required_plugins` block:
```hcl
packer {
@ -66,18 +78,25 @@ string](/docs/templates/hcl_templates/blocks/packer#version-constraints).
version = ">= 2.7.0"
source = "azr/myawesomecloud"
}
happycloud = ">= 2.7.0"
happycloud = {
version = ">= 1.1.3"
source = "azr/happycloud"
}
}
}
```
2. Run [`packer init`](/docs/commands/init) to install all missing plugin binaries.
2. Run [`packer init`](/docs/commands/init) from your project directory (the
directory containing your Packer templates) to install all missing plugin
binaries. Given the above example, Packer will try to look for a GitHub
repository owned by user or organization `azr` named
`packer-plugin-myawesomecloud` and `packer-plugin-happycloud`.
## Names and Addresses
Each plugin has two identifiers:
* A `source` address, which is only used when requiring a plugin outside the HashiCorp domain.
* A unique **local name**, which is used everywhere else in a Packer
* A `source` address, which is only necessary when requiring a plugin outside the HashiCorp domain.
* A unique **local name**, which is used everywhere else in a Packer
configuration.
### Local Names
@ -85,18 +104,49 @@ Each plugin has two identifiers:
Local names allow you to access the components of a plugin and must be unique
per configuration.
Here using the previous example if the required plugin `azr/myawesomecloud`
defined an `ebs` builder, this builder will be made available by using a
`myawesomecloud-ebs` source block :
This is best explained using an example. In the above `required_plugins` block,
we declared the local name "myawesomecloud" for the plugin `azr/myawesomecloud`.
If the "myawesomecloud" plugin contains both an "ebs" builder and a "import"
post-processor, then the builder will be accessed in a source block by using:
```hcl
source "myawesomecloud-ebs" "example" {...}
source "myawesomecloud-ebs" "example" {
// builder configuration...
}
```
similarly, the import post-processor would be accessed by declaring the
post-processor block:
```hcl
post-processor "myawesomecloud-import" {
// post-processor configuration...
}
```
If we change the required_plugins block to use a different local name "foo":
```hcl
required_plugins {
foo = {
version = ">= 2.7.0"
source = "azr/myawesomecloud"
}
}
```
Then we'd instead access that builder using the source:
```hcl
source "foo-ebs" "example" {
// builder configuration...
}
```
## Source Addresses
A plugin's source address is its global identifier. It also specifies the
primary location where Packer can download it.
A plugin's source address is its global identifier. It also tells Packer where
to download it.
Source addresses consist of three parts delimited by slashes (`/`), as
follows:
@ -105,11 +155,13 @@ follows:
* **Hostname** (optional): The hostname of the location/service that
distributes the plugin. If omitted, this defaults to
`github.com`, we recommend explicitly setting the hostname.
`github.com`, we recommend explicitly setting the hostname. Currently, the
only valid "hostname" is github.com, but we plan to eventually support plugins
downloaded from other domains.
* **Namespace:** An organizational namespace within the specified host.
This often is the organization that publishes the plugin. If omitted, this
defaults to `hashicorp`, we recommend explicitly setting the namespace.
defaults to `hashicorp`. We recommend explicitly setting the namespace.
* **Type:** A short name for the platform or system the plugin manages. The
type is usually the plugin's preferred local name.
@ -117,14 +169,17 @@ follows:
For example, the fictional `myawesomecloud` plugin could belong to the
`hashicorp` namespace on `github.com`, so its `source` could be
`github.com/hashicorp/myawesomecloud`, `hashicorp/myawesomecloud` or
`myawesomecloud`.
`myawesomecloud`. Note: the actual _repository_ that myawesomecloud comes from
must always have the name format
`www.github.com/hashicorp/packer-plugin-myawesomecloud`, but the
`required_plugins` block omits the redundant `packer-plugin-` repository prefix
for brevity.
The source address with all three components given explicitly is called the
plugin's _fully-qualified address_. You will see fully-qualified address in
various outputs, like error messages, but in most cases a simplified display
version is used. This display version omits the source host when it is the
public registry, so you may see the shortened version `"myawesomecloud"` instead
of `"github.com/hashicorp/myawesomecloud"`.
version is used. Therefore you may see the shortened version `"myawesomecloud"`
instead of `"github.com/hashicorp/myawesomecloud"`.
-> **Note:** We recommend using explicit source addresses for all plugins.
@ -153,7 +208,7 @@ Packer will error if you set the `packer-plugin-` prefix in a `source`. This
will avoid conflicting with other plugins for other tools, like Terraform.
</Tab>
<Tab heading="manually">
<Tab heading="manually (multi-component plugin)">
The easiest way to manually install a plugin is to name it correctly, then place
it in the proper directory. To name a plugin correctly, make sure the binary is
@ -198,5 +253,8 @@ The valid types for plugins are:
- `provisioner` - A provisioner to install software on images created by a
builder.
</Tab>
<Tab heading="manually (single-component plugin)">
</Tab>
</Tabs>

View File

@ -14,85 +14,29 @@ and the Packer plugin SDK. Prior to version 1.7.0, the Packer core repository (h
The goal of the SDK is to clearly separate the Packer core from the Packer plugins; as a plugin maintainer, you should only have to import the SDK and not the core. The SDK will allow us to use semantic versioning to express the changes to the API that our maintainers are expected to use, and will help us keep a clearer promise to you about the location and functionality of our helper tools.
Packer is currently backwards compatible with the old API because the plugin API itself has not changed yet, so if you choose to continue using the old Packer imports, you will be able to do so until you need to upgrade the version of Packer you are importing to create your plugin.
### How to update plugins to use the packer-plugin-SDK
In a best-case scenario, all you'll have to do is update the packer imports to use the packer-plugin-sdk import path. Replacing `github.com/hashicorp/packer` with `github.com/hashicorp/packer-plugin-sdk`.
We have created a [packer-sdk-migrator](https://github.com/hashicorp/packer-sdk-migrator) cli tool to help you migrate your plugin to use the new import paths. To use it, follow the installation instructions for the migration tool, then call `packer-sdk-migrator migrate` from the root of your plugin directory. More details can be found in the migrator's [README](https://github.com/hashicorp/packer-sdk-migrator/blob/main/README.md).
To help with the migration of existing plugins the [packer-sdk-migrator](https://github.com/hashicorp/packer-sdk-migrator#packer-sdk-migrator) CLI tool is available to automatically rewrite all of the import paths to use to `packer-plugin-sdk` module. The migration CLI provides an [eligibility check](https://github.com/hashicorp/packer-sdk-migrator#packer-sdk-migrator-check-check-eligibility-for-migration) that can be used to determine if a plugin can be migrated with the tool.
Once you have migrated your plugin to the new SDK, your users can continue using your plugin as they always have, manually installing it in their `plugins` directories. However, Packer v1.7 also introduced the `packer init` command, and since you're here anyway it's a great time to upgrade your plugin so it can be found by `packer init`. Keep reading for more details!
## Upgrade Plugins to use the new multi-component RPC server
With the creation of the SDK the import paths may not map directly to the paths within Packer core, as they have been refactored to make it easier to discover and use helpful modules. Below are a few common import paths. For a full list of available imports see [Packer Plugin SDK Docs](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/)
### Why is there a new RPC server?
There are two main reasons we wrote the new server type.
| Old Path | New Path |
| ---------| -------- |
| github.com/hashicorp/packer/template/config | github.com/hashicorp/packer-plugin-sdk/template/config |
| github.com/hashicorp/packer/packer | github.com/hashicorp/packer-plugin-sdk/packer |
| github.com/hashicorp/packer/template/interpolate | github.com/hashicorp/packer-plugin-sdk/template/interpolate |
First, it enables multiple related components, for example a builder and a post-processor that share a hypervisor or cloud, to live together in the same plugin. This helps maintainers who are experts in a specific technology to focus on that technology without having to maintain several repositories that submodule the same common code. You can think of a multi-component plugin as being in some ways analogous to Terraform providers, in the sense that both frequently bundle common resources or components by hypervisor or cloud.
Here is a more comprehensive list of old imports and their new replacements:
| Old Path | New Path |
| ---------| -------- |
| github.com/hashicorp/packer/helper/builder/testing | github.com/hashicorp/packer-plugin-sdk/acctest |
| **new** | github.com/hashicorp/packer-plugin-sdk/acctest/provisioneracc
| github.com/hashicorp/packer/helper/tests | github.com/hashicorp/packer-plugin-sdk/acctest/testutils |
| github.com/hashicorp/packer/common/adapter | github.com/hashicorp/packer-plugin-sdk/adapter |
| github.com/hashicorp/packer/common/bootcommand | github.com/hashicorp/packer-plugin-sdk/bootcommand |
| github.com/hashicorp/packer/common/chroot | github.com/hashicorp/packer-plugin-sdk/chroot |
| github.com/hashicorp/packer/common | github.com/hashicorp/packer-plugin-sdk/common |
| github.com/hashicorp/packer/helper/communicator | github.com/hashicorp/packer-plugin-sdk/communicator |
| github.com/hashicorp/packer/helper/config | github.com/hashicorp/packer-plugin-sdk/template/config |
| github.com/hashicorp/packer/template/interpolate | github.com/hashicorp/packer-plugin-sdk/template/interpolate |
| github.com/hashicorp/packer/communicator/ssh | github.com/hashicorp/packer-plugin-sdk/communicator/ssh |
| github.com/hashicorp/packer/helper/communicator/sshkey | github.com/hashicorp/packer-plugin-sdk/communicator/sshkey |
| github.com/hashicorp/packer/provisioner | github.com/hashicorp/packer-plugin-sdk/guestexec |
| github.com/hashicorp/packer/common/json | github.com/hashicorp/packer-plugin-sdk/json |
| github.com/hashicorp/packer/helper/multistep | github.com/hashicorp/packer-plugin-sdk/multistep |
| steps from github.com/hashicorp/packer/common | github.com/hashicorp/packer-plugin-sdk/multistep/commonsteps |
| github.com/hashicorp/packer/common/net | github.com/hashicorp/packer-plugin-sdk/net |
| github.com/hashicorp/packer/packer | github.com/hashicorp/packer-plugin-sdk/packer |
| github.com/hashicorp/packer/builder | github.com/hashicorp/packer-plugin-sdk/packerbuilderdata |
| github.com/hashicorp/packer/packer | github.com/hashicorp/packer-plugin-sdk/pathing |
| github.com/hashicorp/packer/packer/plugin | github.com/hashicorp/packer-plugin-sdk/plugin |
| github.com/hashicorp/packer/common/random | github.com/hashicorp/packer-plugin-sdk/random |
| github.com/hashicorp/packer/common | github.com/hashicorp/packer-plugin-sdk/retry |
| github.com/hashicorp/packer/packer/rpc | github.com/hashicorp/packer-plugin-sdk/rpc |
| github.com/hashicorp/packer/communicator | github.com/hashicorp/packer-plugin-sdk/sdk-internals/communicator |
| github.com/hashicorp/packer/common/shell | github.com/hashicorp/packer-plugin-sdk/shell |
| github.com/hashicorp/packer/common/shell-local | github.com/hashicorp/packer-plugin-sdk/shell-local |
| github.com/hashicorp/packer/helper/builder/localexec | github.com/hashicorp/packer-plugin-sdk/shell-local/localexec |
| github.com/hashicorp/packer/common/shutdowncommand | github.com/hashicorp/packer-plugin-sdk/shutdowncommand |
| github.com/hashicorp/packer/template | github.com/hashicorp/packer-plugin-sdk/template |
| github.com/hashicorp/packer/packer/tmp | github.com/hashicorp/packer-plugin-sdk/tmp |
| github.com/hashicorp/packer/helper/useragent | github.com/hashicorp/packer-plugin-sdk/useragent |
| github.com/hashicorp/packer/common/uuid | github.com/hashicorp/packer-plugin-sdk/uuid |
| **new** | github.com/hashicorp/packer-plugin-sdk/version
When in doubt you can search the packer-plugin-sdk repo for the name of your imported structs or functions to see where they exist in the SDK now; we have not changed struct or function names. Or refer to the [Packer Plugin SDK Docs](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/) for help.
## Upgrade plugins to use new multi-plugin rpc server
### Why upgrade to the new rpc server?
The goal of this server is to enable multiple related plugins, for example plugins that share a hypervisor or cloud, to rely on shared libraries without having to submodule common code. The helps maintainers who are experts in a specific technology to focus on that technology without having to maintain several repositories. Having a single global plugin wrapper also helps the Packer core register, download, and install plugins. Finally, it allows more clarity in the plugin binary naming and name-usage within a template.
### Do I have to upgrade?
For now, you do not have to upgrade your plugin, but we recommend doing it. Packer will continue to understand how to discover and launch old-style plugins. However, you will not be able to register your plugin with HashiCorp until you have upgraded the plugin.
Second, the new server provides the Packer core with structured metadata that allows the new `packer init` feature to work. This data includes semantic versioning that we need to implement the new `required_plugins` block, which allows us to ensure that users are using the correct _version_ of their plugin for a given Packer template. If you don't upgrade, your users cannot take advantage of the `packer init` tooling.
### How to upgrade the plugin
We've created a scaffolding repository over at https://github.com/hashicorp/packer-plugin-scaffolding/
If you already have a working plugin, the [builder plugin scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/builder/scaffolding) should look pretty familiar. You'll notice that there are some changes to main.go.
We've created a [scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/) that you can either clone or use as a reference example. If you already have a working plugin, it should look pretty familiar. You'll notice, however, that there are some changes to main.go.
Previously, you may have had a main.go that looks something like:
```golang
```go
package main
import (
@ -112,13 +56,12 @@ func main() {
```
With this single-plugin binary you'd install it by putting it into the plugin directory with the name
`packer-provisioner-foo`.
With this single-component plugin binary you'd install it by putting it into the plugin directory with the name `packer-provisioner-foo`, and to access your provisioner in your Packer template, you would use the type `foo`.
In the new multi-plugin binary you'll want to use the NewSet() function to create a server.
To use the new multi-component server, you'll want to use the NewSet() function to create a server:
```golang
```go
package main
import (
@ -130,7 +73,7 @@ import (
func main() {
pps := plugin.NewSet()
pps.RegisterProvisioner("foo", new(CommentProvisioner))
pps.RegisterProvisioner(plugin.DEFAULT_NAME, new(MyProvisioner))
err := pps.Run()
if err != nil {
fmt.Fprintln(os.Stderr, err.Error())
@ -140,67 +83,37 @@ func main() {
```
The implementation is extremely similar, but now we use the "NewSet" function to create the plugin Server, and call Run() instead of Serve().
The implementation is similar, but now we use the "NewSet" function to create the component Server, and call Run() instead of Serve().
With the multi-plugin binary example, it is important to note that the name of the binary must be in the form of `packer-pluign-name`. Otherwise Packer will not register it as a multi-plugin, nor to download it.
You build it as you would any go binary:
```go
go build -o packer-plugin-bar
```
Then you'd install it by putting it into the plugin directory with the name `packer-plugin-name`.
Then you install it by putting it into the plugin directory with the name `packer-plugin-foo`, and reference it in your template using the type "foo".
With the multi-plugin binary you would then refer to your newly installed plugin as `bar-foo` within your Packer template. Whereas, in the single binary setup, you'd have used the name "bar" in your Packer template.
Notice that the new naming convention uses the generic name "plugin" as opposed to saying the specific component type like the old style did; `packer-pluign-name` instead of `packer-provisioner-name`. You need to respect this new convention when installing the plugin, or Packer will not register it as a multi-component plugin.
At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries provide one or more plugin types to Packer which depend on the same cloud or other APIs. See [registering multiple plugins](#registering-multiple-plugins) for details on how to register more than one plugin type.
What does that `plugin.DEFAULT_NAME` do? It tells the Packer core that in your template you'll be using the root plugin name instead of a compound name composed of the plugin name and the component name. If you are only serving a single component, you may want to use `plugin.DEFAULT_NAME` to prevent your templates from stuttering.
If your plugin providers a single plugin type please continue reading for instruction on using the Set architecture for single plugins.
Here's an example using a custom string instead of `plugin.DEFAULT_NAME`:
## Using the Set Architecture for Single Plugins
***THIS IS ON THE PACKER MASTER BRANCH AND WILL BE AVAILABLE IN V1.7.0***
The naming described above could be awkward for users who have given plugins singular names. For example, in a prior world, you may have created a provisioner saved as `packer-provisioner-foo` and accessed it in your template using `"type": "foo"`; Now, if you install a plugin named `packer-plugin-foo`, and register it using NewSet() and RegisterProvisioner() with the name "foo", you'll have to access it in your template using `"type": "foo-foo"` which stutters and breaks backwards compatibility.
To solve this we have a specialized string constant you can use to register your plugins instead:
instead of registering your plugin using
```golang
pps.RegisterProvisioner("foo", new(MyProvisioner))
```go
pps.RegisterProvisioner("bar", new(MyProvisioner))
```
you can register it using
When you do this, if you install your plugin as `packer-plugin-foo`, the Packer core expects you to access the provisioner in your template using the type `foo-bar`, where "foo" is the last portion of the plugin name as installed and "bar" is the name of the provisioner that you registered.
```golang
pps.RegisterProvisioner(plugin.DEFAULT_NAME, new(MyProvisioner))
```
## Registering multiple components
This will tell the Packer core that when consuming this plugin set, the given plugin should just use the root plugin name, rather than a compound name. Therefore, when accessing a plugin registered this way, you can again reference it in your Packer template using `"type": "foo"`.
## Registering multiple plugins
This is useful where "bar" is the name of the hypervisor or environment, and "foo" is the specific plugin type. Many Packer plugins already follow this naming convention. To use an example from the Packer core; if you were writing a custom VSphere plugin, you could now name the plugin `packer-plugin-vsphere`, and register different VSphere builder plugins using:
```golang
func main() {
pps := plugin.NewSet()
pps.RegisterBuilder("iso", new(ISOBuilder))
pps.RegisterBuilder("clone", new(CloneBuilder))
err := pps.Run()
if err != nil {
fmt.Fprintln(os.Stderr, err.Error())
os.Exit(1)
}
}
```
And then reference your plugin in the Packer template using "vsphere-iso" and "vsphere-clone" just like Packer currently does.
You can still use the `DEFAULT_NAME` constant with a set implementation; in this example, whatever you register with the DEFAULT_NAME will be referenced in the Packer template as "vsphere". You can only register one plugin per plugin set using the DEFAULT_NAME
The goal of this guide is to help you upgrade a single-component plugin to use the new SDK and plugin server; check out our [Plugin Development Basics](/docs/plugins/creation#plugin-development-basics) guide for details on how to add new components to your plugin.
## Distributing migrated plugins
Once a plugin has been migrated to use the `packer-plugin-sdk` it can be released as it normally would and used by Packer by [installing the plugin](/docs/plugins#installing-plugins) manually into the respective Packer plugin directory.
If however, you would like to take advantage of Packer's automated plugin installation process via `packer init` -- the preferred method of installation -- you will need to make the plugin available on GitHub and in a repository named after the multi-plugin `https://github.com/<yourorg>/packer-plugin-name`. See [Creating a GitHub Release](/docs/plugins/creation#creating-a-github-release) for details on the recommended practice for releasing Packer plugins on GitHub.
Once a plugin has been migrated to use the `packer-plugin-sdk` it can be released as it normally would and used by Packer by [installing the plugin](/docs/plugins#installing-plugins) manually into the Packer plugin directory. But this method will not allow your users to take advantage of the `packer init` command.
If you want Packer to be able to automatically install your plugin for your users via`packer init` -- the preferred method of installation -- you need to make the plugin available on GitHub in a repository named after the multi-component plugin. `https://github.com/<yourorg>/packer-plugin-name`. We recognize that this may require you to rename or fork your plugin repository, but we think that it is worth the inconvenience to reduce ambiguity in the `init` call.
See our documentation on [Creating a GitHub Release](/docs/plugins/creation#creating-a-github-release) for details on the recommended practice for releasing Packer plugins on GitHub.

View File

@ -7,10 +7,11 @@ description: Learn how to generate the HCL2 configuration of your component easi
# Auto Generate the HCL2 code of a plugin
From v1.5, Packer can be configured using HCL2. Because Packer has so many
builders, provisioner & post-processors, we relied on code generation to iterate
more easily. The good news is that you can benefit from this code generator to
get the HCL2 spec code of your component simply. It's a Go binary package and is
located in [`cmd/mapstructure-to-hcl2`](https://github.com/hashicorp/packer/tree/master/cmd/mapstructure-to-hcl2).
builders, provisioners, and post-processors, we created a on code generation
tool to add the HCL2-enabling code more easily. You can use this code generator
to create the HCL2 spec code of your custom plugin simply. It's a Go binary
package and is located in
[`cmd/mapstructure-to-hcl2`](https://github.com/hashicorp/packer/tree/master/cmd/mapstructure-to-hcl2).
Say you want to configure the `Config` struct of a `Builder` in a package
located in `my/example-plugin/config.go`. Here are some simple steps you can
@ -29,7 +30,7 @@ follow to make it HCL2 enabled:
- Make sure that all the nested structs of `Config` are also auto generated the
same way.
- Now we only need to make Builder implement the interface by adding the
- Now we only need to make your Builder implement the interface by adding the
following snippet:
```go