2020-03-24 16:17:34 +01:00
---
2023-06-02 21:41:36 -07:00
title_tag: "Unit Testing Pulumi Programs"
2020-03-24 16:17:34 +01:00
meta_desc: "Guide to unit testing of Pulumi programs: mock-based tests across Node.js, Python, Go, and .NET."
2023-05-15 15:25:28 -07:00
title: Unit testing
h1: Unit testing Pulumi programs
2023-06-08 16:15:52 -07:00
meta_image: /images/docs/meta-images/docs-meta.png
2020-03-24 16:17:34 +01:00
weight: 1
menu:
2023-05-15 15:25:28 -07:00
usingpulumi:
2020-03-24 16:17:34 +01:00
parent: testing
2023-05-15 15:25:28 -07:00
aliases:
- /docs/guides/testing/unit/
2020-03-24 16:17:34 +01:00
---
2022-05-03 21:23:32 -07:00
Pulumi programs are authored in a general-purpose language like TypeScript, Python, Go, C# or Java. The full power of each language is available, including access to tools and libraries for that runtime, including testing frameworks.
2020-03-25 17:53:22 +01:00
2021-03-14 07:14:21 -07:00
When running an update, your Pulumi program talks to the Pulumi CLI to orchestrate the deployment. The idea of _unit tests_ is to cut this communication channel and replace the engine with mocks. The mocks respond to the commands from within the same OS process and return dummy data for each call that your Pulumi program makes.
2020-03-25 17:53:22 +01:00
Because mocks don't execute any real work, unit tests run very fast. Also, they can be made deterministic because tests don't depend on the behavior of any external system.
## Get Started
2023-05-15 15:25:28 -07:00
Let's build a sample test suite. The example uses AWS resources, but the same capabilities and workflow apply to any Pulumi provider. To follow along, complete the [Get Started with AWS ](/docs/clouds/aws/get-started/ ) guide to set up a basic Pulumi program in your language of choice.
2020-03-25 17:53:22 +01:00
2023-09-12 12:49:40 +00:00
Note that unit tests are supported in all [existing Pulumi runtimes ](https://www.pulumi.com/docs/languages-sdks/ ).
2020-03-25 17:53:22 +01:00
## Sample Program
Throughout this guide, we are testing a program that creates a simple AWS EC2-based webserver. We want to develop unit tests to ensure:
- Instances have a Name tag.
- Instances must not use an inline `userData` script— we must use a virtual machine image.
- Instances must not have SSH open to the Internet.
Our starting code is loosely based on the [aws-js-webserver example ](https://github.com/pulumi/examples/tree/master/aws-js-webserver ):
2021-05-20 13:59:30 -07:00
{{< notes > }}
Choose a language below to adjust the contents of this guide. Your choice is applied throughout the guide.
{{< / notes > }}
2020-03-25 17:53:22 +01:00
{{< chooser language " typescript , python , go , csharp " / > }}
< div > < / div >
{{% choosable language "typescript" %}}
2021-03-14 07:14:21 -07:00
index.ts:
2020-03-25 17:53:22 +01:00
```typescript
import * as aws from "@pulumi/aws ";
export const group = new aws.ec2.SecurityGroup("web-secgrp", {
ingress: [
{ protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] },
{ protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
],
});
const userData = `#!/bin/bash echo "Hello, World!" > index.html nohup python -m SimpleHTTPServer 80 &` ;
export const server = new aws.ec2.Instance("web-server-www", {
instanceType: "t2.micro",
securityGroups: [ group.name ], // reference the group object above
ami: "ami-c55673a0", // AMI for us-east-2 (Ohio)
userData: userData, // start a simple webserver
});
```
{{% /choosable %}}
{{% choosable language "python" %}}
2021-03-14 07:14:21 -07:00
infra.py:
2020-03-25 17:53:22 +01:00
```python
import pulumi
from pulumi_aws import ec2
group = ec2.SecurityGroup('web-secgrp', ingress=[
{ "protocol": "tcp", "from_port": 22, "to_port": 22, "cidr_blocks": ["0.0.0.0/0"] },
{ "protocol": "tcp", "from_port": 80, "to_port": 80, "cidr_blocks": ["0.0.0.0/0"] },
])
user_data = '#!/bin/bash echo "Hello, World!" > index.html nohup python -m SimpleHTTPServer 80 & '
server = ec2.Instance('web-server-www;',
instance_type="t2.micro",
security_groups=[ group.name ], # reference the group object above
ami="ami-c55673a0", # AMI for us-east-2 (Ohio)
user_data=user_data) # start a simple web server
```
{{% /choosable %}}
{{% choosable language "go" %}}
2021-03-14 07:14:21 -07:00
main.go:
2020-03-25 17:53:22 +01:00
```go
package main
import (
2021-08-13 05:05:39 -07:00
"github.com/pulumi/pulumi-aws/sdk/v4/go/aws/ec2"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
2020-03-25 17:53:22 +01:00
)
type infrastructure struct {
2021-08-13 05:05:39 -07:00
group *ec2.SecurityGroup
server *ec2.Instance
2020-03-25 17:53:22 +01:00
}
func createInfrastructure(ctx *pulumi.Context) (*infrastructure, error) {
2021-08-13 05:05:39 -07:00
group, err := ec2.NewSecurityGroup(ctx, "web-secgrp", & ec2.SecurityGroupArgs{
Ingress: ec2.SecurityGroupIngressArray{
ec2.SecurityGroupIngressArgs{
Protocol: pulumi.String("tcp"),
FromPort: pulumi.Int(22),
ToPort: pulumi.Int(22),
CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
},
ec2.SecurityGroupIngressArgs{
Protocol: pulumi.String("tcp"),
FromPort: pulumi.Int(80),
ToPort: pulumi.Int(80),
CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
},
},
})
if err != nil {
return nil, err
}
const userData = `#!/bin/bash echo "Hello, World!" > index.html nohup python -m SimpleHTTPServer 80 &`
server, err := ec2.NewInstance(ctx, "web-server-www", & ec2.InstanceArgs{
InstanceType: pulumi.String("t2-micro"),
SecurityGroups: pulumi.StringArray{group.ID()}, // reference the group object above
Ami: pulumi.String("ami-c55673a0"), // AMI for us-east-2 (Ohio)
UserData: pulumi.String(userData), // start a simple web server
})
if err != nil {
return nil, err
}
return & infrastructure{
group: group,
server: server,
}, nil
2020-03-25 17:53:22 +01:00
}
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
2021-03-14 07:14:21 -07:00
WebserverStack.cs:
2020-03-25 17:53:22 +01:00
``` csharp
using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Ec2.Inputs;
public class WebserverStack : Stack
{
public WebserverStack()
{
var group = new SecurityGroup("web-secgrp", new SecurityGroupArgs
{
Ingress =
{
new SecurityGroupIngressArgs { Protocol = "tcp", FromPort = 22, ToPort = 22, CidrBlocks = { "0.0.0.0/0" } },
new SecurityGroupIngressArgs { Protocol = "tcp", FromPort = 80, ToPort = 80, CidrBlocks = { "0.0.0.0/0" } }
}
});
var userData = "#!/bin/bash echo \"Hello, World!\" > index.html nohup python -m SimpleHTTPServer 80 &";
var server = new Instance("web-server-www", new InstanceArgs
{
InstanceType = "t2.micro",
SecurityGroups = { group.Name }, // reference the group object above
Ami = "ami-c55673a0", // AMI for us-east-2 (Ohio)
UserData = userData // start a simple webserver
});
}
}
```
{{% /choosable %}}
This basic Pulumi program allocates a security group and an instance. Notice, however, that we are violating all three of the rules stated above— let's write some tests!
## Install the unit testing framework
You are free to use your favorite frameworks and libraries for writing unit tests and assertions.
{{% choosable language "typescript" %}}
This guide uses Mocha as the testing framework. [Install Mocha ](https://mochajs.org/#installation ) to your development environment.
2023-03-06 11:58:23 -08:00
```bash
npm install --global mocha
```
2020-03-25 17:53:22 +01:00
Then, install additional NPM modules to your program:
```bash
2023-09-12 12:49:40 +00:00
npm install mocha @types/mocha ts-node --global --save-dev
2020-03-25 17:53:22 +01:00
```
{{% /choosable %}}
{{% choosable language python %}}
We use the built-in [`unittest` ](https://docs.python.org/3/library/unittest.html ) framework, so no need to install anything.
{{% /choosable %}}
{{% choosable language go %}}
We use the built-in `go test` command, so no need to install anything.
{{% /choosable %}}
{{% choosable language "csharp" %}}
We use [NUnit ](https://nunit.org/ ) test framework to define and run the tests, [Moq ](https://github.com/moq/moq4 ) for mocks, and [FluentAssertions ](https://github.com/fluentassertions/fluentassertions ) for assertions.
Install the corresponding NuGet packages to your program:
```bash
dotnet add package NUnit
dotnet add package NUnit3TestAdapter
dotnet add package Moq
dotnet add package FluentAssertions
2023-09-12 14:39:33 +00:00
dotnet add package Microsoft.NET.Test.Sdk
2020-03-25 17:53:22 +01:00
```
{{% /choosable %}}
## Add Mocks
Let's add the following code to mock the external calls to the Pulumi CLI.
{{% choosable language "typescript" %}}
2021-03-14 07:14:21 -07:00
ec2tests.ts:
2020-03-25 17:53:22 +01:00
```ts
import * as pulumi from "@pulumi/pulumi ";
pulumi.runtime.setMocks({
2021-04-19 23:59:43 +01:00
newResource: function(args: pulumi.runtime.MockResourceArgs): {id: string, state: any} {
2020-03-25 17:53:22 +01:00
return {
2021-04-19 23:59:43 +01:00
id: args.inputs.name + "_id",
state: args.inputs,
2020-03-25 17:53:22 +01:00
};
},
2022-02-15 16:15:16 -08:00
call: function(args: pulumi.runtime.MockCallArgs) {
2021-04-19 23:59:43 +01:00
return args.inputs;
2020-03-25 17:53:22 +01:00
},
unit testing guide: show how to set flag `dryRun` (#1642)
* Add PULUMI_TEST_MODE to guides/testing/unit
I was pulling my hair out trying to figure out why `pulumi.isDryRun`
was returning `false` when I was running my unit tests, given that
[the docs on the method][1] say:
> Note that we always consider executions in test mode to be
> “dry-runs”, since we will never actually carry out an update, and
> therefore certain output properties will never be resolved.
I’m still not sure if/where there is any canonical documentation on
“test mode” but I feel that it should probably always be enabled when
running unit tests. I’d prefer that there be a programmatic/imperative
way to enable it, e.g. `runtime.setDryRun(true)` or something similar,
but in lieu of that I feel the examples should include setting the env
var so as to plant seeds of awareness about this feature and how it
works.
[1]: https://www.pulumi.com/docs/reference/pkg/nodejs/pulumi/pulumi/runtime/#isDryRun
* Revert "Add PULUMI_TEST_MODE to guides/testing/unit"
f4adeec976811d64fe066b5084a9eeaf91b63814
As per discussion in pulumi/pulumi-hugo#1642
* guides/testing/unit: show how to set dryRun flag
Which, as per the discussion in pulumi/pulumi-hugo#1642, should
really/probably be renamed to `isPreview` or something along those
lines.
Anyway, it looks like the C# example already includes passing a value
for this flag, and I don’t know enough Go to add it to the Go example.
Now that I’m writing this commit message, the situation with `dryRun`
is confusing enough (especially given the docs on `runtime.isDryRun`)
that it could probably merit adding a small dedicated section to this
page. I’ll ask in the PR whether that’d be desirable.
* guides/testing/unit.md: better description of a flag
Co-authored-by: Ian Wahbe <ian@wahbe.com>
Co-authored-by: Ian Wahbe <ian@wahbe.com>
2022-06-27 13:44:02 -04:00
},
"project",
"stack",
false, // Sets the flag `dryRun` , which indicates if pulumi is running in preview mode.
);
2020-03-25 17:53:22 +01:00
```
{{% /choosable %}}
{{% choosable language python %}}
2021-03-14 07:14:21 -07:00
test_ec2.py:
2020-03-25 17:53:22 +01:00
```python
import pulumi
class MyMocks(pulumi.runtime.Mocks):
2021-04-19 23:59:43 +01:00
def new_resource(self, args: pulumi.runtime.MockResourceArgs):
return [args.name + '_id', args.inputs]
def call(self, args: pulumi.runtime.MockCallArgs):
2020-03-25 17:53:22 +01:00
return {}
unit testing guide: show how to set flag `dryRun` (#1642)
* Add PULUMI_TEST_MODE to guides/testing/unit
I was pulling my hair out trying to figure out why `pulumi.isDryRun`
was returning `false` when I was running my unit tests, given that
[the docs on the method][1] say:
> Note that we always consider executions in test mode to be
> “dry-runs”, since we will never actually carry out an update, and
> therefore certain output properties will never be resolved.
I’m still not sure if/where there is any canonical documentation on
“test mode” but I feel that it should probably always be enabled when
running unit tests. I’d prefer that there be a programmatic/imperative
way to enable it, e.g. `runtime.setDryRun(true)` or something similar,
but in lieu of that I feel the examples should include setting the env
var so as to plant seeds of awareness about this feature and how it
works.
[1]: https://www.pulumi.com/docs/reference/pkg/nodejs/pulumi/pulumi/runtime/#isDryRun
* Revert "Add PULUMI_TEST_MODE to guides/testing/unit"
f4adeec976811d64fe066b5084a9eeaf91b63814
As per discussion in pulumi/pulumi-hugo#1642
* guides/testing/unit: show how to set dryRun flag
Which, as per the discussion in pulumi/pulumi-hugo#1642, should
really/probably be renamed to `isPreview` or something along those
lines.
Anyway, it looks like the C# example already includes passing a value
for this flag, and I don’t know enough Go to add it to the Go example.
Now that I’m writing this commit message, the situation with `dryRun`
is confusing enough (especially given the docs on `runtime.isDryRun`)
that it could probably merit adding a small dedicated section to this
page. I’ll ask in the PR whether that’d be desirable.
* guides/testing/unit.md: better description of a flag
Co-authored-by: Ian Wahbe <ian@wahbe.com>
Co-authored-by: Ian Wahbe <ian@wahbe.com>
2022-06-27 13:44:02 -04:00
pulumi.runtime.set_mocks(
MyMocks(),
preview=False, # Sets the flag `dry_run` , which is true at runtime during a preview.
)
2020-03-25 17:53:22 +01:00
```
{{% /choosable %}}
{{% choosable language go %}}
2021-03-14 07:14:21 -07:00
main_test.go:
2020-03-25 17:53:22 +01:00
```go
import (
2021-08-13 05:05:39 -07:00
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
2020-03-25 17:53:22 +01:00
)
type mocks int
2021-08-13 05:05:39 -07:00
func (mocks) NewResource(args pulumi.MockResourceArgs) (string, resource.PropertyMap, error) {
2021-04-19 23:59:43 +01:00
return args.Name + "_id", args.Inputs, nil
2020-03-25 17:53:22 +01:00
}
2021-08-13 05:05:39 -07:00
func (mocks) Call(args pulumi.MockCallArgs) (resource.PropertyMap, error) {
2021-04-26 16:39:16 -05:00
return args.Args, nil
2020-03-25 17:53:22 +01:00
}
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
2021-03-14 07:14:21 -07:00
Testing.cs:
2020-03-25 17:53:22 +01:00
```csharp
2023-09-12 14:39:33 +00:00
using System.Collections.Immutable;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Testing;
namespace UnitTesting
2020-03-25 17:53:22 +01:00
{
2023-09-12 14:39:33 +00:00
class Mocks : IMocks
2020-03-25 17:53:22 +01:00
{
2023-09-12 14:39:33 +00:00
public Task< (string? id, object state)> NewResourceAsync(MockResourceArgs args)
{
var outputs = ImmutableDictionary.CreateBuilder< string , object > ();
outputs.AddRange(args.Inputs);
if (args.Type == "aws:ec2/instance:Instance")
{
outputs.Add("publicIp", "203.0.113.12");
outputs.Add("publicDns", "ec2-203-0-113-12.compute-1.amazonaws.com");
}
args.Id ??= $"{args.Name}_id";
return Task.FromResult< (string? id, object state)>((args.Id, (object)outputs));
}
public Task< object > CallAsync(MockCallArgs args)
{
var outputs = ImmutableDictionary.CreateBuilder< string , object > ();
if (args.Token == "aws:index/getAmi:getAmi")
{
outputs.Add("architecture", "x86_64");
outputs.Add("id", "ami-0eb1f3cdeeb8eed2a");
}
return Task.FromResult((object)outputs);
}
}
public static class Testing
{
public static Task< ImmutableArray < Resource > > RunAsync< T > () where T : Stack, new()
{
return Deployment.TestAsync< T > (new Mocks(), new TestOptions { IsPreview = false });
}
public static Task< T > GetValueAsync< T > (this Output< T > output)
{
var tcs = new TaskCompletionSource< T > ();
output.Apply(v =>
{
tcs.SetResult(v);
return v;
});
return tcs.Task;
}
2020-03-25 17:53:22 +01:00
}
}
```
{{% /choosable %}}
2023-09-12 12:49:40 +00:00
The definition of the mocks interface is available at the [runtime API reference page ](https://www.pulumi.com/docs/reference/pkg/nodejs/pulumi/pulumi/runtime/#Mocks ).
2020-04-02 12:49:32 +02:00
2020-03-25 17:53:22 +01:00
## Write the Tests
{{% choosable language "typescript" %}}
The overall structure and scaffolding of our tests will look like any ordinary Mocha testing:
2021-03-14 07:14:21 -07:00
ec2tests.ts:
2020-03-25 17:53:22 +01:00
```typescript
import * as pulumi from "@pulumi/pulumi ";
import "mocha";
pulumi.runtime.setMocks({
// ... mocks as shown above
});
describe("Infrastructure", function() {
2023-09-12 12:49:40 +00:00
let infra: typeof import("./index");
2020-05-14 03:04:10 +02:00
before(async function() {
// It's important to import the program _after_ the mocks are defined.
2023-09-12 12:49:40 +00:00
infra = await import("./index");
2020-05-14 03:04:10 +02:00
})
2020-03-25 17:53:22 +01:00
describe("#server ", function() {
// TODO(check 1): Instances have a Name tag.
// TODO(check 2): Instances must not use an inline userData script.
});
describe("#group ", function() {
// TODO(check 3): Instances must not have SSH open to the Internet.
});
});
```
{{% /choosable %}}
{{% choosable language "python" %}}
The overall structure and scaffolding of our tests will look like any ordinary Python's unittest testing:
2021-03-14 07:14:21 -07:00
test_ec2.py:
2020-03-25 17:53:22 +01:00
```python
import unittest
import pulumi
# ... MyMocks as shown above
pulumi.runtime.set_mocks(MyMocks())
# It's important to import `infra` _after_ the mocks are defined.
import infra
class TestingWithMocks(unittest.TestCase):
# TODO(check 1): Instances have a Name tag.
# TODO(check 2): Instances must not use an inline userData script.
# TODO(check 3): Instances must not have SSH open to the Internet.
```
{{% /choosable %}}
{{% choosable language "go" %}}
The overall structure and scaffolding of our tests will look like any ordinary Go test:
2021-03-14 07:14:21 -07:00
main_test.go:
2020-03-25 17:53:22 +01:00
```go
package main
import (
2020-06-17 14:48:11 +03:00
"sync"
"testing"
2020-03-25 17:53:22 +01:00
2021-04-19 23:59:43 +01:00
"github.com/pulumi/pulumi-aws/sdk/v4/go/aws/ec2"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
2020-06-17 14:48:11 +03:00
"github.com/stretchr/testify/assert"
2020-03-25 17:53:22 +01:00
)
// ... mocks as shown above
func TestInfrastructure(t *testing.T) {
2021-08-13 05:05:39 -07:00
err := pulumi.RunErr(func(ctx *pulumi.Context) error {
infra, err := createInfrastructure(ctx)
assert.NoError(t, err)
2020-03-25 17:53:22 +01:00
2021-08-13 05:05:39 -07:00
var wg sync.WaitGroup
wg.Add(3)
2020-03-25 17:53:22 +01:00
2021-08-13 05:05:39 -07:00
// TODO(check 1): Instances have a Name tag.
// TODO(check 2): Instances must not use an inline userData script.
// TODO(check 3): Instances must not have SSH open to the Internet.
2020-03-25 17:53:22 +01:00
2021-08-13 05:05:39 -07:00
wg.Wait()
return nil
}, pulumi.WithMocks("project", "stack", mocks(0)))
assert.NoError(t, err)
2020-03-25 17:53:22 +01:00
}
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
The overall structure and scaffolding of our tests will look like any ordinary NUnit testing:
2021-03-14 07:14:21 -07:00
WebserverStackTests.cs:
2020-03-25 17:53:22 +01:00
```csharp
2023-09-12 14:39:33 +00:00
using System.Linq;
using System.Threading.Tasks;
using FluentAssertions;
2020-03-25 17:53:22 +01:00
using NUnit.Framework;
2023-09-12 14:39:33 +00:00
using Pulumi.Aws.Ec2;
2020-03-25 17:53:22 +01:00
namespace UnitTesting
{
[TestFixture]
public class WebserverStackTests
{
// TODO(check 1): Instances have a Name tag.
// TODO(check 2): Instances must not use an inline userData script.
// TODO(check 3): Instances must not have SSH open to the Internet.
}
}
```
{{% /choosable %}}
Now let's implement our first test: ensuring that instances have a `Name` tag. To verify this we need to grab hold of the EC2 instance object, and check the relevant property:
{{% choosable language "typescript" %}}
```typescript
// check 1: Instances have a Name tag.
it("must have a name tag", function(done) {
2021-03-04 14:08:09 -08:00
pulumi.all([infra.server.urn, infra.server.tags]).apply(([urn, tags]) => {
2020-03-25 17:53:22 +01:00
if (!tags || !tags["Name"]) {
done(new Error(`Missing a name tag on server ${urn}` ));
} else {
done();
}
});
});
```
{{% /choosable %}}
{{% choosable language "python" %}}
```python
# check 1: Instances have a Name tag.
@pulumi .runtime.test
def test_server_tags(self):
def check_tags(args):
urn, tags = args
self.assertIsNotNone(tags, f'server {urn} must have tags')
self.assertIn('Name', tags, 'server {urn} must have a name tag')
return pulumi.Output.all(infra.server.urn, infra.server.tags).apply(check_tags)
```
{{% /choosable %}}
{{% choosable language "go" %}}
```go
// check 1: Instances have a Name tag.
pulumi.All(infra.server.URN(), infra.server.Tags).ApplyT(func(all []interface{}) error {
2021-08-13 05:05:39 -07:00
urn := all[0].(pulumi.URN)
2023-09-12 13:41:41 +00:00
tags := all[1].(map[string]string)
2020-03-25 17:53:22 +01:00
2021-08-13 05:05:39 -07:00
assert.Containsf(t, tags, "Name", "missing a Name tag on server %v", urn)
wg.Done()
return nil
2020-03-25 17:53:22 +01:00
})
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
```csharp
// check 1: Instances have a Name tag.
[Test]
public async Task InstanceHasNameTag()
{
var resources = await Testing.RunAsync< WebserverStack > ();
var instance = resources.OfType< Instance > ().FirstOrDefault();
instance.Should().NotBeNull("EC2 Instance not found");
2023-09-12 14:39:33 +00:00
var tags = await instance.Tags.GetValueAsync();
2020-03-25 17:53:22 +01:00
tags.Should().NotBeNull("Tags are not defined");
tags.Should().ContainKey("Name");
}
```
{{% /choosable %}}
This looks like a normal test, with a few noteworthy pieces:
- Since we're querying resource state without doing a deployment, there are many properties whose values will be undefined. This includes any output properties computed by your cloud provider that you did not explicitly return from the mocks. That's fine for these tests— we're checking for valid inputs anyway.
2023-09-12 14:45:17 +00:00
- Because all Pulumi resource properties are [outputs ](/docs/concepts/inputs-outputs/ )— since many of them are computed asynchronously— we need to use the `apply` method to get access to the values (see the `GetValueAsync` function in the `Testing.cs` file).
2020-03-25 17:53:22 +01:00
- Finally, since these outputs are resolved asynchronously, we need to use the framework's built-in asynchronous test capability.
After we've gotten through that setup, we get access to the raw inputs as plain values. The tags property is a map, so we make sure it is (1) defined, and (2) not missing an entry for the `Name` key. This is very basic, but we can check anything!
Now let's write our second check to assert that `userdata` property is empty:
{{% choosable language "typescript" %}}
```typescript
// check 2: Instances must not use an inline userData script.
it("must not use userData (use an AMI instead)", function(done) {
2021-03-04 14:08:09 -08:00
pulumi.all([infra.server.urn, infra.server.userData]).apply(([urn, userData]) => {
2020-03-25 17:53:22 +01:00
if (userData) {
done(new Error(`Illegal use of userData on server ${urn}` ));
} else {
done();
}
});
});
```
{{% /choosable %}}
{{% choosable language "python" %}}
```python
# check 2: Instances must not use an inline userData script.
@pulumi .runtime.test
def test_server_userdata(self):
def check_user_data(args):
urn, user_data = args
self.assertFalse(user_data, f'illegal use of user_data on server {urn}')
return pulumi.Output.all(infra.server.urn, infra.server.user_data).apply(check_user_data)
```
{{% /choosable %}}
{{% choosable language "go" %}}
```go
// check 2: Instances must not use an inline userData script.
pulumi.All(infra.server.URN(), infra.server.UserData).ApplyT(func(all []interface{}) error {
2021-08-13 05:05:39 -07:00
urn := all[0].(pulumi.URN)
2023-09-12 13:41:41 +00:00
userData := all[1].(string)
2020-03-25 17:53:22 +01:00
2023-09-12 13:41:41 +00:00
assert.Emptyf(t, userData, "illegal use of userData on server %v", urn)
2021-08-13 05:05:39 -07:00
wg.Done()
return nil
2020-03-25 17:53:22 +01:00
})
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
```csharp
// check 2: Instances must not use an inline userData script.
[Test]
public async Task InstanceMustNotUseInlineUserData()
{
var resources = await Testing.RunAsync< WebserverStack > ();
var instance = resources.OfType< Instance > ().FirstOrDefault();
instance.Should().NotBeNull("EC2 Instance not found");
2023-09-12 14:39:33 +00:00
var tags = await instance.UserData.GetValueAsync();
2020-03-25 17:53:22 +01:00
tags.Should().BeNull();
}
```
{{% /choosable %}}
And finally, let's write our third check. It’ s a bit more complex because we're searching for ingress rules associated with a security group— of which there may be many— and CIDR blocks within those ingress rules— of which there may also be many. But it's still several lines of code:
{{% choosable language "typescript" %}}
```typescript
// check 3: Instances must not have SSH open to the Internet.
it("must not open port 22 (SSH) to the Internet", function(done) {
2021-03-04 14:08:09 -08:00
pulumi.all([infra.group.urn, infra.group.ingress]).apply(([ urn, ingress ]) => {
2020-03-25 17:53:22 +01:00
if (ingress.find(rule =>
rule.fromPort === 22 & & (rule.cidrBlocks || []).find(block => block === "0.0.0.0/0"))) {
done(new Error(`Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group ${urn}` ));
} else {
done();
}
});
});
```
{{% /choosable %}}
{{% choosable language "python" %}}
```python
# check 3: Test if port 22 for ssh is exposed.
@pulumi .runtime.test
def test_security_group_rules(self):
def check_security_group_rules(args):
urn, ingress = args
ssh_open = any([rule['from_port'] == 22 and any([block == "0.0.0.0/0" for block in rule['cidr_blocks']]) for rule in ingress])
self.assertFalse(ssh_open, f'security group {urn} exposes port 22 to the Internet (CIDR 0.0.0.0/0)')
return pulumi.Output.all(infra.group.urn, infra.group.ingress).apply(check_security_group_rules)
```
{{% /choosable %}}
{{% choosable language "go" %}}
```go
// check 3: Test if port 22 for ssh is exposed.
pulumi.All(infra.group.URN(), infra.group.Ingress).ApplyT(func(all []interface{}) error {
2021-08-13 05:05:39 -07:00
urn := all[0].(pulumi.URN)
ingress := all[1].([]ec2.SecurityGroupIngress)
for _, i := range ingress {
openToInternet := false
for _, b := range i.CidrBlocks {
if b == "0.0.0.0/0" {
openToInternet = true
break
}
}
assert.Falsef(t, i.FromPort == 22 & & openToInternet, "illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group %v", urn)
}
wg.Done()
return nil
2020-03-25 17:53:22 +01:00
})
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
```csharp
// check 3: Test if port 22 for ssh is exposed.
[Test]
public async Task SecurityGroupMustNotHaveSshPortsOpenToInternet()
{
var resources = await Testing.RunAsync< WebserverStack > ();
foreach (var securityGroup in resources.OfType< SecurityGroup > ())
{
2023-09-12 14:39:33 +00:00
var urn = await securityGroup.Urn.GetValueAsync();
var ingress = await securityGroup.Ingress.GetValueAsync();
2020-03-25 17:53:22 +01:00
foreach (var rule in ingress)
{
(rule.FromPort == 22 & & rule.CidrBlocks.Any(b => b == "0.0.0.0/0"))
.Should().BeFalse($"Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group {urn}");
}
}
}
```
{{% /choosable %}}
That's it— now let's run the tests.
## Run the Tests
{{% choosable language "typescript" %}}
The command line to run your Mocha tests would therefore be:
```bash
2020-04-01 11:03:53 +02:00
$ mocha -r ts-node/register ec2tests.ts
2020-03-25 17:53:22 +01:00
```
{{% /choosable %}}
{{% choosable language "python" %}}
Run the following command to execute your Python tests:
```bash
$ python -m unittest
```
{{% /choosable %}}
{{% choosable language "go" %}}
Run the following command to execute your Go tests:
```bash
$ go test
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
Run the following command to execute your Python tests:
```bash
$ dotnet test
```
{{% /choosable %}}
Running this will tell us that we have three failing tests, as we had planned.
{{% choosable language "typescript" %}}
```bash
Infrastructure
#server
1) must have a name tag
2) must not use userData (use an AMI instead)
#group
3) must not open port 22 (SSH) to the Internet
0 passing (454ms)
3 failing
```
{{% /choosable %}}
{{% choosable language "python" %}}
```bash
======================================================================
FAIL: test_security_group_rules (test_ec2.TestingWithMocks)
----------------------------------------------------------------------
...
======================================================================
FAIL: test_server_tags (test_ec2.TestingWithMocks)
----------------------------------------------------------------------
...
======================================================================
FAIL: test_server_userdata (test_ec2.TestingWithMocks)
----------------------------------------------------------------------
...
----------------------------------------------------------------------
Ran 3 tests in 0.034s
FAILED (failures=3)
```
{{% /choosable %}}
{{% choosable language "go" %}}
```bash
--- FAIL: TestInfrastructure (0.00s)
...
Error: Should be false
Test: TestInfrastructure
Messages: illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group urn:pulumi:stack::project::aws:ec2/securityGroup:SecurityGroup::web-secgrp
...
Error: Expected nil, but got: (*string)(0xc000217390)
Test: TestInfrastructure
Messages: illegal use of userData on server urn:pulumi:stack::project::aws:ec2/instance:Instance::web-server-www
...
Error: "map[]" does not contain "Name"
Test: TestInfrastructure
Messages: missing a Name tag on server urn:pulumi:stack::project::aws:ec2/instance:Instance::web-server-www
FAIL testing-unit-go 0.501s
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
```bash
X InstanceHasNameTag [387ms]
Error Message:
Expected tags not to be < null > because Tags are not defined.
X InstanceMustNotUseInlineUserData [17ms]
Error Message:
Expected tags to be < null > , but found "#!/bin/bash echo "Hello, World!" > index.html nohup python -m SimpleHTTPServer 80 & ".
X SecurityGroupMustNotHaveSshPortsOpenToInternet [11ms]
Error Message:
Expected boolean to be false because Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group urn:pulumi:stack::project::pulumi:pulumi:Stack$aws:ec2/securityGroup:SecurityGroup::web-secgrp, but found True.
Test Run Failed.
Total tests: 3
Failed: 3
```
{{% /choosable %}}
Let's fix our program to comply:
{{% choosable language "typescript" %}}
2021-03-14 07:14:21 -07:00
index.ts:
2020-03-25 17:53:22 +01:00
```typescript
import * as aws from "@pulumi/aws ";
export const group = new aws.ec2.SecurityGroup("web-secgrp", {
ingress: [
{ protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
],
});
export const server = new aws.ec2.Instance("web-server-www", {
instanceType: "t2.micro",
securityGroups: [ group.name ], // reference the group object above
ami: "ami-c55673a0", // AMI for us-east-2 (Ohio)
tags: { Name: "www-server" }, // name tag
});
```
{{% /choosable %}}
{{% choosable language "python" %}}
2021-03-14 07:14:21 -07:00
infra.py:
2020-03-25 17:53:22 +01:00
```python
import pulumi
from pulumi_aws import ec2
group = ec2.SecurityGroup('web-secgrp', ingress=[
{ "protocol": "tcp", "from_port": 80, "to_port": 80, "cidr_blocks": ["0.0.0.0/0"] },
])
server = ec2.Instance('web-server-www;',
instance_type="t2.micro",
security_groups=[ group.name ], # reference the group object above
tags={'Name': 'webserver'}, # name tag
ami="ami-c55673a0") # AMI for us-east-2 (Ohio)
```
{{% /choosable %}}
{{% choosable language "go" %}}
2021-03-14 07:14:21 -07:00
main.go:
2020-03-25 17:53:22 +01:00
```go
package main
import (
2021-08-13 05:05:39 -07:00
"github.com/pulumi/pulumi-aws/sdk/v4/go/aws/ec2"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
2020-03-25 17:53:22 +01:00
)
type infrastructure struct {
2021-08-13 05:05:39 -07:00
group *ec2.SecurityGroup
server *ec2.Instance
2020-03-25 17:53:22 +01:00
}
func createInfrastructure(ctx *pulumi.Context) (*infrastructure, error) {
2021-08-13 05:05:39 -07:00
group, err := ec2.NewSecurityGroup(ctx, "web-secgrp", & ec2.SecurityGroupArgs{
Ingress: ec2.SecurityGroupIngressArray{
ec2.SecurityGroupIngressArgs{
Protocol: pulumi.String("tcp"),
FromPort: pulumi.Int(80),
ToPort: pulumi.Int(80),
CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
},
},
})
if err != nil {
return nil, err
}
server, err := ec2.NewInstance(ctx, "web-server-www", & ec2.InstanceArgs{
InstanceType: pulumi.String("t2-micro"),
SecurityGroups: pulumi.StringArray{group.ID()}, // reference the group object above
Ami: pulumi.String("ami-c55673a0"), // AMI for us-east-2 (Ohio)
2023-09-12 13:41:41 +00:00
Tags: pulumi.StringMap{"Name": pulumi.String("webserver")},
2021-08-13 05:05:39 -07:00
})
if err != nil {
return nil, err
}
return & infrastructure{
group: group,
server: server,
}, nil
2020-03-25 17:53:22 +01:00
}
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
2021-03-14 07:14:21 -07:00
WebserverStack.cs:
2020-03-25 17:53:22 +01:00
```csharp
using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Ec2.Inputs;
public class WebserverStack : Stack
{
public WebserverStack()
{
var group = new SecurityGroup("web-secgrp", new SecurityGroupArgs
{
Ingress =
{
new SecurityGroupIngressArgs { Protocol = "tcp", FromPort = 80, ToPort = 80, CidrBlocks = { "0.0.0.0/0" } }
}
});
var server = new Instance("web-server-www", new InstanceArgs
{
InstanceType = "t2.micro",
SecurityGroups = { group.Name }, // reference the group object above
Ami = "ami-c55673a0", // AMI for us-east-2 (Ohio)
Tags = { { "Name", "webserver" }}// name tag
});
}
}
```
{{% /choosable %}}
And then rerun our tests:
{{% choosable language "typescript" %}}
```
Infrastructure
#server
✓ must have a name tag
✓ must not use userData (use an AMI instead)
#group
✓ must not open port 22 (SSH) to the Internet
3 passing (454ms)
```
{{% /choosable %}}
{{% choosable language "python" %}}
```
----------------------------------------------------------------------
Ran 3 tests in 0.022s
OK
```
{{% /choosable %}}
{{% choosable language "go" %}}
```
PASS
ok testing-unit-go 0.704s
```
{{% /choosable %}}
{{% choosable language "csharp" %}}
```
Test Run Successful.
Total tests: 3
Passed: 3
```
{{% /choosable %}}
All the tests passed!
## Full Example
{{% choosable language "typescript" %}}
2021-03-14 07:14:21 -07:00
2021-07-01 11:54:39 +01:00
The full code for this guide is available in the examples repository: [Unit Tests in TypeScript ](https://github.com/pulumi/examples/tree/master/testing-unit-ts ).
2020-04-02 12:49:32 +02:00
2020-03-25 17:53:22 +01:00
{{% /choosable %}}
2021-03-14 07:14:21 -07:00
2020-03-25 17:53:22 +01:00
{{% choosable language "python" %}}
2021-03-14 07:14:21 -07:00
2021-07-01 11:54:39 +01:00
The full code for this guide is available in the examples repository: [Unit Tests in Python ](https://github.com/pulumi/examples/tree/master/testing-unit-py ).
2020-04-02 12:49:32 +02:00
2020-03-25 17:53:22 +01:00
{{% /choosable %}}
2021-03-14 07:14:21 -07:00
2020-03-25 17:53:22 +01:00
{{% choosable language "go" %}}
2021-03-14 07:14:21 -07:00
2021-07-01 11:54:39 +01:00
The full code for this guide is available in the examples repository: [Unit Tests in Go ](https://github.com/pulumi/examples/tree/master/testing-unit-go ).
2020-04-02 12:49:32 +02:00
2020-03-25 17:53:22 +01:00
{{% /choosable %}}
2021-03-14 07:14:21 -07:00
2020-03-25 17:53:22 +01:00
{{% choosable language "csharp" %}}
2021-03-14 07:14:21 -07:00
2021-07-01 11:54:39 +01:00
The full code for this guide is available in the examples repository: [Unit Tests in C# ](https://github.com/pulumi/examples/tree/master/testing-unit-cs ).
2020-04-02 12:49:32 +02:00
2021-03-14 07:14:21 -07:00
2020-03-25 17:53:22 +01:00
{{% /choosable %}}