iSharkFly-Docs/pulumi-hugo-cn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
pulumi-hugo-cn/themes/default/content/docs/clouds/kubernetes/guides/app-services.md
susan evans ad4f7527b9
add meta imgs for docs (#3018)
2023-06-08 16:15:52 -07:00

18 KiB
Raw Permalink Blame History

title_tag, meta_desc, title, h1, meta_image, menu, aliases
title_tag meta_desc title h1 meta_image menu aliases
Deploy Kubernetes App Services | Crosswalk This page provides an overview of how to deploy Kubernetes App services to different cloud providers. App services Kubernetes App services /images/docs/meta-images/docs-clouds-kubernetes-meta-image.png
clouds
parent identifier weight
kubernetes-guides kubernetes-guides-app-svcs 8
/docs/guides/crosswalk/kubernetes/app-services/

{{< chooser cloud "aws,azure,gcp" / >}}

App services are general services scoped at the Kubernetes application level. These services tend to include datastores, and managers for ingress, DNS, and TLS. They can be shared amongst several apps or be specific to workloads, and are usually a mix of cloud provider and custom services.

{{% choosable cloud aws %}}

{{% /choosable %}}

{{% choosable cloud azure %}}

{{% /choosable %}}

{{% choosable cloud gcp %}}

{{% /choosable %}}

Overview

We'll explore how to setup:

Prerequisites

{{< chooser cloud "aws,azure,gcp" / >}}

{{% choosable cloud aws %}}

Authenticate as the admins role from the Identity stack.

$ aws sts assume-role --role-arn `pulumi stack output adminsIamRoleArn` --role-session-name k8s-admin
$ export KUBECONFIG=`pwd`/kubeconfig-admin.json

{{% /choosable %}}

{{% choosable cloud azure %}}

Authenticate as the ServicePrincipal from the Identity stack.

$ az login --service-principal --username $ARM_CLIENT_ID --password $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID
$ export KUBECONFIG=`pwd`/kubeconfig-admin.json

{{% /choosable %}}

{{% choosable cloud gcp %}}

Authenticate as the admins ServiceAccount from the Identity stack.

$ gcloud auth activate-service-account --key-file k8s-admin-sa-key.json
$ export KUBECONFIG=`pwd`/kubeconfig.json

{{% /choosable %}}

Datastores

Apps may want to persist data to databases or in-memory datastores. Often times these services are provisioned directly with the cloud provider to simplify running and managing their lifecycles.

{{% choosable cloud aws %}}

Postgres Database

Create a Postgres database instance in AWS RDS, and store its connection information in a Kubernetes Secret for apps to refer to and consume.

import * as aws from "@pulumi/aws";
import * as random from "@pulumi/random";
import * as k8s from "@pulumi/kubernetes";

// Generate a strong password for the Postgres DB.
const postgresDbPassword = new random.RandomString(`${projectName}-db-password`, {
	length: 20,
	special: true
}, {additionalSecretOutputs: ["result"]}).result;

// Create a Postgres DB instance of RDS.
const dbSubnets = new aws.rds.SubnetGroup(`${projectName}-subnets`, {
    subnetIds: privateSubnetIds
});
const db = new aws.rds.Instance("postgresdb", {
    engine: "postgres",
    instanceClass: "db.t2.micro",
    allocatedStorage: 20,
    dbSubnetGroupName: dbSubnets.id,
    vpcSecurityGroupIds: securityGroupIds,
    name: "testdb",
    username: "alice",
    password: postgresDbPassword,
    skipFinalSnapshot: true,
});

// Create a Secret from the DB connection information.
const provider = new k8s.Provider("eks-provider", {kubeconfig: config.kubeconfig.apply(JSON.stringify)});
const dbConn = new k8s.core.v1.Secret("postgres-db-conn",
    {
        data: {
            host: db.address.apply(addr => Buffer.from(addr).toString("base64")),
            port: db.port.apply(port => Buffer.of(port).toString("base64")),
            username: db.username.apply(user => Buffer.from(user).toString("base64")),
            password: postgresDbPassword.apply(pass => Buffer.from(pass).toString("base64")),
        },
    },
    {provider: provider},
);

Redis Datastore

Create a Redis datastore instance in AWS ElastiCache, and store its connection information in a Kubernetes ConfigMap for apps to refer to and consume.

import * as aws from "@pulumi/aws";
import * as k8s from "@pulumi/kubernetes";

// Create a Redis instance.
const cacheSubnets = new aws.elasticache.SubnetGroup(`${projectName}-cache-subnets`, {
    subnetIds: privateSubnetIds,
});
const cacheCluster = new aws.elasticache.Cluster("cachecluster", {
    engine: "redis",
    nodeType: "cache.t2.micro",
    numCacheNodes: 1,
    subnetGroupName: cacheSubnets.id,
    securityGroupIds: securityGroupIds,
});

// Create a ConfigMap from the cache connection information.
const cacheConn = new k8s.core.v1.ConfigMap("redis-db-conn",
    {
        data: {
            host: cacheCluster.cacheNodes[0].address.apply(addr => Buffer.from(addr).toString("base64")),
        },
    },
    {provider: provider},
);

{{% /choosable %}}

{{% choosable cloud azure %}}

MongoDB

Create a MongoDB database instance in Azure CosmosDB, and store its connection information in a Kubernetes Secret for apps to refer to and consume.

import * as azure from "@pulumi/azure";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const name = pulumi.getProject();

// Define a separate resource group for app services.
const resourceGroup = new azure.core.ResourceGroup(name);

// Create a MongoDB-flavored instance of CosmosDB.
const cosmosdb = new azure.cosmosdb.Account("k8s-az-mongodb", {
    resourceGroupName: resourceGroup.name,
    kind: "MongoDB",
    consistencyPolicy: {
        consistencyLevel: "Session",
    },
    offerType: "Standard",
    geoLocations: [
        { location: resourceGroup.location, failoverPriority: 0 },
    ],
});

// A k8s provider instance of the cluster.
const provider = new k8s.Provider(`${name}-aks`, {
    kubeconfig: config.kubeconfig,
});

// Create secret from MongoDB connection string.
const mongoConnStrings = new k8s.core.v1.Secret(
    "mongo-secrets",
    {
        metadata: { name: "mongo-secrets", namespace: config.appsNamespaceName},
        data: mongoHelpers.parseConnString(cosmosdb.connectionStrings),
    },
    { provider },
);

{{% /choosable %}}

{{% choosable cloud gcp %}}

Postgres Database

Create a Postgres database instance in CloudSQL, and store its connection information in a Kubernetes Secret for apps to refer to and consume.

import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";
import * as random from "@pulumi/random";

// Generate a strong password for the Postgres DB.
const postgresDbPassword = new random.RandomString(
    `${projectName}-db-password`,
    {
        length: 20,
        special: true,
    },
    { additionalSecretOutputs: ["result"] },
).result;

// Create a Postgres DB instance.
const db = new gcp.sql.DatabaseInstance("postgresdb", {
    project: config.project,
    region: "us-west1",
    databaseVersion: "POSTGRES_9_6",
    settings: { tier: "db-f1-micro" },
});

// Configure a new SQL user.
const user = new gcp.sql.User("default", {
    project: config.project,
    instance: db.name,
    password: postgresDbPassword,
});

// Create a new k8s provider.
const provider = new k8s.Provider("provider", {
    kubeconfig: config.kubeconfig,
});

// Create a Secret from the DB connection information.
const dbConn = new k8s.core.v1.Secret(
    "postgres-db-conn",
    {
        data: {
            host: db.privateIpAddress.apply(addr => Buffer.from(addr).toString("base64")),
            port: Buffer.from("5432").toString("base64"),
            username: user.name.apply(user => Buffer.from(user).toString("base64")),
            password: postgresDbPassword.apply(pass => Buffer.from(pass).toString("base64")),
        },
    },
    { provider: provider },
);

Redis Datastore

Create a Redis datastore instance in Google Cloud MemoryStore, and store its connection information in a Kubernetes ConfigMap for apps to refer to and consume.

import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";

// Create a Redis instance.
const cache = new gcp.redis.Instance("redis", {
    tier: "STANDARD_HA",
    memorySizeGb: 1,
    redisVersion: "REDIS_3_2",
});

// Create a ConfigMap from the cache connection information.
const cacheConn = new k8s.core.v1.ConfigMap(
    "postgres-db-conn",
    {
        data: {
            host: cache.host.apply(addr => Buffer.from(addr).toString("base64")),
        },
    },
    { provider: provider },
);

{{% /choosable %}}

General App Services

General app services stack on GitHub

NGINX Ingress Controller

The NGINX Ingress Controller is a custom Kubernetes Controller. It manages L7 network ingress / north-south traffic between external clients, and the servers in the cluster's apps.

Install NGINX

{{< chooser k8s-language "typescript,yaml" >}}

{{% choosable k8s-language yaml %}}

Deploy the example YAML manifests into the ingress-nginx namespace, and publicly expose it to the Internet using a load balanced Service.

Note: NGINX requires a privileged PSP given its use of allowPrivilegeEscalation: true.

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/cloud-generic.yaml

Check that the NGINX deployment is running.

$ kubectl get pods -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-7dcc95dfbf-k99k6   1/1     Running   0          21s

{{% /choosable %}}

{{% choosable k8s-language typescript %}}

Deploy the Helm chart into the app-svcs namespace created in Configure Cluster Defaults, and publicly expose it to the Internet using a load balanced Service.

Note: NGINX requires a privileged PSP given its use of allowPrivilegeEscalation: true.

import * as k8s from "@pulumi/kubernetes";

// Deploy the NGINX ingress controller using the Helm chart.
const nginx = new k8s.helm.v3.Chart("nginx",
    {
        namespace: config.appSvcsNamespaceName,
        chart: "nginx-ingress",
        version: "1.24.4",
        fetchOpts: {repo: "https://charts.helm.sh/stable/"},
        values: {controller: {publishService: {enabled: true}}},
        transformations: [
            (obj: any) => {
                // Do transformations on the YAML to set the namespace
                if (obj.metadata) {
                    obj.metadata.namespace = config.appSvcsNamespaceName;
                }
            },
        ],
    },
    {providers: {kubernetes: provider}},
);

{{% /choosable %}}

{{< /chooser >}}

Deploy a Workload

Deploy a kuard Pod, service, and ingress resources to test the NGINX ingress controller.

Create the ingress resource for kuard that NGINX will manage by keying off the ingress.class used.

NGINX will front the app through it's desired host and paths, and the apps are will be accessible to the public internet as they share the public load balancer endpoint provisioned for NGINX's service.

Traffic is then routed to the app by inspecting the host headers and paths expected by NGINX onto the service that the kuard Pod runs.

{{< chooser k8s-language "typescript,yaml" >}}

{{% choosable k8s-language yaml %}}

$ kubectl run --generator=run-pod/v1 kuard --namespace=`pulumi stack output appsNamespaceName` --image=gcr.io/kuar-demo/kuard-amd64:blue --port=8080 --expose

$ cat > ingress.yaml << EOF
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: kuard
  labels:
    app: kuard
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: apps.example.com
    http:
      paths:
        - path: "/"
          backend:
            serviceName: kuard
            servicePort: http
EOF

$ kubectl apply -f ingress.yaml --namespace=`pulumi stack output appsNamespaceName`

Check that the ingress is created, and after a few moments the Address will be set to the NGINX LoadBalancer Service address.

$ kubectl describe ingress kuard --namespace=`pulumi stack output appsNamespaceName`

{{% /choosable %}}

{{% choosable k8s-language typescript %}}

import * as k8s from "@pulumi/kubernetes";

// Create a kuard Deployment
const name = "kuard"
const labels = {app: name}
const deployment = new k8s.apps.v1.Deployment(name,
    {
        metadata: {
            namespace: config.appsNamespaceName,
            labels: {app: name},
        },
        spec: {
            replicas: 1,
            selector: { matchLabels: labels },
            template: {
                metadata: { labels: labels, },
                spec: {
                    containers: [
                        {
                            name: name,
                            image: "gcr.io/kuar-demo/kuard-amd64:blue",
                            resources: {requests: {cpu: "50m", memory: "20Mi"}},
                            ports: [{ name: "http", containerPort: 8080 }]
                        }
                    ],
                }
            }
        },
    },
    {provider: provider}
);

// Create a Service for the kuard Deployment
const service = new k8s.core.v1.Service(name,
    {
        metadata: {labels: labels, namespace: config.appsNamespaceName},
        spec: {ports: [{ port: 8080, targetPort: "http" }], selector: labels},
    },
    {provider: provider}
);

// Export the Service name and public LoadBalancer endpoint
export const serviceName = service.metadata.name;

// Create the kuard Ingress
const ingress = new k8s.extensions.v1beta1.Ingress(name,
    {
        metadata: {
            labels: labels,
            namespace: config.appsNamespaceName,
            annotations: {"kubernetes.io/ingress.class": "nginx"},
        },
        spec: {
            rules: [
                {
                    host: "apps.example.com",
                    http: {
                        paths: [
                            {
                                path: "/",
                                backend: {
                                    serviceName: serviceName,
                                    servicePort: "http",
                                }
                            },
                        ],
                    },
                }
            ]
        }
    },
    {provider: provider}
);

Check that the ingress is created, and after a few moments the Address will be set to the NGINX LoadBalancer Service address.

$ kubectl describe ingress kuard-<POD_SUFFIX> --namespace=`pulumi stack output appsNamespaceName`

{{% /choosable %}}

{{< /chooser >}}

Use the NGINX LoadBalancer Service address to access kuard on its expected hosts & paths. We simulate the headers using curl.

$ curl -Lv -H 'Host: apps.example.com' <INGRESS_ADDRESS>

Clean Up

{{< chooser k8s-language "typescript,yaml" >}}

{{% choosable k8s-language yaml %}}

Delete the pod, service, and ingress controller.

$ kubectl delete pod/kuard svc/kuard ingress/kuard
$ kubectl delete -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/cloud-generic.yaml

{{% /choosable %}}

{{% choosable k8s-language typescript %}}

Delete the nginx definition in the Pulumi program, and run a Pulumi update.

{{% /choosable %}}

{{< /chooser >}}