Alice Williams 1dd560977b
Adds Reporting CLI documentation (#2170)
* add files back from lost local files

Signed-off-by: Alice Williams <alicejw@amazon.com>

* nest reporting CLI under main Reporting page

Signed-off-by: Alice Williams <alicejw@amazon.com>

* rough draft examples for AWS Lambda and Cron

Signed-off-by: Alice Williams <alicejw@amazon.com>

* better title for cron job report example

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for more details from tech reviewer

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for better cron job usage description

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for more details

Signed-off-by: Alice Williams <alicejw@amazon.com>

* remove needs info markers and add links to events doc

Signed-off-by: Alice Williams <alicejw@amazon.com>

* more info

Signed-off-by: Alice Williams <alicejw@amazon.com>

* more details from demonstration

Signed-off-by: Alice Williams <alicejw@amazon.com>

* remove reporting directory to keep both parallel in left nav

Signed-off-by: Alice Williams <alicejw@amazon.com>

* small addition

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for two configuration settings in the function

Signed-off-by: Alice Williams <alicejw@amazon.com>

* more details from eng meeting adding to the steps

Signed-off-by: Alice Williams <alicejw@amazon.com>

* more details from eng meeting adding to the steps

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for more details

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for additional steps to create and test the lambda function

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for scheduling reports with lambda

Signed-off-by: Alice Williams <alicejw@amazon.com>

* merge redundant set of steps together

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for tech review comments

Signed-off-by: Alice Williams <alicejw@amazon.com>

* to add download correct title and link

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for reporting directory to nest new cli page

Signed-off-by: Alice Williams <alicejw@amazon.com>

* un-nest reporting directory

Signed-off-by: Alice Williams <alicejw@amazon.com>

* to remove reporting directory - will nest with other page in separate PR

Signed-off-by: Alice Williams <alicejw@amazon.com>

* tech review comments and some reorg

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for tech review comments

Signed-off-by: Alice Williams <alicejw@amazon.com>

* to minor changes

Signed-off-by: Alice Williams <alicejw@amazon.com>

* more info about crontab sequence

Signed-off-by: Alice Williams <alicejw@amazon.com>

* cron updates

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for cron syntax

Signed-off-by: Alice Williams <alicejw@amazon.com>

* small updates to lambda proc

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for final writer checklist

Signed-off-by: Alice Williams <alicejw@amazon.com>

* cron link update

Signed-off-by: Alice Williams <alicejw@amazon.com>

* format fix

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for doc review cron & lambda sections

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for cron reference to man page

Signed-off-by: Alice Williams <alicejw@amazon.com>

* add internal link to cron ref

Signed-off-by: Alice Williams <alicejw@amazon.com>

* reorg single topic into 7 pages

Signed-off-by: Alice Williams <alicejw@amazon.com>

* minor update

Signed-off-by: Alice Williams <alicejw@amazon.com>

* link to set ses permissions

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for tech review meeting updates

Signed-off-by: Alice Williams <alicejw@amazon.com>

* small update

Signed-off-by: Alice Williams <alicejw@amazon.com>

* npm download command name changed to remove opensearch prepended name

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for new download via artifacts hub

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for new download instructions

Signed-off-by: Alice Williams <alicejw@amazon.com>

* small fix signature file

Signed-off-by: Alice Williams <alicejw@amazon.com>

* typo

Signed-off-by: Alice Williams <alicejw@amazon.com>

* for cleaner image

Signed-off-by: Alice Williams <alicejw@amazon.com>

* remove image for push commands we won't use

Signed-off-by: Alice Williams <alicejw@amazon.com>

---------

Signed-off-by: Alice Williams <alicejw@amazon.com>
2023-02-08 17:52:19 -08:00

7.9 KiB

layout title nav_order parent
default Scheduling reports with AWS Lambda 30 Creating reports with the Reporting CLI

Scheduling reports with AWS Lambda

You can use AWS Lambda with the Reporting CLI tool to specify an AWS Lambda function to trigger the report generation.

This requires that you use an AMD64 system and Docker.

Prerequisites

To use the Reporting CLI with AWS Lambda, you need to do the following preliminary steps.

Step 1: Create a container image with a Dockerfile

You need to assemble the container image by running a Dockerfile. When you run the Dockerfile, it downloads the OpenSearch artifact required to use the Reporting CLI. To learn more about Dockerfiles, see Dockerfile reference.

Copy the following sample configurations into a Dockerfile:

# Define function directory
ARG FUNCTION_DIR="/function"

# Base image of the docker container
FROM node:lts-slim as build-image

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# AWS Lambda runtime dependencies
RUN apt-get update && \
    apt-get install -y \
        g++ \
        make \
        unzip \
        libcurl4-openssl-dev \
        autoconf \
        automake \
        libtool \
        cmake \
        python3 \
        libkrb5-dev \
        curl

# Copy function code
WORKDIR ${FUNCTION_DIR}
RUN curl -LJO https://artifacts.opensearch.org/reporting-cli/opensearch-reporting-cli-1.0.0.tgz
RUN tar -xzf opensearch-reporting-cli-1.0.0.tgz
RUN mv package/* .
RUN npm install && npm install aws-lambda-ric

# Build Stage 2: Copy Build Stage 1 files in to Stage 2. Install chrome, then remove chrome to keep the dependencies.
FROM node:lts-slim
# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}
# Copy in the build image dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

# Install latest chrome dev package and fonts to support major char sets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer installs, work.
RUN apt-get update \
    && apt-get install -y wget gnupg \
    && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
      --no-install-recommends \
    && apt-get remove -y google-chrome-stable \
    && rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["/usr/local/bin/npx", "aws-lambda-ric"]

ENV HOME="/tmp"
CMD [ "/function/src/index.handler" ]

Next, run the following build command within the same directory that contains the Dockerfile:

docker build -t opensearch-reporting-cli .

Step 2: Create a private repository with Amazon ECR

You need to follow the instructions to create an image repository, see Getting started with Amazon ECR using the AWS Management Console.

Give your repository the name opensearch-reporting-cli.

In addition to the Amazon ECR instructions, you need to make several adjustments for the Reporting CLI to function properly as described in the following steps in this procedure.

Step 3: Push the image to the private repository

You need to get several commands from the AWS ECR Console to run within the Dockerfile directory.

  1. After you create your repository, select it from Private repositories.
  2. Choose view push commands.
  3. Copy and run each command shown in Push commands for opensearch-reporting-cli sequentially in the Dockerfile directory.

For more details about Docker push commands, see Pushing a Docker image in the Amazon ECR user guide.

Step 4: Create a Lambda function with the container image

Now that you have a container image created for the Reporting CLI, you need to create a function defined as the container image.

  1. Open the AWS Lambda console and choose Functions.

  2. Choose Create function, then choose Container image and fill in a name for the function.

  3. In Container image URI, choose Browse images and select opensearch-reporting-cli for the image repository.

  4. In Images select the image, and choose Select image.

  5. In Architecture, choose x86_64.

  6. Choose Create function.

  7. Go to Lambda > functions and choose the function you created.

  8. Choose Configuration > General configuration > Edit timeout and set the timeout in lambda to 5 minutes to allow the Reporting CLI to generate the report.

  9. Change the Ephemeral storage setting to at least 1024MB. The default setting is not a sufficient storage amount to support report generation.

  10. Next, test the function either by providing values JSON format or by providing AWS Lambda environment variables.

  • If the function contains fixed values, such as email address you do not need a JSON file. You can specify an environment variable in AWS Lambda.
  • If the function takes a variable key-value pair, then you need to specify the values in the JSON with the same naming convention as command options, for example the --credentials option requires the username and password. {: .note }

The following example shows fixed values provided for the sender and recipient email addresses:

{
  "url": "https://playground.opensearch.org/app/dashboards#/view/084aed50-6f48-11ed-a3d5-1ddbf0afc873",
  "transport": "ses",
  "from": "sender@amazon.com", 
  "to": "recipient@amazon.com", 
  "subject": "Test lambda docker image"
}

To learn more about AWS Lambda functions, see Deploying Lambda functions as container images in the AWS Lambda documentation.

Step 5: Add the trigger to start the AWS Lambda function

Set the trigger to start running the report. AWS Lambda can use any AWS service as a trigger, such as SNS, S3, or an AWS CloudWatch EventBridge.

  1. In the Triggers section, choose Add trigger.
  2. Select a trigger from the list. For example, you can set an AWS CloudWatch Event. To learn more about Amazon ECR events you can schedule, see Sample events from Amazon ECR.
  3. Choose Test to initiate the function.

(Optional) Step 6: Add the role permission for Amazon SES

If you want to use Amazon SES for the email transport, you need to set up permissions.

  1. Select Configuration and choose Execution role.
  2. In Summary, choose Permissions.
  3. Select {}JSON to open the JSON policy editor.
  4. Add the permissions for the Amazon SES resource that you want to use.

The following example provides the resource ARN for the send email action:

{
"Effect": "Allow",
"Action": [
      "ses:SendEmail",
      "ses:SendRawEmail"
            ],
"Resource": "arn:aws:ses:us-west-2:555555511111:identity/username@amazon.com"
}

To learn more about setting role permissions, see Permissions in the AWS Lambda user guide.