amazon-bedrock-agentcore-samples/01-tutorials/07-AgentCore-E2E/lab-04-agentcore-runtime.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5c0122e65c053f38",
   "metadata": {},
   "source": [
    "## Lab 4: Deploy to Production - Use AgentCore Runtime with Observability\n",
    "\n",
    "### Overview\n",
    "\n",
    "In Lab 3 we scaled our Customer Support Agent by centralizing tools through AgentCore Gateway with secure authentication. Now we'll complete the production journey by deploying our agent to AgentCore Runtime with comprehensive observability. This will transform our prototype into a production-ready system that can handle real-world traffic with full monitoring and automatic scaling.\n",
    "\n",
    "[Amazon Bedrock AgentCore Runtime](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agents-tools-runtime.html) is a secure, fully managed runtime that empowers organizations to deploy and scale AI agents in production, regardless of framework, protocol, or model choice. It provides enterprise-grade reliability, automatic scaling, and comprehensive monitoring capabilities.\n",
    "\n",
    "**Workshop Journey:**\n",
    "\n",
    "- **Lab 1 (Done):** Create Agent Prototype - Built a functional customer support agent\n",
    "- **Lab 2 (Done):** Enhance with Memory - Added conversation context and personalization\n",
    "- **Lab 3 (Done):** Scale with Gateway & Identity - Shared tools across agents securely\n",
    "- **Lab 4 (Current):** Deploy to Production - Used AgentCore Runtime with observability\n",
    "- **Lab 5:** Build User Interface - Create a customer-facing application\n",
    "\n",
    "### Why AgentCore Runtime & Production Deployment Matter\n",
    "\n",
    "Current State (Lab 1-3): Agent runs locally with centralized tools but faces production challenges:\n",
    "\n",
    "- Agent runs locally in a single session\n",
    "- No comprehensive monitoring or debugging capabilities\n",
    "- Cannot handle multiple concurrent users reliably\n",
    "\n",
    "After this lab, we will have a production-ready agent infrastructure with:\n",
    "\n",
    "- Serverless auto-scaling to handle variable demand\n",
    "- Comprehensive observability with traces, metrics, and logging\n",
    "- Enterprise reliability with automatic error recovery\n",
    "- Secure deployment with proper access controls\n",
    "- Easy management through AWS console and APIs and support for real-world production workloads.\n",
    "\n",
    "\n",
    "### Adding comprehensive observability with AgentCore Observability\n",
    "\n",
    "Additionally, AgentCore Runtime integrates seamlessly with [AgentCore Observability](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability.html) to provide full visibility into your agent's behavior in production. AgentCore Observability automatically captures traces, metrics, and logs from your agent interactions, tool usage, and memory access patterns. In this lab we will see how AgentCore Runtime integrates with CloudWatch GenAI Observability to provide comprehensive monitoring and debugging capabilities.\n",
    "\n",
    "For request tracing, AgentCore Observability captures the complete conversation flow including tool invocations, memory retrievals, and model interactions. For performance monitoring, it tracks response times, success rates, and resource utilization to help optimize your agent's performance.\n",
    "\n",
    "During the observability flow, AgentCore Runtime automatically instruments your agent code and sends telemetry data to CloudWatch. You can then use CloudWatch dashboards and GenAI Observability features to analyze patterns, identify bottlenecks, and troubleshoot issues in real-time.\n",
    "\n",
    "### Architecture for Lab 4\n",
    "<div style=\"text-align:left\"> \n",
    "    <img src=\"images/architecture_lab4_runtime.png\" width=\"75%\"/> \n",
    "</div>\n",
    "\n",
    "*Agent now runs in AgentCore Runtime with full observability through CloudWatch, serving production traffic with auto-scaling and comprehensive monitoring. Memory and Gateway integrations from previous labs remain fully functional in the production environment.*\n",
    "\n",
    "### Key Features\n",
    "\n",
    "- **Serverless Agent Deployment:** Transform your local agent into a scalable production service using AgentCore Runtime with minimal code changes\n",
    "- **Comprehensive Observability:** Full request tracing, performance metrics, and debugging capabilities through CloudWatch GenAI Observability\n",
    "\n",
    "### Prerequisites\n",
    "\n",
    "- Python 3.12+\n",
    "- AWS account with appropriate permissions\n",
    "- Docker, Finch or Podman installed and running\n",
    "- Amazon Bedrock AgentCore SDK\n",
    "- Strands Agents framework\n",
    "\n",
    "**Note**: You MUST enable [CloudWatch Transaction Search](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Enable-TransactionSearch.html) to be able to see AgentCore Observability traces in CloudWatch.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d867d862-3b01-4b97-ac00-0ff38ff57fc0",
   "metadata": {},
   "source": [
    "### Step 1: Import Required Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa06404e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "import os\n",
    "import json\n",
    "import boto3\n",
    "from strands import Agent\n",
    "from strands.models import BedrockModel\n",
    "from lab_helpers.lab2_memory import create_or_get_memory_resource\n",
    "\n",
    "create_or_get_memory_resource()  # Just in case the memory lab wasn't executed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6820ca8f-a8a8-4f34-b4ef-b6dad3776261",
   "metadata": {},
   "source": [
    "### Step 2: Preparing Your Agent for AgentCore Runtime\n",
    "\n",
    "#### Creating the Runtime-Ready Agent\n",
    "\n",
    "Let's first define the necessary AgentCore Runtime components via Python SDK within our previous local agent implementation.\n",
    "\n",
    "Observe the `#### AGENTCORE RUNTIME - LINE i ####` comments below to see where is the relevant deployment code added. You'll find 4 such lines that prepare the runtime-ready agent:\n",
    "\n",
    "1. Import the Runtime App with `from bedrock_agentcore.runtime import BedrockAgentCoreApp`\n",
    "2. Initialize the App with `app = BedrockAgentCoreApp()`\n",
    "3. Decorate our invocation function with `@app.entrypoint`\n",
    "4. Let AgentCore Runtime control the execution with `app.run()`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e79eba2-ca59-463f-9ebf-56e362d7ae66",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile ./lab_helpers/lab4_runtime.py\n",
    "from bedrock_agentcore.runtime import (\n",
    "    BedrockAgentCoreApp,\n",
    ")  #### AGENTCORE RUNTIME - LINE 1 ####\n",
    "from strands import Agent\n",
    "from strands.models import BedrockModel\n",
    "from scripts.utils import get_ssm_parameter\n",
    "from lab_helpers.lab1_strands_agent import (\n",
    "    get_return_policy,\n",
    "    get_product_info,\n",
    "    SYSTEM_PROMPT,\n",
    "    MODEL_ID,\n",
    ")\n",
    "\n",
    "from lab_helpers.lab2_memory import (\n",
    "    CustomerSupportMemoryHooks,\n",
    "    memory_client,\n",
    "    ACTOR_ID,\n",
    "    SESSION_ID,\n",
    ")\n",
    "\n",
    "# Lab1 import: Create the Bedrock model\n",
    "model = BedrockModel(model_id=MODEL_ID)\n",
    "\n",
    "# Lab2 import : Initialize memory via hooks\n",
    "memory_id = get_ssm_parameter(\"/app/customersupport/agentcore/memory_id\")\n",
    "memory_hooks = CustomerSupportMemoryHooks(\n",
    "    memory_id, memory_client, ACTOR_ID, SESSION_ID\n",
    ")\n",
    "\n",
    "# Create the agent with all customer support tools\n",
    "agent = Agent(\n",
    "    model=model,\n",
    "    tools=[get_return_policy, get_product_info],\n",
    "    system_prompt=SYSTEM_PROMPT,\n",
    "    hooks=[memory_hooks],\n",
    ")\n",
    "\n",
    "# Initialize the AgentCore Runtime App\n",
    "app = BedrockAgentCoreApp()  #### AGENTCORE RUNTIME - LINE 2 ####\n",
    "\n",
    "\n",
    "@app.entrypoint  #### AGENTCORE RUNTIME - LINE 3 ####\n",
    "def invoke(payload):\n",
    "    \"\"\"AgentCore Runtime entrypoint function\"\"\"\n",
    "    user_input = payload.get(\"prompt\", \"\")\n",
    "\n",
    "    # Invoke the agent\n",
    "    response = agent(user_input)\n",
    "    return response.message[\"content\"][0][\"text\"]\n",
    "\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    app.run()  #### AGENTCORE RUNTIME - LINE 4 ####"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8855aceb-b79f-4aaa-b16f-8577c059816a",
   "metadata": {},
   "source": [
    "#### What happens behind the scenes?\n",
    "\n",
    "When you use `BedrockAgentCoreApp`, it automatically:\n",
    "\n",
    "- Creates an HTTP server that listens on port 8080\n",
    "- Implements the required `/invocations` endpoint for processing requests\n",
    "- Implements the `/ping` endpoint for health checks\n",
    "- Handles proper content types and response formats\n",
    "- Manages error handling according to AWS standards\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e8aa1fb-4e80-4dbd-864a-8e7bf9eab714",
   "metadata": {},
   "source": [
    "### Step 3: Deploying to AgentCore Runtime\n",
    "\n",
    "Now let's deploy our agent to AgentCore Runtime using the [AgentCore Starter Toolkit](https://github.com/aws/bedrock-agentcore-starter-toolkit).\n",
    "\n",
    "#### Configure the Secure Runtime Deployment (AgentCore Runtime + AgentCore Identity)\n",
    "\n",
    "First we will use our starter toolkit to configure the AgentCore Runtime deployment with an entrypoint, the execution role we will create and a requirements file. We will also configure the identity authorization using an Amazon Cognito user pool and we will configure the starter kit to auto create the Amazon ECR repository on launch.\n",
    "\n",
    "During the configure step, your docker file will be generated based on your application code\n",
    "\n",
    "<div style=\"text-align:left\"> \n",
    "    <img src=\"images/configure.png\" width=\"75%\"/> \n",
    "</div>\n",
    "\n",
    "**Note**: The Cognito access_token is valid for 2 hours only. If the access_token expires you can vend another access_token by using the `reauthenticate_user` method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4581baa2-9edc-425d-becf-09968565081a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from lab_helpers.utils import setup_cognito_user_pool, reauthenticate_user\n",
    "\n",
    "print(\"Setting up Amazon Cognito user pool...\")\n",
    "cognito_config = (\n",
    "    setup_cognito_user_pool()\n",
    ")  # You'll get your bearer token from this output cell.\n",
    "print(\"Cognito setup completed ✓\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17a32ab8-7701-4900-8055-e24364bdf35c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bedrock_agentcore_starter_toolkit import Runtime\n",
    "from lab_helpers.utils import create_agentcore_runtime_execution_role\n",
    "\n",
    "# Initialize the runtime toolkit\n",
    "boto_session = boto3.session.Session()\n",
    "region = boto_session.region_name\n",
    "\n",
    "execution_role_arn = create_agentcore_runtime_execution_role()\n",
    "\n",
    "agentcore_runtime = Runtime()\n",
    "\n",
    "# Configure the deployment\n",
    "response = agentcore_runtime.configure(\n",
    "    entrypoint=\"lab_helpers/lab4_runtime.py\",\n",
    "    execution_role=execution_role_arn,\n",
    "    auto_create_ecr=True,\n",
    "    requirements_file=\"requirements.txt\",\n",
    "    region=region,\n",
    "    agent_name=\"customer_support_agent\",\n",
    "    authorizer_configuration={\n",
    "        \"customJWTAuthorizer\": {\n",
    "            \"allowedClients\": [cognito_config.get(\"client_id\")],\n",
    "            \"discoveryUrl\": cognito_config.get(\"discovery_url\"),\n",
    "        }\n",
    "    },\n",
    ")\n",
    "\n",
    "print(\"Configuration completed:\", response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e1b84cc-798e-472c-ac0b-2c315f4b704d",
   "metadata": {},
   "source": [
    "#### Launch the Agent\n",
    "\n",
    "Now let's launch our agent to AgentCore Runtime. This will create an AWS CodeBuild pipeline, the Amazon ECR repository and the AgentCore Runtime components.\n",
    "\n",
    "<div style=\"text-align:left\"> \n",
    "    <img src=\"images/launch.png\" width=\"100%\"/> \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afa6ac09-9adb-4846-9fc1-4d12aeb74853",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Launch the agent (this will build and deploy the container)\n",
    "from lab_helpers.utils import put_ssm_parameter\n",
    "\n",
    "launch_result = agentcore_runtime.launch()\n",
    "print(\"Launch completed:\", launch_result.agent_arn)\n",
    "\n",
    "agent_arn = put_ssm_parameter(\n",
    "    \"/app/customersupport/agentcore/runtime_arn\", launch_result.agent_arn\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0ae9c09-09db-4a76-871a-92eacd96b9c3",
   "metadata": {},
   "source": [
    "#### Check Deployment Status\n",
    "\n",
    "Let's wait for the deployment to complete:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d909e42-e1a0-407f-84c2-3d16cc889cd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "# Wait for the agent to be ready\n",
    "status_response = agentcore_runtime.status()\n",
    "status = status_response.endpoint[\"status\"]\n",
    "\n",
    "end_status = [\"READY\", \"CREATE_FAILED\", \"DELETE_FAILED\", \"UPDATE_FAILED\"]\n",
    "while status not in end_status:\n",
    "    print(f\"Waiting for deployment... Current status: {status}\")\n",
    "    time.sleep(10)\n",
    "    status_response = agentcore_runtime.status()\n",
    "    status = status_response.endpoint[\"status\"]\n",
    "\n",
    "print(f\"Final status: {status}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7f89c56-918a-4cab-beaa-c7ac43a2ba29",
   "metadata": {},
   "source": [
    "### Step 4: Invoking Your Deployed Agent\n",
    "\n",
    "Now that our agent is deployed and ready, let's test it with some queries. We invoke the agent with the right authorization token type. In out case it'll be Cognito access token. Copy the access token from the cell above\n",
    "\n",
    "<div style=\"text-align:left\"> \n",
    "    <img src=\"images/invoke.png\" width=\"100%\"/> \n",
    "</div>\n",
    "\n",
    "#### Using the AgentCore Starter Toolkit\n",
    "\n",
    "We can validate that the agent works using the AgentCore Starter Toolkit for invocation. The starter toolkit can automatically create a session id for us to query our agent. Alternatively, you can also pass the session id as a parameter during invocation. For demonstration purpose, we will create our own session id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be15c4b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import uuid\n",
    "\n",
    "# Create a session ID for demonstrating session continuity\n",
    "session_id = uuid.uuid4()\n",
    "\n",
    "# Test different customer support scenarios\n",
    "user_query = \"My Iphone is not connecting with the Bluetooth. What should I do?\"\n",
    "\n",
    "bearer_token = reauthenticate_user(\n",
    "    cognito_config.get(\"client_id\"), \n",
    "    cognito_config.get(\"client_secret\")\n",
    ")\n",
    "\n",
    "response = agentcore_runtime.invoke(\n",
    "    {\"prompt\": user_query}, \n",
    "    bearer_token=bearer_token,\n",
    "    session_id=str(session_id)\n",
    ")\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79840836-5cd5-4139-a188-09d181438840",
   "metadata": {},
   "source": [
    "#### Invoking the agent with session continuity\n",
    "\n",
    "Since we are using AgentCore Runtime, we can easily continue our conversation with the same session id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e378d3f-1fcf-4d8e-b8eb-d34578a788ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "user_query = \"I've turned my Bluetooth on and off but it still does not work\"\n",
    "response = agentcore_runtime.invoke(\n",
    "    {\"prompt\": user_query}, \n",
    "    bearer_token=bearer_token,\n",
    "    session_id=str(session_id)\n",
    ")\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "687d63ab-c72d-46a7-a9da-d8b008c75136",
   "metadata": {},
   "source": [
    "#### Invoking the agent with a new user\n",
    "In the example below we have not mentioned the Iphone device in the second query, but our agent still has the context of it. This is due to the AgentCore Runtime session continuity. The agent won't know the context for a new user."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92a84f17-9a71-4025-ba9e-c10682e7cd8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating a new session ID for demonstrating new customer\n",
    "session_id2 = uuid.uuid4()\n",
    "\n",
    "user_query = \"Still not working. What is going on?\"\n",
    "response = agentcore_runtime.invoke(\n",
    "    {\"prompt\": user_query}, \n",
    "    bearer_token=bearer_token,\n",
    "    session_id=str(session_id2)\n",
    ")\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "087ad1d9",
   "metadata": {},
   "source": [
    "In this case our agent does not have the context anymore and needs more information. \n",
    "\n",
    "And it is all it takes to have a secure and scalable endpoint for our Agent with no need to manage all the underlying infrastructure!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca82b1ee",
   "metadata": {},
   "source": [
    "### Step 5: AgentCore Observability\n",
    "\n",
    "[AgentCore Observability](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability.html) provides monitoring and tracing capabilities for AI agents using Amazon OpenTelemetry Python Instrumentation and Amazon CloudWatch GenAI Observability.\n",
    "\n",
    "#### Agents\n",
    "\n",
    "Default AgentCore Runtime configuration allows for logging our agent's traces on CloudWatch by means of **AgentCore Observability**. These traces can be seen on the AWS CloudWatch GenAI Observability dashboard. Navigate to Cloudwatch &rarr; GenAI Observability &rarr; Bedrock AgentCore.\n",
    "\n",
    "![Agents Overview on CloudWatch](images/observability_agents.png)\n",
    "\n",
    "#### Sessions\n",
    "\n",
    "The Sessions view shows the list of all the sessions associated with all agents in your account.\n",
    "\n",
    "![sessions](images/sessions_lab5_observability.png)\n",
    "\n",
    "#### Traces\n",
    "\n",
    "Trace view lists all traces from your agents in this account. To work with traces:\n",
    "\n",
    "- Choose Filter traces to search for specific traces.\n",
    "- Sort by column name to organize results.\n",
    "- Under Actions, select Logs Insights to refine your search by querying across your log and span data or select Export selected traces to export.\n",
    "\n",
    "![traces](images/traces_lab4_observability.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c243e86-a214-483c-aef1-d5243f28ca9e",
   "metadata": {},
   "source": [
    "### Congratulations! 🎉\n",
    "\n",
    "You have successfully completed **Lab 4: Deploy to Production - Use AgentCore Runtime with Observability!**\n",
    "\n",
    "Here is what you accomplished:\n",
    "\n",
    "##### Production-Ready Deployment:\n",
    "\n",
    "- Prepared your agent for production with minimal code changes (only 4 lines added)\n",
    "- Validated proper session isolation between different customers\n",
    "- Confirmed session continuity + memory persistence and context awareness per session\n",
    "\n",
    "##### Enterprise-Grade Security & Identity:\n",
    "\n",
    "- Implemented secure authentication using Cognito integration with JWT tokens\n",
    "- Configured proper IAM roles and execution permissions for production workloads\n",
    "- Established identity-based access control for secure agent invocation\n",
    "\n",
    "##### Comprehensive Observability:\n",
    "\n",
    "- Enabled AgentCore Observability for full request tracing across all customer sessions\n",
    "- Configured CloudWatch GenAI Observability dashboard monitoring\n",
    "\n",
    "##### Current Limitations (We'll fix these next!):\n",
    "\n",
    "- **Developer Focused Interaction** - Agent accessible via SDK/API calls but no user-friendly web interface\n",
    "- **Manual Session Management** - Requires programmatic session creation rather than intuitive user experience\n",
    "\n",
    "##### Next Up [Lab 4: Build User Interface →](lab-05-frontend.ipynb)\n",
    "In Lab 5, you'll complete the customer experience by building a user-friendly interface !! Lets go !!\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}