"## Lab 4: Deploy to Production - Use AgentCore Runtime with Observability\n",
"\n",
"### Overview\n",
"\n",
"In Lab 3 we scaled our Customer Support Agent by centralizing tools through AgentCore Gateway with secure authentication. Now we'll complete the production journey by deploying our agent to AgentCore Runtime with comprehensive observability. This will transform our prototype into a production-ready system that can handle real-world traffic with full monitoring and automatic scaling.\n",
"\n",
"[Amazon Bedrock AgentCore Runtime](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agents-tools-runtime.html) is a secure, fully managed runtime that empowers organizations to deploy and scale AI agents in production, regardless of framework, protocol, or model choice. It provides enterprise-grade reliability, automatic scaling, and comprehensive monitoring capabilities.\n",
"\n",
"**Workshop Journey:**\n",
"\n",
"- **Lab 1 (Done):** Create Agent Prototype - Built a functional customer support agent\n",
"- **Lab 2 (Done):** Enhance with Memory - Added conversation context and personalization\n",
"- **Lab 3 (Done):** Scale with Gateway & Identity - Shared tools across agents securely\n",
"- **Lab 4 (Current):** Deploy to Production - Used AgentCore Runtime with observability\n",
"- **Lab 5:** Build User Interface - Create a customer-facing application\n",
"\n",
"### Why AgentCore Runtime & Production Deployment Matter\n",
"\n",
"Current State (Lab 1-3): Agent runs locally with centralized tools but faces production challenges:\n",
"\n",
"- Agent runs locally in a single session\n",
"- No comprehensive monitoring or debugging capabilities\n",
"After this lab, we will have a production-ready agent infrastructure with:\n",
"\n",
"- Serverless auto-scaling to handle variable demand\n",
"- Comprehensive observability with traces, metrics, and logging\n",
"- Enterprise reliability with automatic error recovery\n",
"- Secure deployment with proper access controls\n",
"- Easy management through AWS console and APIs and support for real-world production workloads.\n",
"\n",
"\n",
"### Adding comprehensive observability with AgentCore Observability\n",
"\n",
"Additionally, AgentCore Runtime integrates seamlessly with [AgentCore Observability](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability.html) to provide full visibility into your agent's behavior in production. AgentCore Observability automatically captures traces, metrics, and logs from your agent interactions, tool usage, and memory access patterns. In this lab we will see how AgentCore Runtime integrates with CloudWatch GenAI Observability to provide comprehensive monitoring and debugging capabilities.\n",
"\n",
"For request tracing, AgentCore Observability captures the complete conversation flow including tool invocations, memory retrievals, and model interactions. For performance monitoring, it tracks response times, success rates, and resource utilization to help optimize your agent's performance.\n",
"\n",
"During the observability flow, AgentCore Runtime automatically instruments your agent code and sends telemetry data to CloudWatch. You can then use CloudWatch dashboards and GenAI Observability features to analyze patterns, identify bottlenecks, and troubleshoot issues in real-time.\n",
"*Agent now runs in AgentCore Runtime with full observability through CloudWatch, serving production traffic with auto-scaling and comprehensive monitoring. Memory and Gateway integrations from previous labs remain fully functional in the production environment.*\n",
"\n",
"### Key Features\n",
"\n",
"- **Serverless Agent Deployment:** Transform your local agent into a scalable production service using AgentCore Runtime with minimal code changes\n",
"- **Comprehensive Observability:** Full request tracing, performance metrics, and debugging capabilities through CloudWatch GenAI Observability\n",
"\n",
"### Prerequisites\n",
"\n",
"- Python 3.12+\n",
"- AWS account with appropriate permissions\n",
"- Docker, Finch or Podman installed and running\n",
"- Amazon Bedrock AgentCore SDK\n",
"- Strands Agents framework\n",
"\n",
"**Note**: You MUST enable [CloudWatch Transaction Search](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Enable-TransactionSearch.html) to be able to see AgentCore Observability traces in CloudWatch.\n"
"create_or_get_memory_resource() # Just in case the memory lab wasn't executed"
]
},
{
"cell_type": "markdown",
"id": "6820ca8f-a8a8-4f34-b4ef-b6dad3776261",
"metadata": {},
"source": [
"### Step 2: Preparing Your Agent for AgentCore Runtime\n",
"\n",
"#### Creating the Runtime-Ready Agent\n",
"\n",
"Let's first define the necessary AgentCore Runtime components via Python SDK within our previous local agent implementation.\n",
"\n",
"Observe the `#### AGENTCORE RUNTIME - LINE i ####` comments below to see where is the relevant deployment code added. You'll find 4 such lines that prepare the runtime-ready agent:\n",
"\n",
"1. Import the Runtime App with `from bedrock_agentcore.runtime import BedrockAgentCoreApp`\n",
"2. Initialize the App with `app = BedrockAgentCoreApp()`\n",
"3. Decorate our invocation function with `@app.entrypoint`\n",
"4. Let AgentCore Runtime control the execution with `app.run()`\n"
"First we will use our starter toolkit to configure the AgentCore Runtime deployment with an entrypoint, the execution role we will create and a requirements file. We will also configure the identity authorization using an Amazon Cognito user pool and we will configure the starter kit to auto create the Amazon ECR repository on launch.\n",
"\n",
"During the configure step, your docker file will be generated based on your application code\n",
"**Note**: The Cognito access_token is valid for 2 hours only. If the access_token expires you can vend another access_token by using the `reauthenticate_user` method.\n"
"Now let's launch our agent to AgentCore Runtime. This will create an AWS CodeBuild pipeline, the Amazon ECR repository and the AgentCore Runtime components.\n",
" status = status_response.endpoint[\"status\"]\n",
"\n",
"print(f\"Final status: {status}\")"
]
},
{
"cell_type": "markdown",
"id": "b7f89c56-918a-4cab-beaa-c7ac43a2ba29",
"metadata": {},
"source": [
"### Step 4: Invoking Your Deployed Agent\n",
"\n",
"Now that our agent is deployed and ready, let's test it with some queries. We invoke the agent with the right authorization token type. In out case it'll be Cognito access token. Copy the access token from the cell above\n",
"We can validate that the agent works using the AgentCore Starter Toolkit for invocation. The starter toolkit can automatically create a session id for us to query our agent. Alternatively, you can also pass the session id as a parameter during invocation. For demonstration purpose, we will create our own session id."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be15c4b6",
"metadata": {},
"outputs": [],
"source": [
"import uuid\n",
"\n",
"# Create a session ID for demonstrating session continuity\n",
"session_id = uuid.uuid4()\n",
"\n",
"# Test different customer support scenarios\n",
"user_query = \"My Iphone is not connecting with the Bluetooth. What should I do?\"\n",
"\n",
"bearer_token = reauthenticate_user(\n",
" cognito_config.get(\"client_id\"), \n",
" cognito_config.get(\"client_secret\")\n",
")\n",
"\n",
"response = agentcore_runtime.invoke(\n",
" {\"prompt\": user_query}, \n",
" bearer_token=bearer_token,\n",
" session_id=str(session_id)\n",
")\n",
"response"
]
},
{
"cell_type": "markdown",
"id": "79840836-5cd5-4139-a188-09d181438840",
"metadata": {},
"source": [
"#### Invoking the agent with session continuity\n",
"\n",
"Since we are using AgentCore Runtime, we can easily continue our conversation with the same session id."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2e378d3f-1fcf-4d8e-b8eb-d34578a788ff",
"metadata": {},
"outputs": [],
"source": [
"user_query = \"I've turned my Bluetooth on and off but it still does not work\"\n",
"response = agentcore_runtime.invoke(\n",
" {\"prompt\": user_query}, \n",
" bearer_token=bearer_token,\n",
" session_id=str(session_id)\n",
")\n",
"response"
]
},
{
"cell_type": "markdown",
"id": "687d63ab-c72d-46a7-a9da-d8b008c75136",
"metadata": {},
"source": [
"#### Invoking the agent with a new user\n",
"In the example below we have not mentioned the Iphone device in the second query, but our agent still has the context of it. This is due to the AgentCore Runtime session continuity. The agent won't know the context for a new user."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "92a84f17-9a71-4025-ba9e-c10682e7cd8e",
"metadata": {},
"outputs": [],
"source": [
"# Creating a new session ID for demonstrating new customer\n",
"session_id2 = uuid.uuid4()\n",
"\n",
"user_query = \"Still not working. What is going on?\"\n",
"response = agentcore_runtime.invoke(\n",
" {\"prompt\": user_query}, \n",
" bearer_token=bearer_token,\n",
" session_id=str(session_id2)\n",
")\n",
"response"
]
},
{
"cell_type": "markdown",
"id": "087ad1d9",
"metadata": {},
"source": [
"In this case our agent does not have the context anymore and needs more information. \n",
"\n",
"And it is all it takes to have a secure and scalable endpoint for our Agent with no need to manage all the underlying infrastructure!"
]
},
{
"cell_type": "markdown",
"id": "ca82b1ee",
"metadata": {},
"source": [
"### Step 5: AgentCore Observability\n",
"\n",
"[AgentCore Observability](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability.html) provides monitoring and tracing capabilities for AI agents using Amazon OpenTelemetry Python Instrumentation and Amazon CloudWatch GenAI Observability.\n",
"\n",
"#### Agents\n",
"\n",
"Default AgentCore Runtime configuration allows for logging our agent's traces on CloudWatch by means of **AgentCore Observability**. These traces can be seen on the AWS CloudWatch GenAI Observability dashboard. Navigate to Cloudwatch → GenAI Observability → Bedrock AgentCore.\n",
"\n",
"\n",
"\n",
"#### Sessions\n",
"\n",
"The Sessions view shows the list of all the sessions associated with all agents in your account.\n",
