As your AI brokers transfer from prototype to manufacturing, the challenges shift from getting them to work to holding them quick and environment friendly. In Half 1 of this collection, we walked via debugging two frequent agent failures: infinite loops and gear invocation errors. These eventualities handled brokers that have been damaged. On this publish, we sort out a distinct problem: brokers that work accurately however carry out poorly. Sluggish response instances and unbounded reminiscence progress are the most typical operational points that floor after you resolve the preliminary debugging issues. They don’t set off error alerts, however they erode person belief and enhance prices over time.
Utilizing AgentCore Observability, a functionality of Amazon Bedrock AgentCore, and Amazon CloudWatch, you’ll discover ways to establish efficiency bottlenecks throughout your agent’s execution path and diagnose reminiscence points in long-running classes. Additionally, you will implement monitoring practices that catch degradation earlier than customers discover it. For added data and finest practices, evaluation AgentCore Evaluations, a functionality of Amazon Bedrock AgentCore, and AgentCore Insights.
You want an AWS account with Amazon Bedrock AgentCore entry, CloudWatch Transaction Search enabled, and a deployed agent. See Half 1 for full setup particulars.
Situation 3: Efficiency bottlenecks
Brokers expertise efficiency bottlenecks once they work accurately however reply too slowly. You count on sub-second responses however expertise multi-second delays. Brokers full duties efficiently, however the latency makes them impractical for interactive use instances. This situation is especially difficult as a result of gradual is subjective. What’s acceptable for a batch processing agent is unacceptable for a customer support chatbot. You will need to set up efficiency budgets on your particular use case, then systematically establish which elements violate these budgets.
Signs to observe for
Efficiency degradation typically manifests steadily. Brokers may begin with acceptable 2-second response instances, however as you add options, combine extra instruments, or accumulate extra reminiscence, latency creeps to five seconds, then 10, then turns into unusable. P95 response instances exceed your thresholds, customers abandon classes, however error charges keep low. The agent works accurately however responds too slowly.
Determine 1 — Session particulars exhibiting a number of traces with persistently excessive latency. The three invocations took 7.5-8.2 seconds (common span latency), demonstrating a systemic efficiency bottleneck quite than occasional slowness. This sample signifies the agent’s structure wants optimization.
To search out bottlenecks, begin by figuring out high-latency requests. Question CloudWatch for agent invocations that exceed your efficiency price range:
This question returns agent invocations that took longer than 3 seconds (regulate the brink based mostly in your necessities), sorted by latency. Choose a consultant high-latency request and be aware its RequestId.
Subsequent, analyze the request timeline to grasp the place time is spent:
The question reveals you the sequence of operations throughout the request and the way lengthy every took. Search for operations that devour disproportionate time. Frequent culprits embrace reminiscence retrieval operations, instrument invocations, token era, and sequential operations that might run in parallel.
Determine 2 — OpenTelemetry hint timeline exhibiting 17 spans throughout three sequential execute_event_loop_cycle operations. The instruments (customer_lookup, order_history) execute one after one other quite than in parallel, with every cycle ready for the earlier one to finish. This sequential sample compounds latency throughout every instrument invocation.
Test reminiscence retrieval latency particularly:
Reminiscence retrieval ought to full in beneath 200 milliseconds, representing the purpose the place customers start perceiving noticeable delays in interactive purposes. Increased latency suggests inefficient reminiscence group.
Study instrument invocation latency to establish gradual integrations:
The question identifies which instruments contribute most to total latency. A single gradual instrument can bottleneck your entire agent workflow.
Root trigger evaluation
Efficiency bottlenecks sometimes stem from three root causes. Sluggish instrument execution happens when exterior instruments take seconds to reply due to poor optimization, overload, or community points. Latencies compound with sequential calls, so a 2-second instrument referred to as thrice turns into a 6-second bottleneck. Extreme token era is an element as a result of basis fashions (FMs) produce tokens sequentially, which means a 500-token response takes 5x longer than a 100-token one, impacting each latency and value. Lastly, sequential processing, performing unbiased operations one by one as an alternative of in parallel, will increase each price and compute instances.
The repair
For gradual instrument execution, implement caching, connection pooling, and correct database indexing to cut back response instances. Set timeout limits and think about sooner instrument alternate options. If a instrument persistently lags, profile it independently. The problem is perhaps community latency, chilly begins, or useful resource rivalry quite than the instrument’s logic itself.
For reminiscence retrieval, exchange single massive namespaces with topic-specific partitions, similar to preferences, historical past, and area information, to cut back search house. Summarize outdated conversations into compact entries quite than storing them verbatim, and set measurement limits per namespace, similar to 100 preferences, 50 latest messages, or 500 area details.
For token era, optimize prompts to encourage transient, direct solutions of two–3 sentences except extra element is requested. Add specific size constraints and monitor token utilization with alerts for unexpectedly lengthy responses.
For sequential processing, run unbiased instrument calls in parallel. Sequential calls totaling 4.5s (2s + 1.5s + 1s) drop to solely 2s when parallelized, typically reducing latency by 50 p.c or extra with minimal effort. To confirm your optimizations, re-run the latency question from the figuring out bottlenecks part. Affirm that P95 response instances now fall inside your efficiency price range and that the hint timeline reveals parallel execution the place anticipated.
Situation 4: Reminiscence points in long-running classes
Brokers expertise reminiscence points in long-running classes once they preserve classes the place reminiscence utilization grows unbounded. Brokers accumulate context, and finally, brokers hit token limits, lose essential context, or exhaust obtainable reminiscence. Classes fail unexpectedly, and also you lose dialog state. This situation is especially problematic for brokers that help prolonged workflows, similar to customer support classes, analysis assistants, or monitoring brokers. With out correct reminiscence administration, these use instances grow to be impractical.
Signs to observe for
When abnormally lengthy agent classes exist, token utilization grows linearly with session length. Reminiscence retrieval latency will increase over time as reminiscence shops develop. Classes terminate unexpectedly with out-of-memory errors or context window exceeded errors.
Determine 3 — Session particulars for the memorygrowth_Agent exhibiting 6 traces, 15.7K complete tokens consumed, and a median hint latency of three,757 ms inside a single session. Token utilization grows with every successive invocation, demonstrating unbounded context accumulation. In manufacturing classes spanning hours, this sample results in context window exhaustion and sudden session failures.
To establish long-running classes with excessive reminiscence utilization:
The question returns classes lasting longer than one hour (3600 seconds), sorted by reminiscence measurement. Choose a session with unusually excessive reminiscence utilization and be aware its SessionId.
Study reminiscence extraction patterns to confirm consolidation is happening:
Determine 4 — CloudWatch Logs Insights exhibiting 209 memory-related log entries concentrated round 18:20. The spike in reminiscence operations signifies the agent storing data with out consolidation. Every invocation provides new reminiscence entries (via add_conversation_note and add_user_context instruments) with out pruning or summarizing outdated knowledge, demonstrating the unbounded progress sample.
Reminiscence extraction ought to happen commonly all through the session. If you happen to see gaps the place no extraction occurs for prolonged intervals, the agent isn’t consolidating recollections correctly.
Test for reminiscence extraction failures:
Failed reminiscence extraction stops brokers from consolidating context, inflicting unbounded progress. Frequent failure causes embrace token limits exceeded throughout summarization, invalid reminiscence codecs that may’t be processed, community timeouts when writing to reminiscence storage, and permission errors that block reminiscence updates.
Analyze reminiscence namespace group:
Recollections are distributed throughout namespaces. Poor namespace group can result in inefficient reminiscence retrieval and consolidation. If you happen to see a single namespace containing hundreds of recollections, that’s a pink flag indicating your agent wants higher reminiscence group.
Root trigger evaluation
Reminiscence points sometimes stem from misconfigured settings, filter on inbuilt datetime metadata of the report. For a deeper understanding of how AgentCore reminiscence works, see AgentCore reminiscence.
The repair
To resolve reminiscence points, confirm your reminiscence methods embrace a consolidation configuration so AgentCore reminiscence merges and summarizes information over time quite than accumulating them indefinitely. Set up information utilizing namespace templates in your technique definitions to ensure retrieval stays scoped to related context. Set eventExpiryDuration to manage how lengthy uncooked occasions persist (between 7–one year). For implementation particulars, see AgentCore reminiscence implementation.
Having coated the 4 failure eventualities throughout each elements, we now flip to manufacturing finest practices that cease these points earlier than they happen. The troubleshooting workflows we’ve coated make it easier to reply when issues go mistaken, however a well-architected observability technique helps you proactively catch points. The mixing of AgentCore with CloudWatch supplies the muse for this proactive strategy, providing you with real-time visibility into agent well being and efficiency.
Activate complete instrumentation for manufacturing brokers, reminiscence methods, and gateways. Configure CloudWatch logs, CloudWatch metrics, and OpenTelemetry traces.
Configure CloudWatch alarms for important metrics. Set thresholds for error charges (5 p.c), ninety fifth percentile (P95) latency (3 seconds), and token utilization per session. Don’t look forward to customers to report issues. Let CloudWatch warn you when metrics exceed acceptable thresholds.
Create operational dashboards. Construct a major dashboard exhibiting complete invocations (final 24 hours), error charge (present in comparison with baseline), P50/P95/P99 latency, token utilization developments, and lively classes depend. Create per-agent dashboards exhibiting agent-specific invocation patterns, instrument utilization breakdown, reminiscence consumption developments, error sorts distribution, and value per session. Overview these dashboards every day to identify developments earlier than they grow to be issues.
Spend money on observability infrastructure earlier than you want it. Construct monitoring into your improvement course of from day one. Share dashboards with product managers and stakeholders so everybody understands agent efficiency. When somebody diagnoses a difficult manufacturing situation, share the strategy with the group to construct institutional information about frequent failure patterns.
To observe instrument accuracy at scale, you should use Amazon Bedrock AgentCore Evaluators to repeatedly and routinely assess agent habits. As a substitute of manually reviewing traces after failures happen, Evaluators examine agent classes in actual time, scoring them towards predefined high quality standards. Amazon Bedrock AgentCore Insights (preview) builds off Evaluators and supplies triage evaluation. Insights can present failure evaluation, person intent extraction, and execution abstract.
After testing the optimization strategies on this publish, clear up sources to keep away from pointless expenses.
For CloudWatch sources, delete take a look at CloudWatch dashboards created throughout debugging, take away CloudWatch alarms arrange for testing functions, and think about archiving or deleting outdated CloudWatch log teams if now not wanted.
For AgentCore sources, if you happen to created take a look at brokers particularly for this tutorial, delete them via the AgentCore console. Take away any take a look at reminiscence namespaces created throughout reminiscence optimization testing. Delete any momentary instrument integrations used for efficiency testing.
For price optimization, evaluation your Amazon CloudWatch Logs retention settings and regulate based mostly in your compliance necessities. Think about using CloudWatch Logs knowledge safety to chop storage prices for older logs.
To delete CloudWatch log teams:
To take away CloudWatch alarms:
Efficiency bottlenecks and reminiscence points signify the most typical operational challenges for manufacturing brokers past the debugging eventualities coated in Half 1. With systematic prognosis utilizing CloudWatch traces and metrics, you’ll be able to establish whether or not latency stems from gradual instruments, inefficient reminiscence retrieval, extreme token era, or sequential processing. For long-running classes, monitoring reminiscence progress patterns and implementing consolidation methods retains brokers secure over prolonged interactions.
Subsequent steps
Able to put these strategies into observe? Begin by turning on AgentCore Observability on your manufacturing brokers if you happen to haven’t already. This one-time setup supplies the muse for all the things we’ve coated. Arrange CloudWatch alarms for important metrics. Create troubleshooting runbooks on your particular brokers and workflows, documenting the queries you run, the thresholds that point out issues, and the fixes you implement.
Observe debugging in non-production environments. Intentionally introduce failures and observe diagnosing them utilizing the workflows we’ve coated. Construct muscle reminiscence for troubleshooting earlier than you want it in manufacturing. Share this data along with your group to assist affirm everybody who operates your brokers understands these debugging strategies and is aware of how you can use the observability options of AgentCore.
Manufacturing brokers typically fail for preventable causes. With the correct observability instruments, systematic troubleshooting approaches, and a tradition that treats each situation as an opportunity to enhance, you’ll be able to catch issues early, resolve them rapidly, and construct brokers that earn lasting belief.
Be taught extra
For extra details about AgentCore and observability options, go to the AgentCore documentation. To get began with AgentCore, go to the AgentCore console. For added CloudWatch monitoring finest practices, see the CloudWatch Person Information.
Concerning the authors
