Immediate caching in Amazon Bedrock can scale back your enter token prices by as much as 90 % whenever you repeatedly ship the identical context to basis fashions, based mostly on Amazon Bedrock immediate caching pricing. With out caching, a ten,000-token contract despatched alongside 50 person questions means 500,000 enter tokens billed at full value for content material the mannequin has already processed.
You’ll be able to mitigate this situation by shortening prompts, lowering context home windows, or implementing application-level caching. Every possibility includes a trade-off:
- Shortened prompts scale back token rely however may also scale back context high quality.
- Smaller context home windows decrease price on the expense of the mannequin’s potential to cause over full info.
- Response caching handles an identical queries nicely, but offers no profit when the identical context is paired with totally different questions.
Immediate caching in Amazon Bedrock helps scale back this problem on the infrastructure stage. If you cache elements of your dialog context (system prompts, paperwork, instrument definitions), Amazon Bedrock reads the cached tokens on subsequent requests as a substitute of reprocessing them. This may scale back time-to-first-token (TTFT) and decrease prices for cached enter tokens by as much as 90 % on cache hits, with out altering your mannequin or immediate high quality.
This put up walks by six sensible immediate caching eventualities utilizing the Converse API in Amazon Bedrock, progressing from primary to superior patterns:
- Message content material caching: Cache lengthy paperwork for multi-question evaluation.
- System immediate caching: Cache persona definitions and directions throughout conversations.
- Software definition caching: Cache instrument schemas for agentic workflows.
- Blended TTL caching: Assign totally different cache lifetimes to totally different content material tiers.
- Tenant isolation: Implement per-tenant cache separation in multi-tenant purposes.
- LangChain integration: Use immediate caching with the LangChain framework.
How immediate caching works
Immediate caching shops a snapshot of partially processed enter in order that subsequent requests with the identical prefix skip redundant computation. This part covers the request circulation, supported fashions, and pricing.
If you embrace a cachePoint marker in your request, Amazon Bedrock evaluates whether or not the content material previous that marker matches an current cache entry. If it does (a cache hit), the mannequin can skip reprocessing these tokens and start era from the cached state. If no match exists (a cache miss), the mannequin processes the complete content material and writes the end result to cache for potential future requests.
This diagram exhibits the circulation:
Determine 1: Immediate caching request circulation, the place the primary request writes to cache and the second request reads from cache, lowering TTFT and enter token price
With this circulation in thoughts, 4 key ideas decide how caching behaves in observe:
- Cache scope: Cache entries are scoped to particular person AWS accounts and AWS Areas.
- Token thresholds: Every cache checkpoint should meet a minimal token threshold to activate. For instance, Anthropic Claude Sonnet 4.5 and Sonnet 4.6 require at the very least 1,024 tokens per checkpoint, whereas Opus fashions require at the very least 4,096.
- Time-to-live (TTL): Cache entries expire based mostly on the TTL specified within the request. The default is 5 minutes, with choose fashions supporting as much as 1 hour.
- Mannequin-agnostic syntax: The Converse API
cachePointsyntax is an identical throughout supported mannequin households, together with Anthropic Claude and Amazon Nova.
For the most recent mannequin help info, see the Amazon Bedrock Immediate Caching documentation.
Pricing
Immediate caching introduces two token classes along with customary enter and output tokens:
| Token Sort | Description | Price vs Normal Enter |
cacheWriteInputTokens |
Tokens written to cache (first request) | 25% larger than customary enter |
cacheReadInputTokens |
Tokens learn from cache (subsequent requests) | 90% decrease than customary enter |
cacheWriteInputTokens (1-hour TTL) |
Tokens written to cache with 1-hour TTL | 100% larger than customary enter (2x) |
For workloads with repeated context, the financial savings attain roughly 75 % on enter token prices. For instance, for those who ship a ten,000-token doc with 10 totally different questions, the primary request incurs a cache write price. The remaining 9 requests every learn from cache at 90 % decreased price, leading to a internet financial savings of roughly 75 % on enter token prices for that doc context. This assumes all subsequent requests happen inside the TTL window. Requests after expiration set off a brand new cache write, lowering the web financial savings. See Amazon Bedrock pricing for detailed pricing info.
Stipulations
Earlier than getting began with the eventualities, ensure you have the next:
- An AWS account with Amazon Bedrock entry in a supported AWS Area (equivalent to
us-west-2). - Mannequin entry enabled for the goal mannequin. The examples on this put up use Anthropic Claude Sonnet 4.5 (
international.anthropic.claude-sonnet-4-5-20250929-v1:0). See Handle mannequin entry for directions. For the most recent mannequin and Area availability, see Supported fashions by AWS Area in Amazon Bedrock. It is a cross-Area inference profile. Requests mechanically route throughout Areas, which may often enhance cache write frequency. - Python 3.10 or later with the next dependencies put in:
Word: Boto3 1.43.0 or later is required for the
ttlparameter incachePointutilized in State of affairs 4 (Blended TTL).
- AWS credentials configured by the default profile or surroundings variables. See Configure the AWS Command Line Interface (AWS CLI) for setup directions.
State of affairs 1: Message content material caching
A typical use case for immediate caching is caching lengthy paperwork or reference content material that you simply question repeatedly. For instance, in a Retrieval Augmented Technology (RAG) software, you ask a number of questions on the identical doc, or a coding assistant references a big codebase.
On this state of affairs, you place a cachePoint marker between the static doc and the dynamic query. Amazon Bedrock caches the doc on the primary name and may reuse it on subsequent calls.
How message content material caching works
Place a cachePoint content material block after the static content material and earlier than the dynamic query. Amazon Bedrock caches the whole lot earlier than the checkpoint and reuses it on subsequent requests:
The next code places this sample into observe with a whole working instance.
Implementation
First, arrange the Amazon Bedrock runtime shopper and outline a pattern doc. In a manufacturing software, this doc generally is a PDF, a data base article, or different content material exceeding the 1,024-token threshold:
Subsequent, outline the caching perform. The important thing components are the cachePoint block positioned between the static doc and the dynamic query:
Run two requests to look at the caching habits. The preliminary name populates the cache, and a subsequent name with a unique query reuses it:
Studying cache metrics
The response utilization object contains two cache-specific fields:
| Subject | Description |
cacheWriteInputTokens |
Tokens written to cache (seems on first request) |
cacheReadInputTokens |
Tokens learn from cache (seems on subsequent requests) |
In our testing with Anthropic Claude Sonnet 4.5 and a doc exceeding 1,024 tokens, the preliminary response exhibits a cache write:
A subsequent request with the identical doc prefix produces a cache learn:
Discover that cacheReadInputTokens now displays the 1,898 tokens learn from cache. All the doc prefix was reused with out reprocessing. Solely 28 tokens (the query itself) had been processed as customary enter. These cached tokens are billed on the decreased cache-read charge (90 % decrease than customary enter).
Simplified cache administration
Claude fashions on Amazon Bedrock help simplified cache administration. You’ll be able to place a single cachePoint, and Amazon Bedrock mechanically checks for cache hits on prefixes as much as roughly 20 content material blocks earlier than that marker. You don’t want to manually place a number of cache checkpoints to get cache hits on earlier parts of your dialog.
For extra granular management, you may place a number of cachePoint markers after every part of content material:
This method helps partial cache hits. If solely the primary two sections match a earlier request, the mannequin reuses the cache for these sections and processes the remaining content material.
Streaming variant
The identical caching syntax works with converse_stream. The important thing distinction is that cache metrics arrive within the metadata occasion on the finish of the stream moderately than within the speedy response:
TTFT benchmark
To quantify the latency enchancment, you may measure TTFT with and with out caching:
Immediate caching can scale back TTFT, with the profit rising because the cached prefix dimension will increase. For smaller paperwork (roughly 2,000–5,000 tokens), the development might not be statistically vital throughout a small variety of iterations. The precise enchancment varies based mostly on doc dimension, mannequin, and present load. For big cached prefixes (over 10,000 tokens), the discount in TTFT turns into extra pronounced.
State of affairs 2: System immediate caching
Many purposes use detailed system prompts that outline the mannequin’s persona, tips, and area experience. These system prompts can span 1000’s of tokens and stay fixed throughout person interactions. With system immediate caching, you pay the complete processing price as soon as and reuse the cached system immediate for each subsequent message.
The cache level goes contained in the system parameter, separate from person messages.
Configuration
The Converse API system parameter accepts an array of content material blocks. Place a cachePoint after the system textual content:
The following instance exhibits this in a full request with an in depth persona immediate.
Implementation
The next instance defines a complete system immediate: an Skilled Area Science Advisor persona with detailed response tips. The immediate exceeds the two,048-token threshold required for caching:
The caching perform locations the cachePoint within the system parameter, retaining person messages separate:
The person message adjustments between requests, however the system immediate stays an identical. Amazon Bedrock is designed to cache the system immediate prefix and reuse it, no matter what the person asks.
When to make use of system immediate caching
System immediate caching is good for:
- Persona-based assistants with detailed position descriptions and behavioral tips.
- Agentic workflows with in depth directions that keep fixed throughout turns.
- Customer support bots with advanced firm insurance policies and response protocols.
- Area-specific assistants with embedded data bases within the system immediate.
Agentic purposes typically outline dozens of instruments with complete JSON schemas. These instrument definitions can collectively include 1000’s of tokens and infrequently change between requests. Software definition caching prevents reprocessing these schemas on each flip.
Setup
Append a cachePoint because the final factor within the instruments array inside toolConfig:
The instance beneath demonstrates this with a set of instrument schemas that collectively exceed the token threshold.
Implementation
Right here we outline a set of space-themed instruments with complete schemas. In a manufacturing software, these is perhaps API integrations, database queries, or exterior service calls:
We construct the Converse API instrument format and append the cachePoint:
Software definition caching is beneficial for agentic workflows the place the identical set of instruments is invoked throughout many dialog turns. By caching the instrument schemas as soon as, you keep away from reprocessing 1000’s of tokens of schema definitions on each flip.
State of affairs 4: Blended TTL caching
Cached content material doesn’t have a single lifecycle. Core reference materials (area data, product catalogs, compliance guidelines) not often adjustments and advantages from longer cache durations. Session-specific context (latest dialog turns, person preferences) adjustments extra often and advantages from shorter expiration. With combined TTL caching, you may assign totally different expiration occasions to totally different content material tiers inside a single request.
Method
Every cachePoint can embrace a ttl discipline. There may be one ordering constraint: longer TTL checkpoints should seem earlier than shorter ones within the request:
Implementation
On this state of affairs, the content material is break up into two tiers: a core reference part cached for 1 hour and a session context part cached for five minutes:
Understanding cacheDetails
The Converse API response features a cacheDetails discipline that exhibits the per-TTL token breakdown. This exhibits whether or not each TTL tiers are working appropriately. When each sections independently exceed the mannequin’s minimal token threshold, you will note one entry per TTL:
If just one part exceeds the edge, the response will present a single TTL entry. Every content material tier should independently meet the mannequin’s minimal token requirement to get separate cache entries for every TTL.
TTL ordering constraint
Cache checkpoints have to be ordered from longest to shortest TTL inside a single request. The API returns an error if a shorter TTL seems earlier than an extended one:
- ✅ Legitimate:
1hthen5m. - ❌ Invalid:
5mthen1h.
This constraint applies throughout cache checkpoint places (messages, system immediate, and gear definitions).
When to make use of combined TTL
The next desk maps content material varieties to really helpful TTLs based mostly on how often the content material adjustments:
| Content material Sort | Really useful TTL | Examples |
| Area data | 1 hour | Product catalogs, compliance guidelines, reference documentation |
| Session context | 5 minutes | Current dialog turns, person preferences, session state |
| Software definitions | 1 hour | API schemas, perform signatures that not often change |
| Dynamic knowledge | No caching | Actual-time knowledge, user-specific queries |
With combined TTL in place, the following state of affairs addresses a typical multi-tenant problem: stopping one tenant’s cached content material from being learn by one other.
State of affairs 5: Tenant isolation
In multi-tenant purposes, it’s essential to stop one tenant’s cached content material from being learn by one other tenant. Immediate caching in Amazon Bedrock scopes entries by account and Area, however inside the similar account and Area, cache entries is perhaps shared throughout requests. The SHA-256 hash prefix sample offers tenant isolation with solely roughly 16 tokens of overhead, with out requiring separate AWS accounts.
Sample
Prepend a SHA-256 hash of the tenant_id to the cached content material. As a result of the hash adjustments the content material prefix, Amazon Bedrock creates a separate cache entry for every tenant:
Implementation
The next perform applies the SHA-256 tenant prefix to the cached content material:
You’ll be able to verify the isolation habits throughout tenants:
The anticipated habits throughout the 4 requests:
| Request | Tenant | Anticipated Cache Habits |
| 1 | tenant1 | cacheWriteInputTokens > 0 (new cache entry) |
| 2 | tenant1 | cacheReadInputTokens > 0 (reuses tenant1’s cache) |
| 3 | tenant2 | cacheWriteInputTokens > 0 (new entry, totally different hash prefix) |
| 4 | tenant2 | cacheReadInputTokens > 0 (reuses tenant2’s cache) |
This sample offers three benefits:
- No server-side configuration: You obtain isolation purely by content material prefixing.
- Unbiased cache entries per tenant: Every tenant’s cached content material stays separate.
- Minimal overhead: The SHA-256 hash provides solely 64 characters (roughly 16 tokens) to the immediate.
State of affairs 6: LangChain integration
For groups utilizing the LangChain framework, immediate caching integrates with the ChatBedrockConverse class. LangChain offers a create_cache_point() technique that generates the proper cachePoint content material block with out requiring you to handle the uncooked API format.
Utilization
Use ChatBedrockConverse.create_cache_point() inside message content material arrays or ChatPromptTemplate definitions:
Implementation: Direct message development
The next instance caches a doc and asks questions with immediate caching enabled:
Implementation: LCEL chain with cache level
For reusable chain patterns, you may combine cache factors straight right into a ChatPromptTemplate:
Inspecting cache metrics in LangChain
The usage_metadata on the response object contains input_token_details with cache-specific fields:
cache_creation: tokens written to cache (first request)cache_read: tokens learn from cache (subsequent requests)
Evaluating the Converse API and InvokeModel API
The eventualities on this put up use the Converse API, which offers a model-agnostic cachePoint syntax. In case your software makes use of the InvokeModel API, remember that the caching syntax differs by mannequin household:
| Characteristic | Converse API (supported fashions) | InvokeModel API (Anthropic) |
| Cache marker | {"cachePoint": {"kind": "default"}} |
"cache_control": {"kind": "ephemeral", "ttl": "5m"} |
| Placement | Standalone content material block after cached content material | Contained in the content material block being cached |
| TTL help | {"cachePoint": {"kind": "default", "ttl": "1h"}} |
"ttl" discipline inside cache_control |
cacheDetails response |
Sure | No |
| Utilization key format | cacheWriteInputTokens |
cache_creation_input_tokens |
Use the Converse API for brand new purposes as a result of the cachePoint syntax works identically throughout the supported mannequin households. This implies you may change between Anthropic Claude and Amazon Nova with out altering your caching code.
Finest practices
Based mostly on the patterns demonstrated on this put up, we suggest the next finest practices for manufacturing deployments:
- Profile your prompts: Determine which elements are static (system prompts, instrument schemas, reference paperwork) and that are dynamic (person questions, session context). Cache the static elements.
- Meet the token threshold: Every cache checkpoint should exceed the mannequin’s minimal token requirement. Amazon Bedrock processes content material beneath the edge usually with out caching.
- Select acceptable TTLs: Use 1-hour TTLs for rarely-changing content material (area data, instrument definitions) and 5-minute TTLs for session-specific context. Do not forget that longer TTLs should seem earlier than shorter ones.
- Monitor cache metrics: Observe
cacheWriteInputTokensandcacheReadInputTokensin your software logs. A low cache-hit ratio may point out that your content material is altering too often or that your TTL is simply too brief. - Apply accountable AI controls: For manufacturing deployments, use Amazon Bedrock Guardrails so as to add content material filtering and grounding validation alongside your caching patterns.
- Implement tenant isolation for multi-tenant programs: Use the SHA-256 hash prefix sample to forestall cross-tenant cache sharing when serving a number of clients from the identical AWS account.
- Use simplified cache administration: For Anthropic Claude fashions, a single
cachePointcan cowl a number of previous content material blocks. You don’t all the time want to position checkpoints after each part. - Mix caching places: You’ll be able to cache system prompts, message content material, and gear definitions concurrently in a single request. This stacks the financial savings throughout the three places.
Clear up
The examples on this put up use on-demand Amazon Bedrock inference and don’t create persistent AWS sources. No cleanup is required past stopping any working Jupyter pocket book kernels.
Conclusion
This put up walked by six sensible immediate caching eventualities utilizing the Amazon Bedrock Converse API, progressing from primary doc caching to superior patterns like combined TTL and tenant isolation. Every state of affairs addresses a selected manufacturing problem:
- Cache content material to scale back prices whenever you ask a number of questions on the identical doc.
- Reuse the immediate and keep away from reprocessing detailed persona definitions on each dialog flip.
- Optimize agentic workflows by effectively caching instrument schemas as soon as.
- Tune combined TTLs for fine-grained management over cache lifetimes for content material with totally different replace frequencies.
- Isolate tenant caches for secure multi-tenant deployments with per-tenant cache separation.
- Combine with LangChain to carry immediate caching to your framework with minimal code adjustments.
The cachePoint syntax is model-agnostic throughout the Converse API. The identical code works with Anthropic Claude, Amazon Nova, and different supported fashions that help immediate caching on Amazon Bedrock. You’ll be able to undertake immediate caching incrementally, beginning with the highest-impact state of affairs to your workload and increasing to further patterns as wanted.
For subsequent steps, profile your current Amazon Bedrock purposes to establish alternatives for immediate caching. Begin with the state of affairs that matches your workload (doc evaluation, persona-based assistants, or agentic instrument use) and measure the TTFT and price enhancements.
Every state of affairs contains working code which you could run instantly in opposition to Amazon Bedrock. The whole set of notebooks and scripts referenced all through this put up is out there within the amazon-bedrock-samples GitHub repository. For extra particulars on immediate caching configuration and supported fashions, see the Amazon Bedrock Immediate Caching documentation.
To get began, go to Amazon Bedrock or open the Amazon Bedrock console to allow mannequin entry and start utilizing immediate caching in your purposes.
Concerning the creator

