Should you’re utilizing Retrieval-Augmented Era (RAG) for advanced analytical duties that span a whole lot of paperwork, equivalent to monetary due diligence or regulatory compliance evaluations, you’ve possible hit its ceiling. Similarity search surfaces related fragments however typically misses cross-document connections. This submit exhibits you tackle that hole utilizing task-aware information compression (TAKC), a method that pre-compresses complete information bases into task-specific representations deployed on AWS. You possibly can deploy an entire open-source implementation in your individual account.
Activity-aware information compression
Contemplate a personal fairness agency evaluating a $500 million acquisition of a producing firm. The due diligence group should analyze monetary statements spanning 12 subsidiaries and 5 years. In addition they face 200+ provider contracts, environmental compliance reviews from 8 amenities, and 50+ authorized circumstances. When an analyst asks about consolidated monetary dangers given present provider phrases and pending litigation, RAG’s similarity search can’t floor that response. A whole lot of paperwork maintain the related info, and the connections between them share no lexical similarity.
TAKC addresses the sort of downside by utilizing an LLM to supply shorter, task-focused summaries of paperwork, with totally different summaries for various duties.
Completely different duties require totally different info from the identical doc. An annual report compressed for monetary evaluation wants income figures, margins, and money movement knowledge. The identical report compressed for a compliance evaluation wants regulatory citations and violation histories. Generic summarization makes an attempt to cowl every part, which dilutes the data density for any particular use case. TAKC compresses paperwork by the lens of a selected process, holding what issues and discarding the remaining. The Ingestion pipeline part exhibits how the compression immediate specifies precisely what info to protect. For manufacturing deployments, retailer task-type prompts in a versioned configuration (equivalent to AWS Methods Supervisor Parameter Retailer or a devoted Amazon Easy Storage Service (Amazon S3) prefix) in order that immediate modifications are auditable and recompression could be triggered when prompts are up to date.
The system compresses paperwork offline, as soon as per doc per process sort. At question time, the system retrieves a pre-compressed illustration fairly than the unique doc. The system then solutions questions utilizing the compressed model as a substitute of the complete doc. If the compressed illustration lacks ample element, the question complexity analyzer routes the query to a decrease compression tier that retains extra context.
TAKC gives entry to the complete information base in compressed type, not simply the top-k chunks {that a} similarity search returns. The system preserves connections between paperwork as a result of the compression sees paperwork collectively. It additionally produces totally different compressed outputs for various duties from the identical supply materials. The monetary compression of a 10-Ok submitting seems nothing just like the authorized threat compression of that very same submitting. The compression reduces token rely by 8x to 64x whereas concentrating on task-relevant info for retention.
Multi-rate compression
Completely different queries require totally different ranges of constancy. A query equivalent to “What was Q3 income?” wants far much less context than a request to investigate relationships between provider cost phrases and quarterly money movement throughout subsidiaries.
TAKC addresses this by sustaining 4 compression tiers for every process sort. On the lightest compression (8x), the system retains roughly 87.5% much less context. It preserves sufficient element for multi-step reasoning and cross-document synthesis. On the medium tier (16x), context discount reaches roughly 93.8%. This degree fits analytical queries with reasonable complexity. The excessive tier (32x) reduces context by roughly 96.9% and serves factual lookups and well-defined questions. On the extremely tier (64x), discount reaches roughly 98.4%. This degree is suitable for classification duties and key phrase lookups.
A question complexity analyzer routes incoming inquiries to the suitable tier primarily based on indicators like question size, query sort, and presence of analytical language. Simple factual questions hit the ultra-compressed cache whereas advanced analytical questions use the calmly compressed cache. This occurs transparently to the person.
Most enterprise queries are lookups that may be served from the upper compression tiers at minimal value. The rare advanced queries devour bigger context budgets solely when wanted. This tier-based routing enhances current RAG optimizations like metadata filtering and question reformatting, which may slender the doc set earlier than compression is utilized. To validate compression high quality, examine LLM responses at every tier in opposition to responses generated from the complete uncompressed doc. The reference implementation consists of take a look at scripts that run this comparability in your particular process varieties and paperwork.
Structure on AWS
The implementation runs on AWS as two decoupled serverless pipelines: one for ingestion and one for queries. Determine 1 illustrates each pipelines.
TAKC structure. The pipeline movement handles knowledge ingestion and compression. The person movement handles authenticated queries
We selected AWS Lambda for compute as a result of every perform invocation is short-lived and event-driven. The workload processes knowledge in bursts throughout ingestion and handles variable question masses between bursts, making serverless a pure match.
We selected Amazon API Gateway to show the question interface as a REST endpoint. For caching, we selected Amazon ElastiCache Serverless for reads on composite keys (takc:{process}:{fee}) with out shard administration. Amazon Cognito handles JWT issuance and token refresh with out customized auth code, lowering the implementation floor space.
Ingestion pipeline
When a doc lands in Amazon S3 below a task-type prefix (for instance, raw-data/monetary/), an S3 occasion notification triggers an AWS Lambda perform. This perform chunks the doc into 256-token segments with 50-token overlap to forestall info loss at boundaries. It then invokes a compression Lambda perform for every chunk asynchronously, enabling parallel processing. For giant-scale ingestion, configure reserved concurrency on the compression perform and place an Amazon Easy Queue Service (Amazon SQS) queue between the chunking and compression steps to deal with throttling gracefully.
The second perform calls Amazon Bedrock to compress the chunks in any respect 4 compression tiers. Every compression name features a task-aware immediate that tells the mannequin what info to protect:
TASK: Monetary evaluation. Protect income metrics, margins, money movement, debt obligations, and monetary threat indicators.
COMPRESSION TARGET: Cut back to roughly 1/16 of authentic size.
INSTRUCTIONS:
- Concentrate on details and relationships related to the duty
- Protect numerical knowledge and metrics
- Keep entities and their attributes
- Hold causal relationships and dependencies
- Take away redundant or irrelevant infoThe mannequin is aware of what to protect as a result of the immediate specifies what issues for the duty. That distinction is what makes this task-aware fairly than generic compression. The system shops compressed outputs in Amazon ElastiCache Serverless with keys like takc:monetary:medium, and backs them as much as S3 for sturdiness. The Redis OSS knowledge mannequin helps the hierarchical key construction wanted for multi-rate cache lookups. Cache entries use a 24-hour TTL and are backed as much as S3. If a cache entry is evicted or expires, the question perform falls again to the S3 backup and re-populates the cache on learn.
Question pipeline
A person authenticates by Amazon Cognito, receives a JWT, and sends a question by Amazon API Gateway. AWS WAF fronts the API for fee limiting and menace safety. A Lambda perform analyzes question complexity utilizing heuristics (key phrase indicators and question size), retrieves the suitable compressed cache from Amazon ElastiCache Serverless, and sends the compressed context plus question to Amazon Bedrock for inference. When routing confidence is low, the system defaults to the medium compression tier as a protected fallback.
The upper-cost Bedrock compression calls occur as soon as throughout ingestion. The question path is a cache lookup plus inference on compressed context.
The stack makes use of Amazon Bedrock (Anthropic Claude 3 Haiku, Claude 3 Sonnet, and Amazon Titan Textual content) for compression and inference. The mannequin choice is configurable by CDK context values with out code modifications. AWS Lambda (Python 3.12+) handles knowledge processing and question logic. Amazon ElastiCache Serverless shops the compressed cache, whereas Amazon S3 holds uncooked knowledge, chunks, and cache backups (KMS-encrypted). Amazon API Gateway exposes the REST endpoints, and Amazon Cognito gives JWT-based authentication. AWS WAF fronts the API with fee limiting and managed safety guidelines. Amazon CloudWatch gives monitoring and metrics, and AWS Key Administration Service (AWS KMS) manages encryption keys.
Outline the infrastructure as a single AWS Cloud Improvement Package (AWS CDK) stack and deploy it with a single command. AWS CDK gives repeatable deployments and allows you to customise parameters like compression chunk measurement, Lambda reminiscence allocation, and ElastiCache storage limits by context values. For manufacturing deployments, think about separating stateful assets (S3, ElastiCache, Cognito) into their very own stack to cut back blast radius and allow impartial lifecycle administration of compute and storage layers.
Value comparability
Enter token utilization for a 100,000-token information base queried 1,000 instances per day (output tokens are excluded as a result of response size is impartial of context measurement):
| Strategy | Enter tokens per question | Every day enter tokens | Relative enter value |
| Full context | 100,000 | 100,000,000 | 100% |
| RAG (top-10 chunks) | ~10,000 | 10,000,000 | 10% |
| TAKC Gentle (8Ă—) | ~12,500 | 12,500,000 | 12.5% |
| TAKC Medium (16Ă—) | ~6,250 | 6,250,000 | 6.25% |
| TAKC Excessive (32Ă—) | ~3,125 | 3,125,000 | 3.1% |
| TAKC Extremely (64Ă—) | ~1,563 | 1,563,000 | 1.6% |
The token financial savings from compression are derived immediately from the compression ratios. Precise financial savings rely in your particular Bedrock mannequin pricing and question patterns.
TAKC requires upfront compression value by one-time Bedrock calls throughout ingestion. This value amortizes for information bases that change sometimes and get queried repeatedly. For information bases that change hourly, RAG’s per-query retrieval mannequin could also be extra sensible.
When to decide on TAKC, RAG, or each
This desk summarizes when every strategy is the higher match, primarily based in your workload traits:
| Issue | Favors TAKC | Favors RAG |
| Question sort | Cross-document reasoning, synthesis | Slim factual lookups |
| Data base stability | Modifications every day or much less | Modifications hourly or extra |
| Activity predictability | Effectively-defined process varieties | Unpredictable question patterns |
| Protection requirement | Should think about full corpus | Just a few paperwork are related |
| Supply attribution | Not required | Required (person must see supply) |
| Token funds | Tight | Versatile |
In follow, a manufacturing system typically advantages from each. RAG handles fast lookups effectively. TAKC handles the analytical queries the place retrieval-based approaches miss connections. A question complexity analyzer can route between them. Use RAG when customers must hint responses again to particular supply paperwork. For regulated workloads that require each cross-document reasoning and auditability, mix TAKC with RAG: use TAKC for the analytical response and RAG to retrieve the supporting supply paperwork for the audit path.
Getting began
Stipulations
To deploy this reference implementation, verify that you’ve the next:
Deploy and take a look at
Clone the repository aws-samples/sample-bedrock-takc-compression and deploy the CDK stack:
After deployment completes, take a look at the pipeline:
- Add a doc to the S3 bucket:
- Wait 2-3 minutes for the ingestion pipeline to chunk and compress the doc in any respect 4 compression tiers.
- Question the API endpoint:
The system handles chunking, multi-rate compression, caching, and question routing with out further configuration.
Clear up
To keep away from ongoing expenses, destroy the CDK stack if you’re carried out testing. First, empty the S3 bucket as a result of CDK can’t delete buckets that include objects:
Then destroy the stack:
This removes the Lambda capabilities, API Gateway, Amazon ElastiCache Serverless cache, Amazon Cognito person pool, WAF internet ACL, and Amazon CloudWatch alarms. The KMS secret’s retained with a 30-day pending deletion window. To schedule its deletion instantly, run the next command:
Conclusion
Complicated analytical duties that span a whole lot of paperwork require greater than fragment retrieval. TAKC gives an strategy constructed for this. Compress the complete information base offline by the lens of particular duties, cache these representations at a number of constancy ranges, and match the fitting compression tier to every question’s complexity.
The AWS implementation makes use of Amazon Bedrock for compression and inference, Amazon ElastiCache Serverless for caching, and a completely serverless structure that scales with demand. Entry the reference implementation, CDK infrastructure, and deployment scripts at aws-samples/sample-bedrock-takc-compression.
To get began, deploy the CDK stack with your individual paperwork and see how the system responds to totally different question varieties. In case your workload includes cross-document reasoning over steady information bases, TAKC can cut back token prices whereas enhancing response high quality.
References
Â
Concerning the authors
