Clever doc processing at scale with generative AI and Amazon Bedrock Knowledge Automation

Extracting info from unstructured paperwork at scale is a recurring enterprise job. Widespread use instances embrace creating product function tables from descriptions, extracting metadata from paperwork, and analyzing authorized contracts, buyer opinions, information articles, and extra. A traditional method to extracting info from textual content is called entity recognition (NER). NER identifies entities from predefined classes, reminiscent of individuals and organizations. Though varied AI providers and options help NER, this method is restricted to textual content paperwork and solely helps a set set of entities. Moreover, traditional NER fashions can’t deal with different information varieties reminiscent of numeric scores (reminiscent of sentiment) or free-form textual content (reminiscent of abstract). Generative AI unlocks these prospects with out pricey information annotation or mannequin coaching, enabling extra complete clever doc processing (IDP).

AWS just lately introduced the overall availability of Amazon Bedrock Knowledge Automation, a function of Amazon Bedrock that automates the technology of invaluable insights from unstructured multimodal content material reminiscent of paperwork, pictures, video, and audio. This service presents pre-built capabilities for IDP and knowledge extraction by way of a unified API, assuaging the necessity for advanced immediate engineering or fine-tuning, and making it a superb alternative for doc processing workflows at scale. To be taught extra about Amazon Bedrock Knowledge Automation, discuss with Simplify multimodal generative AI with Amazon Bedrock Knowledge Automation.

Amazon Bedrock Knowledge Automation is the advisable method for IDP use case attributable to its simplicity, industry-leading accuracy, and managed service capabilities. It handles the complexity of doc parsing, context administration, and mannequin choice mechanically, so builders can deal with their enterprise logic fairly than IDP implementation particulars.

Though Amazon Bedrock Knowledge Automation meets most IDP wants, some organizations require extra customization of their IDP pipelines. For instance, corporations may want to make use of self-hosted basis fashions (FMs) for IDP attributable to regulatory necessities. Some prospects have builder groups who may want to take care of full management over the IDP pipeline as a substitute of utilizing a managed service. Lastly, organizations may function in AWS Areas the place Amazon Bedrock Knowledge Automation isn’t obtainable (obtainable in us-west-2 and us-east-1 as of June 2025). In such instances, builders may use Amazon Bedrock FMs immediately or carry out optical character recognition (OCR) with Amazon Textract.

This submit presents an end-to-end IDP application powered by Amazon Bedrock Knowledge Automation and different AWS providers. It supplies a reusable AWS infrastructure as code (IaC) that deploys an IDP pipeline and supplies an intuitive UI for reworking paperwork into structured tables at scale. The applying solely requires the consumer to supply the enter paperwork (reminiscent of contracts or emails) and a listing of attributes to be extracted. It then performs IDP with generative AI.

The applying code and deployment directions are available on GitHub below the MIT license.

Answer overview

The IDP solution introduced on this submit is deployed as IaC utilizing the AWS Cloud Improvement Package (AWS CDK). Amazon Bedrock Knowledge Automation serves as the first engine for info extraction. For instances requiring additional customization, the answer additionally supplies various processing paths utilizing Amazon Bedrock FMs and Amazon Textract integration.

We use AWS Step Capabilities to orchestrate the IDP workflow and parallelize processing for a number of paperwork. As a part of the workflow, we use AWS Lambda features to name Amazon Bedrock Knowledge Automation or Amazon Textract and Amazon Bedrock (relying on the chosen parsing mode). Processed paperwork and extracted attributes are saved in Amazon Easy Storage Service (Amazon S3).

A Step Capabilities workflow with the enterprise logic is invoked by way of an API name carried out utilizing an AWS SDK. We additionally construct a containerized net utility operating on Amazon Elastic Container Service (Amazon ECS) that’s obtainable to end-users by way of Amazon CloudFront to simplify their interplay with the answer. We use Amazon Cognito for authentication and safe entry to the APIs.

The next diagram illustrates the structure and workflow of the IDP resolution.

The IDP workflow contains the next steps:

1. A consumer logs in to the net utility utilizing credentials managed by Amazon Cognito, selects enter paperwork, and defines the fields to be extracted from them within the UI. Optionally, the consumer can specify the parsing mode, LLM to make use of, and different settings.
2. The consumer begins the IDP pipeline.
3. The applying creates a pre-signed S3 URL for the paperwork and uploads them to Amazon S3.
4. The applying triggers Step Capabilities to start out the state machine with the S3 URIs and IDP settings as inputs. The Map state begins to course of the paperwork concurrently.
5. Relying on the doc kind and the parsing mode, it branches to totally different Lambda features that carry out IDP, save outcomes to Amazon S3, and ship them again to the UI:
   1. Amazon Bedrock Knowledge Automation – Paperwork are directed to the “Run Knowledge Automation” Lambda perform. The Lambda perform creates a blueprint with the user-defined fields schema and launches an asynchronous Amazon Bedrock Knowledge Automation job. Amazon Bedrock Knowledge Automation handles the complexity of doc processing and attribute extraction utilizing optimized prompts and fashions. When the job outcomes are prepared, they’re saved to Amazon S3 and despatched again to the UI. This method supplies the perfect steadiness of accuracy, ease of use, and scalability for many IDP use instances.
   2. Amazon Textract – If the consumer specifies Amazon Textract as a parsing mode, the IDP pipeline splits into two steps. First, the “Carry out OCR” Lambda perform is invoked to run an asynchronous doc evaluation job. The OCR outputs are processed utilizing the amazon-textract-textractor library and formatted as Markdown. Second, the textual content is handed to the “Extract attributes” Lambda perform (Step 6), which invokes an Amazon Bedrock FM given the textual content and the attributes schema. The outputs are saved to Amazon S3 and despatched to the UI.
   3. Dealing with workplace paperwork – Paperwork with suffixes like .doc, .ppt, and .xls are processed by the “Parse workplace” Lambda perform, which makes use of LangChain doc loaders to extract the textual content content material. The outputs are handed to the “Extract attributes” Lambda perform (Step 6) to proceed with the IDP pipeline.
6. If the consumer chooses an Amazon Bedrock FM for IDP, the doc is distributed to the “Extract attributes” Lambda perform. It converts a doc right into a set of pictures, that are despatched to a multimodal FM with the attributes schema as a part of a customized immediate. It parses the LLM response to extract JSON outputs, saves them to Amazon S3, and sends it again to the UI. This move helps .pdf, .png, and .jpg paperwork.
7. The online utility checks the state machine execution outcomes periodically and returns the extracted attributes to the consumer when they’re obtainable.

Stipulations

You’ll be able to deploy the IDP resolution out of your native laptop or from an Amazon SageMaker pocket book occasion. The deployment steps are detailed within the solution README file.

If you happen to select to deploy utilizing a SageMaker pocket book, which is advisable, you’ll need entry to an AWS account with permissions to create and launch a SageMaker pocket book occasion.

Deploy the answer

To deploy the answer to your AWS account, full the next steps:

1. Open the AWS Administration Console and select the Area during which you need to deploy the IDP resolution.
2. Launch a SageMaker pocket book occasion. Present the pocket book occasion identify and pocket book occasion kind, which you’ll set to ml.m5.massive. Depart different choices as default.
3. Navigate to the Pocket book occasion and open the IAM position connected tothe pocket book. Open the position on the AWS Identification and Entry Administration (IAM) console.
4. Connect an inline coverage to the position and insert the next coverage JSON:

{
  "Model": "2012-10-17",
  "Assertion": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudformation:*",
        "s3:*",
        "iam:*",
        "sts:AssumeRole"
      ],
      "Useful resource": "*"
    },
    {
      "Impact": "Permit",
      "Motion": [
        "ssm:GetParameter",
        "ssm:GetParameters"
      ],
      "Useful resource": "arn:aws:ssm:*:*:parameter/cdk-bootstrap/*"
    }
  ]
}

6. When the pocket book occasion standing is marked as InService, select Open JupyterLab.
7. Within the JupyterLab atmosphere, select File, New, and Terminal.
8. Clone the solution repository by operating the next instructions:

cd SageMaker
git clone https://github.com/aws-samples/intelligent-document-processing-with-amazon-bedrock.git

9. Navigate to the repository folder and run the script to put in necessities:

cd intelligent-document-processing-with-amazon-bedrock
sh install_deps.sh

10. Run the script to create a digital atmosphere and set up dependencies:

sh install_env.sh
supply .venv/bin/activate

11. Inside the repository folder, copy the config-example.yml to a config.yml to specify your stack identify. Optionally, configure the providers and point out the modules you need to deploy (for instance, to disable deploying a UI, change deploy_streamlit to False). Ensure you add your consumer e mail to the Amazon Cognito customers listing.
12. Configure Amazon Bedrock mannequin entry by opening the Amazon Bedrock console within the Area specified within the config.yml file. Within the navigation pane, select Mannequin Entry and ensure to allow entry for the mannequin IDs laid out in config.yml.
13. Bootstrap and deploy the AWS CDK in your account:

Notice that this step could take a while, particularly on the primary deployment. As soon as deployment is full, you need to see the message as proven within the following screenshot. You’ll be able to entry the Streamlit frontend utilizing the CloudFront distribution URL supplied within the AWS CloudFormation outputs. The short-term login credentials will probably be despatched to the e-mail laid out in config.yml in the course of the deployment.

Utilizing the answer

This part guides you thru two examples to showcase the IDP capabilities.

Instance 1: Analyzing monetary paperwork

On this state of affairs, we extract key options from a multi-page monetary assertion utilizing Amazon Bedrock Knowledge Automation. We use a sample document in PDF format with a mix of tables, pictures, and textual content, and extract a number of monetary metrics. Full the next steps:

1. Add a doc by attaching a file by way of the answer UI.

2. On the Describe Attributes tab, both manually listing the names and descriptions of the attributes or upload these fields in JSON format. We need to discover the next metrics:
   1. Present money in belongings in 2018
   2. Present money in belongings in 2019
   3. Working revenue in 2018
   4. Working revenue in 2019

3. Select Extract attributes to start out the IDP pipeline.

The supplied attributes are built-in right into a customized blueprint with the inferred attributes listing, which is then used to invoke an information automation job on the uploaded paperwork.

After the IDP pipeline is full, you will note a desk of ends in the UI. It contains an index for every doc within the _doc column, a column for every of the attributes you outlined, and a file_name column that accommodates the doc identify.

From the next assertion excerpts, we will see that Amazon Bedrock Knowledge Automation was capable of appropriately extract the values for present belongings and working revenue.

The IDP resolution can be capable of do advanced calculations past well-defined entities. Let’s say we need to calculate the next accounting metrics:

• Liquidity ratios (Present belongings/Present liabilities)
• Working capitals (Present belongings – Present liabilities)
• Income enhance ((Income yr 2/Income yr 1) – 1)

We outline the attributes and their formulation as elements of the attributes’ schema. This time, we select an Amazon Bedrock LLM as a parsing mode to display how the applying can use a multimodal FM for IDP. When utilizing an Amazon Bedrock LLM, beginning the IDP pipeline will now mix the attributes and their description right into a custom prompt template, which is distributed to the LLM with the paperwork transformed to photographs. As a consumer, you may specify the LLM powering the extraction and its inference parameters, reminiscent of temperature.

The output, together with the complete outcomes, is proven within the following screenshot.

Instance 2: Processing buyer emails

On this state of affairs, we need to extract a number of options from a listing of emails with buyer complaints attributable to delays in product shipments utilizing Amazon Bedrock Knowledge Automation. For every e mail, we need to discover the next:

• Buyer identify
• Cargo ID
• Electronic mail language
• Electronic mail sentiment
• Cargo delay (in days)
• Abstract of situation
• Instructed response

Full the next steps:

1. Add enter emails as .txt recordsdata. You’ll be able to obtain pattern emails from GitHub.

2. On the Describe Attributes tab, listing names and descriptions of the attributes.

You’ll be able to add few-shot examples for some fields (reminiscent of delay) to clarify to the LLM how these fields values ought to be extracted. You are able to do this by including an instance enter and the anticipated output for the attribute to the outline.

3. Select Extract attributes to start out the IDP pipeline.

The supplied attributes and their descriptions will probably be built-in right into a customized blueprint with the inferred attributes listing, which is then used to invoke an information automation job on the uploaded paperwork. When the IDP pipeline is full, you will note the outcomes.

The applying permits downloading the extraction outcomes as a CSV or a JSON file. This makes it easy to make use of the outcomes for downstream duties, reminiscent of aggregating buyer sentiment scores.

Pricing

On this part, we calculate value estimates for performing IDP on AWS with our resolution.

Amazon Bedrock Knowledge Automation supplies a clear pricing schema relying on the enter doc measurement (variety of pages, pictures, or minutes). When utilizing Amazon Bedrock FMs, pricing will depend on the variety of enter and output tokens used as a part of the knowledge extraction name. Lastly, when utilizing Amazon Textract, OCR is carried out and priced individually based mostly on the variety of pages within the paperwork.

Utilizing the previous situations as examples, we will approximate the prices relying on the chosen parsing mode. Within the following desk, we present prices utilizing two datasets: 100 20-page monetary paperwork, and 100 1-page buyer emails. We ignore prices of Amazon ECS and Lambda.

The price evaluation means that utilizing Amazon Bedrock FMs with a customized immediate template is an economical methodology for IDP. Nevertheless, this method requires a much bigger operational overhead, as a result of the pipeline must be optimized relying on the LLM, and requires guide safety and privateness administration. Amazon Bedrock Knowledge Automation presents a managed service that makes use of a alternative of high-performing FMs by way of a single API.

Clear up

To take away the deployed assets, full the next steps:

1. On the AWS CloudFormation console, delete the created stack. Alternatively, run the next command:

   cdk destroy --region <YOUR_DEPLOY_REGION>

2. On the Amazon Cognito console, delete the consumer pool.

Conclusion

Extracting info from unstructured paperwork at scale is a recurring enterprise job. This submit mentioned an end-to-end IDP utility that performs info extraction utilizing a number of AWS providers. The answer is powered by Amazon Bedrock Knowledge Automation, which supplies a totally managed service for producing insights from paperwork, pictures, audio, and video. Amazon Bedrock Knowledge Automation handles the complexity of doc processing and knowledge extraction, optimizing for each efficiency and accuracy with out requiring experience in immediate engineering. For prolonged flexibility and customizability in particular situations, our resolution additionally helps IDP utilizing Amazon Bedrock customized LLM calls and Amazon Textract for OCR.

The answer helps a number of doc varieties, together with textual content, pictures, PDF, and Microsoft Workplace paperwork. On the time of writing, correct understanding of knowledge in paperwork wealthy with pictures, tables, and different visible components is barely obtainable for PDF and pictures. We suggest changing advanced Workplace paperwork to PDFs or pictures for greatest efficiency. One other resolution limitation is the doc measurement. As of June 2025, Amazon Bedrock Knowledge Automation helps paperwork as much as 20 pages for customized attributes extraction. When utilizing customized Amazon Bedrock LLMs for IDP, the 300,000-token context window of Amazon Nova LLMs permits processing paperwork with as much as roughly 225,000 phrases. To extract info from bigger paperwork, you’d at present want to separate the file into a number of paperwork.

Within the subsequent variations of the IDP resolution, we plan to maintain including help for state-of-the-art language fashions obtainable by way of Amazon Bedrock and iterate on immediate engineering to additional enhance the extraction accuracy. We additionally plan to implement methods for extending the dimensions of supported paperwork and offering customers with a exact indication of the place precisely within the doc the extracted info is coming from.

To get began with IDP with the described resolution, discuss with the GitHub repository. To be taught extra about Amazon Bedrock, discuss with the documentation.

Concerning the authors

Nikita Kozodoi, PhD, is a Senior Utilized Scientist on the AWS Generative AI Innovation Middle, the place he works on the frontier of AI analysis and enterprise. With wealthy expertise in Generative AI and numerous areas of ML, Nikita is captivated with utilizing AI to resolve difficult real-world enterprise issues throughout industries.

Zainab Afolabi is a Senior Knowledge Scientist on the Generative AI Innovation Centre in London, the place she leverages her in depth experience to develop transformative AI options throughout numerous industries. She has over eight years of specialized expertise in synthetic intelligence and machine studying, in addition to a ardour for translating advanced technical ideas into sensible enterprise functions.

Aiham Taleb, PhD, is a Senior Utilized Scientist on the Generative AI Innovation Middle, working immediately with AWS enterprise prospects to leverage Gen AI throughout a number of high-impact use instances. Aiham has a PhD in unsupervised illustration studying, and has {industry} expertise that spans throughout varied machine studying functions, together with laptop imaginative and prescient, pure language processing, and medical imaging.

Liza (Elizaveta) Zinovyeva is an Utilized Scientist at AWS Generative AI Innovation Middle and is predicated in Berlin. She helps prospects throughout totally different industries to combine Generative AI into their current functions and workflows. She is captivated with AI/ML, finance and software program safety matters. In her spare time, she enjoys spending time together with her household, sports activities, studying new applied sciences, and desk quizzes.

Nuno Castro is a Sr. Utilized Science Supervisor at AWS Generative AI Innovation Middle. He leads Generative AI buyer engagements, serving to a whole bunch of AWS prospects discover probably the most impactful use case from ideation, prototype by way of to manufacturing. He has 19 years expertise in AI in industries reminiscent of finance, manufacturing, and journey, main AI/ML groups for 12 years.

Ozioma Uzoegwu is a Principal Options Architect at Amazon Net Companies. In his position, he helps monetary providers prospects throughout EMEA to remodel and modernize on the AWS Cloud, offering architectural steering and {industry} greatest practices. Ozioma has a few years of expertise with net improvement, structure, cloud and IT administration. Previous to becoming a member of AWS, Ozioma labored with an AWS Superior Consulting Associate because the Lead Architect for the AWS Follow. He’s captivated with utilizing newest applied sciences to construct a contemporary monetary providers IT property throughout banking, fee, insurance coverage and capital markets.

Eren Tuncer is a Options Architect at Amazon Net Companies targeted on Serverless and constructing Generative AI functions. With greater than fifteen years expertise in software program improvement and structure, he helps prospects throughout varied industries obtain their enterprise targets utilizing cloud applied sciences with greatest practices. As a builder, he’s captivated with creating options with state-of-the-art applied sciences, sharing information, and serving to organizations navigate cloud adoption.

Francesco Cerizzi is a Options Architect at Amazon Net Companies exploring tech frontiers whereas spreading generative AI information and constructing functions. With a background as a full stack developer, he helps prospects throughout totally different industries of their journey to the cloud, sharing insights on AI’s transformative potential alongside the best way. He’s captivated with Serverless, event-driven architectures, and microservices usually. When not diving into expertise, he’s an enormous F1 fan and loves Tennis.