Constructing RAG-based generative AI purposes on AWS utilizing Amazon Bedrock and Amazon FSx for NetApp ONTAP

This put up was co-authored by Michael Shaul and Sasha Korman of NetApp.

Generative synthetic intelligence (AI) purposes are generally constructed utilizing a method known as Retrieval Augmented Era (RAG). RAG gives the underlying mannequin (FM) with entry to extra information that was not obtainable throughout coaching. This information is used to counterpoint the generative AI prompts, offering extra context-specific and correct responses with out regularly retraining the FM, enhancing transparency and minimizing hallucinations.

On this put up, we current an answer utilizing Amazon FSx for NetApp ONTAP with Amazon Bedrock to offer a RAG expertise for generative AI purposes on AWS by ingesting enterprise-specific unstructured consumer file information into Amazon Bedrock in a straightforward, quick, and safe means.

Our answer makes use of FSx for ONTAP file programs as a supply of unstructured information to constantly populate an Amazon OpenSearch Serverless vector database with customers’ current information and folders and related metadata, which then enriches generative AI prompts utilizing Amazon Bedrock APIs to allow RAG situations with enterprise-specific information retrieved from the OpenSearch Serverless vector database.

When utilizing RAG to develop generative AI purposes corresponding to Q&A chatbots, prospects are additionally involved about sustaining information safety and making certain finish customers can’t question data from unauthorized information sources. Our answer makes use of FSx for ONTAP to allow customers to increase their present information safety and entry mechanisms to reinforce mannequin responses from Amazon Bedrock. We use FSx for ONTAP as a supply of related metadata, particularly the safety entry management record (ACL) configurations of customers hooked up to information and folders, to populate that metadata into OpenSearch Serverless. By combining entry management operations with file occasions that notify the RAG software of latest and adjusted information on the file system, our answer reveals how FSx for ONTAP allows Amazon Bedrock to make sure that embeds from approved information are solely obtainable to particular customers connecting to our generative AI software.

AWS serverless companies present automated scaling, built-in excessive availability, and a pay-per-use mannequin, making it simpler to concentrate on constructing generative AI purposes. Occasion-driven computing with AWS Lambda is well-suited for compute-intensive, on-demand duties corresponding to doc embedding and versatile large-scale language mannequin (LLM) orchestration, and Amazon API Gateway gives an API interface that permits pluggable frontends and event-driven invocation of LLMs. Our answer additionally reveals the best way to use API Gateway and Lambda to construct a scalable, automated, API-driven serverless software layer on prime of Amazon Bedrock and FSx for ONTAP.

Answer overview

This answer is FSx for ONTAP Multi-AZ A file system with storage digital machines (SVMs) joined to an AWS managed Microsoft AD area. The OpenSearch serverless vector search assortment gives scalable, high-performance similarity search capabilities. Use an Amazon Elastic Compute Cloud (Amazon EC2) Home windows server as an SMB/CIFS shopper for the FSx for ONTAP quantity and configure information sharing and ACLs for the SMB shares within the quantity. Use this information and ACLs to check permission-based entry to embedding in a RAG state of affairs with Amazon Bedrock.

The embedded container element of our answer is deployed on an EC2 Linux server and mounted as an NFS shopper on an FSx for ONTAP quantity. We periodically migrate current information and folders, together with their safety ACL settings, to OpenSearch Serverless. We populate the OpenSearch Serverless vector search assortment index with company-specific information (and related metadata and ACLs) from the NFS share on the FSx for ONTAP file system.

The answer implements a RAG Retrieval Lambda perform that allows RAG on Amazon Bedrock through the use of company-specific information and related metadata (together with ACLs) retrieved from an OpenSearch Serverless index populated by an embedded container element to energy generative AI prompts utilizing the Amazon Bedrock API. The RAG Retrieval Lambda perform shops the dialog historical past of consumer interactions in an Amazon DynamoDB desk.

Finish customers work together with the answer by sending pure language prompts by the chatbot software or API gateway interface. The chatbot software container Streamlyt is fronted by an AWS Utility Load Balancer (ALB). When a consumer makes use of the ALB to ship a pure language immediate to the chatbot UI, the chatbot container interacts with the API Gateway interface, which then invokes the RAG Get Lambda perform to get the consumer’s response. Customers can even ship immediate requests on to API Gateway to get the response. You point out permission-based entry to the RAG doc by explicitly getting the consumer’s SID and utilizing that SID within the chatbot or API Gateway request. The RAG Get Lambda perform then matches the SID in opposition to the Home windows ACLs configured for the doc. As a further authentication step in a manufacturing atmosphere, you can even authenticate the consumer in opposition to an identification supplier and match the consumer in opposition to the permissions configured for the doc.

The next diagram reveals the end-to-end circulate of our answer. First, we use FSx for ONTAP to arrange information shares and ACLs, then periodically scan these with an embedding container. The embedding container splits paperwork into chunks and creates vector embeddings from these chunks utilizing the Amazon Titan Embeddings mannequin. We then retailer these vector embeddings together with related metadata in a vector database by indexing a vector assortment in OpenSearch Serverless. The next diagram reveals the end-to-end circulate.

The next structure diagram reveals the completely different elements of our answer.

Conditions

Full the next prerequisite steps:

1. Ensure your mannequin is accessible in Amazon Bedrock. This answer makes use of Anthropic Claude v3 Sonnet On Amazon Bedrock.
2. Set up the AWS Command Line Interface (AWS CLI).
3. Install Docker.
4. Install Terraform.

Deploy the answer

You possibly can obtain this answer right here GitHub repositoryBy cloning the repository and utilizing the Terraform templates, all of the elements are provisioned with the required configuration.

1. Clone this answer repository.

   sudo yum set up -y unzip
   git clone https://github.com/aws-samples/genai-bedrock-fsxontap.git
   cd genai-bedrock-fsxontap/terraform

2. From the terraform folder, deploy the complete answer with Terraform.

   terraform init
   terraform apply -auto-approve

This course of can take 15-20 minutes to finish, and as soon as full the output of the terraform command will look much like the next:

api-invoke-url = "https://9ng1jjn8qi.execute-api.<area>.amazonaws.com/prod"
fsx-management-ip = toset([
"198.19.255.230",])
fsx-secret-id = "arn:aws:secretsmanager:<area>:<account-id>:secret:AmazonBedrock-FSx-NetAPP-ONTAP-a2fZEdIt-0fBcS9"
fsx-svm-smb-dns-name = "BRSVM.BEDROCK-01.COM"
lb-dns-name = "chat-load-balancer-2040177936.<area>.elb.amazonaws.com"

Load information and set permissions

To check the answer, it’s good to create an EC2 Home windows server (ad_host) to mount the FSx for ONTAP quantity as an SMB/CIFS shopper to share pattern information and set consumer permissions that will likely be utilized by the embedded container element of the answer to populate the OpenSearch Serverless index. Observe these steps to mount the FSx for ONTAP SVM information quantity as a community drive, add information to this shared community drive, and set permissions primarily based on Home windows ACLs.

1. Acquiring ad_host Get the occasion DNS from the output of the Terraform template.
2. Go to AWS Programs Supervisor Fleet Supervisor within the AWS console. ad_host Create an occasion and comply with the directions right here to log in with Distant Desktop. Use a site admin consumer bedrock-01Admin Get the password from AWS Secrets and techniques Supervisor. You’ll find the password utilizing Secrets and techniques Supervisor. fsx-secret-id The key ID from the output of your Terraform template.
3. To mount an FSx for ONTAP information quantity as a community drive, This PCChoose (proper click on) community And select Mapping a Community Drive.
4. Choose a drive letter and use the FSx for ONTAP share path for mounting.
   (<svm>.<area >c$<volume-name>):
   
5. Add the Amazon Bedrock Consumer Information to a shared community drive and set permissions to solely the admin consumer (be sure to disable inheritance). Superior):
6. Add the Amazon FSx for ONTAP Consumer Information to the shared drive and make sure that the permissions are set. everybody:
7. Above ad_host On the server, open a command immediate and kind the next command to get the SID for the Administrator consumer:

   wmic useraccount the place title="Admin" get sid

Use a chatbot to check permissions

To check permissions utilizing the chatbot, lb-dns-name Get the URL from the output of the Terraform template and entry it from an online browser.

The immediate question asks a typical query within the publicly accessible FSx for ONTAP Consumer Information. In our state of affairs, we requested, “How do I create an FSx for ONTAP file system?” and the mannequin replied in a chat window with detailed directions and supply attributes for creating an FSx for ONTAP file system utilizing the AWS Administration Console, AWS CLI, or FSx API.

Now, let’s ask a query in regards to the Amazon Bedrock Consumer Information, which is just obtainable to administrator entry. On this state of affairs, you ask “How do I take advantage of foundational fashions with Amazon Bedrock?” and the mannequin responds that it has inadequate data to offer an in depth reply to your query.

Use the admin SID within the Consumer (SID) filter search within the chat UI and ask the identical query within the immediate, this time replying with detailed directions on how your mannequin makes use of FM with Amazon Bedrock and offering the supply attribute your mannequin used within the response.

Testing permissions utilizing API Gateway

It’s also possible to use API Gateway to question the mannequin instantly. api-invoke-url Parameters from the output of the Terraform template.

curl -v '<api-invoke-url>/bedrock_rag_retreival' -X POST -H 'content-type: software/json' -d '{"session_id": "1","immediate": "What's an FSxN ONTAP filesystem?", "bedrock_model_id": "anthropic.claude-3-sonnet-20240229-v1:0", "model_kwargs": {"temperature": 1.0, "top_p": 1.0, "top_k": 500}, "metadata": "NA", "memory_window": 10}'

Subsequent, begin the API Gateway. everybody To entry queries associated to the FSx for ONTAP consumer information, set the metadata parameter worth to NA and everybody entry:

curl -v '<api-invoke-url>/bedrock_rag_retreival' -X POST -H 'content-type: software/json' -d '{"session_id": "1","immediate": "what's bedrock?", "bedrock_model_id": "anthropic.claude-3-sonnet-20240229-v1:0", "model_kwargs": {"temperature": 1.0, "top_p": 1.0, "top_k": 500}, "metadata": "S-1-5-21-4037439088-1296877785-2872080499-1112", "memory_window": 10}'

cleansing

To keep away from recurring fees, clear up your account after you’ve got tried the answer: Delete the answer’s Terraform templates from the terraform folder.

terraform apply --destroy

Conclusion

On this put up, we launched an answer that makes use of FSx for ONTAP with Amazon Bedrock and FSx for ONTAP file possession and ACL help to offer permission-based entry in RAG situations for generative AI purposes. Our answer allows you to construct generative AI purposes on Amazon Bedrock and enrich generative AI prompts in Amazon Bedrock with company-specific unstructured consumer file information from FSx for ONTAP file programs. With this answer, you possibly can present extra related, context-specific, and correct responses whereas making certain that solely approved customers have entry to that information. Lastly, this answer demonstrates the usage of AWS serverless companies with FSx for ONTAP and Amazon Bedrock to allow autoscaling, event-driven computing, and API interfaces for generative AI purposes on AWS.

To study extra about how one can get began constructing with Amazon Bedrock and FSx for ONTAP, see the next sources:

In regards to the Writer

Kanishk Mahajan He’s a Principal of Options Structure at AWS. He leads cloud transformation and options structure for AWS ISV prospects and companions. He focuses on containers, cloud operations, migration and modernization, AI/ML, resiliency, safety, and compliance. He’s additionally a Technical Discipline Group (TFC) member for every of AWS’ domains.

Michael Scholl He’s a Principal Architect within the CTO’s Workplace at NetApp and has over 20 years of expertise constructing information administration programs, purposes, and infrastructure options. He has distinctive and deep perception into cloud applied sciences, builders, and AI options.

Sasha Corman He’s the technical chief of a dynamic growth and QA workforce unfold throughout Israel and India. With 14 years of expertise at NetApp as a programmer, he has hands-on expertise and management in efficiently delivering advanced tasks with a concentrate on innovation, scalability and reliability.