Construct a generative AI Slack chat assistant utilizing Amazon Bedrock and Amazon Kendra

Regardless of the proliferation of knowledge and information in enterprise environments, staff and stakeholders usually discover themselves trying to find data and struggling to get their questions answered rapidly and effectively. This may result in productiveness losses, frustration, and delays in decision-making.

A generative AI Slack chat assistant may help handle these challenges by offering a available, clever interface for customers to work together with and procure the knowledge they want. Through the use of the pure language processing and era capabilities of generative AI, the chat assistant can perceive consumer queries, retrieve related data from varied information sources, and supply tailor-made, contextual responses.

By harnessing the ability of generative AI and Amazon Net Providers (AWS) providers Amazon Bedrock, Amazon Kendra, and Amazon Lex, this answer offers a pattern structure to construct an clever Slack chat assistant that may streamline data entry, improve consumer experiences, and drive productiveness and effectivity inside organizations.

Why use Amazon Kendra for constructing a RAG software?

Amazon Kendra is a completely managed service that gives out-of-the-box semantic search capabilities for state-of-the-art rating of paperwork and passages. You should use Amazon Kendra to rapidly construct high-accuracy generative AI purposes on enterprise information and supply probably the most related content material and paperwork to maximise the standard of your Retrieval Augmented Era (RAG) payload, yielding higher giant language mannequin (LLM) responses than utilizing standard or keyword-based search options. Amazon Kendra provides simple-to-use deep studying search fashions which are pre-trained on 14 domains and don’t require machine studying (ML) experience. Amazon Kendra can index content material from a variety of sources, together with databases, content material administration techniques, file shares, and net pages.

Additional, the FAQ characteristic in Amazon Kendra enhances the broader retrieval capabilities of the service, permitting the RAG system to seamlessly change between offering prewritten FAQ responses and dynamically producing responses by querying the bigger information base. This makes it well-suited for powering the retrieval part of a RAG system, permitting the mannequin to entry a broad information base when producing responses. By integrating the FAQ capabilities of Amazon Kendra right into a RAG system, the mannequin can use a curated set of high-quality, authoritative solutions for generally requested questions. This may enhance the general response high quality and consumer expertise, whereas additionally lowering the burden on the language mannequin to generate these primary responses from scratch.

This answer balances retaining customizations by way of mannequin choice, immediate engineering, and including FAQs with not having to take care of phrase embeddings, doc chunking, and different lower-level complexities sometimes required for RAG implementations.

Resolution overview

The chat assistant is designed to help customers by answering their questions and offering data on a wide range of matters. The aim of the chat assistant is to be an internal-facing Slack device that may assist staff and stakeholders discover the knowledge they want.

The structure makes use of Amazon Lex for intent recognition, AWS Lambda for processing queries, Amazon Kendra for looking by FAQs and net content material, and Amazon Bedrock for producing contextual responses powered by LLMs. By combining these providers, the chat assistant can perceive pure language queries, retrieve related data from a number of information sources, and supply humanlike responses tailor-made to the consumer’s wants. The answer showcases the ability of generative AI in creating clever digital assistants that may streamline workflows and improve consumer experiences based mostly on mannequin decisions, FAQs, and modifying system prompts and inference parameters.

Structure diagram

The next diagram illustrates a RAG strategy the place the consumer sends a question by the Slack software and receives a generated response based mostly on the info listed in Amazon Kendra. On this submit, we use Amazon Kendra Net Crawler as the info supply and embody FAQs saved on Amazon Easy Storage Service (Amazon S3). See Knowledge supply connectors for an inventory of supported information supply connectors for Amazon Kendra.

The step-by-step workflow for the structure is the next:

1. The consumer sends a question resembling What's the AWS Properly-Architected Framework? by the Slack app.
2. The question goes to Amazon Lex, which identifies the intent.
3. At present two intents are configured in Amazon Lex (Welcome and FallbackIntent).
4. The welcome intent is configured to reply with a greeting when a consumer enters a greeting resembling “hello” or “good day.” The assistant responds with “Good day! I may help you with queries based mostly on the paperwork offered. Ask me a query.”
5. The fallback intent is fulfilled with a Lambda operate.
   1. The Lambda operate searches Amazon Kendra FAQs by the search_Kendra_FAQ technique by taking the consumer question and Amazon Kendra index ID as inputs. If there’s a match with a excessive confidence rating, the reply from the FAQ is returned to the consumer.

      def search_Kendra_FAQ(query, kendra_index_id):
          """
          This operate takes within the query from the consumer, and checks if the query exists within the Kendra FAQs.
          :param query: The query the consumer is asking that was requested through the frontend enter textual content field.
          :param kendra_index_id: The kendra index containing the paperwork and FAQs
          :return: If present in FAQs, returns the reply together with any related hyperlinks. If not, returns False after which calls kendra_retrieve_document operate.
          """
          kendra_client = boto3.shopper('kendra')
          response = kendra_client.question(IndexId=kendra_index_id, QueryText=query, QueryResultTypeFilter="QUESTION_ANSWER")
          for merchandise in response['ResultItems']:
              score_confidence = merchandise['ScoreAttributes']['ScoreConfidence']
              # Taking solutions from FAQs which have a really excessive confidence rating solely
              if score_confidence == 'VERY_HIGH' and len(merchandise['AdditionalAttributes']) > 1:
                  textual content = merchandise['AdditionalAttributes'][1]['Value']['TextWithHighlightsValue']['Text']
                  url = "None"
                  if merchandise['DocumentURI'] != '':
                      url = merchandise['DocumentURI']
                  return (textual content, url)
          return (False, False)

   2. If there isn’t a match with a excessive sufficient confidence rating, related paperwork from Amazon Kendra with a excessive confidence rating are retrieved by the kendra_retrieve_document technique and despatched to Amazon Bedrock to generate a response because the context.

      def kendra_retrieve_document(query, kendra_index_id):
          """
          This operate takes within the query from the consumer, and retrieves related passages based mostly on default PageSize of 10.
          :param query: The query the consumer is asking that was requested through the frontend enter textual content field.
          :param kendra_index_id: The kendra index containing the paperwork and FAQs
          :return: Returns the context to be despatched to the LLM and doc URIs to be returned as related information sources.
          """
          kendra_client = boto3.shopper('kendra')
          paperwork = kendra_client.retrieve(IndexId=kendra_index_id, QueryText=query)
          textual content = ""
          uris = set()
          if len(paperwork['ResultItems']) > 0:
              for i in vary(len(paperwork['ResultItems'])):
                  score_confidence = paperwork['ResultItems'][i]['ScoreAttributes']['ScoreConfidence']
                  if score_confidence == 'VERY_HIGH' or score_confidence == 'HIGH':
                      textual content += paperwork['ResultItems'][i]['Content'] + "n"
                      uris.add(paperwork['ResultItems'][i]['DocumentURI'])
          return (textual content, uris)

   3. The response is generated from Amazon Bedrock with the invokeLLM technique. The next is a snippet of the invokeLLM technique throughout the success operate. Learn extra on inference parameters and system prompts to change parameters which are handed into the Amazon Bedrock invoke mannequin request.

      def invokeLLM(query, context, modelId):
          """
          This operate takes within the query from the consumer, together with the Kendra responses as context to generate a solution
          for the consumer on the frontend.
          :param query: The query the consumer is asking that was requested through the frontend enter textual content field.
          :param paperwork: The response from the Kendra doc retrieve question, used as context to generate a greater
          reply.
          :return: Returns the ultimate reply that might be offered to the end-user of the appliance who requested the unique
          query.
          """
          # Setup Bedrock shopper
          bedrock = boto3.shopper('bedrock-runtime')
          # configure mannequin specifics resembling particular mannequin
          modelId = modelId
      
          # physique of information with parameters that's handed into the bedrock invoke mannequin request
          physique = json.dumps({"max_tokens": 350,
                  "system": "You're a truthful AI assistant. Your purpose is to offer informative and substantive responses to queries based mostly on the paperwork offered. When you have no idea the reply to a query, you honestly say you have no idea.",
                  "messages": [{"role": "user", "content": "Answer this user query:" + question + "with the following context:" + context}],
                  "anthropic_version": "bedrock-2023-05-31",
                      "temperature":0,
                  "top_k":250,
                  "top_p":0.999})
      
          # Invoking the bedrock mannequin along with your specs
          response = bedrock.invoke_model(physique=physique,
                                          modelId=modelId)
          # the physique of the response that was generated
          response_body = json.masses(response.get('physique').learn())
          # retrieving the precise completion subject, the place you reply might be
          reply = response_body.get('content material')
          # returning the reply as a remaining consequence, which finally will get returned to the top consumer
          return reply

   4. Lastly, the response generated from Amazon Bedrock together with the related referenced URLs are returned to the top consumer.

   When deciding on web sites to index, adhere to the AWS Acceptable Use Coverage and different AWS phrases. Keep in mind which you could solely use Amazon Kendra Net Crawler to index your individual net pages or net pages that you’ve authorization to index. Go to the Amazon Kendra Net Crawler information supply information to be taught extra about utilizing the net crawler as a knowledge supply. Utilizing Amazon Kendra Net Crawler to aggressively crawl web sites or net pages you don’t personal is not thought-about acceptable use.

   Supported options

   The chat assistant helps the next options:

   1. Help for the next Anthropic’s fashions on Amazon Bedrock:
      • claude-v2
      • claude-3-haiku-20240307-v1:0
      • claude-instant-v1
      • claude-3-sonnet-20240229-v1:0
   2. Help for FAQs and the Amazon Kendra Net Crawler information supply
   3. Returns FAQ solutions provided that the arrogance rating is VERY_HIGH
   4. Retrieves solely paperwork from Amazon Kendra which have a HIGH or VERY_HIGH confidence rating
   5. If paperwork with a excessive confidence rating aren’t discovered, the chat assistant returns “No related paperwork discovered”

   Conditions

   To carry out the answer, you might want to have following stipulations:

   • Primary information of AWS
   • An AWS account with entry to Amazon S3 and Amazon Kendra
   • An S3 bucket to retailer your paperwork. For extra data, see Step 1: Create your first S3 bucket and the Amazon S3 Consumer Information.
   • A Slack workspace to combine the chat assistant
   • Permission to put in Slack apps in your Slack workspace
   • Seed URLs for the Amazon Kendra Net Crawler information supply
     • You’ll want authorization to crawl and index any web sites offered
   • AWS CloudFormation for deploying the answer sources

   Construct a generative AI Slack chat assistant

   To construct a Slack software, use the next steps:

   1. Request mannequin entry on Amazon Bedrock for all Anthropic fashions
   2. Create an S3 bucket within the us-east-1 (N. Virginia) AWS Area.
   3. Add the AIBot-LexJson.zip and SampleFAQ.csv information to the S3 bucket
   4. Launch the CloudFormation stack within the us-east-1 (N. Virginia) AWS Area.
   5. Enter a Stack title of your alternative
   6. For S3BucketName, enter the title of the S3 bucket created in Step 2
   7. For S3KendraFAQKey, enter the title of the SampleFAQs uploaded to the S3 bucket in Step 3
   8. For S3LexBotKey, enter the title of the Amazon Lex .zip file uploaded to the S3 bucket in Step 3
   9. For SeedUrls, enter as much as 10 URLs for the net crawler as a comma delimited record. Within the instance on this submit, we give the publicly obtainable Amazon Bedrock service web page because the seed URL
   10. Depart the remainder as defaults. Select Subsequent. Select Subsequent once more on the Configure stack choices
   11. Acknowledge by deciding on the field and select Submit, as proven within the following screenshot
       
   12. Look ahead to the stack creation to finish
   13. Confirm all sources are created
   14. Check on the AWS Administration Console for Amazon Lex
       1. On the Amazon Lex console, select your chat assistant ${YourStackName}-AIBot
       2. Select Intents
       3. Select Model 1 and select Check, as proven within the following screenshot
          
       4. Choose the AIBotProdAlias and select Verify, as proven within the following screenshot. If you wish to make modifications to the chat assistant, you need to use the draft model, publish a brand new model, and assign the brand new model to the AIBotProdAlias. Be taught extra about Versioning and Aliases.
          
       5. Check the chat assistant with questions resembling, “Which AWS service has 11 nines of sturdiness?” and “What’s the AWS Properly-Architected Framework?” and confirm the responses. The next desk exhibits that there are three FAQs within the pattern .csv file.
          

          _question	_answer	_source_uri
          Which AWS service has 11 nines of sturdiness?	Amazon S3	https://aws.amazon.com/s3/
          What’s the AWS Properly-Architected Framework?	The AWS Properly-Architected Framework allows clients and companions to evaluation their architectures utilizing a constant strategy and offers steerage to enhance designs over time.	https://aws.amazon.com/structure/well-architected/
          In what Areas is Amazon Kendra obtainable?	Amazon Kendra is presently obtainable within the following AWS Areas: Northern Virginia, Oregon, and Eire	https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/

       6. The next screenshot exhibits the query “Which AWS service has 11 nines of sturdiness?” and its response. You’ll be able to observe that the response is identical as within the FAQ file and features a hyperlink.
          
       7. Based mostly on the pages you might have crawled, ask a query within the chat. For this instance, the publicly obtainable Amazon Bedrock web page was crawled and listed. The next screenshot exhibits the query, “What are brokers in Amazon Bedrock?” and and a generated response that features related hyperlinks.
          
   15. For integration of the Amazon Lex chat assistant with Slack, see Integrating an Amazon Lex V2 bot with Slack. Select the AIBotProdAlias below Alias within the Channel Integrations

   Run pattern queries to check the answer

   1. In Slack, go to the Apps part. Within the dropdown menu, select Handle and choose Browse apps.
      
   2. Seek for ${AIBot} in App Listing and select the chat assistant. This may add the chat assistant to the Apps part in Slack. Now you can begin asking questions within the chat. The next screenshot exhibits the query “Which AWS service has 11 nines of sturdiness?” and its response. You’ll be able to observe that the response is identical as within the FAQ file and features a hyperlink.
      
   3. The next screenshot exhibits the query, “What's the AWS Properly-Architected Framework?” and its response.
      
   4. Based mostly on the pages you might have crawled, ask a query within the chat. For this instance, the publicly obtainable Amazon Bedrock web page was crawled and listed. The next screenshot exhibits the query, “What are brokers in Amazon Bedrock?” and and a generated response that features related hyperlinks.
      
   5. The next screenshot exhibits the query, “What's amazon polly?” As a result of there isn’t any Amazon Polly documentation listed, the chat assistant responds with “No related paperwork discovered,” as anticipated.
      

   These examples present how the chat assistant retrieves paperwork from Amazon Kendra and offers solutions based mostly on the paperwork retrieved. If no related paperwork are discovered, the chat assistant responds with “No related paperwork discovered.”

   Clear up

   To wash up the sources created by this answer:

   1. Delete the CloudFormation stack by navigating to the CloudFormation console
   2. Choose the stack you created for this answer and select Delete
   3. Verify the deletion by coming into the stack title within the offered subject. This may take away all of the sources created by the CloudFormation template, together with the Amazon Kendra index, Amazon Lex chat assistant, Lambda operate, and different associated sources.

   Conclusion

   This submit describes the event of a generative AI Slack software powered by Amazon Bedrock and Amazon Kendra. That is designed to be an internal-facing Slack chat assistant that helps reply questions associated to the listed content material. The answer structure consists of Amazon Lex for intent identification, a Lambda operate for fulfilling the fallback intent, Amazon Kendra for FAQ searches and indexing crawled net pages, and Amazon Bedrock for producing responses. The submit walks by the deployment of the answer utilizing a CloudFormation template, offers directions for operating pattern queries, and discusses the steps for cleansing up the sources. General, this submit demonstrates use varied AWS providers to construct a strong generative AI–powered chat assistant software.

   This answer demonstrates the ability of generative AI in constructing clever chat assistants and search assistants. Discover the generative AI Slack chat assistant: Invite your groups to a Slack workspace and begin getting solutions to your listed content material and FAQs. Experiment with totally different use instances and see how one can harness the capabilities of providers like Amazon Bedrock and Amazon Kendra to reinforce your corporation operations. For extra details about utilizing Amazon Bedrock with Slack, check with Deploy a Slack gateway for Amazon Bedrock.

   ---

   In regards to the authors

   Kruthi Jayasimha Rao is a Companion Options Architect with a deal with AI and ML. She offers technical steerage to AWS Companions in following greatest practices to construct safe, resilient, and extremely obtainable options within the AWS Cloud.

   Mohamed Mohamud is a Companion Options Architect with a deal with Knowledge Analytics. He makes a speciality of streaming analytics, serving to companions construct real-time information pipelines and analytics options on AWS. With experience in providers like Amazon Kinesis, Amazon MSK, and Amazon EMR, Mohamed allows data-driven decision-making by streaming analytics.