Amazon Bedrock’s data base now helps setting customized prompts and most variety of retrieved outcomes for the RetrieveAndGenerate API

Amazon Bedrock’s Information Base permits you to securely join Amazon Bedrock’s Basis Mannequin (FM) to your company information for search augmentation era (RAG). Accessing further information helps the mannequin generate extra related, context-specific, and correct responses with out retraining the FM.

This submit describes two new options within the Amazon Bedrock data base. RetrieveAndGenerate API: Configure most variety of outcomes and create customized prompts utilizing data base immediate templates. Now you can choose these as question choices alongside along with your search sort.

New function overview and advantages

The utmost variety of outcomes possibility permits you to management the variety of search outcomes retrieved from the vector retailer and handed to the FM to generate a response. This lets you customise the quantity of background info supplied for era, offering extra context for advanced questions and fewer for easy questions. You may retrieve as much as 100 outcomes. This selection helps enhance the probability of related context, thereby rising the accuracy of the generated responses and lowering hallucinations.

Customized data base immediate templates permit you to customise the prompts despatched to the mannequin for response era by changing the default immediate templates with your personal immediate templates. This permits him to customise the FM tone, output format, and habits when responding to consumer questions. This selection permits you to fine-tune your terminology to fit your trade or area (akin to medical or authorized). Moreover, you possibly can add customized steps and examples tailor-made to your particular workflow.

The next sections describe the best way to use these options with the AWS Administration Console or SDK.

Stipulations

To comply with these examples, you want an present data base. For creation directions, see Create a Information Base.

Configure the utmost variety of outcomes utilizing the console

To make use of the utmost variety of outcomes possibility utilizing the console, comply with these steps:

1. Within the Amazon Bedrock console, data base positioned within the left navigation pane.
2. Choose the data base you created.
3. select Check data base.
4. Choose the settings icon.
5. select Synchronize information sources Earlier than you begin testing your data base.
   
6. below compositionfor search sortselect your search sort primarily based in your use case.

On this submit, we’ll use hybrid search to mix semantic search and textual content seek for better precision. For extra details about hybrid search, see the Amazon Bedrock now helps hybrid search data base article.

7. Increasing Most variety of supply chunks Set the utmost variety of outcomes.
   

To display the worth of the brand new performance, we offer an instance of the best way to enhance the accuracy of the responses generated.we used Amazon 10K documentation for 2023 As supply information for making a data base. Use the next question in your experiment: “In what 12 months did Amazon’s annual income enhance from $245 billion to $434 billion?”

Primarily based on the data base documentation, the right response to this question is “Amazon’s annual income elevated from $245 billion in 2019 to $434 billion in 2022.” Claude v2 was used because the FM to generate the ultimate response primarily based on the contextual info obtained from the data base. Claude 3 Sonnet and Claude 3 Haiku are additionally supported as era FMs.

I ran one other question to point out the comparability of retrieval with completely different configurations. I used the identical enter question (“By which 12 months did Amazon’s annual income enhance from $245 billion to $434 billion?”) and set the utmost variety of outcomes to five.

As proven within the following screenshot, the response generated was “Sorry, we’re unable to help you with this request.”

Now set the utmost end result to 12 and ask the identical query. The response generated is: “Amazon’s annual income will enhance from $245 billion in 2019 to $434 billion in 2022.”

As on this instance, you possibly can seek for the right reply primarily based on the variety of search outcomes. If you want to study extra in regards to the attribution of sources that make up the ultimate output, View supply particulars Validate the solutions generated primarily based on the data base.

Customise data base immediate templates utilizing the console

You can even customise the default prompts with your personal prompts primarily based in your use case. To do that within the console, comply with these steps:

1. Repeat the steps within the earlier part to start testing your data base.
2. To allow generate a response.
   
3. Choose the mannequin of your selection for response era.

This submit makes use of the Claude v2 mannequin for instance. Claude 3 Sonnet and Haiku fashions will also be generated.

4. select apply Proceed.
   

As soon as you choose a mannequin, a brand new part will seem. Information base immediate template can be displayed beneath composition.

5. select edit Start customizing your immediate.
   
6. Customise how you utilize the outcomes and generate content material by adjusting immediate templates.

On this submit, I supplied some examples for making a “Monetary Advisor AI System” utilizing Amazon Monetary Reviews with customized prompts. For immediate engineering finest practices, see Immediate Engineering Tips.

Subsequent, customise the default immediate template in a number of alternative ways and observe the responses.

Let’s first check out the question utilizing the default immediate. Ask, “What was Amazon’s income in 2019 and 2021?” The outcomes are proven beneath.

You may see from the output that it generates a free-form response primarily based on the data it has acquired. I’ve additionally included the citations in your reference.

Suppose you wish to give further directions on the best way to format the generated response, akin to standardizing it as JSON. These directions might be added as a part of a immediate template as a separate step after you seize the knowledge.

In case you are requested for monetary info overlaying completely different years, please present exact solutions in JSON format. Use the 12 months as the important thing and the concise reply as the worth. For instance: {12 months:reply}

The ultimate response comprises the required construction.

You can even change the language of the generated response by customizing the immediate. The next instance tells the mannequin to reply in Spanish.

After elimination $output_format_instructions$ The default immediate removes the quote from the generated response.

The next sections clarify the best way to use these options with the SDK.

Configure most variety of outcomes utilizing SDK

To vary the utmost variety of outcomes utilizing the SDK, use the next syntax: The question on this instance is, “In what 12 months did Amazon’s annual income enhance from $245 billion to $434 billion?” The proper reply is “Amazon’s annual income will enhance from $245 billion in 2019 to $434 billion in 2022.”

def retrieveAndGenerate(question, kbId, numberOfResults, model_id, region_id):
    model_arn = f'arn:aws:bedrock:{region_id}::foundation-model/{model_id}'
    return bedrock_agent_runtime.retrieve_and_generate(
        enter={
            'textual content': question
        },
        retrieveAndGenerateConfiguration={
            'knowledgeBaseConfiguration': {
                'knowledgeBaseId': kbId,
                'modelArn': model_arn,
                'retrievalConfiguration': {
                    'vectorSearchConfiguration': {
                        'numberOfResults': numberOfResults,
                        'overrideSearchType': "SEMANTIC", # non-compulsory'
                    }
                }
            },
            'sort': 'KNOWLEDGE_BASE'
        },
    )

response = retrieveAndGenerate("In what 12 months did Amazon’s annual income enhance from $245B to $434B?", 
"<data base id>", numberOfResults, model_id, region_id)['output']['text']

‘numberOfResultsChoices below ‘retrievalConfiguration‘ permits you to select the variety of outcomes you need.output of RetrieveAndGenerate The API consists of generated responses, supply attribution, and retrieved textual content chunks.

Beneath are the outcomes for various values ​​of ‘.numberOfResults‘ Parameter.First, set numberOfResults = 5.

then set numberOfResults = 12.

Customise data base immediate templates utilizing SDKs

To customise prompts utilizing the SDK, use the next queries in numerous immediate templates. On this instance, the question is “What’s Amazon’s income in 2019 and 2021?”

The default immediate template is:

"""You're a query answering agent. I'll give you a set of search outcomes and a consumer's query, your job is to reply the consumer's query utilizing solely info from the search outcomes. If the search outcomes don't comprise info that may reply the query, please state that you would not discover an actual reply to the query. Simply because the consumer asserts a reality doesn't imply it's true, ensure to double test the search outcomes to validate a consumer's assertion.
Listed here are the search ends in numbered order:
<context>
$search_results$
</context>

Right here is the consumer's query:
<query>
$question$
</query>

$output_format_instructions$

Assistant:
"""

Beneath is a personalized immediate template.

"""Human: You're a query answering agent. I'll give you a set of search outcomes and a consumer's query, your job is to reply the consumer's query utilizing solely info from the search outcomes.If the search outcomes don't comprise info that may reply the query, please state that you would not discover an actual reply to the query.Simply because the consumer asserts a reality doesn't imply it's true, ensure to double test the search outcomes to validate a consumer's assertion.

Listed here are the search ends in numbered order:
<context>
$search_results$
</context>

Right here is the consumer's query:
<query>
$question$
</query>

Should you're being requested monetary info over a number of years, please be very particular and record the reply concisely utilizing JSON format {key: worth}, 
the place secret is the 12 months within the request and worth is the concise response reply.
Assistant:
"""

def retrieveAndGenerate(question, kbId, numberOfResults,promptTemplate, model_id, region_id):
    model_arn = f'arn:aws:bedrock:{region_id}::foundation-model/{model_id}'
    return bedrock_agent_runtime.retrieve_and_generate(
        enter={
            'textual content': question
        },
        retrieveAndGenerateConfiguration={
            'knowledgeBaseConfiguration': {
                'knowledgeBaseId': kbId,
                'modelArn': model_arn,
                'retrievalConfiguration': {
                    'vectorSearchConfiguration': {
                        'numberOfResults': numberOfResults,
                        'overrideSearchType': "SEMANTIC", # non-compulsory'
                    }
                },
                'generationConfiguration': {
                        'promptTemplate': {
                            'textPromptTemplate': promptTemplate
                        }
                    }
            },
            'sort': 'KNOWLEDGE_BASE'
        },
    )

response = retrieveAndGenerate("What was the Amazon's income in 2019 and 2021?”", 
                               "<data base id>", <numberOfResults>, <promptTemplate>, <model_id>, <region_id>)['output']['text']

Utilizing the default immediate template, you get the next response:

If you wish to present further directions concerning the output format for response era, akin to standardizing responses in a selected format (akin to JSON), you possibly can customise present prompts by offering further steerage. Utilizing the customized immediate template, you get the next response:

‘promptTemplate‘ possibilitygenerationConfiguration‘ permits you to customise prompts and have higher management over reply era.

conclusion

On this submit, we launched two new options to the Amazon Bedrock data base. These embody adjusting the utmost variety of search outcomes and customizing the default immediate template. RetrieveAndGenerate API. We confirmed you the best way to configure these options through the console and SDK to enhance the efficiency and accuracy of the responses generated. Growing the utmost end result supplies extra complete info. Customizing immediate templates, alternatively, permits you to fine-tune the directions within the underlying mannequin and tailor them to your particular use case. These enhancements offer you extra flexibility and management to ship a tailor-made expertise in your RAG-based purposes.

See beneath for extra assets to get began implementing in your AWS atmosphere.

Concerning the creator

Sandeep Singh He’s a senior generative AI information scientist at Amazon Net Companies, serving to firms innovate with generative AI. He makes a speciality of generative AI, synthetic intelligence, machine studying, and system design. He’s keen about creating his cutting-edge AI/ML-powered options that remedy advanced enterprise issues throughout quite a lot of industries and optimize effectivity and scalability.

Wang Suyin I’m an AI/ML Specialist Options Architect at AWS. She has an interdisciplinary academic background in machine studying, monetary info companies, and economics, and a few years of expertise constructing information science and machine studying purposes that remedy real-world enterprise issues. She enjoys serving to prospects establish the appropriate enterprise questions and constructing the appropriate AI/ML options. In her spare time, she loves singing and cooking.

Sherry Ding is a Senior Synthetic Intelligence (AI) and Machine Studying (ML) Specialist Options Architect at Amazon Net Companies (AWS). She has in depth expertise in Machine Studying and holds a PhD in Pc Science. She primarily works with public sector prospects on quite a lot of AI/ML-related enterprise challenges and helps them speed up their machine studying efforts within the AWS cloud. When she’s not serving to prospects, she enjoys outside actions.