Summarize name transcriptions securely with Amazon Transcribe and Amazon Bedrock Guardrails

Given the amount of conferences, interviews, and buyer interactions in fashionable enterprise environments, audio recordings play an important position in capturing helpful data. Manually transcribing and summarizing these recordings generally is a time-consuming and tedious activity. Fortuitously, developments in generative AI and computerized speech recognition (ASR) have paved the best way for automated options that may streamline this course of.

Customer support representatives obtain a excessive quantity of calls every day. Beforehand, calls had been recorded and manually reviewed later for compliance, laws, and firm insurance policies. Name recordings needed to be transcribed, summarized, after which redacted for private identifiable data (PII) earlier than analyzing calls, leading to delayed entry to insights.

Redacting PII is a essential apply in safety for a number of causes. Sustaining the privateness and safety of people’ private data shouldn’t be solely a matter of moral duty, but additionally a authorized requirement. On this submit, we present you tips on how to use Amazon Transcribe to get close to real-time transcriptions of calls despatched to Amazon Bedrock for summarization and delicate knowledge redaction. We’ll stroll by way of an structure that makes use of AWS Step Features to orchestrate the method, offering seamless integration and environment friendly processing

Amazon Bedrock is a completely managed service that provides a selection of high-performing basis fashions (FMs) from main mannequin suppliers comparable to AI21 Labs, Anthropic, Cohere, Meta, Stability AI, Mistral AI, and Amazon by way of a single API, together with a broad set of capabilities you must construct generative AI functions with safety, privateness, and accountable AI. You need to use  Amazon Bedrock Guardrails to redact delicate data comparable to PII discovered within the generated name transcription summaries. Clear, summarized transcripts are then despatched to analysts. This offers faster entry to name developments whereas defending buyer privateness.

Resolution overview

The structure of this answer is designed to be scalable, environment friendly, and compliant with privateness laws. It consists of the next key parts:

1. Recording – An audio file, comparable to a gathering or help name, to be transcribed and summarized
2. Step Features workflow – Coordinates the transcription and summarization course of
3. Amazon Transcribe – Converts audio recordings into textual content
4. Amazon Bedrock – Summarizes the transcription and removes PII
5. Amazon SNS – Delivers the abstract to the designated recipient
6. Recipient – Receives the summarized, PII-redacted transcript

The next diagram exhibits the structure overflow –

The workflow orchestrated by Step Features is as follows:

1. An audio recording is supplied as an enter to the Step Features workflow. This could possibly be carried out manually or routinely relying on the particular use case and integration necessities.
2. The workflow invokes Amazon Transcribe, which converts the multi-speaker audio recording right into a textual, speaker-partition transcription. Amazon Transcribe makes use of superior speech recognition algorithms and machine studying (ML) fashions to precisely partition audio system and transcribe the audio, dealing with numerous accents, background noise, and different challenges.
3. The transcription output from Amazon Transcribe is then handed to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock by way of AWS Lambda. This mannequin was chosen as a result of it has comparatively decrease latency and price than different fashions. The mannequin first summarizes the transcript based on its abstract directions, after which the summarized output (the mannequin response) is evaluated by Amazon Bedrock Guardrails to redact PII. To be taught the way it blocks dangerous content material, confer with How Amazon Bedrock Guardrails works. The directions and transcript are each handed to the mannequin as context.
4. The output from Amazon Bedrock is saved in Amazon Easy Storage Service (Amazon S3) and despatched to the designated recipient utilizing Amazon Easy Notification Service (Amazon SNS). Amazon SNS helps numerous supply channels, together with e mail, SMS, and cellular push notifications, ensuring that the abstract reaches the meant recipient in a well timed and dependable method

The recipient can then evaluation the concise abstract, shortly greedy the important thing factors and insights from the unique audio recording. Moreover, delicate data has been redacted, sustaining privateness and compliance with related laws.

The next diagram exhibits the Step Features workflow –

Conditions

Observe these steps earlier than beginning:

1. Amazon Bedrock customers have to request entry to fashions earlier than they’re accessible to be used. It is a one-time motion. For this answer, you must allow entry to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock. For extra data, confer with Entry Amazon Bedrock basis fashions. Deployment, as described beneath, is at the moment supported solely within the US West (Oregon) us-west-2 AWS Area. Customers could discover different fashions if desired. You may want some customizations to deploy to different Areas with completely different mannequin availability (comparable to us-east-1, which hosts Anthropic’s Claude 3.5 Sonnet). Be sure to think about mannequin high quality, velocity, and price tradeoffs earlier than selecting a mannequin.
2. Create a guardrail for PII redaction. Configure filters to dam or masks delicate data. This selection might be discovered on the Amazon Bedrock console on the Add delicate data filters web page when making a guardrail. To learn to configure filters for different use circumstances, confer with Take away PII from conversations by utilizing delicate data filters.

Deploy answer assets

To deploy the answer, download an AWS CloudFormation template to routinely provision the required assets in your AWS account. The template units up the next parts:

• A Step Features workflow
• Lambda features
• An SNS subject
• An S3 bucket
• AWS Key Administration Service (AWS KMS) keys for knowledge encryption and decryption

By utilizing this template, you possibly can shortly deploy the pattern answer with minimal guide configuration. The template requires the next parameters:

• Electronic mail deal with used to ship abstract – The abstract can be despatched to this deal with. It’s essential to acknowledge the preliminary Amazon SNS affirmation e mail earlier than receiving extra notifications.
• Abstract directions – These are the directions given to the Amazon Bedrock mannequin to generate the abstract
• Guardrail ID – That is the ID of your lately created guardrail, which might be discovered on the Amazon Bedrock Guardrails console in Guardrail overview

The Abstract directions are learn into your Lambda operate as an surroundings variable.

 
# Use the supplied directions to supply the abstract. Use a default if no intructions are supplied.
SUMMARY_INSTRUCTIONS = os.getenv('SUMMARY_INSTRUCTIONS')
 
These are then used as a part of your payload to Anthropic’s Claude 3 Haiku mannequin. That is shared to provide you an understanding of tips on how to go the directions and textual content to the mannequin.
 
# Create the payload to supply to the Anthropic mannequin.
        user_message = {"position": "person", "content material": f"{SUMMARY_INSTRUCTIONS}{transcript}"}
        messages = [user_message]
response = generate_message(bedrock_client, 'anthropic.claude-3-haiku-20240307-v1:0"', "", messages, 1000)
 
The generate_message() operate incorporates the invocation to Amazon Bedrock with the guardrail ID and different related parameters.
 
def generate_message(bedrock_runtime, model_id, system_prompt, messages, max_tokens):
    physique = json.dumps(
        {
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": max_tokens,
            "system": system_prompt,
            "messages": messages
        }
    )
print(f'Invoking mannequin: {BEDROCK_MODEL_ID}')
 
    response = bedrock_runtime.invoke_model(
        physique=physique,
        modelId=BEDROCK_MODEL_ID,
        # contentType=contentType,
        guardrailIdentifier =BEDROCK_GUARDRAIL_ID,
        guardrailVersion ="1",
        hint ="ENABLED")
    response_body = json.masses(response.get('physique').learn())
    print(f'response: {response}')
    return response_body

Deploy the answer

After you deploy the assets utilizing AWS CloudFormation, full these steps:

1. Add a Lambda layer.

Though AWS Lambda frequently updates the model of AWS Boto3 included, on the time of penning this submit, it nonetheless offers model 1.34.126. To make use of Amazon Bedrock Guardrails, you want model 1.34.90 or greater, for which we’ll add a Lambda layer that updates the Boto3. You possibly can observe the official developer guide on tips on how to add a Lambda layer.

There are alternative ways to create a Lambda layer. A easy technique is to make use of the steps outlined in Packaging the layer content material, which references a pattern software repo. You need to be capable of exchange requests==2.31.0 inside necessities.txt content material to boto3, which can set up the newest accessible model, then create the layer.

So as to add the layer to Lambda, be sure that the parameters laid out in Creating the layer match the deployed Lambda. That’s, you must replace compatible-architectures to x86_64.

2. Acknowledge the Amazon SNS e mail affirmation that it’s best to obtain a number of moments after creating the CloudFormation stack
3. On the AWS CloudFormation console, discover the stack you simply created
4. On the stack’s Outputs tab, search for the worth related to AssetBucketName. It’ll look one thing like summary-generator-assetbucket-xxxxxxxxxxxxx.
5. On the Amazon S3 console, discover your S3 property bucket.

That is the place you’ll add your recordings. Legitimate file codecs are MP3, MP4, WAV, FLAC, AMR, OGG, and WebM.

6. Add your recording to the recordings folder in Amazon S3

Importing recordings will routinely set off the AWS Step Features state machine. For this instance, we use a pattern group assembly recording from the sample recording.

7. On the AWS Step Features console, discover the summary-generator state machine. Select the title of the state machine run with the standing Working.

Right here, you possibly can watch the progress of the state machine because it processes the recording. After it reaches its Success state, it’s best to obtain an emailed abstract of the recording. Alternatively, you possibly can navigate to the S3 property bucket and consider the transcript there within the transcripts folder.

Broaden the answer

Now that you’ve got a working answer, listed below are some potential concepts to customise the answer in your particular use circumstances:

• Attempt altering the method to suit your accessible supply content material and desired outputs:
  • For conditions the place transcripts can be found, create an alternate AWS Step Features workflow to ingest current text-based or PDF-based transcriptions
  • As a substitute of utilizing Amazon SNS to inform recipients by way of e mail, you should utilize it to ship the output to a distinct endpoint, comparable to a group collaboration web site or to the group’s chat channel
• Attempt altering the abstract directions for the AWS CloudFormation stack parameter supplied to Amazon Bedrock to supply outputs particular to your use case. The next are some examples:
  • When summarizing an organization’s earnings name, you may have the mannequin concentrate on potential promising alternatives, areas of concern, and issues that it’s best to proceed to observe
  • In the event you’re utilizing the mannequin to summarize a course lecture, it may establish upcoming assignments, summarize key ideas, listing info, and filter out small discuss from the recording
• For a similar recording, create completely different summaries for various audiences:
  • Engineers’ summaries concentrate on design choices, technical challenges, and upcoming deliverables
  • Mission managers’ summaries concentrate on timelines, prices, deliverables, and motion gadgets
  • Mission sponsors get a short replace on challenge standing and escalations
  • For longer recordings, attempt producing summaries for various ranges of curiosity and time dedication. For instance, create a single sentence, single paragraph, single web page, or in-depth abstract. Along with the immediate, you may need to modify the max_tokens_to_sample parameter to accommodate completely different content material lengths.

Clear up

Clear up the assets you created for this answer to keep away from incurring prices. You need to use an AWS SDK, the AWS Command Line Interface (AWS CLI), or the console.

1. Delete Amazon Bedrock Guardrails and the Lambda layer you created
2. Delete the CloudFormation stack

To make use of the console, observe these steps:

3. On the Amazon Bedrock console, within the navigation menu, choose Guardrails. Select your guardrail, then choose Delete.
4. On the AWS Lambda console, within the navigation menu, choose Layers. Select your layer, then choose Delete.
5. On the AWS CloudFormation console, within the navigation menu, choose Stacks. Select the stack you created, then choose Delete.

Deleting the stack gained’t delete the related S3 bucket. In the event you now not require the recordings or transcripts, you possibly can delete the bucket individually. Amazon Transcribe is designed to routinely delete transcription jobs after 90 days. Nevertheless, you possibly can choose to manually delete these jobs earlier than the 90-day retention interval expires.

Conclusion

As companies flip to knowledge as a basis for decision-making, being able to effectively extract insights from audio recordings is invaluable. By utilizing the facility of generative AI with Amazon Bedrock and Amazon Transcribe, your group can create concise summaries of audio recordings whereas sustaining privateness and compliance. The proposed structure demonstrates how AWS providers might be orchestrated utilizing AWS Step Features to streamline and automate complicated workflows, enabling organizations to concentrate on their core enterprise actions.

This answer not solely saves effort and time, but additionally makes positive that delicate data is redacted, mitigating potential dangers and selling compliance with knowledge safety laws. As organizations proceed to generate and course of giant volumes of audio knowledge, options like this can develop into more and more essential for gaining insights, making knowledgeable choices, and sustaining a aggressive edge.

Concerning the authors

Yash Yamsanwar is a Machine Studying Architect at Amazon Net Companies (AWS). He’s accountable for designing high-performance, scalable machine studying infrastructure that optimizes the total lifecycle of machine studying fashions, from coaching to deployment. Yash collaborates intently with ML analysis groups to push the boundaries of what’s doable with LLMs and different cutting-edge machine studying applied sciences.

Sawyer Hirt is a Options Architect at AWS, specializing in AI/ML and cloud architectures, with a ardour for serving to companies leverage cutting-edge applied sciences to beat complicated challenges. His experience lies in designing and optimizing ML workflows, enhancing system efficiency, and making superior AI options extra accessible and cost-effective, with a selected concentrate on Generative AI. Exterior of labor, Sawyer enjoys touring, spending time with household, and staying present with the newest developments in cloud computing and synthetic intelligence.