Introducing OpenAI fashions on Amazon Bedrock for in-country inferencing in India

Amazon Bedrock now supports the OpenAI GPT-5.6 models, Terra and Luna, in India, with India geographic cross-Area inference. When you have native knowledge processing necessities in India, together with in monetary providers, healthcare, and the general public sector, now you can use these OpenAI fashions at scale. Amazon Bedrock processes inference requests and knowledge inside India.

Each fashions supply a 1-million-token context window, settle for textual content and picture enter, and produce textual content output. Your purposes can course of lengthy paperwork, giant code bases, and combined text-and-image workloads in a single request. The processing by no means leaves the nation.

On this submit, we stroll via how India geographic cross-Area inference works from the Mumbai and Hyderabad Areas. We additionally present the right way to get began from the Amazon Bedrock console and with code, utilizing the OpenAI Responses API, OpenAI Chat Completions API, and the Amazon Bedrock Converse API.

India geographic cross-Area inference

Cross-Area inference robotically routes inference requests throughout a number of AWS Areas to assist enhance throughput, with out you having to handle capability in every Area your self. It’s primarily a capability mechanism. As a substitute of being sure to at least one Area’s capability, your requests draw on a broader pool of compute. That helps you keep throughput and constant efficiency beneath load, which issues most throughout site visitors peaks.

With India geographic cross-Area inference, Amazon Bedrock routes requests solely inside the India geography throughout Areas reminiscent of Asia Pacific (Mumbai) Area (ap-south-1) and Asia Pacific (Hyderabad) Area (ap-south-2). You possibly can scale to satisfy demand whereas maintaining knowledge processed inside India to satisfy knowledge residency necessities. You name the profile from both India Area because the supply, and Amazon Bedrock routes the request to the vacation spot India Area based mostly on capability. For probably the most present details about mannequin availability in every Area, see Regional availability by fashions within the Amazon Bedrock Person Information.

Cross-Area inference works via inference profiles. You name a profile ID because the mannequin, and it defines the mannequin and the AWS Areas Amazon Bedrock can route your request to. The India geographic inference profiles preserve that routing inside India. There are two profiles:

• in.openai.gpt-5.6-terra for GPT-5.6 Terra.
• in.openai.gpt-5.6-luna for GPT-5.6 Luna.

With inference profiles, billing and quota consumption are tracked in opposition to your account within the supply Area, no matter which backend Area dealt with the request. Amazon CloudWatch and AWS CloudTrail document log entries within the supply Area solely, so your monitoring stays in a single place.

Selecting between Amazon Bedrock Mantle and Runtime endpoints

For brand new purposes, we advocate the bedrock-runtime endpoint. It helps the Bedrock-native InvokeModel and Converse APIs, the OpenAI-compatible Responses and Chat Completions APIs, and the Anthropic Messages API, and it’s the place Amazon Bedrock options reminiscent of Guardrails, clever immediate routing, and cross-Area inference can be found.

Information residency

The India geographic profile retains inference inside India. Requests route solely between ap-south-1 and ap-south-2. Your enter prompts and output outcomes would possibly transfer between these two Areas. Information is encrypted in transit throughout the Amazon community. Amazon Bedrock makes use of a zero knowledge retention (ZDR) knowledge safety mannequin. Which means that by default, Amazon Bedrock doesn’t retailer mannequin inputs or outputs. Nonetheless, for sure fashions, together with GPT-5.6, content material flagged by the Amazon Bedrock automated abuse-detection classifiers is retained for offline abuse detection. Please see Abuse detection within the Amazon Bedrock Person Information for extra particulars.

World cross-Area inference

Amazon Bedrock additionally gives world cross-Area inference and world inference profiles (prefixed world.) in India that path to supported business AWS Areas worldwide for max capability. You ship your request to the India Area endpoint, both Asia Pacific (Mumbai) ap-south-1 or Asia Pacific (Hyderabad) ap-south-2, utilizing the worldwide profile ID because the mannequin ID. Amazon Bedrock then decides which vacation spot Area serves the request. World cross-Area inference helps OpenAI GPT-5.6 fashions, together with Sol, Terra, and Luna. Nonetheless, in case your workload has native knowledge processing necessities, use the India (prefixed in.) profiles as a substitute, as a result of they preserve inference inside the nation. To learn extra about world cross-Area inference, see Introduce cross-Area inference for OpenAI GPT-5.6 fashions on Amazon Bedrock.

Entry GPT-5.6 fashions from the Amazon Bedrock console

You possibly can attempt GPT-5.6 within the textual content playground within the Amazon Bedrock console, which requires no coding or SDK setup. You possibly can ship prompts, regulate inference parameters, and change between variants to get a really feel for every mannequin earlier than you combine the API.

1. Open the Amazon Bedrock console in a Area the place the fashions can be found, reminiscent of Asia Pacific (Mumbai) ap-south-1.
2. Within the navigation pane, beneath Take a look at, select Playground.
3. Select Choose mannequin in the midst of the web page.
4. Seek for OpenAI GPT-5.6 Terra, choose IN OpenAI GPT-5.6 Terra, and select Apply.
5. Enter a immediate and select Run to generate a response.

Name GPT-5.6 fashions with the OpenAI Responses API

GPT-5.6 fashions on Amazon Bedrock natively assist the OpenAI Responses API format. In case your utility already calls OpenAI fashions, you’ll be able to level your current OpenAI SDK consumer on the Amazon Bedrock endpoint in both India Area, Mumbai or Hyderabad, and cross an India geographic inference profile ID because the mannequin parameter.

For authentication, Amazon Bedrock accepts both commonplace AWS credentials or an Amazon Bedrock API key. The API key path is the simple match for the OpenAI SDK, which passes it because the bearer token. For manufacturing, generate short-term API keys programmatically with the aws-bedrock-token-generator package deal. It derives a bearer token out of your current AWS credentials, so no static key must be saved.

The next instance makes use of Asia Pacific (Mumbai) ap-south-1 endpoint and India geographic inference profile ID to name OpenAI GPT-5.6 Terra on Amazon Bedrock.

from aws_bedrock_token_generator import provide_token
from openai import OpenAI

# Level the OpenAI SDK on the Amazon Bedrock OpenAI-compatible
# endpoint within the Asia Pacific (Mumbai) Area.
consumer = OpenAI(
    base_url="https://bedrock-runtime.ap-south-1.amazonaws.com/openai/v1",
    api_key=provide_token(area="ap-south-1"),  # short-term Amazon Bedrock API key, legitimate as much as 12 hours
)

# Geographic (India) inference profile ID for GPT-5.6 Terra.
model_id = "in.openai.gpt-5.6-terra"  # "in.openai.gpt-5.6-luna" for luna
response = consumer.responses.create(
    mannequin=model_id,
    enter="Extract the fee due date and whole quantity from the bill textual content that follows, and return them as JSON. <bill textual content>",
    max_output_tokens=512,
)
print(response.output_text)

The Responses API makes use of a single enter discipline and returns the generated textual content in output_text, with the output restrict set via max_output_tokens. The identical consumer additionally works with the Chat Completions API, helpful in case your utility already makes use of this format.

Controlling reasoning depth

To regulate reasoning depth, set the non-obligatory reasoning parameter, for instance reasoning={"effort": "low"}. GPT-5.6 fashions on Amazon Bedrock assist the next reasoning effort ranges: none, low, medium, excessive, xhigh, and max. For extra data, see the associated submit Get began with OpenAI GPT-5.6 Sol, Terra, and Luna on Amazon Bedrock. Omitting the parameter makes use of the mannequin default.

response = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    enter="Extract the fee due date and whole quantity from the bill textual content that follows, and return them as JSON. <bill textual content>",
    reasoning={"effort": "excessive"},  # none | low | medium | excessive | xhigh | max
    max_output_tokens=512,
)

For the total record of supported parameters, see the OpenAI mannequin documentation within the Amazon Bedrock Person Information.

Multi-turn conversations with server-side state

Set retailer=True to have Amazon Bedrock retain the response server-side, then reference it on the following flip with previous_response_id. You ship solely the brand new flip. You don’t resend the prior dialog historical past.

first = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    enter="Bear in mind this: my favourite quantity is 42. Reply with simply 'saved'..",
    max_output_tokens=50,
    retailer=True,
)

second = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    previous_response_id=first.id,  # the mannequin remembers the sooner flip
    enter="what's my favorite quantity?",
    max_output_tokens=50,
    retailer=True,
)

print(second.output_text)

When you chain with previous_response_id, the response you chain from should have been created with retailer=True. Chaining from an unstored response returns an error.

Streaming responses

For streaming, set stream=True and iterate over the occasions:

stream = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    enter="Draft a brief standing replace for a delayed cargo, in a well mannered and direct tone.",
    max_output_tokens=512,
    stream=True,
)

for occasion in stream:
    if occasion.kind == "response.output_text.delta":
        print(occasion.delta, finish="")

Name GPT-5.6 fashions with the Converse API

When you favor the AWS SDK and SigV4 authentication over an API key, the Amazon Bedrock Converse API helps GPT-5.6 fashions with the identical unified interface it offers for different fashions on Amazon Bedrock. ConverseStream covers the streaming case.

import boto3

# Commonplace AWS credentials (SigV4), no API key wanted.
consumer = boto3.consumer("bedrock-runtime", region_name="ap-south-1")
model_id = "in.openai.gpt-5.6-terra"  # India Geo inference profile
response = consumer.converse(
    modelId=model_id,
    messages=[
        {
            "role": "user",
            "content": [{"text": "Classify this support ticket as billing, technical, or account: <ticket text>"}],
        }
    ],
    inferenceConfig={"maxTokens": 512},
)
print(response["output"]["message"]["content"]

Immediate caching

Your prompts would possibly share an extended, steady prefix, reminiscent of a system instruction, a information base excerpt, or a set of few-shot examples. Once they do, GPT-5.6 fashions on Amazon Bedrock assist immediate caching. Cached reads are billed at a 90 % low cost in comparison with uncached enter tokens. This provides up shortly for Retrieval Augmented Technology (RAG) and agent workloads that repeat the identical context throughout many turns. You’ll find particulars on Amazon Bedrock immediate caching documentation.

Immediate caching works with the India geographic inference profiles, so you retain the financial savings beneath the India data-residency boundary. Immediate caching runs in two modes, specific and implicit. With implicit caching, Amazon Bedrock locations the cache breakpoints for you robotically. Observe that the minimal prefix size is 1,024 tokens. With specific caching, you mark the cache boundary your self for exact management, and the cached prefix stays heat for at the least half-hour.

response = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    prompt_cache_key="ticket-agent-ver123",  # identical key throughout all requests
    enter=[
        {
            "type": "message",
            "role": "developer",
            "content": [{
                "type": "input_text",
                "text": SYSTEM_INSTRUCTIONS,  # long, static: guidelines, KB excerpts (>= 1,024 tokens)
                "prompt_cache_breakpoint": {"mode": "explicit"},
            }],
        },
        {
            "kind": "message",
            "position": "consumer",
            "content material": [{
                "type": "input_text",
                "text": user_question,  # changes on every request
            }],
        },
    ],
    extra_body={"prompt_cache_options": {"mode": "specific"}},
)

Each response tells you what the cache did, in utilization.input_tokens_details:

particulars = response.utilization.input_tokens_details
print(f"cached: {particulars.cached_tokens}, written: {particulars.cache_write_tokens}")

On the primary name, you will note cache_write_tokens populated because the prefix is saved. On subsequent calls, the identical tokens come again as cached_tokens, billed on the cache-read fee. As a result of these counts are a part of input_tokens somewhat than added on prime, a cached token is counted and charged as soon as.

IAM permissions for India geographic cross-Area inference

To permit an AWS Id and Entry Administration (IAM) position to invoke GPT-5.6 fashions via the India geographic inference profile, grant the position entry to 3 sources. These are the India geographic inference profile itself, the muse mannequin (FM) within the supply Area, and the muse mannequin in every vacation spot Area listed within the profile (ap-south-1 and ap-south-2). You should utilize the AmazonBedrockLimitedAccess managed coverage or create your personal.

The next instance grants permission to make use of the GPT-5.6 Terra mannequin via the India geographic inference profile. The supply Area is ap-south-1, and the vacation spot Areas are ap-south-1 and ap-south-2. Substitute <ACCOUNT> together with your account ID, and duplicate the sources for in.openai.gpt-5.6-luna if the position wants each fashions. When you supply from Hyderabad, use the ap-south-2 inference-profile ARN within the first assertion. The muse-model sources within the second assertion keep the identical, as a result of they already record each India Areas.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "GrantGeoCrisInferenceProfileAccess",
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel*"
            ],
            "Useful resource": [
                "arn:aws:bedrock:ap-south-1:<ACCOUNT>:inference-profile/in.openai.gpt-5.6-terra",
                "arn:aws:bedrock:ap-south-1:<ACCOUNT>:project/default"
            ]
        },
        {
            "Sid": "GrantGeoCrisModelAccess",
            "Impact": "Permit",
            "Motion": [
                "bedrock:InvokeModel*"
            ],
            "Useful resource": [
                "arn:aws:bedrock:ap-south-1::foundation-model/openai.gpt-5.6-terra",
                "arn:aws:bedrock:ap-south-2::foundation-model/openai.gpt-5.6-terra"
            ],
            "Situation": {
                "StringEquals": {
                    "bedrock:InferenceProfileArn": "arn:aws:bedrock:ap-south-1:<ACCOUNT>:inference-profile/in.openai.gpt-5.6-terra"
                }
            }
        },
        {
            "Sid": "AllowBearerTokenAuth",
            "Impact": "Permit",
            "Motion": ["bedrock:CallWithBearerToken"],
            "Useful resource": "*"
        }
    ]
}

The primary assertion grants entry to the India geographic inference profile for requests originating from the supply Area. The second grants entry to the muse mannequin within the supply Area and in each vacation spot Area listed within the profile. This entry is scoped to that one profile via the bedrock:InferenceProfileArn situation. These insurance policies grant bedrock:CallWithBearerToken and bedrock:InvokeModel on the challenge/default useful resource. The OpenAI Responses and Chat Completions APIs use this useful resource to authenticate with an Amazon Bedrock API key and run inference.

Service Management Coverage (SCP)

In case your group makes use of Service Management Insurance policies to dam unused Areas, your Area-specific SCP situations should enable each vacation spot Area listed within the geographic inference profile.

For the India geographic inference profile, each ap-south-1 and ap-south-2 have to be permitted. Blocking both vacation spot Area causes cross-Area inference to fail though the supply Area stays accessible.

Organising Codex with GPT-5.6 on Amazon Bedrock

Codex can use the geographic inference profiles via Amazon Bedrock Runtime. Set up the newest Codex CLI (codex-cli 0.149.1 or later) to make use of the Amazon Bedrock Runtime mannequin supplier. For organizations whose identification supplier is Okta, Auth0, Microsoft Entra ID, Amazon Cognito, or AWS IAM Id Heart, the AWS OIDC Auth Helper repository offers a pattern credential helper.

First, observe the information to configure your identification supplier, the corresponding AWS federation useful resource, and an IAM position carrying the Bedrock permissions proven earlier. Then add a named profile to ~/.aws/config, so the helper doesn’t exchange credentials resolved by the default profile.

[profile <AWS_OIDC_PROFILE>] 
credential_process = <ABSOLUTE_PATH_TO_CREDENTIAL_PROCESS> --profile <OIDC_HELPER_PROFILE> 
area = ap-south-1
output = json 

This federation helper exchanges an OIDC token for short-term AWS credentials, and Codex reads via the usual AWS credential chain with no additional configuration. Subsequent, create or replace ~/.codex/config.toml and reference the AWS profile, see the Codex configuration reference for different supported settings:

mannequin = "in.openai.gpt-5.6-terra" 
model_provider = "amazon-bedrock-runtime" 
model_reasoning_effort = "excessive" 
 
[model_providers.amazon-bedrock-runtime.aws] 
profile = "<AWS_OIDC_PROFILE>" 
area = "ap-south-1" 

If the helper doesn’t have a sound cached session, it opens the configured sign-in web page in your browser. After you authenticate, the helper returns short-term AWS credentials via credential_process. Requests are signed with AWS SigV4. When the profile is backed by AWS IAM Id Heart, the credentials are already short-term and rotate with the SSO session. To make use of this within the Asia Pacific (Hyderabad) Area, set the Area to ap-south-2 within the AWS profile and Codex configuration.

If you have already got AWS credentials configured regionally, Codex can use them straight. Level the Amazon Bedrock Runtime supplier at an AWS profile and an India Area, and set the mannequin to an India geographic inference profile ID.

Create or replace ~/.codex/config.toml:

  mannequin = "in.openai.gpt-5.6-terra"
  model_provider = "amazon-bedrock-runtime"
  model_reasoning_effort = "excessive"

  [model_providers.amazon-bedrock-runtime.aws]
  profile = "default"
  area = "ap-south-1"

Monitoring and logging

The India geographic inference profiles work with the identical account-level controls you already use for different fashions on Amazon Bedrock. Requests seem in Amazon Bedrock mannequin invocation logging the identical manner on-demand requests do. You possibly can ship these logs to Amazon Easy Storage Service (Amazon S3) or Amazon CloudWatch Logs. CloudWatch metrics for invocation rely, token rely, latency, throttles, and errors are printed per inference profile. You possibly can monitor GPT-5.6 fashions utilization from Mumbai or Hyderabad with out correlating logs throughout Areas your self.

To see which vacation spot Area processed a given request, CloudTrail occasions embody an additionalEventData discipline with an inferenceRegion key. For the India profile, that is at all times ap-south-1 or ap-south-2, which provides you an auditable document that inference stayed inside India. Utilization is itemized on AWS Price Explorer and the AWS Price and Utilization Report. Spend attribution by mannequin works the identical manner as for the remainder of your Amazon Bedrock workloads.

Conclusion

Amazon Bedrock brings the OpenAI GPT-5.6 Terra and Luna fashions, with a 1-million-token context window, to clients constructing in India. The India geographic inference profiles can be found on the bedrock-runtime  endpoint, which is the endpoint we advocate for brand spanking new purposes. With India geographic cross-Area inference, you’ll be able to scale throughout the Mumbai and Hyderabad Areas whereas maintaining inference inside the nation. You possibly can name the fashions via the OpenAI Responses API, the OpenAI Chat Completions API, or the Amazon Bedrock Converse API. When you have an current workload on the bedrock-mantle endpoint, it stays absolutely supported and no adjustments are required; to undertake the India geographic profiles, level that workload on the bedrock-runtime  endpoint in Mumbai or Hyderabad. Assessment the Amazon Bedrock pricing web page for present GPT-5.6 charges earlier than sizing a manufacturing workload.

To get began, overview the mannequin playing cards for GPT-5.6 Terra and Luna and the Cross-Area inference part within the Amazon Bedrock Person Information. Strive a immediate within the Amazon Bedrock console playground in Mumbai or Hyderabad, then wire up the code examples shared earlier on this submit. When you run into a problem or need to ask the group, AWS re:Post is an efficient place to go looking or submit a query.

In regards to the authors