Accessing OpenAI GPT fashions on Amazon Bedrock with world cross-Area inference in Thailand, Malaysia, Singapore, Indonesia, and Taiwan

Single-Area inference can throttle when demand exceeds native capability. Groups in Thailand, Malaysia, Singapore, Indonesia, and Taiwan want entry to the most recent OpenAI fashions past that capability. Amazon Bedrock affords OpenAI GPT-6 Sol and GPT-6 Luna by means of world cross-Area inference from all 5 AWS Areas, and GPT-6 Astra from Asia Pacific (Singapore). This put up reveals you the way to configure the decision, set permissions, and monitor inference out of your supply Area.

GPT-6 Sol, GPT-6 Luna, and GPT-6 Astra are all out there by means of world cross-Area inference from the previous Areas, on the bedrock-runtime endpoint. See Cross-Area inference within the Amazon Bedrock Consumer Information. For the complete checklist of supply and vacation spot Areas in every profile, see Supported Areas and fashions for inference profiles.

On this put up, we introduce the worldwide cross-Area inference profiles and invoke them from Asia Pacific (Singapore) utilizing the Converse API, the OpenAI-compatible APIs, and coding shoppers. We then cowl the AWS Id and Entry Administration (IAM) and repair management coverage (SCP) configuration, quota administration, and monitoring.

International cross-Area inference profiles for OpenAI GPT-6

The GPT-6 household affords three fashions, every out there by means of world cross-Area inference.

• GPT-6 Sol brings robust reasoning and coding capabilities to complicated duties carried out all through the week.
• GPT-6 Luna makes targeted, repeatable duties sensible at excessive quantity.
• GPT-6 Astra handles complicated reasoning, coding, and long-document work.

All three settle for textual content and picture enter, client-side instrument use, and immediate caching. They differ on reasoning effort. GPT-6 Sol and GPT-6 Luna settle for none, low, medium, excessive, xhigh, and max. GPT-6 Astra accepts the identical ranges besides none. For GPT-6 Sol and GPT-6 Luna, see Carry extra intelligence to on a regular basis work with GPT-6 Sol and GPT-6 Luna on Amazon Bedrock. For GPT-6 Astra, see Take in your most formidable work with GPT-6 Astra on Amazon Bedrock.

International cross-Area inference routes requests throughout supported industrial AWS Areas the place the mannequin is deployed, providing you with broad entry to Amazon Bedrock capability. You invoke it by passing a world cross-Area inference profile ID rather than a mannequin ID. Your supply Area is the place you make the API name. Amazon Bedrock selects the vacation spot Area for processing. Billing, quota monitoring, AWS CloudTrail, and Amazon CloudWatch all keep in your supply Area.

The next desk lists the worldwide cross-Area inference profile IDs and the supply Areas coated on this put up.

Inference profile availability and Area assist change as AWS provides Areas and profiles, so verify each on the mannequin card earlier than you construct: GPT-6 Sol, GPT-6 Luna, and GPT-6 Astra. For per-token charges, see Amazon Bedrock pricing.

Getting began with OpenAI fashions on Amazon Bedrock

Begin with the stipulations, then invoke any of the three profiles by means of the Converse API, the OpenAI-compatible APIs, or your present coding consumer.

Stipulations

1. An IAM id with permission to invoke the profile. The IAM coverage within the Safety and permissions part grants solely what these examples want.
2. Your supply Area activated. Solely the Area you name from wants activating. You don’t activate vacation spot Areas, which run on Amazon Bedrock managed capability. Singapore (ap-southeast-1) is a Area that’s enabled by default. Jakarta (ap-southeast-3), Malaysia (ap-southeast-5), Thailand (ap-southeast-7), and Taipei (ap-east-2) are opt-in Areas you could activate first. For extra data, see Allow or disable AWS Areas in your account.
3. A present AWS SDK or CLI. The OpenAI-compatible examples additionally require the OpenAI SDK and the Amazon Bedrock token generator. Set up them with pip set up boto3 openai aws-bedrock-token-generator.

The Amazon Bedrock console lists all three world cross-Area inference profiles below system-defined inference profiles. See Determine 1.

Invoke GPT-6 fashions by means of Amazon Bedrock Runtime

Every mannequin helps three entry paths on this endpoint: the Amazon Bedrock Converse API, the OpenAI Responses API, and the OpenAI Chat Completions API. The OpenAI-compatible APIs are known as on the /openai/v1 paths fairly than by means of the AWS SDKs, and the endpoint accepts both Signature Model 4 (SigV4) or an Amazon Bedrock API key. Move the worldwide cross-Area inference profile ID because the mannequin in each case.

Converse API. This instance calls GPT-6 Sol from Asia Pacific (Singapore) and logs the response. Set BEDROCK_PROFILE_ID to world.openai.gpt-6-luna or world.openai.gpt-6-astra to change fashions, or name converse_stream for streaming.

import logging
import os
import boto3

# AWS_REGION overrides this default, so set it to your supply Area or unset it.
REGION = os.environ.get("AWS_REGION", "ap-southeast-1")
PROFILE_ID = os.environ.get("BEDROCK_PROFILE_ID", "world.openai.gpt-6-sol")
PROMPT = (
    "In three bullet factors, clarify when a workload ought to use "
    "cross-Area inference as a substitute of calling a mannequin in a single Area."
)

logging.basicConfig(format="%(levelname)s %(message)s")
log = logging.getLogger("bedrock-cris")
log.setLevel(logging.INFO)

consumer = boto3.consumer("bedrock-runtime", region_name=REGION)

# AccessDeniedException factors on the IAM coverage. ValidationException often
# means the mannequin ID isn't a world cross-Area inference profile.
response = consumer.converse(
    modelId=PROFILE_ID,
    messages=[{"role": "user", "content": [{"text": PROMPT}]}],
    inferenceConfig={"maxTokens": 1024},
    # GPT-6 Sol and Luna settle for none by means of max. GPT-6 Astra rejects none.
    additionalModelRequestFields={"reasoning": {"effort": "low"}},
)

# Any effort above none can prepend a reasoningContent block, so choose the textual content
# block by key fairly than by place.
textual content = "".be a part of(b["text"] for b in response["output"]["message"]["content"] if "textual content" in b)

log.data("Known as %s from %s", PROFILE_ID, REGION)
log.data("Cease cause: %s", response["stopReason"])
log.data("Response:npercents", textual content)

OpenAI Chat Completions API. Level the OpenAI SDK on the Regional endpoint and go the identical profile ID.

import logging
import os
from aws_bedrock_token_generator import provide_token
from openai import OpenAI

REGION = os.environ.get("AWS_REGION", "ap-southeast-1")
PROFILE_ID = os.environ.get("BEDROCK_PROFILE_ID", "world.openai.gpt-6-sol")
PROMPT = "Checklist three makes use of for a message queue."

logging.basicConfig(format="%(levelname)s %(message)s")
log = logging.getLogger("bedrock-cris")
log.setLevel(logging.INFO)

# provide_token mints a short-term key out of your AWS credentials, legitimate as much as 12 hours.
consumer = OpenAI(
    base_url=f"https://bedrock-runtime.{REGION}.amazonaws.com/openai/v1",
    api_key=provide_token(area=REGION),
)

response = consumer.chat.completions.create(
    mannequin=PROFILE_ID,
    messages=[{"role": "user", "content": PROMPT}],
    max_completion_tokens=256,
)

log.data("Known as %s from %s", PROFILE_ID, REGION)
log.data("Response:npercents", response.selections[0].message.content material)

When migrating an present OpenAI consumer:

• Set the mannequin parameter on each request, together with these referencing a earlier response.
• Use supported instrument sorts: perform, mcp, customized, namespace, and tool_search.
• Configure the inference profile ID instantly in your consumer fairly than querying the fashions endpoint.

Arrange the ChatGPT desktop app and Codex CLI

The ChatGPT desktop app and the Codex CLI share ~/.codex/config.toml, so one change covers each. Of the 2 built-in Amazon Bedrock suppliers, solely amazon-bedrock-runtime accepts an inference profile.

mannequin = "world.openai.gpt-6-astra"
model_provider = "amazon-bedrock-runtime"
[model_providers.amazon-bedrock-runtime.aws]
area = "ap-southeast-1"

SigV4 signing means there isn’t any key to rotate. Add profile = "<your-profile>" for a non-default profile, and set area explicitly, as a result of the consumer in any other case falls again to AWS_REGION. Substitute one other profile ID for mannequin to vary fashions.

Codex stories a 272,000-token window for these profiles, which is its personal bundled determine fairly than a service restrict. Increase it with model_context_window. This configuration wants Codex CLI 0.148 or later.

Determine 2 reveals the end result within the app.

Determine 2: The ChatGPT desktop app working GPT-6 Astra on Amazon Bedrock

Safety and permissions

Amazon Bedrock affords a variety of controls over inference. Two of them matter right here. AWS Id and Entry Administration (IAM) insurance policies management which roles can invoke which profiles, and repair management insurance policies (SCPs) apply organization-wide controls, together with Area scope. This part covers each, and the place request information lands.

IAM permissions for world cross-Area inference

To name any of those fashions by means of a world cross-Area inference profile, your IAM coverage wants 4 statements:

1. Entry to the inference profile and the mission/default useful resource in your supply Area.
2. Entry to the muse mannequin in your supply Area, scoped to the inference profile.
3. The Area-agnostic Amazon Useful resource Identify (ARN) for the muse mannequin, arn:aws:bedrock:::foundation-model/openai.gpt-6-sol, with aws:RequestedRegion set to unspecified to allow cross-Area routing.
4. bedrock:CallWithBearerToken for API-key authentication.

This IAM coverage grants permission to invoke GPT-6 Sol by means of world cross-Area inference from ap-southeast-1. Change together with your AWS account ID and ap-southeast-1 together with your supply Area.

{
  "Model": "2012-10-17",
  "Assertion": [
    {
      "Sid": "GrantGlobalCrisProfileAndProjectAccess",
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel"],
      "Useful resource": [
        "arn:aws:bedrock:ap-southeast-1:<ACCOUNT>:inference-profile/global.openai.gpt-6-sol",
        "arn:aws:bedrock:ap-southeast-1:<ACCOUNT>:project/default"
      ],
      "Situation": {
        "StringEquals": { "aws:RequestedRegion": "ap-southeast-1" }
      }
    },
    {
      "Sid": "GrantGlobalCrisInRegionModelAccess",
      "Impact": "Permit",
      "Motion": ["bedrock:InvokeModel"],
      "Useful resource": ["arn:aws:bedrock:ap-southeast-1::foundation-model/openai.gpt-6-sol"],
      "Situation": {
        "StringEquals": {
          "aws:RequestedRegion": "ap-southeast-1",
          "bedrock:InferenceProfileArn": "arn:aws:bedrock:ap-southeast-1:<ACCOUNT>:inference-profile/world.openai.gpt-6-sol"
        }
      }
    },
    {
      "Sid": "GrantGlobalCrisGlobalModelAccess",
      "Impact": "Permit",
      "Motion": ["bedrock:InvokeModel"],
      "Useful resource": ["arn:aws:bedrock:::foundation-model/openai.gpt-6-sol"],
      "Situation": {
        "StringEquals": {
          "aws:RequestedRegion": "unspecified",
          "bedrock:InferenceProfileArn": "arn:aws:bedrock:ap-southeast-1:<ACCOUNT>:inference-profile/world.openai.gpt-6-sol"
        }
      }
    },
    {
      "Sid": "AllowBearerTokenAuth",
      "Impact": "Permit",
      "Motion": ["bedrock:CallWithBearerToken"],
      "Useful resource": "*"
    }
  ]
}

In the event you use streaming (converse_stream or stream=True), add bedrock:InvokeModelWithResponseStream to the Motion within the first three statements. Converse authorizes as bedrock:InvokeModel. In the event you solely name Converse with SigV4 credentials, drop the fourth assertion.

To modify fashions, change openai.gpt-6-sol with openai.gpt-6-luna or openai.gpt-6-astra all through. For the entire reference, see IAM coverage necessities for world cross-Area inference.

Service management coverage concerns for world cross-Area inference

A worldwide cross-Area inference request carries no Area title. Amazon Bedrock units aws:RequestedRegion to unspecified, so a Area allowlist in an SCP denies the decision, and the error names solely the coverage sort, not which coverage. That could be a frequent cause a primary name fails in an enterprise account.

Somewhat than widening the allowlist, scope the exception. Amazon Bedrock populates the bedrock:InferenceProfileArn situation key throughout authorization, so an SCP can enable these profiles whereas holding Area controls for every thing else. The next SCP does that.

{
  "Model": "2012-10-17",
  "Assertion": [
    {
      "Sid": "DenyOutsideApprovedRegionsExceptBedrockInvoke",
      "Effect": "Deny",
      "NotAction": [
        "iam:*", "sts:*", "organizations:*", "account:*",
        "route53:*", "cloudfront:*", "waf:*", "wafv2:*", "shield:*",
        "support:*", "trustedadvisor:*", "health:*",
        "budgets:*", "ce:*", "cur:*", "tax:*", "billing:*",
        "bedrock:Invoke*"
      ],
      "Useful resource": "*",
      "Situation": {
        "StringNotEquals": { "aws:RequestedRegion": ["ap-southeast-1"] }
      }
    },
    {
      "Sid": "DenyBedrockInferenceOutsideApprovedRegionsUnlessCris",
      "Impact": "Deny",
      "Motion": ["bedrock:Invoke*"],
      "Useful resource": "*",
      "Situation": {
        "StringNotEquals": { "aws:RequestedRegion": ["ap-southeast-1"] },
        "ArnNotLike": {
          "bedrock:InferenceProfileArn": [
            "arn:aws:bedrock:*:*:inference-profile/global.*"
          ]
        }
      }
    }
  ]
}

The primary assertion enforces your Area allowlist for each service besides Amazon Bedrock inference. The second denies that inference outdoors these Areas until it routes by means of a world cross-Area inference profile. For geographic profiles, add their prefix (for instance inference-profile/us.*) to the ArnNotLike checklist, as a result of geographic requests carry an actual Area title. Hold your supply Area within the allowlist, and add the worldwide providers your group makes use of to NotAction.

A blunter different is so as to add unspecified to your present allowlist, which works however opens the identical exception to each different service. For a ready-to-use coverage file, see the aws-samples service-control-policy-examples repository.

Knowledge retention and abuse detection

For automated offline abuse detection, classifier-flagged visitors is retained by AWS for as much as 30 days and processed programmatically. Underneath cross-Area inference that retention occurs within the vacation spot Area, which for a world profile is any supported industrial AWS Area the place the mannequin is deployed. Eligible prospects would possibly request full zero information retention by means of their AWS account crew. For extra data, see Amazon Bedrock abuse detection and Knowledge retention.

In case your group has information residency necessities, take this to your threat and compliance crew early. See the AWS Monetary Providers Compliance Middle nation pages for these 5 nations. We suggest you acquire your individual recommendation on compliance with the necessities related to what you are promoting.

Quota administration

GPT-6 quotas on bedrock-runtime are measured in tokens per minute (TPM), tracked towards the supply Area you name from. Enter and cache-write tokens rely 1:1, and every output token consumes 10 tokens of quota. For instance, a request with 2,000 enter tokens and 1,000 output tokens depletes 12,000 tokens out of your quota.

Amazon Bedrock reserves the complete max_tokens up entrance, so set it near the completions you count on. One quota covers InvokeModel, Converse, Responses, and Chat Completions. Assessment and lift it on the Service Quotas console out of your supply Area. Earlier than manufacturing, request will increase early and alarm on EstimatedTPMQuotaUsage within the AWS/Bedrock namespace at 70–80 p.c of your allocation. See How tokens are counted in Amazon Bedrock and Amazon Bedrock endpoints and quotas for present values.

Monitoring and logging

Your operational floor stays in your supply Area. Amazon CloudWatch metrics, AWS CloudTrail occasions, mannequin invocation logs, quota accounting, and billing all stay within the Area you known as from. You pay that Area’s charges, not the processing Area’s.

CloudTrail information the place every request was processed. AWS paperwork a world cross-Area inference profile’s locations as its industrial Areas fairly than enumerating them. GetInferenceProfile returns the Area-agnostic and source-Area mannequin ARNs fairly than a vacation spot checklist. Your per-request file is additionalEventData.inferenceRegion in CloudTrail in your supply Area. Inference on this endpoint is a administration occasion, logged by default at no further value. CloudTrail Occasion Historical past retains 90 days, so longer horizons want a path or mannequin invocation logging. Retrieve the CloudTrail occasions with this command.

aws cloudtrail lookup-events 
  --region ap-southeast-1 
  --lookup-attributes AttributeKey=EventSource,AttributeValue=bedrock.amazonaws.com 
  --start-time <START> 
  --end-time <END> 
  --max-results 1 
  --query 'Occasions[0].CloudTrailEvent' --output textual content | python3 -m json.instrument

Set the window round a name you simply made, as a result of CloudTrail supply lags by a couple of minutes. If the latest occasion was a failed name, it carries an errorCode and no inferenceRegion. Increase --max-results and choose an occasion with out errorCode.

{
  "eventName": "Converse",
  "eventSource": "bedrock.amazonaws.com",
  "awsRegion": "ap-southeast-1",
  "managementEvent": true,
  "requestParameters": {
    "modelId": "world.openai.gpt-6-sol",
    "inferenceConfig": { "maxTokens": 1024 }
  },
  "additionalEventData": {
    "inferenceRegion": "us-east-1"
  }
}

Metrics and coding-agent utilization keep in CloudWatch. The place builders attain GPT-6 by means of a coding agent, CloudWatch Coding Agent Insights provides ready-made dashboards below GenAI Observability that break token use down by person, crew, and price middle. The brokers emit OpenTelemetry metrics natively. For Codex, see Arrange OpenTelemetry for OpenAI Codex.

Clear up

This put up provides an IAM coverage and optionally an SCP assertion and a logging vacation spot. Detach or delete the IAM coverage you connected, and revert the SCP entry when you added one. Delete the mannequin invocation logging vacation spot in Amazon Easy Storage Service (Amazon S3) or Amazon CloudWatch Logs when you created one, to keep away from storage prices. The worldwide cross-Area inference profiles are system-defined, so there may be nothing to delete.

Conclusion

This put up confirmed the way to invoke GPT-6 Sol and GPT-6 Luna from 5 Asia Pacific supply Areas, and GPT-6 Astra from Asia Pacific (Singapore), by means of world cross-Area inference on bedrock-runtime. Move a world cross-Area inference profile ID because the mannequin parameter, and configure the IAM coverage and SCP exception from this put up. The identical endpoint serves the OpenAI-compatible APIs, so an present OpenAI integration strikes throughout by altering a base URL and a key.

To get began, connect the IAM coverage from this put up, and ship a primary request by means of world.openai.gpt-6-sol. Dimension your tokens-per-minute quota on the 10x burndown fee, and make sure in CloudTrail that additionalEventData.inferenceRegion seems in your calls.

For extra data, see the Amazon Bedrock Consumer Information, Route mannequin inference requests throughout AWS Areas with cross-Area inference, and Amazon Bedrock pricing.

Concerning the authors