NVIDIA Nemotron 3 Nano 30B MoE mannequin now out there on Amazon SageMaker JumpStart

At this time, NVIDIA Nemotron 3 The Nano 30B mannequin with 3B lively parameters is now typically out there within the Amazon SageMaker JumpStart Mannequin Catalog. Nemotron 3 Nano on Amazon Internet Companies (AWS) allows you to speed up innovation and ship tangible enterprise worth with out managing complicated mannequin deployments. SageMaker JumpStart gives managed deployment capabilities that you need to use to energy your generated AI functions with Nemotron capabilities.

Nemotron 3 Nano is a small-language hybrid mixed-expert (MoE) mannequin with the very best computational effectivity and accuracy, permitting builders to drive extremely expert agent duties at scale. The mannequin is totally open with open weights, datasets, and recipes, permitting builders to seamlessly customise, optimize, and deploy the mannequin to their infrastructure to satisfy privateness and safety necessities. Nemotron 3 Nano excels in coding and reasoning, and excels in benchmarks comparable to SWE Bench Verified, GPQA Diamond, AIME 2025, Enviornment Laborious v2, and IFBench.

About Nemotron 3 Nano 30B

Nemotron 3 Nano is differentiated from different fashions by its structure and precision, boasting sturdy efficiency with quite a lot of superior technical expertise.

• Structure:
  • ο MoE with hybrid Transformer-Mamba structure ο Helps token price range to offer optimum accuracy with minimal inference token technology
• Accuracy:
  • Superior accuracy for coding, scientific reasoning, arithmetic, and following directions
  • Leads in benchmarks like LiveCodeBench, GPQA Diamond, AIME 2025, BFCL, IFBench (in comparison with different open language fashions underneath 30B)
• Ease of use:
  • 30B parameter mannequin with 3 billion lively parameters
  • Has a context window of as much as 1 million tokens
  • Textual content-based fundamental mannequin. Use textual content for each enter and output.

Stipulations

To start out utilizing Nemotron 3 Nano with Amazon SageMaker JumpStart, you want a provisioned Amazon SageMaker Studio area.

Attempt utilizing NVIDIA Nemotron 3 Nano 30B with SageMaker JumpStart

To check the Nemotron 3 Nano mannequin with SageMaker JumpStart, open and choose SageMaker Studio mannequin within the navigation pane. Seek for “NVIDIA” within the search bar and choose it NVIDIA Nemotron 3 Nano 30B As a mannequin.

On the mannequin particulars web page, increase Observe the prompts to deploy the mannequin.

As soon as your mannequin is deployed to your SageMaker AI endpoint, you may take a look at it. You may entry the mannequin utilizing the next AWS Command Line Interface (AWS CLI) code instance. can be utilized nvidia/nemotron-3-nano as a mannequin ID.

cat > enter.json << EOF
{
"mannequin": "${MODEL_ID}",
"messages": [
{
 	"role": "system",
 	"content": "You are a helpful assistant."
 },
 {
 	"role": "user",
       	"content": "What is NVIDIA? Answer in 2-3 sentences."
}],
"max_tokens": 512,
"temperature": 0.2,
"stream": False, # Set to False for non-streaming mode,
   	"chat_template_kwargs": {"enable_thinking": False} # Set to False for non-reasoning mode
}
EOF
 
aws sagemaker-runtime invoke-endpoint 
--endpoint-name ${ENDPOINT_NAME} 
--region ${AWS_REGION} 
--content-type 'utility/json' 
--body fileb://enter.json 
> response.json

Alternatively, you may entry the mannequin utilizing the SageMaker SDK and Boto3 code. The next Python code instance reveals ship a textual content message to an NVIDIA Nemotron 3 Nano 30B utilizing the SageMaker SDK. For added code examples, see. NVIDIA GitHub repository.

runtime_client = boto3.consumer('sagemaker-runtime', region_name=area) 
payload = {
        "messages": [
            {"role": "user", "content": prompt}
        ],
        "max_tokens": 1000
    }
    
    strive:
        response = self.runtime_client.invoke_endpoint(
            EndpointName=self.endpoint_name,
            ContentType="utility/json",
            Physique=json.dumps(payload)
        )
        
        response_body = response['Body'].learn().decode('utf-8')
        raw_response = json.masses(response_body)
        
        # Parse the response utilizing our customized parser
        return self.parse_response(raw_response)
        
    besides Exception as e:
        elevate Exception(
            f"Did not invoke endpoint '{self.endpoint_name}': {str(e)}. "
            f"Examine that the endpoint is InService and you've got least-privileged IAM permissions assigned."
        )

at present out there

NVIDIA Nemotron 3 Nano is now totally managed and out there with SageMaker JumpStart. See the mannequin bundle for out there AWS Areas. If you want to study extra, please see beneath. Nemotron Nano model page, NVIDIA GitHub Sample Notebook for Nemotron 3 Nano 30Band the Amazon SageMaker JumpStart pricing web page.

Attempt the Nemotron 3 Nano mannequin as we speak with Amazon SageMaker JumpStart and ship us your suggestions. AWS re:Post for SageMaker JumpStart or by way of your common AWS Assist contacts.

Concerning the writer

Dan Ferguson I am an AWS options architect primarily based in New York, USA. Dan is a machine studying providers knowledgeable devoted to serving to prospects combine ML workflows effectively, successfully, and sustainably.

Pooja Karaj He leads product and strategic partnerships for Amazon SageMaker JumpStart, the machine studying and generative AI hub inside SageMaker. She is concentrated on accelerating prospects’ AI adoption by simplifying the invention and deployment of underlying fashions, enabling them to construct production-ready, generative AI functions throughout the mannequin lifecycle, from onboarding to customization to deployment.

benjamin crabtree He’s a senior software program engineer on the Amazon SageMaker AI workforce, specializing in delivering “final mile” experiences to prospects. He’s obsessed with democratizing the most recent synthetic intelligence breakthroughs by offering easy-to-use options. Ben additionally has intensive expertise constructing large-scale machine studying infrastructure.

timothy ma He’s a lead specialist in generative AI at AWS, working with prospects to design and deploy cutting-edge machine studying options. He additionally leads go-to-market methods for generative AI providers, serving to organizations harness the potential of superior AI applied sciences.

Abdullahi Olaoye He’s a Senior AI Options Architect at NVIDIA, specializing in integrating NVIDIA AI libraries, frameworks, and merchandise with cloud AI providers and open supply instruments to optimize AI mannequin deployment, inference, and technology AI workflows. He works with AWS to reinforce the efficiency of AI workloads and drive adoption of NVIDIA-powered AI and generative AI options.

Nirmal Kumar Jullu He’s a product advertising supervisor at NVIDIA, driving the adoption of AI software program, fashions, and APIs within the NVIDIA NGC catalog and NVIDIA AI Basis fashions and endpoints. He beforehand labored as a software program developer. Nirmal holds an MBA from Carnegie Mellon College and a Bachelor’s diploma in Pc Science from BITS Pilani.

vivian chen As a Deep Studying Options Architect at NVIDIA, I assist groups bridge the hole between complicated AI analysis and real-world efficiency. Vivian focuses on inference optimization and cloud-integrated AI options, with a deal with turning the heavy lifting of machine studying into quick, scalable functions. She is obsessed with serving to purchasers navigate NVIDIA’s accelerated computing stack to make sure their fashions not solely work within the lab, but in addition in manufacturing.