From Gemma 3 270M to FunctionGemma, how Google AI constructed a compact function-call specialist for edge workloads

Google has launched FunctionGemma, a particular model of the Gemma 3 270M mannequin. It’s particularly skilled for operate calls and designed to run as an edge agent that maps pure language to executable API actions.

So what’s FunctionGemma?

FunctionGemma is a 270M parameter text-only transformer primarily based on Gemma 3 270M. It maintains the identical structure as Gemma 3 and is launched as an open mannequin beneath the Gemma license, however its coaching targets and chat format are centered on operate calls somewhat than free-form dialogs.

This mannequin is meant to be fine-tuned for particular function-calling duties. It’s not positioned as a common chat assistant. The principle design aim is to transform consumer directions and gear definitions into structured operate calls and optionally summarize the device’s response to the consumer.

From an interface perspective, FunctionGemma is offered as an ordinary causal language mannequin. The enter and output are textual content sequences, the enter context is 32K tokens, and the output funds is as much as 32K tokens per request, shared with the enter size.

Structure and coaching knowledge

This mannequin makes use of the Gemma 3 transformer structure and the identical 270M parameter scale because the Gemma 3 270M. The coaching and runtime stack reuses the analysis and infrastructure utilized in Gemini, together with JAX and ML pathways on massive TPU clusters.

FunctionGemma makes use of Gemma’s 256K vocabulary, which is optimized for JSON constructions and multilingual textual content. This improves token effectivity for operate schemas and gear responses, and reduces sequence size for latency and memory-intensive edge expansions.

The mannequin is skilled with 6T tokens and has a information cutoff in August 2024. The dataset focuses on two fundamental classes.

• Public instruments and API definitions
• The device makes use of interactions that embrace prompts, operate calls, operate responses, and follow-up messages in pure language that summarize the output or request clarification.

This coaching sign teaches each the syntax, equivalent to which operate to name and format the arguments, and the intent, equivalent to when to name the operate and when to request extra data.

Dialog model and management tokens

FunctionGemma doesn’t use a free-form chat format. We count on strict dialog templates that separate roles and tool-related areas. Dialog turns are wrapped as <start_of_turn>function ... <end_of_turn> The place does the function normally lie? developer, consumer or mannequin.

Inside these turns, FunctionGemma depends on a hard and fast set of management token pairs.

• <start_function_declaration> and <end_function_declaration> For device definition
• <start_function_call> and <end_function_call> For mannequin device calls
• <start_function_response> and <end_function_response> For serialized device output

These markers permit the mannequin to tell apart between pure language textual content and useful schemas and execution outcomes. faces hugging one another apply_chat_template The API and official Gemma templates robotically generate this construction for messages and gear lists.

Fantastic-tuning and efficiency of cellular actions

FunctionGemma is already skilled to make use of general-purpose instruments out of the field. Nevertheless, the official Cell Actions information and mannequin card emphasize that small-scale fashions attain production-level reliability solely after some task-specific fine-tuning.

The Cell Actions demos use datasets, and every instance exposes a small set of instruments for interacting with the Android system, equivalent to creating contacts, setting calendar occasions, controlling a flashlight, and displaying maps. FunctionGemma learns to map utterances like “Create a calendar occasion for tomorrow’s lunch” or “Activate the flashlight” to instruments utilizing structured arguments.

Within the Cell Actions analysis, the bottom FunctionGemma mannequin reached 58% accuracy on the performed take a look at set. After fine-tuning utilizing a publicly obtainable cookbook recipe, the accuracy elevated to 85%.

Edge agent and reference demo

FunctionGemma’s major deployment goal is edge brokers operating domestically on telephones, laptops, and small accelerators equivalent to NVIDIA Jetson Nano. The low parameter depend of 0.3B and help for quantization allow low reminiscence and low latency inference on shopper {hardware}.

Google presents a number of reference experiences by the Google AI Edge Gallery

• cellular motion reveals a totally offline assistant-style agent for machine management utilizing FunctionGemma fine-tuned with the Cell Actions dataset and deployed to the machine.
• tiny backyard is a voice-controlled sport wherein the mannequin breaks down instructions equivalent to “plant and water sunflowers within the prime row” into domain-specific capabilities equivalent to: plant_seed and water_plots Use express grid coordinates.
• FunctionGemma Physics Playground Runs solely inside the browser utilizing Transformers.js, customers can clear up physics puzzles by pure language directions that the mannequin interprets into simulation actions.

These demos validate that with the suitable tweaks and tooling interfaces, the 270M parameter operate caller can help multi-step logic on the machine with out server calls.

Necessary factors

1. FunctionGemma is a text-only variant of Gemma 3 with 270M parameters, skilled particularly for operate calls somewhat than open-ended chat, and launched as an open mannequin beneath Gemma’s Phrases of Service.
2. The mannequin maintains a Gemma 3 transformer structure and a 256,000 token vocabulary, helps 32,000 tokens per request shared between enter and output, and is skilled on 6T tokens.
3. FunctionGemma makes use of strict chat templates. <start_of_turn>function ... <end_of_turn> Devoted management tokens for operate declarations, operate calls, and performance responses. Required for dependable device use in manufacturing methods.
4. Within the Cell Actions benchmark, accuracy elevated from 58% of the bottom mannequin to 85% after task-specific fine-tuning. This reveals that small operate callers require area knowledge greater than immediate engineering.
5. With 270 million scale and quantization help, FunctionGemma can run on telephones, laptops, and Jetson-class gadgets, and the mannequin is already built-in into ecosystems like Hugging Face, Vertex AI, LM Studio, and edge demos like Cell Actions, Tiny Backyard, and Physics Playground.

Please examine technical details and HF model. Please be happy to observe us too Twitter Remember to affix us 100,000+ ML subreddits and subscribe our newsletter. cling on! Are you on telegram? You can now also participate by telegram.