MBZUAI’s IFM Releases PAN 2, an Open 29B World Mannequin for Bodily AI

The Institute of Basis Fashions (IFM) at MBZUAI launched PAN 2, an open world mannequin for bodily intelligence. The PAN 2 world mannequin takes a scene and a textual content motion and simulates the result as video. IFM announced it on X on November 10, 2026. Weights, training code and inference code ship beneath Apache 2.0. For robotics and agent builders, the pitch is easy: check a plan in simulation earlier than appearing on it.

TL;DR

Measurement: 29B-parameter diffusion transformer (MMDiT), dense. Inputs: textual content plus an non-obligatory first body. Output: 704×1248 video at 24 fps, as much as 20 s.
Runs on: Information-center GPUs. Every employee holds the total 29B mannequin in BF16, with block offloading as fallback.
Efficiency: Leads 6 examined fashions on full plans. Trails MiniMax-H3 on single actions and human choice.
Finest: 34.0% full-plan completion on 100 multi-step duties, the very best of all fashions examined (self-reported).
Worst: About 2 in 3 plans miss a step. Physics-IQ system default: 37.48 vs 42.91 for MiniMax-H3.
Backside line: Finest: a completely open world mannequin that leads on multi-step plans. Worst: lengthy plans nonetheless fail usually, and {hardware} wants are excessive.

What’s PAN 2?

PAN 2 is a general-purpose world mannequin from IFM that simulates how a scene adjustments after an motion. It follows the Generative Latent Prediction (GLP) structure from IFM’s Critique of World Models paper. It succeeds PAN, launched in November 2025.

The objective is simulative reasoning. An agent proposes plans, PAN 2 simulates each, and the agent acts on the very best.

How does PAN 2 work?

PAN 2 splits simulation into 2 sorts of reasoning:

Conceptual reasoning: An LLM reads the commentary and the requested actions. It writes the subsequent world state as structured JSON: occasions, timed actions sure to actors, and digicam movement. Any instruction-following LLM can fill this position. IFM’s report makes use of Kimi K3 by default.
Perceptual reasoning: A 29B MMDiT predicts the video latents. IFM trained it from scratch on lots of of thousands and thousands of annotated movies. A frozen Qwen3.5-9B textual content encoder circumstances it. The Wan 2.2 VAE decoder renders the ultimate frames.

IFM’s ablation exhibits the LLM issues. With no conceptual reasoning, full-plan completion drops from 34.0% to 0.0%. Step completion falls from 75.1% to eight.7%. A format-only rewrite recovers a part of the hole, reaching 27.8% full-plan completion.

IFM additionally dropped gameplay, animation and simulation footage after early coaching. A 9B check mannequin educated with out it gained 58% of blinded votes.

<div class="pane on" id="p-how">
<div class="pipe">
<div class="node" data-i="0"><div class="n">STEP 1</div><div class="t">Observe</div><div class="s">Non-compulsory picture + textual content motion</div></div>
<div class="node" data-i="1"><div class="n">STEP 2</div><div class="t">Conceptual</div><div class="s">LLM writes the subsequent state</div></div>
<div class="node" data-i="2"><div class="n">STEP 3</div><div class="t">Encode</div><div class="s">Frozen Qwen3.5-9B</div></div>
<div class="node" data-i="3"><div class="n">STEP 4</div><div class="t">Perceptual</div><div class="s">29B MMDiT predicts latents</div></div>
<div class="node" data-i="4"><div class="n">STEP 5</div><div class="t">Render</div><div class="s">Wan 2.2 VAE decodes video</div></div>
</div>
<div class="move"><i id="mtpFlow"></i></div>
<div class="element" id="mtpDetail"></div>
<div class="ctrl">
<button class="btn" id="mtpPrev"> Prev</button>
<button class="btn pri" id="mtpNext">Subsequent </button>
<button class="btn" id="mtpPlay"> Auto-play</button>
<span class="trace">Click on any step to leap</span>
</div>
</div>

<div class="pane" id="p-abl">
<div class="seg" id="mtpAblSeg">
<button class="chip" data-a="0">None: motion despatched straight to diffusion</button>
<button class="chip" data-a="1">Format-only rewrite</button>
<button class="chip on" data-a="2">Full conceptual reasoning</button>
</div>
<div class="abl">
<div class="card"><div class="ok">6 capabilities</div><div class="v" id="ab0">84.6<small>%</small></div><div class="mini"><i id="ib0"></i></div></div>
<div class="card"><div class="ok">Step completion</div><div class="v" id="ab1">75.1<small>%</small></div><div class="mini"><i id="ib1"></i></div></div>
<div class="card"><div class="ok">Full-plan completion</div><div class="v" id="ab2">34.0<small>%</small></div><div class="mini"><i id="ib2"></i></div></div>
</div>
<p class="notice">IFM ablation with Kimi K3 because the reasoner; the diffusion mannequin stays fastened. With out the LLM, no multi-step plan completes. Supply: IFM technical weblog.</p>
</div>

<div class="pane" id="p-run">
<div class="grid">
<div class="spec"><div class="ok">Generator</div><div class="v">29B MMDiT, BF16, educated from scratch</div></div>
<div class="spec"><div class="ok">Duties</div><div class="v">Textual content-to-video, image-to-video</div></div>
<div class="spec"><div class="ok">Output</div><div class="v">704×1248 at 24 fps, as much as 20 s (481 frames)</div></div>
<div class="spec"><div class="ok">Default clip</div><div class="v">241 frames, about 10 s</div></div>
<div class="spec"><div class="ok">Sampler</div><div class="v">50 steps · move shift 9 · CFG 2.5</div></div>
<div class="spec"><div class="ok">Caption finances</div><div class="v">1,536 tokens arduous restrict</div></div>
<div class="spec"><div class="ok">Serving</div><div class="v">Diffusers · vLLM · FastVideo · generate.py</div></div>
<div class="spec"><div class="ok">GPU reminiscence</div><div class="v">Full 29B per employee; block offload fallback; VRAM not disclosed</div></div>
<div class="spec"><div class="ok">Security</div><div class="v">pan2-guardrail: unsafe T2V 33.9% → 1.5%</div></div>
<div class="spec"><div class="ok">License</div><div class="v">Apache 2.0: weights, coaching + inference code</div></div>
</div>
<pre>hf obtain IFM/PAN2 –include "transformer/*"
hf obtain Qwen/Qwen3.5-9B
hf obtain Wan-AI/Wan2.2-TI2V-5B-Diffusers</pre>
</div>

<div class="ft">
<span>Sources: <a href="https://huggingface.co/IFM/PAN2" goal="_blank" rel="noopener">HF mannequin card</a> · <a href="https://ifm.ai/weblog/pan2/" goal="_blank" rel="noopener">IFM weblog</a> · scores self-reported</span>
<span class="mtp">© Marktechpost</span>
</div>
</div>
<script>
(operate(){
var R=doc.getElementById(‘mtp-pan2′);
operate submit(){strive{mum or dad.postMessage({sort:’mtp-pan2-h’,h:R.offsetHeight+40},’*’)}catch(e){}}
// tabs
R.querySelectorAll(‘.tab’).forEach(operate(b){b.addEventListener(‘click on’,operate(){
R.querySelectorAll(‘.tab’).forEach(operate(x){x.classList.take away(‘on’)});
R.querySelectorAll(‘.pane’).forEach(operate(x){x.classList.take away(‘on’)});
b.classList.add(‘on’);doc.getElementById(‘p-‘+b.dataset.p).classList.add(‘on’);
if(b.dataset.p===’bench’)drawBars(cur);
if(b.dataset.p===’abl’)setAbl(abl);
setTimeout(submit,60);
})});
// pipeline
var D=[
[‘Observe: current world + action’,’Input is an optional starting image plus text, and actions described in text with a duration. Example: "Accelerate mangrove forest growth to protect the coast."’,”],
[‘Conceptual reasoning backbone (LLM)’,’An instruction-following LLM reasons over the scene and the action. It writes the next world state as structured, time-grounded JSON. IFM’s report uses Kimi K3 by default.’,'{ "events": [{"t":"0-3s","actor":"surfer","action":"paddles"},n {"t":"3s","actor":"surfer","action":"pops up"}],n "digicam": "sluggish pan proper" }’],
[‘Frozen text encoder’,’Qwen3.5-9B maps the structured caption to conditioning features. Captions have a soft limit of 1,480 and a hard limit of 1,536 tokens.’,”],
[‘Perceptual reasoning backbone (29B MMDiT)’,’A 29B diffusion transformer, trained from scratch on hundreds of millions of videos in the same caption format, fills in geometry, contact and motion. It predicts video latents conditioned on the state and the starting frame.’,”],
[‘Render the next observation’,’The Wan 2.2 VAE decoder turns latents into 704×1248 video at 24 fps, up to 20 s. An agent compares simulated outcomes and acts on the best plan.’,”]
];
var step=0,timer=null,nodes=R.querySelectorAll(‘.node’);
operate present(i){step=(i+5)%5;nodes.forEach(operate(n,ok){n.classList.toggle(‘on’,ok===step);n.classList.toggle(‘achieved’,ok<step)});
doc.getElementById(‘mtpFlow’).fashion.width=((step+1)/5*100)+’%’;
var d=D[step];doc.getElementById(‘mtpDetail’).innerHTML='<h4>’+d[0]+'</h4><p>’+d[1]+'</p>’+(d[2]?'<pre>’+d[2]+'</pre>’:”);setTimeout(submit,30);}
nodes.forEach(operate(n){n.addEventListener(‘click on’,operate(){cease();present(+n.dataset.i)})});
doc.getElementById(‘mtpNext’).onclick=operate(){cease();present(step+1)};
doc.getElementById(‘mtpPrev’).onclick=operate(){cease();present(step-1)};
var pb=doc.getElementById(‘mtpPlay’);
operate cease(){if(timer){clearInterval(timer);timer=null;pb.textContent=” Auto-play”}}
pb.onclick=operate(){if(timer){cease();return}pb.textContent=”❚❚ Pause”;timer=setInterval(operate(){present(step+1)},2600)};
present(0);
// bars
var M={
plan:{n:’Full-plan completion’,u:’%’,max:40,notice:’100 selfish duties, 456 steps. Share of simulations with each step achieved. † closed mannequin, single seed.’,d:[[‘PAN 2’,34.00,1],[‘MiniMax-H3’,32.50],[‘FLUX 3 †’,26.77,0,1],[‘Seedance 2.5 †’,24.49,0,1],[‘Cosmos 3 Super’,13.83],[‘LTX 2.5′,13.00]]},
step:{n:’Step completion’,u:’%’,max:80,notice:’Share of particular person plan steps carried out. PAN 2 and MiniMax-H3 are successfully tied.’,d:[[‘MiniMax-H3’,75.12],[‘PAN 2’,75.10,1],[‘FLUX 3 †’,72.44,0,1],[‘Seedance 2.5 †’,68.95,0,1],[‘Cosmos 3 Super’,57.23],[‘LTX 2.5′,55.77]]},
six:{n:’6 capabilities’,u:’%’,max:90,notice:’300 single-action instances (201 I2V, 99 T2V). MiniMax-H3 leads all 6 classes.’,d:[[‘MiniMax-H3’,87.20],[‘PAN 2’,84.48,1],[‘Seedance 2.5 †’,78.59,0,1],[‘FLUX 3 †’,77.41,0,1],[‘LTX 2.5’,72.25],[‘Cosmos 3 Super’,71.34]]},
piq:{n:’Physics-IQ Verified’,u:”,max:55,notice:’Leaderboard setting; real-take variance = 100. PAN 2 makes use of Claude Opus 5.5 as reasoner right here. LTX 2.5 not on leaderboard.’,d:[[‘FLUX 3 †’,51.11,0,1],[‘PAN 2’,49.38,1],[‘Cosmos 3 Super’,48.62],[‘Seedance 2.5 †’,42.43,0,1],[‘MiniMax-H3’,39.79],[‘Omni Flash 1.1 †’,35.34,0,1]]},
rb:{n:’RBench’,u:”,max:65,notice:’Robotics benchmark, unweighted imply x100.’,d:[[‘Omni Flash 1.1 †’,60.79,0,1],[‘FLUX 3 †’,57.96,0,1],[‘PAN 2’,57.04,1],[‘Seedance 2.5 †’,56.38,0,1],[‘MiniMax-H3’,56.16],[‘LTX 2.5’,51.55],[‘Cosmos 3 Super’,50.64]]},
elo:{n:’Human Elo’,u:”,max:1700,min:1250,notice:’Blind pairwise choice: 140 prompts, 13 reviewers, 1,400 comparisons. Open fashions solely.’,d:[[‘MiniMax-H3’,1620],[‘PAN 2 (GPT-6 Astra)’,1595,1],[‘PAN 2 (Claude Opus)’,1559,1],[‘Cosmos 3 Super’,1380],[‘LTX 2.5’,1346]]}
};
var cur=”plan”,seg=doc.getElementById(‘mtpSeg’);
Object.keys(M).forEach(operate(ok){var b=doc.createElement(‘button’);b.className=”chip”+(ok===cur?’ on’:”);b.textContent=M[k].n;b.onclick=operate(){cur=ok;seg.querySelectorAll(‘.chip’).forEach(operate(x){x.classList.take away(‘on’)});b.classList.add(‘on’);drawBars(ok)};seg.appendChild(b)});
operate drawBars(ok){var m=M[k],field=doc.getElementById(‘mtpBars’);field.innerHTML=”;
m.d.forEach(operate(r){var row=doc.createElement(‘div’);row.className=”row”+(r[2]?’ me’:”);
var v=m.u===’%’?r[1].toFixed(2)+’%’:(ok===’elo’?String(r[1]):r[1].toFixed(2));
row.innerHTML='<div class="lab’+(r[3]?’ cl’:”)+’">’+r[0]+'</div><div class="trk"><div class="fill"></div></div><div class="val">’+v+'</div>’;field.appendChild(row);
var mn=m.min||0,w=Math.max(2,(r[1]-mn)/(m.max-mn)*100);
requestAnimationFrame(operate(){requestAnimationFrame(operate(){row.querySelector(‘.fill’).fashion.width=w+’%’})});});
doc.getElementById(‘mtpNote’).textContent=m.notice+(m.min?’ Axis begins at ‘+m.min+’.’:”);setTimeout(submit,60);}
// ablation
var A=[[45.1,8.7,0.0],[78.2,70.7,27.8],[84.6,75.1,34.0]],abl=2;
operate setAbl(i){abl=i;R.querySelectorAll(‘#mtpAblSeg .chip’).forEach(operate(c){c.classList.toggle(‘on’,+c.dataset.a===i)});
[0,1,2].forEach(operate(j){var el=doc.getElementById(‘ab’+j),to=A[i][j],from=parseFloat(el.textContent)||0,t0=null;
operate tick(ts){if(!t0)t0=ts;var p=Math.min(1,(ts-t0)/600),v=from+(to-from)*p;el.innerHTML=v.toFixed(1)+'<small>%</small>’;if(p<1)requestAnimationFrame(tick)}
requestAnimationFrame(tick);doc.getElementById(‘ib’+j).fashion.width=to+’%’;});}
R.querySelectorAll(‘#mtpAblSeg .chip’).forEach(operate(c){c.onclick=operate(){setAbl(+c.dataset.a)}});
window.addEventListener(‘load’,submit);window.addEventListener(‘resize’,submit);setTimeout(submit,300);
})();
</script>
</physique>
</html>
“>

(operate(){var f=doc.getElementById(‘mtp-pan2-frame’);window.addEventListener(‘message’,operate(e){if(e.knowledge&&e.knowledge.sort===’mtp-pan2-h’&&e.supply===f.contentWindow){f.fashion.top=e.knowledge.h+’px’;}});})();

How does PAN 2 carry out on benchmarks?

All numbers beneath are self-reported by IFM. Its personal check suite covers 6 capabilities throughout 300 single-action instances, plus 100 multi-step duties.

Full-plan completion: 34.00% vs 32.50% for MiniMax-H3, 26.77% for FLUX 3 and 24.49% for Seedance 2.5.
Step completion: 75.10%, stage with MiniMax-H3 at 75.12%.
6 capabilities general: 84.48% vs 87.20% for MiniMax-H3. MiniMax-H3 leads in all 6 classes.
Physics-IQ Verified (leaderboard setting): 49.38, first amongst open fashions. Closed FLUX 3 scores 51.11.
RBench: 57.04 vs 56.16 for MiniMax-H3 and 50.64 for Cosmos 3 Tremendous.
Human Elo: 1595 with a GPT-6 Astra reasoner, behind MiniMax-H3 at 1620.

PAN 2 vs MiniMax-H3, Cosmos 3 Tremendous and LTX-2.5

Characteristic	PAN 2	MiniMax-H3	Cosmos 3 Super	LTX-2.5
Developer	IFM (MBZUAI)	MiniMax	NVIDIA	Lightricks
Generator dimension	29B MMDiT	33B dense transformer (~13B in cacheable AdaLN branches)	64B Combination-of-Transformers	22B transformer (per checkpoint file names)
Planner / encoder	Any LLM + frozen Qwen3.5-9B encoder	Qwen3-VL-32B encoder; hosted Context-IR	Constructed-in autoregressive tower	Gemma-based textual content encoder
Max clip	20 s, 704×1248, 24 fps	15 s, 24 fps, 768p (2K by way of hosted module)	400 frames	Not disclosed
Audio	No	Sure, stereo	Sure	Sure
License	Apache 2.0	MiniMax H3 Neighborhood License	OpenMDW 1.1	LTX-2.x Neighborhood License
Coaching code	Sure	Not disclosed	Framework on GitHub	Not disclosed
Full-plan completion*	34.00%	32.50%	13.83%	13.00%
Physics-IQ Verified (leaderboard)*	49.38	39.79	48.62	Not reported
Human Elo*	1595	1620	1380	1346

*Scores from IFM’s PAN 2 model card, run by IFM. The weblog stories completely different Cosmos 3 Tremendous and LTX-2.5 plan scores.

How do you run PAN 2?

PAN 2 helps text-to-video and image-to-video. You’ll be able to run it by Hugging Face Diffusers, vLLM, FastVideo or the repo’s generate.py.

The default sampler makes use of 50 steps, move shift 9 and steering 2.5. A clip defaults to 241 frames, about 10 seconds. Every GPU employee masses the total 29B mannequin in BF16. IFM publishes no VRAM determine or era time.

Prompts work finest as structured captions. IFM ships a Conceptual Reasoner skill for Claude Code and Codex that rewrites free textual content. Captions have a tough finances of 1,536 tokens.

IFM strongly recommends the separate pan2-guardrail. In IFM exams, it minimize unsafe outputs from dangerous T2V requests from 33.9% to 1.5%. It additionally blocked 11.5% of benign T2V requests.

Key Takeaways

PAN 2 pairs an LLM planner with a 29B diffusion mannequin educated from scratch.
It leads 6 examined fashions on full-plan completion at 34.0%.
MiniMax-H3 nonetheless wins single actions and human choice.
Weights, coaching and inference code are Apache 2.0.
Eradicating the LLM planner drops full-plan completion to 0%.

Try the model weights, GitHub repo, technical blog and press release. Additionally, be happy to comply with us on Twitter and don’t neglect to hitch our 150k+ML SubReddit and Subscribe to our E-newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Must companion with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and so forth.? Connect with us

The submit MBZUAI’s IFM Releases PAN 2, an Open 29B World Mannequin for Bodily AI appeared first on MarkTechPost.

MBZUAI’s IFM Releases PAN 2, an Open 29B World Mannequin for Bodily AI

TL;DR

What’s PAN 2?

How does PAN 2 work?

How does PAN 2 carry out on benchmarks?

PAN 2 vs MiniMax-H3, Cosmos 3 Tremendous and LTX-2.5

How do you run PAN 2?

Key Takeaways

What market knowledge reveals about AI search

Converter

Editors Pick

Newsletter

Categories

Related Posts

Leave a Comment Cancel Reply

Latest

Best selling