Cmpnd has launched dspy-wsp, a prototype optimizer that brings weight-space programming to DSPy. Weight-space programming is a method by which a frontier LLM reads one other mannequin’s weights and edits them immediately. There aren’t any gradients, no coaching loop and no fine-tuning dataset. dspy-wsp packages this as WeightSpace, an optimizer that plugs into any DSPy program and metric. In Cmpnd’s assessments, about $5 to $10 of GPT-6-Astra calls improved each determination mannequin they tried. You’ll be able to test the deployment choices and specs right here by our GPULeaderboard.com chrome extension.
TL;DR
- What it’s: A DSPy optimizer (
WeightSpace) the place a author LLM edits a pupil mannequin’s weights. MIT license,pip set up dspy-wsp, wants DSPy 3.5. - Measurement: Examined on open determination fashions from 0.6B to 9B parameters. One 25-pass run edited a single layer or head.
- Runs on: One Mac for all experiments. 2B fashions used about 6 GB of reminiscence. d1-omni-600M wants 4 GB or extra (Mac or GPU).
- Efficiency: WeightSpace alone added 1.9 to 16.1 factors on Typed Selections. With ReAnchor, positive aspects had been 6.5 to twenty.0 factors.
- Greatest: d1-omni-600M rose from 50.9% to 69.1% (+18.3) with WeightSpace + ReAnchor.
- Worst: Clef-Flash (9B) gained solely +1.9 from WeightSpace alone. With the unique paper’s stricter rule, d1-omni-600M gained simply +2.8.
- Backside line: Greatest: an inexpensive, gradient-free edit that ships as a small, verifiable patch. Worst: very experimental, gradual (78 to 167 minutes per run) and weakest on bigger fashions.
What’s weight-space programming?
Weight-space programming (WSP) treats a neural community’s weights as a program that may be learn and rewritten. The thought comes from MIT analysis: Sparks of Weight-Space Programming.
A frontier mannequin (the analysis group used GPT-6-Astra) appears to be like at weight tensors, causes about what every half computes, and adjustments them to match a objective written in plain English. The analysis group stresses that WSP names an rising functionality, not a hard and fast methodology. Gradients had been banned of their experiments, however the idea doesn’t rule them out.
How is it completely different from fine-tuning or immediate optimization?
It helps to match the three levers builders often pull:
- Immediate optimization (DSPy’s MIPROv2, GEPA) edits textual content: directions and examples. The mannequin’s weights by no means change.
- Positive-tuning (DSPy’s BootstrapFinetune, used inside BetterTogether) adjustments weights with gradient descent. It wants coaching information, a coaching loop and often a GPU.
- Weight-space programming adjustments weights by reasoning. The editor varieties a speculation (“these neurons over-score possibility B”), writes code to patch them, and checks the end result with a couple of ahead passes.
The research post frames WSP as the subsequent step after utilizing LLMs to optimize textual content artifacts. Earlier work let LLMs hill-climb on prompts, harnesses and coaching code. Weights had been left to Adam or to skilled weight-generating networks. WSP argues LLM reasoning can now attain the weights immediately.
What did the unique analysis present?
The MIT group ran 9 experiments. A number of are hanging:
- Studying weights: Astra named the proper sport (Breakout, Pong, Pac-Man, Freeway) for 4 PPO coverage networks from weights alone.
- Fixing a damaged coverage: A Kinetix car-on-ramp coverage failed all 256 episodes. One edit to three.5% of its weights made it achieve all 1,024 take a look at episodes.
- Writing from zero: On MinAtar Breakout, a zero-initialized community written in 1 shot scored 517.4. PPO reached 25.3 after 500M steps.
- Fixing an LLM: Qwen3.5-0.8B looped on math issues. Astra traced the problem to MLP layers 15 and 19 and adjusted 131,072 weights (0.016% of the mannequin). With 50 ahead passes, MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%.
The analysis additionally exhibits limits. On Gemma 3 270M, edits moved MATH-500 from 6.8% to between 7.0% and seven.7%. The analysis group didn’t rely that as an actual acquire. Modifying can take away an impediment, nevertheless it couldn’t add a lacking skill.
How does dspy-wsp work?
dspy-wsp turns that analysis loop into a regular DSPy optimizer. The GitHub Repo describes 3 actors:
- The coed: the small mannequin your DSPy program runs on. Its weights load into an area Python course of.
- The author: a frontier LLM, beneficial as
gpt-6-astraat most reasoning effort. - The sandbox: a kernel the place the author runs code. macOS makes use of built-in
sandbox-exec; Linux makes use of bubblewrap.
The author works in turns. Every flip has a brief rationalization, then 1 code block. That block is both:
- a Python block that reads or edits weights contained in the sandbox, or
- a ahead block that runs your DSPy program on observe examples.
Ahead passes come from a hard and fast price range (max_passes=25 within the assessments). When the price range runs out, the author’s finest edit turns into the patch. A hidden validation set then compares unique and edited weights on the identical examples and seeds.
What information does the author see?
The repo is specific about information separation:
trainsetis the author’s observe set.holdoutstrikes a fraction (0.25 within the instance) of coaching information right into a test set. The author can rating edits on it however by no means sees the examples.valsetstays hidden and runs as soon as on the finish.- Metrics can return
dspy.Prediction(rating=..., suggestions=...). The suggestions textual content is seen to the author.
What did the author really change?
The results write-up lists each edit. The edits are small and focused:
| Mannequin | Tensor edited | Scope | Time | Author price |
|---|---|---|---|---|
| d1-omni-600M | head.scorer.1.weight (+ bias, scorer.3.weight) |
112,640 of 1,048,576 entries, plus 220 | 78 min | $7.63 |
| NanoJev (Qwen3-0.6B) | scalar.weight |
1,024 entries | 167 min | $6.92 |
| d1-2B | layers.23.feed_forward.w2.weight |
All 15.7M entries, rank-9 change | 83 min | $5.42 |
d1-2B has no determination head, so the author edited its spine at layer 23. On d1-omni-600M, the author rewrote a part of the choice head’s scorer.
What does the code appear to be?
The Repo example triages assist tickets. The signature makes use of solely decision-type outputs:
import dspy
from typing import Literal
from dspy_wsp import WeightSpace, load_decision_backend
class Triage(dspy.Signature):
"""Triage a assist ticket."""
ticket: str = dspy.InputField()
pressing: bool = dspy.OutputField(desc="Does this ticket want a reply at this time?")
group: Literal["billing", "technical", "account"] = dspy.OutputField(desc="Which group ought to deal with it?")
backend = load_decision_backend("d1") # d1-omni-600M
optimizer = WeightSpace(
metric=metric,
writer_lm=dspy.LM("openai/gpt-6-astra", model_type="responses", reasoning_effort="max"),
backend=backend,
max_passes=25,
holdout=0.25,
run_dir="runs/triage",
)
compiled = optimizer.compile(dspy.Predict(Triage), trainset=practice, valset=take a look at)
compiled(ticket="I used to be charged twice this month.") # runs on the edited weights
After compile, this system calls the edited native mannequin as a substitute of a hosted API. compiled.wsp_patch.abstract() exhibits what modified.
<html lang="en"><head><meta charset="utf-8"><meta title="viewport" content material="width=device-width,initial-scale=1">
<title>Weight-House Programming Explainer | Marktechpost</title>
<fashion>
html,physique{margin:0;padding:0;background:clear}
#mtp-wsp{–bg:#F1F2F0;–card:#FFFFFF;–ink:#1D2320;–mute:#6B6866;–g:#3A8A50;–gd:#1E4A2C;–gl:#DCEBDF;–gr:#B0AAA6;–ch:#5B5452;–line:#D9DBD8;
max-width:880px;margin:0 auto;background:var(–bg)!essential;colour:var(–ink)!essential;border:1px stable var(–line)!essential;border-radius:14px;font-family:Inter,system-ui,-apple-system,Segoe UI,Roboto,sans-serif;overflow:hidden;box-sizing:border-box}
#mtp-wsp *{box-sizing:border-box}
#mtp-wsp hr,#mtp-wsp p:empty,#mtp-wsp del,#mtp-wsp s{show:none!essential}
#mtp-wsp .hd{padding:20px 24px 10px}
#mtp-wsp .kick{font-size:11px;letter-spacing:.12em;text-transform:uppercase;colour:var(–g)!essential;font-weight:700}
#mtp-wsp h3{margin:4px 0 4px;font-size:21px;line-height:1.25;colour:var(–ink)!essential}
#mtp-wsp .sub{margin:0;colour:var(–mute)!essential;font-size:13.5px}
#mtp-wsp .tabs{show:flex;hole:6px;padding:10px 24px 0;flex-wrap:wrap;border-bottom:1px stable var(–line)}
#mtp-wsp .tab{background:clear!essential;border:0!essential;border-bottom:3px stable clear!essential;padding:9px 10px;font:600 13px Inter,system-ui,sans-serif;colour:var(–mute)!essential;cursor:pointer;border-radius:0}
#mtp-wsp .tab.on{colour:var(–gd)!essential;border-bottom-color:var(–g)!essential}
#mtp-wsp .pn{show:none;padding:20px 24px 22px;animation:mtpf .35s ease}
#mtp-wsp .pn.on{show:block}
@keyframes mtpf{from{opacity:0;rework:translateY(6px)}to{opacity:1;rework:none}}
#mtp-wsp .card{background:var(–card)!essential;border:1px stable var(–line)!essential;border-radius:10px;padding:16px}
#mtp-wsp .row{show:flex;hole:12px;flex-wrap:wrap}
#mtp-wsp .btn{background:var(–card)!essential;border:1px stable var(–line)!essential;colour:var(–ink)!essential;font:600 13px Inter,system-ui,sans-serif;padding:8px 13px;border-radius:999px;cursor:pointer;transition:all .2s}
#mtp-wsp .btn:hover{border-color:var(–g)!essential}
#mtp-wsp .btn.on{background:var(–gd)!essential;colour:#fff!essential;border-color:var(–gd)!essential}
#mtp-wsp .lbl{font-size:12px;colour:var(–mute)!essential;text-transform:uppercase;letter-spacing:.08em;font-weight:700;margin:0 0 8px}
#mtp-wsp p{font-size:14px;line-height:1.55;margin:8px 0;colour:var(–ink)!essential}
#mtp-wsp code,#mtp-wsp pre{font-family:"JetBrains Mono",ui-monospace,Menlo,monospace!essential;background:#E6E9E6!essential;colour:var(–gd)!essential;border:0!essential;border-radius:5px;padding:1px 5px;font-size:12.5px}
/* levers diagram */
#mtp-wsp .lev{show:grid;grid-template-columns:1fr 60px 1fr;align-items:middle;hole:8px;margin-top:14px}
#mtp-wsp .field{border:2px stable var(–line);border-radius:10px;padding:12px;min-height:118px;background:#fff;transition:all .35s}
#mtp-wsp .field.scorching{border-color:var(–g);box-shadow:0 0 0 4px var(–gl)}
#mtp-wsp .field h4{margin:0 0 8px;font-size:13px;colour:var(–ink)}
#mtp-wsp .ptxt{font-size:12px;colour:var(–ch);line-height:1.5;font-family:ui-monospace,Menlo,monospace}
#mtp-wsp .ptxt .chg{background:#FBE7B5;transition:background .4s}
#mtp-wsp .wgrid{show:grid;grid-template-columns:repeat(16,1fr);hole:3px}
#mtp-wsp .wgrid i{show:block;aspect-ratio:1;border-radius:2px;background:#DADDD9;transition:background .4s,rework .4s}
#mtp-wsp .wgrid i.a{background:var(–g);rework:scale(1.15)}
#mtp-wsp .wgrid i.b{background:var(–gr)}
#mtp-wsp .arrow{text-align:middle;font-size:22px;colour:var(–g)}
#mtp-wsp .desc{margin-top:12px}
#mtp-wsp .tags{show:flex;hole:6px;flex-wrap:wrap;margin-top:8px}
#mtp-wsp .tg{font-size:11.5px;font-weight:600;padding:3px 8px;border-radius:999px;background:var(–gl);colour:var(–gd)}
#mtp-wsp .tg.n{background:#ECE7E5;colour:var(–ch)}
/* loop */
#mtp-wsp .steps{show:grid;grid-template-columns:repeat(6,1fr);hole:6px;margin:6px 0 14px}
#mtp-wsp .st{border:2px stable var(–line);border-radius:9px;padding:9px 6px;text-align:middle;font-size:11.5px;font-weight:600;colour:var(–mute);background:#fff;transition:all .3s;cursor:pointer}
#mtp-wsp .st b{show:block;font-size:17px;colour:var(–gr);transition:colour .3s}
#mtp-wsp .st.on{border-color:var(–g);colour:var(–gd);rework:translateY(-3px);box-shadow:0 6px 14px rgba(30,74,44,.12)}
#mtp-wsp .st.on b,#mtp-wsp .st.carried out b{colour:var(–g)}
#mtp-wsp .time period{background:#18201B!essential;colour:#CFE6D4!essential;border-radius:9px;padding:12px 14px;font:12.5px/1.6 ui-monospace,Menlo,monospace;min-height:96px;white-space:pre-wrap}
#mtp-wsp .time period .ok{colour:#8FD19E}
#mtp-wsp .price range{margin-top:12px;show:flex;align-items:middle;hole:10px;font-size:12.5px;colour:var(–mute)}
#mtp-wsp .bud{flex:1;top:10px;background:#DADDD9;border-radius:6px;overflow:hidden}
#mtp-wsp .bud span{show:block;top:100%;background:var(–g);transition:width .6s ease}
#mtp-wsp .ctl{show:flex;hole:8px;margin-top:12px;flex-wrap:wrap}
/* edits */
#mtp-wsp .mat{show:grid;grid-template-columns:repeat(40,1fr);hole:2px;margin-top:10px}
#mtp-wsp .mat i{show:block;aspect-ratio:1;background:#DADDD9;border-radius:1px;transition:background .25s}
#mtp-wsp .mat i.a{background:var(–g)}
#mtp-wsp .mat i.r{background:linear-gradient(135deg,var(–g),var(–gd))}
#mtp-wsp .kv{show:grid;grid-template-columns:repeat(4,1fr);hole:8px;margin-top:12px}
#mtp-wsp .kv div{background:var(–card);border:1px stable var(–line);border-radius:8px;padding:9px}
#mtp-wsp .kv b{show:block;font-size:17px;colour:var(–gd)}
#mtp-wsp .kv span{font-size:11px;colour:var(–mute);text-transform:uppercase;letter-spacing:.06em}
/* bars */
#mtp-wsp .bars{margin-top:12px}
#mtp-wsp .br{show:grid;grid-template-columns:170px 1fr 92px;align-items:middle;hole:10px;margin:9px 0;font-size:13px}
#mtp-wsp .trk{top:20px;background:#E3E5E2;border-radius:4px;place:relative;overflow:hidden}
#mtp-wsp .fill{place:absolute;left:0;high:0;backside:0;width:0;border-radius:4px;transition:width .9s cubic-bezier(.2,.8,.2,1)}
#mtp-wsp .v{font-weight:700;text-align:proper}
#mtp-wsp .v small{font-weight:600;colour:var(–mute);margin-left:4px}
#mtp-wsp .be aware{font-size:11.5px;colour:var(–mute)!essential;margin-top:10px}
#mtp-wsp .ft{show:flex;justify-content:space-between;hole:10px;flex-wrap:wrap;padding:11px 24px;border-top:1px stable var(–line);font-size:11.5px;colour:var(–mute)}
#mtp-wsp .ft a{colour:#76B900!essential;font-weight:700;text-decoration:none}
@media (max-width:640px){
#mtp-wsp .hd{padding:16px 16px 8px}#mtp-wsp .tabs{padding:8px 12px 0}#mtp-wsp .pn{padding:16px}
#mtp-wsp h3{font-size:18px}
#mtp-wsp .lev{grid-template-columns:1fr}#mtp-wsp .arrow{rework:rotate(90deg)}
#mtp-wsp .steps{grid-template-columns:repeat(3,1fr)}
#mtp-wsp .kv{grid-template-columns:repeat(2,1fr)}
#mtp-wsp .br{grid-template-columns:1fr 84px}#mtp-wsp .br .trk{grid-column:1/-1;order:3}
#mtp-wsp .mat{grid-template-columns:repeat(24,1fr)}
}
</fashion></head><physique>
<div id="mtp-wsp">
<div class="hd">
<div class="kick">Interactive explainer · Cmpnd dspy-wsp</div>
<h3>Weight-space programming: an LLM that edits weights like code</h3>
<p class="sub">No gradients. No coaching loop. A author LLM reads, patches and assessments a small mannequin's weights.</p>
</div>
<div class="tabs" position="tablist">
<button class="tab on" data-t="0">1 · The thought</button>
<button class="tab" data-t="1">2 · The loop</button>
<button class="tab" data-t="2">3 · What modified</button>
<button class="tab" data-t="3">4 · Outcomes</button>
<button class="tab" data-t="4">5 · Price & deploy</button>
</div>
<!– 1 concept –>
<div class="pn on" data-p="0">
<p class="lbl">Choose an optimization lever</p>
<div class="row" id="levb">
<button class="btn" data-l="0">Immediate optimization</button>
<button class="btn" data-l="1">Positive-tuning</button>
<button class="btn on" data-l="2">Weight-space programming</button>
</div>
<div class="lev">
<div class="field" id="bx-p"><h4>Immediate / directions</h4>
<div class="ptxt">Triage a assist ticket.<br><span id="pchg">Resolve if pressing and route it.</span><br>ticket → pressing, group</div></div>
<div class="arrow">⇄</div>
<div class="field" id="bx-w"><h4>Mannequin weights</h4><div class="wgrid" id="wg"></div></div>
</div>
<div class="card desc" id="ldesc"></div>
</div>
<!– 2 loop –>
<div class="pn" data-p="1">
<p class="lbl">Step by one WeightSpace run</p>
<div class="steps" id="steps">
<div class="st" data-s="0"><b>01</b>Load pupil</div>
<div class="st" data-s="1"><b>02</b>Examine</div>
<div class="st" data-s="2"><b>03</b>Hypothesize</div>
<div class="st" data-s="3"><b>04</b>Edit</div>
<div class="st" data-s="4"><b>05</b>Ahead move</div>
<div class="st" data-s="5"><b>06</b>Patch</div>
</div>
<div class="time period" id="time period"></div>
<div class="price range">Ahead-pass price range <div class="bud"><span id="budf" fashion="width:100%"></span></div><b id="budn">25 / 25</b></div>
<div class="ctl"><button class="btn" id="prev">← Again</button><button class="btn on" id="subsequent">Subsequent step →</button><button class="btn" id="play"> Auto-play</button></div>
</div>
<!– 3 edits –>
<div class="pn" data-p="2">
<p class="lbl">Tensor the author edited (GPT-6-Astra, 25 passes)</p>
<div class="row" id="edb">
<button class="btn on" data-e="0">d1-omni-600M</button>
<button class="btn" data-e="1">NanoJev</button>
<button class="btn" data-e="2">d1-2B</button>
</div>
<div class="card" fashion="margin-top:12px">
<div fashion="font-size:13px"><code id="edt"></code></div>
<p id="edd" fashion="margin-bottom:0"></p>
<div class="mat" id="mat"></div>
</div>
<div class="kv">
<div><b id="k1"></b><span>Entries modified</span></div>
<div><b id="k2"></b><span>Author turns</span></div>
<div><b id="k3"></b><span>Runtime</span></div>
<div><b id="k4"></b><span>Author price</span></div>
</div>
<p class="be aware">Grid is illustrative of share edited, not precise positions. Supply: dspy-wsp outcomes README (2026-11-07).</p>
</div>
<!– 4 outcomes –>
<div class="pn" data-p="3">
<p class="lbl">Typed Selections take a look at accuracy (400 held-out instances)</p>
<div class="row" id="rsb">
<button class="btn on" data-r="0">d1-omni-600M</button>
<button class="btn" data-r="1">NanoJev 0.6B</button>
<button class="btn" data-r="2">d1-2B</button>
<button class="btn" data-r="3">Clef-Flash 9B</button>
</div>
<div class="bars" id="bars"></div>
<p class="be aware">Author GPT-6-Astra, 25 ahead passes. Features are paired, ±0.7 to 1.4 pts SE. Self-reported by Cmpnd.</p>
</div>
<!– 5 price –>
<div class="pn" data-p="4">
<div class="kv" fashion="margin-top:0">
<div><b>$5.42 to $7.63</b><span>Author price / run</span></div>
<div><b>78 to 167 min</b><span>Runtime / run</span></div>
<div><b>~6 GB</b><span>Reminiscence, 2B mannequin</span></div>
<div><b>MIT</b><span>License</span></div>
</div>
<div class="card" fashion="margin-top:12px">
<p class="lbl">Ship the patch</p>
<p><code>pip set up dspy-wsp</code> (DSPy 3.5, Python 3.10+)</p>
<p>Patch = <code>manifest.json</code> + <code>edit.npz</code> with sha256 hashes of base and edited tensors, so anybody with the general public base mannequin rebuilds the precise weights.</p>
<p>Resolution fashions: <code>wsp_patch.write_checkpoint(…)</code> · LLMs: <code>dspy-wsp apply</code>, then serve on vLLM.</p>
<p fashion="margin-bottom:0">Runs regionally: MLX on Apple silicon, PyTorch on Linux (NVIDIA GPU if current).</p>
</div>
<p class="be aware">Caveat: Cmpnd calls the package deal "very experimental." Beneath the paper’s stricter rule, d1-omni-600M gained solely +2.8.</p>
</div>
<div class="ft"><span>Sources: cmpnd.ai, github.com/cmpnd-ai/dspy-wsp, rikiyat.github.io</span><a href="https://www.marktechpost.com" goal="_blank" rel="noopener">Constructed by Marktechpost</a></div>
</div>
<script>
(operate(){
var R=doc.getElementById(‘mtp-wsp’);
operate q(s){return R.querySelector(s)} operate qa(s){return R.querySelectorAll(s)}
operate publish(){strive{mother or father.postMessage({mtpWspH:R.offsetHeight+40},’*’)}catch(e){}}
qa(‘.tab’).forEach(operate
/* levers */
var wg=q(‘#wg’);for(var i=0;i<64;i++){var c=doc.createElement(‘i’);wg.appendChild(c)}
var cells=wg.youngsters;
var L=[
{p:1,w:0,t:'<b>Prompt optimization</b> (MIPROv2, GEPA) rewrites instructions and examples. Weights never move.’,g:[‘Edits text’,’No GPU needed’],n:[‘Weights frozen’]},
{p:0,w:1,t:'<b>Positive-tuning</b> (BootstrapFinetune) nudges many weights with gradient descent on coaching traces.’,g:[‘Changes weights’],n:[‘Needs gradients’,’Needs a training loop’]},
{p:0,w:2,t:'<b>Weight-space programming</b>: a frontier LLM reads tensors, varieties a speculation about what they compute, writes code to patch a couple of, then assessments the edit with a ahead move.’,g:[‘Changes weights’,’No gradients’,’Targeted, hashed patch’],n:[‘Needs a strong writer LLM’]}
];
operate lever(ok){qa(‘#levb .btn’).forEach(operate(b){b.classList.toggle(‘on’,+b.dataset.l===ok)});var d=L[k];
q(‘#bx-p’).classList.toggle(‘scorching’,d.p===1);q(‘#bx-w’).classList.toggle(‘scorching’,d.w>0);q(‘#pchg’).className=d.p?’chg’:”;
for(var i=0;i<cells.size;i++){cells[i].className=””}
if(d.w===1){var n=0;var iv=setInterval(operate(){for(var j=0;j<cells.size;j++){cells[j].className=Math.random()<.7?’b’:”}if(++n>5)clearInterval(iv)},140)}
if(d.w===2){[19,20,27,28].forEach(operate(x,ix){setTimeout(operate(){cells[x].className=”a”},200+ix*180)})}
q(‘#ldesc’).innerHTML='<p fashion="margin-top:0">’+d.t+'</p><div class="tags">’+d.g.map(operate(x){return ‘<span class="tg">’+x+'</span>’}).be part of(”)+d.n.map(operate(x){return ‘<span class="tg n">’+x+'</span>’}).be part of(”)+'</div>’;setTimeout(publish,60)}
qa(‘#levb .btn’).forEach(operate(b){b.onclick=operate(){lever(+b.dataset.l)}});lever(2);
/* loop */
var S=[
{b:25,t:'<span class="k">load_decision_backend("d1")</span>n# student weights load into a local Python processn# writer: gpt-6-astra, reasoning_effort="max"’},
{b:25,t:'<span class="k">“`python</span> # runs in sandbox (sandbox-exec / bubblewrap)nW = weights["head.scorer.1.weight"]nprint(W.form, W.std()) # learn tensors like supply code’},
{b:25,t:’# author flip = brief rationalization + 1 code blockn"These scorer items push solutions towards the unsuitable possibility.n I’ll rescale them and test on observe examples."’},
{b:25,t:'<span class="ok">“`python</span>nW[rows] *= scale # focused edit, no gradientsn# change is recorded in opposition to the bottom tensor hash’},
{b:24,t:'<span class="ok">“`ahead</span> # run the DSPy program on observe examplesn# rating on holdout (author sees scores, by no means examples)n# preserve the edit provided that the metric improves’},
{b:0,t:’# price range spent: finest edit turns into the patchnpatch/manifest.json + patch/edit.npz (sha256 hashed)n# hidden valset compares unique vs edited weights as soon as’}
];
var cur=0,pl=null;
operate step(ok){cur=Math.max(0,Math.min(5,ok));qa(‘.st’).forEach(operate(s){var i=+s.dataset.s;s.classList.toggle(‘on’,i===cur);s.classList.toggle(‘carried out’,i<cur)});
q(‘#time period’).innerHTML=S[cur].t;q(‘#budf’).fashion.width=(S[cur].b/25*100)+’%’;q(‘#budn’).textContent=S[cur].b+’ / 25′;
if(cur===4){var b=24;var iv=setInterval(operate(){b-=1;if(cur!==4||b<1){clearInterval(iv);return}q(‘#budf’).fashion.width=(b/25*100)+’%’;q(‘#budn’).textContent=b+’ / 25′},90)}}
qa(‘.st’).forEach(operate(s){s.onclick=operate(){step(+s.dataset.s)}});
q(‘#subsequent’).onclick=operate(){step(cur+1)};q(‘#prev’).onclick=operate(){step(cur-1)};
q(‘#play’).onclick=operate(){if(pl){clearInterval(pl);pl=null;q(‘#play’).textContent=”
Auto-play”;return}step(0);q(‘#play’).textContent=”❚❚ Pause”;pl=setInterval(operate(){if(cur>=5){clearInterval(pl);pl=null;q(‘#play’).textContent=”
Auto-play”;return}step(cur+1)},2200)};
step(0);
/* edits */
var E=[
{t:’head.scorer.1.weight (+ bias, scorer.3.weight)’,d:’Rewrote part of the decision head scorer: 112,640 of 1,048,576 entries (about 10.7%), plus 220 bias/output entries.’,k:[‘112,860′,’74’,’78 min’,’$7.63′],m:’a’,f:0.107},
{t:’scalar.weight’,d:’A single small tensor of 1,024 entries on the Qwen3-0.6B spine with a choice head.’,ok:[‘1,024′,’68’,’167 min’,’$6.92′],m:’a’,f:1},
{t:’layers.23.feed_forward.w2.weight’,d:’d1-2B has no determination head, so the author edited the spine: all 15.7M entries modified, as a rank-9 replace.’,ok:[‘15.7M’,’65’,’83 min’,’$5.42′],m:’r’,f:1}
];
var mat=q(‘#mat’);for(var i=0;i<200;i++){mat.appendChild(doc.createElement(‘i’))}
operate ed(ok){qa(‘#edb .btn’).forEach(operate(b){b.classList.toggle(‘on’,+b.dataset.e===ok)});var e=E[k];q(‘#edt’).textContent=e.t;q(‘#edd’).textContent=e.d;
[‘k1′,’k2′,’k3′,’k4’].forEach(operate(id,ix){q(‘#’+id).textContent=e.ok[ix]});
var ms=mat.youngsters,n=ms.size;for(var i=0;i<n;i++)ms[i].className=””;
var on=ok===1?20:Math.spherical(n*e.f);var idx=[];for(var i=0;i<n;i++)idx.push(i);
if(ok===0){idx=idx.slice(0,on)}else if(ok===1){idx=idx.slice(0,20)}
idx.forEach(operate(x,j){setTimeout(operate(){ms[x].className=e.m},j*(ok===2?3:12))});setTimeout(publish,60)}
qa(‘#edb .btn’).forEach(operate(b){b.onclick=operate(){ed(+b.dataset.e)}});ed(0);
/* bars */
var D=[[50.9,60.2,66.9,69.1],[39.2,50.7,52.4,56.4],[60.6,70.1,74.0,77.9],[68.8,70.3,70.7,75.3]];
var N=[‘Unoptimized’,’ReAnchor’,’WeightSpace’,’WeightSpace + ReAnchor’],C=[‘#B0AAA6′,’#5B5452′,’#3A8A50′,’#1E4A2C’];
var bars=q(‘#bars’);N.forEach(operate(n,i){bars.insertAdjacentHTML(‘beforeend’,'<div class="br"><span>’+n+'</span><div class="trk"><div class="fill" fashion="background:’+C[i]+’"></div></div><span class="v"></span></div>’)});
var G=[[0,9.3,16.1,18.3],[0,11.6,13.3,17.3],[0,9.5,13.4,17.3],[0,1.5,1.9,6.5]];
var rc=0;
operate drawBars(ok){rc=ok;qa(‘#rsb .btn’).forEach(operate(b){b.classList.toggle(‘on’,+b.dataset.r===ok)});var r=qa(‘#bars .br’);
r.forEach(operate(row,i){var v=D[k][i];var f=row.querySelector(‘.fill’);f.fashion.width=”0″;var g=i?’ <small>+’+G[k][i].toFixed(1)+'</small>’:”;row.querySelector(‘.v’).innerHTML=v.toFixed(1)+’%’+g;setTimeout(operate(){f.fashion.width=(v/85*100)+’%’},60+i*120)})}
qa(‘#rsb .btn’).forEach(operate(b){b.onclick=operate(){drawBars(+b.dataset.r)}});drawBars(0);
window.addEventListener(‘load’,publish);window.addEventListener(‘resize’,publish);setTimeout(publish,300);
})();
</script>
</physique></html>
“>
(operate(){var f=doc.getElementById(“mtp-wsp-frame”);window.addEventListener(“message”,operate(e){if(e.information&&e.information.mtpWspH&&e.supply===f.contentWindow){f.fashion.top=e.information.mtpWspH+”px”;}});})();
How effectively does it work? Typed Selections benchmark outcomes
Cmpnd examined on Typed Decisions, a benchmark for typed probabilistic choices. A mannequin reads 1 piece of unstructured state and solutions 5 typed questions directly. Every reply is a chance distribution. The take a look at cut up has 400 instances and a couple of,000 choices.
Every mannequin received 25 ahead passes from GPT-6-Astra. Features are paired, with a regular error of ±0.7 to 1.4 factors.
| Mannequin | Unoptimized | ReAnchor | WeightSpace | WeightSpace + ReAnchor |
|---|---|---|---|---|
| d1-omni-600M | 50.9% | 60.2% (+9.3) | 66.9% (+16.1) | 69.1% (+18.3) |
| NanoJev (Qwen3-0.6B) | 39.2% | 50.7% (+11.6) | 52.4% (+13.3) | 56.4% (+17.3) |
| d1-2B | 60.6% | 70.1% (+9.5) | 74.0% (+13.4) | 77.9% (+17.3) |
| Clef-Flash (9B) | 68.8% | 70.3% (+1.5) | 70.7% (+1.9) | 75.3% (+6.5) |
Sources: Cmpnd blog chart for d1-omni-600M, d1-2B and Clef-Flash; results README for NanoJev.
ReAnchor is an earlier Cmpnd optimizer. Resolution fashions output possibilities, and thresholds flip these into solutions. ReAnchor refits every query’s thresholds with no LLM calls. WeightSpace adjustments the mannequin; ReAnchor recalibrates its output. The two stack effectively.
Three patterns stand out:
- WeightSpace beat ReAnchor alone on each mannequin proven. On d1-omni-600M, it added +16.1 versus +9.3.
- The most important mannequin gained least. The 9B Clef-Flash gained solely +1.9 from WeightSpace alone.
- The mixture issues most at 9B. Clef-Flash jumped to +6.5 when each optimizers ran.
Does the author mannequin matter?
The reply is sure. With gpt-6.1-sol as the author, WeightSpace alone gave +5.8 on d1-omni-600M, +12.1 on NanoJev and −0.6 on d1-2B. Astra produced +16.1, +13.3 and +13.4 on the identical fashions.
Did dspy-wsp bend the unique paper’s guidelines?
Barely, and Cmpnd says so. By default, the author can recompute the choice head inside its kernel. Setting simulate_head=False enforces the paper’s stricter rule. Beneath that rule, d1-omni-600M gained solely +2.8 ± 0.7 factors. Including ReAnchor introduced it to +9.3, no higher than ReAnchor alone.
How do you deploy an edited mannequin?
Every run writes a patch to run_dir. The patch holds manifest.json and edit.npz, with solely the modified entries. It additionally shops sha256 hashes of each base and edited tensor. Anybody with the general public base mannequin can rebuild the edited weights precisely.
- Resolution fashions:
compiled.wsp_patch.write_checkpoint(backend.weights_dir, "edited-d1/") - Textual content LLMs:
dspy-wsp apply <base> <patch> <out>/, then serve with vLLM. - Packages:
compiled.save("program.json")and reload withallow_unsafe_lm_state=True.
The edited mannequin runs regionally. On Apple silicon it makes use of MLX. On Linux it makes use of PyTorch, on an NVIDIA GPU if current.
What fashions does dspy-wsp assist?
| Backend | Mannequin | Alias | Structure |
|---|---|---|---|
D1Backend |
Liquid AI d1-omni (textual content path) | "d1" |
Encoder with determination head |
NanoJevBackend |
NanoJev | "nanojev" |
Qwen3 spine with determination head |
D1LMBackend |
Liquid AI d1 on LFM3 | "d1-2b" |
LM answering at query positions |
Textual content-generating LLMs work by VLLMBackend (NVIDIA), HFBackend (CUDA, MPS or CPU) and MLXBackend (Apple silicon). A GEPASearch possibility, contributed by Michael Isaac, provides inhabitants seek for determination fashions.
How does WeightSpace evaluate with different DSPy optimizers?
| Metrics | WeightSpace (dspy-wsp) | ReAnchor | BootstrapFinetune | GEPA |
|---|---|---|---|---|
| What adjustments | Mannequin weights | Resolution thresholds, cuts, weights | Mannequin weights | Immediate textual content |
| How | Frontier LLM causes and edits in a sandbox | Grid search over cached possibilities | Gradient fine-tuning on bootstrapped traces | Reflective immediate evolution |
| Gradients | No | No | Sure | No |
| LLM calls throughout optimization | Sure (author, 25 passes) | None | Trainer for bootstrapping | Sure |
| Typical price | $5.42 to $7.63 per run | Close to zero | Not disclosed | Not disclosed |
| Output | Small hashed patch | Calibrated program | Positive-tuned mannequin | Optimized prompts |
| Supply | dspy-wsp | Cmpnd | BetterTogether paper | GEPA paper |
Key Takeaways
- Weight-space programming lets a frontier LLM edit weights immediately, with out gradients or coaching information.
- dspy-wsp wraps it as a DSPy optimizer:
pip set up dspy-wsp, MIT license. - 25 author passes price $5.42 to $7.63 and took 78 to 167 minutes.
- d1-omni-600M rose from 50.9% to 69.1% with WeightSpace + ReAnchor.
- Features fall at 9B, and the package deal is explicitly experimental.
Take a look at the Cmpnd blog post, the GitHub repo, the full results and the unique Weight-Space Programming research. You’ll be able to test the deployment choices and specs right here by our GPULeaderboard.com chrome extension.
Additionally, be happy to comply with us on Twitter and don’t overlook to affix our 150k+ML SubReddit and Subscribe to our Publication. Wait! are you on telegram? now you can join us on telegram as well.
Have to associate with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and so on.? Connect with us
The publish Cmpnd Releases dspy-wsp: Weight-House Programming Involves DSPy appeared first on MarkTechPost.

