Edge0 Releases Audio8-TTS-Life like, a 0.6B Voice Cloning TTS Mannequin

Edge0 has launched Audio8-TTS-Realistic, an open-weight text-to-speech mannequin for zero-shot voice cloning. Audio8-TTS-Life like is a 0.6B-parameter mannequin that clones a voice from 1 reference clip and transcript. It helps 23 languages, 12 greater than its predecessor. The weights and code ship beneath Apache 2.0. For builders, the catch is aggressive multilingual accuracy at a dimension that’s low-cost to fine-tune and serve.

TL;DR

Measurement: 601M main-model parameters (codec excluded); as much as 2,048 packed textual content/audio positions of context.
Runs on: a CUDA GPU is really helpful; smoke-tested on NVIDIA H20 in BF16. The paper stories 1.1 to 1.2 GiB peak reminiscence on an Apple M2 CPU (INT4 ONNX). No Life like ONNX export is public but.
Efficiency: forward of bigger open fashions on CV3, behind 4B-class fashions on MiniMax.
Greatest: 4.86% common CV3 error throughout 9 languages, versus 7.73% for Qwen3-TTS-1.7B.
Worst: 48.57% WER on Thai (CV3) and 31.35% CER on Cantonese (MiniMax).

What’s Audio8-TTS-Life like?

It’s the second 0.6B launch in Edge0’s Audio8 TTS household. It builds on Audio8-TTS-Preview-0.6b and retains the identical Transformers inference interface. The repository bundles the checkpoint, a 44.1 kHz neural audio codec, tokenizer, processor and customized code. It could possibly additionally generate speech and not using a reference voice.

New languages are Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish and Vietnamese.

How does the structure work?

The mannequin makes use of a dual-autoregressive (DualAR) design impressed by Fish Audio S2 Pro. In accordance with the technical research report, the gradual department is constructed on Qwen2.5 0.5B. It has 24 layers at width 896 and runs at about 21.5 frames per second. Every step predicts textual content tokens plus the primary audio codebook.

A shallow 4-layer quick department then predicts the remaining 9 codebooks in parallel. Sequential compute subsequently occurs solely on the body degree.

How does Edge0 prepare for realism?

The group argues that phrase error charge (WER) and speaker similarity (SIM) miss rhythm, pauses and timing. A mannequin can rating nicely on each and nonetheless sound artificial.

Coaching runs in 3 phases:

Multilingual supervised fine-tuning (SFT).
GRPO with a mixed reward: 0.7 weight on WER, 0.3 on SIM.
GRPO guided by a discovered realism critic.

The critic scores every output in opposition to a matched human recording of the identical sentence. It was skilled on a top quality ladder constructed from 6 open-source TTS programs. These have been ranked in a personal blind listening research, with human recordings on the high. Artificial pause compression provides finer rungs to the ladder.

An identity-erasure loss stops the critic from rewarding generator fingerprints. The critic head has about 3M parameters on high of a frozen WavLM-large encoder. A length penalty additionally kicks in past 1.35x the reference size. This blocks the coverage from gaming the critic with slower speech.

The ultimate stage used solely Chinese language and English prompts. It nonetheless improved each combination metrics on the 13 lower-resource languages. Their error fell from 22.53% to twenty.77%, and SIM rose from 0.637 to 0.679.

<div class="pn on" data-p="arch">
<div class="arch">
<div class="blk on" data-d="0"><b>Textual content + reference</b><span>clip and actual transcript</span></div>
<div class="arr">→</div>
<div class="blk" data-d="1"><b>Gradual AR</b><span>24 layers · width 896</span></div>
<div class="arr">→</div>
<div class="blk" data-d="2"><b>Quick AR</b><span>4 layers · 9 codebooks</span></div>
<div class="arr">→</div>
<div class="blk" data-d="3"><b>Codec decoder</b><span>44.1 kHz waveform</span></div>
</div>
<div class="element" id="a8-detail"></div>
<div class="frames" id="a8-frames"></div>
<div class="legend"><span><i fashion="background:#e7e7e5"></i>codebook 1 (gradual AR)</span><span><i fashion="background:#8d8d93"></i>codebooks 2 to 10 (quick AR, parallel)</span><span>~21.5 frames per second</span></div>
</div>

<div class="pn" data-p="prepare">
<div class="steps">
<div class="stp" data-s="0"><small>STAGE 1</small><b>Multilingual SFT</b><span>Cross-entropy on paired reference and goal recordings.</span></div>
<div class="stp" data-s="1"><small>STAGE 2</small><b>GRPO: WER + SIM</b><span>Reward weights 0.7 on WER, 0.3 on speaker similarity.</span></div>
<div class="stp" data-s="2"><small>STAGE 3</small><b>GRPO: realism critic</b><span>Scores every output in opposition to a matched human recording.</span></div>
</div>
<div class="ctrl">
<button class="btn on" data-g="0">After SFT</button>
<button class="btn" data-g="1">After WER/SIM</button>
<button class="btn" data-g="2">After realism</button>
<button class="btn" id="a8-play"> Step by means of</button>
</div>
<div id="a8-stage"></div>
<p class="word">The realism stage used solely Chinese language and English prompts, but improved each metrics on the 13 further languages. Supply: technical report, Desk 1.</p>
</div>

<div class="pn" data-p="bench">
<div class="ctrl">
<button class="btn on" data-b="cv3">CV3 (9 langs)</button>
<button class="btn" data-b="mm">MiniMax (24 langs)</button>
<button class="btn" data-b="seed">Seed-TTS EN/ZH</button>
</div>
<div id="a8-bench"></div>
<p class="word" id="a8-bnote"></p>
</div>

<div class="pn" data-p="lang">
<div class="ctrl">
<button class="btn on" data-l="az">A to Z</button>
<button class="btn" data-l="finest">Greatest first</button>
<button class="btn" data-l="worst">Worst first</button>
</div>
<div class="lg" id="a8-lang"></div>
<p class="word">CV3 error charge per language (CER for Chinese language, Japanese, Korean; WER in any other case). 500 objects every. Shorter bar is best.</p>
</div>

<div class="pn" data-p="run">
<p><b>Transformers</b> (PyTorch 2.5+, Transformers 4.57+, CUDA GPU really helpful):</p>
<pre><code>model_id = "Edge0/Audio8-TTS-Life like-0.6b"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
mannequin = AutoModel.from_pretrained(model_id, trust_remote_code=True)</code></pre>
<p><b>Serving:</b> SGLang Omni adapter with paged consideration, dynamic batching and an OpenAI-compatible <code>/v1/audio/speech</code> API. Throughput and streaming weren’t benchmarked by Edge0.</p>
<p><b>CPU:</b> the paper stories 1.1 to 1.2 GiB peak on an Apple M2 with INT4 ONNX. The repo states no Life like ONNX export is out there but.</p>
</div>

<div class="ft"><span>Numbers self-reported by Edge0; competitor figures from their very own sources or Boson AI’s analysis.</span><a href="https://www.marktechpost.com" goal="_blank" rel="noopener">© Marktechpost</a></div>
</div>
<script>
(operate(){
var R=doc.getElementById(‘mtp-a8tts’);
operate rs(){attempt{guardian.postMessage({mtpEmbedHeight:R.offsetHeight+40,id:’mtp-a8tts’},’*’)}catch(e){}}
operate $$(s){return Array.prototype.slice.name(R.querySelectorAll(s))}
$$(‘.tab’).forEach(operate
$$(‘.tab’).forEach(operate(x){x.setAttribute(‘aria-selected’,x===t?’true’:’false’)});
$$(‘.pn’).forEach(operate(p){p.classList.toggle(‘on’,p.getAttribute(‘data-p’)===t.getAttribute(‘data-t’))});
if(t.getAttribute(‘data-t’)===’bench’)bench(curB);
if(t.getAttribute(‘data-t’)===’prepare’)stage(curG);
if(t.getAttribute(‘data-t’)===’lang’)lang(curL);
setTimeout(rs,60);};});
/* structure */
var D=[‘A reference clip and its exact transcript condition the model. The model can also generate speech without a reference.’,
‘The slow branch is built on Qwen2.5 0.5B. Once per audio frame it predicts text tokens plus the first of 10 codebooks.’,
‘A 4-layer fast branch reads the slow hidden state and predicts the remaining 9 codebooks in parallel, adding no extra sequential steps.’,
‘The bundled codec turns 10 codebooks of 4,096 entries per frame back into a 44.1 kHz waveform. It is excluded from the 0.6B count.’];
var det=R.querySelector(‘#a8-detail’);det.textContent=D[0];
$$(‘.blk’).forEach(operate(b){b.onclick=b.onmouseenter=operate(){$$(‘.blk’).forEach(operate(x){x.classList.toggle(‘on’,x===b)});det.textContent=D[+b.getAttribute(‘data-d’)]};});
var F=R.querySelector(‘#a8-frames’),N=16,html=””;
for(var i=0;i<N;i++){html+='<div class="fr">’;for(var c=0;c<10;c++)html+='<div class="cb"></div>’;html+='</div>’}
F.innerHTML=html;var fr=$$(‘.fr’),okay=0;
setInterval(operate(){var col=fr[k%N].kids;
if(kpercentN===0)fr.forEach(operate(f){Array.prototype.forEach.name(f.kids,operate(x){x.className=”cb”})});
col[0].className=”cb s”;setTimeout(operate(){for(var c=1;c<10;c++)col[c].className=”cb f”},180);okay++;},420);
/* coaching */
var G=[[5.41,0.639,22.36,0.638],[5.01,0.643,22.53,0.637],[4.86,0.661,20.77,0.679]],curG=0;
var LB=[‘CV3 error % (9 langs)’,’CV3 SIM’,’Extra 13 langs error %’,’Extra 13 langs SIM’],MX=[25,1,25,1];
operate stage(g){curG=g;$$(‘.stp’).forEach(operate(s){s.classList.toggle(‘on’,+s.getAttribute(‘data-s’)<=g)});
$$(‘[data-g]’).forEach(operate(b){b.classList.toggle(‘on’,+b.getAttribute(‘data-g’)===g)});
var h=””;for(var i=0;i<4;i++){h+='<div class="mrow"><span class="lab">’+LB[i]+'</span><div class="trk"><div class="bar me" data-w="’+(G[g][i]/MX[i]*100)+’"></div></div><span class="val">’+(ipercent2?G[g][i].toFixed(3):G[g][i].toFixed(2)+’%’)+(g?'<br><small fashion="colour:#9a9a9f">’+((G[g][i]-G[0][i])>0?’+’:”)+(G[g][i]-G[0][i]).toFixed(ipercent2?3:2)+’ vs SFT</small>’:”)+'</span></div>’}
var el=R.querySelector(‘#a8-stage’);el.innerHTML=h;develop(el);setTimeout(rs,60);}
$$(‘[data-g]’).forEach(operate(b){b.onclick=operate(){stopP();stage(+b.getAttribute(‘data-g’))}});
var P=null;operate stopP(){if(P){clearInterval(P);P=null;R.querySelector(‘#a8-play’).textContent=” Step by means of”}}
R.querySelector(‘#a8-play’).onclick=operate(){if(P){stopP();return}var g=0;stage(0);this.textContent=”■ Cease”;P=setInterval(operate(){g++;if(g>2){stopP();return}stage(g)},1500)};
operate develop(el){requestAnimationFrame(operate(){requestAnimationFrame(operate(){Array.prototype.forEach.name(el.querySelectorAll(‘[data-w]’),operate(b){b.fashion.width=b.getAttribute(‘data-w’)+’%’})})})}
/* benchmarks */
var B={cv3:{max:9,rows:[[‘Higgs TTS 3 (~4B)’,4.41],[‘Audio8-Realistic (0.6B)’,4.86,1],[‘VoxCPM2 (2B)’,5.35],[‘CosyVoice3 (1.5B)’,6.89],[‘Qwen3-TTS (1.7B)’,7.73]],n:’CV3 macro-average error, excluding exhausting subsets. Higgs and Qwen3-TTS figures from Boson AI analysis; VoxCPM2 and CosyVoice3 from the VoxCPM repo.’},
mm:{max:30,rows:[[‘Higgs TTS 3 (~4B)’,2.74],[‘Audio8-Realistic (0.6B)’,5.89,1],[‘Qwen3-TTS (1.7B)’,27.41]],n:’MiniMax 24-language common, together with languages exterior every modelu0027s supported checklist. Qwen3-TTS helps 10 of the 24.’},
seed:{max:2,rows:[[‘Qwen3-TTS (1.7B)’,1.01],[‘Higgs TTS 3 (~4B)’,1.11],[‘Audio8-Realistic (0.6B)’,1.25,1],[‘VoxCPM2 (2B)’,1.41],[‘CosyVoice3 (1.5B)’,1.67]],n:’Common of Seed-TTS English WER and Chinese language CER, excluding exhausting Chinese language. Decrease is best.’}},curB=’cv3′;
operate bench(okay){curB=okay;$$(‘[data-b]’).forEach(operate(b){b.classList.toggle(‘on’,b.getAttribute(‘data-b’)===okay)});
var d=B[k],h=””;d.rows.forEach(operate(r){h+='<div class="mrow"><span class="lab"’+(r[2]?’ fashion="colour:#fff!vital;font-weight:700"’:”)+’>’+r[0]+'</span><div class="trk"><div class="bar’+(r[2]?’ me’:”)+’" data-w="’+(r[1]/d.max*100)+’"’+(r[2]?”:’ fashion="background:#55555b!vital"’)+’></div></div><span class="val">’+r[1].toFixed(2)+’%</span></div>’});
var el=R.querySelector(‘#a8-bench’);el.innerHTML=h;R.querySelector(‘#a8-bnote’).textContent=d.n;develop(el);setTimeout(rs,60);}
$$(‘[data-b]’).forEach(operate(b){b.onclick=operate(){bench(b.getAttribute(‘data-b’))}});
/* languages */
var L=[[‘Chinese’,1.81],[‘English’,2.91],[‘Japanese’,7.51],[‘Korean’,4.80],[‘German’,3.20],[‘French’,7.94],[‘Spanish’,3.39],[‘Italian’,4.93],[‘Russian’,7.26],[‘Arabic’,21.10],[‘Finnish’,32.28],[‘Hindi’,21.71],[‘Indonesian’,13.74],[‘Malay’,17.80],[‘Dutch’,10.87],[‘Norwegian’,17.96],[‘Polish’,19.23],[‘Portuguese’,6.87],[‘Swedish’,19.25],[‘Thai’,48.57],[‘Turkish’,12.57],[‘Vietnamese’,28.11]],curL=’az’;
operate col(v){return v<8?’#f5f5f4′:v<20?’#a1a1a6′:’#5c5c62′}
operate lang(m){curL=m;$$(‘[data-l]’).forEach(operate(b){b.classList.toggle(‘on’,b.getAttribute(‘data-l’)===m)});
var a=L.slice();if(m===’az’)a.kind(operate(x,y){return x[0]<y[0]?-1:1});if(m===’finest’)a.kind(operate(x,y){return x[1]-y[1]});if(m===’worst’)a.kind(operate(x,y){return y[1]-x[1]});
var h=””;a.forEach(operate(r){h+='<div class="lc"><div class="t"><span>’+r[0]+'</span><em>’+r[1].toFixed(2)+’%</em></div><div class="mini"><i data-w="’+(r[1]/50*100)+’" fashion="background:’+col(r[1])+’"></i></div></div>’});
var el=R.querySelector(‘#a8-lang’);el.innerHTML=h;develop(el);setTimeout(rs,60);}
$$(‘[data-l]’).forEach(operate(b){b.onclick=operate(){lang(b.getAttribute(‘data-l’))}});
window.addEventListener(‘load’,rs);window.addEventListener(‘resize’,rs);setTimeout(rs,300);rs();
})();
</script>
“>
window.addEventListener(“message”,operate(e){if(e.information&&e.information.id===”mtp-a8tts”&&e.information.mtpEmbedHeight){var f=doc.getElementById(“mtp-a8tts-frame”);if(f)f.fashion.peak=e.information.mtpEmbedHeight+”px”;}});

How does it carry out on benchmarks?

All numbers under are self-reported error charges, the place decrease is best.

Seed-TTS (English/Chinese language): 1.25% common error, SIM 0.704.
CV3 (9 languages): 4.86% common error, SIM 0.661.
MiniMax (24 languages): 5.89% common error, SIM 0.754.

On CV3, Audio8 beats VoxCPM2 (5.35%), CosyVoice3-1.5B (6.89%) and Qwen3-TTS-1.7B (7.73%). On MiniMax, Qwen3-TTS-1.7B scores 27.41%, however it formally helps solely 10 of the 24 take a look at languages.

Bigger fashions nonetheless lead elsewhere. Higgs TTS 3 stories 4.41% on CV3 and a couple of.74% on MiniMax. Qwen3-TTS-1.7B averages 1.01% on Seed-TTS. Edge0 itself calls its desk a reference comparability, since analysis pipelines and textual content normalization differ.

How does it evaluate with different open TTS fashions?

Function	Audio8-TTS-Realistic	Qwen3-TTS-12Hz-1.7B-Base	VoxCPM2	Higgs TTS 3
Measurement	0.6B (excl. codec)	1.7B	2B	~4B spine
License	Apache 2.0	Apache 2.0	Apache 2.0	Analysis and non-commercial
Languages	23	10	30	100+
Output pattern charge	44.1 kHz	Not disclosed	48 kHz	24 kHz
Seed-TTS EN WER	1.46%	1.24%	1.84%	Not disclosed
Seed-TTS ZH CER	1.03%	0.77%	0.97%	Not disclosed
CV3 avg error (9 langs)	4.86%	7.73% (Boson AI eval)	5.35%	4.41%
MiniMax avg error (24 langs)	5.89%	27.41% (Boson AI eval)	Not disclosed	2.74%
{Hardware} famous	CUDA GPU; INT4 M2 CPU in paper	GPU, FlashAttention 2	~8 GB VRAM	1x H100 serving exams

VoxCPM2’s CV3 common is computed from its per-language desk.

How do you deploy it?

Set up PyTorch 2.5+ and Transformers 4.57+, then load with trust_remote_code=True. Voice cloning wants a clear reference clip and its actual transcript. The GitHub repo provides a CLI, batching and supervised fine-tuning scripts.

For GPU serving, an SGLang Omni adapter gives paged consideration, dynamic batching and an OpenAI-compatible /v1/audio/speech API. Edge0 notes throughput and streaming weren’t benchmarked. A free Playground runs on Hugging Face ZeroGPU.

Key Takeaways

601M parameters, 23 languages, Apache 2.0 weights.
A discovered realism critic drives the ultimate GRPO stage.
4.86% CV3 error beats Qwen3-TTS-1.7B and VoxCPM2.
4B-class Higgs TTS 3 nonetheless leads on CV3 and MiniMax.
Decrease-resource languages stay weak, with Thai close to 49% WER.

Try the Paper, Model Weights, Repo and Samples. Additionally, be happy to observe us on Twitter and don’t neglect to hitch our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Have to accomplice with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and so on.? Connect with us

The publish Edge0 Releases Audio8-TTS-Life like, a 0.6B Voice Cloning TTS Mannequin appeared first on MarkTechPost.

Edge0 Releases Audio8-TTS-Life like, a 0.6B Voice Cloning TTS Mannequin

TL;DR

What’s Audio8-TTS-Life like?

How does the structure work?

How does Edge0 prepare for realism?

How does it carry out on benchmarks?

How does it evaluate with different open TTS fashions?

How do you deploy it?

Key Takeaways

The Symptom-Centered Mannequin of Standard Medication

How typically to run AEO immediate monitoring: A cadence information

Converter

Editors Pick

Newsletter

Categories

Related Posts

Leave a Comment Cancel Reply