Kuaishou’s KwaiKAT workforce is KAT-Coder-V2.5. This can be a coding mannequin that’s educated to work inside an actual, executable repository, fairly than outputting single-turn code. The fashions provided can be found beneath stream lake. Open weight variations KAT-Coder-V2.5-Developmentlaunched individually with Hugging Face below Apache-2.0.
AutoBuilder: An atmosphere that really executes the supposed checks.
On this research, we body verifiable duties as a triad. It requires an correct description of the duty, an executable repository atmosphere, and a set of validation checks. A patch is barely legitimate if it passes all of them.
Duties are mined from the next precise pull requests and commits. SWE bench system. Merged code modifications present golden patches, and accompanying check modifications present check patches. The uncooked drawback textual content is discarded as a specification. As a substitute, the outline is regenerated into three elements. An issue assertion primarily based on the golden patch, necessities derived from the check patch, and interface constraints inferred from each. A readability verify then removes something that’s ambiguous, incomplete, poorly specified, or internally inconsistent.
AutoBuilder handles the atmosphere aspect. The construct agent analyzes the repository, installs dependencies, and creates configuration scripts that run checks from a clear checkout. The validation agent runs the script in an remoted sandbox.
Acceptance guidelines are the fascinating half. Validation doesn’t learn exit codes or grep log patterns. Parse the output of a structured testing framework and solely settle for an atmosphere if at the very least 90% of the anticipated checks are collected and go/fail outcomes are reproduced throughout runs. Failures are fed again as structured data for iterative restore.
Combining a preconfigured base atmosphere, construct system templates, and a retrievable library of extracted construct recipes elevated the construct success charge from 16.5% to 57.2%. The result’s over 100,000 verifiable environments throughout 12 languages. Git historical past, commit metadata, and different exploitable traces are eliminated, so the agent can not learn referenced options from the repository.
Knowledge Scaling Flywheel: Course of Filtering
Filtering trajectories by closing check success is deceptive. Some path execution depends on hard-coding, bypassing mechanisms, or test-oriented shortcuts. Some failed runs comprise worthwhile search, localization, and restore habits.
KwaiKAT works in each instructions. Within the case of a close to miss, focused process-level hints point out what to examine or confirm with out revealing an answer. This alone will enhance the go charge for assignments from zero to round 20%. Since hinted trajectories comprise data that isn’t accessible at inference time, the verified patches are modified and unhinted trajectories are regenerated from the unique process context. Solely samples that go validation, haven’t any lacking hints, and are in keeping with the patch are retained.
For passruns, rule-based gates take away invalid, unstable, or abusive trajectories. A scoring stage then evaluates exploration, localization, pre-edit reasoning, specification constancy, repository conventions, patch minimality, validation high quality, restoration habits, and honesty.
The third mechanism targets harness overfitting. Software names, argument guidelines, output codecs, and immediate templates are randomized whereas preserving performance. Verification is anchored to check outcomes fairly than harness traces, so one process will be reserved for a lot of harness configurations. Reasonable perturbations are additionally injected, equivalent to lacking dependencies, short-term command failures, truncated output, and noisy logs.
Look at the pipeline and numbers
Infrastructure failures restrict rewards earlier than algorithm limits
in the meantime KAT-Coder-V2 In coaching, the sluggish reward curve was initially attributed to the RL algorithm. Nonetheless, an audit revealed that as much as 16% of trajectories may fail on account of sandbox infrastructure points fairly than mannequin insurance policies, and boundary inconsistencies may lead to as much as 40 empty remark steps, compromising rewards.
Three infrastructure fixes adopted. First, the early launch picture deletion coverage lowered disk utilization from 95% to 60% and invalid rollouts on account of timeouts from 6-7% to lower than 1%. Second, modifying the atmosphere variables throughout distant sandbox initialization stopped the system override that reversed rewards in 6-7% of samples and lowered the error to lower than 1%. Third, the gateway server bypassed the mainstream chat endpoint (reapplying and retokening apply_chat_template resulted in 40% token drift on a scale of roughly 200 turns) and immediately known as /generate to make sure alignment of rollout tokens.
These updates lowered the sandbox suggestions error charge from roughly 16% to lower than 2% and lowered coaching collapse by an order of magnitude.
Uneven PPO and three tiers of compensation
The analysis workforce chosen PPO You will need to use GAE over the no-critic trajectory technique as a result of the manufacturing harness splits the session into structurally distinct samples, complicating the group baseline.
use Asymmetrical actor and critic,Critics get privileged coaching context (rewards, checks, protection, patches, metadata, future turns), whereas actors solely see the rollout state. Critic and additional context are discarded at inference time.
There are three ranges of rewards. The core process rating requires passing all fail_to_pass and pass_to_pass checks. Customary habits constraints penalize duplicates, unlawful instrument calls, and debugging particles. Failed Trajectory Incentives will retrieve the file by way of F2 and provides partial check credit score.
5 specialists are fused by way of multi-supervised on-policy distillation utilizing inverse KL, off-policy beginning, and drift-aware truncation. Prune-OPD.
outcome
Beneath the unified Claude code harness, KAT-Coder-V2.5 leads the panel. pinch bench 94.9, which exceeded Opus 4.8’s 93.5. It ranks 2nd in SWE-Bench Professional (65.2 vs 69.2) and Inside KAT Code Bench (53.1 vs 57.3).
Nonetheless, it falls wanting Terminal-Bench 2.1, rating final at 60.7 behind GLM-5.1 (61.8) and Opus 4.8 (84.6). In SciCode, the rating is 50.3, which matches GLM-5.2.
What you need to take note of is open weight. KAT-Coder-V2.5-Development is a separate 35B-total/3B-active MoE, then RL, post-trained on Qwen3.6-35B-A3B utilizing 127K SFT samples. Outcomes evaluated with a unique in-house protocol can’t be in contrast with the primary main desk.
Vital factors
- KwaiKAT treats agent coding as an infrastructure drawback fairly than a mannequin scale drawback.
- AutoBuilder elevated atmosphere constructing success charge from 16.5% to 57.2% and resulted in over 100,000 verifiable environments throughout 12 languages.
- Sandbox audits present that as much as 16% of RL trajectories fail on account of sandboxing fairly than coverage. The repair lowered that to lower than 2%.
- KAT-Coder-V2.5 tops PinchBench with 94.9 and ranks second behind Opus 4.8 with 65.2 in SWE-Bench Professional.
- Openweight KAT-Coder-V2.5-Dev is an unbiased 35B-A3B MoE below Apache-2.0 with its personal benchmark quantity.
Please verify paper, Model’s weight on hugging face,and Product page. All credit score for this research goes to the researchers of this mission.
Michal Sutter is a knowledge science knowledgeable with a grasp’s diploma in knowledge science from the College of Padova. With a powerful basis in statistical evaluation, machine studying, and knowledge engineering, Michal excels at remodeling complicated datasets into actionable insights.
