Tencent Hunyuan’s 3D Digital Human staff has launched HY-Movement 1.0, an open-weight text-to-3D human movement technology household that scales diffusion transformer-based stream matching to 1B parameters within the movement area. This mannequin converts pure language prompts and anticipated durations into 3D human movement clips on an built-in SMPL-H skeleton, and is obtainable on GitHub and Hugging Face with code, checkpoints, and a Gradio interface for native use.
What HY-Movement 1.0 provides builders?
HY-Movement 1.0 is a set of text-to-3D human movement technology fashions constructed on a diffusion transformer DiT educated with a stream matching goal. This mannequin sequence introduces two variants: HY-Movement-1.0 with 1.0B parameters as the usual mannequin and HY-Movement-1.0-Lite with 0.46B parameters as a light-weight choice.
Each fashions generate skeleton-based 3D character animation from easy textual content prompts. The output is a movement sequence on an SMPL-H skeleton that may be built-in into 3D animation or recreation pipelines similar to digital people, cinematics, and interactive characters. This launch consists of inference scripts, batch-oriented CLI, Gradio internet apps, and helps macOS, Home windows, and Linux.
Knowledge engines and taxonomies
Coaching information comes from three sources: actual human movement video, movement seize information, and 3D animation property for recreation manufacturing. The analysis staff begins with a 12M high-quality video clip from HunyuanVideo, performs shot boundary detection to section the scene, performs an individual detector to make sure the clip comprises individuals, and applies the GVHMR algorithm to reconstruct the SMPL X movement monitor. Movement seize classes and 3D animation libraries contribute roughly 500 hours of further movement sequences.
All information is retargeted to the built-in SMPL-H skeleton by way of mesh becoming and retargeting instruments. Multi-stage filters take away artifacts similar to duplicate clips, irregular poses, joint velocity outliers, irregular displacements, lengthy static segments, and foot slips. The movement is then normalized, resampled to 30 fps, and cut up into clips of lower than 12 seconds with a set world body, Y axis up, and character dealing with the constructive Z axis. The ultimate corpus comprises over 3,000 hours of movement, of which 400 hours are high-quality 3D movement with verified captions.
Along with this, the analysis staff has outlined a three-level taxonomy. The highest degree has six lessons: Locomotion, Sports activities and Athletics, Health and Out of doors Actions, Every day Actions, Social Interactions and Leisure, and Sport Character Actions. These lengthen to greater than 200 fine-grained movement classes within the leaf part, overlaying each easy atomic actions and mixtures of simultaneous or sequential motions.
Motion expression and HY-Movement DiT
HY-Movement 1.0 makes use of an SMPL-H skeleton with 22 physique joints with out fingers. Every body is a 201-dimensional vector that concatenates a worldwide root translation in 3D house, a worldwide physique orientation in a steady 6D rotation illustration, 21 native joint rotations in 6D format, and 22 native joint positions in 3D coordinates. Velocity and foot contact labels have been eliminated as they decelerate coaching and don’t assist closing high quality. This illustration is appropriate with animation workflows and is near the DART mannequin illustration.
The core community is a hybrid HY Movement DiT. First, we apply a dual-stream block that processes movement latencies and textual content tokens individually. In these blocks, every modality has its personal QKV projection and MLP, and the joint consideration module permits movement tokens to question semantic options from textual content tokens whereas preserving modality-specific construction. The community then concatenates the movement and textual content tokens into one sequence and processes them with parallel spatial and channel consideration modules right into a single stream block that performs deeper multimodal fusion.
For textual content alignment, the system makes use of a twin encoder scheme. Qwen3 8B supplies token-level embedding, and the CLIP-L mannequin supplies world textual content performance. A bidirectional token refiner corrects the LLM’s causal attentional bias for non-autoregressive technology. These alerts are fed to the DiT by way of the adaptive layer normalization adjustment. Consideration is uneven: a movement token can attend to any textual content token, however a textual content token by no means returns to movement, stopping noisy movement situations from destroying linguistic representations. Temporal consideration throughout the movement department makes use of a slim sliding window of 121 frames. This lets you focus capability on native kinematics whereas protecting prices for lengthy clips extra manageable. Full rotary place embedding is utilized after concatenating textual content and movement tokens to encode the relative place of your entire sequence.
Stream matching, prompt rewriting and coaching
HY-Movement 1.0 makes use of stream matching as a substitute of ordinary denoising diffusion. The mannequin learns the rate subject alongside steady paths that interpolate between Gaussian noise and actual movement information. Throughout coaching, the target is the imply squared error between the anticipated velocity and the bottom fact velocity alongside this path. Throughout inference, the realized atypical differential equations are built-in into clear trajectories from noise, offering secure coaching for lengthy sequences and appropriate with DiT architectures.
Separate period prediction and immediate rewriting modules enhance followability of directions. It makes use of Qwen3 30B A3B as the bottom mannequin and is educated with artificial person model prompts generated from movement captions utilizing VLM and LLM pipelines similar to Gemini 2.5 Professional. This module predicts applicable working occasions and rewrites casual prompts into normalized textual content that’s simpler for DiT to grasp. It’s first educated with supervised fine-tuning after which refined with group-relative coverage optimization utilizing Qwen3 235B A22B because the reward mannequin to attain semantic consistency and interval validity.
The coaching will comply with a three-stage curriculum. Stage 1 performs intensive pre-training on your entire 3,000-hour dataset to study a variety of movement priors and fundamental textual content movement alignment. In Stage 2, we fine-tune the 400-hour high-quality set to sharpen movement particulars and enhance semantic accuracy with a decrease studying fee. Stage 3 applies reinforcement studying. First, we carry out direct desire optimization utilizing 9,228 rigorously chosen human desire pairs sampled from roughly 40,000 generated pairs, after which apply Stream GRPO with composite rewards. This reward combines the semantic rating from the textual content movement retrieval mannequin with a bodily rating that penalizes artifacts similar to foot slip and route drift based mostly on a KL regularization time period to remain near the supervised mannequin.
Benchmarks, scaling habits and limitations
For analysis, the staff constructed a check set of greater than 2,000 prompts throughout six classification classes, together with easy, simultaneous, and sequential actions. Human raters rating followability and movement high quality on a scale of 1 to five. HY-Movement 1.0 has a mean instruction following rating of three.24 and an SSAE rating of 78.6%. Baseline textual content movement methods similar to DART, LoM, GoToZero, and MoMask obtain scores between 2.17 and a couple of.31 with SSAE between 42.7 % and 58.0 %. By way of movement high quality, HY-Movement 1.0 reaches a mean of three.43, whereas the perfect baseline is 3.11.
Within the scaling experiments, we research 0.05B, 0.46B, and 0.46B DiT fashions educated with solely 400 hours and 1B parameters. Instruction followability steadily improves with mannequin dimension, reaching a mean of three.34 for the 1B mannequin. The movement high quality saturates across the 0.46B scale, and the 0.46B and 1B fashions attain comparable averages between 3.26 and three.34. Evaluating a 0.46B mannequin educated in 3,000 hours to a 0.46B mannequin educated in solely 400 hours, we see that bigger information quantity is vital to instruction tuning, and high-quality curation primarily improves realism.
Essential factors
- Billion-scale motion-enabled DiT stream matching: HY-Movement 1.0 is the primary diffuse transformer-based stream matching mannequin scaled to the 1B parameter degree particularly for textual content to 3D human movement, concentrating on high-fidelity command following throughout quite a lot of actions.
- A big and thoroughly chosen movement corpus: The mannequin is pre-trained on over 3,000 hours of reconstruction, movement seize, and animation movement information and fine-tuned on 400 hours of high-quality subsets, all retargeted to a unified SMPL H skeleton and arranged into over 200 movement classes.
- Hybrid DiT structure with highly effective textual content conditioning: HY-Movement 1.0 makes use of hybrid dual-stream and single-stream DiT with uneven consideration, narrowband temporal consideration, and twin textual content encoders, Qwen3 8B and CLIP L to fuse token-level and world semantics to movement trajectories.
- Immediate rewriting and coaching pipeline for RL: A devoted Qwen3 30B-based module predicts working occasions and rewrites person prompts. DiT can also be additional coupled with direct-first optimization and stream GRPO utilizing semantic and bodily rewards to enhance realism and followability of directions past supervised coaching.
Please verify paper and Full code here. Additionally, be happy to comply with us Twitter Do not forget to hitch us 100,000+ ML subreddits and subscribe our newsletter. dangle on! Are you on telegram? You can now also participate by telegram.
Asif Razzaq is the CEO of Marktechpost Media Inc. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of synthetic intelligence for social good. His newest endeavor is the launch of Marktechpost, a man-made intelligence media platform. It stands out for its thorough protection of machine studying and deep studying information, which is technically sound and simply understood by a large viewers. The platform boasts over 2 million views per 30 days, demonstrating its recognition amongst viewers.
