Improved Base Model and Trajectory Estimation
Coming soon
BeyondRetarget Plus: Data Engine for Generating Robot Motion Data from Internet-Scale Human Videos
1School of Electronic Science and Engineering, Nanjing University, Nanjing, China
2School of Intelligence Science and Technology, Nanjing University, Suzhou, China
3Jiangsu Mobile Information System Integration Co., Ltd., Nanjing, China
4China Mobile Zijin (Jiangsu) Innovation Research Institute Co., Ltd., Nanjing, China
*Equal contribution · †Corresponding author
BeyondRetarget (base) focuses on real-time monocular-video retargeting and teleoperation. BeyondRetarget Plus extends the base method with global trajectory modeling, moving-camera support, and improved generalization and motion reconstruction for offline robot motion data generation from Internet-scale human videos.
Video demonstrations
Teleoperation Video
Teleoperation video coming soon
Video 2. Placeholder for the real-time teleoperation demonstration.
Abstract
Internet human videos contain rich demonstrations of everyday behavior, but converting them into executable robot motion at scale remains challenging. BeyondRetarget Plus presents a scalable framework for collecting robot-ready motion data directly from massive in-the-wild human videos. The framework combines visual motion understanding, robot-aware adaptation, temporal consistency, and contact-aware refinement to transform unconstrained video observations into physically plausible and executable robot trajectories. By supporting diverse humanoid embodiments and reducing the need for costly motion-capture collection, BeyondRetarget Plus provides a practical path toward large-scale robot motion data construction from Internet videos.
Approach
BeyondRetarget Plus extends the base method with global trajectory modeling, moving-camera support, and delivers stronger generalization capability and motion reconstruction performance.
A Larger, More Comprehensive Training Dataset
Coming soon
Improved Robot Motion Supervision
Coming soon
Hover over a card to reveal its description.
Failure Case
Camera motion estimation may struggle with videos featuring overly uniform backgrounds, potentially leading to abnormal trajectories. Highly dynamic and challenging motions may cause estimation failures due to motion blur and out-of-distribution (OOD) inputs.
BibTeX
@misc{xiong2026beyondretarget,
title={BeyondRetarget: Learning Executable Humanoid Motions Directly from Monocular Video},
author={Tianyu Xiong and Yi Lu and Jinrui Wang and Ziqi Liang and Dandan Lei and Xiaoyang Zhou and Xiao-xiao Long and Qiu Shen and Xun Cao},
year={2026},
eprint={2609.29850},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2609.29850},
}