TwinAligner: Visual and Physical Real2Sim2Real All-in-One for Robotic Manipulation

The robotics field is evolving towards data-driven, end-to-end learning, inspired by multimodal large models. However, the reliance on costly real-world data limits progress. Simulators offer cost-effective alternatives, but the gap between simulation and reality challenges effective policy transfer.

We introduces TwinAligner, a novel Real2Sim2Real system addressing both visual and physics gaps. The visual alignment module achieves pixel-level alignment through SDF reconstruction and editable 3DGS rendering, while the physics alignment module ensures dynamic consistency via robotic system identification and trajectory-based parameter identification. TwinAligner enhances robot learning by providing scalable data collection and establishing a trustworthy iterative cycle, accelerating algorithm development. Quantitative evaluations highlight TwinAligner's strong capabilities in visual and physics real-to-sim alignment. This system enables policies trained in simulation to achieve strong zero-shot generalization to the real world. The high consistency between real-world and simulated policy performance underscores TwinAligner's potential to significantly advance scalable robot learning.