Researchers at Apple’s AI/ML group have posted a paper describing HALO, a humanoid autonomy system trained only on synthetic data from 1,000 procedurally generated homes, then tested in real rooms the robot had never seen, according to an arXiv preprint submitted Nov. 4 and a project page under Apple’s research organization.
The paper, “Perceptive Humanoid Motion from Simulation,” argues that mapping pixels and free-form language to joint commands has been limited by a lack of large paired datasets, and that human teleoperation can take at least an hour of human time to produce an hour of data. HALO instead synthesizes vision-language-motion data so the corpus “scales with compute rather than human labor,” the authors wrote. They say the system learns skills including sitting on chairs, navigating to named objects, stepping sideways through half-open doors, crouching and crawling beneath tables, and pushing boxes, and can orchestrate those skills for longer tasks.
“Without ever seeing a real home during training, HALO transfers zero-shot to the physical world, achieving a 91% success rate across 360 trials in six unmapped real-world environments,” the abstract says. Those results are the authors’ reported experimental figures, not an independent replication or a product announcement from Apple. The project page lists training details including imitation learning on 64 NVIDIA B200 GPUs and reinforcement learning on 4 H100s, with the corpus generated on 250 H100s, and names authors including Siyuan Hu, Abulikemu Abuduweili, Yue Meng, Shipeng Liu, Quentin Leboutet, Chen Chen, Matthias Müller and Vladlen Koltun. Code and videos are linked from the project site.