Apple researchers train humanoid robot in a thousand virtual rooms

A humanoid robot that can be asked in everyday language to sit on a chair or push a box aside: to do that, it must see where it is, understand the instruction and move its whole body correctly. In a paper dated last Wednesday, 4 November, Apple researchers describe a system that learned this mainly in a computer simulation, a simulated world inside a computer. They call it HALO.

In the test with a humanoid robot from the Chinese brand Unitree, the G1 model, the tasks succeeded in 91 per cent of 360 attempts, according to the researchers. This happened in six real spaces that the robot had never seen before and that had not been mapped in advance.

Why practise in a simulation

A robot learns such tasks from examples: what it sees, the instruction and the matching movement. Often a person demonstrates those movements, for example by controlling the robot remotely. Every hour of examples then costs at least an hour of human work. As a result, the amount of training material grows slowly.

So the researchers let the computer do most of the work. They built a thousand virtual rooms: living rooms, offices and bedrooms. An AI model that had learned from real human movements worked out how the robot's body should move in them. Automatic checks threw out movements that are impossible in reality, such as feet sinking through the floor or sliding over the ground. This produced about 300 hours of training material. It did not work entirely without people: for each new task, the researchers still wrote the instructions and set a few key poses.

Three layers on the robot itself

Everything runs on the robot, without a cable to the outside. A Mac mini, a small computer from Apple, sits in a case on its back. It runs an existing language model that also understands images: Qwen3.6, an open model from the Chinese company Alibaba. Every two seconds, that model breaks an instruction down into small steps. A second part uses the camera image to make a movement plan for the next five seconds. A third part controls the joints fifty times per second. For this it uses a depth camera, which measures how far away everything is, and data on the robot's own posture.

The robot could sit down, walk to a named object, step sideways through a half-open door, crouch and crawl under a table, and push boxes. It could also carry out these skills one after another, for example first going through a door and then walking to an object. Such sequences succeeded in the real world in 70 per cent of cases, tested in one space. Failures happened mainly when the goal of the next step was still out of view. The code and videos are on the HALO project page. Apple has not announced a robot: HALO is a research project.

What does this mean for the road to the singularity?

Robots that learn to walk or move an arm in a simulation are already common. What is new about HALO is that the interplay of seeing, understanding an instruction and moving the whole body also comes from the computer, instead of from hours of demonstrations by people. Anyone who makes training material with computing power instead of human work can let a robot practise in many more rooms and on many more tasks for the same money. And every time computers get cheaper, the number of situations in which a robot has already practised grows.

That puts pressure on other robot makers. A company that mainly has people demonstrate movements pays roughly the same for every new hour of training material, while a competitor using simulations pays less with each new generation of computers. Because Apple is making the code public, anyone can adopt the approach and improve it. HALO still sees only with cameras and loses track of an object that disappears from view; that is the next thing to improve. If the trend continues, a household robot will not need to get to know a home before it can help there, because it has already practised in thousands of similar rooms. For someone who has trouble walking, such a robot could then simply push the box out of the hallway.

What is the technological singularity? Read the explanation →