NederlandsNVIDIA releases free software that adapts human movements for robots
With SOMA Retargeter, developers can take a recorded human movement, such as walking or waving, and adapt it to the body of a humanoid robot. Robots can then use that movement as an example to practise on a computer first.
Anyone who wants to teach a humanoid robot to walk or wave can start with a person who shows how it is done. But a robot cannot simply copy such a movement: its arms and legs have different lengths, its joints sit in different places and cannot turn as far. That is why NVIDIA has released SOMA Retargeter, free open-source software that adapts a human movement to the body of a robot. the company explained in a blog post how it works. Developers can find the program on GitHub, a website where programmers share code.
From human to robot
The program works with movements stored on a fixed digital skeleton from NVIDIA, the SOMA skeleton. Such a movement can come from motion capture, in which an actor is filmed wearing a suit with measuring points. It can also be taken from an ordinary video, or be generated by an AI model.
First, the program scales the human skeleton to the size of the robot. Then, for every frame of the movement, it calculates how the robot's joints must be positioned to copy the pose as closely as possible. If the robot cannot follow everything exactly, settings decide which part of the body takes priority. A separate function makes sure the feet do not slide across the floor at moments when they should stand still on the ground. The result is a file with the robot's pose for every frame.
The current version has ready-made settings for five humanoid robots: the G1 and H2 from Unitree, the T1 from Booster, and the X2 Ultra and A3T3 from AgiBot. Developers can add another robot themselves. The software comes with example movements from the company Bones, which records human movements.
Practising before it is real
A converted movement like this is not yet a way to control the robot, but an example. The robot then has to learn how to drive its motors to follow that example without falling over. It first practises this in a simulation, a recreated world on the computer. The simulation also shows whether it keeps its balance and stands firmly on the ground. NVIDIA advises always testing a movement this way before a real robot performs it. The program is still in a test phase, so its features may still change.
According to NVIDIA, MotionBricks, an AI model that creates smooth movements for the Unitree G1, was partly trained with movements converted by this software. It was shown on a real G1. In a separate experiment, the small robot MicroDuck learned to move in a simulation using converted human movements.
What does this mean for the road to the singularity?
Robots already learn from humans. NVIDIA's GR00T 2 learned partly from 100,000 hours of video that people filmed through their own eyes, and practising in a recreated world has become normal for robot makers. What is new is a free tool that makes one recorded human movement suitable for robots with very different builds. Then not every robot maker has to film actors in suits themselves: a movement that has been recorded once can serve many robots.
This gives robot makers a reason to have their robot included in such a shared program. Developers will start work sooner with a robot whose settings are already in place than with one where they have to work everything out themselves. NVIDIA does not earn money from the free software itself, but it does from the chips on which all that practice runs. If that trend continues, a shared collection of human movements will grow, just as there are now huge collections of text that language models learn from. A new robot would then no longer start from zero, but could practise with thousands of human movements on its very first day: walking, lifting, handing something over. That makes it cheaper to build robots that can soon help out at home or in healthcare.
What is the technological singularity? Read the explanation →
