Robot arm with artificial skin feels where it is touched, even without looking

Robots see the world mainly through cameras. That works well as long as everything is in view. But what if a person bumps into the arm from the side, or something blocks the view? People notice that at once, because we feel all over our body. Most robot arms only feel something in their 'hand', or nothing at all.

Researchers at the Hong Kong University of Science and Technology (HKUST) therefore built a robot arm that feels touch over its upper arm and forearm. They call it EmArm. The research, led by Professor Shen Yajing, appeared in the journal Nature Sensors.

A skin inside the skeleton

The researchers looked at the human body: bones provide strength, muscles and skin are soft and sensitive. EmArm works in the same way. Inside the hard frame of the arm is a soft layer that acts as skin. That skin covers more than 710 square centimetres, slightly more than a sheet of A4 paper.

The skin works with light instead of electricity. Tiny cameras, smaller than a millimetre, look at the skin from the inside. When something presses against the arm, the skin deforms slightly. The cameras see that deformation and the computer calculates from it where the arm is being touched, how hard, over what area and in how many places at once. This way, no complicated network of wires is needed, as with electronic skin.

According to the researchers, the skin feels more finely than the skin on a human arm. The robot can already tell apart two touches about 2 centimetres apart. For the skin on our arm, that has to be 3 to 5 centimetres. The arm stays sturdy meanwhile: more than 15 kilos can press on the skin without the feeling or the work suffering.

Feeling and moving in one loop

Feeling alone is not enough. The robot also has to understand quickly what it feels and what it should then do. According to Shen, that is the most important part of the work. The arm combines what it feels with what it knows about the position of its own joints. It compares that with a previously learned approach, and a new movement follows immediately. The researchers deliberately did not choose a large AI model that learns everything from examples: that needs a lot of labelled data and a lot of computing power, and then a robot reacts too slowly.

In trials, the arm showed three things:

It kept a basketball lying off-centre on the arm in balance by adjusting its posture and force.
When a hand got in the way, the arm felt it, stopped, judged whether the obstacle would stay there and looked for a new route. That also worked when the camera saw nothing.
It recognised five kinds of touch (stroking, squeezing, poking, tickling and hitting) in up to 99.8 per cent of cases, and responded with a 'feeling', such as happy, curious or shy.

What comes next?

The research was done in the lab, with one arm. The researchers now want to apply the same approach to other parts of a robot, such as legs and the torso. They are thinking mainly of humanoid robots that work safely alongside people in factories, at home or in care, precisely in places where a camera alone is not enough.

WATTBA wrote earlier about a soft sheet that measures how it bends and twists. Sensors like that also help robots to feel their own body better.

What does this mean for the road to the singularity?

AI models keep getting smarter, but a robot that works with people also has to be able to feel the world. EmArm shows that touch over a large part of the robot is possible, without complicated wiring and without slow calculations. When this kind of skin soon works together with AI models that also see, hear and plan, robots will gain something that resembles our own sense of the body. That is an important step towards robots that take part safely and naturally in our daily lives.

What is the technological singularity? Read the explanation →