What A Time To Be Alive
Choose language: English

Autonomous John Deere loader learns in simulation to balance on two wheels

Researchers made a five-tonne John Deere loader balance on two wheels, spin and drive over bumps on its own. The controller learned this in a computer simulation, using an hour and a half of measurements on the real machine.

Yellow skid-steer loader seen from the side, balancing forwards on its two front wheels: the rear wheels hang in the air and the bucket floats clear above a dark floor with a glowing blue grid.
Illustration: a five-tonne loader balances on its front wheels, as the controller first learned in a simulation

Most people have only seen a five-tonne loader balance on its front wheels at shows by top operators. Researchers at the Swiss technical university ETH Zürich have now made a John Deere 332G do it without a driver. The machine lifted its rear end, stayed balanced on its two front wheels, turned on the spot, drove over bumps and swung its bucket back and forth. Afterwards it set itself down gently on four wheels again.

The work comes from the Robotic Systems Lab at ETH Zürich, together with Blue River Technology (a subsidiary of John Deere), the Swiss company Gravis Robotics and Intel's Intelligent Systems Lab. It was published in the journal Science Robotics. According to the researchers, until now only a small number of very experienced operators had managed such manoeuvres, during official world-record attempts.

A difficult machine to control

The 332G is a skid-steer loader: a compact machine with four wheels on fixed axles. It steers by turning the wheels on the left and right at different speeds, so that it slides over the ground and changes direction. The wheels are driven hydraulically, which means by oil pressure. Exactly how such a drive responds to a steering command is hard to calculate. And practising by trial and error on a real five-tonne machine is too dangerous and too expensive.

Practising on the computer first

That is why the researchers first spent about an hour and a half measuring how the wheels responded to all kinds of steering commands. Later they added twenty minutes of measurements taken while the machine was balancing. With these measurements they trained a small neural network that predicts how fast the wheels will turn after a command. That network was part of a simulation of the loader.

In that simulation the controller learned through reinforcement learning: trying again and again, and being rewarded for what goes well. To bridge the gap between the simulation and the real world, the researchers varied things such as the position of the loader arm, properties of the machine and the simulated sensor readings. They taught it separate skills, such as lifting, balancing, driving and landing, and on the real machine they switched between those skills.

The same lab earlier taught a humanoid robot to climb ladders and stand up again after a fall, also after a lot of practice in a simulation. That robot also learned from recordings of people moving. For the loader, the lessons came only from measurements on the machine itself.

The tests

Whether lifting onto two wheels succeeds depends on how high the loader arm is. With an arm angle of 28 degrees it worked in all 11 attempts, and at 31 degrees in all 12. At 34 degrees, the highest position of the arm, it worked 11 out of 14 times; the other three times the bucket touched the ground. The angles of 28 and 34 degrees were outside the range the controller had practised in the simulation.

The controller had only practised on flat ground and knew nothing about obstacles. Even so, the loader drove forwards and backwards on two wheels over bumps 75 centimetres high. An existing control method, which keeps calculating ahead what the machine must do, could not get over the first bump.

The machine only sensed how it was tilting and how fast its wheels were turning. It had no cameras and no lidar, a sensor that scans the surroundings with laser light. The controller ran on an ordinary laptop with an Intel Core i7 processor, without a graphics card. For safety, the loader had a frame with a roller at the back to stop it tipping over backwards, and extra small wheels next to the bucket. As a next step, the researchers want to add cameras or lidar, so that the machine can see difficult terrain coming, and to bring the separate skills together in a single controller.

What does this mean for the road to the singularity?

Robots have been learning movements in simulation for a few years, mainly walking robots with electric motors. What is new is that it also works for a heavy machine with a hydraulic drive that is hard to predict, and that an hour and a half of measurements on the real machine was enough. Construction and farm machines differ by brand and type. If each type gets its own practice copy on the computer after a few hours of measuring, a manufacturer can first try out a new controller thousands of times virtually before a machine takes it onto the building site.

Whoever builds better automatic controllers faster this way puts pressure on other manufacturers to do the same. A machine that handles the difficult parts of the work by itself then becomes normal. If perception with cameras and lidar is added, heavy machines can do a larger share of the work on their own, including in places that are dangerous for people. People then supervise and do the work that requires insight. For that, such a machine must respond safely to people nearby, and that is the next step to work out and test.

What is the technological singularity? Read the explanation →

Sources

  1. Tom's Hardware
  2. www.science.org
  3. DongA Science
  4. arXiv: Heavy Metal Acrobatics (Lorentz e.a.)
  5. ETH Zürich Robotic Systems Lab: projectpagina Heavy Metal Acrobatics
  6. Dryad: data bij Heavy Metal Acrobatics