AI company pays plumbers and technicians to teach robots home repairs

A leaking tap, a circuit breaker that keeps tripping or an air conditioner that no longer cools: anyone living in Minneapolis, Chicago or Miami can now have such a job fixed for free. The American company HUMXN, which collects training data for AI, pays the bill. There is one condition: during the work, the technician wears a camera that records what they see and do. Those recordings become learning material for robots.

HUMXN announced the programme on 6 October. It covers plumbing, electrics, heating and air conditioning, air duct cleaning and carpet cleaning. Local trade businesses do the work and are paid by HUMXN.

Why robots need technicians

Language models such as ChatGPT learned from enormous amounts of text and images from the internet. A robot that has to repair something itself needs something else: examples of how people work with their hands in the real world. Such examples are scarce.

An instruction video usually shows how a repair should be done. HUMXN wants to capture the thinking of the skilled worker instead. Where does a plumber look first when water is leaking? What does an electrician test to find a fault? Which tool does a technician pick up, what do they do when something goes differently than expected, and how do they know the job is finished? Because the camera is on the technician, the footage looks the way they see it themselves, with their hands and tools in the centre. Robotics researchers call this egocentric video: footage from the point of view of the person doing the work.

According to founder Thomas Unise, the next generation of AI needs more than what is on the internet. In his view, technicians carry out millions of technical jobs every day, and robots can learn a lot from them.

The choice of three very different cities is also deliberate. A cooling system in warm Miami develops different problems than one in cold Minneapolis, and the pipes in an old house in Chicago are different from those in a home in Miami. The more of that variety a robot sees, the better it will later cope with situations outside the lab.

How it works

Residents describe their job on HUMXN's website. A staff member assesses whether it fits the programme. It has to be a real repair or maintenance job of normal size: a service visit or at most a day's work. Major renovations, new construction, roofing and emergency jobs are not yet included.

If the job fits, a participating trade business makes an appointment. The technician works as usual, just with the camera on. This creates a model with three winners: the resident pays nothing, the trade business gets its money, and AI and robotics companies get data that is hard for them to collect themselves.

On privacy, HUMXN says the recordings are about the work, not the resident. The camera focuses on hands, tools and equipment. According to the company, faces, names, post, screens and other recognisable details that still end up in the picture are made unrecognisable before anything is shared, and the address is never passed on. Residents do not have to be on camera themselves, and nothing is scheduled or recorded before they agree. The footage goes under licence to approved AI and robotics companies. Los Angeles, New York and San Francisco are on the list as the next cities.

A growing market for real footage

HUMXN is not the only one working this way. Since the end of May, the German company MicroAGI has been offering free cleaning through its app Shift in New York, with the cleaners wearing a camera on their heads. Mecka AI pays people to film themselves with body sensors and a smartphone during everyday tasks, such as making coffee or repairing a car. According to TechCrunch, Mecka is about to raise money at a valuation of about 500 million dollars (about 430 million euros). Competitor XDOF is reportedly in talks at a valuation of 1.2 billion dollars (about 1 billion euros).

A week after the start of the home programme, HUMXN itself announced that it wants to buy 50 million dollars' worth of data (about 43 million euros) from trade businesses, farmers and manufacturers. There too, the aim is to make accumulated practical knowledge usable for AI and robots.

What still needs to happen

The programme has only just started. The coming months will show how many jobs it produces and which robotics companies start working with the footage. The privacy arrangements are currently HUMXN's own; clear rules and independent oversight of the anonymisation of footage from homes are a logical next step for this whole sector.

Something is also changing for the skilled workers themselves. Their knowledge is gaining a new value, and HUMXN says that the people who provide that value should also share in the benefits. How that works out in practice for technicians will be an important question now that robots can do more step by step.

Video alone is also only a beginning. A robot also has to learn how much force to apply to a nut or a fragile pipe. Footage of real jobs does help it understand what needs to happen and in what order, and that is exactly the part that is most lacking now.

What does this mean for the road to the singularity?

In recent years, AI has mainly learned from what people wrote down. The next leap requires knowledge that has never been written down: the craftsmanship in the hands of plumbers, electricians and technicians. Initiatives like HUMXN's make that knowledge visible to machines on a large scale for the first time. That is an important building block for robots that find their way not only in a factory, but also in ordinary homes. There is still a long way to go before a robot can fix a leak on its own, but with every recorded job, the textbook from which it will learn keeps growing.

What is the technological singularity? Read the explanation →