NederlandsKAIST's AI fills in the ground beneath clouds on satellite images
Researchers at South Korea's KAIST built an AI model that learns what the Earth looks like from above, including location and scale. This lets it remove clouds from satellite photos and turn radar images into ordinary photos. That will help with disasters and nature management.
Satellites take millions of photos of the Earth every day. With them we follow forests, fields, cities and the effects of floods. But an ordinary camera in space has the same problem as you on a grey day: through thick clouds you see nothing, and at night neither. At any moment, a large part of the Earth is covered by cloud.
There is a solution: radar. A radar satellite sends its own signals to the ground and catches the reflection. That works through clouds and in the dark. Only radar images do not look like an ordinary photo. They consist of speckles and contrasts that are hard to read even for experts.
An AI model that knows the Earth
A team from the Korea Advanced Institute of Science and Technology (KAIST), led by Professor Munchurl Kim, has built an AI model that tackles both problems. It is called GeoCore-9B. The model creates images, just as well-known AI drawing programs do, but it has only ever seen satellite images. It was trained on about ten million satellite photos from all over the world, each with a short description, such as "a harbour with ships and sheds".
What is special is that with each photo the model also learned where it was taken and how large the piece of ground shown is. That is important, because from above a shed can sometimes look like a factory hall. If you know the scale, you do not make that mistake. Other models usually start from an AI trained on ordinary holiday and street photos and then adapt it. KAIST started from scratch with only images of the Earth.
During training, the model also got help from a second AI that specialises in satellite images. This taught it better how roads run and where fields end. That help is only needed during learning, not in use.
Clouds away and radar as a photo
The researchers then taught the model two practical tasks with a small amount of extra training. The first is removing clouds: the model fills in the cloudy parts of a photo with what the ground beneath probably looks like. The second is turning radar images into images that look like an ordinary photo.
According to the researchers, the model did better at removing clouds than existing models made specially for that one task, measured by brightness, colour and how well the shapes match. When converting radar images, it scored best on some of the measurements.
The next steps
The research has been online as a preprint since August. That means other scientists have not yet formally reviewed it. The researchers now want to test the model with images from more kinds of satellites and from more regions. After that, they want to teach it to predict how the ground changes, for example through floods, forest fires or growing cities.
Important to know: a filled-in spot beneath a cloud is a well-founded estimate by the model, not a real photo. For a quick overview after a disaster that can help a lot; for exact measurements, real observations remain necessary.
What does this mean for the road to the singularity?
This research shows how AI models are increasingly learning a field of their own, in this case the Earth's surface, rather than just ordinary photos. One model that understands the Earth can be used straight away for many tasks. That saves building a separate system for each task.
The best prospect lies in the combination. An AI that can read satellite images can work together with language models that know how floods arise, how farming works and how relief work is organised. Such a system could soon see what is happening on Earth in any weather and help decide at once where help is most urgently needed. That brings an Earth we can see continuously and clearly a good deal closer.
What is the technological singularity? Read the explanation →
