New experiments show that in-orbit ML payloads can be retrained on new images

We are experiencing a ‘golden age’ for Earth observations (EO), with increasing numbers of satellites being launched into orbit every month. The sheer numbers of Earth-facing cameras present an amazing opportunity: independent satellites could act in concert to share data, processing power and sensors, leading to a much greater capability for monitoring the health of our planet.

However, there are still significant challenges to overcome before fulfilling the vision of a cooperative network of EO satellites. Managing large satellite constellations is a complex task, which can be made easier by giving satellites more autonomy. In addition, downloading big data from orbit is difficult and costly because of limited communication bandwidth and issues coordinating available ground stations. 

The solution to these issues may lie in machine learning (ML) capabilities. Onboard intelligence could help automate analysis in orbit so that only high-level data products are downloaded, while “federated learning” — which trains a model in multiple sessions with unique datasets — could help satellites share smaller representations of what they have learned. 

The foundations of this solution have now been established. In June 2021, the D-Orbit Wildride mission was launched into space by a SpaceX Falcon 9 rocket from Cape Canaveral. On board this mission was the Unibap Nebula module: an on-demand, in-orbit cloud computing platform containing a new ML Payload called “WorldFloods”. WorldFloods was initially developed in partnership with ESA Φ-Lab during the 2019 Frontier Development Lab (FDL) Europe program to prototype in-orbit flood-mapping on resource-constrained satellite hardware. 

A series of experiments have run on Nebula since then, with support from ESA Φ-Lab, D-Orbit, and Unibap. In these experiments, the WorldFloods payload successfully created accurate maps of water, land and cloud from remote sensing images, and then rapidly transmitted compressed flood maps back to Earth. Moreover, the WorldFloods payload was quickly retrained to generate flood maps from a completely new camera, and this new capability demonstrated in-orbit.

These experiments have shown that ML-based models deployed in-orbit can be updated if new information is available, paving the way for agile integration of onboard and on-ground processing as well as continuous learning.

Advances like these are moving us towards a new, intelligent space age where multiple intelligence spacecraft work together to provide data and services faster, and with more accuracy. 

Spacecraft working together with spacecloud infrastructure can enable hybrid observation and adaptive in-space services, with the potential to revolutionise how we respond to disasters, such as flooding and wildfires, manage emissions and pollution, improve weather forecasts, and enable next generation space situational awareness. Challenges such as space weather, planetary defence from asteroids, and mitigating orbital debris will also benefit from these capabilities. 

Read the full paper in the Nature Scientific Journal.