Machine learning has brought significant improvements to pattern recognition systems in the past decade [5]. The three main factors that this success is attributed to are the use of deep neural networks, very large annotated training datasets, and abundant computing resources. However, for many earth observations problems only small training datasets are available. One promising family of machine learning techniques for dealing with limited training data is meta-learning—also known as learning to learn [4]. Given several pattern recognition problems from the same application domain (e.g., object detection in satellite images) these approaches are able to automatically construct new machine learning algorithms that are specialised for the application domain of interest. When deployed on novel tasks from the same domain, these meta-learned algorithms require less data and computing resources in order to train an effective model compared to conventional machine learning approaches.

Meta-learning methods are most commonly applied to datasets containing images of people or everyday objects, with the goal of creating learning algorithms that are able to quickly train models on images that contain novel object categories. Such methods are unsuitable for the earth observation problem setting, because of the additional structure in earth observation data compared to conventional everyday photo collections. In addition, in many situations satellites will acquire multiple images of the same region, and there will be considerable overlap in content between these images. This overlapping content violates standard independence assumptions made by machine learning methods. Naively ignoring this structure will yield machine learning models with suboptimal performance, but bespoke meta-learning approaches that instead explicitly take advantage of these temporal and spatial structures in the data have the potential to provide even more powerful models.

The objective of this project is to develop a meta-learning framework that is suitable for use with remote sensing applications, thus enabling scientists working with earth observation data to rapidly train accurate models without costly data annotation processes or machine learning expertise. The meta-learning approaches will take advantage of the multiple image modalities commonly captured by satellite imaging systems (e.g., RGB, infrared, multispectral/hyperspectral, range/LiDAR, etc), and leverage additional contextual information of the geographic areas in which datasets are gathered (e.g., latitude and longitude, historical weather patterns, population density, etc) and the relative position of different regions of interest within the same dataset. By combining deep learning with novel Bayesian prediction heads, the framework will be able to take advantage of all the benefits provided by well[1]calibrated uncertainty estimates. Examples of such benefits include easily detecting anomalies [2], being able to fuse information from other sources with the information extracted from satellite images, and active learning [1] to further make the most effective use of limited label annotation resources.

Case Study on Marine Heatwave Detection: The improved detection of Marine Heatwaves (MHWs) will serve as a case study to validate the efficacy of the proposed methods. MHWs are prolonged and anomalous warm water events in a given ocean region. Severe MHWs have significant impacts on marine ecosystems and potentially on weather patterns [3]. The approach proposed here has the advantage of considering the nonstationary behaviour of Sea Surface Temperature and its spatial variability as compared to traditional methods based on a pixel-by-pixel analysis [3]. The proposed project is highly relevant, as it will develop methods for anomaly detection that can integrate data multiple sources, such as satellite and sub-surface measurements (e.g., Argo floats).

References

[1] Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning, pages 1183–1192. PMLR, 2017.

[2] David J Hill, Barbara S Minsker, and Eyal Amir. Real-time bayesian anomaly detection for en[1]vironmental sensor data. In Proceedings of the Congress-International Association for Hydraulic Research, volume 32, page 503, 2007.

[3] Alistair J Hobday, Lisa V Alexander, Sarah E Perkins, Dan A Smale, Sandra C Straub, Eric CJ Oliver, Jessica A Benthuysen, Michael T Burrows, Markus G Donat, Ming Feng, et al. A hierar[1]chical approach to defining marine heatwaves. Progress in Oceanography, 141:227–238, 2016.

[4] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):5149–5169, 2021.

[5] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–444, 2015.