Mauro Dalla Mura

Neural koopman prior for data assimilation

By Anthony Frion, Lucas Drumetz, Mauro Dalla Mura, Guillaume Tochon, Abdeldjalil Aissa El Bey

2024-06-02

In IEEE Transactions on Signal Processing

Abstract

With the increasing availability of large scale datasets, computational power and tools like automatic differentiation and expressive neural network architectures, sequential data are now often treated in a data-driven way, with a dynamical model trained from the observation data. While neural networks are often seen as uninterpretable black-box architectures, they can still benefit from physical priors on the data and from mathematical knowledge. In this paper, we use a neural network architecture which leverages the long-known Koopman operator theory to embed dynamical systems in latent spaces where their dynamics can be described linearly, enabling a number of appealing features. We introduce methods that enable to train such a model for long-term continuous reconstruction, even in difficult contexts where the data comes in irregularly-sampled time series. The potential for self-supervised learning is also demonstrated, as we show the promising use of trained dynamical models as priors for variational data assimilation techniques, with applications to e.g. time series interpolation and forecasting.

Continue reading

Learning sentinel-2 reflectance dynamics for data-driven assimilation and forecasting

By Anthony Frion, Lucas Drumetz, Guillaume Tochon, Mauro Dalla Mura, Abdeldjalil Aïssa El Bey

2023-05-29

In Proceedings of the 31th european signal processing conference (EUSIPCO)

Abstract

Over the last few years, massive amounts of satellite multispectral and hyperspectral images covering the Earth’s surface have been made publicly available for scientific purpose, for example through the European Copernicus project. Simultaneously, the development of self-supervised learning (SSL) methods has sparked great interest in the remote sensing community, enabling to learn latent representations from unlabeled data to help treating downstream tasks for which there is few annotated examples, such as interpolation, forecasting or unmixing. Following this line, we train a deep learning model inspired from the Koopman operator theory to model long-term reflectance dynamics in an unsupervised way. We show that this trained model, being differentiable, can be used as a prior for data assimilation in a straightforward way. Our datasets, which are composed of Sentinel-2 multispectral image time series, are publicly released with several levels of treatment.

Continue reading

Leveraging neural koopman operators to learn continuous representations of dynamical systems from scarce data

By Anthony Frion, Lucas Drumetz, Mauro Dalla Mura, Guillaume Tochon, Abdeldjalil Aïssa-El-Bey

2023-02-17

In Proceedings of the 48th IEEE international conference on acoustics, speech, and signal processing (ICASSP)

Abstract

Over the last few years, several works have proposed deep learning architectures to learn dynamical systems from observation data with no or little knowledge of the underlying physics. A line of work relies on learning representations where the dynamics of the underlying phenomenon can be described by a linear operator, based on the Koopman operator theory. However, despite being able to provide reliable long-term predictions for some dynamical systems in ideal situations, the methods proposed so far have limitations, such as requiring to discretize intrinsically continuous dynamical systems, leading to data loss, especially when handling incomplete or sparsely sampled data. Here, we propose a new deep Koopman framework that represents dynamics in an intrinsically continuous way, leading to better performance on limited training data, as exemplified on several datasets arising from dynamical systems.

Continue reading

Assimilation de données variationnelle de séries temporelles d’images sentinel-2 avec un modèle dynamique auto-supervisé

By Anthony Frion, Lucas Drumetz, Mauro Dalla Mura, Guillaume Tochon, Abdeldjalil Aïssa-El-Bey

2022-06-14

In 29e colloque sur le traitement du signal et des images

Abstract

Au cours des dernières années, l’apprentissage profond a acquis une importance croissante dans de nombreux domaines scientifiques, notamment en ce qui concerne le traitement d’images, et en particulier pour le traitement des données issues de satellites. Le paradigme le plus courant en ce qui concerne l’apprentissage profond est l’apprentissage supervisé, qui requiert une grande quantité de données annotées représentant la vérité terrain pour la tâche d’intérêt. Or, obtenir des données correctement annotées pose souvent des difficultés financières ou techniques importantes. Pour cette raison, nous nous plaçons ici dans le cadre de l’apprentissage auto-supervisé. Nous proposons un modèle d’apprentissage profond inspiré de la théorie de l’opérateur de Koopman qui apprend, à partir de séries temporelles d’images multispectrales Sentinel-2, à modéliser les dynamiques de long terme de réflectance des pixels. Après son entraînement, notre modèle peut être utilisé dans divers problèmes inverses faisant intervenir la dynamique temporelle pour résoudre différentes tâches telles que l’interpolation ou le débruitage de données.

Continue reading

Learning endmember dynamics in multitemporal hyperspectral data using a state-space model formulation

By Lucas Drumetz, Mauro Dalla Mura, Guillaume Tochon, Ronan Fablet

2020-01-24

In Proceedings of the 45th IEEE international conference on acoustics, speech, and signal processing (ICASSP)

Abstract

Hyperspectral image unmixing is an inverse problem aiming at recovering the spectral signatures of pure materials of interest (called endmembers) and estimating their proportions (called abundances) in every pixel of the image. However, in spite of a tremendous applicative potential and the avent of new satellite sensors with high temporal resolution, multitemporal hyperspectral unmixing is still a relatively underexplored research avenue in the community, compared to standard image unmixing. In this paper, we propose a new framework for multitemporal unmixing and endmember extraction based on a state-space model, and present a proof of concept on simulated data to show how this representation can be used to inform multitemporal unmixing with external prior knowledge, or on the contrary to learn the dynamics of the quantities involved from data using neural network architectures adapted to the identification of dynamical systems.

Continue reading

Braids of partitions for the hierarchical representation and segmentation of multimodal images

By Guillaume Tochon, Mauro Dalla Mura, Miguel Angel Veganzones, Thierry Géraud, Jocelyn Chanussot

2019-07-01

In Pattern Recognition

Abstract

Hierarchical data representations are powerful tools to analyze images and have found numerous applications in image processing. When it comes to multimodal images however, the fusion of multiple hierarchies remains an open question. Recently, the concept of braids of partitions has been proposed as a theoretical tool and possible solution to this issue. In this paper, we demonstrate the relevance of the braid structure for the hierarchical representation of multimodal images. We first propose a fully operable procedure to build a braid of partitions from two hierarchical representations. We then derive a framework for multimodal image segmentation, relying on an energetic minimization scheme conducted on the braid structure. The proposed approach is investigated on different multimodal images scenarios, and the obtained results confirm its ability to efficiently handle the multimodal information to produce more accurate segmentation outputs.

Continue reading

Constructing a braid of partitions from hierarchies of partitions

By Guillaume Tochon, Mauro Dalla Mura, Jocelyn Chanussot

2019-03-13

In Mathematical morphology and its application to signal and image processing – proceedings of the 14th international symposium on mathematical morphology (ISMM)

Abstract

Braids of partitions have been introduced in a theoretical framework as a generalization of hierarchies of partitions, but practical guidelines to derive such structures remained an open question. In a previous work, we proposed a methodology to build a braid of partitions by experimentally composing cuts extracted from two hierarchies of partitions, notably paving the way for the hierarchical representation of multimodal images. However, we did not provide the formal proof that our proposed methodology was yielding a braid structure. We remedy to this point in the present paper and give a brief insight on the structural properties of the resulting braid of partitions.

Continue reading

Advances in utilization of hierarchical representations in remote sensing data analysis

By Guillaume Tochon, Mauro Dalla Mura, Miguel-Angel Veganzones, Silvia Valero, Philippe Salembier, Jocelyn Chanussot

2017-11-08

In Comprehensive remote sensing, 1st edition

Abstract

The latest developments in sensor design for remote sensing and Earth observation purposes are leading to images always more complex to analyze. Low-level pixel-based processing is becoming unadapted to efficiently handle the wealth of information they contain, and higher levels of abstraction are required. Region-based representations intend to exploit images as collections of regions of interest bearing some semantic meaning, thus easing their interpretation. However, the scale of analysis of the images has to be fixed beforehand, which can be problematic as different applications may not require the same scale of analysis. On the other hand, hierarchical representations are multiscale descriptions of images, as they encompass in their structures all potential regions of interest, organized in a hierarchical manner. Thus, they allow to explore the image at various levels of details and can serve as a single basis for many different further processings. Thanks to its flexibility, the binary partition tree (BPT) representation is one of the most popular hierarchical representations, and has received a lot of attention lately. This article draws a comprehensive review of the most recent works involving BPT representations for various remote sensing data analysis tasks, such as image segmentation and filtering, object detection or hyperspectral classification, and anomaly detection.

Continue reading

Object tracking by hierarchical decomposition of hyperspectral video sequences: Application to chemical gas plume tracking

By Guillaume Tochon, Jocelyn Chanussot, Mauro Dalla Mura, Andrea Bertozzi

2017-04-20

In IEEE Transactions on Geoscience and Remote Sensing

Abstract

It is now possible to collect hyperspectral video sequences at a near real-time frame rate. The wealth of spectral, spatial and temporal information of those sequences is appealing for various applications, but classical video processing techniques must be adapted to handle the high dimensionality and huge size of the data to process. In this article, we introduce a novel method based on the hierarchical analysis of hyperspectral video sequences to perform object tracking. This latter operation is tackled as a sequential object detection process, conducted on the hierarchical representation of the hyperspectral video frames. We apply the proposed methodology to the chemical gas plume tracking scenario and compare its performances with state-of-the-art methods, for two real hyperspectral video sequences, and show that the proposed approach performs at least equally well.

Continue reading

Region-based classification of remote sensing images with the morphological tree of shapes

By Gabriele Cavallaro, Mauro Dalla Mura, Edwin Carlinet, Thierry Géraud, Nicola Falco, Jón Atli Benediktsson

2016-04-12

In Proceedings of the IEEE international geoscience and remote sensing symposium (IGARSS)

Abstract

Satellite image classification is a key task used in remote sensing for the automatic interpretation of a large amount of information. Today there exist many types of classification algorithms using advanced image processing methods enhancing the classification accuracy rate. One of the best state-of-the-art methods which improves significantly the classification of complex scenes relies on Self-Dual Attribute Profiles (SDAPs). In this approach, the underlying representation of an image is the Tree of Shapes, which encodes the inclusion of connected components of the image. The SDAP computes for each pixel a vector of attributes providing a local multiscale representation of the information and hence leading to a fine description of the local structures of the image. Instead of performing a pixel-wise classification on features extracted from the Tree of Shapes, it is proposed to directly classify its nodes. Extending a specific interactive segmentation algorithm enables it to deal with the multi-class classification problem. The method does not involve any statistical learning and it is based entirely on morphological information related to the tree. Consequently, a very simple and effective region-based classifier relying on basic attributes is presented.

Continue reading

Search

Tags