Elliot Vincent

Pixel-wise Agricultural Image Time Series Classification: Comparisons and a Deformable Prototype-based Approach

Elliot Vincent1,2Jean Ponce3,4Mathieu Aubry1

1LIGM, Ecole des Ponts, Univ Gustave Eiffel, CNRS
2Inria Paris3Department of Computer Science, Ecole normale superieure (ENS-PSL, CNRS, Inria)4Courant Institute of Mathematical Sciences and Center for Data Science, New York University


Improvements in Earth observation by satellites allow for imagery of ever higher temporal and spatial resolution. Leveraging this data for agricultural monitoring is key for addressing environmental and economic challenges. Current methods for crop segmentation using temporal data either rely on annotated data or are heavily engineered to compensate the lack of supervision. In this paper, we present and compare datasets and methods for both supervised and unsupervised pixel-wise segmentation of satellite image time series (SITS). We also introduce an approach to add invariance to spectral deformations and temporal shifts to classical prototype-based methods such as K-means and Nearest Centroid Classifier (NCC). We show this simple and highly interpretable method leads to meaningful results in both the supervised and unsupervised settings and significantly improves the state of the art for unsupervised classification of agricultural time series on four recent SITS datasets.

Proposed method

1. A prototype-based method


Overview. Given an agricultural satellite image time series, we assign to each pixel independently the label of the prototype which leads to the best reconstruction. Each prototype follows spectral and temporal deformations in order to match a given input time series.

2. Introducing spectral and temporal invariances


Learned transformations. We show the visual interpretations of our time series deformations. The offset deformation is time-independent and performed on each spectral band separately. On the other hand, time warping is channel-independent and achieved by translating landmark time-steps, allowing targeted temporal adjustments.


If you find this work useful for your research, please cite:
      title={Pixel-wise Agricultural Image Time Series Classification: Comparisons and a Deformable Prototype-based Approach}, 
      author={Elliot Vincent and Jean Ponce and Mathieu Aubry},

Further information

If you like this project, check out related works from our group:


The work of MA was partly supported by the European Research Council (ERC project DISCOVER, number 101076028). JP was supported in part by the Louis Vuitton/ENS chair on artificial intelligence and the French government under management of Agence Nationale de la Recherche as part of the Investissements d'avenir program, reference ANR19-P3IA0001 (PRAIRIE 3IA Institute). This work was granted access to the HPC resources of IDRIS under the allocation 2021-AD011013067 made by GENCI. We thank Antoine Guedon for inspiring discussions; Tom Monnier, Romain Loiseau, Ricardo Garcia Pinel, Theo Bodrito, Yannis Siglidis and Loic Landrieu for manuscript feedback and constructive insights.