Image Collation

Illustrations of different plants from the same manuscript

Same illustration in three manuscripts with different styles

Abstract

Illustrations are an essential transmission instrument. For an historian, the first step in studying their evolution in a corpus of similar manuscripts is to identify which ones correspond to each other. This image collation task is daunting for manuscripts separated by many lost copies, spreading over centuries, which might have been completely re-organized and greatly modified to adapt to novel knowledge or belief and include hundreds of illustrations. Our contributions in this paper are threefold. First, we introduce the task of illustration collation and a large annotated public dataset to evaluate solutions, including 6 manuscripts of 2 different texts with more than 2 000 illustrations and 1 200 annotated correspondences. Second, we analyze state of the art similarity measures for this task and show that they succeed in simple cases but struggle for large manuscripts when the illustrations have undergone very significant changes and are discriminated only by fine details. Finally, we show clear evidence that significant performance boosts can be expected by exploiting cycle-consistent correspondences.

1 min Video

Dataset

The manuscripts in our dataset come from three different texts, have diverse number of illustrations and come from diverse. In total, it includes more than 2 000 illustrations and 1 300 annotated correspondences.

To download the dataset, please refer to ManuscriptDownloader .

Visual Results

Query and 5 nearest neighbors according to our similarity score

Query and 5 nearest neighbors (correct in Green) according to our normalized score

Query and 5 nearest neighbors (correct in Green) after information propagation

Code and Paper

Code
Paper

Slides

Poster

To cite our paper,

@inproceedings{kaoua2021imagecollation,
  title={Image Collation: Matching illustrations in manuscripts},
  author={Kaoua, Ryad and Shen, Xi and Durr, Alexandra and Lazaris, Stavros and Picard, David and Aubry, Mathieu},
  booktitle={International Conference on Document Analysis and Recognition (ICDAR)},
  year={2021}
}

Acknowledgment

This work was supported in part by ANR project EnHerit ANR-17-CE23-0008, project Rapid Tabasco, and gifts from Adobe. We thank Alexandre Guilbaud for fruitful discussions.