teaser teaser
(a) Historical watermark recognition
teaser teaser
(b) One-shot repeated pattern detection
We present a unified approach to two problems: (a) historical watermark recognition, and (b) repeated-pattern detection in artwork collections.

Abstract

Progress in the digitization of cultural assets leads to online databases that become too large for a human to analyze. Moreover, some analyses might be challenging, even for experts. In this paper, we explore two applications of computer vision to analyze historical data: watermark recognition and one-shot repeated pattern detection in artwork collections. Both problems present computer vision challenges which we believe to be representative of the ones encountered in cultural heritage applications: limited supervision is available, the tasks are fine-grained recognition, and the data comes in several different modalities. Both applications are also highly practical, as recognizing watermarks makes it possible to date and locate documents, while detecting repeated patterns allows exploring visual links between artworks. We demonstrate on both tasks the benefits of relying on deep mid-level features. More precisely, we define an image similarity score based on geometric verification of mid-level features and show how spatial consistency can be used to fine-tune out-of-the-box features for the target dataset with weak or no supervision. This paper relates and extends our previous works ArtMiner and Historical Watermark Recognition

Method



    teaser
    Mid-level feature learning with weak supervisions or without supervisions.
  • (a) Our approach relies on candidate correspondences obtained by matching the features of a proposal region (in red) to the full database.
  • (b) In case of category-level supervisions (top), we verify whether matched pairs belong to the same category and their spatial difference is small. If no supervisions is available (bottom), the candidate correspondences are verified by matching the features of the verification region (in blue) of the query in the candidate images and checking for consistency.
  • (c) Finally, we extract features from the positive (in green) and negative (in yellow) regions and use them to improve the features using a metric learning loss.


    • Visual Results




    One-shot historical watermark recognition, more results can be found here



    teaser


    One-shot artwork retrieval on Venus database, results on other datasets can be found here



    teaser       teaser
    Left : Query outlined in blue                                     Right :Top six detection results


    Code, Data and Paper


    Code & Data ArtMiner
    Code & Data Watermark
    Paper

    To cite our paper, 

    @article{shen2021spatially,
  title={Spatially-consistent Feature  Matching and Learning for Art Collections and Watermark Recognition},
  author={Shen, Xi and Champenois, Robin and Ginosar, Shiry and Pastrolin, Ilaria and Rousselot, Morgane and Bounou, Oumayma and Monnier, Tom and Gidaris, Spyros and Bougard, Fran{\c{c}}ois and Raverdy, Pierre-Guillaume and Limon, Marie-Fran{\c{c}}oise and B{\'{e}}n{\'{e}}vent, Christine and Smith, Marc and Poncet, Olivier and Bender, K and B{\'{e}}atrice, Joyeux-Prunel and Honig, Elizabeth and Efros, Alexei A and Aubry, Mathieu},
  journal = {International Journal of Computer Vision (IJCV)},
  year={2022}
}




    Acknowledgment

    This work was supported in part by ANR project EnHerit ANR-17-CE23-0008, PSL Filigrane pour tous project, project Rapid Tabasco, gifts from Adobe to Ecole des Ponts. We thank Minsu Cho, Pascal Monasse and Renaud Marlet for fruitful discussions, and Kenny Oh and Davienne Shields for thier help on defining the task and building the Brueghel dataset.