teaser
  • Given an initial similarity graph between a query and its top candidates in image retrieval (left), we propose a module dubbed Subgraph Similarity Refiner (SSR) to improve the similarity graph (right).

    • Abstract

    In the problems of image retrieval and few-shot classification, the mainstream approaches focus on learning a better feature representation. However, directly tackling the distance or similarity measure between images could also be efficient. To this end, we revisit the idea of re-ranking the top-k retrieved images in the context of image retrieval (e.g., the k-reciprocal nearest neighbors \cite{qin2011hello,zhong2017re}) and generalize this idea to transductive few-shot learning. We propose to meta-learn the re-ranking updates such that the similarity graph converges towards the target similairty graph induced by the image labels. Specifically, the re-ranking module takes as input an initial similarity graph between the query image and the contextual images using a pre-trained feature extractor, and predicts an improved similarity graph by leveraging the structure among the involved images. We show that our re-ranking approach can be applied to unseen images and can further boost existing approaches for both image retrieval and few-shot learning problems. Our approach operates either independently or in conjunction with classical re-ranking approaches, yielding clear and consistent improvements on image retrieval (CUB, Cars, SOP, rOxford5K, rParis6K) and transductive few-shot classification (Mini-ImageNet, tiered-ImageNet and CIFAR-FS) benchmarks.

    Video

    Method



    Overview of SSR

            Inputs: a similarity graph.
            Goal: updating the similarity graph.
            How: SSR decomposes the N * N similarity graph into N subgraphs where rows and columns of the matrix are ordered depending on similarities to the subgraph reference image.
            Output: an improved similarity matrix.
    Trulli
    Overview of SSR

    Image retrieval

        SSR module

            Rows: the subgraph reference image (red) and the query image (green).
            Columns: top-retrieved images.
    Trulli
    SSR for image retrieval

        Results on image retrieval benchmarks: rOxford5K and rParis6K

    Trulli

    Transductive few-shot learning

        SSR module

            We illustrate our idea with an 1-shot-2-way example:
                Rows: the subgraph reference image (red) and the support set S;
                Columns: the support set S and the query set Q.
    Trulli
    SSR for transductive few-shot classification

        Results on transductive few-shot classification: Mini-ImageNet, Tiered-ImageNet and CIFAR-FS

    Trulli


    Code and Paper


    Code
    Paper
    Slides

    To cite our paper, 

    @inproceedings{shen2021reranking,
  title={Re-ranking for image retrieval and transductive few-shot classification},
  author={Shen, Xi and Xiao, Yang and Hu, Shell Xu, and Sbai, Othman and Aubry, Mathieu},
  booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
  year={2021}
}




    Acknowledgment

    This work was supported in part by ANR project EnHerit ANR-17-CE23-0008, project Rapid Tabasco, and IDRIS under the allocation AD011011160R1 made by GENCI.