Abstract and key ingredients of the paper

Multi-view structure from motion (SfM) estimates the position and orientation of pictures in a common 3D coordinate frame. When views are treated incrementally, this external calibration can be subject to drift, contrary to global methods that distribute residual errors evenly. We propose a new global calibration approach based on the fusion of relative motions between image pairs. We improve an existing method for robustly computing global rotations. We present an efficient a contrario trifocal tensor estimation method, from which stable and precise translation directions can be extracted. We also define an efficient translation registration method that recovers accurate camera positions.These components are combined into an original SfM pipeline. Our experiments show that, on most datasets, it outperforms in accuracy other existing incremental and global pipelines. It also achieves strikingly good running times: it is about 20 times faster than the other global method we could compare to, and as fast as the best incremental method. More importantly, it features better scalability properties.

Hereby our key ingredients of our Global Structure from Motion chain:

ACRANSAC

Robust and adaptive robust parametric model estimation.

Convex Optimization

Use convex optimization under l-∞ norm in order to compute optimal solution for a provided set of geometric constraints.

Reduced Trifocal Tensor

Fast, robust and accurate estimation of relative translations in an image triplet from known rotations. We use ACRANSAC for robustness and a minimal convex optimization to estimate relative translations really fast.

Fusion of relative translations

A new method to combine relative translations in a common coordinate system using convex optimization.

Our Global Structure from Motion chain is composed as:

  • Pictures

  • Relative pairwise rotations

    Robust essential matrices estimation is performed with ACRANSAC from local features putative matches.

  • Rotation consistency

    Outlier relative rotations are removed using a sequential Bayesian inference over the whole graph using an adjusted cycle length weighting.

  • Global Rotations

    The global rotations are computed using a sparse eigenvalue solver.

  • Relative translations computations

    We use a triplet based coverage estimation of the graph. By using an edge coverage of the graph we compute relative translations thanks to a contrario reduced trifocal tensor from known rotations. ACRANSAC is used for robustness, convex optimization for optimality and a minimal number of constraints is built from 4-uplets of tracks for speed.

  • Global Translations computation

    We integrate the relative translation directions and compute global translations using a l-∞ method.

  • Final structure and motion

    The preceding steps provide a good estimation of the motions, as well as structure per triplet. We link points in these structures by feature tracking and compute 3D point positions per track by triangulation. This global structure and the translations are then refined by bundle adjustment.

  • 3D

Showcase

Some results obtained from our Global Structure from Motion pipeline.

  • MayaHead (50 pictures).

    MayaHead (50 pictures).

    Thanks to Cicero Moraes for the rendering.

    View Details
  • PavillonAurore (421 pictures).

    PavillonAurore (421 pictures).


    View Details

More info

.

Code

released in OpenMVG library
For best performance of l-∞ minimization code please consider the Mosek v6 LP backend. Help

Acknowledgments

This work was supported by Mikros Image and Agence Nationale de la Recherche ANR-09-CORD-003 (Callisto project).

Thanks to Cicero Moraes for the video rendering of the MayaHead mesh.