Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.



Keynote talk given at the ISPRS TC V Symposium. Riva del Garda, 23/6/14

Andrea Fusiello

on 31 May 2016

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of ISPRS-TCV-2014-Keynote

Compute essential matrix E
Extract cameras from E
Reconstruct model by intersection
Refine with bundle adjustment
Separate exterior orientation of two images

Solve two resections, with GCP
Combined, single stage orientation (bundle block adjustment with block=2)

Use GCP, but can work without (arbitrarily fix the gauge freedom)
Two-steps combined orientation
relative orientation
absolute orientation

Use GCP but can work without

Stereo-models are correct up to a similarity

Separate exterior orientation of two images
Combined single stage orientation
Relative + absolute orientation
Global modeling
Merge of partial models
Camera (frame) space
Points space
Global motion-first
Independent models
block adjustment
Hierachical variation

Multiple views (block)
Use points
and all of them simultaneously
Alignment via points correspondences

Points are used to compute orientations:
absolute in IBA
exterior in resection/intersection
both in the hierarchical method

Only some points/image are used at intermediate steps. The final model results from a merge.
Use two-view tools iteratively.
Alignment of camera frames (w/o points)
Intersection only after cameras are aligned

This is partially global: the first stage is local (many relative orientations, but the alignment of camera frames is global (w/o points though).
Computer Vision
Two views (stereo)
Epipolar geometry
Let's hybridize
This was my home page
And this was the state of the art in modeling from images (CVPR'97)
I was challenged by my PhD supervisor with the problem of reconstructing an object's geometry from casual, uncalibrated images.
Of Course, I did not succeeded, but (as a partial apology):
"It is good to have an end to journey toward; but
it is the journey that matters
, in the end." [E. Hemingway]
Andrea Fusiello
DIEG - University of Udine

It's image-based modelling
It's structure and/from motion
It's shape reconstruction
It's stereo/block processing
It's block adjustment
It's aerial/photogrammetric triangulation
Structure from motion: state of the art, examples and open issues - the Computer Vision perspective
Piazza Erbe, Udine
Back in 1996...
What was a challenge in 1997 has become a commodity.

There are now software applications that solves the problem off-the-shelf.
Let's agree on names
Let's put it in perspective
Some background
A possible taxonomy

Specific CV references are in the relevant slides. These are general refs on the CV-PH connection

R. I. Hartley and J. L. Mundy, "
The relationship between photogrammetry and computer vision
," SPIE93-photogrammetry, 1993, pp. 92--105.
Förstner. "
Computer Vision and Photogrammetry Mutual Questions: Geometry, Statistics and Cognition.
" Int. Symp. Photogrammetry meets Geoinformatics, 2002
Computer Vision and Remote Sensing – Lessons Learned
. Photogrammetrische Woche 2009
Remondino, Fraser.
Digital camera calibration methods: Considerations and comparisons
. ISPRS Comm. V Symp. 2006.

Fabio Crosilla for his guidance into the Photogrammetry world

Fabio Remondino for his suggestions that helped to improve this presentations

Roberto Toldo is the main programmer of "Samantha" and produced the results that have been shown
Seitz, Dyer. Photorealistic Scene Reconstruction by Voxel Coloring
Pollefeys, Van Gool. A Stratified Approach to Metric Self-Calibration
Stein, Shashua. Model-based Brightness Constraints: on Direct Estimation of Structure and Motion
Bundle Block
Multifocal constraints
Structure and motion
Compute 3D points (structure) and camera exterior parameters (motion) from image point correspondences (tie-points)

The gold-standard method is bundle adjustment, used in CV to refine solutions provided by other methods.

Focusing on automation, BA is not seen in CV as a stand-alone method.

CV methods concentrate on a free-network solution, as GCP are considered an optional piece of information. If GCPs are available, solve an absolute orientation in the end.

CV methods are devised for irregular, unknown blocks;
a lot of effort is put on recovering the
epipolar graph
(= block structure)

Sequential SfM (Bundler) and its hierarchical variation (Samantha) proved the most effective in practical applications. They are cousins of IMBA.

Global motion-first methods are very promising (use frames vs points).

For each pair of overlapping photographs a stereo-model is built (by relative orientation)
Then, all these independent models are simultaneously transformed into a common reference frame by similarity transformations.

The common reference system is usually the ground ref. system, but if no GCP are available it can be any model coordinates system.
CV tools: a quick overview
All points must be seen in all views (in principle)
Due to the low-rank constraint, matrix completion can be used

Projective reconstruction (Euclidean upgrade is required)
Initial guess needed
Moulon, Monasse, Marlet. Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion. ICCV 2013
Robust global motion estimation with matrix completion - F. Arrigoni, B. Rossi, F. Malapelle, P. Fragneto and A. Fusiello.
WG1 poster
(Tue PM)
Equivalent to collinearity equations
Like in the case of the bifocal tensor (E), camera matrices can be extracted from the trifocal and quadrifocal tensors.

This can be seen as generalization to 3 and 4 views of the relative orientation
The first step step of the global-motion first methods is linked to the global registration of multiple 3D point sets in
frame space

The second step of the IMBA is linked to the global registration of multiple 3D point sets in
point space

note: rigid vs similarity
Global registration
of multiple 3D point sets
Sharp, Lee, Wehe, 2001. Toward multiview registration in frame space
Benjemaa, Schmit, 1998. A solution for the registration of multiple 3D point sets using unit quaternions
Beinat, A., Crosilla, F., 2001. Generalized procrustes analysis for size and shape 3d object reconstruction

R. Gherardi, M. Farenzena, A. Fusiello. Improving the Efficiency of Hierarchical Structure-and-Motion. CVPR 2010 (Samantha)

Snavely, Seitz, Szeliski, 2008. Modeling the world from internet photo collections (Bundler)
Bill Triggs, Philip F. McLauchlan, Richard I. Hartley, and Andrew W. Fitzgibbon. Bundle adjustment - a modern synthesis. International Workshop on Vision Algorithms, 2000.
[Diagram from Triggs et al]
On the two-views methods there is a good agreement.

Relative orientation is central, even if the Essential matrix formulation is peculiar of CV, it is equivalent to the coplanarity constraint

Exterior orientation (resection) is not used in the context of two-views processing, but it is known in CV (also as PnP or :camera pose")

BA is common background. In CV used as a refinement step (discussed ahead)
Photographs courtesy of F. Remondino
Results obtained with 3Dflow's
software (
Demo available at the exposition booth
Results obtained with "Samantha" (code available from

P. Sturm and B. Triggs. A factorization based algorithm for multi-image projective structure and motion. EECV 1996

John Oliensis and Richard Hartley. Iterative extensions of the sturm/triggs algorithm: Convergence and nonconvergence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2007.
A relative orientation variant
This presentation on Prezi web site:
A unifying note: The Gauss-Helmert method can be seen as a Gauss-Newton iteration with a special approximation of the Hessian [Kanatani, Niitsuma 2011]
Modulo different notation, in the end there is agreement on the problems to be solved, usually non-linear LS.

Also the solution methods are largely overlapping, after a closer look

Thanks to the projective geometry approach, CV also developed fast, linear sub-optimal methods (DLT-like) that can be used for initialization.

Assessment of accuracy of results, sensitivity/reliability analysis is less a concern in CV
Ref: Börlin, Grussenmeyer. BUNDLE ADJUSTMENT WITH AND WITHOUT DAMPING. The Photogrammetric Record 28(144): 2013
The numerical implementations of BA can differ, but all of them (from CV and PH) stem from the Gauss-Newton method.

Indeed, if the cost function is weighted by the true measurement covariances, there is no difference between the Gauss-Newton method and the so-called Gauss-Markov adjustment (from PH).

L-M (from CV) is basically the Gauss-Newton method, to which the gradient descent principle is combined to improve convergence.

This can be seen as an instance of a more general class of damped Gauss-Newton methods (see ref.)
Trifocal constraint
To travel is to discover that everyone is wrong about other countries

[W. Förstner citing A. Huxley]
Full transcript