Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Transcript of ISPRS-TCV-2014-Keynote
Extract cameras from E
Reconstruct model by intersection
Refine with bundle adjustment
Separate exterior orientation of two images
Solve two resections, with GCP
Combined, single stage orientation (bundle block adjustment with block=2)
Use GCP, but can work without (arbitrarily fix the gauge freedom)
Two-steps combined orientation
Use GCP but can work without
Stereo-models are correct up to a similarity
Separate exterior orientation of two images
Combined single stage orientation
Relative + absolute orientation
Merge of partial models
Camera (frame) space
Multiple views (block)
and all of them simultaneously
Alignment via points correspondences
Points are used to compute orientations:
absolute in IBA
exterior in resection/intersection
both in the hierarchical method
Only some points/image are used at intermediate steps. The final model results from a merge.
Use two-view tools iteratively.
Alignment of camera frames (w/o points)
Intersection only after cameras are aligned
This is partially global: the first stage is local (many relative orientations, but the alignment of camera frames is global (w/o points though).
Two views (stereo)
This was my home page
And this was the state of the art in modeling from images (CVPR'97)
I was challenged by my PhD supervisor with the problem of reconstructing an object's geometry from casual, uncalibrated images.
Of Course, I did not succeeded, but (as a partial apology):
"It is good to have an end to journey toward; but
it is the journey that matters
, in the end." [E. Hemingway]
DIEG - University of Udine
It's image-based modelling
It's structure and/from motion
It's shape reconstruction
It's stereo/block processing
It's block adjustment
It's aerial/photogrammetric triangulation
Structure from motion: state of the art, examples and open issues - the Computer Vision perspective
Piazza Erbe, Udine
Back in 1996...
What was a challenge in 1997 has become a commodity.
There are now software applications that solves the problem off-the-shelf.
Let's agree on names
Let's put it in perspective
A possible taxonomy
Specific CV references are in the relevant slides. These are general refs on the CV-PH connection
R. I. Hartley and J. L. Mundy, "
The relationship between photogrammetry and computer vision
," SPIE93-photogrammetry, 1993, pp. 92--105.
Computer Vision and Photogrammetry Mutual Questions: Geometry, Statistics and Cognition.
" Int. Symp. Photogrammetry meets Geoinformatics, 2002
Computer Vision and Remote Sensing – Lessons Learned
. Photogrammetrische Woche 2009
Digital camera calibration methods: Considerations and comparisons
. ISPRS Comm. V Symp. 2006.
Fabio Crosilla for his guidance into the Photogrammetry world
Fabio Remondino for his suggestions that helped to improve this presentations
Roberto Toldo is the main programmer of "Samantha" and produced the results that have been shown
Seitz, Dyer. Photorealistic Scene Reconstruction by Voxel Coloring
Pollefeys, Van Gool. A Stratified Approach to Metric Self-Calibration
Stein, Shashua. Model-based Brightness Constraints: on Direct Estimation of Structure and Motion
Structure and motion
Compute 3D points (structure) and camera exterior parameters (motion) from image point correspondences (tie-points)
The gold-standard method is bundle adjustment, used in CV to refine solutions provided by other methods.
Focusing on automation, BA is not seen in CV as a stand-alone method.
CV methods concentrate on a free-network solution, as GCP are considered an optional piece of information. If GCPs are available, solve an absolute orientation in the end.
CV methods are devised for irregular, unknown blocks;
a lot of effort is put on recovering the
(= block structure)
Sequential SfM (Bundler) and its hierarchical variation (Samantha) proved the most effective in practical applications. They are cousins of IMBA.
Global motion-first methods are very promising (use frames vs points).
For each pair of overlapping photographs a stereo-model is built (by relative orientation)
Then, all these independent models are simultaneously transformed into a common reference frame by similarity transformations.
The common reference system is usually the ground ref. system, but if no GCP are available it can be any model coordinates system.
CV tools: a quick overview
All points must be seen in all views (in principle)
Due to the low-rank constraint, matrix completion can be used
Projective reconstruction (Euclidean upgrade is required)
Initial guess needed
Moulon, Monasse, Marlet. Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion. ICCV 2013
Robust global motion estimation with matrix completion - F. Arrigoni, B. Rossi, F. Malapelle, P. Fragneto and A. Fusiello.
Equivalent to collinearity equations
Like in the case of the bifocal tensor (E), camera matrices can be extracted from the trifocal and quadrifocal tensors.
This can be seen as generalization to 3 and 4 views of the relative orientation
The first step step of the global-motion first methods is linked to the global registration of multiple 3D point sets in
The second step of the IMBA is linked to the global registration of multiple 3D point sets in
note: rigid vs similarity
of multiple 3D point sets
Sharp, Lee, Wehe, 2001. Toward multiview registration in frame space
Benjemaa, Schmit, 1998. A solution for the registration of multiple 3D point sets using unit quaternions
Beinat, A., Crosilla, F., 2001. Generalized procrustes analysis for size and shape 3d object reconstruction
R. Gherardi, M. Farenzena, A. Fusiello. Improving the Efficiency of Hierarchical Structure-and-Motion. CVPR 2010 (Samantha)
Snavely, Seitz, Szeliski, 2008. Modeling the world from internet photo collections (Bundler)
Bill Triggs, Philip F. McLauchlan, Richard I. Hartley, and Andrew W. Fitzgibbon. Bundle adjustment - a modern synthesis. International Workshop on Vision Algorithms, 2000.
[Diagram from Triggs et al]
On the two-views methods there is a good agreement.
Relative orientation is central, even if the Essential matrix formulation is peculiar of CV, it is equivalent to the coplanarity constraint
Exterior orientation (resection) is not used in the context of two-views processing, but it is known in CV (also as PnP or :camera pose")
BA is common background. In CV used as a refinement step (discussed ahead)
Photographs courtesy of F. Remondino
Results obtained with 3Dflow's
Demo available at the exposition booth
Results obtained with "Samantha" (code available from
P. Sturm and B. Triggs. A factorization based algorithm for multi-image projective structure and motion. EECV 1996
John Oliensis and Richard Hartley. Iterative extensions of the sturm/triggs algorithm: Convergence and nonconvergence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2007.
A relative orientation variant
This presentation on Prezi web site:
A unifying note: The Gauss-Helmert method can be seen as a Gauss-Newton iteration with a special approximation of the Hessian [Kanatani, Niitsuma 2011]
Modulo different notation, in the end there is agreement on the problems to be solved, usually non-linear LS.
Also the solution methods are largely overlapping, after a closer look
Thanks to the projective geometry approach, CV also developed fast, linear sub-optimal methods (DLT-like) that can be used for initialization.
Assessment of accuracy of results, sensitivity/reliability analysis is less a concern in CV
Ref: Börlin, Grussenmeyer. BUNDLE ADJUSTMENT WITH AND WITHOUT DAMPING. The Photogrammetric Record 28(144): 2013
The numerical implementations of BA can differ, but all of them (from CV and PH) stem from the Gauss-Newton method.
Indeed, if the cost function is weighted by the true measurement covariances, there is no difference between the Gauss-Newton method and the so-called Gauss-Markov adjustment (from PH).
L-M (from CV) is basically the Gauss-Newton method, to which the gradient descent principle is combined to improve convergence.
This can be seen as an instance of a more general class of damped Gauss-Newton methods (see ref.)
To travel is to discover that everyone is wrong about other countries
[W. Förstner citing A. Huxley]