Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Large Scale Face Recognition Using Metric Learning

No description
by

messy whatever

on 2 July 2015

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Large Scale Face Recognition Using Metric Learning

Large scale face recognition in unconstrained settings using metric learning
Face Database
Liveris Avgerinidis
Supervisors
Prof. Dr. T. Gevers
Dr. R. Valenti

Motivation
Real-life data are open-ended:
350 millions of images uploaded daily

72 hours of video content generated every 1 minute

Ability to analyze < ability to create
1-vs-rest SVMs:
Scale well with big data
Fast test time
Limitations:
When a new class arrives, train from scratch
Storage space
Outline
Introduction
Related Work and Background
Proposed Method
Experimental Evaluation
Conclusion
Face Recognition Challenges
Face recognition a hard and special case of object recognition
Face is non rigid, flexible object
Face recognition tasks
Face Recognition Pipeline
Face detection
Sightcorp software
Face Recognition
Verification/
Identification
Image
Representation
Image Representations
Holistic-based
Feature-based
Holistic-based
Principal Components Analysis (PCA)
Projects data in the most dominant eigenvectors
Linear Discriminant Analysis (LDA)
Maximizes the ratio of between class and within class variance
Projects data to fishervectors
Feature-Based
Gabor wavelets
Histogram of Oriented Gradients (HOG)
Local Binary Patterns (LBP)
Facial Self Similarity (FSS)
Gabor wavelets
Gabor kernels, inspired by human visual system:
Gabor convolution with an image:
Histogram of Oriented Gradients (HOG)
HOG capture:
Local appearance
Shape
Local Binary Patterns (LBP)
LBP capture:
Local structure
Texture
Facial Self Similarity (FSS)
FSS capture:
Geometric layouts
Local self-similarities
Metric learning
Metric learning methods successful in object recognition
Great results on imageNet dataset (14M images and 22K classes)
Face verification tasks
Basic idea:
Nearest Class Mean Metric Learning (NCMML) classifier
Distance based metric learning
Allows addition of new classes at nearly zero cost
Nearest Class Mean Classifier
Face identification:
closed-universe (accuracy)
open-universe (ROC curves)
Introduction
Related Work and Background
Proposed Method
Experimental Evaluation
Conclusion
Outline
Introduction
Related Work and Background
Proposed Method
Experimental Evaluation
Conclusion
Outline
We focus on:
Large scale face recognition
Addition of new face images and classes on the fly
Face Detection
Face detection components:
Face registration
Face normalization
Rotate the face so that the angle between the two eyes is zero (horizontal position).
Scale the face so that the distance between the two eyes remains constant.
Translate the face so that the eyes are always centered horizontally and at a specific desired height.
Face Registration
Face normalization
Face normalization:
Gamma correction
Difference of Gausians (DOG) filtering
Contrast equalization
Feature Extraction
Features:
Gabor wavelets
HOG
LBP
FSS
Face Recognition
Discussion
High similarity of frontal view
Can NCM be used with success in face recognition domain?
Face recognition domain
Object recognition domain
Robust feature representations (SIFT)
Research questions
We study NCM in face recognition field
Specifically:
Which image representations are suitable for metric learning?
How does the system perform in open and closed universe identifications tasks?
Can generalize to new classes?
Nearest Class Mean metric learning (NCMML)
Low rank projection matrix W : d x D
Mahalanobis distance computed by:
Allows image compression to d dimensions
Cheap computation of distances
Nearest class mean classifier

Probabilistic formulation using the soft-min formulation:
Discriminative maximum likelihood training:
Three not linearly separable classes
Visualization of learned distance
Identification

closed-universe
open-universe
Overview of proposed method
Introduction
Related Work and Background
Proposed Method
Experimental Evaluation
Conclusion
Outline
Controlled datasets
CMU MultiPie
Extended Yale B
AR
Uncontrolled dataset
Pubfig+LFW
Error analysis
Generalization to new classes
Evaluation
CMU MultiPie
Systematically measure performance across head poses with various features
Extended Yale B
AR
Similar performance to state of the art
Higher projection dimensions lead to better performance
Complementary nature of features
Discussion
Uncontrolled setting
Pubfig83+LFW Dataset:
8,720 images with 83 face classes from Pubfig83
12,066 distractors from LFW
closed-universe
open-universe
Closed-universe
Comparison with methods that use the same features and face detection software
Closed-universe
Comparison with other published works which use different features and face detection software:
Different experimental setup
More training images per class used
Open universe
Open-universe performance across different projection dimensions
Open universe
Comparison with other methods
Mislabelled cases
Occlusions
In certain poses, people look similar
Low resolution
Hairstyle
Generalization performance
Addition of new classes consists of computing their means in the learned space
Almost real time procedure
Generalization performance
Effect of number of trained classes into Pubfig83 dataset with 83 face classes
Generalization performance
Impact of adding new classes
Outline
Introduction
Related Work and Background
Proposed Method
Experimental Evaluation
Conclusion
Conclusion

Research questions:
Which image representations are suitable for metric learning?
combination of HOG, Gabor, FSS, LBP
How does the system perform in open and closed universe identifications tasks?
on par with state of the art
Can generalize to new face classes?
competitive performance
Further improvements
Try multiple centroids for class representation, lead to non linear classifier
Dimensionally reduced with PCA
Zero-meaned
Concatenated
Photometric normalization
Color information used
Cropped faces
Background used
Family 101 dataset

Discussion
Competitive performance with SVM in open and closed universe tasks
Higher projected dimensions, but still a lot less than the initial, perform better
Perform better when we increase training data
3:1 trained-to-added class ratio
1:1 trained-to-added class ratio
1:3 train-to-added class ratio
Questions?
Thank you!
retrain
new image
Open universe task
new face class
learned metric W
class addition
3 trained classes
+ 1 added
=total 4 classes
Full transcript