Prezi

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in the manual

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

AVEDac

Automated Video Event Detection and Classification
by Danelle Cline on 5 February 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of AVEDac

Classifier software developed by Perona student Marc’Aurelio Ranzato at Caltech and Universita’ degli studi di Padova

Developed to analyze biological particles (pollen and urine samples)
Implemented in Matlab
Processes color square images of the detections cropped from the scene

Features based on local jets
3x108 component feature vector per image
Use Fisher’s Linear Discriminant (FLD) to reduce dimension

Square images classified using a classifier obtained from a Mixture of Gaussians (MoG) generative model Classifier Overview AVEDac Overview Sept 29, 2011 MBARI Example Transect Video Detections AVEDac Overview Sept 29, 2011 MBARI Eye-in-the-Sea AVEDac Overview Sept 29, 2011 MBARI Detection works well on benthic or midwater video with non-cluttered backgrounds and uniform lighting
Classifier improvements needed to improve accuracy
Different features
Different classifiers
Ensemble learning (multiple classifiers)
Performance improvement needed for real-time use
5 frames per second processing rates on 8-node, dual-XEON 2.4 GHz CPU Beowulf cluster* Conclusions and Future Work AVEDac Overview Sept 29, 2011 MBARI Example Time-Lapse Image Detections AVEDac Overview Sept 29, 2011 MBARI http://www.mbari.org/aved/
Danelle Cline
Duane Edgington

Monterey Bay Aquarium Research Institute (MBARI) The Automated Visual Event Detection and Classification System
(AVEDac)

Project information, papers, etc.
http://www.mbari.org/aved/

Software freely available
http://avedac.googlecode.com


Danelle Cline dcline@mbari.org
Duane Edgington duane@mbari.org Thank You AVEDac Overview Sept 29, 2011 MBARI
Two features used:

Appearance-based method
Local Jets (Schmid et al. 1997)

Image and power spectrum principle components (Torralba et. al. 2003) Classifier Features Used AVEDac Overview Sept 29, 2011 MBARI Developed by Ocean Research and Conservation Association (ORCA)
Uses far-red lights and a intensified low-light camera to view animals natural behaviors unobtrusively
In 2009-2010, EITS was connected to MARS observatory
6 months of video recorded 24 hours per day 7 days a week during this period Eye-in-the-Sea (EITS)
on Monterey Accelerated Research System (MARS) AVEDac Overview Sept 29, 2011 MBARI Scale Space Approach
Biologically Inspired Computer Vision AVEDac Overview Sept 29, 2011 MBARI Convolution of input image with derivatives of the Gaussian kernel
Linear combination of local jets can be invariant with respect to
Shift
Rotation
Animals are not oriented, therefore shift and rotation invariance are very important Features Based on Local Jets AVEDac Overview Sept 29, 2011 MBARI If the features capture the peculiarities of appearance of our particle, then any classifier will perform a competent job (Duda & Hart, 1973) Importance of Feature Extraction AVEDac Overview Sept 29, 2011 MBARI Develop Automated Visual Event Detection and classification (AVEDac) software system
Detection
Tracking
Classification (what, e.g. Family:asterridae, Genus: Rathbunaster) Approach AVEDac Overview Sept 29, 2011 MBARI ~67,000 time-lapse still-frame images taken once per hour, representing 16 years of data at the Station M site off the coast of central California Station M camera tripod deployment Station M Time-lapse AVEDac Overview Sept 29, 2011 MBARI 3D Map of Saliency Output Typically every
5th input frame Color, Intensity, Orientation computed on 8 spatial scales then combined into saliency map
Peaks in saliency map indicate the points with most visual attention
Map scanned in order decreasing saliency through WTA
Each “winner” point is output as a coordinate, e.g. (142,30), (132,12).
These coordinate points are then used in tracking system (Kalman filter or nearest neighbor) Saliency Attention Model Detection
Biologically Inspired Computer Vision AVEDac Overview Sept 29, 2011 MBARI Station M time-lapse camera launch Ventana ROV launch MARS observatory AUV descending Video and still images collected from:
Underwater cabled observatories
Remotely Operated Vehicles (ROVs)
Autonomous Underwater Vehicles (AUVs)
Tripod ocean-bottom time-lapse camera systems
Abundance and distribution studies
Behavioral studies Project Motivation AVEDac Overview Sept 29, 2011 MBARI Every frame events.XML Detection Segmentation
and
Tracking Post-
processing Image Preprocessing
Remove scan lines
Mask time code overlay
etc. Detection and Tracking
Algorithm Overview AVEDac Overview Sept 29, 2011 MBARI Step 2: Classification Step 1: Detection Detection and Classification
Two Step Approach Other Benthocodon
(jellyfish) Macrourids
(rattail fish) Echinocrepis
(sea urchin) Biological
Classifier AVEDac Overview Sept 29, 2011 MBARI Monochromatic
feature extractor 100 x 100 = 10000 values Blue 108 values per B image Monochromatic
feature extractor 100 x 100 = 10000 values Green 108 values per G image 324 values
per image 100 x 100 = 10000 values Red Monochromatic
feature extractor 108 values per R image D values per image Fisher
Linear
Discriminant 100 x 100 x 3 = 30000 values Color Averaged Local Jets AVEDac Overview Sept 29, 2011 MBARI Actual Class Predicted Class Leukothele/Rathbunaster Test Results
Confusion Matrix AVEDac Overview Sept 29, 2011 MBARI 100 x 100 = 10000 values 108 values per image 27 values per image 27 values per image 27 values per image Average 9 values per pixel Invariants 27 values per pixel Positive
Negative
Absolute Average 9 values per pixel Invariants 27 values per pixel Positive
Negative
Absolute Average 9 values per pixel Invariants 27 values per pixel Positive
Negative
Absolute 27 values per image Average 9 values per pixel Invariants 27 values per pixel Positive
Negative
Absolute Scale Space Approach AVEDac Overview Sept 29, 2011 MBARI Blurring Blurring Blurring Can you see MBARI ? Try blurring your eyes *for 29.97 frames-per-second, interlaced, NTSC video About half-way through the video you’ll see a Humboldt Squid (Dosidicus gigas) attracted to the artificial e-jelly “lure” mimicking an Atolla jellyfish.
For more information see http://www.mbari.org/mars/science/eits.html Eye-in-the-Sea Typically, every 5th frame
for 30 fps video Monterey Bay Station M 1-hour time lapse Monterey Bay ROV benthic (bottom) transect Features Based on Image and Power Spectrum Principle Components Monterey Bay ROV midwater transect Holothurian Classification Test Results
Confusion Matrix Scotoplanes globosa Predicted Class (%) Actual Class (%) Based on 7.5 minutes of benthic (ocean bottom) transect video
Trained classifier with 3 classes totaling ~6000 images
Tested 1031 events or ~21000 images
Used 80% probability threshold & probability voting on each event Based on 10 minutes of benthic (ocean bottom) transect video
Trained classifier with 8 classes totaling ~7000 images / 5 minutes segment
Tested 46 events or ~4700 images / 5 minutes segment
Used 80% probability threshold & probability voting on each event Every frame
See the full transcript