Prezi

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in the manual

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Deep-sea Guide - 2013 MLML Presentation

No description
by Brian Schlining on 11 October 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Deep-sea Guide - 2013 MLML Presentation

What does one do with 18,000 hours of underwater video? The path to MBARI's Deep-Sea Guide
Some Background About MBARI
About Me
Brian Schlining
1995 - Graduate Student at MLML in Physical Oceanography
1995 - Took the MATLAB programming class at MLML
1997 - Internship at MBARI
1998 - Began working at MBARI
1999 - Graduated from MLML
The mission of MBARI is to achieve and maintain a position as a world center for advanced research and education in ocean science and technology, and to do so through the development of better instruments, systems, and methods for scientific research in the deep waters of the ocean.
MBARI emphasizes the peer relationship between engineers and scientists as a basic principle of its operation.
All of the activities of MBARI must be characterized by excellence, innovation, and vision.


First ROV dive was recorded on August 1st 1988

MBARI's three ROVs have performed over 4750 dives

The entire ROV dive is recorded. All video is archived

Those dives have produced about 24,000 tapes
ROV Trivia
About the ROV Video
Benthic ROV Coverage
Why Video?
Powerful tool for capturing scientifically
valuable information

Immediate and substantial contributions
to education and outreach programs
The Problem
How do you use video data to
produce good science?
A picture is worth a thousand words ... if you can find it
Use video for quantitative and qualitative science ... especially if you can associate it with other data (e.g. position, temperature, salinity, etc.)
Video Annotations
Trained science staff identify animal taxonomy, animal behavior, geological features, evidence of human impact and other objects recorded in the ROV video stream
Video lab research technicians, MLML Alumni
What is an Annotation?
Taonius at 02:25:30:28 on Tape T0325-03
Identification of the object of interest
Index into Media
ID of the Media or Tape
time: 2002-12-21 17:02:57
latitude: 36.331
longitude: -122.901
depth: 984.8 m
salinity: 34.383 psu
temperature: 3.940 C
Ancillary Data
surface-color | self | red
swimming | nil | nil
image-quality | self | good
Descriptive Information
How do scientists create Annotations?
Video Annotation and Reference System (VARS)
Annotation
Query
Knowledgebase
http://vars.sourceforge.net
cockatoo squid!
but wait that's a common name
... I need a scientific name
I'll use the family name: Cranchidea
or is it Cranchidae?
or maybe Cranchiidae?
Are Annotations Useful?
VARS has been used in nearly 245 peer-reviewed publications to date.
Also, important non peer-reviewed publications number in the hundreds and include intern and student research projects, technical reports, surveys for permitting
requirements, curriculum generation, reports for policy makers, news releases, outreach presentations, and articles, images, and award-winning video presentations for our website.
Spelling Matters
The Road to the Deep-Sea Guide
Overview ...
About MBARI
ROV Video
Video Annotations
The Road to the Deep-Sea Guide
Video Annotations
Normalizing Data
Spatial Context
Visualization
Custom Processing
Analysis is 'Hard'
4,048,817 Annotations on Feb 25th, 2013
Using 2663 concepts out of 4223 choices in the knowledgebase
SQL or Text Processing
Area
Time
Volume
Effort
...
Digital Elevation Model
Constrain by spatial bounds
Remove Duplicate Annotations
Create Custom Groupings
Calculate Percent Coverage
GIS
"Browsing" annotation data is difficult.
It requires a great deal of processing
but we can automate some of it!!
Motivations
Quality Control of Annotations
Identification Guide
MATLAB is the gateway drug to programming.
Java
C/Objective-C
Scala
Matlab
R
Excel
Ocean Data View
etc.
Funding and Resources
2010 - Proposed Internal MBARI Project
“Discussions with the Biodiversity Initiative researchers have indicated that the analyses of video annotations are critical first steps to establishing a
baseline for biodiversity research

Video Annotation Analysis and Presentation
"We are proposing to aggregate existing analysis from other projects and systematically apply them to the video annotations."
"In order to provide access to the data products, we are proposing to create a web based interface, dubbed the Deep-Sea Guide."
So what did that proposal get us?
2010
- 6 weeks of Software Engineer time + a summer intern

2011
- 6 weeks of Software Engineer time + a summer intern

2012
- A summer intern
Walk-through of an Analysis
Preparing the data
VARS Database
4+ million
Expedition Database
27 million Camera records
50 million CTD records
90 million Navigation records
Still more processing
Even more processing
Expedition Data is Merged with VARS annotations
CTD, Navigation and Camera data are
merged
nightly

'Smart' code:
Uses best available navigation
Checks for edited data if raw data was previously used
Example: Creating a Depth Histogram
Read Annotations
Filter for Duplicates
Tape Set:
Concept:
Link:
V1688
Gripoteuthis
identity-reference | self | 1
Filter by 'Zones'
Normalize by Effort
Create Data Artifacts
Register in DSG
database
A Zone is typically some spatial bounds

but we can use anything to filter a 'zone':
Altitude (i.e. only benthic annotations)
Time (start and end dates)
Actinopoda
Acantharea
Heliozoa
Radiolaria
Phaeodaria
Polycystina
Knowledgebase
Phaeoconchia
Phaeocystina
...
...
...
...
Coelodendridae
Concharidae
Astracanthidae
Aulacanthidae
...
Collodaria
Nassellaria
...
...
...
...
...
Astrostrobiidae
Cannobotryidae
...
Collozoida
V0001
T0001
D0001
...
V3668
T1163
D0332
List all Dives
Read Start and End Dates
T1163

2001-12-20 14:52:15Z
2001-12-20 21:56:15Z
Fetch best navigation data
CleanNavData
40,000,000+
RawNavData
46,000,000+
Estimate duration of Navigation
datum
1. Calculate dt between each
record
2. Toss out outlier values
3. Use mean(dt) as duration
Why do we Need a Deep-Sea Guide?
2 Files
x 4223 Concepts = 8446
x 3 Zones = 25,338
x 2 = 50,676
(normalized)
repeat for other data products
X
= A lot of files
It would be handy to 'Google' for data ....
... and see thumbnails of available data
Metadata would be nice too.
and maybe the data's parent data as well
18,000 hours of video = 750 days = 6,000 8 hour work days = 25 work years
(Assuming 240 work days per year)
A (very) Brief Architecture Overview
The processing needs to be very, very fast.
The usual tools, Matlab, Excel, R are waaaay too slow.
Data processing is run on the Java Virtual Machine
Written in Scala
Compact syntax, Easy to parallelize
Instead of writing this:
We can write this:
Did I mention it was fast?
A batch process runs 1 per week

Generates about 200,000 files in 4 hours
That's 14 files per second
Contributors
Engineer:
Brian Schlining

Video Lab:
Nancy Jacobsen Stout
Kyra Schlining
Susan von Thun
Linda Kuhnz
Lonny Lundsten
Scientists:
Charlie Paull
Steve Haddock

Interns:
Erica Curles
Hana Hashim
funded by The David and Lucile Packard Foundation
Tour of Deep-Sea Guide
MBARI video annotations are publicly
available at
http://www.mbari.org/vars/vars_query.html
The Deep-Sea Guide
Server side is also Java/Scala
Reuses VARS software libraries

Uses Industry Standard Technologies
Java Application Servers
JPA
JDBC
REST-ful URIs
Each ROV has: HD Camera + plenty of lights
See the full transcript