Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


A Taste of Vision


Amy Nau

on 8 November 2012

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of A Taste of Vision

A Taste of Vision Right now we can take the blind from no light perception
to a state of “ultra low vision”

This forms a shadowy, pixellated world without color, texture, detail or depth                       Cortical Implants
Retinal Implants
Sensory Substitution Devices
Eye Regeneration (future)
Eye Transplantation (future) “Ultra Low Vision” Hardware
Input device/camera
Power supplies, RF (wireless)

Converts video into processed image

How the device communicates with the brain Artificial Vision Devices       . What if you don’t have eyes? Laboratory of Neural Prosthesis at Illinois Institute Of Technology (IIT), Chicago, is developing a visual prosthetic using Intracortical Iridium Oxide (AIROF) electrodes arrays.

Dr. Mohamad Sawan, Professor and Researcher at Polystim neurotechnologies Laboratory at the Ecole Polytechnique de Montreal Dobelle- cortical implant (64 electrodes)- Human implantation Cortical Approaches Phosphenes/outlines

Unstable spatial resolution

Percepts that do not match reality

Can induce seizures

Surgical risks


Limited use (hours per day)

Use of an existing, intact sense to relay afferent sensory information about a
different sense to the brain

Cross modal plasticity refers to the adaptive reorganization of neurons to integrate the functions of two or more sensory systems Sensory Substitution Visual Cortex BrainPort VoICE AuxDeco Sensory Substitution Options peristriate occipital visual areas activated in :
early , late blind & sighted subjects seeing with sound.com vertical location =frequency,
horizontal location =stereo panning,
brightness =loudness Targeting an occipital cortex site with rTMS led to a impairment in interpretation Subretinal -- no camera needed;
100’s of microelectrodes between RPE
and Outer retina Epiretinal implanted on to the inner retina
camera coupled to ganglion cells http://www.anatomy.unimelb.edu.au/researchlabs/rees/images/retina.jpg

http://singularityhub.com/2009/10/12/mits-retinal-implant-is-moving-forward-but-hasnt-caught-up-with-argus-ii/ Momentous engineering effort

Arrays are improving

Surgical techniques improving

Optic nerve needs to be healthy

Some functioning retina

Gross forms and shadows are
now possible Retinal Implant
Summary Amy Nau, O.D., F.A.A.O
Assistant Professor
UPMC Eye Center/McGowan Institute
Director, Optometric Services
Director, Sensory Substitution Lab
University of Pittsburgh Medical Center
Fox Center for Vision Restoration Using a modified tangent screen, (6’x6’) ; 50mm target, the BrainPort provides
60 degrees of space recognition in the horizontal meridian
57 degrees of space recognition in the vertical meridian Visual Field Results Obstacle
Course Obstacle Course- Object Detection DMDRP / DOD study
Hardware upgrades
Software upgrades
Collaborate with RI at CMU
Facial recognition
Moblity enhancement
20 subjects FDA safety study
Is the device safe?
Explore additional functional
70 subjects @ 8 sites,
18 subjects @ UPMC Veteran’s Study
First study with at home use
Device feedback
10 subjects Outcomes Study
Design and validate clinical tests for artificial vision devices
Assess neuroplastic changes (PET/MRI)
30 subjects BrainPort Studies 2009-present *outcomes study only
** DMDRP study only EMR
Flash VEP
Object recognition
Word recognition
Mobility tasks
MRI (DTI/tractography)*
Visual field
Depression Screen
QoL (VFQ-25 and AI)
Obstacle Course Assessments We Measure Baseline
assessments of visual and mobility function
Structured training --20 hours
Repeat assessments of visual and mobility function
+/- send patients home for X duration
Return to repeat assessments of visual and mobility function Study Design we spent 12 months figuring out which tests would work,
which would not and how to administer the tests...........
Psychophysical tests have to be standardized
Zoom set to encompass entire computer screen (approx 35 degree FOV)
½ meter testing distance
Subjects allowed to control intensity
Tasks had 3 minute time limit
In many cases, the camera is on a tripod It is important to control the experimental conditions……. Subjects can identify objects with practice but words are harder Eye Movement Recording n=3 subjects
Grating stimulus
A video-based eye tracking system (Eyelink 1000) was used, 500 Hz sampling frequency. Nystagmus Reduction VEP results Neuroimaging protocol Analysis of Fiber Architecture in Brain Port Training using Fractional Anisotropy and PET based ROI’s View object with BrainPort only
Sense object with hands only
Interpret object with hands and BrainPort
FDG18 PET scan study Baseline structural MRI and DTI scan
Baseline PET scan (using BrainPort, no training)
Training 15-20h
Repeat PET scan (using BrainPort)

Differences in architecture?
Differences in function?
Subgroups according to duration of blindness
(n=6 blindfolded controls and 24 blind)

regressors : lifetime fraction; onset rate; age at scan; pre/post training

In this case we included family-wise error correction for multiple comparisons (0.05) and required 50 voxel cluster, p=.001. Overview Projects Selected
Results To Be Done... VoICE Diffusion tensor imaging (DTI) of microstructural integrity in congenitally blind, acquired blind and control adult brains. (Left) fractional anisotropy (FA) maps of congenitally blind and control brains. Red pixels indicate regions of interest (ROIs) with significantly different activations among the 3 groups after BrainPort stimulation in functional PET. (Right) Histogram of the distributions of FA within the red ROIs selected. The congenitally blind appeared to possess fewer structures with high FA and more structures with low FA compared to the acquired blind and control. Psychophysics Facilitates in-home training with a sighted caregiver BrainPort Companion Device Over 2,000 hours of training
Not enough LVOT domestically or internationally
Device abandonment, boredom, complacency, lack of follow up are all barriers to success
Common to all low vision devices Rehabilitation and Training
Lessons Learned Alternative Cameras Investigate the use of time-of-flight sensors and IR to recognize and interpret the environment. Averaged 66% correct estimating self distance to box

Detect ed the presence of the box from an average 18.75 feet

Two box discrimination distance averaged 9 inches

Estimated boxes placed at random distances within 1.75 feet error. Depth Perception Standing tasks may be predictive of navigation abilities Detectedseparation between cylinders 2.75 inches minimal distance

Determined spatial relationship between shapes with 79% accuracy

Estimated distance between cylinders with at least 0.5 inches distance

Determined the spatial relationship between overlapping shapes with 83% accuracy. Using a device with a single camera, can you perceive spatial relationships? Depth Perception with AV
-Computer vision algorithms are used to recognize and render signage information to assist in safe navigation
The challenge in addressing this problem is the variation in lighting and view Face Differencing features that distinguish any face from an “average” face.

Highlight areas of the new face that differ from the “average” face by more than a certain threshold. Making cameras “smart”
Might enhance functionality
Recapitulating sight is extremely difficult

Allow software to provide heavy computational lifting
Allow the human to make the decisions 80/20 rule Incorporating What Exists
Machine Vision The Versatile and Integrated System for Telerehabilitation (VISYTER)
-VISYTER is a secure, integrated system that combines high-quality videoconferencing with access to electronic health records

-Stimuli presentation, remote multiple camera control, remote control of the display screen, and an eye contact teleprompter.

-The software platform is suitable for supporting low-volume services to homes, yet scalable to support high-volume enterprise-wide telehealth services.

-It represents a capacity building strategy for long term rehabilitation and in-home assessments. Telerehabilitation
Requires that each SSD device be shown, in advance, a fixed set of faces that it needs to recognize - Have the software do the recognition
- Render an easily-discernable code to the BrainPort user. Alternative Approaches to facial/object recognition Word recognition
Object detection
Signage detection
Seated and standing depth perception
Tangent Screen
Neuroimaging (PET/ DTI/MRI Outcomes Assessments What is the best way to convey this information on a limited array? Generating GPS based maps on the electrode array fMRI Adult -v - Older Adult
Cortical Blindness
Two SSD in one individual
Pediatric populations Chrissie Pintar
Christopher Fisher
Jacki Fisher
Dongsheng Yang
Rich Hertle
Valeria Fu
Aimee Arnoldussen
Rich Hogle
Kevin Chan
Charles Laymon
Vincent Lee
Fernando Boada
Yaser Sheikh
Amy Rebovich
Ken Wojznik
Mark Kislan
Deborah Fenton
Pam Howe
Melissa Lowalkowski
Judith Shanahan
Barry Fell
Wendy Chen
Tobin Vijayin
Jenna Sembrat
Julie Steinbrink
Alex Keifer
Cody Wolfe
Kathleen Janesco
Justin Kaiser
Myles Nightingale Acknowlegements Fine Foundation
DCED State of PA
Louis J. Fox Center for Vision Restoration
Lion’s Club
OTERO Fellowship
University of Pittsburgh Aging Institute ARGUS II (60 electrodes)
object localization 96%
motion discrimination 57%
grating orientation 23%
Best VA 20/1260
70% no SAE Humayun et al. Interim Results from the International Trial of Second Sight's Visual Prosthesis. Ophthalmology Vol 119(4), 779-788, April 2012 Corticl
Approach The image shows the anatomical ROI of optic radiations (in pink)
overlaid on MNI template.
The voxels in blue are areas show significant differences amongst all three groups a non-parametric voxel wise multiple comparison (Kruskal-Wallis test).

Wilke R, et al. Spatial resolution and perception of patterns mediated by a subretinal 16-electrode array in patients blinded by hereditary retinal dystrophies.Invest Ophthalmol Vis Sci. 2011 Jul 29;52(8):5995-6003. Print 2011 Jul. -Visual percepts were elicited reliably in 8 of 11 patients.

-On single-electrode activation, percepts were generally described as round spots of light of distinguishable localization in the visual field.

-On activation of a pattern of electrodes, percepts matched that pattern when electrodes were activated sequentially.

-Patterns such as horizontal or vertical bars were identified reliably

-One participant was able to recognize simplified letters presented on the 16-electrode array.
-Estimated logMAR of 1.78. Subretinal electric stimulation (Tubingen) The lateral-occipital tactile-visual area (LOtv) is activated when objects are recognized by vision or touch. LOtv is also activated in sighted and blind humans extracting shape information from visual-to-auditory sensory substitution soundscapes. A. Amedi, W. Stern, J. A. Camprodon, F. Bermpohl, L. Merabet, S. Rotman, C. Hemond, P. Meijer and A. Pascual-Leone, ``Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex,'' Nature Neuroscience, Vol. 10, No. 6, pp. 687 - 689, June 2007 TELEREHABILITATION
Full transcript