Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Wearable Device For Visually Impaired People

No description
by

Wafa Mannaei

on 15 August 2017

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Wearable Device For Visually Impaired People

A Highly Accurate and Reliable Data Fusion Framework for Guiding Visually Impaired
Contents
Introduction
Lose of sense decreases the independence and self worth!
Proposed Work
Prototype's Hardware
Demonstration
Testing and Results
Conclusion and Future Work
By
Wafa Elmannai
Under supervision
Professor. Khaled Elleithy
Committee Members
Prof. Elif Kongar
Prof. Miad Faezipour
Prof. Xingguo Xiong
Prof. Mohsen Guizani

Ph.D. Dissertation Prospectus Defense
Motivation and Research Problem
Percentage Distribution of Blind
Worldwide
Number (in thousand) of People Visually Impaired
per Million Total Population
The Most Popular Techniques
Problem
There is no single framework so far can be considered as a white cane replacement.
Objectives
An accurate data fusion algorithm that integrates both the computer vision algorithms and fused data from multiple sensors to aid the visually impaired people.
Providing a wider range of detection and performing indoor and outdoor with a simple utilization.
Proposing a fast responsive obstacle avoidance algorithm.
Navigation system.
State of The Art
Evaluation
Static/ Dynamic Obstacle Detection Algorithm
Using Computer Vision
Extraction of Interested Points Using
Oriented Fast and Rotated BRIEF (ORB)
3
4
5
6
7
8
9
10
12
11
13
14
15
16
17
19
20
ORB: an efficient alternative to SIFT or SURF in 2011.

ORB is a good choice in low-power devices.
21
Mathematical Model - Part 1
FAST Key point Orientation (FAST Feature Detection with directions)
Low threshold t to find more than N key points
Harris measure pick the top N points
Defines the moments of a patch
A Radio Frequency Identification Walking Stick (RFIWS)
A sidewalk
navigator was proposed by Mohammad Farid Saaid et al..., 2009.

A Radio Frequency Identification (RFID) is used to transfer and receive information.

Advantages:
RFID technology has a perfect reading function
between the tags and readers that makes the
device reliable in the level of detection.

Limitation:
Scope limitation
Expensive
Tags can easily distrusted from their function
Eye Substitution
Bharambe et al. developed an android application for VI people , 2013.

Used Sensors:

2 Ultrasonic Sensors, Vibrator sensors.

Advantages:
A low power consumption
Usages two sensors to overcome the issue of narrow cone angle.

Limitation:
Short detection range (Max 3m)
Discomfort design
limited use by only Android devices.
Expensive $1790 + software budget.
To find the centroid
The orientation of the patch is:
22
Mathematical Model - Part 2
Rotated BRIEF Descriptor
The BRIEF descriptor is a bit string description of an image patch constructed from a set of binary intensity tests.
r is the path radius, whereas x and y run between [-r, r].
“steer” BRIEF according to the orientation of keypoints.
For any feature set of n binary tests at location (x_i, y_i), define a 2 x n matrix, S which contains the coordinates of these pixels.
Using the patch orientation and the corresponding rotation
matrix R, we construct a “steered” version S of S:
24
23
Brute-Force Matcher
It takes the descriptor of one feature in first set and is matched with all other features in second set using some distance calculation. And the closest one is returned.
25
Demonstration of Brute Force Matcher
26
Random sample consensus (RANSAC)
Eliminate Outliners (more then %50) using the homographic matrix.
27
Demonstration of RANSAC
28
k- Means Clustering
It is the classification of an image into different groups
Initialize number of cluster k and centre C_k.
For each pixel , calculate the Euclidean distance d, between the center and each pixel of an image using the relation given below.
Assign all the pixels to the nearest centre based on distance d.
After all pixels have been assigned, recalculate new position of the centre
Repeat the process until it satisfies the tolerance or error value.
29
Demonstration of K-Means Clustering
Proposed Proximity Measurement Method
for Obstacle Avoidance
30
Smart Cane
Smart cane for navigation was deployed by Mohd Helmy Abd Wahab et al ... based on a fuzzy decision, 2011.

Used Sensors:
Ultrasonic sensors and Water detector.

Advantages:
Obstacle avoidance system for hearing
and visually impaired.

Limitation:
The water sensor can’t detect the water if it is less than 0.5 deep.
The buzzer won’t stop before it is dry.
Short detection range (Max 1.5m).
Only static object can be detected.
Krishna Kumar et al. developed Ultrasonic Cane as a navigation
aid, 2014.

Used Sensors:
3 Ultrasonic sensors

Advantages:
Wider range of view can be detected.

Limitation:
Small detection range (1.5m).
Static object detection only.
Navigation task is depend on the user.
Ultrasonic Cane
SUGAR Indoor System
Martinez-Sala designed an indoor navigation for visually impaired people, 2015.

Used Sensors:
Ultra-wide band Sensors

Advantages:
Huge detection range indoor only.
UWB technology offers robustness
because it does not need direct line of sight between tags and sensors.
location and orientation can be obtained by UWB signals.

Limitation:
Sensors would have to be deployed in every room.
The room has to be mapped beforehand.
User needs to select destination beforehand.
It is not suitable for outside use.
32
Proposed Obstacle Avoidance Algorithm
31
Obstacle Avoidance Region
33
.Net Gadgeteer Framework
Battery
We have used a .NET Gadgeteer compatible main board and modules from GHI Electronics using C# programing language.
Version: GHI NETMF 4.3 SDK
Visual Studio version: 2013

Prototype's Appearance
Gyro Module
GPS Module
Camera Module
Can stream images as large as 320 X 240.
The dimension is 47 X 27 X 28.55 mm
The weight 12 g
Lens Diameter is 14mm
FEZ Spider
The board supports all of the .NET Micro Framework core features and additionally supports other features such as USB host, WiFi and RLP (loading native code).
72MHz. 32-bit ARM7 processor
16 MB RAM
Full TCP/IP Stack with SSL, HTTP, TCP, UDP, DHCP
Ethernet, WiFi driver and PPP ( GPRS/ 3G modems) and
DPWS
Low power and hibernate support
Power Supply
It provides access to a USB client connector and also includes an advanced power supply, allowing the mainboard to be powered from USB or from the power jack.
Dimensions  42 x 37 x 15.2 mm
Weight 13 g
Voltage  7V to 30V

Wi-Fi RS21 Module
The GHI Electronics Wi-Fi RS21 Module can establish WiFi connections based on .NET Sockets.
Size: 42mm x 42mm
Weight: 7g

The Designed Device
Music Module
The Music Module includes an audio decoder capable of playing MP3, WMA, OGG, MIDI and WAV files.
Size: 50mm x 39mm
Weight: 12g
34
Power Consumption
35
The Functional Capability of Proposed Method
over the Scenarios
36
37
38
40
Example of Real Time Experiments
39
Video Data set
41
42
The Experiments' Results
43
41
45
Conclusion
We proposed and implemented an obstacle avoidance algorithm based on the image depth.

The approximately measurement algorithm allows us to approximately measure the distance between the user and the object based on the size and (x,y) coordination of that object in that particular frame.

Our hardware and software implementation provides a framework that facilitates the VI people’s mobility indoor and outdoor by integrating computer vision technologies and modules' data.
46
Remaining Work
Adding infrared and ultrasonic sensors to increase the accuracy of an objects detection algorithm.
Using the GPS, Compass and Gyro data to provide a navigation system.
Creating an app that analyze the location of the user and allow the user to contact his/her relative in any emergency.
Vibration motors can be added for visually and hearing impaired.
47
Publications and Grants
Journal:
Elmannai, W.; Elleithy, K. Sensor-Based Assistive Devices for Visually-Impaired People: Current Status, Challenges, and Future Directions. Sensors 2017, 17, 565.

[Impact Factor: 2.033 ]


Posters:
Wafa Elmannai and Khaked Elleithy, "A Framework for Guiding Visually Impaired Based on Data Fusion", The 2017 ASEE-NE Annual Conference will be held at the University from Massachusetts Lowell from April 27 to April 29, 2017.
Wafa Elmannai and Khaked Elleithy, "An Accurate Data Fusion System for Visually Impaired", Connecticut Microelectronics and Optoelectronic Symposium Program (CMOS), April 5, 2017.
[Best Poster Prize]
Wafa Elmannai and Khaked Elleithy, "A Highly Accurate and Reliable Data Fusion Framework for Guiding Visually Impaired", Faculty Research Day, University of Bridgeport, March 12, 2017.
Wafa Elmannai, Khaled Elleithy and Yassir Ellrithy, An Extensible, Wearable, Assistive Device for the Visually Impaired Indoor and Outdoor, the Annual IEEE Connecticut Conference on Industrial Electronics, Technology & Automation (CT-IETA 2016), Bridgeport, CT, October 14-15, 2016.

Grants:
$20,000 AAUW grant from the American Association of University Women for 2015-2016.
The Results of the Decision Table For an Obstacle Avoidance Algorithm
Comparison
Ref: 1. World Health Organization. Visual Impairment and Blindness. Available online: http://www.Awho.int/mediacentre/factsheets/fs282/en/ (Accessed on January 2016).
2. American Foundation for the Blind. Available online: http://www.afb.org/ (Accessed on January 2016).
Ref: Bharambe, S.; Thakker, R.; Patil, H.; Bhurchandi, K.M. Substitute Eyes for Blind with Navigator Using Android. The India Educators Conference (TIIEC), Bangalore, India, 4–6 April 2013; pp. 38–43.
Ref: Saaid, M.F.; Ismail, I.; Noor, M.Z.H. Radio frequency identification walking stick (RFIWS): A device for the blind. 5th International Colloquium on Signal Processing & Its Applications, Kuala Lumpur, Malaysia, 6–8 March 2009.
Ref: Wahab, A.; Helmy, M.; Talib, A.A.; Kadir, H.A.; Johari, A.; Noraziah, A.; Sidek, R.M.; Mutalib, A.A. Smart Cane: Assistive Cane for Visually-impaired People. IJCSI International Journal of Computer Science Issues, Vol. 8, Issue 4, No 2, July 2011.
Ref: Kumar, K.; Champaty, B.; Uvanesh, K.; Chachan, R.; Pal, K. and Anis, A. Development of an ultrasonic cane as a navigation aid for the blind people. The 2014 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT), Kanyakumari District, India, 10–11 July 2014.
Ref: Martinez-Sala, A.S.; Losilla, F.; Sánchez-Aarnoutse, J.C. and García-Haro, J. Design, implementation and evaluation of an indoor navigation system for visually-impaired people. Sensors 2015, 15, 32168–32187.
Ref: Elmannai, W.; Elleithy, K. Sensor-Based Assistive Devices for Visually-Impaired People: Current Status, Challenges, and Future Directions. Sensors 2017, 17, 565. [Impact Factor: 2.033 ]
Architecture of The System
18

Ref: Rublee, Ethan, et al. "ORB: An efficient alternative to SIFT or SURF." Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 2011.
Ref: Brute Force Matcher. http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_matcher/py_matcher.html, [access Jan 2016].
Ref: Barclay, Adam, and Hannes Kaufmann. "FT-RANSAC: Towards robust multi-modal homography estimation." Pattern recognition in remote sensing (PRRS), 2014 8th IAPR workshop on. IEEE, 2014.
Ref: Kanungo, Tapas, et al. "An efficient k-means clustering algorithm: Analysis and implementation." IEEE transactions on pattern analysis and machine intelligence 24.7 (2002): 881-892.
44
• Introduction
o Motivation
o Research Problem
o The Most Popular Techniques
o Objectives
• State of The Art
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Algorithm for an Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents
• Introduction
• State of The Art
o Existing Techniques
o Evaluation
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents

Introduction
• State of The Art
• Proposed Work
o Data Fusion Algorithm
o Architecture of The System
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents

Introduction
• State of The Art
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents

Introduction
• State of The Art
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
o Obstacle Avoidance Region
o Proposed Obstacle Avoidance Algorithm
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents

Introduction
• State of The Art
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents

Introduction
• State of The Art
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents

Introduction
• State of The Art
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
o Example of Real Time Experiments
o The Functional Capability of Proposed Method over the Scenarios
o Video Data set
o The Results of the Decision Table For an Obstacle Avoidance Algorithm
o The Experiments' Results
o Comparison
• Conclusion and Remaining Work
• Publications
Contents

Introduction

• State of The Art
• Proposed Work
• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Contents

Introduction
• State of The Art
• Proposed Work

• Static/ Dynamic Obstacle Detection Algorithm Using Computer Vision
• Proposed Proximity Measurement Method for Obstacle Avoidance
• Prototype's Hardware
• Demonstration
• Testing and Results
• Conclusion and Remaining Work
• Publications
Assistive Technologies for Disable People
Data Fusion Algorithm

Demonstration of ORB
Table1. The most important features that correspond to the user’s needs
2
1
Full transcript