Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Upper Body Motion Analysis Using Kinect for Stroke Rehabilitation at the Home

No description
by

Tingfang Du

on 9 July 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Upper Body Motion Analysis Using Kinect for Stroke Rehabilitation at the Home

M.S. Thesis Defense
Tingfang Du
November 21, 2012
School of Arts, Media and Engineering
Arizona State University
Human Motion Analysis
Home-based Adaptive Mixed Reality Stroke Rehabilitation System
Multimodal Sensing Module
Capture the human movements by tracking a variety of body joints using motion capture devices.
Upper Body Tracking
Depth-fused mean shift tracking approach
Meanshift tracking
Kinematic evaluation of torso movement quality
Video Surveillance
Animation
Medicine
Low-cost rehabilitation system provides engaging, purposeful feedback to help patients self-assess performance on physical tasks and improve motor function in long-term at home therapy.
System Architecture
Components
- Motion Capture ( Opti-Track, Kinect, FSRs, IMU )
- Motion Analysis ( kinematic features )

- Multimedia Feedback ( audio, visual, tangible )
- Adaptation

Objective
Hospital System
Home System
Hospital System
Home System
6-camera motion capture system with
12 optical markers on arm, hand, torso
inertial measure unit (IMU) on
Back board
Kinematic Representation
Devices
Finger
Endpoint (Wrist)
Elbow
Shoulder
Torso
Devices
4-camera Opti-Track system
1 optical marker on endpoint (wrist)
6 FSRs sensors on chair
Kinematic Representation
Endpoint (Wrist)
Torso
Looking for Improvement
Expensive
Noisy data from chair sensors
- unreliable torso data
Kinect
Torso Kinect v.s. Chair Sensors?
Skeletal Body Tracking
Depth Image
Body parts
3D joint proposals
Estimate centroids of body parts by find local modes in feature space using Meanshift algorithm.
Map depth value into grayscale value.
A classifier trained based on random decision forests
1 million samples in training set.
categorize the body regions into different body parts.
No elbow and shoulder data
Torso
Represent torso movement using three joints --
Left shoulder, right shoulder and spine (torso).
Skeleton tracking
Skeleton identification
Upper Body Tracking
Endpoint
Skeleton tracking algorithm works poorly.
Low accuracy
Hand joint position varies from different hand poses.
Not robust
mean shift procedure
Kalman Filter
Find local maxima of Bhattacharyya coefficient, a similarity measurement.
Torso Compensation
Leaning forward or backward
twisting toward or away from the target
Difficult to measure
different targets - distance, direction
different compensation strategies
System Structure
Leaning angle
Compensation : Leaning
Compensation : Twisting
Compensation at initial
Compensation at the target
Pronaunce leaning and moderate twisting
......
Twisting angle
Endpoint trajectories
Feature Extraction
Classifiers
Feedback
Kinematic evaluation for torso movement quality
Feature Extraction
Real-time based features
Trial based features
leaning angle
twisting angle
max leaning angle
standard deviation of leaning angle
max offset X
......
Normal
Mean Leaning angle
Mean Twisting angle
Max Leaning angle difference
Max twisting angle difference
Standard deviation of leaning angle
Standard deviation of twisting angle
Max OffsetX
Max OffsetZ
Real-time
train reference for each target
use threshold based classifier
Post trial
Feature Extraction -- calculations
leaning angle and twisting angle are euler angles of rotation matrix between current torso coordinates and rest torso coordinates
mean leaning and twisting angle:
use off-the-shelf classification technology
e.g. SVM, KNN, NaiveBayes
Classification
System Setup
Physical
Platform and Software
Mac OSX -- Dash

Windows -- Tracking tools and Kinect program

Parallels

OpenNI, NITE
Endpoint Tracking Errors
Classification Results
Conclusions and Future work
Classification Results
Endpoint Kinect v.s. Opti-Track ?
A promising solution for HAMRR
Elbow
Kinect ?
Upper body Tracking
Robust torso tracking
Promising end-point tracking

Future
multi Kinect system
gesture recognition

Input
Segmentation
Tracking
Conclusions and Future work
Thank you!
Overview

Upper-body tracking using Kinect
Analysis of torso compensatory movement
Experiment results

Stroke Rehabilitation
virtual reality
Each year, 795,000 people in the U.S. suffer from stroke.
60% of them experience upper limb disability.

traditional
expensive facilities
System Structure:
Stroke Rehab System
traditional physical tasks
limited availability
affordable facilities
accessible, at home
new forms of tasks
HAMRR
3D vision
Real time
Inexpensive
easy to set up
Skeletal body tracking
User Friendly
References
[3] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake. Real-time human pose recognition in parts from single depth images. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1297–1304, Washington, DC, USA, 2011. IEEE Com- puter Society.
[2] G. G. Saposnik. Virtual reality in stroke rehabilitation: A meta-analysis and implications for clinicians. Stroke (1970), 42(5):1380–1386, -05 2011.
[1] P. Meer. Kernel-based object tracking. IEEE Transactions on pattern analysis and machine intelligence, 25(5), 2003
[4] Yinpeng Chen. Constraint-aware computational adaptation framework to support realtime multimedia applications. PhD thesis, Arizona State University, Tempe, AZ, USA, 2009.
Upper Body Tracking
Torso
Localization
(R, G, B) --> (H, S, V)
Kalman filter predict
Kalman filter correct
Torso
Shoulders
Endpoints
Elbows
......
Background
Overview
Analysis of torso movement during physical rehab tasks
Experiment and Results

Background
Upper-body tracking using Kinect
Overview
Experiment and Results
Background
Upper-body tracking using Kinect
Analysis of torso movement during physical rehab tasks
Overview
Experiment and Results
Background
Upper-body tracking using Kinect
Analysis of torso movement during physical rehab tasks
Target Representation
Real-time feedback
post-trial feedback
post-set feedback
Multi-layer feedback
intuitive, immediate
comprehensive, reflective
Full transcript