### Present Remotely

Send the link below via email or IM

CopyPresent to your audience

Start remote presentation- Invited audience members
**will follow you**as you navigate and present - People invited to a presentation
**do not need a Prezi account** - This link expires
**10 minutes**after you close the presentation - A maximum of
**30 users**can follow your presentation - Learn more about this feature in our knowledge base article

# FINAL FYP12013 Kinect in Physiotherapy

No description

#### Transcript of FINAL FYP12013 Kinect in Physiotherapy

Physiotherapy Researching Console using Kinect (PRCK) in Physiotherapy FYP12013 Philip Chan

Ricky Cheng

Charlene Yu

Kenneth Leung Dr. C. K. Chui

Dr. W. S. Ho Perf rms exercise Training of patients (users) with mobility difficulties 3D Depth Sensors RGB Camera Actions of users are captured and measured Recorded data and user information can be accessed Capture and Measure Computation & Analysis Further analysis can be carried out to assist physiotherapists in examining user performance Accurate measurement of user performance and recording is needed Facilitates daily training and monitors progress of each individual Automate the collection of raw data from walking No human input Automate the collection of raw data from walking Compute and analyze the collected data to produce relevant results Automate the collection of raw data from walking Compute and analyze the collected data to produce relevant results Provide a user interface for accessing analyzed data We've created a system named... Database Design step length measured coordinates of joints time spent etc... Data are gathered automatically by motion-sensing tool distance walked step length etc... Provide a user interface for accessing the analyzed data total distance average step length total time speed Provide a user interface for accessing the analyzed data Name

Gender

DOB

HKID

Health Status

... Stroke Patients at Royal Berkshire Benefit from Playing Kinect Related Works Opect in Tokyo Women's Medical University in Japan Tedesys Uses Kinect in Hospitals Problem Setting Overview Objective Automate the collection of raw data from walking, e.g. coordinates of joints, time spent etc. Compute and analyze the collected data to produce relevant results, e.g. distance walked, step length etc. Provide a user interface for accessing analyzed data Carried out test in real life environment Project Description Walking is a common practice physiotherapists use to monitor alderfly’s body function Depth sensor range of Kinect is from 800mm to 4000mm Effective sensible depth ranged from about 1.5m to 4m by our measurement Kinect's Performance under this setting: Modeling Kinect tracks skeleton data which consists of 20 joints Coordinate system for these data is a full 3D system Data extraction tests were conducted to clarify the data collection rate Result: around 1700-1800 milliseconds to record 100 datasets Problem Formulation replay Produce useful information by computation and analysis detects 20 joints coordinates Replay Depth Frame Window Control Panel (self-implemented) Distance from sensor

(self-implemented) distance between feet D (cm) Sum {Step length} = Total distance Speed = Distance / Time Detects and analyzes motion Color Frame Window User Interface Implementation Implementation Case 1: Normal Case 3: Simulated Parkinson's Disease 6 out of 7 of the items concern about angles and percentage of time

Generates a suggested GARSM score automatically GARSM Analysis Engine Definition “degree of loss of hip range of motion (ROM) seen during a gait cycle”

Angle between the thigh and the vertical during double support should not be too small (>10 degrees)

Double support – when both legs are in contact with the ground during a walking cycle Hip ROM steadier walking pace greater amplitude Easy Comparison Reads the same data as real-time analyzer

Data are read from the database; not captured directly Offline Analyzer Arm-heel-strike Synchrony Modelling Assess whether the arm and its contralateral leg are in phase

3 methods:

Comparing the sign of slope

Modelling

Normalization Arm-heel-strike Synchrony Definition “the extent to which the contralateral movements of an arm and leg are out of phase”

Arm and its contralateral leg should be in phase at all times during walking Scoring scale used to assess the risk of recurrent falls in elderly

7 items each in a 3-point scale Coordinates

Selection Menu Graph Engine Method 3 – Normalization Arm-heel-strike Synchrony Normalize the actual curve to make it look like a typical sine curve

Phase difference vs. area between two curves Phase difference: calculate area between two curves

Area between two curves at minimum when in phase, maximum when out of phase Method 2 – Modelling Arm-heel-strike Synchrony Method 2 – Modelling Model a typical sine curve according to the actual data collected Arm-heel-strike Synchrony Arm-heel-strike Synchrony Method 1 – Comparing the sign of slope Most straight-forward method

Two curves in phase: go up and go down at the same time

When curve is increasing, the value of the slope will be positive

Conclusion: sigh of slopes are the same at any given time (in phase) 20 joints in total Implementation Application P(H|X)

Probability that hypothesis H holds given the observed data sample X

Calculating P(X|H) is complex Bayes’ Theorem Relies on probability

Calculate the explicit probabilities for hypothesis

Practical approaches and standard method Bayesian Classification Model Construction

Model Classification A technique in data mining

Predict group membership for data instances

Based on model data set and the values in

Classifying attribute Classification Organize data in systematic and scientific way

Hidden Information

Prediction

Provide reference to physiotherapist Potential Abilities Potential in further data analysis

Hidden information can be found

Bayesian Classification: one kind of classification model only

Can be replaced by any prediction model

Other exercise and analysis in the same approach Summary Combining Result of Step 1 and Step 2

By applying Bayes’ thoery: P (Ci|x) = P (X| Ci) * P (Ci)

The highest probability among the classes

the unknown data should fall in that class Steps Steps Class Label : Having Parkinson’s Disease

System Data Feature Vectors

Age Range

Gender

Stroke History

Degree of resting tremor

Degree of rigidity

Degree of Bradykinesia

Degree of postural instability

Goal: Return the whether the newly inserted unknown record is a Potential Parkinson Patient Classifier Model Assumption: Class Independence

Values of attributes are conditionally independent of one another

Makes the calculation much simpler Naïve Bayesian Classification One Vector = One tuple

One feature = One dimension Data Representation Model Classification Model Construction … P (Xj|Ci)

Xj is the criteria of the feature (for j = 1- 7)

Ci is the two class labels.

E.g. P(age group <= 20| having Parkinson’s disease) Steps Intelligence System Analysis System Reporting System Our Goals Automate the collection of raw data from walking, e.g. coordinates of joints, time spent etc.

Compute and analyze the collected data to produce relevant results, e.g. distance walked, step length etc.

Provide a user interface for accessing analyzed data. Classification Classification Two steps Feature Vector Bayes’ Theorem In early-April 2013, the University of South California announced their research result on “detecting depression from facial actions using Kinect”

Making use of the facial recognition technology of Kinect

Around 90% accuracy Prologue – Successful Example in World Study of Parkinson’s Disease using Kinect Motion Sensing Application Measuring Methods Possible Classification Model Demonstration of Parkinson’s Disease Gait Four Cardinal Symptoms of Parkinson’s Disease Recap and Review Resting Tremor

Counting number of cycles

Rigidity

Included angle between the thigh bone and the shinbone Measuring Methods * Jankovic J. Parkinson’s disease: clinical features and diagnosis. J Neurol Neurosurg Psychiatry. 2008;79:368-376. Overview of the Classification Model * Jankovic J. Parkinson’s disease: clinical features and diagnosis. J Neurol Neurosurg Psychiatry. 2008;79:368-376. According to Prof. Jankovic’s journal*, symptoms of Parkinson’s Disease can be categorized into two main categories

Motor symptoms

Non-motor symptoms Understanding Parkinson’s Disease Future Development SimSensei Step 1 P(X1|C1) P(X|C1) P(X|C2) P(X7|C1) P(X1|C2) P(X7|C2) … Step 2 Compute the Probability of P(Ci)

e.g. P (having Parkinson’s disease) Final Step Method 1 - Comparing the sign of slope Method 2 - Modelling Method 2 - Modelling Method 2 - Modelling Method 3 - Normalization Implementation Method 3 - Normalization Method 3 - Normalization Method 3 - Normalization Observations Observations Model Implementation Hip ROM Modelling Calculate the angle between the leg and the vertical:

Applied at the point where the y-coordinates of ankle joint of both legs intersect: Double support Development Environment Language: C#, MySQL, XAML Software Tools: Kinect SDK v1.6, Kinect Developer Toolkit v1.6, Visual Studio 2010, MySQL Connector Net 6.2.5, GitHub ...With the aid of motion sensing system Observations Case 2: Simulated same arm same heel Observations Case 4: Simulated elderly Experimental Setup Kinect

sensor Target Resting Tremor

Why Resting Tremor? Rigidity

Bradykinesia (Slowness of movement)

Postural Instability

Data Transformation & Relevant Analysis

Data Cleaning: take out data with error GARSM How to solve? Value

Differentiation

Integration Sign of Conclusion Method 1 is trivial but gives reasonable results

Method 2 is too specialize to normal walking pattern Use more data to build Build differentiated models

Method 3 works most as expected

Full transcriptRicky Cheng

Charlene Yu

Kenneth Leung Dr. C. K. Chui

Dr. W. S. Ho Perf rms exercise Training of patients (users) with mobility difficulties 3D Depth Sensors RGB Camera Actions of users are captured and measured Recorded data and user information can be accessed Capture and Measure Computation & Analysis Further analysis can be carried out to assist physiotherapists in examining user performance Accurate measurement of user performance and recording is needed Facilitates daily training and monitors progress of each individual Automate the collection of raw data from walking No human input Automate the collection of raw data from walking Compute and analyze the collected data to produce relevant results Automate the collection of raw data from walking Compute and analyze the collected data to produce relevant results Provide a user interface for accessing analyzed data We've created a system named... Database Design step length measured coordinates of joints time spent etc... Data are gathered automatically by motion-sensing tool distance walked step length etc... Provide a user interface for accessing the analyzed data total distance average step length total time speed Provide a user interface for accessing the analyzed data Name

Gender

DOB

HKID

Health Status

... Stroke Patients at Royal Berkshire Benefit from Playing Kinect Related Works Opect in Tokyo Women's Medical University in Japan Tedesys Uses Kinect in Hospitals Problem Setting Overview Objective Automate the collection of raw data from walking, e.g. coordinates of joints, time spent etc. Compute and analyze the collected data to produce relevant results, e.g. distance walked, step length etc. Provide a user interface for accessing analyzed data Carried out test in real life environment Project Description Walking is a common practice physiotherapists use to monitor alderfly’s body function Depth sensor range of Kinect is from 800mm to 4000mm Effective sensible depth ranged from about 1.5m to 4m by our measurement Kinect's Performance under this setting: Modeling Kinect tracks skeleton data which consists of 20 joints Coordinate system for these data is a full 3D system Data extraction tests were conducted to clarify the data collection rate Result: around 1700-1800 milliseconds to record 100 datasets Problem Formulation replay Produce useful information by computation and analysis detects 20 joints coordinates Replay Depth Frame Window Control Panel (self-implemented) Distance from sensor

(self-implemented) distance between feet D (cm) Sum {Step length} = Total distance Speed = Distance / Time Detects and analyzes motion Color Frame Window User Interface Implementation Implementation Case 1: Normal Case 3: Simulated Parkinson's Disease 6 out of 7 of the items concern about angles and percentage of time

Generates a suggested GARSM score automatically GARSM Analysis Engine Definition “degree of loss of hip range of motion (ROM) seen during a gait cycle”

Angle between the thigh and the vertical during double support should not be too small (>10 degrees)

Double support – when both legs are in contact with the ground during a walking cycle Hip ROM steadier walking pace greater amplitude Easy Comparison Reads the same data as real-time analyzer

Data are read from the database; not captured directly Offline Analyzer Arm-heel-strike Synchrony Modelling Assess whether the arm and its contralateral leg are in phase

3 methods:

Comparing the sign of slope

Modelling

Normalization Arm-heel-strike Synchrony Definition “the extent to which the contralateral movements of an arm and leg are out of phase”

Arm and its contralateral leg should be in phase at all times during walking Scoring scale used to assess the risk of recurrent falls in elderly

7 items each in a 3-point scale Coordinates

Selection Menu Graph Engine Method 3 – Normalization Arm-heel-strike Synchrony Normalize the actual curve to make it look like a typical sine curve

Phase difference vs. area between two curves Phase difference: calculate area between two curves

Area between two curves at minimum when in phase, maximum when out of phase Method 2 – Modelling Arm-heel-strike Synchrony Method 2 – Modelling Model a typical sine curve according to the actual data collected Arm-heel-strike Synchrony Arm-heel-strike Synchrony Method 1 – Comparing the sign of slope Most straight-forward method

Two curves in phase: go up and go down at the same time

When curve is increasing, the value of the slope will be positive

Conclusion: sigh of slopes are the same at any given time (in phase) 20 joints in total Implementation Application P(H|X)

Probability that hypothesis H holds given the observed data sample X

Calculating P(X|H) is complex Bayes’ Theorem Relies on probability

Calculate the explicit probabilities for hypothesis

Practical approaches and standard method Bayesian Classification Model Construction

Model Classification A technique in data mining

Predict group membership for data instances

Based on model data set and the values in

Classifying attribute Classification Organize data in systematic and scientific way

Hidden Information

Prediction

Provide reference to physiotherapist Potential Abilities Potential in further data analysis

Hidden information can be found

Bayesian Classification: one kind of classification model only

Can be replaced by any prediction model

Other exercise and analysis in the same approach Summary Combining Result of Step 1 and Step 2

By applying Bayes’ thoery: P (Ci|x) = P (X| Ci) * P (Ci)

The highest probability among the classes

the unknown data should fall in that class Steps Steps Class Label : Having Parkinson’s Disease

System Data Feature Vectors

Age Range

Gender

Stroke History

Degree of resting tremor

Degree of rigidity

Degree of Bradykinesia

Degree of postural instability

Goal: Return the whether the newly inserted unknown record is a Potential Parkinson Patient Classifier Model Assumption: Class Independence

Values of attributes are conditionally independent of one another

Makes the calculation much simpler Naïve Bayesian Classification One Vector = One tuple

One feature = One dimension Data Representation Model Classification Model Construction … P (Xj|Ci)

Xj is the criteria of the feature (for j = 1- 7)

Ci is the two class labels.

E.g. P(age group <= 20| having Parkinson’s disease) Steps Intelligence System Analysis System Reporting System Our Goals Automate the collection of raw data from walking, e.g. coordinates of joints, time spent etc.

Compute and analyze the collected data to produce relevant results, e.g. distance walked, step length etc.

Provide a user interface for accessing analyzed data. Classification Classification Two steps Feature Vector Bayes’ Theorem In early-April 2013, the University of South California announced their research result on “detecting depression from facial actions using Kinect”

Making use of the facial recognition technology of Kinect

Around 90% accuracy Prologue – Successful Example in World Study of Parkinson’s Disease using Kinect Motion Sensing Application Measuring Methods Possible Classification Model Demonstration of Parkinson’s Disease Gait Four Cardinal Symptoms of Parkinson’s Disease Recap and Review Resting Tremor

Counting number of cycles

Rigidity

Included angle between the thigh bone and the shinbone Measuring Methods * Jankovic J. Parkinson’s disease: clinical features and diagnosis. J Neurol Neurosurg Psychiatry. 2008;79:368-376. Overview of the Classification Model * Jankovic J. Parkinson’s disease: clinical features and diagnosis. J Neurol Neurosurg Psychiatry. 2008;79:368-376. According to Prof. Jankovic’s journal*, symptoms of Parkinson’s Disease can be categorized into two main categories

Motor symptoms

Non-motor symptoms Understanding Parkinson’s Disease Future Development SimSensei Step 1 P(X1|C1) P(X|C1) P(X|C2) P(X7|C1) P(X1|C2) P(X7|C2) … Step 2 Compute the Probability of P(Ci)

e.g. P (having Parkinson’s disease) Final Step Method 1 - Comparing the sign of slope Method 2 - Modelling Method 2 - Modelling Method 2 - Modelling Method 3 - Normalization Implementation Method 3 - Normalization Method 3 - Normalization Method 3 - Normalization Observations Observations Model Implementation Hip ROM Modelling Calculate the angle between the leg and the vertical:

Applied at the point where the y-coordinates of ankle joint of both legs intersect: Double support Development Environment Language: C#, MySQL, XAML Software Tools: Kinect SDK v1.6, Kinect Developer Toolkit v1.6, Visual Studio 2010, MySQL Connector Net 6.2.5, GitHub ...With the aid of motion sensing system Observations Case 2: Simulated same arm same heel Observations Case 4: Simulated elderly Experimental Setup Kinect

sensor Target Resting Tremor

Why Resting Tremor? Rigidity

Bradykinesia (Slowness of movement)

Postural Instability

Data Transformation & Relevant Analysis

Data Cleaning: take out data with error GARSM How to solve? Value

Differentiation

Integration Sign of Conclusion Method 1 is trivial but gives reasonable results

Method 2 is too specialize to normal walking pattern Use more data to build Build differentiated models

Method 3 works most as expected