Loading presentation...
Prezi is an interactive zooming presentation

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Modélisation et détection des émotions à partir de données e

No description
by

Franck Berthelon

on 19 December 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Modélisation et détection des émotions à partir de données e

"
A complex system is a system composed of

interconnected parts

that as a whole exhibit one or more properties (behavior among the possible properties)

not obvious from the properties of the individual parts
."
Palomba 97
Heart-rate valence information
Emotion overview
Why ?
universality
attention
Use case
Serious games

Learning
Expression
Psycho-evolutionist theory
James-Lange theory
Cannon-Bard theory
The two-factor theory
Component Process Model
Emotion overview
II - Our prototype
I - State of the art
"serious games, that is, (digital) games used for purposes other than mere entertainment"

Tarja Susi
Field
Human Machine Interaction
Perception
Attention
Person
Stimulus
HMI "à pointe fine" Joëlle Coutaz
New interaction
Unintentional action
Picard 91
A definition attempt
Kleinginna
Person
Stimulus
How to improve emotion detection ?
Adaptability
Reliability
Performance
How to develop a sensor independent detector ?
Use case constraint : Real-time computation
How to use contextual data in emotion detection process ?
Hypothesis : multi-modal approach improve detection reliability
How to unify emotion knowledge ?
Table of contents
IV -
Em
otion
O
ntology
for
C
ontext
A
wareness

III - Personalized
Emotional Maps

V - Evaluations
Conclusion
State of the art
Architecture & Prototype
Personalized Emotional Maps
Emotion Ontology for context
Awareness

Evaluations
Conclusion
Person
Stimulus
Person
Stimulus
Person
Stimulus
Person
Stimulus
Fear
Surprise
Joy
Disgust
Anger
Perception
Expression
Charles Darwin
William James and Carl Lange
Walter B. Cannon and Philip Bard
Klaus R. Scherer
Stanley Schachter and Jerome E. Singer
Perception
Expression
Feedback
Feeling
Perception
Expression
Feeling
Perception
Expression
Context
Hyper-arousal
Feeling
Emotion
Mood
Speed
Frequency
Intensity
Duration
Trigger
Effect on attention
Physiological and
behavioral component
Fast
Slow
Rare
Frequent
High
Low
Easy
Hard
Seconds/minutes
Hours/days
Present
Absent
High
Low
Ekman & Davidson 94
Detection process
Dimensional
Hypothesis
Categorial
Emotion modelization and detection from expressive and contextual data
Presented by Franck Berthelon
Sensors
Psychological
Behavioral
David Marr 82
Physiological
Internal changes
Easy to use
Hard to falsify
High invasivity
Continuous data
EG :
Heart rate
Skin conductance
Skin temperature
EEG
High
decrease
Low
decrease
Negative
valence
Positive
valence
Increase
Positive
arousal
External changes
Hard to use
Easy to falsify
Low invasivity
Mixed data
EG :
Facial expressions
Voice
Text
External changes
Hard to use
Easy to falsify
Low invasivity
Essentially posture
Related to action tendancy
Camera
Microsoft Kinect
Frijda 87
EG :
Positioning
Sensor agnostic
Multi-sensors
Models
Ontology
Psychology
Positioning
Other
Detectors
Overview
Facial expressions
Demo
Modules
Sensors processing
Expression classification
Contextualization
8 extracted features
Eyebrow
Mouth
Color-based detection
Luminosity
Constrast
Morphologic functions
Haar classifier for face detection
Facial expression
Personalized emotional maps
Russel's model
Emotion Ontology for Context Awareness
Scherer's model and PEM
Why a complex system ?
Definition :
For emotion :
Many interconnected expressions. All driven by one emotion.
Some individual expression easily understandable.
Non specific response for emotion

PEM model
Calibration
Detection
Synthesis
Context restriction
Use case
Ontology
Reasoning on context
Contextualisation
Categorisation
Our proposal
Future work
Thanks for your attention
Publications :
F. Berthelon & P. Sander, "
Em
otion
O
ntology for
C
ontext
A
wareness
", IEEE CogInfoCom 2013
F. Berthelon & P. Sander, "
Regression algorithm for emotion detection
", IEEE CogInfoCom 2013 (best paper award)
Extend context
Extend sensor pool
PEM comparison
PEM evaluation
Contextualisation
Hysteresis
Synthetisation
Methodology
Adaptation to experimental constraints
Obstruction/power cant' be detected
n emotion features
continuous approach
eg. : Russell's model
VS
n emotions terms
discrete approach
eg. : Plutchik's model
Scherer's complex model
Emotion as a complex system
René Thom's catastrophe theory
Hysteresis to describe emotion
Emergent behavior
No restriction on output model
Follow Scherer's model for expression recognition
What is context ?
Memory
Personality traits
Stimulus
Environment (intern/extern)
...
Our context
Phobias
Philias
Why a restriction ?
Feasibility
(Serious) games compatibility
Long term knowledge
How context impacts emotion experience ?
Convert dimensional measure to categorial
Definition : Compute expression from continuous emotion
Why an ontology ?
Arachnophobia
Cynophilia
and
and
Modelization of intangible knowledge
Semantic relation between concept (triplet)
Reasoning capabilities
Throw inference engine
Interoperability with other knowledge base
Reuse data define elsewhere
Expose data in standardized way
Tom
Cat
Mammal
Conceptual level
Instance level
Search for a contextual impact on emotion detection
Model
+
Arachnophobic
>
+
Non-arachnophobic
(Valence, Arousal)
Vocabulary
Joy
Fear
Anger
Sadness
...
Measure
(0.8, 0.7)
Cartesian
Polar
Metrics :
16 classes
9 properties
Allow to describe any kind
of philia or phobia and their
impact on emotion
W3C Technology :
RDF
RDFs
Developed by our team
SPARQL request language
Request
Corese inference engine
Goal :
Retrieve activated personality trait
Apply impact on measure
Search for a compatible emotion term
Perception
Expression
Feeling
Knowledge
Results
Discussion
Metrics
Proceedings method
Average deviation
by features
Average deviation
by emotion
Average deviation
Calibration: 1232 images, 6 emotions, 6 sequences
Test: 5757 images, 6 emotions, 17 sequences
Influence of K parameter on detection
Summary
5757 images
17 sequences
17 transitions detected
14 emotions matched
82% success
Main difficulty on disgust
1232 images
6 emotions
K = 1 to 15
K = 6 highest recognition rate
1.4% on valence
2.1% on arousal
Overall evaluation
Methodology
Result
Proceedings method
High influence of K parameter on detection
KNN well adapted
Good recognition rate
Real-time processing
Low anger/disgust distinction
Add more anger/disgust sequences
Observe more modalities
E.g. : Heart rate or new facial features
Result
Feedback
Calibration
Ground truth
Satellite image classification
Methodology
Result
Discussion
Proceedings method
Mimic
}
Elicited emotion
...
{
Sensor measures
n : emotion dimensions
m : number of sensors
...
{
Sensor measures
m : number of sensors
Regression algorithm
based on KNN
Symetric sequence
Normal sequence
Hysteresis on valence
Hysteresis on arousal
No hysteresis on valence
No hysteresis on arousal
Dimensional
Categorial
{
Sensor measures
m : number of sensors
...
Methodology
Results
Discussion
Metrics
Proceedings method
3
Naive approach
Not dedicated system
Expression tendency
Russel's model
Emotion discrimination problem
Selection strategy (average)
Expression attenuation
Contribution
Improvement
Compute emotion from one sequence
Store detected emotion
Repeat with symetric sequence
1.
2.
Discussion
First observation of hysteresis
Contribute to Scherer's complex model
Not caused by conservative algorithm
Need more investigation
PEM, inspired by Scherer's complex model
Sensor agnostic
Overall good recognition rate
Hysteresis observation
Emotion synthetization capability
Feasibility of emotion contextualization with EMOCA
Complete prototype
Paper
Type
Signals
Methodology
Emotions
Haag 04
Multimodal
EMG, GSR, skin
temperature, BVP,
ECG, respiratory rate
Neural network
Valence/Arousal
Picard 01
Multimodal
EMG, GSR, BVP, respiratory rate
SFFS-FP
KNN
neutral, anger, hate, sadness, like, love, joy, reverence
Bos 06
Unimodal
EEG
FDA classifier
Valence/Arousal
Li 09
Unimodal
EEG
SVM + CSP
joy, sadness
Gunes 07
Multimodal
Face, body
Bayesien classifier
anxiety, anger, disgust, fear, joy, uncertainty
HEO :
based on EmotionML
GRIHO :
Describe emotion through expression
EMO :
detection annotation, distinguish among emotion temporality
Questionnaire
Step 1 : recognize expressed emotion without context
Step 2 : recognize expressed emotion with context
Participants
10 persons from 22 and 36 years old
4 women and 6 men
Annotation dissimilarity according to context
Agreement among participants on contextual annotation
Sometime emotion recognition is hard even for humans
Error on anger/disgust or fear/surprise
Strong impact of context
29% emotion change
57% intensity change
Hard to generalize impact on emotion
52% accordance between participants
Dissimilarity according to context
Agreement among participants
Emotion features
Cowie
Pleasant
Unpleasant
Time
Eyebrow left
Mouth left
Stimulus
Expression
Emotion
Adjustement
Detector
Affective computing
Cognitive info-communication
1
2
3
4
5
6
7
{
}
{
}
1
2
4
5
3
6
7
11
12
15
13
14
16
17
18
19
20
22
23
24
1
25
26
27
31
35
28
29
30
32
33
34
21
8
9
10
OpenCV
Architecture & prototype
Personalized Emotional Maps
Adaptability
Reliability
Performance
How to develop a sensor independent detector ?
Real-time computation
Hypothesis : multi-modal approach improve detection reliability
Emotion Ontology for Context Awareness
Adaptability
Reliability
Performance
How to unify emotion knowledge ?
Real-time computation
How to use contextual data in emotion detection process ?
Evaluations
“Emotion is a complex set of interactions among subjective and objective
factors, mediated by neural/hormonal systems, which can
a) give rise to affective experiences such as feelings of arousal, pleasure/displeasure;

Kleinginna 81
b) generate
cognitive processes
such as emotionally relevant perceptual effects,
appraisals, labeling processes
;

c) activate widespread
physiological adjustments
to the arousing conditions;
and

d) lead to
behavior
that is often, but not always,
expressive
,
goaldirected
, and
adaptive
.”

“We [will] consider emotion in an inclusive sense rather than in the
narrow sense of
episodes
where a strong rush of feeling briefly
dominates
a person’s awareness
.
Cowie 05
... emotion in the broad sense
pervades
human
communication and cognition
.
Human beings have positive or negative
feelings about most things, people, events and symbols. These feelings
strongly
influence the way they attend, behave, plan, learn and select.

Generate cognitive process
appraisal
Physiological adjustments
Behavior adjustments
Pervasive emotion
Episodic emotion
High impact on awareness
Both change behavior, learning and attention
Full transcript