Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Thesis Prez

No description
by

Saee Paliwal

on 7 February 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Thesis Prez

Model inversion is inferring on theta given D and lambda by maximising the posterior distribution


Model encoded by parameter vectors theta and lambda.



Trial-wise learning model, that updates


Overview
A Model-based Analysis of Impulsivity Using a Slot-Machine Gambling Paradigm
What is impulsivity?
Various definitions including the following characteristics:
rapid and unplanned (Moeller et al)
risk-taking (Eyesenck et al)
novelty seeking (Cloninger et al)
lack of reason or careful deliberation (Stedman’s Medical Dictionary 1995)
Methods
Our approach
Mathematical Framework
Modelling
Results
A Bayesian Approach
The model
Naturalistic slot-machine paradigm
Impulsivity and Gambling
Why gambling?
Diagnostic Statistical Manual (DSM IV: Impulse Control Disorder Not Elsewhere Classified
DSM V: Addictive Disorder (2013)
Characteristics of impulsivity like novelty-seeking and risk-taking are expressed when people engage in a gambling task. FUN!
Research in Pathological Gambling gaining interest
Number of Papers per 100k in the Medline DB
Engaging experiment
Relevant to current research
Demonstrates Impulsivity
Game Variables:
The game:
Win/loss
Money in the machine
Overall performance
Bets (min/max/switch)
Top-up
Machine Switches
Cashouts
Gambles
Future Work:
Perceptual Readouts
Response Readouts:
Thank you!
TNU Zürich
NSC Masters Program
Professor Klaas Enno Stephan
Dr. David Cole
Dr. Andreea Diaconescu
Dr. med. Helene Haker Rössler
Dr. Jakob Heinzle
Dr. med. Dr. nat. rer. Quentin Huys
Dr. Frederike Petzschner
Dr. Sudhir Shankar Raman
Dr. Gabor Stefanics
Eduardo Aponte
Sandra Iglesias
Lars Kasper
Kate Lomakina
Daniel Renz
Tina Wentz
Dimitris Bolis
Falk Lieder
Thomas Baumgartner
Sebastian Grässli
Silvia Princz
Nicole Welti
Suzanne Wilde
Moritz v. Looz
Lilian Weber
Dr. Michael Pfeiffer
David Bontrager
Nawal El Boghdady
Aniruddh Galgali
Dennis Goldschmidt
Raphael Holca
Suraj Honnuraiah
Asim Iqbal
Sofia Jativa
Mitra Javadzadeh
Jonas Klein
Annahita Sedgi
Gerick Lee
Tom Lorimer
Hazael Montanaro
Daniel Neil
Asim Sengor
Joana Soldado Magraner
Saray Soldado Magraner
Atanas Stankov
Ivan Voitov
Valance Wang
Pegah Kassraian Fard
Institute of
Neuroinformatics
Special thanks to Professor Klaas Enno Stephan and Dr. Frederike Petzschner for their continued support through this project.
Bayes Theorem
Free Energy and Model Comparison
Models and Model Inversion
Variational Clustering
What is free energy?
Use in model comparison
Model inversion:
Variational Gaussian Mixture Model
Hierarchical Gaussian Filters: Perception
Reinforcement Learning:
Scaling likelihood, P(D|H) by belief, P(H), or the
prior
The problem to tackle:
What does our model of the world look like?
Given data
D
,
how probable is our hypothesis,
H
?
Generative models and inference
The answer: Bayes
likelihood
prior
model evidence
Evidence:
Belief scaling:
Stems from thermodynamics: A = U - TS
As natural systems minimise free energy, the brain minimises surprise.
Surprise
FE = log joint probability - Shannon entropy
= accuracy - complexity
Model fit
log(p(D)) = KL[q||p] + FE
Lower bound on Log model evidence
sensory input
input perception
volatility of perception: ,
group-level perception:
updates occur via precision-weighted prediction errors
Analog to Reinforcement Learning:
1. Measure of accuracy while correcting for complexity
2. Minimize surprise
Rescorla-Wagner reinforcement learning model:


salience
learning rate
association
maximum
conditioning
Propose an approximate posterior q and optimize that distribution
Bayes theorem:
posterior
Approximate inference: Variational Bayes
Analytic approximation to the posterior
Assumes data arises from a set of Gaussians
MLE to calculate optimal memberships
Optimizes clusters using maximum a posteriori
Use Bayesian model selection to pick optimum number of clusters
Variational GMM
Vanilla GMM
Conclusion
Comparison with Rescorla Wagner
Our model has a higher
negative FE.

--> Our model wins!
We've created a naturalistic paradigm that gives us a more rigorous, and richer in readout than a questionnaire.
Parameter-based clusters corroborate with the BIS, an established measure of impulsivity.
Observe
Validation
Binary HGFs provides a flexible framework to model the exploratory quality of impulsitvity mechanistically, as an abberation in perception.
Model
Current collaboration with the Zentrum für Spielsucht in Zürich
Continued work with the MPI Cologne to implement this paradigm on pre-treatment Parkinsonian patients.
Parkinson's Study
Try to eke out gambling behaviour in financial data with an eye to regulation.
Financial Gambling
ZSV Zürich
Broad construct
Modelling impulsivity
Mechanistically model impulsivity across the axes of
perception
and
decision-making
.
cut for time?
Apply the field of cognitive modelling to clinical applications and diagnosis
Translational neuromodeling
What does this entail?
Barratt Impulsiveness Scale (BIS)
Current approaches
Create a new,
naturalistic behavioural paradigm
: in our case, we choose gambling.
Observe
Model the mechanism behind an impulsive action from the pespective of
aberrant

learning
or
aberrant decision-making
. Characterise subjects based on model parameters.
Model
Cross-check our modelling results with the industry gold standard, the
BIS
.
Validate
Scaling likelihood, P(D|H) by belief, P(H), or the
prior
The problem to tackle:
What does our model of the world look like?
Given data
D
,
how probable is our hypothesis,
H
?
Generative models and inference
The answer: Bayes
likelihood
prior
model evidence
posterior
Hierarchical Gaussian Filters: Perception
sensory input
input perception
volatility of perception: ,
"self" perception:
win/loss (gross)-binary
win/loss (net)-binary
overlearning-binary
full performance-cont
machine performance-cont
Perceptual axis:
Hierarchical Gaussian Filters: Decision-making
Softmax-binary response function on second-level mean, which is the prediction of the perceived variable on trial t+1

Free parameter:
beta
, controls curvature
bet (min/max)
switching bets
switching bets, switching machines
switching bets, switching machines and gamble option
all switches (switching bets, switching machines, cashing out, gamble option)
Response axis (all binary)
External Validation:
Minimum free energy
Maximum sensitivity
Players track gross win/loss
Switching behaviour is most informative
Meaning:
Cluster differences:
Clustering
Interpretation
Impulsivity as
exploration
Nesters
Foragers
Along the lines of risk-taking and novelty-seeking, the subgroups we unearth separate those who explore their environments and those who do not.
Clusters
Supplementary Material
48 healthy male volunteers at the Max Planck Institute, Cologne
Course of Experiment:
Experimental readouts:
Participants
-Barratt Impulsiveness Scale (BIS)
-UPPS Impulsive Behaviour Scale
-Temporal Discounting Task
External Measures
Impulsivity
-Beck Depression Inventory (BDI)
-Sensation Seeking Scale (SSS)
-Sensetivity to Reward and
Punishment (SPSRQ)
Sensation-seeking
-Barratt Impulsiveness Scale (BIS)
-UPPS Impulsive Behaviour Scale
-Temporal Discounting Task
Addiction
-Cambridge Gambling Task (CGT)
-TNU Slot-machine Game
Gambling
Subject data:
Subject interaction:
Variable 1
Variable 2
Correlation
Bet variance
Bet variance
Bet mean
Bet switches
Cashout
Gamble %
Bet mean
Gamble %
Bet mean
Machine Switch %
0.4187*
0.4892*
0.0525
0.1172
0.4604*
Questionnaire Readout:
Barratt Impulsiveness Scale
Variable 1
Variable 2
Correlation
Bet variance
Bet Mean
Bet Switches
Gamble Pct
Machine Switches
Cashout
BIS Total
BIS Total
BIS Total
BIS Total
BIS Total
BIS Total
0.4514*
0.1510*
0.4664*
0.3430*
0.4604*
0.2227
Overview of Topic
Underlying structure:
The trace contains:
true wins
true losses
fake wins
near misses

The structure of the trace is as follows:
1. losing streak
2. section high in fake wins
3. section high in non-sequential losses
4. section high in true wins
Simulations
Trace characteristics:
Return to Player (RTP):
(sum of all wins - sum of all losses)/total bet amount

Minimum requirement of 75% RTP (GLI LLC) --> our trace has a 90% RTP
Return to Player:
Clustering
Selecting the optimal number of clusters
FE comparison
Sensitivity analysis
Model Selection
Indep Var
Dependent Var
R-squared
Bet Switch %
Machine Switch %
Cashout %
0.0432
0.0197
0.0301
0.3486
0.4316
0.3477
omega
theta
beta
Regressions with paradigm readouts
-0.0000
-0.0224
0.0083
-0.0180
-0.0093
-0.0265
Indep Var
Dependent Var
R-squared
BIS Total
1.2656
0.2484
omega
theta
beta
0.9169
-3.3033
Regression with model parameters
Regressions
Experimental Procedure
- 48 healthy male volunteers
- Max Planck Institute, Cologne
- Took paradigm and a suite of
questionnaires, namely, the BIS
- Order: paradigm 1st, BIS 2nd
Belief Scaling
2nd level variance
3rd level
variance
probability of win
prior belief of machine
posterior belief of win
Exploitation
Exploration
BIS Threshold:
74
Low theta, high omega: high variable uncertainty but stronger belief in own inference.
Low theta, low omega: no uncertainty, no need to explore.
Parameter-based clustering on perceptual parameters omega and theta.
Same mean? 2-sample T-test: p-val 0.0014
Idenitical distributions? Man-Whitney-Wilcoxon: p-val 0.0002
Why impulsivity?
Full transcript