Model encoded by parameter vectors theta and lambda.

Trial-wise learning model, that updates

**Overview**

**A Model-based Analysis of Impulsivity Using a Slot-Machine Gambling Paradigm**

What is impulsivity?

Various definitions including the following characteristics:

rapid and unplanned (Moeller et al)

risk-taking (Eyesenck et al)

novelty seeking (Cloninger et al)

lack of reason or careful deliberation (Stedman’s Medical Dictionary 1995)

**Methods**

Our approach

Mathematical Framework

**Modelling**

**Results**

**A Bayesian Approach**

The model

Naturalistic slot-machine paradigm

Impulsivity and Gambling

Why gambling?

Diagnostic Statistical Manual (DSM IV: Impulse Control Disorder Not Elsewhere Classified

DSM V: Addictive Disorder (2013)

Characteristics of impulsivity like novelty-seeking and risk-taking are expressed when people engage in a gambling task. FUN!

Research in Pathological Gambling gaining interest

Number of Papers per 100k in the Medline DB

Engaging experiment

Relevant to current research

Demonstrates Impulsivity

Game Variables:

The game:

Win/loss

Money in the machine

Overall performance

Bets (min/max/switch)

Top-up

Machine Switches

Cashouts

Gambles

**Future Work:**

Perceptual Readouts

Response Readouts:

**Thank you!**

TNU Zürich

NSC Masters Program

Professor Klaas Enno Stephan

Dr. David Cole

Dr. Andreea Diaconescu

Dr. med. Helene Haker Rössler

Dr. Jakob Heinzle

Dr. med. Dr. nat. rer. Quentin Huys

Dr. Frederike Petzschner

Dr. Sudhir Shankar Raman

Dr. Gabor Stefanics

Eduardo Aponte

Sandra Iglesias

Lars Kasper

Kate Lomakina

Daniel Renz

Tina Wentz

Dimitris Bolis

Falk Lieder

Thomas Baumgartner

Sebastian Grässli

Silvia Princz

Nicole Welti

Suzanne Wilde

Moritz v. Looz

Lilian Weber

Dr. Michael Pfeiffer

David Bontrager

Nawal El Boghdady

Aniruddh Galgali

Dennis Goldschmidt

Raphael Holca

Suraj Honnuraiah

Asim Iqbal

Sofia Jativa

Mitra Javadzadeh

Jonas Klein

Annahita Sedgi

Gerick Lee

Tom Lorimer

Hazael Montanaro

Daniel Neil

Asim Sengor

Joana Soldado Magraner

Saray Soldado Magraner

Atanas Stankov

Ivan Voitov

Valance Wang

Pegah Kassraian Fard

Institute of

Neuroinformatics

Special thanks to Professor Klaas Enno Stephan and Dr. Frederike Petzschner for their continued support through this project.

Bayes Theorem

Free Energy and Model Comparison

Models and Model Inversion

Variational Clustering

What is free energy?

Use in model comparison

Model inversion:

Variational Gaussian Mixture Model

Hierarchical Gaussian Filters: Perception

Reinforcement Learning:

Scaling likelihood, P(D|H) by belief, P(H), or the

prior

The problem to tackle:

What does our model of the world look like?

Given data

D

,

how probable is our hypothesis,

H

?

Generative models and inference

The answer: Bayes

likelihood

prior

model evidence

Evidence:

Belief scaling:

Stems from thermodynamics: A = U - TS

As natural systems minimise free energy, the brain minimises surprise.

Surprise

FE = log joint probability - Shannon entropy

= accuracy - complexity

Model fit

log(p(D)) = KL[q||p] + FE

Lower bound on Log model evidence

sensory input

input perception

volatility of perception: ,

group-level perception:

updates occur via precision-weighted prediction errors

Analog to Reinforcement Learning:

1. Measure of accuracy while correcting for complexity

2. Minimize surprise

Rescorla-Wagner reinforcement learning model:

salience

learning rate

association

maximum

conditioning

Propose an approximate posterior q and optimize that distribution

Bayes theorem:

posterior

Approximate inference: Variational Bayes

Analytic approximation to the posterior

Assumes data arises from a set of Gaussians

MLE to calculate optimal memberships

Optimizes clusters using maximum a posteriori

Use Bayesian model selection to pick optimum number of clusters

Variational GMM

Vanilla GMM

**Conclusion**

Comparison with Rescorla Wagner

Our model has a higher

negative FE.

--> Our model wins!

We've created a naturalistic paradigm that gives us a more rigorous, and richer in readout than a questionnaire.

Parameter-based clusters corroborate with the BIS, an established measure of impulsivity.

Observe

Validation

Binary HGFs provides a flexible framework to model the exploratory quality of impulsitvity mechanistically, as an abberation in perception.

Model

Current collaboration with the Zentrum für Spielsucht in Zürich

Continued work with the MPI Cologne to implement this paradigm on pre-treatment Parkinsonian patients.

**Parkinson's Study**

Try to eke out gambling behaviour in financial data with an eye to regulation.

**Financial Gambling**

**ZSV Zürich**

Broad construct

Modelling impulsivity

Mechanistically model impulsivity across the axes of

perception

and

decision-making

.

cut for time?

Apply the field of cognitive modelling to clinical applications and diagnosis

Translational neuromodeling

What does this entail?

Barratt Impulsiveness Scale (BIS)

Current approaches

Create a new,

naturalistic behavioural paradigm

: in our case, we choose gambling.

Observe

Model the mechanism behind an impulsive action from the pespective of

aberrant

learning

or

aberrant decision-making

. Characterise subjects based on model parameters.

Model

Cross-check our modelling results with the industry gold standard, the

BIS

.

Validate

Scaling likelihood, P(D|H) by belief, P(H), or the

prior

The problem to tackle:

What does our model of the world look like?

Given data

D

,

how probable is our hypothesis,

H

?

Generative models and inference

The answer: Bayes

likelihood

prior

model evidence

posterior

Hierarchical Gaussian Filters: Perception

sensory input

input perception

volatility of perception: ,

"self" perception:

win/loss (gross)-binary

win/loss (net)-binary

overlearning-binary

full performance-cont

machine performance-cont

Perceptual axis:

Hierarchical Gaussian Filters: Decision-making

Softmax-binary response function on second-level mean, which is the prediction of the perceived variable on trial t+1

Free parameter:

beta

, controls curvature

bet (min/max)

switching bets

switching bets, switching machines

switching bets, switching machines and gamble option

all switches (switching bets, switching machines, cashing out, gamble option)

Response axis (all binary)

External Validation:

Minimum free energy

Maximum sensitivity

Players track gross win/loss

Switching behaviour is most informative

Meaning:

Cluster differences:

**Clustering**

**Interpretation**

Impulsivity as

exploration

Nesters

Foragers

Along the lines of risk-taking and novelty-seeking, the subgroups we unearth separate those who explore their environments and those who do not.

Clusters

**Supplementary Material**

48 healthy male volunteers at the Max Planck Institute, Cologne

Course of Experiment:

Experimental readouts:

Participants

-Barratt Impulsiveness Scale (BIS)

-UPPS Impulsive Behaviour Scale

-Temporal Discounting Task

External Measures

Impulsivity

-Beck Depression Inventory (BDI)

-Sensation Seeking Scale (SSS)

-Sensetivity to Reward and

Punishment (SPSRQ)

Sensation-seeking

-Barratt Impulsiveness Scale (BIS)

-UPPS Impulsive Behaviour Scale

-Temporal Discounting Task

Addiction

-Cambridge Gambling Task (CGT)

-TNU Slot-machine Game

Gambling

Subject data:

Subject interaction:

Variable 1

Variable 2

Correlation

Bet variance

Bet variance

Bet mean

Bet switches

Cashout

Gamble %

Bet mean

Gamble %

Bet mean

Machine Switch %

0.4187*

0.4892*

0.0525

0.1172

0.4604*

Questionnaire Readout:

Barratt Impulsiveness Scale

Variable 1

Variable 2

Correlation

Bet variance

Bet Mean

Bet Switches

Gamble Pct

Machine Switches

Cashout

BIS Total

BIS Total

BIS Total

BIS Total

BIS Total

BIS Total

0.4514*

0.1510*

0.4664*

0.3430*

0.4604*

0.2227

Overview of Topic

Underlying structure:

The trace contains:

true wins

true losses

fake wins

near misses

The structure of the trace is as follows:

1. losing streak

2. section high in fake wins

3. section high in non-sequential losses

4. section high in true wins

Simulations

Trace characteristics:

Return to Player (RTP):

(sum of all wins - sum of all losses)/total bet amount

Minimum requirement of 75% RTP (GLI LLC) --> our trace has a 90% RTP

Return to Player:

Clustering

Selecting the optimal number of clusters

FE comparison

Sensitivity analysis

Model Selection

Indep Var

Dependent Var

R-squared

Bet Switch %

Machine Switch %

Cashout %

0.0432

0.0197

0.0301

0.3486

0.4316

0.3477

omega

theta

beta

Regressions with paradigm readouts

-0.0000

-0.0224

0.0083

-0.0180

-0.0093

-0.0265

Indep Var

Dependent Var

R-squared

BIS Total

1.2656

0.2484

omega

theta

beta

0.9169

-3.3033

Regression with model parameters

Regressions

Experimental Procedure

- 48 healthy male volunteers

- Max Planck Institute, Cologne

- Took paradigm and a suite of

questionnaires, namely, the BIS

- Order: paradigm 1st, BIS 2nd

Belief Scaling

2nd level variance

3rd level

variance

probability of win

prior belief of machine

posterior belief of win

Exploitation

Exploration

BIS Threshold:

74

Low theta, high omega: high variable uncertainty but stronger belief in own inference.

Low theta, low omega: no uncertainty, no need to explore.

Parameter-based clustering on perceptual parameters omega and theta.

Same mean? 2-sample T-test: p-val 0.0014

Idenitical distributions? Man-Whitney-Wilcoxon: p-val 0.0002

Why impulsivity?