Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Bayesian statistical inference illustrated by toy models

No description
by

Yen Ting Lin

on 24 June 2018

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Bayesian statistical inference illustrated by toy models

We need the "likelihood" which quantifies the probability of a a1 to reproduce the data point. Since each data point is independent, we have
Bayesian statistical inference
and
Uncertainty quantification (UQ)

2018 q-Bio summer school Southwest Campus

Yen Ting Lin
6/20/2018
Statistical inference

Definition
: A process to deduce the underlying (model, parameters, etc.) probability distributions using
data
.
Uncertainty quantification
Definition
: (less well-defined) The process of quantifying the uncertainty
1. Frequentists' approach: bootstraping, hypothesis testing (
p
-value)...
2.
Bayesian approach
1. Forward propagation UQ: you have done it last week!
2. Backward propagating UQ: you will learn it today!
Why do we want to go beyond maximum likelihood (best fit) picture and perform statistical inference & UQ?
Data and measurement are not perfect; there are always some noises.
The noise can come from intrinsic behaviour of the system, or measurements (sampling). Is it purely out of luck that we obtain a good experimental (data) support, or is it something "significant"?
A toy example to illustrate the methods
A linear equation with independent Gaussian noise:
Task: Given a set of synthetic data,
find out
model parameters.
MLE
: best-fit parameters which minimize the chi-square.
Statistical inference
: find the "distribution" of the model parameters
Bayesian inference framework
A natural generalization of stochastic models
Deterministic models
deterministic parameters, deterministic observables
Frequentists' stochastic model
deterministic parameters, stochastic observables
Bayesian stochastic model
stochastic parameters, stochastic observables
Concept
:
Joint
probability distribution of the
parameters
and the
observed data
, linked by the model prediction using the Bayes formula.
"Forward"
"Backward"
Bayesian inference framework
A natural generalization of stochastic models
Bayesian stochastic model
stochastic parameters
θ
, stochastic observables
D
Understand Bayesian inference through the toy model
Assume we know everything but the slope
a1
, which will be inferred
We need a "prior" , which is often described as the "prior belief" of the parameter
a1.
We assume it's uniform in between (0, 100).
A naïve way to compute the "posterior" distribution by evaluating its value at every possible (or interested)
a1
value.
Understand Bayesian inference through the toy model
Note that the posterior is updated sequentially (online updating)
for independent experiments.
Effect of the prior I
Effect of the prior II
Usage of the posterior II: test for significance
Uncertainty of a1
Usage of the posterior I: "correlation" between parameters
Let's consider a 2-parameter model: inferring
a0
and
a1:
Usage of the posterior II: test for significance
The
joint
posterior distribution is more informative than the
marginalized

Usage of the posterior II: test for signficance
Usage of the posterior III: evidence, marginalized likelihood
quantification of how much a model explains the data
Probability of a model with parameters
θ
to explain data
D
Prior probability of the parameters
θ
Sum over all possible parameters
θ
Total (all possible
θ)
probability that the model can explain the data
Getting technical: high-dimensional inference
Up to now, we adopted the naïve way to compute the "posterior" distribution by evaluating its value at every possible (or interested)
θ
value. The
grid-based
scanning procedure becomes impossible when the dimensionality of the parameter space is larger than 3...

Solution
: use the sampling techniques in statistical physics:
Monte Carlo methods
.
An intuitive method termed as
Markov chain Monte Carlo
constructs a random walk in the parameter space, such that the
stationary distribution
of the random walk coincides with the
posterior distribution.

Using a 2-state, discrete model as an example:

The art of the game: formulating the likelihood
Extension:
sequential Monte Carlo (sMC)
for stochastic dynamical systems
and/or hidden Markov process
Questions
?

(illustrated by toy models)
Full transcript