**Ensemble Models**

**and their use in modeling macro-micro relationships**

We can make use of this when working with directed graphical models.

Which relationships (perhaps of a limited number of possible alternatives) hold between the variables we are working with is itself a random variable. The evidence for particular relationships holding can provide information about the value of the external variable.

Let us be modeling a switching system. The variables of the system we are working with relate to each other differently depending on the value of an external environmental variable. Assume we have all and only the models appropriate for each value of the external variable.

Using Bayes rule and an appropriate prior, the likelihood of the models given (current) data provides a means of calculating a distribution over the states of the external variable.

Motivating Thoughts

Let M be a categorical variable taking the values {m1, m2 … mn}. Let our models be such that each encodes the relationships taken to hold if M takes its ith value.

Where the degrees of freedom of a conditional distribution is exponential on the number of variables being conditioned upon, we may obtain major reduction of the degrees of freedom of our models by moving to using an external variable. Let us imagine conditional distribution consisting of variables with v values, except for a potential external variable, A, with w=2 values.

This generalizes under simple assumptions about regularity. (That each possible conditional distribution produced when using A as an external variable connects Z with the same number (n/w) of parents.)

Often

Naively duplicating whole networks for external variables relevant to one or few conditional distributions would be ridiculous.

But it does make sense to treat particular local mechanisms in this way:

Interpret as interlaced networks or as latent variables* as desired...

What would this look like? What advantages would this have?

Use in Directed Graphical Models

Use in Directed Graphical Models

Use in Directed Graphical Models

Illustrative Example: Market Sentiment

Market sentiment is:

Important to trading.

Emergent.

Not directly observable.

Observable in hindsight.

Dynamically extended.

Assume we have:

Historical data of stock movements.

Retrospective labels for each day as bearish, neutral or bullish.

From the label sequence we train a dynamics model.

Assume MP of order 1

Separate data into three datasets based on labels.

Train networks for each using structural learning algorithms.

Encoding relationships holding in each MS state.

Illustrative Example: Market Sentiment

This gives us a Hidden Markov Model, where the sensor conditional distribution is given by a the technique discussed.

If we wish to know the expected movement in stock A tomorrow, we:

estimate the distribution over market sentiment tomorrow.

estimate the distribution over A's movement for all market sentiments, using the appropriate networks.

combine these estimates of A's distribution, weighting according to the distribution over market sentiment.

We can estimate state in the usual way. If we have queries for the micro variables, we can then use the state estimates as weights for Bayesian model averaging.

=

.28

.16

.56

p(A )=

t+1

*Not always possible to 'flatten' into a DAG

Specification

Example

Illustrative Example: Market Sentiment

Illustrative Example: Market Sentiment

Motivation

**Overview**

Motivating Thoughts Expanded