### Present Remotely

Send the link below via email or IM

CopyPresent to your audience

Start remote presentation- Invited audience members
**will follow you**as you navigate and present - People invited to a presentation
**do not need a Prezi account** - This link expires
**10 minutes**after you close the presentation - A maximum of
**30 users**can follow your presentation - Learn more about this feature in our knowledge base article

# Variance

E1

by

Tweet## Jacob Hoke

on 17 February 2012#### Transcript of Variance

What is win-rate? Calculating the mean What is variance? Calculating the variance What is standard deviation? Using Standard Deviation Scaling An example What will we cover in this series? Tolerance You can easily use the standard deviation to find the probability that an observed sample will fall in a certain range. In a normal distribution, a result will be within 1 S.D. from the mean 68% of the time, 2 S.D. 95% of the time, and 3 S.D. from the mean 99.7% of the time.

You can scale these numbers so that you can take a look at how results can change over larger and smaller sample sizes. You can easily scale the MEAN and the VARIANCE by multiplying to get the desired sample size. Take a poker player with a win-rate of 4bb/100 (0.04bb/hand) and a standard deviation of 60bb/100 hands. Pretty straightforward here. Our mean is 0.04bb/hand. We want to take a look at a 50k hand sample so we simply the mean by 50,000.

50,000 * 0.04 = 2000 bb Remember, we have to scale the variance, not the standard deviation. We were given the standard deviation (as most poker tracking software will give us). So, let's start by finding the variance/100 hands, then the variance/hand, and then we can find the standard deviation/50,000 hands. Per 100 hands:

(standard deviation)^2 = variance

60^2 = 3600

Per hand:

variance/hand = 3600/100 = 36/hand

(standard deviation/hand = sqrt(36) = 6) Now we simply multiply our variance per hand (36) by 50k to get our variance per 50k hands. We then take the square root of the result to find the standard deviation per 50k hands.

36*50,000 = 1,800,000

sqrt (1,800,000) = 1342

s.d./50k = 1342 bb What is his mean and standard deviation over 50k hands? mean = 2000 bb

S.D. = 1342 bb

As sample size increases, the mean overtakes the standard deviation. At 100 hands the mean is 4 and the standard deviation is 60. At 50k hands the mean is 2000 and the standard deviation is 1342. As the sample size gets larger the results will GRADUALLY converge on the mean (this is the concept often referred to as "The Law of Large Numbers".) The larger the mean is in relationship to the standard deviation the less swingy the results will be. It is the relationship between these two numbers that determines how wide a range our results will fall in.

In any given 50k hand sample, the results will lie between:

+648 bb and +3342 bb (1 SD) 68% of the time

-684 bb and +4684 bb (2SDs) 95% of the time

-2026 bb and +6026 bb (3 SDs) 99.7% of the time. The trouble with win-rates Our observed win-rate is unlikely to be our actual win-rate because: Our sample size is almost always small

It takes a long time to acquire a reasonable sample size

Our play and our opponents' play varies over time

The game conditions constantly change

Our personal skill level changes A theoretical 2bb/100 winner Let's say we have a player, Sammy Supernova, who we somehow know runs at 2bb/100 with a standard deviation of 70bb/100.

Let's take a look at the range of outcomes we can expect from Sammy over his next 100k hands (a light month for Sammy). mean = 2000 bb

S.D. = 2216 bb

Along with these numbers, we can use the normal distribution to find Sammy's chances of having results in certain ranges How likely is Sammy to win more than twice his mean winrate? He will exceed twice his mean winrate 18% of the time given a 100k hand sample. Link: http://davidmlane.com/hyperstat/z_table.html Months like these can easily skew our perception of our win-rate. Poker players have a natural tendency to believe that these extra-successful months are owed to us for all our past bad luck when it all actuality it is just a stretch of positive variance. We all will have a lot of both positive AND negative variance. We often don't recognize the 'run good' for what it is: simply 'run good'! This leads to a tendency to overestimate our winrate.

Because poker is a high-variance game, most of the time we will either run good or run bad. We very rarely will run "normal". How likely is Sammy to run close to his winrate? How about breaking even? Sammy has about a 12% chance of breaking even over 100k hands (I compute break-even as lying between -500 and +500 bb).

He has a 18% chance of running hot (>4000 bb), and a 12% chance of breaking even over his next 100k hands. These numbers are suprising similar! The Dude The Dude is the guy who is crushing the stakes. Let's give him a win-rate of 8bb/100 and a slightly larger standard deviation of 80bb/100. The Dude plays 1 million hands a year.

The Dude does everything a great poker player should do. He table selects well. He is completely tiltless. He is indeed... The Dude.

How much can The Dude's winrate vary just due to variance over 1 million hands? mean = 80,000

S.D. = 8,000

To see where 99.7% of our results will lie we add 3 S.D's to the mean and subtract 3 S.D.'s from the mean. 8,000*3 = 24,000

24,000 + 80,000 = 104,000 bb

-24,000 + 80,000 = 56,000 bb The "best case scenario" of 104,000 is the equivalent of winning at 10.4bb/100.

The "worst case scenario" of 56,000 is the equivalent of winning at 5.6bb/100.

Both scenarios are 30% away from The Dude's actual win-rate

68% of the time the Dude will run within 10% of his actual win-rate

If this were a $1/2 player's yearly volume it would be the difference in an income of $208,000 and $112,000, just due to variance! Some things to learn from The Dude The Dude is winning at a high win-rate and even he can experience big swings. If you have a smaller win-rate, you can experience even greater swings.

Even a very good player can run quite hot over a large sample.

Running hot and seeing really amazing results doesn't tell us as much as we would intuitively believe.

A reasonable winner can look like The Dude over a reasonable sample.

Don't let other players' amazing graphs get your down (selection bias).

The guys who quickly ran to the nose bleed stakes, while certainly awesome players, had some good luck on their side to make the exceptional run. Of course, it would be easier to get those sort of runs in juicier games (which provide higher win-rates, leading to less negative variance). What about a good solid player? Say we have a player who wins at 6bb/100 and a S.D. of 70/100 hands over a 170k sample. His next 100k hands he breaks even. 0.3% What can we gather from this? Assuming that our observed 6bb/100 winner is a strong player who studies and thinks about the game correctly, it is unlikely that he is actually a break-even player. However, it is also fairly unlikely that he is a 6bb/100 winner. He probably ran hot.

It is also fairly likely that his play has deteriorated due to his bad run, which caused his winrate to go down, which causes the probability of a bad run to increase, which causes his play to deteriorate, which causes his win-rate to go down.... you can see the vicious cycle. Don't let your observed win-rate influence your emotional state or your self-perception.

We really can only assess our level of play via hand analysis and observation.

Be aware of variance. Don't let it cloud your judgment or affect your learning.

Winning players run hot over large samples and we will often (always?) hear about them.

Don't count on your winrate being high. You never really know!

The long run is really, really long.

Don't let the vicious cycle continue. Keep studying. Keep learning. Set goals that aren't related to results (more on this in future episodes). What is a win-rate?

What is variance?

What is standard deviation?

What is fair to expect?

How long is the long run? Win-rates, standard deviation, mathematical variance

Recognizing variance and its effects

The Learning Process

Developing a Learning Plan

Identifying variance

Identifying "run good"/"run bad"

Understanding variance's effect on the mind

Comparing results to EV How do you learn?

Learning in small chunks

The Conscious-Competance Learning Matrix

Pre/Post-Session routines

Setting attainable goals Pre-session routine

Post-session routine

The Session Cycle

Executing Your Strategy

Using spreadsheets

Using PokerStove

Taking time off A player's win-rate is the average amount he or she wins per some sample size or time (hourly). It is most commonly expressed in terms of big blinds per 100 hands (bb/100). In terms of statistical calculations, this is our "mean". To calculate the mean we simply multiply the probability of all the events by their respective value and sum the results.

Say we want to calculate the mean of a coin flip. Let's give heads a value of 1 and tails a value of 0. Thus...

mean = P(heads)*(value of heads) + P(tails)*(value of tails)

mean = (0.5)*(1) + (0.5)*(0) = 0.5 + 0 = 0.5 Variance is a mathematical measurement of how far the results may fall from the mean.

Variance is often used in the poker community to talk about negative swings (this causes some ambiguity), but this is only half of the true mathematical definition. There is positive variance, too!

Many poker players tend to downplay positive variance and exaggerate negative variance. This leads to a skewed outlook on the game (more on this later). Variance is calculated by multiplying the probability of an outcome by the square of its distance from the mean. Sum this number over all possible outcomes, and you have the variance.

variance = P(outcome 1)[(value of outcome 1 - mean)^2] +... p(outcome n)[(value of outcome n - mean)^2]

Using the coin flip example (mean = 1/2, value of heads = 1, value of tails = 0):

variance = p(heads)*[(value of heads - mean)^2] + p(tails)*[(value of tails - mean)^2]

variance = (1/2)*[(1 - 1/2)^2] + (1/2)*[(0 - 1/2)^2]

= (1/2)*[(1/2)^2] + (1/2)*[(-1/2)^2]

= (1/2)*[1/4] + (1/2)*[1/4]

= 1/8 + 1/8 = 1/4

variance (coin flip) = 1/4 It's the square root of variance

It is more useful than the variance (more on this later)

With standard deviation and win-rate we can mathematically determine what to expect from results

Standard deviation of a coin flip is sqrt(1/4) = 1/2 Let's say that "close" means within 10% of his mean winrate.

10% 0f 2000 = 200

2000 + 200 = 2200

2000 - 200 = 1800

So, we are looking for the probability that Sammy's results will lie between 1800 and 2200 bb. 7.2% Only about 7.2% of Sammy's 100k hand samples will end up within 10% of his mean winrate. And this is over what most poker players would believe is a large sample! This goes to show how much variance there can be in poker, even over great numbers of hands. A yearly breakdown using these numbers ~2.2 months of running hot (> 4000 bb)

~1.4 months of break even (-500 to 500 bb)

~1.6 months losing at least 5BI (< -500 bb)

~6.8 months (500 to 4000 bb) About 30% of the time Sammy has a "run good" month or a "run bad" month. They are similarly likely. That's fairly often.

Even though the hot months occur about as frequently as the cold months, we tend to feel like the hot months are a more accurate depiction of our poker prowess. By that logic, we should feel that our cold months are also accurate indicators of our prowess (sometimes we do!). Of course, neither of these are even close to reliable indicators.

All of those months when you feel like you did alright, but could have run better, you probably ran pretty close to average. How often does Sammy lose more than 5BI's? 13% Of course, both of these results are outliers, but it goes to show how extreme our results can get.

In fact, his results will lie in the range of 9.6bb/100 to 10.4bb/100 only 2% of the time. His results will lie in range of 5.6bb/100 to 6.4bb/100 2% of the time as well. On pretty graphs There are a lot of good players out there. A lot of them are playing a lot of hands. A hypothetical 6bb/100 winner will have his next 300k hands run at 10bb/100 just 1% of the time. However, if you consider how many players are out there, that most of them have statistical tracking software, and that many will love to show off their pretty graphs, it is not too far-fetched to imagine that you are going to see these graphs pop up from time to time. While it almost certainly means that player is quite strong, it doesn't ensure that he is the best player in the game. Not even close. There is a strong selection bias here as poker players will rarely post mediocre graphs, unless to complain about how bad they are running.

Most players graphs aren't THAT pretty. How likely will a 6bb/100 winner break even over his next 100k hands? How likely will a breakeven player win 6bb/100 over his next 100k hands? This is why poker can be so frustrating! Tolerance This guy knows the mean! Mean?

Full transcriptYou can scale these numbers so that you can take a look at how results can change over larger and smaller sample sizes. You can easily scale the MEAN and the VARIANCE by multiplying to get the desired sample size. Take a poker player with a win-rate of 4bb/100 (0.04bb/hand) and a standard deviation of 60bb/100 hands. Pretty straightforward here. Our mean is 0.04bb/hand. We want to take a look at a 50k hand sample so we simply the mean by 50,000.

50,000 * 0.04 = 2000 bb Remember, we have to scale the variance, not the standard deviation. We were given the standard deviation (as most poker tracking software will give us). So, let's start by finding the variance/100 hands, then the variance/hand, and then we can find the standard deviation/50,000 hands. Per 100 hands:

(standard deviation)^2 = variance

60^2 = 3600

Per hand:

variance/hand = 3600/100 = 36/hand

(standard deviation/hand = sqrt(36) = 6) Now we simply multiply our variance per hand (36) by 50k to get our variance per 50k hands. We then take the square root of the result to find the standard deviation per 50k hands.

36*50,000 = 1,800,000

sqrt (1,800,000) = 1342

s.d./50k = 1342 bb What is his mean and standard deviation over 50k hands? mean = 2000 bb

S.D. = 1342 bb

As sample size increases, the mean overtakes the standard deviation. At 100 hands the mean is 4 and the standard deviation is 60. At 50k hands the mean is 2000 and the standard deviation is 1342. As the sample size gets larger the results will GRADUALLY converge on the mean (this is the concept often referred to as "The Law of Large Numbers".) The larger the mean is in relationship to the standard deviation the less swingy the results will be. It is the relationship between these two numbers that determines how wide a range our results will fall in.

In any given 50k hand sample, the results will lie between:

+648 bb and +3342 bb (1 SD) 68% of the time

-684 bb and +4684 bb (2SDs) 95% of the time

-2026 bb and +6026 bb (3 SDs) 99.7% of the time. The trouble with win-rates Our observed win-rate is unlikely to be our actual win-rate because: Our sample size is almost always small

It takes a long time to acquire a reasonable sample size

Our play and our opponents' play varies over time

The game conditions constantly change

Our personal skill level changes A theoretical 2bb/100 winner Let's say we have a player, Sammy Supernova, who we somehow know runs at 2bb/100 with a standard deviation of 70bb/100.

Let's take a look at the range of outcomes we can expect from Sammy over his next 100k hands (a light month for Sammy). mean = 2000 bb

S.D. = 2216 bb

Along with these numbers, we can use the normal distribution to find Sammy's chances of having results in certain ranges How likely is Sammy to win more than twice his mean winrate? He will exceed twice his mean winrate 18% of the time given a 100k hand sample. Link: http://davidmlane.com/hyperstat/z_table.html Months like these can easily skew our perception of our win-rate. Poker players have a natural tendency to believe that these extra-successful months are owed to us for all our past bad luck when it all actuality it is just a stretch of positive variance. We all will have a lot of both positive AND negative variance. We often don't recognize the 'run good' for what it is: simply 'run good'! This leads to a tendency to overestimate our winrate.

Because poker is a high-variance game, most of the time we will either run good or run bad. We very rarely will run "normal". How likely is Sammy to run close to his winrate? How about breaking even? Sammy has about a 12% chance of breaking even over 100k hands (I compute break-even as lying between -500 and +500 bb).

He has a 18% chance of running hot (>4000 bb), and a 12% chance of breaking even over his next 100k hands. These numbers are suprising similar! The Dude The Dude is the guy who is crushing the stakes. Let's give him a win-rate of 8bb/100 and a slightly larger standard deviation of 80bb/100. The Dude plays 1 million hands a year.

The Dude does everything a great poker player should do. He table selects well. He is completely tiltless. He is indeed... The Dude.

How much can The Dude's winrate vary just due to variance over 1 million hands? mean = 80,000

S.D. = 8,000

To see where 99.7% of our results will lie we add 3 S.D's to the mean and subtract 3 S.D.'s from the mean. 8,000*3 = 24,000

24,000 + 80,000 = 104,000 bb

-24,000 + 80,000 = 56,000 bb The "best case scenario" of 104,000 is the equivalent of winning at 10.4bb/100.

The "worst case scenario" of 56,000 is the equivalent of winning at 5.6bb/100.

Both scenarios are 30% away from The Dude's actual win-rate

68% of the time the Dude will run within 10% of his actual win-rate

If this were a $1/2 player's yearly volume it would be the difference in an income of $208,000 and $112,000, just due to variance! Some things to learn from The Dude The Dude is winning at a high win-rate and even he can experience big swings. If you have a smaller win-rate, you can experience even greater swings.

Even a very good player can run quite hot over a large sample.

Running hot and seeing really amazing results doesn't tell us as much as we would intuitively believe.

A reasonable winner can look like The Dude over a reasonable sample.

Don't let other players' amazing graphs get your down (selection bias).

The guys who quickly ran to the nose bleed stakes, while certainly awesome players, had some good luck on their side to make the exceptional run. Of course, it would be easier to get those sort of runs in juicier games (which provide higher win-rates, leading to less negative variance). What about a good solid player? Say we have a player who wins at 6bb/100 and a S.D. of 70/100 hands over a 170k sample. His next 100k hands he breaks even. 0.3% What can we gather from this? Assuming that our observed 6bb/100 winner is a strong player who studies and thinks about the game correctly, it is unlikely that he is actually a break-even player. However, it is also fairly unlikely that he is a 6bb/100 winner. He probably ran hot.

It is also fairly likely that his play has deteriorated due to his bad run, which caused his winrate to go down, which causes the probability of a bad run to increase, which causes his play to deteriorate, which causes his win-rate to go down.... you can see the vicious cycle. Don't let your observed win-rate influence your emotional state or your self-perception.

We really can only assess our level of play via hand analysis and observation.

Be aware of variance. Don't let it cloud your judgment or affect your learning.

Winning players run hot over large samples and we will often (always?) hear about them.

Don't count on your winrate being high. You never really know!

The long run is really, really long.

Don't let the vicious cycle continue. Keep studying. Keep learning. Set goals that aren't related to results (more on this in future episodes). What is a win-rate?

What is variance?

What is standard deviation?

What is fair to expect?

How long is the long run? Win-rates, standard deviation, mathematical variance

Recognizing variance and its effects

The Learning Process

Developing a Learning Plan

Identifying variance

Identifying "run good"/"run bad"

Understanding variance's effect on the mind

Comparing results to EV How do you learn?

Learning in small chunks

The Conscious-Competance Learning Matrix

Pre/Post-Session routines

Setting attainable goals Pre-session routine

Post-session routine

The Session Cycle

Executing Your Strategy

Using spreadsheets

Using PokerStove

Taking time off A player's win-rate is the average amount he or she wins per some sample size or time (hourly). It is most commonly expressed in terms of big blinds per 100 hands (bb/100). In terms of statistical calculations, this is our "mean". To calculate the mean we simply multiply the probability of all the events by their respective value and sum the results.

Say we want to calculate the mean of a coin flip. Let's give heads a value of 1 and tails a value of 0. Thus...

mean = P(heads)*(value of heads) + P(tails)*(value of tails)

mean = (0.5)*(1) + (0.5)*(0) = 0.5 + 0 = 0.5 Variance is a mathematical measurement of how far the results may fall from the mean.

Variance is often used in the poker community to talk about negative swings (this causes some ambiguity), but this is only half of the true mathematical definition. There is positive variance, too!

Many poker players tend to downplay positive variance and exaggerate negative variance. This leads to a skewed outlook on the game (more on this later). Variance is calculated by multiplying the probability of an outcome by the square of its distance from the mean. Sum this number over all possible outcomes, and you have the variance.

variance = P(outcome 1)[(value of outcome 1 - mean)^2] +... p(outcome n)[(value of outcome n - mean)^2]

Using the coin flip example (mean = 1/2, value of heads = 1, value of tails = 0):

variance = p(heads)*[(value of heads - mean)^2] + p(tails)*[(value of tails - mean)^2]

variance = (1/2)*[(1 - 1/2)^2] + (1/2)*[(0 - 1/2)^2]

= (1/2)*[(1/2)^2] + (1/2)*[(-1/2)^2]

= (1/2)*[1/4] + (1/2)*[1/4]

= 1/8 + 1/8 = 1/4

variance (coin flip) = 1/4 It's the square root of variance

It is more useful than the variance (more on this later)

With standard deviation and win-rate we can mathematically determine what to expect from results

Standard deviation of a coin flip is sqrt(1/4) = 1/2 Let's say that "close" means within 10% of his mean winrate.

10% 0f 2000 = 200

2000 + 200 = 2200

2000 - 200 = 1800

So, we are looking for the probability that Sammy's results will lie between 1800 and 2200 bb. 7.2% Only about 7.2% of Sammy's 100k hand samples will end up within 10% of his mean winrate. And this is over what most poker players would believe is a large sample! This goes to show how much variance there can be in poker, even over great numbers of hands. A yearly breakdown using these numbers ~2.2 months of running hot (> 4000 bb)

~1.4 months of break even (-500 to 500 bb)

~1.6 months losing at least 5BI (< -500 bb)

~6.8 months (500 to 4000 bb) About 30% of the time Sammy has a "run good" month or a "run bad" month. They are similarly likely. That's fairly often.

Even though the hot months occur about as frequently as the cold months, we tend to feel like the hot months are a more accurate depiction of our poker prowess. By that logic, we should feel that our cold months are also accurate indicators of our prowess (sometimes we do!). Of course, neither of these are even close to reliable indicators.

All of those months when you feel like you did alright, but could have run better, you probably ran pretty close to average. How often does Sammy lose more than 5BI's? 13% Of course, both of these results are outliers, but it goes to show how extreme our results can get.

In fact, his results will lie in the range of 9.6bb/100 to 10.4bb/100 only 2% of the time. His results will lie in range of 5.6bb/100 to 6.4bb/100 2% of the time as well. On pretty graphs There are a lot of good players out there. A lot of them are playing a lot of hands. A hypothetical 6bb/100 winner will have his next 300k hands run at 10bb/100 just 1% of the time. However, if you consider how many players are out there, that most of them have statistical tracking software, and that many will love to show off their pretty graphs, it is not too far-fetched to imagine that you are going to see these graphs pop up from time to time. While it almost certainly means that player is quite strong, it doesn't ensure that he is the best player in the game. Not even close. There is a strong selection bias here as poker players will rarely post mediocre graphs, unless to complain about how bad they are running.

Most players graphs aren't THAT pretty. How likely will a 6bb/100 winner break even over his next 100k hands? How likely will a breakeven player win 6bb/100 over his next 100k hands? This is why poker can be so frustrating! Tolerance This guy knows the mean! Mean?