The Math Forum

Ask Dr. Math - Questions and Answers from our Archives
Associated Topics || Dr. Math Home || Search Dr. Math

Probabilities in a Dice Game Using Markov Analysis

Date: 03/22/2004 at 21:36:05
From: John
Subject: probability

Anne, Bob and Carmel are going to take turns rolling a fair die, in 
the order Anne, Bob, Carmel, Anne, Bob Carmel, etc.  The first person 
to roll a 6 will win $100.

a) Find the probability that Anne will win $100 if the first four 
numbers rolled are 3, 2, 5, 5.

b) Find the probability that Anne will win $100 if a draw is to be 
declared with nobody winning $100 in the event of a 6 not being 
rolled within the first 8 rolls in total.

c) Suppose that a draw is to be declared if ever two 1's are rolled 
in a row (by two different players).  Also suppose that the game is 
in fact over and did not end in a draw.  Find the probability that 
Anne is the player who won $100.

I was able to do part (a) by directly summing an infinite series 
and I checked this by performing a first step analysis.  However, I 
am unsure how to solve the rest of the problems.

Date: 03/25/2004 at 19:57:37
From: Doctor Mitteldorf
Subject: Re: probability

Dear John,

The Markov method uses "state vectors" and matrices, and is probably
more abstract than you're comfortable with, unless you're an
extraordinary 17-year-old.  Here's one way to solve a problem like
this without having to use an infinite series.

The probability of A winning on her first roll is (1/6).  The
probability that no one prevails in round 1, so we have to go to round
2, is (5/6)^3.  But now A is in the same situation she was in at the
beginning of round 1.  We can use this fact to make an equation.

Let's say A's total probability of winning is p.  Then one part of p
is the probability she'll win in the first round, (1/6).  The second
part of p is just p times the probability that she'll make it to round
2.  So our equation is

   p = 1/6 + (5/6)^3 * p

   We can solve this for p, to find p = 36/91 = 0.3956.

I have been mulling your problem (c) with the two-1's rule for much
of the day, and I don't have a handy explanation.  The best I can do
is a Markov analysis, or a computer simulation.  Computer simulations
are my specialty--they're fun and they're easy for problems like this, 
and they give you an instant answer.  If you have a little knowledge
of BASIC or C++, perhaps you'd like to try it.  The answer I get is 
that Ann wins 0.35357 of the time.

Incidentally, the same computer simulation has convinced me that the 
"draw" situation with two-1's in a row occurs exactly 0.125 or 1/8 of 
the time.  Here's the reasoning and the algebra that supports that

Think of the "system" as having two states.  As long as the game
isn't over, then either a 1 has just been rolled, or anything else has
just been rolled (2,3,4,5).  Let u be the probability of two 1's
eventually preceding a 6 if we're in state 1, and let v be the
probability of a draw eventually starting in state 2.  We'll write two
equations connecting u and v that come from the probabilities for the
next roll of the die, starting from state 1 or state 2.

    Starting in state 1 (a 1 has just been rolled),
             there's a 1/6 chance of a 1, ending in a draw
             there's a 1/6 chance of a 6, ending in a win
             there's a 4/6 chance of anything else -> state 2

From this, we write the equation:

    u = 1/6 (1) + 1/6 (0) + 4/6 (v)  (our first equation)  

    Starting in state 2 (a 2,3,4 or 5 has just been rolled, or just

             there's a 1/6 chance of a 1, -> state 1
             there's a 1/6 chance of a 6, ending in a win
             there's a 4/6 chance of anything else -> state 2

From this, we write the equation:

    v = 1/6 (u) + 1/6 (0) + 4/6 (v)  (our second equation) 

Solve the two equations together, to find that v = 1/8, agreeing with
my numerical experiment.  It also tells us that u = 1/4.

This method is in the spirit of Markov analysis, because we're 
thinking in terms of "state space" and "transition probabilities".  In
a full Markov analysis, you'd make a matrix of transition 
probabilities from each possible state to every other possible state.

This suggests how we might go about solving the full problem, the
probability of Ann winning.  It also suggests why it might be messy
and tedious!

In the full problem, there are 6 states in the system:  Either a 1
has just been rolled, or not.  And Ann, Bob, or Carmel might be next 
to roll.  Thus 2 * 3 = 6 states.
Using the same kind of analysis that I just illustrated in the simple 
problem, you could define 6 probabilities u,v,w,x,y and z for Ann 
winning starting in each of the 6 states.  Then you could write down 6 
equations that describe the probabilities for going from each of those 
6 states to each of the other states.  Then you'd have 6 equations in 
6 unknowns, and you could solve the whole mess.
Of course, it won't be quite so bad, because the 3 players rotate
in fixed order, so, for example, the equation for u will just have a w
and an x in it, but no v, y and z.  Still it might take some thinking
to write down the equations, and a few pages of algebra to solve them.

Let me know if you take it on...           

- Doctor Mitteldorf, The Math Forum 
Associated Topics:
College Probability
High School Probability

Search the Dr. Math Library:

Find items containing (put spaces between keywords):
Click only once for faster results:

[ Choose "whole words" when searching for a word like age.]

all keywords, in any order at least one, that exact phrase
parts of words whole words

Submit your own question to Dr. Math

[Privacy Policy] [Terms of Use]

Math Forum Home || Math Library || Quick Reference || Math Forum Search

Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.