Associated Topics || Dr. Math Home || Search Dr. Math

### Random Variables and Order Statistics

```
Date: 08/16/2001 at 00:42:11
From: Sam
Subject: Random variables

Can you give me some hints concerning the following problem?

Let X1, X2, ....., Xn be n independent and continuous random variables
with the same distribution.

Let X denote the maximum and Y the minimum of these random variables.

Let U = X/Y.

Find the distribution of X, Y, and U.
```

```
Date: 08/16/2001 at 11:57:35
From: Doctor Jordi
Subject: Re: Random variables

Hello, Sam - thanks for writing to Ask Dr. Math.

The problem you propose is an exercise in order statistics and
transformations. Let us first look at the distribution of your maximum
order statistic and of your minimum.  I will denote the minimum
observation by X_min and the maximum by X_max.

Suppose you have ordered your sample of n Xi observations. That is,
say we have a streak such that

W1 < W2 < ... < Wn

where every Wi corresponds to some Xi and X_min = W1 = min(X1, X2,
..., Xn) and X_max = Wn = max(X1, X2, ..., Xn).  Let F_X(x) denote the
cumulative distribution function of the Xi. Given that the Xi are
i.i.d., the distribution of each may be found according to the
following argument.

We will proceed by first finding the distribution function of X_max.
That is, we seek F_max(x) = P(X_max < x). Now, think about this
problem this way: the probability that the maximum is less than any
given number is not an event independent of the other observations.
In fact, X_max will be less than any given x if and only if all the
other observations are also less than this given x.

What is the probability that W1 is less than x (hint: what is the
meaning of the cumulative distribution function F_X(x) of W1)?  What
is the probability that W2 is also less than x? What is the
probability that W1 AND W2 are less than x, given that W1 and W2 are
independent (how do you find the probability of two _independent_
events)?  Therefore, what is the probability that W1 AND W2 AND W3 AND
... AND Wn = X_max are all less than x?

I hope the above hints are enough for you. Once you have found the
cumulative distribution function of X_max, remember to differentiate
to obtain its density function.

The argument for finding the distribution of the minimum proceeds
similarly. That is, we seek

F_min(x) = P(X_min < x) = 1 - P(X > x)

and we know that the minimum will be greater than any given x if and
only if all the other observations are greater than this given x.
Remember that

P(Xi > x) = 1 - P(Xi < x) = 1 - F_X(x)

The last step is to find the distribution function of the quotient of
these two random variables. This can be done with similar reasoning.

We seek

F_U(u) = P(U < u) = P(X_max / X_min < u)

That is, the probability of the set (x_max, x_min) such that
x_max / x_min < u. If x_min is greater than 0, then this will be the
set x_max < u*x_min. If x_min is less than 0, the set in question will
then be x_max > u*x_min. This allows us to set up the following double
integrals to evaluate the probability in question:

F_U(u) = Int[-oo, 0]Int[u*x_min, oo] f(x_max, x_min) dx_max dx_min  +
Int[0, oo]Int[-oo, u*x_min] f(x_max, x_min) dx_max dx_min

Where the symbol Int[a, b] f(x)dx is used to denote the definite
integral of f(x) evaluated from a to b. (I've omitted the appropriate
subscripts of the density functions for better readability). To
evaluate these integrals, it is helpful to make the change-of-variable
x_max = x_min*v in the inner integrals.

Does this help? Please write back if you feel I need to explain myself
better or if you would like to talk about anything else.

- - Doctor Jordi, The Math Forum
http://mathforum.org/dr.math/
```

```
Date: 08/16/2001 at 18:39:16
From: Sam
Subject: Re: Random variables

I am extremely thankful for the answer you provided, and I think I got
the idea concerning the calculation of the first part. However,
concerning the part that deals with the ratio ( max / min ) , I am
still wondering how I can calculate the joint p.d.f (f (x_max, x_min))
you used in the double integrals.

Is there any general formula that allows me to calculate it for the
ordered set of random variables you talked about at the beginning?

If yes I would be glad to receive your answer.

Thank you again.
Sincerely, Sam
```

```
Date: 08/17/2001 at 21:02:36
From: Doctor Jordi
Subject: Re: Random variables

Hello again.

I will treat in more detail the general case for the transformation of
the quotient of two continuous random variables (without assuming
independence).  I hope you have no difficulty applying it to the
problem at hand.

Suppose we have two random variables, X and Y, with joint density
f(x,y), and we seek the density function of Z = Y/X.  We proceed first
by finding the cdf of Z.  F_Z(z) = P(Z < z) = P(Y/X < z).

Now, this last probability can be expressed as the sum of two disjoint
events: [1]: P(Y < xz) if x > 0 and [2]: P(Y > xz) if x < 0.  (Since I
am assuming continuous random variables, it is okay to use strict
inequalities). By the additive axiom of disjoint events of the
probability function,

P(Z < z) = P(Y < xz|x > 0) + P(Y > xz|x < 0)

It is now simple to setup the appropriate integrals to evaluate this
probability.  In [1] Y ranges from -oo to xz, while x ranges from
0 to oo.  In the other summand above, Y ranges from xz to oo and x
from -oo to 0. The resulting double integrals look as follows:

F_Z(z) = Int[o,oo]Int[-oo,xz]f(x,y)dydx +
Int[-oo,0]Int[xz,oo]f(x,y)dydx

In order to remove the dependence of x in the inner limits of
integration,
we make the change-of-variable v = y/x; dv = dy/x, which gives

Int[0,oo]Int[-oo,z] xf(x,xv) dvdx + Int[-oo,0]Int[z,-oo] xf(x,xv) dvdx

(take careful note of the changes in the limits of integration and
make sure you understand why they changed as they did).

Flipping the inner limits of integration on the right integral and
using the absolute value function allows us to express this more
neatly as follows:

Int[0,oo]Int[-oo,z] xf(x,xv) dvdx + Int[-oo,0]Int[-oo,z] -
xf(x,xv) dvdx

Int[0,oo]Int[-oo,z] |x|f(x,xv) dvdx

Now, if we can use the assumption that f(x,xv) is continuous, we can
exchange the order of integration and differentiate with respect to z
to obtain the appropriate density function

f_z(z) = Int[0,oo] |x|f(x,xz) dx

as required.

I will now address your question about the distribution of the kth
order statistic.

The setup, recall, is an i.i.d. sample of n observations. We take
these observations and order them from least to greatest.  Now we want
their distribution.

Let X1, X2, ..., Xn be independent, identically distributed random
variables with common density f_x(x) and let Y1, Y2, ..., Yn be such
that every Yi corresponds to some Xi and Y1 < Y2 < ... < Yn.  (note:
since I am restricting myself to the continuous case, the previous
inequalities are strict, although to be perfectly rigorous and
general, they should be "less than or equal to").  We have already
considered previously the distribution of Y1 and Yn (the min and the
max). Let us now concentrate on the case n=2 and turn to the joint
distribution of Y1 and Y2.

It is apparent that no two-order statistics will be independent. So
whatever the joint distribution of Y1 and  Y2 is, it will not be the
product of their marginal distributions.

The compound event (Y1 < y1, Y2 < y2) means that either
(X1 < y1, X2 < y2) or (X2 < y1, X1 < y2). Notice that Y1 can be X1 or
X2, whichever one is smaller. This means that the event (Y1 < y1 and
Y2 < y2) is the union of two events. Recall the additive law of
probability that states that for two events A and B we have

P(A union B) = P(A) + P(B) - P(A intersection B).

So, since y1 < y2, for F_Y1,Y2(y1, y2)  = P(Y1 < y1, Y2 < y2) we have
the following:

P(Y1 < y1, Y2 < y2) = P(X1 < y1, X2 < y2) + P(X2 < y1, X1 < y2) -
P(X1 < y1, X2 < y2)

Since X1 and X2 are independent with common distribution F_x(x), we
can calculate

F_Y1,Y2(y1, y2) = F(y1)F(y2) + F(y2)F(y1) - F(y1)F(y1)

= 2F(y1)F(y2) - [F(y1)]^2

To obtain the joint density f_Y1,Y2(y1, y2) we differentiate first
with respect to y1 and then with respect to y2 to obtain simply

f_Y1,Y2(y1, y2) = 2f(y1)f(y2) for y1 < y2 and 0 elsewhere

This method can be applied to any number of n order statistics. In
general, we obtain the following joint density of all the order
statistics:

f_Y1,Y2,...Yn(y1, y2, ..., yn) = n!f(y1)f(y2)...f(yn)
for y1 < y2 < ... < yn
and 0 elsewhere

From this joint density, the marginal of the kth order statistic can
be found by integration.

As an example, say n = 3 and we seek the marginal density function of
Y2. We set up the following double integral and "integrate out" y1
and y3. The limits of integration are set by the requirement that
y1 < y2 < y3.

f_Y2(y2) = Int[-oo,y2]Int[y2,oo] 3! f(y1)f(y2)f(y3) dy1dy3

= 3!f_X(y2)F_X(y2)[1 - F_X(y2))

Although this procedure can be generalized to obtain the density of
the kth statistic, it is easier to proceed by a heuristic differential
argument, which can be made rigorous if desired.

Think of the density function of a continuous random variable as an
approximate measure of the probability that the random variable
realizes a value in that neighbourhood.  P( x < X < x + dx) is
approximately equal to f(x)dx for small dx. These approximations
become exact in the limit as dx -> 0.

Say now we seek the marginal density function f_Yk(yk) of the kth
order statistic. This can be done if we consider the problem as an
exercise in applying the multinomial distribution. We have n
observations. We want to know the probability that k-1 of those
observations are less than yk, that exactly one of them is close to
yk, and that the remaining n-k are greater than yk. The following
table summarizes a bit of this information.

Class                  How many of them       Associated probability
--------------------------------------------------------------------
Yi's less than yk           k-1                 [F_X(yk)]^(k-1)

Yi's close to yk             1                   f_X(yk)

Yi's greater than yk        n-k                 [1 - F_X(yk)]^(n-k)

From the multinomial distribution, we now compute the probability of
the event f_X(yk) = P("Yk is close to yk").

n!
f_Yk(yk) = --------------- [F_X(yk)]^(k-1)*f_X(yk)*[1 - F_X(yk)]^(n-k)
(k-1)! 1! (n-k!)

An entirely analogous argument allows us to find the joint density of
any two-order statistics. You will have to use this when you find the
joint density of the maximum and the minimum. HINT: If two-order
statistics are under consideration then there will be 5 classes for
the multinomial distribution instead of 3. What are these classes?

This should be enough to help you answer your question.  If you still
need anything else, please let us know.

- Doctor Jordi, The Math Forum
http://mathforum.org/dr.math/
```
Associated Topics:
College Probability

Search the Dr. Math Library:

 Find items containing (put spaces between keywords):   Click only once for faster results: [ Choose "whole words" when searching for a word like age.] all keywords, in any order at least one, that exact phrase parts of words whole words

Submit your own question to Dr. Math
Math Forum Home || Math Library || Quick Reference || Math Forum Search

Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.
http://mathforum.org/dr.math/