Re-Calculating the Standard Deviation

Date: 03/20/2002 at 11:38:49
From: Gareth Williams
Subject: Re-Calculating the Standard Deviation

Hi Dr. Math,

How can I re-calculate the standard deviation of a set of values when 
I insert a new value but I don't know what the existing values are?

For instance, I have a set of 5 values. They have an average of 20 
with a standard deviation of 12. If I add the value 26 to the set I 
can re-calculate the new average of 21, but how do I re-calculate the 
new standard deviation?

Thanks in advance,
Gareth

Date: 03/21/2002 at 03:24:51
From: Doctor Twe
Subject: Re: Re-Calculating the Standard Deviation

Hi Gareth - thanks for writing to Dr. Math.

Yes, this can be done.

If you are working with population mean and standard deviation, an 
alternate formula for the population standard deviation (the one 
calculators and computers actually use) is

               n*Sum[x^2] - (Sum[x])^2
     s = Sqrt[ ----------------------- ]
                         n^2

where 

     n        = the population size (number of data points)
     Sum[x^2] = the sum of the squares of all data points
     Sum[x]   = the sum of all data points
     
So if you could recover the values of Sum[x^2] and Sum[x], then 
the standard deviation after adding a new value, v, would be
     
                (n+1)*(v^2 + Sum[x^2]) - (v + Sum[x])^2
     s' = Sqrt[ --------------------------------------- ]
                         (n+1)^2

Recovering Sum[x] is easy: it's just n times the mean, mu:

  Sum[x] = n*mu
  
Recovering Sum[x^2] is a little trickier.  Starting from the formula,   

                                n*Sum[x^2] - (Sum[x])^2
                      s = Sqrt[ ----------------------- ]
                                          n^2


                           n*Sum[x^2] - (Sum[x])^2
                    s^2 =  ----------------------- 
                                     n^2
                        
               
                n^2 s^2 =  n*Sum[x^2] - (Sum[x])^2
   
   
   n^2 s^2 + (Sum[x])^2 = n*Sum[x^2]
  
   
     n^2 s^2 + (n*mu)^2 = n*Sum[x^2]   


     n^2 s^2 + n^2 mu^2 = n*Sum[x^2]
   

        n^2(s^2 + mu^2) = n*Sum[x^2]
   

          n(s^2 + mu^2) = Sum[x^2]
   
So from n, mu, and s, we can recover Sum[x] and Sum[x^2], which means 
we can compute s'.  

A similar computation can be made if you have a sample mean and 
standard deviation. The alternate formula for the sample standard 
deviation is:

               n*Sum[x^2]-(Sum[x])^2
     s = Sqrt[ --------------------- ]
                      n*(n-1)

So you start from there, but the rest is the same. 

I hope this helps. If you have any more questions, write back.

- Doctor TWE, The Math Forum
  http://mathforum.org/dr.math/   

Date: 03/22/2002 at 06:26:19
From: Gareth Williams
Subject: Re-Calculating the Standard Deviation

No need for a follow up; your well-explained answer told me all I 
needed to know. Thanks, Dr. Math.

Gareth