Best Line Fit: Least Squares

Date: 03/23/97 at 18:27:36
From: Mike Litrownik
Subject: Best Line Fit: Least Squares

Hi there. I'm a freshman in high school and in math now we're plotting 
data points and trying to find a line of best fit. We figured out that 
by using our calculators and using a linear regression it will give us 
the exact line of best fit. I know it uses something called least 
squares fitting or something like that to determine the line. My 
question is: What is least squares fitting? How exactly does it do it? 

Also, with cubic and quartic regressions and logistic and other 
regressions...how exactly do they work? 

Any help would be appreciated. 
Thanks.

Date: 03/23/97 at 19:46:32
From: Doctor Steven
Subject: Re: Best Line Fit: Least Squares

Imagine you're trying to fit a line through and you want it to be as 
close as possible to the points.  One way to do this is to minimize 
the sum of the distances of the points from the line.  But this might 
lead to the line passing through some points while other points are 
very far away from the line.  So instead what least squares fitting 
does is tries to get every point as equally close to the line as it 
can get it.  The way to do this is to make sure that no point is far 
from the average distance away from the line.  In statistical terms 
this means you are trying to minimize the standard deviation of the 
distances of the points from the line.

We use squares of numbers to figure out the standard deviation.  
If a point is really far away from the line that really increases the 
standard deviation since the distance away from the line is squared 
which makes its value much bigger.  On the other hand if the distance 
between the point and the line is small, then squaring it makes it 
even smaller. (.5^2 = .25) So we get this effect of really disliking 
points far away from the line and really liking points close to the 
line.  This creates a fit that gives each point close to an average 
distance away from the line.  The actual equations used to get this 
line are pretty complicated but if you continue to take courses in 
math in college you'll probably encounter it at the freshman level.

Cubic and quartic regressions rely on cubic and quartic equations 
which model the data points.  The trick is to pick three or four 
points at random and then create a cubic or quartic polynomial to fit 
them and hope it fits the rest of the data points.  Calculators would 
probably do several of these and then choose the best fit out of the 
bunch.  Logistic regression use logarithmic expressions to fit a set 
of data points in much the same way as cubic and quartic regressions 
do.

Hope this helps.

-Doctor Steven,  The Math Forum
 Check out our web site!  http://mathforum.org/dr.math/