Line of Best Fit Using Least Squares Method
Date: 06/08/2005 at 00:12:07 From: Matthew Subject: Least Squares for an Eighth Grader In my geometry class, we were given a worksheet. In each of the problems we were given several points and we had to find the best fitting line. In a recent issue of "College Mathematics Journal" there were many references to the least squares method. I've done some research and found that this can be used to find the best line. Unfortunately, for the most part, the web sites I've used get a bit technical. I understand all the concepts (okay, I don't know how to work with partial derivatives, but other than that...), but I haven't found the explanations very lucid. Could you possibly explain the least squares method of linear regression in simple terms that an eighth grader (who has taken Algebra 2 and knows a bit about differentiation) could understand? I researched the method on Wikipedia and Mathworld. I also glanced through a calculus textbook that was lying around in my geometry classroom, but I couldn't find anything.
Date: 06/08/2005 at 10:46:27 From: Doctor Ian Subject: Re: Least Squares for an Eighth Grader Hi Matthew, Here's one way to understand it. Suppose you have a bunch of data points, and you just draw _any_ line to represent that data. (It doesn't have to be a particularly good fit--in fact, it could be a terrible fit.) Now, from each point in the data, you can draw a vertical line segment that connects the data point to the line. The length of the segment is the distance of the point from the line. And you can add up all those distances. If you do this with two different lines, the sum of the distances will be different for the two lines. In particular, for one of them it will be smaller. One way of comparing the goodness of fit for the lines is to say that the smaller sum indicates the better fit. (Note that this is just one of many possible ways that we might make such a comparison. For example, instead of measuring distances vertically, we might measure them perpendicular to the line. But this would be a lot more complicated, and it's not clear that it would provide a 'better' measure. But it would certainly provide a different one.) One improvement we can make is that, instead of minimizing the sum of the distances, we minimize the sum of the _squares_ of the distances-- it gives you the same result, but without having to compute a lot of square roots, which are expensive operations. Does this make sense so far? Now we're in sort of the same situation we're in when we want to find the square root of some number that isn't a perfect square, e.g., the square root of 7429. The simplest method is to make a guess, square it, and let the result (is it too high, or too low?) tell us how to make our next guess. Similarly, we could make guesses with lines (to make a guess, we have to pick a slope and an intercept), and use those guesses to make better guesses, until we get a fit that is "good enough". Or, as we learn more math, we can start getting more sophisticated. In the case of square roots, we can use calculus to create techniques that will move us more quickly towards the actual answer. This works by looking not just at how close our answer is, but at how much of a change in the answer we'd expect to see if we change our guess by any given amount. (This is where derivatives--which measure rate of change--enter the picture.) So now we can move, not just in the right direction, but by an optimal amount in that direction. Similarly in the case of least squares: we can look at not just how small the sum of squares is for the line that represents a given guess, but at how we would expect the sum to change if we change the parameters that determine the line. But that's the basic idea. As a physical model, you can imagine that each data point is represented by a pin stuck in a board, and your line is a stick. If you connect the stick to each pin with a rubber band, there will be some way to place the stick so that the various forces exerted by the rubber bands cancel out. In any other configuration, they'll be working to move the stick by pulling it or twisting it. Because of the way rubber bands actually work, this would give you a different answer than the least squares operation. But in the sense that rubber hands are "happier" when they're being stretched as little as possible, it has the same sense of trying to minimize the lengths of the rubber bands. Does this make sense? Is it at the level you wanted? - Doctor Ian, The Math Forum http://mathforum.org/dr.math/
Date: 06/08/2005 at 21:01:29 From: Matthew Subject: Thank you (Least Squares for an Eighth Grader) Yes, that was great! Thank you very much! -Matthew
Search the Dr. Math Library:
Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.