Date: 10/26/96 at 12:37:54 From: Charles E. Knoop, Jr. Subject: latitude longitude Dear Dr. Math, I used a formula I found on the Web for calculating distances between the two (latitude, longitude) points of (0382203N, 0854546W) and (0292127N,1005548W). I came up with 1068.819. Using another software program that calculates air miles, I got 1269.8, which I thought should be less than the arc on the earth. I decided to use a formula I used in geometry - sqr(a) + sqr(b) = sqr(c) a = lat1-lat2 in seconds b = long1-long2 in seconds c = sqrt(sqr(a)+sqr(b)) distance for one second = circumference/(360*3600) distance = (distance for one second) * c Using the above formula I came up with 1220.299. My question is: Why is the arc length so much shorter than using the Pythagorean formula? Thanks in advance, Charlie
Date: 10/29/96 at 20:7:14 From: Doctor Donald Subject: Re: latitude longitude First of all, the formula from geometry is not correct in general since points with the same longitudinal separation (say, 1 degree) are much closer together if they lie near the poles than if they lie near the equator. Two points with the same latitude and differing by one degree of longitude are about 70 statute miles apart at the equator and about 92 feet apart if they are within a mile of the north pole. The formula from geometry is close to correct for points near the equator and too large everywhere else. You can correct the Pythagorean theorem so that it becomes a better approximation by adjusting it for latitude, but it is basically just wrong and you shouldn't use it. Latitude and longitude lines do not form triangles, they form "spherical triangles." The difference between the two answers (air miles versus the other one) could possibly be accounted for by one of the calculations yielding nautical miles (the one which gave 1068 as the answer) and the other using statute miles. 1068 nautical miles is 1.15*1068 = 1230 statute miles. I'm not sure whether you were calculating statute or nautical miles with your Pythagorean formula - it depends on the number of miles/degree that you used. As you see, converting 1068 to statute miles doesn't come very close to 1269, but it might still be the main thing operating. I suspect 1068 is correct in some set of units, probably nautical miles, and the air miles calculation is just wrong. I don't know a formula for the problem, but an accurate formula would take into account the flattening of the earth at the poles and so on. -Doctor Donald, The Math Forum Check out our web site! http://mathforum.org/dr.math/
Search the Dr. Math Library:
Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.