Spherical Triangles

Date: 10/26/96 at 12:37:54
From: Charles E. Knoop, Jr.
Subject: latitude longitude

Dear Dr. Math,

I used a formula I found on the Web for calculating distances between 
the two (latitude, longitude) points of (0382203N, 0854546W) and 
(0292127N,1005548W). I came up with 1068.819. Using another software 
program that calculates air miles, I got 1269.8, which I thought 
should be less than the arc on the earth. I decided to use a formula 
I used in geometry -

   sqr(a) + sqr(b) = sqr(c)
   a = lat1-lat2 in seconds
   b = long1-long2 in seconds
   c = sqrt(sqr(a)+sqr(b))

   distance for one second = circumference/(360*3600)

   distance = (distance for one second) * c

Using the above formula I came up with 1220.299.

My question is: Why is the arc length so much shorter than using the
Pythagorean formula?

Thanks in advance,
Charlie

Date: 10/29/96 at 20:7:14
From: Doctor Donald
Subject: Re: latitude longitude

First of all, the formula from geometry is not correct in general 
since points with the same longitudinal separation (say, 1 degree) are 
much closer together if they lie near the poles than if they lie near 
the equator. Two points with the same latitude and differing by one
degree of longitude are about 70 statute miles apart at the equator 
and about 92 feet apart if they are within a mile of the north pole.  
The formula from geometry is close to correct for points near the 
equator and too large everywhere else.

You can correct the Pythagorean theorem so that it becomes a better 
approximation by adjusting it for latitude, but it is basically just 
wrong and you shouldn't use it. Latitude and longitude lines do not 
form triangles, they form "spherical triangles."

The difference between the two answers (air miles versus the other 
one) could possibly be accounted for by one of the calculations 
yielding nautical miles (the one which gave 1068 as the answer) and 
the other using statute miles. 1068 nautical miles is 1.15*1068 = 1230 
statute miles. I'm not sure whether you were calculating statute or
nautical miles with your Pythagorean formula - it depends on the 
number of miles/degree that you used.  As you see, converting 1068 to 
statute miles doesn't come very close to 1269, but it might still be 
the main thing operating. I suspect 1068 is correct in some set of 
units, probably nautical miles, and the air miles calculation is just 
wrong. I don't know a formula for the problem, but an accurate formula 
would take into account the flattening of the earth at the poles and 
so on.

-Doctor Donald,  The Math Forum
 Check out our web site!  http://mathforum.org/dr.math/