Length of a Nautical Mile at a Given Latitude
Date: 11/06/2002 at 15:58:11 From: Keith Shears Subject: Length of a Nautical Mile in a given Latitude As the actual length of a nautical mile varies with latitude, I am trying to work out a formula for accurately predicting the exact length of a nautical mile in any given latitude, in metres. I believe this may require the use of some form of spheroidal calculation to allow for the flattening for the earths ellipsoid shape. I was given one formula as follows: true length of a nautical mile = 1852.3 - 9.4 Cosine(2 x Latitude) But I have no reference or proof of this formula or its origins. It would appear to use the length of a nautical mile at Latitude 45 degrees - so I can't prove whether this is accurate or not.
Date: 11/07/2002 at 10:41:01 From: Doctor Rick Subject: Re: Length of a Nautical Mile in a given Latitude Hi, Keith. I have never heard anyone say that the nautical mile is not a fixed distance. See the How Many? web site linked from the Dr. Math FAQ: A Dictionary of Units of Measurement - Russ Rowlett http://www.unc.edu/~rowlett/units/dictN.html It says: "nautical mile (nmi, naut mi or NM) "a unit of distance used primarily at sea and in aviation. The nautical mile is defined to be the average distance on the Earth's surface represented by one minute of latitude. This may seem odd to landlubbers, but it makes good sense at sea, where there are no mile markers but latitude can be measured. Because the Earth is not a perfect sphere, it is not easy to measure the length of the nautical mile in terms of the statute mile used on land. For many years the British set the nautical mile at 6080 feet (1853.18 meters), exactly 800 feet longer than a statute mile; this unit was called the Admiralty mile. Until 1954 the U.S. nautical mile was equal to 6080.20 feet (1853.24 meters). In 1929 an international conference in Monaco redefined the nautical mile to be exactly 1852 meters or 6076.115 49 feet, a distance known as the international nautical mile. The international nautical mile equals about 1.1508 statute miles. There are usually 3 nautical miles in a league. The unit is designed to equal 1/60 degree , although actual degrees of latitude vary from about 59.7 to 60.3 nautical miles." The nautical mile is the AVERAGE distance of a minute of latitude, which is a fixed distance, not the actual distance at each particular point (which does vary with latitude due to the non-sphericity of the earth). Perhaps you are really asking for a formula for the length of a minute of latitude at any latitude on an ellipsoidal model of the earth. The radius of curvature in the meridian (north-south plane) is greater at the poles and less at the equator. The length of a minute of latitude is proportional to the radius of curvature. This is consistent with your formula. Let's check out the details. The formula for the radius of curvature in the meridian at a given latitude is M = a(1-e^2)/(1-e^2*sin^2(lat))^3/2 where a is the semi-major axis (radius of the earth at the equator) and e is the eccentricity of the earth. In the World Geodetic System (WGS), a = 6378137 m e^2 = 0.00669438002290 Thus M = 6335439.32708 / (1 - 0.00669438002290*sin^2(lat))^3/2 One minute of arc is 1/60 degree, and a degree is pi/180 radians, so 1 min arc = pi/10800 = 0.000290888208665 radian We multiply this by the radius of curvature M to get the length of a minute of arc: 1842.90459693 / (1 - 0.00669438002290*sin^2(lat))^3/2 This does not look like 1852.3 - 9.4*cos(2*lat) However, when I calculate both formulas using a spreadsheet, the differences aren't so great: Lat Yours Mine ------------------------------ 0 1842.9 1842.904597 10 1843.466889 1843.462752 20 1845.099182 1845.071468 30 1847.6 1847.540708 40 1850.667707 1850.57721 50 1853.932293 1853.817733 60 1857 1856.871458 70 1859.500818 1859.367085 80 1861.133111 1860.999007 90 1861.7 1861.566326 Maybe with a little work we could put our formulas into forms that would show the relation more clearly. I'll leave it at this, though, unless you want to pursue it further. - Doctor Rick, The Math Forum http://mathforum.org/dr.math/
Date: 11/07/2002 at 14:48:18 From: Keith Shears Subject: Thank you (Length of a Nautical Mile in a given Latitude) A big thank you for your efforts; at least now I have two formulae, and I can see the proof of one of them - yours - and understand its origins! Keith
Date: 11/07/2002 at 15:17:46 From: Doctor Rick Subject: Re: Thank you (Length of a Nautical Mile in a given Latitude) Hi, Keith. I thought I'd take a quick look at how my formula might be approximated. Using the fact that (1+x)^n = 1 + nx + (n(n-1)/2!)x^2 + ... we can approximate my formula 1842.90459693 / (1 - 0.00669438002290*sin^2(lat))^3/2 as 1842.90459693(1 + (3/2)0.00669438002290*sin^2(lat)) = 1842.90459693 + 18.5056555767sin^2(lat) Since sin^2(lat) = 1 - cos^2(lat), this equals 1842.90459693 + 18.5056555767(1-cos^2(lat)) = 1861.4102525067 - 18.5056555767cos^2(lat) Your formula had cos(2*lat). Using cos(2x) = 2*cos^2(x) - 1 cos^2(x) = (1+cos(2x))/2 the approximate formula becomes 1861.4102525067 - 18.5056555767(1+cos(2*lat))/2 = 1852.15742471835 - 9.25282778835cos(2*lat) Compare this with your formula: 1852.3 - 9.4*cos(2*lat) We're pretty close. We can get an idea of the error in the approximation by looking at the next term in the series: (n(n-1)/2!)x^2 = (3/2)(1/2)/2 * (0.00669438002290*sin^2(lat))^2 = 0.0000168055214591*sin^4(lat) This error is greatest at the poles, where sin(lat) = 1 and we get an error of 0.0000168 * 1842.90459693 = 0.031 meters. The difference between our formulas at 90 degrees latitude was 0.14 meter; but then, the coefficients weren't the same as in my approximation. - Doctor Rick, The Math Forum http://mathforum.org/dr.math/
Search the Dr. Math Library:
Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.