Associated Topics || Dr. Math Home || Search Dr. Math

### Length of a Nautical Mile at a Given Latitude

```Date: 11/06/2002 at 15:58:11
From: Keith Shears
Subject: Length of a Nautical Mile in a given Latitude

As the actual length of a nautical mile varies with latitude, I am
trying to work out a formula for accurately predicting the exact
length of a nautical mile in any given latitude, in metres. I believe
this may require the use of some form of spheroidal calculation to
allow for the flattening for the earths ellipsoid shape. I was given
one formula as follows:

true length of a nautical mile = 1852.3 - 9.4 Cosine(2 x Latitude)

But I have no reference or proof of this formula or its origins. It
would appear to use the length of a nautical mile at Latitude 45
degrees - so I can't prove whether this is accurate or not.
```

```
Date: 11/07/2002 at 10:41:01
From: Doctor Rick
Subject: Re: Length of a Nautical Mile in a given Latitude

Hi, Keith.

I have never heard anyone say that the nautical mile is not a fixed
distance. See the How Many? web site linked from the Dr. Math FAQ:

A Dictionary of Units of Measurement - Russ Rowlett
http://www.unc.edu/~rowlett/units/dictN.html

It says:

"nautical mile (nmi, naut mi or NM)

"a unit of distance used primarily at sea and in aviation. The
nautical mile is defined to be the average distance on the Earth's
surface represented by one minute of latitude. This may seem odd to
landlubbers, but it makes good sense at sea, where there are no mile
markers but latitude can be measured. Because the Earth is not a
perfect sphere, it is not easy to measure the length of the nautical
mile in terms of the statute mile used on land. For many years the
British set the nautical mile at 6080 feet (1853.18 meters), exactly
800 feet longer than a statute mile; this unit was called the
Admiralty mile. Until 1954 the U.S. nautical mile was equal to
6080.20 feet (1853.24 meters). In 1929 an international conference in
Monaco redefined the nautical mile to be exactly 1852 meters or
6076.115 49 feet, a distance known as the international nautical
mile. The international nautical mile equals about 1.1508 statute
miles. There are usually 3 nautical miles in a league. The unit is
designed to equal 1/60 degree [2], although actual degrees of
latitude vary from about 59.7 to 60.3 nautical miles."

The nautical mile is the AVERAGE distance of a minute of latitude,
which is a fixed distance, not the actual distance at each particular
point (which does vary with latitude due to the non-sphericity of the
earth).

Perhaps you are really asking for a formula for the length of a minute
of latitude at any latitude on an ellipsoidal model of the earth.

The radius of curvature in the meridian (north-south plane) is greater
at the poles and less at the equator. The length of a minute of
latitude is proportional to the radius of curvature. This is
consistent with your formula. Let's check out the details.

The formula for the radius of curvature in the meridian at a given
latitude is

M = a(1-e^2)/(1-e^2*sin^2(lat))^3/2

where a is the semi-major axis (radius of the earth at the equator)
and e is the eccentricity of the earth. In the World Geodetic System
(WGS),

a = 6378137 m
e^2 = 0.00669438002290

Thus

M = 6335439.32708 / (1 - 0.00669438002290*sin^2(lat))^3/2

One minute of arc is 1/60 degree, and a degree is pi/180 radians, so

1 min arc = pi/10800 = 0.000290888208665 radian

We multiply this by the radius of curvature M to get the length of a
minute of arc:

1842.90459693 / (1 - 0.00669438002290*sin^2(lat))^3/2

This does not look like

1852.3 - 9.4*cos(2*lat)

However, when I calculate both formulas using a spreadsheet, the
differences aren't so great:

Lat  Yours         Mine
------------------------------
0    1842.9        1842.904597
10   1843.466889   1843.462752
20   1845.099182   1845.071468
30   1847.6        1847.540708
40   1850.667707   1850.57721
50   1853.932293   1853.817733
60   1857          1856.871458
70   1859.500818   1859.367085
80   1861.133111   1860.999007
90   1861.7        1861.566326

Maybe with a little work we could put our formulas into forms that
would show the relation more clearly. I'll leave it at this, though,
unless you want to pursue it further.

- Doctor Rick, The Math Forum
http://mathforum.org/dr.math/
```

```
Date: 11/07/2002 at 14:48:18
From: Keith Shears
Subject: Thank you (Length of a Nautical Mile in a given Latitude)

A big thank you for your efforts; at least now I have two formulae,
and I can see the proof of one of them - yours - and understand its
origins!

Keith
```

```
Date: 11/07/2002 at 15:17:46
From: Doctor Rick
Subject: Re: Thank you (Length of a Nautical Mile in a given Latitude)

Hi, Keith.

I thought I'd take a quick look at how my formula might be
approximated. Using the fact that

(1+x)^n = 1 + nx + (n(n-1)/2!)x^2 + ...

we can approximate my formula

1842.90459693 / (1 - 0.00669438002290*sin^2(lat))^3/2

as

1842.90459693(1 + (3/2)0.00669438002290*sin^2(lat)) =
1842.90459693 + 18.5056555767sin^2(lat)

Since sin^2(lat) = 1 - cos^2(lat), this equals

1842.90459693 + 18.5056555767(1-cos^2(lat)) =
1861.4102525067 - 18.5056555767cos^2(lat)

cos(2x) = 2*cos^2(x) - 1

cos^2(x) = (1+cos(2x))/2

the approximate formula becomes

1861.4102525067 - 18.5056555767(1+cos(2*lat))/2 =
1852.15742471835 - 9.25282778835cos(2*lat)

1852.3 - 9.4*cos(2*lat)

We're pretty close. We can get an idea of the error in the
approximation by looking at the next term in the series:

(n(n-1)/2!)x^2 = (3/2)(1/2)/2 * (0.00669438002290*sin^2(lat))^2 =
0.0000168055214591*sin^4(lat)

This error is greatest at the poles, where sin(lat) = 1 and we get an
error of 0.0000168 * 1842.90459693 = 0.031 meters. The difference
between our formulas at 90 degrees latitude was 0.14 meter; but then,
the coefficients weren't the same as in my approximation.

- Doctor Rick, The Math Forum
http://mathforum.org/dr.math/
```
Associated Topics:
College Higher-Dimensional Geometry
High School Higher-Dimensional Geometry

Search the Dr. Math Library:

 Find items containing (put spaces between keywords):   Click only once for faster results: [ Choose "whole words" when searching for a word like age.] all keywords, in any order at least one, that exact phrase parts of words whole words

Submit your own question to Dr. Math
Math Forum Home || Math Library || Quick Reference || Math Forum Search