The notion of the earth as a perfect sphere has served navigators quite well over the centuries and it continues to provide a basis for instruction and practical navigation. Aside from the fact that modern navigators employ refined models of the earth such as the WGS84 ellipsoid, which together with satellite positioning gives unprecedented positioning accuracy, the question arises as to just how good are the spherical results for distance when compared to the results for distance obtained from the spheroidal model. In this document a comparison is made of great circle distance (GCD) on the sphere with great ellipse distance (GED) on the spheroid. This comparison is then repeated for rhumbline distances. In each case it is concluded that the difference in using the sphere when compared to the spheroid is near 0·5%.
2. SPHERICAL AND SPHEROIDAL EARTH
Since the spherical earth is a convenient assumption, the question of which sphere should be chosen for navigation purposes naturally arises. A sphere having equal volume or surface area could be selected, but neither of these is as convenient as the navigation sphere defined here as one upon which, on any great circle, a span of one minute of arc is equal to one International Nautical mile (n.m) of 1852 metres. The navigation sphere therefore has a radius of: or 3437·7468 n.m.
On the WGS84 ellipsoid, a natural unit of distance is the span of one minute of longitude at the equator also called the geodetic or geographic mile. It is equal to 1855·3249 metres and the equatorial radius is: or 3437·7468 geodetic miles. The geodetic mile is equal to 1·0018 nautical miles.
3. COMPARISON OF GCD AND GED
By comparing distance along a great circle (GC) with the distance between the same two points on the great ellipse (GE), their ratio GCD/GED can be used to describe the fractional and percentage error η and η% i.e.
For comparison purposes, it is convenient to consider the great ellipse as an inclined version of a meridian ellipse. Distance L(ψ) from equator to geodetic latitude ψ on the meridian ellipse is given by:
An approximation to Equation 2 has been provided by Snyder [o1], which to the first two terms is:
In this expression terms higher than ε2 have been omitted from Snyder’s series expansion since numerical evaluations have shown that their contributions are minor . For the inclined ellipse shown in Figure 1, the same formulae apply, but whereas on the WGS84 ellipsoid the ellipticity is ε=0·08181919 it must be replaced with a smaller value ε′ (0⩽ε′⩽ε) given by:
Where φv is the geocentric latitude of the vertex, φ′, ψ′ are the geocentric and geodetic elliptical angles respectively which are related by tan φ′=α′ tan ψ′ and for which α′=1−ε′2.
5. DISTANCE ON THE GE AND GC
Equation 3 gives distance to good accuracy when ε and ψ are replaced with ε′ and ψ′. The GED between two points at P1, P2 can then be expressed as:
After applying Equation 3 and manipulating further:
where the constant 1·0018 has been included to convert from geodetic to nautical miles.
Since GC latitude angles (φ) on the navigation sphere are both geodetic and geocentric, the sphere can be considered as a degenerated spheroid for which ε=0, hence when computing GCD, angle φ′ can be replaced with ψ′ with no loss of generality i.e.
7. RHUMBLINE COMPARISONS
Analysis of the length of the rhumbline on the navigation sphere between specific points is readily compared to the corresponding length on the spheroid between the same points. The rhumbline distance d1 on the sphere between defined positions is given by d1=DLAT sec γ where DLAT is the difference in latitude between two points in n.m, while on the spheroid, the corresponding distance is d2=1·0018DLP sec γ where DLP is the number of latitude parts on the spheroid separating the same two points. The coefficient 1·0018 converts the DLP in geodetic miles on the spheroid to nautical miles and γ is the azimuth angle. The ratio of these two measures is d1/d2 and the fractional error is or:
which is independent of γ.
On the navigation sphere, DLAT is simply a(φ2−φ1) and DLP=L(ψ2)−L(ψ1). We then have:
As before, latitude φ can be replaced with ψ with no loss of generality and with ε=0·08181919 (WGS84) Equation 16 becomes:
which reduces approximately to:
From an inspection of Equation 18, the greater values of the fractional error occur for small distances i.e. when . Maximum errors occur for with values very close to ±0·005 or ±0·5% respectively. For moderate values of δ the error is less. As ψ0 approaches middle latitudes the error diminishes approaching zero at . Elsewhere for other valid values of δ in the range , the distance for a rhumbline on the navigation sphere is within 0·5% of the distance on the rhumbline on the WGS84 spheroid.
We can conclude that any error likely to be incurred in using the navigation sphere is, when rounded to one significant digit, within half of one percent and generally less for GC tracks when compared to GE tracks between the same points. When rhumbline courses are considered, the rounded error is also no greater than half of one percent. From results developed here, it appears that the spherical model can serve well in an instructional setting and meet most of the needs of navigational practice in the range of the usual navigational latitudes reached by great circle and rhumbline tracks. In more demanding applications involving high tonnage or high-speed vessels where operating cost and safety are of prime importance, the spheroidal model is more suitable.