The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Courses » ap-calculus

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: [ap-calculus] e
Replies: 2   Last Post: Oct 23, 2012 9:10 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Jeff Stuart

Posts: 1,086
Registered: 12/6/04
Re: [ap-calculus] e
Posted: Oct 23, 2012 8:08 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

This ap-calculus EDG will be closing in the next few weeks. Please sign up for the new AP Calculus
Teacher Community Forum at
and post messages there.

Welcome to the world of floating point arithmetic. The problem is that
calculators and most numerical software do not treat all real numbers
the same way, and that except for a few special numbers, at every step
of the computation process, numbers are replaced by "convenient"
approximations. For simplicity, imagine that you calculator only keeps
six digits. Thus, your calculator would replace pi with 3.14159. If
you ask your calculator for (pi - 3.14159)^(-1), you will get an
error, because the first computation your calculator does is pi -
3.14159 = 0, and it then faces 0^(-1), which it knows is undefined. As
another example, 1/3 stores as 3.33333 x 10^(-1), so 3 x (1/3) is
computed as 3.00000 x 3.33333 x 10^(-1) =9.99999 x 10^(-1). Notice that
1 - 3(1/3) computes as 1.00000 -0.999999, and the first thing that
happens in our simplified calculator is that the last 9 is dropped so
the digits line up. Thus the difference computed is 0.00001, rather
than 0.000001 . Further, the result is stored as 1.00000 x 10^(-4), so
that neither of the first two digits is correct! This is called
catastrophic cancellation. Now suppose you ask your calculator to
compute (1 + 1/x )^x, where x has been chosen to have exactly six digits
(or at least six digits in scientific notation). The first computation
is 1/x, which is stored as a six digit approximation "a", and then your
calculator computes 1 + a, which it stores as a six digit approximation
"b", and then, finally, it computes b^x. If x > 10^5, then 1/x <
10(-5), so a =< 10^(-5), meaning that as a decimal, a
=0.00000##########...#000000... where at most six of the # digits are
nonzero.. When the calculator computes 1 + a as 1.00000 + a, it only
keeps the first FIVE decimal places of a, so 1 + a = 1.00000 rather
than 1.00000####....#0000... we
Since b = 1.00000, it is not surprising that b^x = 1. OH, by the way,
all of this happens base 2 or base 8 or base 16 rather than base base 10.

You might look up "floating point arithmetic", "IEEE floating point",
and/or "truncation versus rounding".

Jeff S.

Professor Jeff Stuart, Chair
Department of Mathematics
Pacific Lutheran University
Tacoma, WA 98447 USA
(253) 535 - 7403

To search the list archives for previous posts go to

Date Subject Author
Read Re: [ap-calculus] e
Jeff Stuart
Read re:[ap-calculus] e
Michael Sweeney

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.