Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: computation precision and display
Replies: 3   Last Post: Dec 4, 2012 12:32 PM

 Messages: [ Previous | Next ]
 Roger Stafford Posts: 5,929 Registered: 12/7/04
Re: computation precision and display
Posted: Dec 3, 2012 11:17 PM

"Jose" wrote in message <k9ji0m\$rcv\$1@newscl01ah.mathworks.com>...
> Another important question is that Matlab seems to make this kind of rounding off randomly so many other complex numbers when operated as above produce results sometimes displayed as an exact one or a one follwed by many zeroes.
- - - - - - - - - -
I would like to correct your assertion that "Matlab seems to make this kind of rounding off randomly." It may appear random on casual observation, but in fact in computers that adhere to the IEEE 754 Standard for Floating Point Arithmetic, the rules for "rounding to nearest" are very precise when it involves the four operations of addition, subtraction, multiplication, and division. In each case the precise result (to infinite precision) must be rounded to the very nearest floating point value that is available in that precision format - that is, double precision or single precision. There is nothing random about it.

In the examples you give, the precise answer would not be an exact 1, because the sums of the squares of your defined real and imaginary parts are not precisely 1. These parts cannot be expressed to exact precision, but only to that allowed by the computer's format. If you receive an answer of exactly 1, it is only because that happened to be the closest to the precise answer. An answer 1.000000000000000 is simply the display to 15 places of a number that is presumably different from an exact 1 in its least bit. However, strictly speaking, both answers are incorrect because the computer is incapable of expressing the exact answer.

This is the nature of rounding errors in computers and it is necessary for all users to take such behavior into consideration and make appropriate allowances.

Roger Stafford

Date Subject Author
12/3/12 Jose
12/3/12 Nasser Abbasi
12/4/12 Steven Lord
12/3/12 Roger Stafford