The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Software » comp.soft-sys.matlab

Topic: computation precision and display
Replies: 3   Last Post: Dec 4, 2012 12:32 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Roger Stafford

Posts: 5,929
Registered: 12/7/04
Re: computation precision and display
Posted: Dec 3, 2012 11:17 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

"Jose" wrote in message <k9ji0m$rcv$>...
> Another important question is that Matlab seems to make this kind of rounding off randomly so many other complex numbers when operated as above produce results sometimes displayed as an exact one or a one follwed by many zeroes.
- - - - - - - - - -
I would like to correct your assertion that "Matlab seems to make this kind of rounding off randomly." It may appear random on casual observation, but in fact in computers that adhere to the IEEE 754 Standard for Floating Point Arithmetic, the rules for "rounding to nearest" are very precise when it involves the four operations of addition, subtraction, multiplication, and division. In each case the precise result (to infinite precision) must be rounded to the very nearest floating point value that is available in that precision format - that is, double precision or single precision. There is nothing random about it.

In the examples you give, the precise answer would not be an exact 1, because the sums of the squares of your defined real and imaginary parts are not precisely 1. These parts cannot be expressed to exact precision, but only to that allowed by the computer's format. If you receive an answer of exactly 1, it is only because that happened to be the closest to the precise answer. An answer 1.000000000000000 is simply the display to 15 places of a number that is presumably different from an exact 1 in its least bit. However, strictly speaking, both answers are incorrect because the computer is incapable of expressing the exact answer.

This is the nature of rounding errors in computers and it is necessary for all users to take such behavior into consideration and make appropriate allowances.

Roger Stafford

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2017. All Rights Reserved.