Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Topic: computation precision and display
Replies: 3   Last Post: Dec 4, 2012 12:32 PM

 Messages: [ Previous | Next ]
 Jose Posts: 1 Registered: 12/3/12
computation precision and display
Posted: Dec 3, 2012 8:01 PM

Hi,

It just happens in my linux 64 bits PC, running Matlab 2010a (on format long g) that when I calculate

>> A=(0.714844838298060 + 0.699283102297360i)*conj(0.714844838298060 + 0.699283102297360i)

A =

1.000000000000000

Now when I calculate

>> B=(0.757342604591773 + 0.653017748051421i)*conj(0.757342604591773 + 0.653017748051421i)

B =

1

And the difference is:

>>A-B

ans =

-1.11022302462516e-16

This is important because when I try to plot A and B as elements of a matrix with a colormap using imagesc they get very different colors.

Then my questions are:

How to interpret a number such as A=1.000000000000000?

Is it possible to display its nonzero decimals after the 15th zero?

Another important question is that Matlab seems to make this kind of rounding off randomly so many other complex numbers when operated as above produce results sometimes displayed as an exact one or a one follwed by many zeroes.

So, how can this be controlled or predicted?

Thanks.

Date Subject Author
12/3/12 Jose
12/3/12 Nasser Abbasi
12/4/12 Steven Lord
12/3/12 Roger Stafford