Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: How to import data without losing much precision
Replies: 5   Last Post: Dec 3, 2012 9:44 PM

 Messages: [ Previous | Next ]
 Roger Stafford Posts: 5,929 Registered: 12/7/04
Re: How to import data without losing much precision
Posted: Dec 3, 2012 9:44 PM

"Jeff" wrote in message <k9j6fs\$ih3\$1@newscl01ah.mathworks.com>...
> Actually, I don't know how much precision I should preserve. My adviser has been characteristically slow to respond to my questions and I've learned that I need to keep working while waiting for his response. If I don't need that much precision, then hopefully I will have at least learned something valuable along the way.
>
> Maybe my real question should have been "How do I determine how much precision my version of Matlab has?"

- - - - - - - - -
Determining the magnitude of computation errors due to rounding can be a fairly complicated matter. It is of course heavily dependent on the number of bits used to represent numbers. However it also depends in an important way on the nature of the computation that is being performed.

As a very simple (but admittedly extreme) example of the latter, suppose you wish to compute the two roots of the quadratic equation "a*x^2+b*x+c=0" where a and c are small numbers and b is large and negative, say, (I generated these using 'rand')

a = 5.2498740541049627e-01;
b = -4.6332273421032481e+07;
c = 6.5193872929175320e-02;

Matlab has no trouble with the large root using the familiar formula that we all learned in high school

x1 = (-b+sqrt(b^2-4*a*c))/(2*a); % = 8.8254066561472103e+07

However, for the other root, using the other familiar formula

x2 = (-b-sqrt(b^2-4*a*c))/(2*a);

matlab is faced in this case with the very small difference between two very large numbers in the numerator and loses relative accuracy - that is, the ratio of error to total size - in a serious way getting the erroneous answer

x2 = 7.0959612746310526e-09

On the other hand if an equivalent formula is used (multiply numerator and denominator by "-b+sqrt(b^2-4*a*c)" and simplify to see this)

x2 = 2*c/(-b+sqrt(b^2-4*a*c));

x2 = 1.4070941940781315e-09

which is in fact correct out to all but the 17th digit.

The lesson to be learned here is that in all the computations you perform on your data you should definitely take into account the nature of the algorithms you use in handling that data in judging how much precision you require, and of course you should always select for numerical procedures those with the best accuracy.

Roger Stafford

Date Subject Author
12/2/12 Jeff
12/2/12 Steven Lord
12/3/12 Jeff
12/3/12 Steven Lord
12/3/12 Nasser Abbasi
12/3/12 Roger Stafford