Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
NCTM or The Math Forum.



Re: How to import data without losing much precision
Posted:
Dec 3, 2012 9:44 PM


"Jeff" wrote in message <k9j6fs$ih3$1@newscl01ah.mathworks.com>... > Actually, I don't know how much precision I should preserve. My adviser has been characteristically slow to respond to my questions and I've learned that I need to keep working while waiting for his response. If I don't need that much precision, then hopefully I will have at least learned something valuable along the way. > > Maybe my real question should have been "How do I determine how much precision my version of Matlab has?"          Determining the magnitude of computation errors due to rounding can be a fairly complicated matter. It is of course heavily dependent on the number of bits used to represent numbers. However it also depends in an important way on the nature of the computation that is being performed.
As a very simple (but admittedly extreme) example of the latter, suppose you wish to compute the two roots of the quadratic equation "a*x^2+b*x+c=0" where a and c are small numbers and b is large and negative, say, (I generated these using 'rand')
a = 5.2498740541049627e01; b = 4.6332273421032481e+07; c = 6.5193872929175320e02;
Matlab has no trouble with the large root using the familiar formula that we all learned in high school
x1 = (b+sqrt(b^24*a*c))/(2*a); % = 8.8254066561472103e+07
However, for the other root, using the other familiar formula
x2 = (bsqrt(b^24*a*c))/(2*a);
matlab is faced in this case with the very small difference between two very large numbers in the numerator and loses relative accuracy  that is, the ratio of error to total size  in a serious way getting the erroneous answer
x2 = 7.0959612746310526e09
On the other hand if an equivalent formula is used (multiply numerator and denominator by "b+sqrt(b^24*a*c)" and simplify to see this)
x2 = 2*c/(b+sqrt(b^24*a*c));
the answer is
x2 = 1.4070941940781315e09
which is in fact correct out to all but the 17th digit.
The lesson to be learned here is that in all the computations you perform on your data you should definitely take into account the nature of the algorithms you use in handling that data in judging how much precision you require, and of course you should always select for numerical procedures those with the best accuracy.
Roger Stafford



