The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Software » comp.soft-sys.matlab

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: How to import data without losing much precision
Replies: 5   Last Post: Dec 3, 2012 9:44 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Roger Stafford

Posts: 5,929
Registered: 12/7/04
Re: How to import data without losing much precision
Posted: Dec 3, 2012 9:44 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

"Jeff" wrote in message <k9j6fs$ih3$>...
> Actually, I don't know how much precision I should preserve. My adviser has been characteristically slow to respond to my questions and I've learned that I need to keep working while waiting for his response. If I don't need that much precision, then hopefully I will have at least learned something valuable along the way.
> Maybe my real question should have been "How do I determine how much precision my version of Matlab has?"

- - - - - - - - -
Determining the magnitude of computation errors due to rounding can be a fairly complicated matter. It is of course heavily dependent on the number of bits used to represent numbers. However it also depends in an important way on the nature of the computation that is being performed.

As a very simple (but admittedly extreme) example of the latter, suppose you wish to compute the two roots of the quadratic equation "a*x^2+b*x+c=0" where a and c are small numbers and b is large and negative, say, (I generated these using 'rand')

a = 5.2498740541049627e-01;
b = -4.6332273421032481e+07;
c = 6.5193872929175320e-02;

Matlab has no trouble with the large root using the familiar formula that we all learned in high school

x1 = (-b+sqrt(b^2-4*a*c))/(2*a); % = 8.8254066561472103e+07

However, for the other root, using the other familiar formula

x2 = (-b-sqrt(b^2-4*a*c))/(2*a);

matlab is faced in this case with the very small difference between two very large numbers in the numerator and loses relative accuracy - that is, the ratio of error to total size - in a serious way getting the erroneous answer

x2 = 7.0959612746310526e-09

On the other hand if an equivalent formula is used (multiply numerator and denominator by "-b+sqrt(b^2-4*a*c)" and simplify to see this)

x2 = 2*c/(-b+sqrt(b^2-4*a*c));

the answer is

x2 = 1.4070941940781315e-09

which is in fact correct out to all but the 17th digit.

The lesson to be learned here is that in all the computations you perform on your data you should definitely take into account the nature of the algorithms you use in handling that data in judging how much precision you require, and of course you should always select for numerical procedures those with the best accuracy.

Roger Stafford

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.