Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
NCTM or The Math Forum.


dpb
Posts:
9,257
Registered:
6/7/07


about float value
Posted:
Jan 11, 2013 12:55 AM


Hi, I have a problem which is bothering me all the time.
I have a float value, say AAA, which is equal to 0.004
now in a very simple judgement function:
if AAA == 0.004 ...
it will never enter into the condition section. Every time, I have to do this:
if round(AAA*1000)/1000 == 0.004 ...
<http://matlab.wikia.com/wiki/FAQ#Why_is_0.3__0.2__0.1_.28or_similar.29_not_equal_to_zero.3F>
Also DAGS for 'GoldbergWhat every computer scientist should know about floating point'  I seem to have mislaid my convenient link at the moment, sorry.
It's inherent in floating point as the above show in lesser/greater detail.
if (abs(AAA)setValue) > tolVal
is alternative way to make a comparison where tolVal is something like 1E7*setValue or other suitable tolerance value.
The point is that _exact_ comparisons to floating point values is often going to cause grief owing to a bit or so in the representation that can be off owing to whether the value is computed or entered at the command line or otherwise obtained in slightly different manner. Even order of computation can cause a rounding difference as can whether an intermediate value is held at a higher precision in a floating point register during computations rather than stored between operations.




