Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » Software » comp.soft-sys.matlab

Topic: about float value
Replies: 0  

Advanced Search

Back to Topic List Back to Topic List  
dpb

Posts: 8,102
Registered: 6/7/07
about float value
Posted: Jan 11, 2013 12:55 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

Hi, I have a problem which is bothering me all the time.

I have a float value, say AAA, which is equal to 0.004

now in a very simple judgement function:

if AAA == 0.004
...

it will never enter into the condition section.
Every time, I have to do this:

if round(AAA*1000)/1000 == 0.004
...

<http://matlab.wikia.com/wiki/FAQ#Why_is_0.3_-_0.2_-_0.1_.28or_similar.29_not_equal_to_zero.3F>

Also DAGS for 'Goldberg-What every computer scientist should know about
floating point' -- I seem to have mislaid my convenient link at the
moment, sorry.

It's inherent in floating point as the above show in lesser/greater detail.

if (abs(AAA)-setValue) > tolVal

is alternative way to make a comparison where tolVal is something like
1E-7*setValue or other suitable tolerance value.

The point is that _exact_ comparisons to floating point values is often
going to cause grief owing to a bit or so in the representation that can
be off owing to whether the value is computed or entered at the command
line or otherwise obtained in slightly different manner. Even order of
computation can cause a rounding difference as can whether an
intermediate value is held at a higher precision in a floating point
register during computations rather than stored between operations.

--




Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.