Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » sci.math.* » sci.math

Topic: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Replies: 16   Last Post: Feb 27, 2014 6:52 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
gnasher729

Posts: 418
Registered: 10/7/06
Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Posted: Feb 22, 2014 6:08 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

On Friday, February 21, 2014 9:22:58 AM UTC, Thomas Plehn wrote:
> Excuse my short description, but what about the precision here?
>
> I am noot good in numerics, excuse me...


You have exactly the same problems with a naive calculation as with 32 bit, 64 bit or 80 bit precision.

1. If either a or b or both are large enough, then calculating a^2 + b^2 will overflow and you won't get a reasonable result, even though sqrt (a^2 + b^2) is nowhere near large enough to overflow.

2. If a and b are both quite small, then calculating a^2 + b^2 will underflow with massive loss of precision, or even give a result of zero, so you won't get a reasonable result, even though sqrt (a^2 + b^2) is nowhere near small enough to underflow.

Two solutions: a. You prove that a and b are not large enough that a^2 + b^2 could overflow. You also prove that either a or b is large enough so that the square doesn't underflow, and that an underflow in the other square won't afffect the result much. b. You check the magnitude of a and b. If the numbers are too big or too small you multiply both with some power of two so that the calculation will be fine, then divide the result by the same power of two.


Date Subject Author
2/21/14
Read numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Thomas Plehn
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Thomas Plehn
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Peter Percival
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Wizard-Of-Oz
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
David C. Ullrich
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Wizard-Of-Oz
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
gnasher729
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Wizard-Of-Oz
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Roland Franzius
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
gnasher729
2/23/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Roland Franzius
2/27/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Phil Carmody
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Roland Franzius
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
James Waldby
2/27/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Phil Carmody
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
gnasher729

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.