Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » sci.math.* » sci.math.independent

Topic: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Replies: 16   Last Post: Feb 27, 2014 6:52 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
gnasher729

Posts: 418
Registered: 10/7/06
Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Posted: Feb 22, 2014 6:20 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

On Friday, February 21, 2014 9:34:42 PM UTC, Roland Franzius wrote:

> Yo can have it exactly to one bit. Suppose a>b and set a=2^m A, b=2^m B
> with 1/sqrt(2) < A < sqrt(2). Then ...


You can very easily have a result that is almost always correctly rounded by doing the calculation with 32 bit. Now using 32 bit instead of 16 would presumably cost extra time but is very easy to do. Therefore to be useful, a solution using only 16 bit arithmetic must not be more costly than just switching to 32 bit.


Date Subject Author
2/21/14
Read numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Thomas Plehn
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Thomas Plehn
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Peter Percival
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Wizard-Of-Oz
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
David C. Ullrich
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Wizard-Of-Oz
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
gnasher729
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Wizard-Of-Oz
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Roland Franzius
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
gnasher729
2/23/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Roland Franzius
2/27/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Phil Carmody
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
Roland Franzius
2/21/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point
numbers
James Waldby
2/27/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
Phil Carmody
2/22/14
Read Re: numerical precision of sqrt(a^2+b^2) with 16 bit floating point numbers
gnasher729

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.