The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Software » comp.soft-sys.matlab

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: precision of fractional numbers
Replies: 7   Last Post: Oct 8, 2013 10:38 AM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]

Posts: 2,359
Registered: 12/6/04
Re: precision of fractional numbers
Posted: Oct 8, 2013 10:16 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <l30tbi$jau$>...
> Sorry should read
> >
> > round(x*100)/100
> >

Consider the OPs original request:
More specifically, I would like Matlab to consider 3.257 as 3.26. It is feasible for a single number, but I am looking for a command to use in the first line of the script and that applies same format to all the numbers along the calculations.

First, the above would apply "for a single number" but not for every number in the entire script.

Second, although 3.257 would APPEAR to be 3.26 (to two decimal digits), I don't believe it would necessarrily be EXACTLY 3.26 due to the way MATLAB (or a computer, in general) stores floating point numbers. (I don't have MATLAB installed on this computer, so I can't give an example using the short and long formats.)

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.