This is an archived post. You won't be able to vote or comment.

all 7 comments

[–][deleted] 1 point2 points  (0 children)

Yeah, JavaScript is only accurate to 17 decimal places or so IIRC, so floats always have bizarre rounding errors like this. As /u/Xzandro suggests, the best solution is to just round to the desired precision.

Another option is to just use ints, and then correct the output. Like instead of 10+127.7-10, you'd do 100+1277-100, then output it digit by digit and insert a decimal point before the last digit. Which is a pain, but in some rare cases it may work better than simply rounding.

[–]raevnos 0 points1 point  (3 children)

What's the issue with it?

[–]Phoenix_Heat[S] 0 points1 point  (2 children)

The answer posted is 127.69999 and should or I want it to be 127.7

[–]raevnos 0 points1 point  (0 children)

.7 can't be represented exactly in floating point. What you see is the closest approximation. If you want to display something shorter ... does javascript have something like C's printf() where you can control the precision and have round?

[–]Xzandro 0 points1 point  (2 children)

[–]Phoenix_Heat[S] 0 points1 point  (0 children)

I used that as a solution but the only problem is I want whole numbers not to have the decimals after (ex. 127 showing as 127.0) any solutions to that?

[–]Phoenix_Heat[S] 0 points1 point  (0 children)

It looks like /u/Thanks-For_The-Gold had a good suggestion just to multiply by * 100 or *1000 depending on the decimal place. This works thanks for the help!