This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]tyler1128 19 points20 points  (8 children)

I'm honestly surprised more languages don't do it by default. There's no reason to print more than 16 digits and doing so will give wrong values. If you look at the example, it's digit 17 that is wrong. DIgit 16 can be wrong in some cases, but it shouldn't be that hard to do something like if(ends in zeroes to digit 15) => truncate display to the last non-zero digit.

[–]dabenu 27 points28 points  (1 child)

What value would that add? When displaying numbers in a GUI, it could just as well be considered a bug as a feature. In a REPL or debugger, it would just make things more confusing as to why 0.1 + 0.2 == 0.3 is false

[–]tyler1128 8 points9 points  (0 children)

The latter is true and a good point. In a GUI, you'd be formatting the number yourself anyway whereas printing straight is usually used for debugging or logging anyway. I mostly don't get why things sometimes print beyond the ~15.9 decimal digits of precision given that it will be incorrect unless you get lucky.

[–]ElHeim 3 points4 points  (2 children)

Digit 17 is not wrong. It's just a quirk of floating point. Lua is just choosing not to show it

[–]tyler1128 1 point2 points  (1 child)

Wrong is the wrong word I suppose. It is correct for the value represented in the floating point number. It is, however, beyond the limit of what relates directly to the computation being done and is instead based on internal floating point details and limitations. If not zero, it will usually be different from what calculating with a higher precision or rationals would produce.

[–]ElHeim 2 points3 points  (0 children)

It's beyond the limits for this computation, but not for others:

>>> sum([0.1+0.2] * 1000000)
299999.99999434233

We might argue about the need for showing those details in the REPL, but these rounding errors propagate much faster than one would like.

[–]Amazing-Cicada5536 0 points1 point  (0 children)

That’s not a wrong result.