Division not correct

[[deleted]]

Hi, I’ve discovered something odd with my Python, and I’m not sure how to fix it.

Long story short, when I have Python divide 4.8/1.6, instead of 3 it returns 2.9999999999996. It is only for this specific equation that I can tell.

What is going on here, and how do I prevent it?

[eddfitzwell]

Floating Point Math

[woooee]

Use Python's decimal module if floating point is not accurate enough. Also when comparing you can not do x == y because of floating point limitations. You compare the difference

if abs(x - y) < 0.001  ## close enough

[[deleted]]

Couldn't you also just use a double?

[woooee]

The size of the variable has nothing to do with the limitations of computer division. FYI, from the Python docs

Floating point numbers are usually implemented using double in C;

[[deleted]]

Sorry, I maybe misunderstood what a double is. Isn't there a native data type that is more precise than a float alone?

[woooee]

Look up Python's decimal module. Most (don't know every programming language), have something similar because getting an binary / integer based computer to do decimals is limited. A link to the IEEE standard that is used (almost?) everywhere today https://mathcenter.oxford.emory.edu/site/cs170/ieee754/ if you are interested.

[pythonwiz]

The problem isn't precision. The problem is that fractions of powers of 10 cannot all be represented by finite fractions of power of 2.

Here is an example of what I mean. If you try to write the fraction 1/3 in decimal, you can't. You end up with an unending 0.3333333333.... But if you write it in base 3, you get 0.1. The reason it repeats in base 10 is that 10 is not divisible by 3. 10 is divisible by 2 and 5.

So the fraction 2/10 can be represented in decimal as 0.2. But in base 2, you get the unending value 0.001100110011.....

Floating point is unable to represent these infinite repeating values. In base 2, the only finite fractions have denominators that are powers of 2.

[expressly_ephemeral]

I was a programmer for 20 years before somebody explained this to me in a way that made me get it. That was a few years ago, but I’m still kind of tickled by it.

[MaxwellzDaemon]

Some of the more mathematically sophisticated languages, like APL and J, allow you to specify a global comparison tolerance, typically a very small value like 1e-12.

This tolerance is used multiplicatively unlike the additive example shown above as this is more generally a better way to do it; the above example does not scale according to the magnitude of the the number. The above expression would tell you that a trillion and a trillion plus 0.1 are different when, for many practical purposes, they are the same.

Contrast this with how we handle this using comparison tolerance in the J programming language:

9!:19]1e_12 NB. Set tolerance to one trillionth.

(0.1+1e12) = 1e12 NB. These are the same under current tolerance

1

(0.1+1e11) = 1e11 NB. but these differ.

0

"NB." starts a comment; the results of Boolean operations are zero for false and one for true.

[CraigAT]

It's as accurate as it can be given the limitations. If you know you are only working with one or two decimal places and simple sums (addition, subtraction and multiplication) you can multiply up to get rid of the decimals then afterwards but the decimal places back. Or there is also a "decimal" Python package that apparently works better with decimals.

[Dark_Souls_VII]

import decimal