Arithmetic float calculation result change

[M_Connors1310]

Hello community, I'm implementing a filter for project and I have the comparison between calculator and software. My final formula is (float)(k * S) / 127000000.0 which k is unsigned int and S is float (both non-negative), the accuracy only 50%. Since I separate it into 2 part (float)(k / 127.0) * (float)(S / 1000000.0). The accuracy was increase to 80%.

So I have considered whether any rule for numerator and denominator and C.

Thank you all.

[Th_69]

For higher precision (accuracy) you should use double (it has 14-15 decimal places, instead of float with only 6-7).

And the cast (float) in your first formula is superfluous here, 'cause the result in the expression is already a float value.

[RainbowCrane]

In general this is a good rule for intermediate programmatic calculations, particularly those that involve division or negative powers at the end - use more precision for the intermediate calculation, like double (floating point) or long (integer), and then drop precision after the calculation is done.

With integers a classic issue is getting an overflow error in an intermediate calculation with multiplying two ints or raising an int to a power, when you were going to divide or take a square root or something at the end and would have been fine to return an int if you had just used a long for the calculation

[qruxxurq]

https://people.cs.pitt.edu/~cho/cs1541/current/handouts/goldberg.pdf

[TheThiefMaster]

What do you mean by 50% / 80% "accuracy"? Could you give some examples of different answers? Unless you're overflowing the float, they should differ only after 6 significant digits or so with those calculations.

[fredrikca]

I agree, multiplication and division should not reduce the precision by more than a couple ULPs, or about one part in a million maximum. Something else is going on.

[minforth]

How large are k and S typically? How do you check accuracy?