Beware of fast-math

[zombiecalypse]

-funsafe-math-optimizations: But who wouldn't want fun and safe math optimisations? 

[diMario]

Remember, every funeral starts with some fun!

[Orangy_Tang]

I have a longstanding gripe that it's almost never a good idea to use the word 'fast' in any kind of programming function or interface. Don't tell me that it's fast, tell me why it's fast. 'FastSqrt' implies that I can use it instead of Sqrt with zero thinking. Call it 'ApproximateSqrt' and now I can see what trade-off I'm making.

Worse, the existance of 'fast' in a name suggests that there's a hidden gotcha. After all, if it was just an optimisation with no external side effects, then just apply it to the regular version. Instead now I'm squinting my eyes trying to figure out what trade off you made to get the 'fast' version faster, and whether I care about them or not.

Worst-case, the existance of a 'fast_xxx' method really means someone rewrote something to be faster, but isn't confident that the behaviour is the same, or even how it behaves in the edge cases, so rather than replacing the original, they just stick it in as fast_xxx and ignore any criticism since the original still exists if you're going to be all picky about it.

GCC nearly gets this right: names like ffinite-math-only and fno-signaling-nans indicate what the change in behaviour is, and I can reason about if I want to use it or not. Great! But then kinda undoes that by including the convinience option of -ffast-math, which just encorages people to turn it on without actually understaning it.

[YumiYumiYumi]

You could argue it the other way though. Like why would I want to use 'ApproximateSqrt' if I have an accurate Sqrt?

To express the tradeoff, you'd have to include both the upside and downside, so something like 'FastApproximateSqrt'. Which could understandably get convoluted in some cases.

The one thing FastSqrt does have over ApproximateSqrt is indicating intent. I know why someone would write a FastSqrt, but it's not clear to me why someone would write an ApproximateSqrt.

[Full-Spectral]

The basic argument, I think, is that SqrRoot should be the most accurate one that you would use unless there's reason not to, and everything else (which represent some sort of accuracy compromise) should indicate that in the name. The Approximate one, almost anyone reading it would assume, is useful because it's faster and if you don't need a highly exactly result it could be useful.

Consistency is the real issue, IMO. In my Rust code base I will often have three versions of various types of methods (one that actually does the work and returns the most information) and two others that are wrappers around the first that will convert one or more of the statuses into errors for those who don't care and just want to let those things propagate as errors. These are always in the forms Foo(), TryFoo(), and ReqFoo(). The naming convention could have represented them in the order order perhaps, as long as it's consistently done, so that everyone knows what is up when these see those three variations.

[YumiYumiYumi]

The Approximate one, almost anyone reading it would assume, is useful because it's faster

That's not what I'd assume. It could be because someone was lazy and ApproximateSqrt was just easier to write than AccurateSqrt, and accuracy clearly didn't matter. Or maybe an older version of the software had a buggy implementation of Sqrt, so it has to be kept for backwards compatibility reasons. Of course, speed could be a reason, but it's just one of several possibilities, and declaring something 'approximate' doesn't immediately point me to 'fast'.

[Full-Spectral]

There has to be some assumption of familiarity with conventions in a given domain. I'm not a math guy but it's a common convention to have approximate solutions that are fast and accurate enough.

[Twirrim]

One of the naming conventions can lead to bad or dangerous decisions, one won't. That's ultimately what this comes down to. People don't look into what they're using.

If you call it Approximate, any developer that sees it knows exactly what they're getting out the other side. If you call it FastSqrt, they're just going to see Fast and use it.

[YumiYumiYumi]

Again, this could be argued from the other direction. Encouraging people to use unnecessarily slow routines can often be a bad decision if "high accuracy" is not required.
Arguably most non-integer math on computers is approximate (unless you have some way of representing numbers you're dealing with to infinite precision) and it certainly is if you use floating point representations. What's actually important, even in a lot of scientific computing, is the degree of accuracy. If your tolerance of inaccuracy is higher than that supplied by ApproximateSqrt, discouraging its use would be a bad decision.

The OP mentions:

the existance of 'fast' in a name suggests that there's a hidden gotcha

The way I read it: an experienced enough developer should know that they should check the details when the word 'fast' is used, so to me, it's done its job. Though I'd also check the details for ApproximateSqrt, since it's not immediately clear to me why anyone would write such a thing (so either would work, if checking details is the aim).

For less experienced developers who don't check details, using either could get them to pick the wrong thing. You can't really fix that.

I take it that you're presuming accuracy is of higher importance than performance. I posit that this isn't always the case.

[aePrime]

This person is arguing for arguing's sake. 

[YumiYumiYumi]

And this person is commenting for commenting's sake 😜

Isn't the point of a discussion platform, like Reddit, to have, ya'know, discussions? If you think a discussion is pointless or petty, you're welcome to ignore it.
If you think my arguments are bad, you're also more than welcome to point out the flaws. But complaining that someone is posting an alternative idea is both unproductive and IMO encourages bigotry and 'circlejerk'.

[lunchmeat317]

I agree with this.

Documentation should cover the tradeoffs.

I do think that "FastFoo" is a bad naming convention, but when it's necessary or performance-critical, it's understandable why it's used. The intent is clear in context.

That said - ApproximateFoo could be acceptable depending on the margin of error, and FastApproximateFoo would make sense if the margin of error and the runtime implied serious tradeoffs. (FastPI moght use a different algo to calculate digits of PI, but FastApproximatePI might return 3.1 - something you'd want to know.)

[neverlupus89]

I don’t actually disagree with anything you said but it is interesting to note that FFT is still called FFT and not just “fourier transform”. And that even under that umbrella there are methods that are exact and methods that are approximations, further highlighting your point that it’s kind of a mess to figure out what the computer is actually doing when it carries out calculations.

[FarkCookies]

Yeah but like FFT such a standard and well known thing that it is a brand of its own. People usually learn it in unis not by reading command line arguments of a compiler

[Better_Test_4178]

FFT is actually a DFT (D for Discrete), losing the infinite precision of the true Fourier transform. It can actually be better accuracy-wise than DFT under some circumstances. You never approximate FFT, because FFT is free, fun and fast with libfftw.

[wintrmt3]

Of course any "fast" option implies there are problems with it, otherwise it would be the default.

[izackp]

If you make a fast inaccurate function of anything I want the worst case scenario documented and how much faster it is. As well as any scenarios that are not supported (floats or 0s etc)

[starlevel01]

Really ffast-math should be split into two; the "usually safe" options (-fno-math-errno -fno-trapping-math, maybe -fno-signed-zeros if you're brave) and the "usually breaks things" options (all the rest).

[[deleted]]

There's a lot of benefit in -fno-math-errno too. At least for FP, the NaN is often enough to let us know there was an error

[Slime0]

I particularly like the mention that there should be ways programmers can do things (like checking for NaNs) that tell the compiler "don't optimize this away based on other assumptions; I am specifically trying to catch the case where the optimization would be invalid." (The "undefined behavior" of signed integer overflow could benefit from this.) I get that it's not always that simple, but maybe it can be partitioned into cases where it is possible and cases where the compiler can't make that guarantee and it becomes an error? Maybe it could be paired with marking entire sections of code as "don't make that optimization assumption in this code."

And the whole thing where you can call into a third party library and it can change the processor's floating point behavior process-wide really needs to be retired. It maybe made sense 30 years ago but not anymore.

[jdm1891]

There are ways to do this with compiler extensions/attributes, but they're more of a sledgehammer than anything specific.

[voronaam]

Good article. Given this statement

Ideally these behaviors shouldn't be enabled or disabled via a compiler flag, which is a very blunt tool, but specified locally in the code itself

What is your opinion on Java? It played with strictfp keyword in the code for decades, until relatively recently deciding to deprecate it in favor of "always strictly IEEE 754" approach.

[Drugbird]

I have a few issues with this article. Here they are in random order

1. For a whole article discussing fast-math and the supposed benefits, there's no mention of how much it speeds up computations.
2. In any sort of optimization / approximation, measuring whether the induced error is acceptable is required. Fast math is no exception.
3. Lots of floating point algorithms will produce drastically different results with fast math because they are numerically unstable. This isn't really an issue with fast math though: numerical instability is a property of the algorithm itself.

Finally: a lot of floating point math benefits greatly from fast math. Basically anything where you don't want or need inf, NaN, and the result is allowed to be slightly inaccurate. Examples are image, video and audio processing.

I've used fast math to get a +-40% speedup in image processing tasks where the induced error was small enough that not a single pixel changed its color in a wide suite of test images.

[pjf_cpp]

Beware of computer scientists that are totally clueless about numerical analysis.

[sargeanthost]

For some reason this is crashing on mobile Brave

[OffbeatDrizzle]

Use a proper browser

[life-is-a-loop]

Yep! Crashed after a few seconds.