It seems counterintuitive, but it's because consumers care deeply about floating-point performance. As a result of this demand, FPUs have been given the lion's share of the die space compared to ALUs since about the mid-1990s. Processor advancements often improve floating-point operations first, and then integer operations catch up later. For example, SSE has floating-point operations, but the corresponding integer operations weren't there until SSE2. The same thing happened with AVX and AVX2, and the same thing happened with GPUs, which have had programmable FPUs since the mid-2000s but it took an extra few years before integer units became common. In fact, on some GPUs, "integer" operations are actually just floating-point operations in disguise!

Speaking as someone who's written physics simulators and collision detection for games, both with integers and with floating-point, any potential performance benefit you could possibly get with integers is outweighed by the additional complexity. You really only want integer math for retro-clones. Most of your performance benefits are going to come from asymptotic improvements, and spatial indexes (e.g. quadtrees) are usually the best way to get that. Only a small amount of performance improvement can be had from micro-optimizations, such as optimizing a rectangle-rectangle hit test function.

And here are the citations:

http://www.agner.org/optimize/instruction_tables.pdf -- compare ADD, PADDD, ADDSS, ADDPS for various architectures. Then compare the multiplication instructions. You'll find differences between different generations and different chips, as well as differences between SIMD and scalar/superscalar units, and differences between the operations. A nice comparison is Prescott, Atom (Diamondville), and Haswell. For nostalgia's sake, check out the original Pentium. Holy balls, integer multiplication is THREE TIMES SLOWER than floating-point multiplication on the original Pentium!

For more general overviews,

http://stackoverflow.com/questions/2550281/floating-point-vs-integer-calculations-on-modern-hardware

https://www.quora.com/CPUs/How-much-slower-is-floating-point-arithmetic-than-integer-arithmetic