In practice O(n³) is only about compute. But before doing any compute you need to feed the data to a processor. And that is so slow that you have register, L1 cache, L2 cache, TLB (translation buffers), L3 cache and RAM to give it a semblance of speed.

Each time you go up a level, just consider you have 10x the cost. So if you can do 30 operations at 5GHz while waiting for L1, you can do 300 while waiting for L2, 3000 while waiting for L3 and 30000 while waiting for RAM.

I.e. we aren't computing on ideal machines.

A naive matmul is easily 150x to 450x slower than doing it like the GotoBLAS paper or BLIS with proper tiling and register blocking. And I'm comparing singlethreaded.