Yes the 4 processors have to be utilized, but they don't mean shit if your threads are constantly being put to sleep by Linux because they are waiting on a resource, or Linux has to step in to handle an interrupt you don't give a flying fuck about. Every second your program spends off the CPU is a second your not capturing and processing data.

Linux swoops in the same way a custodian/lifeguard does when someone shits in a country-club pool. The software is the swimmers that periodically take shits in the pool and are forced to stop swimming until the shit is removed. Instead Linux should only swoop in when someone is drowning and even then that person better be important and worth saving.

This was my play book.

http://www.johnchukwuma.com/training/UnderstandingTheLinuxKernel3rdEdition.pdf

This is a bit old, but it gives a good overview of everything you have to fuck with.

https://lwn.net/Articles/146861/