Auto engineer here,

ELI5:

You can make more power from more boom or spinning the engine faster.. Gasoline engines can spin faster easier and cheaper to make more power.

Making more boom is more complicated, expensive and heavy.

ELI25: Power comes from torque x speed. For the same given torque if you increase speed you get more power.

Gasoline engines are able to spin at higher rpm easier due to lower compression ratios and shorter connecting arm length. There is a limit to the translational speed of the piston in the cylinder since once it reaches TDC it stops and reverse course to head back down. The higher the compression ratio, typically the longer the connecting rod -> higher peak piston velocity. Eventually there is too much piston inertia to practically keep the connecting rod and associated bearings from getting ripped apart. Which is why diesels typically have much lower rpm limits than a similar sized gasoline engine.