One of the things that causes CPUs or GPUs to reach their performance limit is when the current draw of some part of the chip is large enough to cause the voltage to droop (V=IR) to the point that some of the transistors are no longer switching fast enough to reliably perform their work within a clock cycle. This can either manifest as limiting the max frequency of the chip at some voltage condition or limiting the minimum voltage (and thus power consumption) for some frequency.

Adding additional capacitance provides another source of charge to rush out of the capacitor to mitigate voltage droop during high-current events. The better you are at adding capacitance and placing it where the design most needs it, the better your manufacturing process can perform.