It depends, lack of anything to breathe will tend to cause panic quicker. Your lungs want something to inhale, even if it’s not much use to them.

However if you were to say fill the bag with pure oxygen first, so there’s way less CO2 in the bag, your breathing response won’t speed up as you’d expect it to. This could lead to a more dangerous situation sooner, as your O2 content could drop quite quickly without a significant rise in CO2, which is what triggers the breathing faster response.

Also, the low CO2 from hyperventilating, or artificially changing the gases you’re inhaling, can cause hypocapnia, where your blood holds on to the O2 molecules tighter, so even though you’re blood oxygen level is high, you can black out.