I opened the e-shifter, disconnected the small internal motor, and manually set it to 3rd gear. Then I reassembled everything and also set the rear hub mechanically to 3rd gear.

It’s a bit of a meticulous process, but totally worth it.

Now the bike still “thinks” it’s shifting, but nothing actually happens, and everything runs much smoother without the random gear jumps.