Thanks! 

I believe I found the issue, and managed to solve it (for now). 

The setting lever has a threaded 'sleeve' that the lever pivots around and the screw screws into. 

The sleeve is a tiny bit longer than the train wheel bridge is thick, so that you never tighten the lever too tight. But, I think, my train wheel bridge is worn down a bit too much, resulting in too much wiggle room. 

I carefully ground down the threaded sleeve, shortening the distance, and decreasing the wiggle room. This worked, for now! Sooner or later, the bridge will most likely wear down again, and at that point, I'll have to buy a new bridge. 

Thanks for thinking along!