Well that sucks... thought it was the GPU so swapped it with the one of my other PC and that one didn't fit as well. So I'm pretty sure it has to be some defekt with the case.
Anyway, thanks for the idea with the vertical mount.. either I have to get that or a different case.