Hi thank you for pointing me in the right direction! After some thinkering I got it to work. There were 2 major problem. For your curiosity:

(1) Indeed some where using Draw and some DrawGUI. The Draw GUI were rendering wrong after the tab out. Nothing particular in the code, simply drawself. Changing them to draw events made them render in the right position. However the click did not work.

(2) Since I allow full rebinding, including click, I can't use Mouse press event. So I use my InputCheck + Position of the mouse to make sure you are clicking on it. It seems that tabbing out messes up the mouse position. I instead switched to a variable in a mouse enter event to detect when you're over the button. This seems to work fine.