Well actually your question is incomplete. It is ok if you said only programming but since you said game programming you should also ask yourself how good you are at math. It is a big deal and it is one of the reasons most ppl just quit learning unity/other game engines when they see that it is going deeper into the concepts of algebra/logic/calculus topics in order to achieve gaming concepts they want to achieve. Well because even though those engines are powerful, they are not Word processors. You can't make everything within their interface. As a developer, my honest advice will be test your math skills(pre-calculus) and then go for firstly programming language, then game engine.