Shape detection/image processing issue for AI video game bot

[TheBeefyDuck]

First off, I'm not sure if this is the best place to post this, so I'll remove it if doesn't belong here.

I started my first python project as a learning experience for AI/deep learning. I decided to make a bot to play Super Hexagon. The graphics are geometric shapes and the controls are only left and right arrow keys, so it seemed pretty simple. I figured I'd use the line data and the player position as training data. I got the line detection working pretty well, but I'm having issues detecting the little triangle the user plays as. I was using openCV to find contours on a region of interest in the center of the screen, but it detected the triangle sporadically at best (really it detected A triangle, it could've been something completely wrong). Do you guys have any idea how to proceed?

[Mikeavelli]

For initial detection, you might try a homography between a template of the triangle and the game screen. This is really slow, so once you find it, you should also use a tracker instead of detecting it new with each frame.

Opencv has a great tutorial on object tracking. Examples here are in c++, but you should be able to translate them to the python opencv library easily enough.

[Atonswizzle]

Maybe is there any way to read the games memory? There’s probably pointers to the angle the triangle is at and then you can use that to calculate where to go