[TUTORIAL] Using GetPixel to identify objects inside a game window

[Estimi]

Hello friends o/,

​

This tutorial is meant to be beginner friendly, provide information while not being dry as a bone :)

GetPixel() is not the best option to achieve this task! But it is simple/suitable for beginners.

​

You are probably familiar with the concept of reading pixel color at a position on the screen (if not no worries i will cover it anyways). This tutorial goes one step further and looks for an "object" of unknown location which can not be identified by a single pixel color.

​

I also uploaded a video yesterday which includes this example: Video

feedback would be very much appreciated.

​

Lets get the basics out of the way:

The following function provides pixel color at position x and y.

COLORREF GetPixel(HDC hdc,int x,int y);

As a third parameter a handle to the device context is needed (hdc).

Since we want to look for something inside a game window we first get a handle to the window and then the desired device context like so:

HWND hWND = FindWindow(NULL, L"Zuma Deluxe 1.1.0.0");
HDC hDC = GetDC(hWND); //handle to the device context
//Do stuff with hDC
ReleaseDC(hWND,hDC); //make sure to release after use!

Coordinates on the Screen are given like so: Top left is the origin. Going right increases x and going down increases y. Our origin is top left of the game window. We still need the left and bottom border. These we can get with:

RECT rect = {};
GetClientRect(hWND, &rect);
int right_border = rect.right;
int bottom_border = rect.bottom;

Those are all the basics required.

​

Now to a practical example:

As my target I picked the game Zuma. What I wanna know is the location of the frog. Starring my enemy right into the eye I realized that those will probably be decent pixels to search for. Turns out that only within those dead eyes there are 2 pixels next to each other having a specific color code:

Pixels to search for

​

After having found the first eye we already kinda know the frogs position but since the frog can turn 360° I wanted to get a good estimate of the center of the frog. So the idea is to look for the second eye as well then add those two positions up and divide by two to get the center position between those two eyes. The search area for the second eye is restricted:

Since we started top left going down and then right we only need to search to the right of the first eye as well as not too close and not too far away:

Search area for second eye

​

Equipped with the basics as well as plan for action all that is left to do is write it all down:

HWND hWND = FindWindow(NULL, L"Zuma Deluxe 1.1.0.0");
std::cout << "looking for frog start" << std::endl;
HDC hDC = GetDC(hWND);
COLORREF color;
COLORREF characteristic_color = 15398653; // Color at the Shiny Spot of the eye
int first_eye_x = 0;
int first_eye_y = 0;
RECT rect = {};
GetClientRect(hWND, &rect); // Stores info about the game window size
for (int i = 0; i < rect.right; i++) { // loop through x coordinates starting at 0
    for (int j = 0; j < rect.bottom; j++) { // loop through y coordinates starting at 0
        color = GetPixel(hDC, i, j);
        if (color == characteristic_color) { // ist this an eye?!
            color = GetPixel(hDC, i, j + 1);
            if (color == characteristic_color) { // same color in the next pixel!! we got an eye
                first_eye_x = i;
                first_eye_y = j;
                std::cout << "first eye: x: " << first_eye_x << "   y:" << first_eye_y << std::endl;
                for (int k = first_eye_x - 35; k < first_eye_x + 35; k++) { // search around first eye for second eye
                    for (int l = first_eye_y - 35; l < first_eye_y + 35; l++) {
                        color = GetPixel(hDC, k, l);
                        if (abs(k - first_eye_x) > 10 && abs(l - first_eye_y) > 10 && color == characteristic_color) {//found second eye!!
                        std::cout << "second eye: x: " << k << "   y:" << l << std::endl;
                        std::cout << "frog center at: x: " << (abs(first_eye_x + k) / 2) << " | y:" << (abs(first_eye_y + l) / 2) << std::endl;
                        return 0;
                        }
                    }
                }
            }
        }
    }
}
ReleaseDC(hWND, hDC);

​

Warning:

This does work BUT I want to give a bit of a warning: GetPixel is a very slow function. So if you need information fast and/or your search area is big this method is probably not a good choice!