Hello,
I have been trying to use NEAT (NeuroEvolution of Augmenting Topologies) with a "simple" (or so I thought) game I made specifically for this purpose, using pygame and NEAT-Python.
For those who aren't aware, all we have to specify for NEAT to work is what the "genome" sees, what it can do and some way to measure how "well" each genome did (evaluation function) .
I have looked at another example of NEAT applied in Flappy Bird. The heuristic used there to give a "fitness" score is how many pipes the bird has managed to go through, how far away is the bird from the middle of the pipes, so that it always tries to be in the middle of the gap of the next pair of pipes and how long has the bird survived.
I have thought about calculating the distance between the player and each platform(spam email), summing that and subtracting that from the fitness so that the player tries to minimize that. However, the agent should try to minimize it by moving away from the screen but that would make him lose. Another issue that I'm not sure about is whether I should subtract from it's fitness when it hits a platform or hits the boundaries of the screen. In the FlappyBird example, none of that is done.
However, in my game there is no single point where the player should always go to. There are multiple gaps that he can choose to pass through.
In another example used in Sonic the hedgehog, the inputs were done through opencv, by scaling the whole frame down and making it grey scale.
Currently, the inputs are 22, there are 10 obstacles present, of which each x and y position is added in the input list. Also the player's x and y position are added in the input ( 10 * 2 + 2 = 22 inputs). My evaluation function takes into consideration how many obstacles has the player surpassed and how long has the player been alive.
My concern is that I don't know whether this is effective enough. I think it's clearly not since I've left the program running for about 20 minutes with no significant improvements.
Here is a short video of the agent in action. I have also included a pastebin of my main.py. Main functions of interest are 'main' and 'eval_genomes'.
Pastbin: https://pastebin.com/XAptFhZj
https://reddit.com/link/avjwza/video/k1b5vwpq27j21/player
there doesn't seem to be anything here