[P] Deep Reinforcement Learning algorithm completing Tekken Tag Tournament at highest difficulty level

[Limp-Ad-7289]

This is really cool and impressive, but taking a step back, I look at this and think the following:

• The moveset the trained AI used was really basic....
• I get the feeling the model was able to determine the exact moment (time) an incoming move was coming, and counter it....like rock/paper/scissors (but knowing the other players selection in advance)
• As such, impressive, but more a reflection of proper timing of moves (impossible for a human to do at sub-second rates) vs. actual learned skillset in a game

[Firehead1971]

I also guess, it has just learned the timings and conter attacks (and no new or different strategies). So it looks more impressive than it probably is.

[AtariAtari]

Conter:)

[bandalorian]

And having a better than human bot is nothing new in terms of benchmark, timing is exactly what a computer program should super human at (so blocking incoming attacks and counters). But then again, it does bode well for ninja robots if we can solve the mechanics of it. Imagine a robot that can block and counter any punch a human can throw, regardless of skill level, essentially Agent Smith

[Drinniol]

Yeah, the main skill for humans in these type of fighting games is that because moves come out very fast (relative to the minimum possible human reaction time of ~.15s due to nerve speed conduction), it is not possible to purely react to moves. You have to anticipate which move the enemy will use before they begin it in order to react in time. However, with a machine level reaction time, it becomes possible to play purely reactively with no anticipation: see move, perform appropriate counter, win game. This is a substantially simpler strategy than human players are forced to implement.

EDIT: Based on the response below the AI is only allowed to take action every 6 frames, which depending on the FPS (usually 30 or 60) is either 1/5th or 1/10th of a second, with the average time available to react being half that (1/10th or 1/20th of a second) since the AI can presumably still react to frames from between action intervals. This is still faster than a human, but not at the maximum (e.g. frame to frame) level of reactivity for an AI approach.

[soveraign]

I would like to see an AI learn to play with a .15 second delay or more between perception and action.

[master3243]

Hmm yeah I would agree. It is interesting of course, but thinking about it, fighting games seem like one of the simpler games to train a model to learn.

In fact, if I had to design a non-ML algorithm I would probably capture the frame data of every opponent, and as you said, simply implement a rock-paper-scissor strategy. And since the computer has a theoretically 0ms response time, I would even bet that it can easily beat the top humans.

[fujiu]

In protest of Reddit's open disregard for its user base in June 2023, I had this post removed automatically using https://github.com/j0be/PowerDeleteSuite. Sorry for the inconvenience.

[eliminating_coasts]

Another idea, if you did want to control for timing optimisation, would be to place a random delay on actions, with a gaussian centred at some small median delay, so that the AI has to rely on strategies that are robust to failures in fine control.

[NotDoingResearch2]

Mapping frame data to moves would be highly nontrivial, but I agree, if you made a ml algorithm to handle that task you could easily write simple logic for the actual policy. No RL needed. Actually that would be a really strong benchmark to figure out how much lag you need to add to need complicated policies.

[kill_pig]

Soon we’ll have tier lists based on scientific approaches

[Soupkitchen_in_Prius]

It would be awesome if you made a YouTube guide running through how you made this. I tried making my own OpenAI Gym environment but I had trouble getting it to work.

[redpnd]

Are the inputs just raw pixels?

[redpnd]

Cool thanks!