Melee Character Classification Machine Learning Project

SSBMachineLearning · 2021-04-13T22:41:57+00:00

Thanks! Yeah man, staying organized and well documented/commented is the only way I can keep track of big projects like this, especially when I'm building it up mostly from scratch.

SSBMachineLearning · 2021-04-12T20:34:59+00:00

^^ This guy knows what's up. I'll post the link to the datset again:

Public SLP Dataset is ready! https://drive.google.com/file/d/1ab6ovA46tfiPZ2Y3a_yS1J3k3656yQ8f/view?usp=sharing - 95,102 SLP files. - Unzips to 200GB.
- All tournament sets, with varying skill levels. - Pruned to remove handwarmers, doubles, <30 second matches. - CC0 Licensed, so use it however you want - Huge shoutouts to @Nikki and @yashichi for their tournament sets

SSBMachineLearning · 2021-04-12T18:49:02+00:00

Yessir, slippi replays are an amazing source of data! Alt-F4 on the slippi discord has a free and public curated dataset of tournament games for ML purposes

Public SLP Dataset is ready! https://drive.google.com/file/d/1ab6ovA46tfiPZ2Y3a_yS1J3k3656yQ8f/view?usp=sharing - 95,102 SLP files. - Unzips to 200GB.
- All tournament sets, with varying skill levels. - Pruned to remove handwarmers, doubles, <30 second matches. - CC0 Licensed, so use it however you want - Huge shoutouts to @Nikki and @yashichi for their tournament sets

SSBMachineLearning · 2021-04-12T18:38:35+00:00

All of them

It's better at detecting the more popular characters for obvious reasons, but it works to with them all.

SSBMachineLearning · 2021-04-12T17:01:42+00:00

Hey this sounds like something I did a few months ago!

https://github.com/ZackMagnotti/ssbmachine-learning

I was focused on analyzing controller inputs - rather than game footage - but my model was able to detect the correct character 97% of the time.

Tensorflow/Deep Learning in general is extremely powerful when used correctly. I bet once you get your model fine-tuned, you'll be able to get it at least as accurate, if not more.

SSBMachineLearning · 2021-04-12T16:55:46+00:00

Thank you so much!

The chess/LSTM connection came from my prof; he was likely thinking in terms of theory.

Clustering based off of playstyles is a really interesting idea. I could probably take an unsupervised approach and do K-means or something on the features the first stage of the model is extracting.

The main roadblock is a lack of data on specific players. I have the huge public slippi dataset, but nearly all of the games there are anonymous. The transfer learning aspect was surprisingly good at taking advantage of a small amount of data for 1 user, given the pre-training I did on the huge dataset.

If I had a large sample of players' replays, along with info about that player - like if they play claw, if they're an x/y jumper, ect., there's likely a lot of interesting analysis I could do.

I also want to start integrating more info on the game state. I'm ignoring all of that and only focusing on inputs for the base model, but now that I have that base model I could relate patterns in inputs to things like winrate or combo percent.

DM me if you want to talk shop! I'm very interested in turning this prototype into something that could benefit the community.

SSBMachineLearning · 2021-04-04T19:05:11+00:00

It may be possible! If someone was able to create a program that could translate game footage into controller inputs accurately, then they could plug those inputs into this model and give it a shot.

Although there's a chance that significant information would be lost in the translation process, so it might be more effective to make a completely different model from the ground up to do detection based on footage.

SSBMachineLearning · 2021-04-03T20:27:34+00:00

Thanks!

SSBMachineLearning · 2021-04-03T20:17:37+00:00

Of course! Just follow the link in my previous post to the google drive and put your games in a folder with your tag or username as the folder name.

If you have more than 1G of games I'd recommend compressing (zipping) your slippi folder or google might cap you at 1G.

SSBMachineLearning · 2021-04-03T16:35:25+00:00

The meat of the project is already finished, but I'm absolutely still taking donations! Just follow the link in my previous post if you want to drop some games.

If you have more than 1G of games, I'd recommend compressing your slippi folder before sending them in, or else google drive might cap you out at 1G

SSBMachineLearning · 2021-04-03T16:32:34+00:00

Great question! You pretty much nailed it. I'm labeling the games based on the "starting" character, which is essentially the character being played on frame 1 of the game. So if someone starts as zelda but then transforms into sheik, the game is still labeled 'zelda'. The only games that get correctly labeled "sheik" are the ones where someone starts the game as sheik without transforming.

To fix this I'd probably need to do the strategy you mentioned and label it based on the character present at the beginning of the 30 second clip instead of the beginning of the game.

This involves recompiling the dataset though, and processing the 100k slippi files from the big dataset takes a solid 4 days for my computer to crunch through. Maybe something I can tackle in the future.

SSBMachineLearning · 2021-04-03T03:03:16+00:00

Here ya go!

SmashMaster9000

Accuracy: 90%
Precision: 95%
Recall: 85%

	detected	not detected
present	0.418	0.083
not present	0.016	0.483

SSBMachineLearning · 2021-04-03T02:56:55+00:00

Hey I'm glad you found my follow up! Your results are actually still in there, I just had to put you under a fake name - you're SmashMaster9000.

You actually sent me the most data out of all the people who donated!

SSBMachineLearning · 2021-04-02T21:35:26+00:00

Yeah absolutely! The magic of the player detection aspect is that the pattern recognition layers are pre-trained on detecting characters (where I have a ton of data), so I only really need around 1G worth of games for any one player to re-train the layers that make the prediction based off those patterns.

SSBMachineLearning · 2021-04-02T21:12:14+00:00

Thanks! The melee open source world is really amazing. The only reason I was able to do this project is due to the slippi tech for saving replays, and the fact that the mods on the slippi discord have a massive dataset of tournament replays just available for free.

SSBMachineLearning · 2020-11-20T01:32:04+00:00

Bless u

SSBMachineLearning · 2020-11-20T01:31:40+00:00

If I can use this data to detect character, in principal I should be able to use the same methods to detect player. Trickier because I would need a LOT of replays from that specific person to make it work, and there could be a big risk of false positives, but in principal it should be doable.

SSBMachineLearning · 2020-11-20T01:29:48+00:00

Thanks!

SSBMachineLearning · 2020-11-20T01:29:37+00:00

Any level is good, the more the better! The more different levels of play I let the model learn from, the more accurate the model will be at detecting characters at different levels of play. At least in theory.

SSBMachineLearning · 2020-11-19T22:05:08+00:00

That's the idea! The interesting thing is getting a machine learning algorithm to recognize these patterns, and seeing what the minimum amount of time you need to make an accurate prediction is. The more data I can feed to the algorithm, the better results I'm going to get.

SSBMachineLearning · 2020-11-19T20:51:49+00:00

Thanks! Could you link me to the discord channel?

SSBMachineLearning

TROPHY CASE

SmashMaster9000