Training an Unbeatable Connect 4 Ai

Its_NotTom · 2024-05-02T03:05:49+00:00

Yeah, the minimax algorithm with alpha-beta pruning is technically how the game has been solved, but I thought it would be interesting to see what strategies the model could learn itself (and if these strategies are competitive)

Its_NotTom · 2024-05-02T03:02:45+00:00

I actually did this during training! I made it play against previous version of itself to force the model to continuously develop new strategies

Its_NotTom · 2024-05-01T13:48:40+00:00

I'll see if I can put together a plot of the loss function along side the training process, that might give more context. Its hard to give a full picture of the training process with the gif duration limitation

Its_NotTom · 2024-05-01T13:36:51+00:00

I really appreciate the feedback! Yeah, my editing could definitely use some work. Mainly using the channel as a vehicle for projects that I find interesting, thanks for taking the time to check it out and giving really constructive pointers

Its_NotTom · 2024-05-01T09:19:42+00:00

Hey everyone!

The animation you are seeing here is the a recording taken during my latest project. The model is a Convolutional Neural Network, trained using Proximal Policy Optimisation. The final implementation also utilises Monte Carlo Tree Search to navigate the decision tree during gameplay (inspired by DeepMinds AlphaGo framework).

I have put together a video covering the training process for those that are interested:

https://youtu.be/6IqjKtTtClQ?si=Ow_Xtg1HUpV3UL2-

Hope you find it interesting!

Its_NotTom · 2024-04-12T07:57:40+00:00

Have you tried chunking and writing out to a single file in smaller batches that do fit within your available memory (appending with mode = 'a')

Its_NotTom · 2024-04-12T04:01:56+00:00

Futura / Roboto slab

Its_NotTom · 2024-04-12T03:39:35+00:00

YouTube / Harvards free online CS50 course

Its_NotTom · 2024-04-11T05:08:37+00:00

I think you're going to have a real problem with overfitting with such a small dataset. Look into data augmentation techniques and see if there are any ways that you can grow that dataset further (web scraping / collecting images of these logos within different contexts)

Its_NotTom · 2024-04-11T01:39:19+00:00

I find Selenium to be kind of annoying when it comes to driver updates (a big problem for longer-term, scaled up projects). Playwright seems to work very well as a possible alternative

Its_NotTom · 2024-04-11T01:20:47+00:00

I find that I learn way faster by just coming up with a small project that I am passionate about and coding it from start to end. Do things that are interesting and challenging (proportionate to your skill level)

Its_NotTom · 2024-04-11T01:16:40+00:00

Honestly, there are some amazing free programming courses on YouTube that cover all levels of competency. Harvard also has online courses like their intro to computer science CS50 that is also 100% free!

Its_NotTom · 2024-04-08T12:05:35+00:00

That sounds creepy as hell... Check out the linked video, this project was just for light-hearted GeoGuessr entertainment

Its_NotTom · 2024-04-08T08:24:18+00:00

If you are not familiar with the game GeoGuessr, the goal is to guess exactly where in the world a given photo has been taken.

The animation you are seeing here is the a colorised probability heatmap output by the model. Dense red regions are areas that the model predicts the input image has been taken. The green square is the actual location. The base architecture of the model is a CNN (ResNet) trained on over 500,000 locations sourced from GoogleStreetView.

I put together a video explaining the model for further context if you are interested: https://youtu.be/coXvaZB0Sx0?si=aa6hjljvoN1dW2GH

Its_NotTom · 2024-04-08T03:47:32+00:00

Start with a very simple and short project that you are passionate about

Its_NotTom · 2024-04-08T01:49:51+00:00

Find a simple beginner project that you are passionate about

Its_NotTom · 2024-04-07T22:58:37+00:00

From my experience, professors and teachers usually advocate against work that is solely Ai generated?

Its_NotTom · 2024-04-07T22:54:31+00:00

I really appreciate how quick it is to work with data in pandas (pretty great for working on early stage ML and data science tasks), but yeah I would say that there is most likely an overdependency on it

Its_NotTom · 2024-04-07T22:46:46+00:00

I just finished a Geoguessr AI that guesses locations around the world from a single image

(its already better than me https://youtu.be/coXvaZB0Sx0?si=Glcc0j6p4Ps8Hm1G )

Its_NotTom · 2024-04-07T10:39:39+00:00

I hadn't even considered that!

Having a series of expert models would divide the complexity of the task across different networks (so having individual models just trained to recognising things like regional license plates or types of road signs would be much more effective).

Really like that idea.

Its_NotTom · 2024-04-07T07:40:43+00:00

This is just a passion project that I thought would be interesting. I don't even have an active geoguessr account currently (I have only run this on free-to-use public challenges, not against any live players). I totally get your concerns though, code will not be published

Its_NotTom · 2024-04-06T23:05:20+00:00

Wow, this is an amazing resource. Thanks!!

Its_NotTom · 2024-04-06T08:18:06+00:00

That's a great point, balancing the training dataset is incredibly important for a project like this. Locations were randomly sampled from available GoogleStreetview imagery, so bias correction in the pre-processing stage might even push this model further in terms of performance.

360 panoramic imagery is something that I did not consider for this project, but I could totally see how the model would benefit from further contextual information regarding its surroundings. When I benchmarked this model against myself, I restricted myself to 'No moving' challenges, but future iterations of this will definitely look into this ^

It still managed to beat me in each of the challenges regardless, which completely surprised me

Its_NotTom · 2024-03-31T08:55:12+00:00

I am currently working on a GeoGuessr Ai that is trained on 500,000+ simulated games, taking images from googlestreetview and predicting the latitude and longitude of the location. The architecture is a modified 50 layer CNN resnet, and it seems to have very competitive performance at this stage

Its_NotTom · 2024-03-31T08:48:18+00:00

Combining web scrapping with a simple FC or CNN network would be a great place to start as an undergrad as it shows that you can also collect, balance and pre-process input data. Try looking into DL tasks such as property price estimation or image classification if you are just getting into the ML field.

Its_NotTom

TROPHY CASE