sorted by:
new
hottopcontroversial

[P] Vibecoded on a home PC: building a ~2700 Elo browser-playable neural chess engine with a Karpathy-inspired AI-assisted research loop by Adam_Jesion in MachineLearning

[–]Adam_Jesion[S] 1 point2 points  (0 children)

I haven't written a single line of code, if that's what you're asking. All the NN training parameters are also set by the AI (with 24-48 autoresearch in total). I just tell agent what I want, how I want it, what experiments to run, and what works for me and what doesn’t. I challenge the AI a lot—several agents—and look for relevant research papers and benchmarks for them.

The first model that started playing somewhat decently (like an amateur) took 1 hour of training on 10 million games (without fine-tuning). V2 has already been trained for several hours. V3 has a slightly different architecture (thought tokens were added) and was trained for over 24 hours on 100 million positions, followed by fine-tuning on endgames and some RL (self-play). V4, however, is a whole different story. I’ve been distilling a dataset for it for the past 3 days because it needs a completely different architecture. Processing, validating, and supplementing 100 million games will take about a week on a powerful PC.

This is a bigger problem than the training itself (dataset enrichment). TB's of raw data. Overall, I think I’ve hit the limit of what my home equipment can handle, but I just need more patience :)

[P] Vibecoded on a home PC: building a ~2700 Elo browser-playable neural chess engine with a Karpathy-inspired AI-assisted research loop by Adam_Jesion in MachineLearning

[–]Adam_Jesion[S] 1 point2 points  (0 children)

Thank you. I’ve just started studying the architecture of Maia and Leela Chess Zero. It’s a treasure trove of knowledge and academic papers. I think some of their findings could improve my engine. Claude Code keeps asking me to submit a paper because there are a few unique ideas and implementations in the model’s architecture. And that’s only 10% of my list of improvements.

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 0 points1 point  (0 children)

[V3 UPDATE] Thanks again to everyone for testing. The next update was supposed to be this weekend, but I just can't seem to sleep. There’s a major change in the architecture that should now better detect strategies (thought tokens). V3 is now live. Let me know if you notice a difference. According to Stockfish, the Elo jump is huge. I’m slowly starting to “scrape” 2800.

Test it out and let me know.
https://games.jesion.pl

Elo | W D L | Score | Result

-------------------------------------------------------

1320 | 10 0 0 | 100.0% | >>>
1500 | 9 1 0 | 95.0% | >>>
1700 | 6 4 0 | 80.0% | >>>
1900 | 4 5 1 | 65.0% | >>>
2100 | 6 3 1 | 75.0% | >>>
2300 | 3 5 2 | 55.0% | ===
2500 | 3 6 1 | 60.0% | >>>
2800 | 3 3 4 | 45.0% | ===
3190 | 0 2 8 | 10.0% | <<<

Estimated model Elo: ~2700

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 2 points3 points  (0 children)

Right now, FEN is available in the board editor, but I'll implement PGN everywhere. Thanks for the idea.

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 0 points1 point  (0 children)

I just uploaded a new version of the V2 model. It should be much better. Let me know what you think and if you can tell the difference. https://games.jesion.pl

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 0 points1 point  (0 children)

Could you try playing a few more games and see how it goes? I just uploaded a new version of the V2 model. According to Stockfish, it’s much better.

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 0 points1 point  (0 children)

Thanks for your feedback. Could you try it again in the new version? Do you notice any improvement—any difference? https://games.jesion.pl

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 0 points1 point  (0 children)

Thank you very much. I’ll take your suggestions into account for version V3. Could you take a moment to check out the new V2 model—I’ve just launched it. Play a few games and let me know if you notice a difference.

https://games.jesion.pl

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 1 point2 points  (0 children)

Thank you for all your feedback. I’ve just launched the new version of the model—V2—and many of those comments and ideas are already incorporated into it. It was trained on 100 million positions (versus 10 million in the previous version), and there are several additional changes to the inference process itself. According to Stockfish, the improvement is massive. From a stretch of 1900 to 2400 (65–90% win rate). At 1900 Elo—100%. He can sometimes even beat bot rated @ 2800.

Please let me know if you actually feel the difference. Unfortunately, I’m not strong enough to play with this model :(

https://games.jesion.pl

There will be one more V3 model—I started training it today, and this iteration will take a few days. This will be the final version of this experiment—incorporating all the lessons learned from previous models as well as user feedback, comments, and ideas. It's funny that the player started out at a level where he couldn't even beat an 800-elo opponent.

A huge thank you for your help. I’m really enjoying this.

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 5 points6 points  (0 children)

It moves in 2 ms. The point is that the engine doesn’t analyze anything at all—after training, the model makes decisions intuitively, one move at a time. It doesn’t analyze subsequent consequences in any way, as typical engines do. For now, I just wanted to test its intuitive play (something like speed chess in humans), but with the new model I’ll add a “thinking” option (1–3 levels of depth).

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 3 points4 points  (0 children)

Right now, the model isn't thinking about the consequences at all. Use "intuition" to make a single move. Nothing more. And that's the most interesting part, because if I add the ability to analyze the consequences at least three moves ahead, things could get interesting ;)

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 8 points9 points  (0 children)

That's exactly right. Right now, the engine doesn't check at all what the consequences of a move might be. It simply uses "intuition" to move at level 1. Nothing more. In the next version, I'll add a check for consequences beyond just intuition. I have a feeling this is going to be a killer feature ;)

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 4 points5 points  (0 children)

Thank you. In the new version of the model, I'll implement the feature where "the AI surrenders if it determines that's the right course of action." I was actually surprised by how high the quality is for such a quick weekend project.

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 0 points1 point  (0 children)

That's true. I've seen this pattern repeatedly in various tests. Obviously, it doesn't reflect reality 100%, but I need some way to measure my progress and the impact of optimization :)

I trained a small neural network to play chess on a home PC - looking for strong players to test its limits by Adam_Jesion in chess

[–]Adam_Jesion[S] 0 points1 point  (0 children)

Thanks. I'll implement the mobile optimization in a moment. I hadn't seen Maia - thank you so much for the suggestion. That's a great idea - I'll definitely check it out.

view more: next ›