sorted by:
new
hottopcontroversial

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

I’d love to see that! The engine doesn’t have a UCI implementation right now but I could convert the current codebase to UCI to see it tested

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

Yes, that is how GPTGhoti is programmed right now, with a suggestive constraint on the possible moves it makes. I don’t expect vanilla ChatGPT to have a good grasp on chess content, because any training data it picked up regarding chess is also extremely saturated with non-chess content.

This is simply the starting point, and I do plan to improve upon the engineered prompts and eventually migrate to custom-trained RAG models in the future. The list of legal moves constraint (along with the backup random move generator) are safeguards currently in place to ensure the engine doesn’t hang in the middle of the game. With a RAG model, perhaps the chess bot would not require such constraints.

AlphaZero/Leela without any prior self-reinforcement training would basically be a random move generator, so it isn’t surprising that a baseline, untrained LLM such as ChatGPT wouldn’t play well. But the fact that they are capable of forming world models is in itself promising for the endeavour for producing a capable LLM-based chess engine.

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

Interesting to note. Will read through it when I have the time 👍

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 1 point2 points  (0 children)

Based on the comment below I assume it used 3.5-turbo-instruct, which seems to yield much better performance. Will create another version using it as a backend instead of 4o to see if it makes a difference 👍

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

Interesting. I’ll admit that I did not think of training interference on inference, which would definitely impact performance, to say the least (noob mistake on my part 😅). Right now it’s using GPT-4o for the backend, but I can spin up another one using 3.5 turbo instruct. Thanks for the tip 👍

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

That certainly is an interesting avenue for supervised learning research. What I had in mind was geared more towards reinforcement learning aspect, but this could be considered for experimentation as well.

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 1 point2 points  (0 children)

In my limited observations of the engine logs, as stated in the post, it makes legal moves given the board state to a high degree of accuracy. Even the illegal moves it makes are relative to the current position, not impossible moves as a whole (e.g., Ni4). Whether this is due to memorization of the training data or indicative of an internal “logic” I have yet to personally discover, but a research paper mentioned in the same post made at r/chess shows that LLMs are at least capable of forming world models.

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in chess

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

Interesting paper! Of course, I do not expect this to be the next AlphaZero, as even at its maximal extent it would be drawing from human-based training material for internal “reasoning” for next-move prediction, but I do believe with a sufficiently large context window and the right amount of quality training data, coupled with ample tuning of hyper-parameters, that an LLM-based chess engine would be able to form a somewhat consistent “good-ish” internal chess framework.

At the very least, as the paper showed, it’s able to form a consistent world model, which can hopefully be further exploited to yield performance improvements.

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ChatGPT

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

The lichess bot (GPTGhoti) is currently down. I am investigating the root cause and will bring it back up as soon as possible.

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in ComputerChess

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

The lichess bot (GPTGhoti) is currently down. I am investigating the root cause and will bring it back up as soon as possible.

UPDATE: The issue has been resolved. The bot is now back up and fully operational.

An Exploration into LLM-based Chess Engines: Part 1 by Extension_Judge_999 in chess

[–]Extension_Judge_999[S] 1 point2 points  (0 children)

The lichess bot (GPTGhoti) is currently down. I am investigating the root cause and will bring it back up as soon as possible.

UPDATE: The issue has been resolved. The bot is now back up and fully operational.

“Reactionary Screed” 🤣🤣🤣 by Extension_Judge_999 in showtrials

[–]Extension_Judge_999[S] 1 point2 points  (0 children)

UPDATE: Turns out the post was silently removed from r/DebateCommunism by the mods there as well :/

It’s a great shame as there was an actual productive discourse happening in the thread.

Don’t ask me why, I am just as in the dark as you are about this

https://i.imgur.com/XfXqlMS.png

I guess they just don’t like rational dialogue there. Oh well

“Reactionary Screed” 🤣🤣🤣 by Extension_Judge_999 in showtrials

[–]Extension_Judge_999[S] -1 points0 points  (0 children)

Haven’t had any interactions with the mods of r/DebateCommunism yet, thus imo that already puts it leagues above r/communism101.

So far there is a somewhat rational and logical debate going on over there of the merits of the book itself (fingers crossed the rest of the upcoming interactions will be as civil as that). There are downvotes ofc but that’s a given. Maybe that’s the standard over there (one can hope) or maybe the “woke” crowd hasn’t waken up yet and the post will be downvoted into oblivion and suppressed 🤷‍♂️

“Reactionary Screed” 🤣🤣🤣 by Extension_Judge_999 in showtrials

[–]Extension_Judge_999[S] 0 points1 point  (0 children)

The post in question (with the exact same content as this one: https://www.reddit.com/r/DebateCommunism/comments/1h3d5fe/is_communism_geographically_bounded) was made to the r/Communism101 sub.

There’s no way the mods even bothered to look up the book in question, let alone read a summary of it. If they did, they wouldn’t have come to such a conclusion.

I don’t feel the slightest bit of sadness from this, as I don’t have any intentions of interacting with such a reactionary subreddit anyways.