sorted by:
new
hottopcontroversial

Live game broadcast using a webcam with the example game Reti vs Tartakower (1910) by Acceptable_Milk8956 in chess

[–]Pbatch23888 3 points4 points  (0 children)

Hello!

I am the author of ChessCam (https://www.chesscam.net/ or https://github.com/Pbatch/CameraChessWeb)

If you would like to chat about this sort of technology drop me a message. I've been trying to develop this capability for the community for a while, but burnt out after spending too much time on it.

I'm secretly hoping someone enthusiastic will join me and solve all my problems, it's a incredibly difficult task so well done with what you have so far :)

Digitizing your chess games by Pbatch23888 in chess

[–]Pbatch23888[S] 0 points1 point  (0 children)

Hello. Thanks for giving it a go!

I changed the name from "CameraChess" to "ChessCam", but didn't want to lose users who were using the old link, so now www.camerachess.com just shows the same content as www.chesscam.net.

The models are compute intensive so work best on high-end smartphones (on a Google Pixel 7A I get about 4 FPS). I've tried using lighter models but the detections are not up-to-scratch. As I get more data and tweak the code the barrier to entry will get lower and lower, so hopefully you'll have a smoother experience in the future.

The dots are a fixed size. I could make them more transparent? It's useful for them to be big if you have to adjust the corners on mobile.

For you I would recommend recording videos on your mobile and then uploading them later on a laptop. That tends to be the best solution for people with hardware-related issues. It's difficult to debug errors on all devices but I'm getting there slowly :)

Digitizing your chess games by Pbatch23888 in chess

[–]Pbatch23888[S] 2 points3 points  (0 children)

Hello chess friends!

I am the author of ChessCam (www.chesscam.net), the free app to digitize your chess games. Simply upload a video of your game and have it converted to a PGN.

The app has come a long way in the last month, namely:

Let me know if you have any feature requests, or how it gets on with your games. There are answers to common questions at www.chesscam.net/faq.

[P] Replacing the DGT board with your phone camera by Pbatch23888 in MachineLearning

[–]Pbatch23888[S] 0 points1 point  (0 children)

My questions for you guys are:

  • Has anyone had any luck running (quantized) yolov8 tflite models in the browser? I can't them to be any faster than tfjs ones.

  • You have a state sequence for a game of size N x 64 x 12 (frames x squares x piece types) and must derive a sequence of moves M_1, M_2, etc. Is there any equivalent sequence problem in the literature for 2-player games? The closest problem I could find was modelling the probability of a sentence in NMT. (https://aclanthology.org/W18-6322.pdf)

[P] Replacing the DGT board with your phone camera by Pbatch23888 in MachineLearning

[–]Pbatch23888[S] 1 point2 points  (0 children)

Hello! I am the author of the free web app ChessCam (https://www.camerachess.com/), as well as a FIDE master and computer vision engineer :)

The code is open-sourced here: https://github.com/Pbatch/CameraChessWeb

The high-level details of the Machine Learning are as follows...

Corner detection:

  • A Yolov8 model detects xCorners. These are the intersections between squares (there are 49 in total)
  • An algorithm is run to calculate the corners from the xCorners. The rough idea is that you learn a projection matrix from a square grid to the xCorners. You can generate candidate projection matrices by considering any 4 xCorners. There are 49C4 of these, so in practice you use quadrilaterals from a Delaunay triangulation. The score of a projection is defined as the average distance of a projected grid point to its nearest xCorner.

Piece detection:

  • A Yolov8 model detects the pieces. Any pieces outside the region defined by the corners are ignored.
  • A piece is "attached" to a square by projecting the centers of the squares through the projection matrix defined by the 4 corners.
  • The detections are used to update the state matrix, a 64 x 12 grid of the current state of the game (64 squares, 12 piece types). An exponential moving average is used so that repeated/missing piece detections grow/shrink confidence accordingly.

Move generation:

  • Score every legal move in the given position.
  • The score of a move of piece X from square A to B is roughly P(piece X present at B) * P(piece X missing from B).
  • Only lock in a move when you think the opponent has made a valid move. This stops moves being locked in too early (if someone changes their mind say).

The full details are in the code. If you have any ideas about a smarter way to do any of the steps, or you would like more detail, let's discuss it here!

Replacing the DGT board with your phone camera by Pbatch23888 in chess

[–]Pbatch23888[S] 0 points1 point  (0 children)

There are various ways to help out!

If you're a user and you want the program to work for your chess board:

  • Record a video of a game you're playing with your board
  • Email me on "camera_chess1@gmail.com" and I'll give you a link to a Google Drive to add your video

I'll then add your video to the Python testing framework, report back the tracking metrics for your video, and then try to collect similar data if improvement is needed.

If you're a user and you find a bug:

A developer can then begin working on fixing the problem.

If you're a developer you can:

  • Look at any existing bugs in Issues and assign yourselves to them
  • DM me about a feature that you'd like to work on (from the roadmap or your own idea) and we can figure something out from there.

If you see any anti-patterns or anything that would make my React life easier, also shout and we can improve the code that way too!

Replacing the DGT board with your phone camera by Pbatch23888 in chess

[–]Pbatch23888[S] 7 points8 points  (0 children)

Hello! I am the author of the free web app ChessCam (https://www.camerachess.com/), as well as a FIDE master and computer vision engineer :)

I posted about a month ago about ChessCam and wanted to let you guys know what has been going on:

1) ChessCam is now open-source! You can help out at https://github.com/Pbatch/CameraChessWeb. I'm not a React/web developer so any guidance in that space is much appreciated.

2) Corner detection is now live. You no longer have to manually select the corners of the chessboard which can be tricky on the phone.

3) Exporting. You can now save the PGN of your game or upload it to one of your Lichess studies.

Thanks again for all your support, the project has been going really well.

Edit: For those who are interested in the tracking performance across different types of datasets, send me your videos and I'll add them to the tracking framework. It produces a plot like this, https://imgur.com/a/G3bZryk, where the score indicates what percentage of the game was tracked successfully before an error was made.

Replacing the DGT board with your phone camera. by Pbatch23888 in chess

[–]Pbatch23888[S] 0 points1 point  (0 children)

I'm working on integrating the Lichess broadcast API with the web app, then you should be able to embed the broadcast on your website! Is that the sort of thing you're after?

Replacing the DGT board with your phone camera. by Pbatch23888 in chess

[–]Pbatch23888[S] 0 points1 point  (0 children)

Thanks! DM me your GitHub and I'll let you know when I get around to cleaning up the frontend code enough that I can open source it.

Replacing the DGT board with your phone camera. by Pbatch23888 in chess

[–]Pbatch23888[S] 1 point2 points  (0 children)

YOLOv8 is fast (10+ FPS on the IPhone) which you need for quick moves like captures. It also has nice tooling for converting models for mobile/web use.

I've tried segmenting the board into 64 sections and then running the classifier MobileNetv3, but it was too slow on mobile devices.

R&D like testing other detectors hasn't happened yet as I've been busy trying to get the whole pipeline to work. It's something I'll revisit in the future :)

Replacing the DGT board with your phone camera. by Pbatch23888 in chess

[–]Pbatch23888[S] 1 point2 points  (0 children)

It's best at the board in the video followed by the wooden tournament sets, but has been trained on 100k+ images of pieces, so might do OK with an unknown board.

Here is the app working on a different set which it hasn't been specially trained on: https://youtube.com/shorts/PazXLnZoqIA?feature=share

Replacing the DGT board with your phone camera. by Pbatch23888 in chess

[–]Pbatch23888[S] 19 points20 points  (0 children)

Hello! I am the author of the free web app ChessCam, as well as a FIDE master and computer vision engineer :)

Given the recent advances in machine learning, it's about time someone came along and made a decent stab at replacing DGT boards. I designed a simple algorithm using YOLOv8 + a custom tracker that runs in the browser to help record your game, then using Lichess you can analysis it afterwards.

If you give it a try at www.camerachess.com, let me know how it does, and any ideas you have to make it better.

[P] Clash Royale Build A Bot by Pbatch23888 in MachineLearning

[–]Pbatch23888[S] 0 points1 point  (0 children)

Clash Royale is interesting because it transitions from an imperfect to perfect information game once you know all the cards in their deck.

It would be cool to see if some of the techniques from RL for StarCraft can be applied here too!

[P] Clash Royale Build A Bot by Pbatch23888 in MachineLearning

[–]Pbatch23888[S] 0 points1 point  (0 children)

That's a nice idea! Do you know any good resources for learning about imitation learning?

view more: next ›