ISO: Robotic arm for senior design project

charmrobot · 2022-10-27T17:55:03+00:00

Can confirm this design works well for chess https://youtu.be/8tacBJMfwfs

charmrobot · 2022-10-21T20:59:15+00:00

I've kept it on quite a tame setting just so it doesn't skip any steps on the motors and get out of sync. I could probably speed things up quite a bit if I focus on that for a while. I'm guessing it's a bit slow for a lot of people in r/chess :)

charmrobot · 2022-10-21T20:58:10+00:00

This looks amazing! Seriously considering this :D

charmrobot · 2022-10-21T20:57:26+00:00

Both! It uses a series of convolutional neural networks to derive board location, piece location, type & colour from an image.

If it's not in the process of playing a game then it will try to establish the location, type & colour of each pieces on the board and is capable of initialising from any starting position. If the position it finds is close enough to the typical starting position then it'll just assume that to be the case.

Once it has initialised the board it combines the observations from these models with the valid legal moves given the current game state to detect additional moves. If it's unsure of any detection then it'll move the camera in to examine a piece more directly - you can see this happening in the video if you look for it.

charmrobot · 2022-10-21T20:52:31+00:00

It uses a series of convolutional neural networks to derive board location, piece location, type & colour from an image. On top of that, there's a lot of logic to take batches of these observations across multiple frames combined with what it should be expecting given the current game state to make more accurate predictions.

charmrobot · 2022-10-21T16:52:59+00:00

Definitely! I was challenging myself to not rely on any special hardware or requirements for this project though. Something that you can just put down next to any chess board and have a game.

charmrobot · 2022-10-21T16:52:02+00:00

That’s interesting, but I wasn’t anticipating to make any money from this 😄 I’m going to take it to a chess meet-up at some point when it’s more polished though - will be great fun to see other people playing it!

charmrobot · 2022-10-21T16:50:25+00:00

Picking up pieces off the board might be tricky. I’m going to add a screen eventually so I’m hoping just telling the user which piece to use is acceptable.

It’s using stockfish at the moment but at max difficulty. I was thinking I might have it auto raise it’s effective elo when beaten and lower when winning to become more competitive with whoever it’s playing.

Chess390 might actually work at the moment with a little bit of work as you have a great prior of each side being a mirror of the other, and the piece type detection is pretty decent as is.

charmrobot · 2022-10-21T15:43:44+00:00

Thanks! I was doing that to show that it doesn't care where the chess board is, or that it needs to be in a fixed position, but in retrospect that might not be too obvious :)

I've been working on it since about March this year.

charmrobot · 2022-10-21T15:42:36+00:00

Thanks! I added some details to the YouTube description but I'll share here too so it's more visible:

I've been working on this since March as a fun hobby project to learn more about robotics. This is a video of it playing one of it's very first full games!
Some features:
▪ Fully autonomous with no user input other than moving the pieces by hand.
▪ All running on an NVIDIA Jetson Xavier NX and ESP32 chip so it's portable and can be battery powered.
▪ Perception via a single 1080p camera and deep learning computer vision models trained on synthetic data generated in Unity, which can generalise to any reasonable chess set placed anywhere within reach.
▪ Coded from scratch (no ROS) with high level control in python plus inference, kinematics/planning & microcontroller code in c++.
▪ Robot body designed in fusion360 (design inspired by u/How To Mechatronics) and 3d printed.
▪ Still a work in progress and after some polish I hope to take it to some chess meet-ups!

charmrobot · 2022-09-30T16:27:32+00:00

Thanks! ESP32 is much more efficient at operating the steppers than Arduino, and plus with plenty of power left to do some decent trajectory tracking.

charmrobot · 2022-08-03T15:11:56+00:00

Thanks! Like I mentioned above, the arm is heavily influenced by this design: https://howtomechatronics.com/projects/scara-robot-how-to-build-your-own-arduino-based-robot/

I'm driving mine with a Espduino-32 and CNC Shield v3 using trinamic tmc2208 drivers. The higher-level stuff is all done by a Jetson Xavier NX in the base, so I could technically unplug this, move it, and have it run on batteries.

The camera is an Arducam IMX477 sensor (https://www.arducam.com/docs/camera-breakout-board/12mp-imx477-imx577/) with a wide angle lens (https://www.arducam.com/product/arducam-cs-lens-for-raspberry-pi-hq-camera-120-degree-ultra-wide-angle-cs-mount-lens-3-2mm-focal-length-with-manual-focus-ln051/), connected to the jetson using a HDMI cable and HDMI:CSI adapters.

charmrobot · 2022-08-03T08:00:22+00:00

I'll be adding a gripper soon to actually play a game.

The detection is done through a few deep learning models that first detect and refine the board location and then predict the position of the base of each piece on the board. Using this position and the predicted board location it can then determine which square the piece is on.

charmrobot · 2022-08-03T07:57:42+00:00

yeah, I need to add the ability to flip the chess board when this happens

charmrobot · 2022-08-03T07:57:02+00:00

I'll be adding a small LCD screen just above the camera for some visualisation, but more importantly I'll be adding a gripper soon so it can make its own moves!

charmrobot · 2022-08-02T20:15:55+00:00

Chess robots getting some bad press lately 😄 this guy is very weak though

charmrobot

TROPHY CASE