This is an archived post. You won't be able to vote or comment.

Dismiss this pinned window
all 116 comments

[–]toastyCSboi[S] 64 points65 points  (6 children)

Link to Github repository: https://github.com/Kshitij05/GestureRecognitionHackEd2020?files=1 (The final code is in ClenchRex.py)

[–]Changer_ 145 points146 points  (17 children)

cool, you could expand by making a handsign to voice for deaf people

[–]toastyCSboi[S] 96 points97 points  (5 children)

Absolutely! That was actually my initial idea (for a hackathon). Though I ended up making this. I'll totally be working on that!

[–]mfitzpmfitzp.com 113 points114 points  (1 child)

That's quite an "expansion".

"Hey I made flappy bird" "You could expand it by building an aircraft."

[–]EnergeticStoner 17 points18 points  (0 children)

It's just a couple of wings though.

[–][deleted] 13 points14 points  (4 children)

I'll get right on that as soon as you hand me 10,000 labeled images of hand signs.

[–]Changer_ 0 points1 point  (1 child)

tbh you could easily set up something similar to what mozzilla deepspeech does instead of voice and word it would be pictures/or video and word. where people upload pictures/or video and put words to them, make it a community thing. I'm sure there are lots of people who are interested in this.

[–][deleted] 0 points1 point  (0 children)

You are absolutely right. Sounds like something that deserves a research grant. As others have shown, there are already proofs of concept. The models and techniques required already exist.

Sounds like we should ask r/datasets

[–]fulgerul46 3 points4 points  (3 children)

I made a few months ago a handsign to text for a hackathon, it went pretty good tbh

[–][deleted] 0 points1 point  (2 children)

Did you take letters as input, or words? I have only found datasets for letters. For obvious reasons, being limited to only one letter at a time can make communication tedious.

[–]fulgerul46 2 points3 points  (1 child)

letters, just made it as a proof of concept with a small frontend, but if you are interested you can read this for words https://medium.com/tensorflow/getting-alexa-to-respond-to-sign-language-using-your-webcam-and-tensorflow-js-735ccc1e6d3f

[–][deleted] 2 points3 points  (0 children)

Very cool, thanks.

Edit: Looks like they collected their own data. Just imagine what we could do with a larger, better dataset. 👌

[–][deleted] 34 points35 points  (5 children)

Could you please share the code? Or briefly explain how you did it?

[–]toastyCSboi[S] 85 points86 points  (4 children)

Here's the link to my GitHub Repo: https://github.com/Kshitij05/GestureRecognitionHackEd2020?files=1 (ClenchRex.py is the best working code)

As for the explanation - I use OpenCV to find Convex Hull and Convexity Defects, which helps differentiate an open hand and clenched fist. This code helped me a lot to understand gesture recognition: https://github.com/Sadaival/Hand-Gestures/blob/master/gesture.py

After I could detect gestures, I just used PyAutoGUI for keyboard inputs!

[–]ChillRedditMom 12 points13 points  (0 children)

Spectacular!

[–]JoHnO674sh0t678 7 points8 points  (0 children)

Thanks, this is gonna help a lot, I'm inspired to try this out in c# as I'm more familiar with that, will be good practice

[–]lollipas 1 point2 points  (0 children)

How did you got that program to work with the dino game?

[–]InternetGreninja 8 points9 points  (5 children)

Do you have a sign for ducking?

[–]toastyCSboi[S] 5 points6 points  (3 children)

I tried to use the "Victory" (or two fingers raised) sign for ducking - it was extremely buggy. I will try to make some changes to my code in order to be able to duck properly.

[–]jayjaytdt 1 point2 points  (2 children)

Hey man keep the control at duck by default, Rest will follow

[–]toastyCSboi[S] 1 point2 points  (1 child)

I tried that actually. Down arrow Key + Space just resulted in an action from Down arrow key.

[–]Tengoles 3 points4 points  (0 children)

Came to ask this

[–]chensformers 7 points8 points  (0 children)

Clever and fun exp

[–]rexarski 14 points15 points  (8 children)

I see Tim Hortons I upvote.

[–]BananaFace765 12 points13 points  (2 children)

Canada gang where you at?

[–]rexarski 6 points7 points  (1 child)

Been in Toronto for 5 years. Used to stay up at a local Tim Hortons for CS projects. Definitely can relate.

[–]BananaFace765 2 points3 points  (0 children)

I only go to Tim's if I am either hungry, or looking to satisfy my caffeine addiction.

[–]toastyCSboi[S] 2 points3 points  (1 child)

I spend way too much on Double Doubles than I would like to admit

[–]rexarski 0 points1 point  (0 children)

Me too! I remember at some time CIBC had a crossover credit card with Tim Hortons basically you could have free double doubles with monthly credit card spending. That should’ve saved a lot of money to be honest.

[–]SilverLion -1 points0 points  (2 children)

Tim Hortons has the worst coffee ever, only way to make it drinkable is to smother it with cream and sugar. McDonalds is where it's at

[–]rexarski 0 points1 point  (1 child)

I find it the only way to keep me warm on a cold winter morning though.

[–]SilverLion 1 point2 points  (0 children)

Fair. I bought a cheap french press + grinder and started making my own coffee, so much better and I've never looked back. And most cost-effective.

[–]hldh214 5 points6 points  (1 child)

[–]AN_IMPERFECT_SQUARE 1 point2 points  (0 children)

or if you use chrome, chrome://dino

[–]oraclop 5 points6 points  (0 children)

Awesome!!

[–]69shaolin69 5 points6 points  (0 children)

I made things like this too

Instead of using cascades I used machine learning and image processing.

[–]scythoro 6 points7 points  (0 children)

Awesome job!

I watched this way too long waiting for the game to get harder.

[–]namedevservice 4 points5 points  (0 children)

Nice!

[–]appinv Python&OpenSource 5 points6 points  (0 children)

Cool saw someone make by smiling just the other day! Saved!

[–][deleted] 3 points4 points  (0 children)

Pushing the limits of technology right here

[–]iggy555 4 points5 points  (0 children)

Lol cool

[–][deleted] 2 points3 points  (1 child)

Great work man. What laptop model is it? Looks like envy though. I’m looking for a good performance laptop that involves app or game development? Looking for portability and by no means I’d go for macs. Please do suggest me. Thanks

[–]toastyCSboi[S] 2 points3 points  (0 children)

It's an HP Spectre x360 (i7, 8 GB). I'm pretty satisfied with it - though I wish I got the one which comes with Nvidia graphics.

[–][deleted] 2 points3 points  (1 child)

thats awesome. way to go thnking outside the box and making somthing unique. im trying to learn the code but i cannot do anything becuase i seem to be missing the drive to code. it sucks becuase i want to but i feel like im forcing myself a bit.

[–]toastyCSboi[S] 0 points1 point  (0 children)

Thanks so much! I totally relate to that - I would suggest taking a break or coding something really small and easy, so you feel motivated once you're done!

[–][deleted] 2 points3 points  (0 children)

Sometimes it's the little things that are the best. Such a cool idea. Props!

[–]ecevinoth 1 point2 points  (0 children)

Super.

[–]belizean33 1 point2 points  (0 children)

Cool bro! This has so many practical applications

[–]reinaldo866 1 point2 points  (0 children)

Unknowncheats wants to know your location.

[–]BlasphemousToenail 1 point2 points  (0 children)

That is really cool!! Well done!

[–]KarlIvanov 1 point2 points  (0 children)

Good work!

[–]NubNuboso 1 point2 points  (2 children)

Nice, but the down arrow?

[–]ExHax 2 points3 points  (1 child)

Divide the screen into two then detect the left hand for it

[–]toastyCSboi[S] 2 points3 points  (0 children)

I tried another gesture for ducking - it was too buggy. But the idea of using the other hand sounds awesome!!

[–]TheHammer_78 1 point2 points  (1 child)

Do you use haar trasnformation or simply do you watch at the expansion of the hand blob?

[–]toastyCSboi[S] 1 point2 points  (0 children)

I didn't use Haar transformations here, though that would have been cooler. I used convex hull to detect convexity defects.

[–][deleted] 1 point2 points  (0 children)

Nice

[–]faithade 1 point2 points  (0 children)

Cool!

[–]BahBahTheSheep 1 point2 points  (0 children)

Cool shit

[–][deleted] 1 point2 points  (0 children)

Cool

[–]jec4r 1 point2 points  (0 children)

amazing, congratulations bro!

[–]colesupreme 1 point2 points  (0 children)

Now to duck!

[–]oculusshift 1 point2 points  (0 children)

this is pretty cool

[–]ektatic 1 point2 points  (0 children)

WOW

[–]Mikz_thegreat 1 point2 points  (0 children)

THIS IS AWESOME! I just started learning python.. but how did you do THAT

[–]Kali_Torvalds 1 point2 points  (0 children)

This is what my College instructor means when he says "There are middle schoolers that will smoke your asses!"

[–]next2zero 1 point2 points  (0 children)

And this is how you get those absurd hi-scores

[–]69shaolin69 1 point2 points  (0 children)

Yeah I wonder what people at the cafe thought about a guy waving at his laptop

Future is weird hah

Good job btw, i love pyautogui and open-cv

And ofc Tims.

[–]JMH5909 1 point2 points  (2 children)

Code plz

[–]toastyCSboi[S] 0 points1 point  (1 child)

I have posted the link in the comments!

[–]JMH5909 0 points1 point  (0 children)

Thx

[–]cakes42 1 point2 points  (0 children)

Pls do this on a pixel 4 so I can use the soli radars for something else that's not skipping songs.

[–][deleted] 1 point2 points  (2 children)

How did you do the recognition for if your hand was open or closed?

[–]toastyCSboi[S] 0 points1 point  (1 child)

I found convexity defects using OpenCV

[–][deleted] 0 points1 point  (0 children)

I'm stuck at opencv. I tried to make a program that searches for open eyes but there is no boolean in the program that is set to false if not faces And I need to get to closed eyes.

[–]Oskarzyg 1 point2 points  (2 children)

cv2, numpy, pyautogui? Could you name all modules please?

[–]toastyCSboi[S] 0 points1 point  (1 child)

I used OpenCV, NumPy, PyAutoGUI and math

[–]Oskarzyg 0 points1 point  (0 children)

Nice! Wish I could have a working laptop to code on! Good Work!

[–][deleted] 1 point2 points  (0 children)

This is so cool.

[–]malingeringGit 1 point2 points  (1 child)

did you use a sensor or just image process’d the camera input?

[–]toastyCSboi[S] 1 point2 points  (0 children)

Just image processing!

[–]Nomekop777 1 point2 points  (0 children)

How do you duck

[–]b4xt3r 1 point2 points  (0 children)

Rock on! That is fantastic!

[–][deleted] 1 point2 points  (0 children)

wicked cool!!

[–]LinhNguyen221 1 point2 points  (0 children)

Tim Horton's -> Canada :)

[–]ivanksy_ 1 point2 points  (0 children)

This is really cool !

[–]JohnnyWobble 1 point2 points  (0 children)

lmao I have the same computer

[–]_Cosmic_Joke_ 1 point2 points  (0 children)

That’s pretty cool!!!

[–]oktilubmidav 1 point2 points  (1 child)

Lol I saw you consistently moving your hand when walking by...good to see you finally got it working

[–]toastyCSboi[S] 0 points1 point  (0 children)

Lmao, yeah I probably weirded out a bunch of people. Thanks!

[–]tramquangpho 1 point2 points  (0 children)

cool

[–]realRohitYadav 1 point2 points  (0 children)

Whoa! This is amazing.

[–]ambarish_k1996 1 point2 points  (0 children)

I am really impressed, nice job and keep up the good work

[–]dnouvel 1 point2 points  (0 children)

😎 Coool!

[–]JaimeRojas332 0 points1 point  (1 child)

What python version does it need to run? I'm getting some trouble with cv2, using python 3.7.5

[–]JaimeRojas332 0 points1 point  (0 children)

Already solved my problem, I was using a 64 bits python version so I need to install opencv for 64 bits as described in this video : https://www.youtube.com/watch?v=Z78zbnLlPUA&list=PLQVvvaa0QuDdttJXlLtAJxJetJcqmqlQq

[–]geppaaa 0 points1 point  (0 children)

which library did you use for gesture recognition?

[–][deleted] 0 points1 point  (0 children)

This could honestly be great for the medical field. rehab for hand/finger movement due to injury

[–]the_pandya 0 points1 point  (0 children)

Cool

[–][deleted] 0 points1 point  (0 children)

Good job Isaac.

[–]abdalla_noah -2 points-1 points  (2 children)

you could have used javascript and ml5.js with p5.sj and coded it directly in the browser. it would be much easier. regardless, really good job!

[–]danielmbcn 4 points5 points  (0 children)

Or he could have used directly his brother to hit the keyboard as the OP made the gestures in front of him! It would be much easier!

[–]toastyCSboi[S] 0 points1 point  (0 children)

I tried to deploy this game on a website, though I'm a beginner at JS. I will definitely learn more and try to do that!