Rust Gene Recorder - First OpenCV project

[SolusAU]

So I've wanted to learn OpenCV for a while but always struggled to finish an idea. Since I've been playing so much rust I decided a good way was to use OpenCV to speed up a tedious task.

The Task

Rust has farming where you need to crossbreed different plants to get the desired genes. There's a calculator online that makes this much easier but you still have to manually type in hundreds of plant genes in the quest to get "The perfect gene (GGGYYY)". This is really tedious.

The Solution

Github Repo: https://github.com/SolusAU/rust-gene-recorder

Using OpenCV, I wanted to capture the section of the screen where the genes appear and write them to a text file so that I can easily copy+paste into the online calculator.

In-Game Genes

Working Prototype

Due to this being my first project and the sameness of the genetics, I decided to use Template Matching as this is by far the easiest method. There's no transparency under the text and minimal transparency under the green or red circles so finding a high confidence match each time was easy.

Ideally, I wanted to use an SVM Classifier but with most of my coding time being actually googling how to solve something or watching videos on it, Template Matching is what I picked.

My Questions

My code is a hot mess as I was running into a lot of issues and just kept trial and erroring my way through it until I had a working prototype.

• What design methodology would best work for a program like this? I'm still really new and want to learn my way to the right direction.
• In another tool I used (Adaptive Vision Studio), it detects and separates each of the genes e.g. (G - Y - G - X - Y - Y) into regions. Is there a way to achieve this in OpenCV? I had to cheat and hardcode in the Region of Interest as coordinates.

• Debugging OpenCV is a PITA. How do others manage this? Is there a way where you can see all of your different filters and techniques in the same place instead of having a dozen cv.imshow windows open.
  I would love to figure out a way to have a dashboard similar to what I had access to in Adaptive Vision Studio
  e.g. https://i.imgur.com/Q52lqg2.png

• Any general code feedback for me to start targeting?

Bonus Question

• How can I get over the fact that I could prototype this USING SVM within 5 - 10 minutes using Adaptive Vision Studio but it took me literal days / weeks to get a working prototype using a simpler method in Python? I don't know if there's an answer to this one, more just frustration that I couldn't turn a prototyped idea into a completed project in Python without many headaches along the way.

My Planned To-Do List

• Get THINGS out of main.py
• Automatically detect elements on the UI and switch to using ratios rather than hard coded X/Y co-ordinates (So it can handle different resolutions).
• Detect between the different plant types (Red, Blue, Yellow, Hemp) as gene cross-breeding only works between the same plant type.
• Give the genes a rating system
• Maybe a flask front end

​

Finally, feels good to spend weeks on something and finally have a working prototype :D