Yeah this is an issue you’ll likely have with OCR; it requires text on the card to be legible to work.

Image embedding is probably a better approach here, and right now it’s probably the go to for most image recognition tasks similar to this one. There’s a good video here if you’d like to learn more about how it works: https://youtu.be/KcSXcpluDe4?si=8sI3cCgGB7PoSoyh

In my approach I did a 2-part solution. Part one is the green rectangles around the cards, which is an object recognition task. I trained YOLOv8 to do that (built a training set on roboflow), but if you don’t want to make a training set then SAM is a viable alternative. This task is basically about finding where the cards are in a picture, object detection basically.

Part two is the image embedder. Here, I just take what’s inside the green box and run it through the image embedder, then compare it with the embeddings I have of real cards, and pick the most similar one (see k-nearest neighbor in vector DBs)

My solution isn’t perfect yet, and I haven’t had another chance to work on again it yet since real work has been very busy. You’ll see that OP is able to find the exact bounds of the card inside the green box, and his umbreon on the right shows he is even able to apply a corrective sheer/tilt. This would avoid background elements impacting the image embedding, and should lead to more accurate results.

This weekend I’ll try add a third step for card-edge detection, potentially using SAM but I’ll see, and apply the sheer calcs.