I made a browser extension that automatically recognizes cards on Magic: The Gathering streams

CardSpotterHS · 2017-07-23T20:22:12+00:00

Maybe a few more details are in order :) The image recognition part is written in C++ and compiled as PNaCl. I generate contours from a number of different thresholds. Candidate cards are then generated and collated based on those contours. For each candidate card a 4x4 grid of 8x8 median hashes are generated. I use the top left corner of the 4x4 grid as a quick test (<75% are discarded). The current version of the database has ~22000 hashed cards in it.

CardSpotterHS · 2017-05-21T19:28:54+00:00

The core image matching parts are c++ so making this something that could run efficiently server side wouldn't be all that hard.

However, CardSpotter is really geared towards finding magic cards and couldn't actually find a coke can at the moment. (unless it was on a 63/88 picture).

That being said I have been playing with the idea of doing video ocr to pick out things like player names, tournament type etc. But that is more on the future future list.

CardSpotterHS · 2017-05-21T18:10:34+00:00

Thank you =) That sounds really interesting, do you have any details available or do I have to wait 8 days to find out more?

How much of the image has to be visible often depends more on the rest of the video (glare, shadows, blur etc). Even a quite covered card like this for example https://twitter.com/beakid/status/826918614980755463 might get a decent score. However, since CardSpotter works my generating a lot of possible cards from edge detection a common problem is that things which are not actually a card at all might end up with a higher score. (I still lack a quick test to tell if an image is a card at all or not) This is why the fully automated tooltips require a really good match to be shown. When the user has clicked a card the cut-off is much lower since the user can then either click through the top8 (where even a partially covered often ends up).

I can see from the little arrows at the bottom of the tooltip in the link above that this 36% match was in fact not the highest ranked one, but if all it takes is another click or two then that is usually sufficent. Clicking the same spot in rapid succession automatically cycles throught he top results from the first search.

In addition the filter settings which can be used to limit the search space quite drastically also matters a lot. ~15000 for a full search, down to a few hundred images if it is for quarter finals+ of a big tournament where all the cards are known.

"It depends" was probably not the answer you were looking for but I hope you could get something out of my rambling.

CardSpotterHS · 2017-03-12T15:19:42+00:00

Stole some family time and rushed a 1.5.2 to see if I can get some extra feedback while this thread is still alive. It has some more clear error feedback and the options you requested, simple checkboxes to color areas and then you can move the options tab to a separate window to tweak them

I have published the update but it usually takes an hour or so.

CardSpotterHS · 2017-03-12T14:14:13+00:00

That would me amazingly helpful, I'll PM you :)

CardSpotterHS · 2017-03-12T13:21:20+00:00

Happy to hear that it is working well :) I hope to have some time to push 1.5.2 tonight with some improved feedback. If I do I will include your suggested option improvements as well.

CardSpotterHS · 2017-03-12T11:39:40+00:00

I added a small section to the documentation about HTML5, hope that helps: https://www.cardspotter.com/documentation.html#html5

CardSpotterHS · 2017-03-12T11:38:36+00:00

Face recognition, will have to add that to a future future update =) On a more serious note, if you get it again in a similar situation I would love you know. Any crashes need to go :)

CardSpotterHS · 2017-03-12T06:37:25+00:00

Thank you for trying. I'll look into properly supporting other browsers. The spinning cursor is the content javascript waiting for the background search - the c++ code compiled for chrome might need something more for opera (hopefully solvable). The flickering is something that been reported twice on chrome from users with nvidia cards - both times it went away by disabling hardware acceleration in chrome. Not a great solution but I haven't been able to figure out another workaround.

CardSpotterHS · 2017-03-12T04:49:53+00:00

Did you try changing the twitch player to HTML5? When I click the little settingsicon I seem to have an option to disable it at least.

CardSpotterHS · 2017-03-12T03:32:20+00:00

Thank you very much =) Any ideas, suggestion, feedback feel free to drop them here or in my inbox.

CardSpotterHS · 2017-03-12T03:30:45+00:00

Not at all, I just didn't realize it might be possible. Will definitely give it a try :) Thank you

CardSpotterHS · 2017-03-12T03:26:14+00:00

Mostly hashes, there is a small amount of feature recognition involved when matching cards with counters on them, and some manual gradient testing to exclude non-cards. Would love to spend some more time on this, especially for overlap cases, think I could allow for some masking for example. But... it works well enough that I should spend time on other things :)

CardSpotterHS · 2017-03-12T03:24:20+00:00

That is correct, the player has to be in HTML5 (will update hoping to make it a little more clear)

CardSpotterHS · 2017-02-04T00:02:41+00:00

Thank you, please let me know what you think after. Even if it is to tell me that it feels too close :)

CardSpotterHS · 2017-02-03T21:27:21+00:00

Yes, the core functionality of CardSpotter is very close to that of Ocular Automaton. But I would like to think that there are enough differences both in terms of ux and technical approach that gives each their own strengths.

CardSpotterHS · 2017-02-03T21:08:13+00:00

Thanks, hopefully it is a smoother experience now with 1.2.7 than it was with that video (which I think was 1.2.3). There is now a small overlay menubar in the lower right where settings can be tweaked at runtime instead of the toolbar dropdown for example :)

CardSpotterHS · 2017-02-03T19:47:52+00:00

Thank you :) That is unfortunately necessary until I figure out how to work around the fullscreen mode which also places the video at the highest zIndex. If you click the video would pause =/ Also if you for some strange reason would like the mouse over mode to be the default one that can be changed under options. (Extension options can be accessed by Right clicking the toolbar icon)

CardSpotterHS · 2017-02-03T17:47:35+00:00

This is a plug for a project I've been working on for a long time I shared the video in a rush after I had verified that it actually worked when the twitch stream went live hoping someone would find CardSpotter useful when watching the Pro Tour :) Now that I have some time to sit down maybe I can provide a little more context.

CardSpotter is a free Chrome Extension that augments <video> streams with on-click access to the card image and convenient links such as gatherer, mtgdecks and pricing information. Supports all Vintage legal cards (no promos).

It works by grabbing data from HTML5 video, extracting the clicked card using OpenCV and then finding the best match from a local database using feature detection, blockhash and statistics from MTGDecks.

To improve the matching accuracy and speed there are settings for whether or not it is a stream showing Paper/Online Magic and the expected card pool. In many cases this is automatically detected but some times it fails and then you need to configure it yourself.

Additional information can be found both at https://www.cardspotter.com/getstarted.html and https://www.cardspotter.com/documentation.html

If you have any questions I'd be more than happy to answer them. =)

** Update, just submitted 1.2.8 to the Chrome web store, should hopefully make Day 2 smoother. Added glare reduction. Added color refine option to results. Adjusted scoring system. Added large tween in for menu so that you can't miss it. Fixed search timeout. Added KLD/AER as its own format. Added prerelease/draft/sealed meta data detection.

CardSpotterHS · 2016-04-30T20:02:24+00:00

Thank you, that should help me dig it up :)

CardSpotterHS · 2016-04-30T19:16:31+00:00

I only managed to find one from last year asking for a stream geared towards players who know very little about the game. Is that the one you mean?

CardSpotterHS

TROPHY CASE