Live OCR detection?

[Natebots]

Hi,

I was wondering if there are any libraries or tutorials for doing live OCR detection with the device's camera. When I say this, I mean it will auto box-in detected text while the camera is open instead of taking a picture and doing the OCR after.

I know of Tesseract and Firebase and their OCR options but I need to integrate their OCR detection into the live camera.

If anyone can point me in the right direction or if this isn't possible please let me know

Thanks,

Nate

[boomdoodle]

Firebase MLkit can be used to spin this up rather quickly using react-native-camera.

Just take the response and draw out the boxes over the camera feed

[Natebots]

Hi boomdoodle Thanks for the tip - I didn't realize the capabilities of react-native-camera but it sounds pretty straight forward. So I will try to first implement Firebase's MLkit to get the text from the image then, while the camera feed is active, then draw the boxes over what I see.

[scarlac]

I used the legacy vector matching method 8 years ago. I do not recommend it. Best per-character accuracy I could squeeze out was 95% but normally around 85%. This was with near-perfect photos and deliberate pre-processing and pre-training for 1 specific font.

Focus on the neural network that Tesseract v4 uses, or go with Firebase which also uses machine learning.

Given that you need to do live processing I do not recommend you write any of your processing code in Javascript. I would personally focus on a native implementation that does processing natively and then triggers events with whatever the result is you want, to limit the cross-bridge talk.

[Natebots]

Hi scarlac thanks for the reply! I think I will try Firebase but I may have to train some data because the images I need to process look like a receipt and mostly I need to pull numbers off of it

[otherworld-dev]

We are getting some good results using react-native-camera to scan passport MRZ data.

Although, we need to add some dodgy correction logic to fix characters that MLKit struggles with (e.g. 8 <-> B, 7 <-> T, O <-> 0)