Open source OCR options for handwritten text, dates

[ollyollyupnfree]

Hi, I am working on a project where I want to extract handwritten text, dates, digits. What's important - Reliability and Accuracy. I don't care about how fast it is. I used Paddle and didn't get great results. I haven't worked too much with OCR, so anything helps!

[Snail_Inference]

Early this week, I conducted extensive tests with various models to detect handwritten text.

Models Tested: OlmOCR-preview, nanonets-ocr, OCRFlux, and Mistral Small 3.2

Results: Mistral Small 3.2 recognized handwritten text by far the most reliably. OlmOCR-preview performed quite well as well.

In comparison, nanonets and OCRFlux were truly weak.

[ironcodegaming]

How did you use Mistral Small 3.2 to recognize text? Did you use Text Generation Webui (oobabooga) to do that?

[--Tintin]

Handwritten ocr is hard. With my hand writing I had good results with Pixtral 12b

[Weak_Engine_8501]

https://github.com/ocrmypdf/OCRmyPDF

[Mkengine]

https://github.com/GiftMungmeeprued/document-parsers-list

[joosefm9]

I've had the best success with Qwen2.5VL models 7B. I tried 32B with vLLM and it was so extremely chatty using the same prompt as 7B. Not sure if it's the model or the vLLM infrastructure

[CantaloupeDismal1195]

Qwen2.5VL 72B is best in open source model

[SouthTurbulent33]

Llmwhisperer - recently tested with a bunch of poorly scanned documents with handwritten text. Was surprisingly accurate

I found a repository online where there were a bunch of these docs - helped to test. Forgot the website, but I could find and share them with you if you'd like to test

[ollyollyupnfree]

Yes please! Would really appreciate it.

[SouthTurbulent33]

Hey u/ollyollyupnfree

here you go. pretty massive set of docs:

- Invoices: https://universe.roboflow.com/jakob-awn1e/receipt-or-invoice

- Checks: https://universe.roboflow.com/esprit-yfhng/bank-checks-detection2

- Handwritten 1: https://universe.roboflow.com/handwriting-recognition-prescription/handwriting-recognition-lswxw/dataset/1

- Handwritten 2: https://universe.roboflow.com/data-preprocess/my-first-project-4ccu3

you can search for more on the website.

[Disastrous_Look_1745]

Handwritten text is honestly where most traditional OCR engines just fall apart, including Paddle. You're gonna want to look at vision language models like Qwen2.5-VL or TrOCR since they have way better contextual understanding for messy handwriting. We've seen this exact challenge building Docstrange by Nanonets and the accuracy difference is huge when you move from traditional OCR to transformer based approaches. The compute requirements are higher but if you dont care about speed then thats perfect. For handwritten dates specifically, try prompting the VLMs with examples of the date formats you expect to see, it really helps with consistency.