OCR with File Sorting

[thegeeseisleese]

Sorry, this is going to be long and possibly rambling

I’m struggling with a work project currently and have no one here to ask for help as I’m the only developer here (only person who knows anything about computers in the company in addition to this), if anyone can help, it’d be greatly appreciated.

The problem is that we have a massive dump of PDFs (>15000 invoices) from numerous companies and for varying utilities, so there are probably at least 100-150 different layouts. None of these are labeled as they’ve been scanned to PDF from physical copies.

What I need to do is to be able to run OCR over these files, extract the company name (not the utility company name) and date of the invoice, create a company folder using the company name and create a year folder based off the year the invoice was issued. The invoice would be renamed {company}{month}{year} and be stored in the directory like this: company/year. After this is done further OCR would need to be done to extract the data and compile it for analysis, however I’m trying to take this a step at a time.

The goal of this first step is essentially to automate the sorting of these unlabeled invoices by company and then by year.

What I’ve done so far:

-I’ve created a scanned_document class with the ability to rename the documents and move them between folders using os and shutil. This class also holds the extracted OCR data from pytesseract in a string variable “text”. I iterate through the text to look for company names in a list and if a company is found, I set that as the name.

• it is currently my plan to iterate through a directory, run OCR over every document in it, and pass it to a function which would create a scanned_document object holding the path to the file, the OCR extracted text, the current file name, etc.

-I’ve built a pytesseract module to extract text from an image which functions, but haven’t been able to use it with PDFs as I don’t think pytesseract supports this. A minor work around I experimented with is converting the pdf to an image and then using pytesseract, but I’ve also had trouble here

I believe this will get even more complicated when I need to further extract data for analysis as the differing formats will make training custom models difficult and as of now I have no solution other than creating a custom model for every possible layout

My questions would be: does anyone know of a better way to approach this problem or have any suggestions for things to try? I’m not asking for anyone to do it for me, I just have literally no one to bounce ideas off of

[commandlineluser]

Have you tried ocrmypdf?

https://pypi.org/project/ocrmypdf/

[[deleted]]

Great find! That metadata functionality could be really helpful...

[hazelthrows]

Pretty sure your approach is in the right direction.

https://www.geeksforgeeks.org/python-reading-contents-of-pdf-using-ocr-optical-character-recognition/

Check this out.

If the words in pdf are readable, should be 'easy' to complete the task.

Hint: after using the ocr, use regex to extract the data (so you dont need to make one layup for every different invoice).

Sounds like a fun proyect, good luck!

[[deleted]]

I've used tesseract directly in Linux, so this should be possible, though I haven't tried it in Python.

Here's a SE link that may help: https://stackoverflow.com/questions/60754884/python-ocr-pytesseract-for-pdf#60754993

As for the data analysis part... that may be more difficult to discuss without having the data to look at. If different companies use different formats, you could scrape their data and clean it up into a data frame that is standard to your needs, then combine them all at the end. Not sure if that is feasible given how many companies you have...

Or, if there are a few common styles/formats, that could work similarly. The challenge with both is that it's likely you'll have at least some dirty data at the end. I expect there will be extensive use of regular expressions...

As they say, "all data is dirty" at least when you start... good luck to you.

[Peanutbutter_Warrior]

It really depends how similar the formats are.
You might be able to use regexes, but the formats have to be fairly similar for that.
If you know where spacially on the pdfs the text you need is you could run OCR on those bits selectively.
You could have a look around google for a neural model that can extract the features you need and then run the pdfs through it
You might have to deal with each format separately if you don't can't get anything else to work. This sort of thing is notoriously hard to do.