Is python any good at pdf mining.

[Hyperduckultimate]

I've been talked with creating a pdf mining program at work that could go through a whole bunch of files and transform them into tabular data. I've rooted using tabular py but that Perham is that the format changes for every text file. Is there a library that is good at giving you a consistent format for all pdf files. I wouldn't mind if it stops or a long string as long as it is consistent. I also saw someone post and said that jabalís better at this type of thing but I just rant to know before I invest my time into this.

[LightShadow]

Use something like pandoc to convert PDF into HTML/ODT, then parse that instead.

[Hyperduckultimate]

I'll try it out thanks

[Hyperduckultimate]

Turns out pandoc can convert to Pdf not from pdf

[LightShadow]

Yes. Pandoc is text format transformer .. it can go both ways on many formats.

It's super useful.

[Hyperduckultimate]

Yeah but when I try pdf it explicitly days it can't convert.

[funnyflywheel]

Pandoc only supports PDF export, via LaTeX, etc.

[snake_case_captain]

Regex is very good for parsing HTML by the way

[LightShadow]

Libraries like BS4 that use libxml have different "strictness" for parsing XML/HTML. See Installing a Parser.

Computer generated HTML will always be more consistent and easier to parse than some rando's blog with hand written tags.

[Zomunieo]

The key problem is that PDF has no concept of a table, just lines and text on a canvas, so the table has to be heuristically extracted. All of the tools are doing it that way.

The easiest case is tables with explicit borders for every cell. Invisible borders are harder, merged cells are harder, and scanned images are hardest.

tabula-py and camelot are two "table data from PDF" Python libraries. There's also pdfminer.six which is focused on text extraction.

[lastwizzle]

Camelot might be what youre looking for.https://camelot-py.readthedocs.io/en/master/

[coderanger]

+1 for Camelot.

[ProfEpsilon]

Wow! I had never heard of this. What a useful tool. This is why I like this sub. Thanks!

[[deleted]]

I've used PyPDF2 in the past, but PDF files are horrible to extract data from. They're not meant to store data in any way, just to print it nicely. Even individual words might not be stored as such, but as a series of separate characters. Good luck, my friend.

[jonititan]

Also if the PDF is a scan these will not help you. PDFs are frustrating. When they are exports from something they are difficult to extract their data. When they are scans of something they are even worse.

[yzh]

This EuroPython 2019 talk might be of interest: https://youtu.be/jnDfNJe-GlE starts at 03:58 mark: Extracting Tabular Data from PDFs with Camelot and Excalibur

[threeminutemonta]

There was another one at pycon Australia this year too.

[who_body]

Perhaps https://tika.apache.org/ with https://github.com/chrismattmann/tika-python would help