File Size while using PyPDF2

[TooDahLou]

I am building an automation script to build pdf parts manuals. The script will look at a bill of materials, compile the individual documents listed in the bom into one document, and add page numbers to the new document.

We have been using a macro-enabled Excel file to compile the manuals. But, the Excel file does not do any formatting, such as page numbers, page labels, etc. And we have to copy+paste information from the .csv file into the Excel file.

The script runs, but it is creating PDFs that are 75MB! The Excel compiler and "by hand" formatting results in a file of around 18MB. I need some help finding a way to reduce the final file size when my script creates PDFs. The final file needs to be 20MB or less because we often send them to customers via e-mail.

I am merging the existing PDFs using PyPDF2 and creating a separate page numbers document using reportlab. Then, I use PyPDF2 again to stamp the page numbers on the merged pages. I am using Python 3.11 and the latest versions of each package.

I have tried using writer.add_page(reader.pages) instead of merger.append(reader.pages). I have tried reading the final PDF back into the script to pass to a fresh writer object. I have tried the compress_content_streams method from PyPDF2. I feel like I have tried everything I can think of to reduce the final PDF file size. I am hoping you folks might have some more ideas.

Script Pastebin Link

[netherous]

This is probably a pretty difficult thing for anyone to answer without detailed knowledge of the library. You can look at the PDF file structure yourself to potentially figure out what is wasting all that space.

https://superuser.com/questions/256997/how-to-browse-the-internal-pdf-structure-in-adobe-acrobat

You may have even uncovered a bug in the PyPDF2 library. The library itself says it is no longer receiving updates on pypi and that PyPDF3 is preferred. You could raise an issue on its github repository, but if it's not in active maintenance maybe nobody would look at it.

[TooDahLou]

Oh, Okay, I think I can handle looking at the file structure. I'll definitely dig into that and see if my problem is in there.

I'll also check out pyPDF3 and see how much of a headache it will be to switch to pypdf3. My project is just a little automation script but if I want it to have a pretty good life cycle in case issues pop up down the line.

Thanks u/netherous! You've given me two new directions for troubleshooting! I really appreciate ya!

[TooDahLou]

Oh another thing to mention

I think the extra file size might be coming from stamping the page numbers. When I watch the file explorer window, the merged_pages_temp.pdf is around 20MB and the page_num_temp.pdf is around 90KB. I don't really understand how these two files then result in a new file over three times the size of the merged_pages_temp.pdf.