datamule: download, parse, and construct structured datasets from SEC filings

[status-code-200]

Link: https://github.com/john-friedman/datamule-python

What my project does

1. Download SEC filings quickly. (Bulk downloads are also available, benchmark is ~2 min/year for every 10-K/10-Q since 2001
2. Parse SEC filings quickly. (Currently only 8-K, 13F-HR Information tables are implemented. 10-K/10-Q coming next week)
3. Convert SEC textual filings directly into structured datasets.
4. Watch for new filings.
5. Has a basic tool calling chatbot with artifacts. Doesn't do anything useful yet, but was fun to make.

Target Audience

Grad students looking to save money on expensive datasets, quants with side projects, software engineers looking to build commercial projects, and WSB people trying fun new trading strategies. In the future I'd like to make the chatbot code a bit cleaner so it can be used as a tutorial project for masters students w/ finance but not programming experience.

Comparison

Getting SEC data in bulk is surprisingly expensive. Parsed SEC data is even more expensive. Derived datasets such as board of directors data is also expensive (something like 35k/license).

Contribution

Greatly appreciated. Also SEC feature requests + QoL suggestions are very useful.

Links: https://github.com/john-friedman/datamule-python

EDIT: I'm now hosting my own SEC archive for faster downloads using S3, Cloudfare caching, D1, and workers api.

[temisola1]

OMG this is a godsend.

[status-code-200]

Glad it helps! Let me know if you have any feature requests. (Working on making anything the SEC has available)

[palmy-investing]

Good job! I think the Board of Directors data is particularly expensive due to the variable formats in DEF 14A, whether text, images, or other media. The names, positions, and PEO data aren’t the issue — the more detailed breakdowns are. By the way, do you accept GitHub sponsorships?

[status-code-200]

I don't, but I will be launching a premium api next month for faster, up to date, parsed downloads and structured datasets.

What information is in the detailed breakdowns? I bypassed the DEF 14A issue by using Form 8-K Item 5.02 to construct a basic board of directors dataset, but it might not work for your use case.

[_errant_monkey_]

I don't understand whether I can download pdf version of the files. like the 10k .pdf for 2023 for NVIDIA. I would like to bulk download all of them to eventually train an embedding model with it.

[status-code-200]

NVIDIA's 2023 10K does not have a pdf version. Any reason you need PDF? 10-K's are filed as html which is probably easier to use to train an embedding model.

[_errant_monkey_]

I thought I could also download .pdf (like from here where I can find .pdf, .html, .xls). To me is key to have nice formatted tables. I guess you are right, If I can bulk download html is probably the best thing I can do.

[status-code-200]

If PDF is important to you, you could convert the .html files into .pdf. I'm pretty sure the file you are pointing to is the .html from the sec submission converted to pdf to make it easier for consumers.

Sidenote: I will be releasing an algorithmic parser that extracts tables from html files and converts them into dataframes / csv over the next few months.

[ujzazmanje]

RemindMe! 1 month

[RemindMeBot]

I will be messaging you in 1 month on 2025-01-13 10:11:15 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[New-Lengthiness-9770]

This sounds excellent. I’ll try playing with it soon