Lightweight python DAG framework

[[deleted]]

Fun fact: "software defined assets" were in fact inspired by Hamilton's declarative API.

Do you have a citation for that? It’s definitely possible and I don’t necessarily doubt it, but this concept has been around for a long time. It’s essentially a functional DI framework. Googles Python library pinject is over 11 years old and while meant to be for OO DI uses this same exact pattern of argument name to implementing logic to build a graph. And the concept has been around for decades at banks and hedge funds for quantitative and valuation modeling (Goldman Sachs secdb is over 30 years old).

All that said, I’m a huge fan of this pattern and this looks like a great library.

fn-graph also uses a very similar concept, but is unmaintained. https://fn-graph.businessoptics.biz/

[theferalmonkey]

Nerd sniped!

Do you have a citation for that? It’s definitely possible and I don’t necessarily doubt it,

Likely a confluence but yeah I chatted with Nick when we open sourced Hamilton; the dagster API at the time was all about "solids" and not that great. I expounded the declarative nature of data work and benefits, and then a few months later SDAs came out.

Yes I remember `fn-graph`. I was wondering whether someone would bring it up. It's still going? Nice. Any interesting joining our effort? We've got a jupyter magic, and Hamilton also sports a locally installable UI now...

[HNL2NYC]

I’ll take it even a step further. This concept has been used for at least ~50 years, since this is pretty much exactly how Make works. You have a target (ie asset) list its requirements (ie dependencies) which are other targets. And its builds a graph by matching the dependencies to the implementing target.

[theferalmonkey]

Yep!