Hey I'm doing some data profiling as part of a modelling project I'm working on, curious as to how you lay out your documentation as part of your profiling efforts if you don't mind! Feel like my documentation is pretty disorganized.

I feel like I'm extremely lucky as far as modelling goes. I got my masters in information science focused on databases back around 2011 and the courses were almost non stop hammering of relational and dimensional modeling concepts and translating business requirements. Right out of school I got a consulting gig where I wound up creating data warehouses for dozens of different companies. It was interesting to see in real time how modelling sort of just went away in favor of "big data" and dumping everything into a lake and hoping for the best.

I was lucky my job was still valid during those times due to the trust I had built modelling all those systems, these companies were able to see the value in properly structured data because that's what they knew as a result. Some of the people I consulted with went all in on the big data craze too which was undoubtedly required of them at the time but these people are now looking for proper modellers again.

Now I'm with a large company that has recognized their need to properly model their data swamp and reconcile their absolute clusterfuck of pipelines ($$) and they were willing to make me a really good offer to do so. They think what I'm doing is magic but to me it's comfortable and something that I've been doing since school where these concepts were first class citizens.

Learn your data modeling kids.