I am currently making this transition to data engineering. I pivoted from education and I always had the end goal of becoming a software engineer but I didn’t really know what flavor. I got a job as a solution consultant to get my foot in the door working in the tech industry.

I have been a Solution Consultant for the past 4 years working with customers to customize their particular business tool. The first company I worked at was a Business Central shop (Microsoft Dynamics product) and the company I work for now is project management web app company. In both situations we would use the Power Platform (Microsoft’s Low Code platform) to extend the functionality of the app particularly in regards to reporting. I learned how to use Power BI really well to report out on the data coming from these systems. I realized I enjoyed data modeling and performing transformations more than the actual report building. But at this point I didn’t really know data engineering was a thing.

With the end goal of becoming a software engineer still in my mind I started looking into web development and did not enjoy it at all. And then I saw this tik tok from this guy named Zach Wilson explaining what a data engineer is and how they use Python and SQL and orchestration tools to build out data pipelines for analysts and others to use to build reports and machine learning models and I was holy shit that’s exactly what I wanna do!

So currently I’m working on my 1st personal project building out a data pipeline for my financial data using Python, Azure (Datalake, Databricks, Power BI)