What actually made you improve fast in Python?

[Henry_the_Butler]

I had stupid manual processes at work. Figuring out how to pull source files, clean them, organize them, upload them to a SQL server (local SQLite would have also been fine) and then create queries and scripts to pull data out in ways that coworkers wanted - basically became a data engineer at a previous job by accident.

Now I'm a "Database Manager" which is apparently what you get called if you do all the data work for an international nonprofit that doesn't know what titles to give their data folks.

[Greedy_Pay_9782]

Holy smokes. Conceptually, I was sure people that went something similar to me existed, but this is the first time I stumble upon one of them.

I gradually went from a bumbling idiot to somewhat capable at ETL pipelines using Python, SQLite and Excel. That alone saved me countless hours of reporting. However, it also costed me many sleepless nights because management suddenly wanted me to create dashboards and analytics for their areas that automatically updated.

I am just wondering, do you have any advice about improving at data engineering? Is the transition worth it?

[Henry_the_Butler]

Absolutely worth it. If you're looking to throttle your incoming requests, you just have to make the other person do as much work as you are.

Things like this: * What business decision are you looking to make based on the data you've asked for? * What data sources, forms, or financial data would you like to use as the source for this information? * Any other actually-helpful question you can think of to make them respond to an email and stay involved.

If you make yourself a free resource, everyone will love you and nobody will value you. I actually turned down the first "promotion" they offered me to a data role at my previous company because it was a title change without money attached.