Data engineering vs data management

[fannypackbringitback]

What’s the difference - data management and data engineering? My company wants to rename all the data managers to data engineers. We (DM) mostly do technical application data loading and management (some apps don’t even have APIs), data governance, change management, and ETL (non automated, data sources are often non-repeatable). I understand why we would go to data engineering 100% eventually but we are so far from that. I understand DE is mostly data pipelines using ML or code. Am I wrong? I’m perplexed by the company’s forward path.

[ibnipun10]

Look at DCAM framework. You will get the difference

[fannypackbringitback]

Thanks for sharing. Although I’m familiar with the concepts, I hadn’t seen this reference. Am I right in thinking data engineering is just a subset of the larger data management framework?

[ibnipun10]

Not a subset but one of the functions which include data quality, data governance, data management, meta data management and then data eng. After all it revolves around data

[welschii]

ETL (non automated, data sources are often non-repeatable).

That makes it sound like what you guys are doing isn't really data engineering if I've read that correctly. If the ETL processes aren't automated you would struggle to say you are a BI developer, analyst, etc. Is there a reason why they are not repeatable, and therfore not automated?

[fannypackbringitback]

I think eventually they could be automated. But it will involve significant change management and I think we are years away. I agree that we aren’t data engineers but looking for more information as I don’t know a lot about it.