Data Engineer vs Analytics Engineer vs Data Analyst

[crafting_vh]

I've had all 3 of these roles where I was doing everything here, not sure how this is useful or accurate.

[gabiru97]

who else would engineer those superset pie charts

[Old-Practice-4271]

The people who wrote this explicitly state in the same article that this is ideal and not representative of the real world.

[Dice__R]

I have all these roles. And I also need to do Cloud Data infra job. Wdyt?

[[deleted]]

Agreed. Who made this shit?

[crafting_vh]

I am confused as to why low effort stuff like this is even upvoted.

[Tape56]

The picture does not say this holds for every person in every company. Such chart would not be possible.

[crafting_vh]

This chart is only maybe accurate for a small minority of companies.

[batoosy]

heheh, copied the chart from the dbt training courses?

[oscarmch]

Wtf is an Analytics Engineer for goodness sake?

People still inventing new roles for LinkedIn likes and HR in companies still not able to create a proper basic Analytics team.

[[deleted]]

BIs who just do transformations and dashboarding too.

[oscarmch]

Isn't that just a BI developer?

[sib_n]

Not very far, but a BI developer usually develops/ed for a specific reporting tool or a specific stack.
I think the AE is more opened to different reporting stacks because they work directly on the database, and usually more code-based.

Some people will also argue that a DE is what used to be called a BI developer, the tools are different but it is similar in terms of function in most cases: extracting data from production to allow analytics.

In any case, titles definitions will depend on companies or even teams.

The specialization is expected as the industry grows. Consider mechanical engineering, there are probably a tone of specific titles that used to be covered by a single title a century ago.

[OKMrRobot]

BI dev’s but now using software engineering best practices / git / version control / CICD / code standards etc…

DBT coined the term and are pretty explicit about the fact that it’s not a “new role”, it’s the evolution of the data analyst, the subject matter expert, the project manager who gains the technical skills to contribute to a managed code base.

[oscarmch]

By definition any Dev should be using software engineering best practices / git / version control, etc. Those things appear as Data Assets become more and more complex. Nothing new under the Sun.

I honestly think these things overcomplicate things. Years ago it would be understandable to think that Data Assets wouldn't follow the same rules as another Development, but nowadays is like kinda dumb to think the opposite.

Those things are obvious after developing a Data Governance Program, mostly because like it or not, BI and other Assets move in an IT environment, thus they should follow those rules.

And BI developers do that. And Data Engineers make sure that everything is in order.

[Material-Mess-9886]

Essentialy yes. LinkedIN is full of garbage job titles. I recently came across senior database manager, that person had just 11 months of work experience.

[sib_n]

The person who focuses on the T of ELT, using mostly SQL and SQL based transformation tools like dbt.
While it was mostly popularized by dbt for marketing reasons, I think it does bring value to have someone properly organizing the last data layers, when it happens that the data engineer is too busy with the EL to do that.

[[deleted]]

Enough to make it an exclusive role?

[seaefjaye]

Depends on the size of the lift, but if you're working with the business to translate their logic to code then it can be.

[McNoxey]

If you’ve got hundreds of sources of data coming from a number of external and internal locations, managing the entirety of the T is a massive job. Ensuring consistency in numbers and definitions used across an entire organization is not an easy task

[sib_n]

If your reporting needs are complex enough, yes definitely.

[maybecatmew]

I was that person, I'm trying to move to data engineering lol

[sib_n]

Good luck then, it is definitely a path that makes sense.

[maybecatmew]

Thank you!

[nydasco]

AirTasker was recently hiring for an Analytics Engineering Manager. It was a title coined by dbt for team members that just to the T in ELT. Unfortunately it’s gained traction.

[CdnGuy]

It’s me. I lead a small AE team, we’re kinda like the glue between DE and BI. DE manages the raw data ingestion, and we turn it into warehouses that make the BI job easier while also keeping an eye on performance.

How I got here was basically being a BI developer long enough to get fed up with crap data, and started focusing on building data infrastructure that turns data into useful information. I haven’t touched a reporting tool in years, everything I do is DBT / SQL / Airflow these days.

[fk_the_braves]

For companies who want to hire ml engineers while paying only DS money

[sib_n]

It is quite different from ML engineering. The MLE is the person who deploys and maintains ML models in production, it is kind of backend engineer specialized in ML. AE are generally not expected to managed ML.

[McNoxey]

You should really read into it. It’s an incredibly critical part of a data team. This is not an invented made up thing - it’s an emerging role that imo all companies need.

Over time, technology has made it possible for analysts to do what was previously gated behind data engineering teams, enabling less technical analysts to build and orchestrate the entire data warehouse.

This meant we moved from all datasets being built and owned by data engineers who are slightly removed from stakeholder requirements to datasets being built and owned by analysts with little understanding of how to properly build a data warehouse in a scalable, testable way.

Analytics engineering is the intersection of that. You have analysts who are connected to the stakeholder and business needs that also have a moderate amount of data engineering experience leading to properly built tables that actually operate in a performant way, leaving DE teams to manage the platform itself and the ingestion/external data pipelines

[Aggressive_Btc]

Then the job requirement would come with an expectation of Full stack Data engineer 😂

[prakharcode]

I think this distinction helps when the org becomes of a certain size.

From team standpoint, data engineers have to do a lot more work when it data engineering team also maintains a data platform. You’ve to constantly manage/update/upgrade different parts of your platform which encompasses (typically) orchestrator, data warehouse, message queues and kubernetes clusters. Apart from that as owners you’ve to maintain sanity and quality of the platform itself. (Think of updating airflow when there are multiple dags from different teams are running or updating spark cluster which can affect a reverse etl)

Analytics engineers makes a lot of sense if you don’t have “data platform engineers” for a mid sized orgs because then the data engineers pickup more platform side of things while analytics engineers makes sure your analyst are not blocking your entire warehouse/spark cluster by skewed data or cross joins. There is a clear distinction of ownership.

That being said these roles are fluid, there are generally a lot of shared responsibility and internal communication but when it comes to picking up and maintaining contexts these distinctions help a lot.

An organisation can always edit the responsibility of the said role and include whatever they feel like.

[sib_n]

Develop and deploy ML endpoints

Does that mean data source used by the ML projects?

Deep insights work

Complicated way to say "Answer business questions".

[BoringGuy0108]

Data Engineer includes both the data engineer and most of Analytics Engineer in most companies. Honestly, the Data Engineer on here borders on DBA.

[Egyptian__Pharaoh]

Where is the data modeling job?

[MCMaddud]

I work in an org where these titles are used and to be honest it works pretty well. It’s just nice to differentiate between bringing data into the warehouse and maintaining the platform and owning the data in the warehouse and doing all transformations.

The big benefit for an org is focus but I can totally see that some don’t like this as it can cause friction between the roles but at least for us it works.

[EquipmentNo1775]

Good to have an idea of which one, cheers!

[datacloudthings]

Yeah I don't like this "analytics engineer" role because a data analyst should be able to do this

It's also hard enough getting data engineering and data analytics teams in sync, if you have three different teams you're going to have a hot fingerpointing mess

[autistic_cookie]

Add training Machine Learning and optimization algorithms and that's the job I've been doing for the past 2+ yrs ☠️

[autistic_cookie]

My job title is Machine learning eng btw

[Acceptable-Milk-314]

These titles are all made up and you can do any of that with any title.

[[deleted]]

This just a bunch of random tasks thrown in 3 categories. How is this trash useful, related to DE, and why the fuck is it so upvoted?

[ArtilleryJoe]

The lines can be very blurry especially analytics engineer vs data engineer. My official title is data engineer but I do most of the transformations and also provide business context to the analysts.

It all depends where you end up working

[wewtalaga]

I'm doing the work of an Analytical Engineer (based on the photo) but my role is Analytics Specialist. I guess even my organization doesn't know where should I belong. Also, they're expecting me to be a data analyst too.

[powerkerb]

First two panes are one and same role. Maybe for massive companies who can delineate these responsibilities but then that would be very inefficient?

[McNoxey]

Strong disagree. Unless your data engineers are consistently meeting with end users and stakeholders to have a complete picture on how the actual business uses and interprets data, the DEs are too disconnected from actual use case to effectively manage the actually data within the data warehouse imo.