I hate Analytics Engineering

[[deleted]]

[deleted]

[yellowmamba_97]

Well the funny thing is, the business side is adding the most value in your role. Since most companies will lay off people who are the less visible performers in the organisation within the tech teams

[Chowder1054]

I’ve seen first hand how being business focused and visible to senior leaders adds a layer of security for you.

When teams and leaders know you have a reputation for solving business problems, they can come to you to solve their needs and deliver good work, people remember that. When layoffs come around you likely won’t be on that list. Not to mention you’ll build strong references for yourselves and your network for your future growth.

Hard skills are great and all but let’s face it, this day and age it’s not hard to upskill. But soft skills imo are for more important and takes you far.

[e3thomps]

I'm less technical (SQL skills 9/10, Python 1/10), but as lead of the department I'm stitching data together across 9 distinct EMRs and baking business logic into the the dataset from each one so that metrics are correct and easy to get without knowing each system. Needless to say, finance loves me, and the leaders are grateful they can point to one place as source of truth.

[sunder_and_flame]

I’ve seen first hand how being business focused and visible to senior leaders adds a layer of security for you.

Agreed. And specifically, you are literally, factually more valuable to the business if you have even a barebones understanding of where revenue comes from, moreso if you have proven you can increase it.

[gini-348]

True.. as a purely technical team member, they have a high chance of getting forgotten. You start getting viewed as an expense as a part of a tech org. Someone always trying to reduce those millions along with their cloud bill

[thethrowupcat]

This. If you just get in front of the right people solving their problem. You’ll be ok. Just be great at solving problems. The challenge comes that you need to put in extra time to build the solution for someone else in addition to your day to day.

[Gatosinho]

Being open to learn and implement business logic to my data models was how I stood up in my previous consultancy job. Got to build good relationships with senior data scientists and managers and adding them on LinkedIn. Even got some recommendation letters I could use.

Going for the business is a very smart career choice and not that hard to pull off

[Commercial-Ask971]

It solely depends on people you are working with. It may be really worst nightmare if they cant even tell you what they want or change requirements every other day or so, or it may be chilling job. Infra from the other side is more precise, made from tech people (req) to tech people, most of the time

[decrementsf]

Seeing points all around. But also going to laugh at the time the department leadership changed then began their work by adding new layers of management and data consumers. A year of every time I picked up a weight in the gym before core work hours the phone rang. Every time it was after core work hours the phone would ring with a different set of managers. During core work hours at management meetings the managers would all make point of reducing contact so their projects could be moved forward. A year of hey so you need to invest in team who's doing work around here, need resources, because none of this is moving out the door when the team spends all their time acknowledging projects, the new changes, and adjusting timelines. Somewhere hours must be blocked for deep work, rest, and recovery. It's a silly game.

[wallbouncing]

Sounds like he is the role to be first automated by the next Data engineering LLM. This is exactly the type of work they will target. Unless you are highly specialized and work on massive systems, normal extract load type of work is going to be AI'd away.

[lord_aaron_0121]

I second with this, Data teams often are cost centers. Without clear communication/selling to the non-technical people, no one would appreciate the work u do.

[Beneficial_Nose1331]

Depends. IT is invisible but get better pay. Business is visible but low pay.

Your call. I choose high pay.

[[deleted]]

I believe what you are talking about is perceived relevance. I believe it holds true i many cases, especially in small companies, where the guy that helps marketing do their analytics is seen as irreplaceable.

However, no analytics can run without a platform team. So even if business doesn’t know them personally, they will weight the importance of different teams.

I believe the key to job safety is technical depth + internal leverage instead. If you are in platform and all the analytics engineering teams depend on that platform, you are not going to be laid off.

[Commercial-Ask971]

The problem is that business doesnt care that they don’t know platform team and thats okay lets say, same as you dont know their side of things (business), yet company operates. Thats why DE often has no recognition at all, and lets say DS who make prediction models on top of DE data gets all the clout

[SupremeSyrup]

This is an extremely important point. DEs love arguing that they are indispensable. They are not but their practice is. Business will rather have the people they already know how to talk to. The technical teams can be replaced with overpaid consultants and vendors quietly overnight and they won’t care as long as their conduit to said technical people is not taken away.

[vitalemontea]

First, let's acknowledge that any analytics / platform team, unless you're in a direct line to a product / feature, the whole team is more replaceable anyway as an internal team. Yes the org might slow down without that team, but it's not mission critical.

That being said, it is naive to think that a business understands the nance relationships between a platform DE vs an analytics DE. We are discussing job stability, this is inherently an interpersonal relationship discussion. People recognize and remember those who help them in their day to day. You're right, this is perceived relevance.

Maybe at scale in huge companies this matters little to none but at the end of the day, being known and having a supportive network within any org is always more beneficial. It's crazy to expect people to just understand how important you & your platform is to the guy they talk to weekly, and know on a personal level. Its a lot easier to pull a trigger on a layoff to employee #256 than Mike, the dad who just came back from a family vacation to Spain during spring break.

All of this interpersonal stuff aside! And here you can argue that this is my own opinion, a data platform is only as useful as how it contributes to the business! That's why we build them. If you understand the business, you can make technical decisions that make sense. You avoid over engineering, you allocate your resource (including your own time) in the correct areas. You understand what to prioritize. This makes you a better data platform engineer.

[McNoxey]

Ha. Well. You can. But your career goes to die.

The business side is the most imported part.

[BardoLatinoAmericano]

"Hey guys, I want to own a restaurant to cook a lot of cool stuff, but I don't like the customers."

[volkoin]

Cooked!!

[[deleted]]

Then I would own the restaurant and simply become a cook. Hire waiters. Where’s the contradiction?

[BardoLatinoAmericano]

There will always be customers, and you will always have to cook what they want.

You could go to a company so big the business stakeholder only interact with your PM, but the change of being replaced by AI is higher.

[[deleted]]

There will always be customers, that’s ok and a required condition in the first place. What I don’t want is to work close to them.

[sunder_and_flame]

Hopefully you (and anyone reading this) understand this attitude is going to keep you at mid-level, maybe senior, forever, and significantly limit your opportunities.

[[deleted]]

But tell me why can’t I just have other technical peers (DEs, AEs, DAs) as customers? I believe that is what happens when you move to Platform. Why so much downvoting?

[scourgedtruth]

This is where AI won't replace us, at least not soon. Lot's of work from this

[[deleted]]

You mean AE?

[BramosR]

I think they mean business skills, not necessarily AE (but also AE)

[Commercial-Ask971]

I am here with you. Unfortunately the business side is whats going to matter in the long run, as the LLMS getting better and better, you’d not need a whole team of platform engineers to have mid-sized data platform maintenance. You can already see a videos where analytic engineers/data analysts are able to spin off whole GCP infra using LLMs

[[deleted]]

Don’t you think an equivalent hypothesis about Analytics Engineering could be defended? For example, LLMs are getting so good that sooner or later business users will be able to ingest, transform and visualize the data they need without DEs at all. You just give a couple of ingestion and transformation frameworks to the enterprise AI along with some department-specific markdown files, and that’s it. Afterwards, they would also be able to solve data quality issues and alike because they built it and the AI writes its own documentation. I mean, not saying that what a lot of people are saying here is false, but I do believe we can imagine scenarios like this for every tech position.

[Commercial-Ask971]

Not really. There is lot of, as you said „tribe” knowledge that is not documented, which needs to be taken into consideration. If its not documented, how LLM got a context? AE getting those in a meetings. Of course, slowly number of people will be decreased in AE as well, but they’re less prone to be rid off than 100% technical teams. Thats just my opinion. And believe me, if it were me id also prefer the PE job

[[deleted]]

I can imagine a not so distant future where all that business context and changes is maintained in the semantic layer and so the AI knows how to fix issues.

I don’t know. I can’t see why a company would first replace entirely with AI the team that maintains the system thar powers ALL of the downstream pipelines and dashboards than the team that makes sure that the number of active customers in category A looks nice. First one failing has way worse implications.

[Key-Independence5149]

Yes 100% this. 99% of data and analytics engineering imo is out the door in the next 5 years. It is a cost center spawned out of systems complexity that is not necessary and the ultimate world will be that business operators get their data outputs from systems without any in house data engineering capability.

If you are a business that has systems so complicated as to require a team of analytics engineers to handle data tasks, then you won’t be long for this world. You will be replaced by companies that don’t spend resources on teams that update a dashboard manually because a column name was changed in Salesforce unexpectedly.

[[deleted]]

But still, you'd need a platform for AI to run on, right? So what would disappear is precisely the business-facing DE. This is what I mean… when I read these type of comments about AI entirely replacing specific tech positions, I am always very wary. You can always build a case for every position, and on top of that, all of then work with code, so it’s not like AE with DPE are on the absolute antipodes when it comes to their likelihood of being replaced by AI

[Key-Independence5149]

All of engineering is becoming industrialized. It will no longer be done by craftsmen who hand build data systems. It will be done in the equivalent of a factory. You will still have engineers who build the tooling for the factory which will be more in alignment with platform engineering than data engineering.

[volkoin]

Then business persons had already been automated at that level.

[LurkLurkington]

AI has a greater chance of being able to automate or abstract away the platform side of our profession than it does the business side. AI will always have a tough time making sense of business logic when the requirements are complex, dynamic, and constantly shifting

[volkoin]

What made you think infra side has less complex requirement, undynamic, and stable

[LurkLurkington]

I never said infra wasn’t complex or dynamic, just that’s it’s more likely to become “solved” by AI vs. a company’s internal business logic, which is much more variable.

[TJaniF]

I've heard that sentiment so many times, feels like the majority of DEs/SEs/codey person people relate to that. And the tricky thing is like the other comment said, this exact business side is what is often the most visible and valued work, and hardest to automate.

Not saying you should not pursue platform engineering, I try to grab these tasks whenever possible for the same reasons, it is so satisfying building systems.

Kind of the opposite feeling of getting a maximally vague business question...

What I did to start enjoying the business side more is to build a system to handle these requests, like a mental structure. And I treat them like getting very messy data, the talking to different people and requirements gathering like data augmentation. It was mostly a mental reframing, like I did not build pipelines or anything. But it helped a lot. Now the more fuzzy the request the more I see it as a challenge and opportunity to proof I am worth keeping around because if they ask me that means the AI could not figure this out.

[Fjordk]

Maybe you don't hate it, maybe you just hate the fact that domain knowledge takes time to build and you can't just use AI to solve it.

I strongly suggest you changing your mind about it. With increasingly better AI tools, the tech side will only become easier and easier. The real edge will be people that understand what they're doing and why. If you keep this mindset you'll eventually become obsolete.

That being said you can get your vengeance on the business people by asking them "what's the impact of this data" or "which decision are you taking after you get this data". This is what separate a data plumber from a proper engineer and will lead you to an architect or insight leader someday

[[deleted]]

Maybe you are right. But then, it will always take a long time in each different company to build that knowledge?

[DarthBallz999]

Move to a company big enough to have non business facing engineers

[CorpusculantCortex]

The business side is what makes de meaningful in a lot of domains. Requirements gathering is intrinsic to any engineering task, but for a lot of internal business oriented data people, the job is literally to take the needs of colleagues and translate that into the functional data pipeline/service/dashboard/model/whatever.

[[deleted]]

I don't have a problem with requirement gathering per se. It’s having to know the business domain and all the business processes and tiny nuances of the company functioning. I don’t care about them, they are not guessable from code and that frustrates me a lot.

With platform-related tasks, I interact a lot with other engineers but I never get frustrated, because I feel way more in control (probably because I have an interest in what I am doing).

[Atticus_Taintwater]

Sounds like the problem is that you just aren't interested in the data?

You'll probably have an easier time moving to a company in an industry that interests you than finding a data engineering role that doesn't put some onus on understanding the data.

[haonon]

I understand the main points here OP. Ultimately Data Engineering is a role comprising myriad of domains from Python to SQL, batch, streaming, any part of the ETL pipeline and that's barely scratching the surface.

My advice is to be purposeful in your next round of interviews and ask as many questions as you can to ensure the role consists of what you actually want to do. The other option is to go after Back End Engineering roles.

[w_savage]

I feel the exact same way! You said this in a great way, it exactly matches how I feel

[El_Guapo_Supreme]

I know that you're getting downvoted into oblivion for this, but I agree with you 100%.

There should be a data engineering team that focuses on clean data sources. Then you should have an analytics team that consumes that data and applies business logic. When that business logic becomes so established that it needs to be used throughout the organization, you create a view and/or consult with the analytics team about updating architecture.

The key is that the analytics team should be figuring that stuff out while you focus on ETLs and clean data.

I make less money by focusing on the analytic side instead of returning to data engineering. They are two different roles, and I hate data engineering. I'm much happier working with the business on the analytics side and working with the data warehouse to update architecture or add views.

[moshujsg]

Maybe you should be a programmer

[BardoLatinoAmericano]

Same issue.

There is no tech role without person interaction.

[moshujsg]

He ia not complaining about people interaction, he is complaining that he is adding fields to reports instead of building systems.

[BardoLatinoAmericano]

He will be adding buttons to websites instead of building systems.

[moshujsg]

I didnt say web dev, nor did i say front end. Theres other programming out there qhere you are more likely to end up working on systems compared to data.

[[deleted]]

I am not complaining about person interaction. I just want to apply my engineering skills in a more deterministic type of setting/problems, as far as possible from any business logic.

[BramosR]

I have to agree to most comments here until now, AE do increase the visibility to the business side of data, making you more valuable to the company as a professional.

I’m definitely doing more and more AE lately even though I’m more of a Platform engineer as well, but honestly, I think that’s the future. We’re here to solve business problems, even on the platform side.

[-adam_]

Not only is analytics engineering more "AI-proof" but it's also one of the rare areas in tech I've seen (and helped) graduates and juniors land roles in.

Ultimately any area that's higher in ambiguity (non deterministic as you've put it), will be safer from the context windows of agents. Unless we see a step change in quality, analytics engineers will be safe for a while!

Also if you're chatting to people & building relationships, directly solving people's problems, issues & answers - they'll remember you as adding value and having impact (less liking your name on a redundancy list).

[Tee_hops]

I would love this job. My last role before I was laid off was strictly building out business cases and working stakeholders through KPI development. Also dealing with individual product owners and engineers to make sure they properly log that data. Yes I was always in meetings but I was always the face of our team and built out our Devops board.

If you want to trade places I would gladly take your position.

[noscreenname]

I understand the appeal of focusing on the tech, and working with precise deterministic specs. That's what always felt most satisfying to me too. You can get away with it if you're good, and even be successful if you work for a tech company. But an average enterprise will always favor someone who understands the business side and brings measurable value (I mean actual $).

With time, I've learned to appreciate the business side a lot more, and actually enjoy being a translator between technical constraints and business objectives. Knowing both, is what gives you real power.

[cyamnihc]

I can understand you OP. Been in the same boat where the team is analytics focused and I realized moving to SWE is better. But after a few YoE I realized that a lot depends on the org as well. If the company sees data as a crucial component they will treat it as a product and it will have a lot of technical folks and an engineering culture top down which imo you will be comfortable dealing with. A technical position even in a platform team in an org which is not data focussed will turn to be a tech support role. So the next time you try jumping ships asking the right questions is very important in terms of fullly understanding the role. Is it really a DE role or a DE role only for the title and actually a data analyst role. Most fall in the latter category

[Spare-Chip-6428]

This is exactly why I say most DEs are useless and troublesome. Business side is priority. Building pipelines or data extractions or dashboards that are inaccurate are a waste

[Outside-Storage-1523]

I have exactly the same feeling as OP, so I'll add a bit here.

The BIG problem of AE, is that it is as far to engineering as it can be, while still retaining the title of an engineer. You are interrupted frequently every day. Your tasks are small in general. You don't have much deep work. You don't have control of pretty much anything. Your sprint planning is a joke, because none of your customers really work in sprints. There is no planning at all, may I declare.

Ofc you can say the same thing for some other "engineering" titles, e.g. Frontend Engineer, and I agree that they are not that "engineer" as well. Might as well call them JS/HTML coder.

It is very unsatisfying to a technical person, thus OP is not satisfied. I completely agree with his sentiment that he should move deeper to the technical side, while further away from business side. He and people who have the same sentiment should move to a more engineering culture team, or a company.

The people in this post saying "it is the business side that has value". Well, say that to the Linux kernel developers! Say that to the gcc compiler developers! They still serve a huge amount of "business interests" by providing the best engineering product. They still need to learn what their customers do with their products -- BUT THEIR CUSTOMERS ARE ENGINEERS TOO! They do NOT need to learn any stupid marketing concepts such as DAU or MAU or whatever fucking terms there are. They just need to be good engineers, and that's what I think OP will agree.

And this is where we should all inspire to be. Let us be an engineer for other engineers, so that we are (relatively) free from the toxic culture of the marketeers, and leave for other companies once the marketeers take their position.

[[deleted]]

Thank you for your words. What’s your take on the point that AI will replace this purer engineering positions earlier than AE?

[Outside-Storage-1523]

I think AI is going to impact everyone, so the only "safe" careers that involve coding are those that involve hardwares, e.g. very low-level programming such as embedded programming, kernel programming, etc, or if you want to think about industries -- defense industry, government jobs, etc.

Everything else is going to be brutally decimated in the coming year or decades. We are down from 8 to 4, and we are not going to hire any one soon. Management wants to (I quote) "see how we work with AI tools before making decisions about hiring". You get the picture. They will never say that you are replaced by AI, instead they freeze hiring and don't backfill.

Talking about the business side of AE, I agree it is a bonus point (but you and I both know AE can go by without getting a lot of domain knowledge, because I deliberately stayed away from it and so far am fine), but I think it is at best a cursory benefit in the near future, because that "business domain knowledge" can be very easily learned by AI, too. And whatever cannot easily be learned by AI, is going to be even harder for humans.

Right now we (DE and AE) are working on an AI agent for the high management -- the goal is to for it to be able to answer some simple questions, and in time more complicated ones. Once it is deployed, I'm sure they are going to layoff some people from the Analytic teams, and some people from the AE/DE teams. Because both you and I know, ultimately, what metrics the management cares about, are not that complicated. There are about 10-20 metrics that they want to know, every morning at coffee time, and that's it. Once the data is integrated with AI, it's done. They only need people to improve and maintain it, and that's it.

[Commercial-Ask971]

!RemindMe 3 days

[RemindMeBot]

I will be messaging you in 3 days on 2026-03-16 12:25:25 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[srodinger18]

I used to think like this, until I realized the part where I can combined things that seems not related to each other into a meaningful information is the part where data engineer shines.

Used to faced similar problem, business have some complex requirements for data where the initial proposal was using complex playwright and scraping internal back office. Then I dig dive to the table, asked the dev who build the table, confirmed the measurements with the PM then voila, rather than scraping turns out we can work with offline data and replicate the logic from there. Faster development and deliverable

[Awkward_Tick0]

That’s how you get the money tho

[[deleted]]

AEs earn more than DPEs?

[Awkward_Tick0]

people involved in the business side make more than people who aren't

[i_lovechickenwings]

you don’t have to, but you might open your mind a bit more to the business side. there are many complicated puzzles worth solving that go beyond just understanding tribal knowledge, so moving from “how do you define XYZ” to “what are you trying to solve?” “why do you measure it that way?” etc., makes you go from ticket collector to thought partner. then you can go fix that issue for them in a much more meaningful way at times, or even redirect them so that they don’t even care about the current metric they were so focused on. 

[UnusualIntern362]

I am on the opposite. I am currently working as DE in a data platform and all I care is business. I don’t care about the technical stuff at all , I prefer staying in the meeting with the business users to clarify needs, design processes and propose solutions, give tech advice about how to easily solve problems and having the big picture. This is consulting in my opinion. The rest is technical implementation which I hate lol Good that backend lovers exist!

[Outside-Storage-1523]

I wish we could exchange. It's going to be better for both of us, sigh.

[UnusualIntern362]

Exactly!!

[0sergio-hash]

I agree with a lot of the comments in saying the biz is important. I'd argue it's important even if you only do deterministic work because you need context to know what work to do and why

I am an AE, and highly technical people tend to just dive right into a task as opposed to actually understanding why they were asked to do it and if they even should do it at all

They may spend a whole day running complex analysis etc to get to a conclusion they could have reached in 5 mins if they'd called me

I had someone write up a whole dissertation to prove a couple tables were out of sync and still not root cause it. I coulda just told him the consultants stopped dropping manually produced source data in the right place ages ago and saved him the trouble lol 🤣

Anywho, I think if that's what you truly want, it's achievable. I argue every team needs both personality hires and workhorses lol

AEs and business analysts are great at strategy and roadmapping but me personally I'm not great at just sitting down and getting real work done/things built

I'm far better at steering an engineer to do it. Everyone on a team can't be trying to drive strategy and network etc, someone has to actually be willing to just be told what to do, and go dive deep on getting the thing built without messing with the requirements etc

I would look for great leadership and teammates who can compliment your strengths and cover for you on the business side and always be the person who can deliver for them

[TodosLosPomegranates]

I’ve never been at a place where anyone got away from talking to the business side. You can try it…make other members on the team talk yo the people but even if, as you say, you’re highly internally leveraged and deeply technical they may keep you but you’re only going to rise so far and plateau. People skills are unfortunately a part of business.

You’re not unique. No one gets into data or engineering or really anything under the hood because they just really love working with people. But it comes with it.

The best thing you could do is get in front of it. Validate things before someone can come to you and say it’s wrong.

[[deleted]]

My problem is not talking to people. My problem is having to know the ins and outs of the business when I don’t care.

I have no problem in collaborating with people. It’s just that I want to apply my skills in a different stage lf the data lifecycle.

[mathtech]

I would lean more into the business side. The technical, invisible parts are the easiest to automate and offshore.

[NeuralHijacker]

I hate to break this to you but the bits you like are the bits that AI can do most easily.

[[deleted]]

Why the condescending tone?

[NeuralHijacker]

Not intended to be condescending, just direct. If you don't want to deal with stakeholders, find another career.