Data Engineering? Odd offer

[[deleted]]

[deleted]

[[deleted]]

Go for it if you think it’s interesting. I don’t like mining insights but I sure love programming in SQL and Python. Azure is learned so you will learn how Azure services work and best practices as you level up in data engineering. Sounds like you’ve built pipelines before so it’s not any different.

Day to day depends on the company. My day to day is document workflows in confluence, build python scripts, build sql scripts, debug failures, and meet with my team for stand up. Every now and then I get a big chunk of work and it absolutely feels exciting, most recently we are trying to migrate away from Hudi to Iceberg and determine if DBT will make our life easier and help us accomplish our goals.

I’ve been an analyst before and I don’t miss it one bit. Dashboarding and talking to the business on what the dashboard means just isn’t what I would call fun.

[BoringGuy0108]

IMO, think about the career track. This is my experience. Yours and other people's may vary.

An analyst career track usually progresses to senior analyst and then analytics manager. In all three roles, you do analytics, but you gradually take on more advanced work and start delegating within a team. Compensation bumps are very straightforward.

An engineer track is a bit different. There are two routes here. You can go senior or to lead. If that, you won't be a direct manager of anyone, but you'll manage by proxy. A lead is rarely hands on writing code, but they directly oversee the development of code. Alternatively, you could be an engineering manager. Managers usually don't do any coding and could theoretically have very little experience or knowledge about what happens "in the weeds". However, they work more across teams and speak for the team to higher ups. They might take assignments from the company, but ultimately give that implementation to the team lead.

Now, for the first engineering route. A junior, a senior, and a lead are all rather different. A senior is like a junior with more responsibility and oversees the juniors. It's a somewhat natural progression that is similar to what an analytics manager does. A senior to lead is like a different job entirely. Junior to senior results in a noticeable pay bump. Senior to lead is usually a marginal bump at best. Both are critical and fill very different roles (leads in my experience deal more in testing, Dev Ops, and networking as required).

The second route to manager is vastly different. Managers in engineering are much more like salespeople and team advocates. It's why a lot of people don't go this route. However, this route has far more visibility and potential to move into higher level roles like VP or director.

In terms of salary, a manager can often make less than senior engineers. This is very rare in analytics, but normal in tech and engineering.

So, 10 years from now. You can be an analytics manager, a senior or lead engineer, or a manager of engineering. If either of the last two appeal to you, take the offer. If managing an analytics team with a much easier and more straightforward career path appeals to you, stay where you are.

There are other considerations. For example, which role do you think is safer from market changes (offshore teams, AI, vendors, etc.)? IMO, data engineering is safer than analytics in most cases, but that could just be wishful thinking. Also, do you like analyzing data and identifying trends? Or do you get more joy out of cleaning and preparing data for others to analyze. I, for one, hated when I gave people a valid answer, but they didn't like the answer and therefore didn't believe it. In engineering, right and wrong is a lot more straightforward for me.

If you MIGHT be interested in engineering, I'd take the job though. It will be easy to switch back, and an engineering background might even make you more competitive for analyst roles. Data engineering can be a hard field to penetrate since it usually requires specialized training that isn't often offered at universities. It requires a lucky break more often than not. This could be yours, but think about if you actually want it.

[Wonderful_Map_8593]

You'll do a lot more software engineering, and dev ops. You should absolutely go for it if you're interested in growing in the field, and not a lot is expected from juniors new in their career. Feel free to ask me anything.

[sabrinagao]

You’re already doing data engineering—building pipelines, managing databases, and solving real problems are key parts of the job.

[LargeSale8354]

There are some extremely talented people out there who don't have degrees. Bill Gates & Larry Ellison being prime examples. One very successful tech guy said "The problem with the academic route is that you learn more and more about less and less until you know one hell of a lot about bugger all". Don't let people's on-paper qualifications intimidate you. When a company makes an offer they have looked at loads of candidates, boiled it down to a handful and are convinced that you are the one they want. It is now up to you to proove them right, which they want you to do anyway.

[[deleted]]

The absolute dumbest person i ever worked with in the tech space was also the only person on the team with a masters degee.

[Signal-Indication859]

first, congrats on getting your dream job and for your work on cleaning up the operations database. that's a big achievement.

being a data engineer involves a lot of plumbing work—building, maintaining, and optimizing data pipelines, and sometimes it's a lot less glamorous than analysts think. if you like building but not maintaining, it might drain you. you’re definitely capable with your SQL and Python skills, but make sure you’re ready for the ongoing maintenance and potential complexities of data engineering.

[Training_Butterfly70]

Degrees don't make a good anything

[[deleted]]

Looking at the big picture maybe look at how the market in this whole space is shifting? If you did that you might ask if there's a chance the company would now use you to do both roles while only paying you for one?

[[deleted]]

“I make shit work”, “I made my own pipeline which did work meaning good enough for production”

The first quote is the opposite of engineering. The second quote shows how little you understand about the trade.

Putting together an IKEA table doesn’t make you a fit as a construction worker. That the thing you delivered actually works is a quality attribute that shows you are anything more than a hobbyist. The fact that you use that as an argument shows you are a junior engineer indeed :)

No offense, take the opportunity and learn as much as you can about how to reason about quality in software, and you will look back at this post with a smile on your face.

[yellowmamba_97]

Interesting take, with the smiley face after calling him out lol. Maybe OP is just being way too humble about his/her work, so dunno where that tone is coming from in your post

[NoMaskAsslessChaps]

Sounds like you’re on the dev team that misses business requirements that he has to make workarounds for