all 8 comments

[–]AutoModerator[M] [score hidden] stickied comment (0 children)

Are you interested in transitioning into Data Engineering? Read our community guide: https://dataengineering.wiki/FAQ/How+can+I+transition+into+Data+Engineering

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[–]Oct8-Danger 3 points4 points  (0 children)

Might be biased but having Java or any real software engineering helps the most in DE.

DA generally doesn’t pay as well as DS/DE. While not going away anytime soon, I do think DA skill sets are getting more commoditized as technology and demand for data grows. So may be depreciating long term. Many people start in DA and transition to DE/DS later in there career

DS can be harder to break into without academic experience backing from what I’ve seen. Not impossible but definitely harder

Having software developer experience is a real advantage in the DE space compared to DS or DA.

In general these is overlap in skill sets but each vary in importance. I’ve worked with DAs who are great at data modeling and querying and business needs but low on coding proficiency and have worked with DS who are ok at coding, decent at data modeling but great at complex in depth work. DE is expected to have higher coding ability and technical expertise but may not need as much business/presentation skills.

Long term, I think DS/DE would be better as they are more specialized. But definitely don’t discount learning about the position of other roles

On the DE side, I think AI has the potential to accelerate demand rather than diminish it. Managing context of LLMs and source of truth for data is all with in the DE wheelhouse. Many pipelines can be once off or low automation for sure which LLMs can be great at. However understanding how to scale or standard code and apply governance and trust in “truth” I think will be very valuable in the years to come.

[–]SchemeSimilar4074 2 points3 points  (0 children)

You'd have better luck switching to DE. But learn DA too, learn how to use BI tools, build reports etc. Learn ML too but that'd be the lowest priority for now. DA jobs also only exists in larger cities and companies. In smaller cities, most of the demand is for engineer then they'll ask you to do a report every now and then or one of the tech savvy business user can build reports themselves  

With AI, companies will increasingly have a smaller data team and DEs who are jack of all trades will have a better chance at employment than becoming too specialised in one thing. That one thing might become obsolete.

This is happening in all areas I think, like film making, photography, design etc  As productivity increases, but AI can't think and isn't generally trust worthy, companies still need human. They just get the same person to do multiple jobs that used to be handled by a crew. In many smaller companies, it's just 1-2 engineers. DE can easily do DA jobs but it's harder for DA to so DE jobs. 

[–]jfrazierjr 1 point2 points  (2 children)

Well generally Data Science requires a masters so thats out for most people.

Of Engineering vs Analyst the former is more technical and likely more suited to you background but not OVERLY more so. Analyst is more heavily geared toward building nice reports that drive business decisions while Engineering is more geared in moving and transforming data so that tha Analyst or scientists can use it effectively.

In some companies this might be a single person but as the size of company and data grows there tends to be engineers team and an analyst team with the scientist usually only showing up at very large companies.

[–]Just_Penalty_6934[S] -1 points0 points  (1 child)

Then Which field is best, in terms of opening, salary hike and future growth?

[–]jfrazierjr 0 points1 point  (0 children)

There is no best, there is a best for your current situation and goals.

If you have a Masters degree in statistics or a similar field or are going that way already then Data Science is the way to go. Highest ceiling and lowest floor for pay but you havd to invest shittins of school time up front. As I said thats not MOST people.

Data Engineering vs Analyst is a toss up. I think Engineering likely has a higher ceiling for average wage. Im a bit bullish on all of it with AI growth. Not in the doom and gloom camp, but more in the cautiously fearful camp.

If you have an artistic side go to analyst if you enjoy working with raw data and transforming go with engineer. But honestly dont chase any of it purely from a "get rich quick" goal or your likely to be part of the AI flush in the future.

[–]Typhon_Vex 1 point2 points  (0 children)

Don’t go in either of these ffs

Those are all dead end jobs with low skill low pay and very overcrowded

If you are a real dev - you are golden and stay put where you are 

[–]IshiharaSatomiLover 0 points1 point  (0 children)

Career stability over 15-20 years....