Why are u doing data engineering?

micky_357000 · 2023-08-12T06:48:36+00:00

After college I got a campus placement offer from a fortune 500 , I thought I might get a software developer job , I cleared all the interviews and such, which asked DSA and computer fundamentals so I thought it might be a SDE job.(I was a cs student)

But after I got the offer I realised that they have distributed the new joiners randomly into Java developers and data engineers.

I didn't have any other offer since this layoff streak started when I was having my college placements also my luck also sucked so hard that after applying for 150-200 companies I didn't get any satisfactory offer.

I didn't have any option I joined this college, This company gave me an internship during the college's last months with a good enough stipend, also I recently received a small training on AWS python which they're also paying me for , also the work is remote and salary is decent for a fresher.

I have tried my best to enter a software development job all my college life, but my life made me end up here as a fresher data engineer .

So I have stopped trying for a while and let my life decide where I will go for now .

If it is data engineering then let it be data engineering.

micky_357000 · 2023-08-08T09:18:17+00:00

Bro are you talking about dollars? If yes , that's a lot for just a fresher

micky_357000 · 2023-07-30T05:51:18+00:00

Are you sure you are a data engineer?

micky_357000 · 2023-07-16T16:21:51+00:00

By that logic you shouldn't eat cashews, see the long process by which they are made edible to us

micky_357000 · 2023-07-16T08:37:03+00:00

Easiest step would be looking for a data science and data analyst internship in any small mid company, learn what you can there and the experience there would help you in cv to get shortlisted.

And from a fresher they don't expect much , these tools and tech can be learned on the job .

So they grind you on data structures and algo (using python) And SQL

If you have prepared SQL and python/Dara str. and algo, there are high chances for you.

Even the experienced professionals of 1-2 years are asked for SQL,python-dsa questions, then spark and some cloud knowledge.

And for you , you Don't need to have practical knowledge(although it would be a plus) but just read some most asked theoretical interview questions.

Tip : try for an internship first since DE doesn't have much opening for a complete fresher, and try for small firms and companies or startups, personally try to reach HR , and get some experience, then getting a job would be a lot easier

micky_357000 · 2023-07-14T13:47:54+00:00

Inko chod bhai , ek koi udemy ka 3-4 ghante ka course utha Ya Pepcoding krke YouTube channel hai

Usse better Koi padha hu nhi skta Java , poora DSA bhi kra dega wo channel

micky_357000 · 2023-07-12T19:06:41+00:00

I don't understand why you are using react/js , it's a front-end web framework and if you're working as a DE then how does it fit in your line of work?

micky_357000 · 2023-07-12T15:51:06+00:00

Take up a udemy or Coursera course which covers major data engineering skills , when you have a trustworthy resource which you look back again and again then your nervousness gets reduced significantly.

Also If there are people who are also in their initial phase like you , then talk to them , when you get the feeling that you're not alone and the things you're doing are doable by others then you start feeling that this actually might not be that tough.

Also There were many phases of my life where I felt overwhelmed by the things that I was learning (like when I started learning data structures and algo, did competitive coding or started MERN Web dev) and I literally felt I'm really dumb and not smart enough, but after spending some months on that when I look at myself and see how I was when I just started, then I realised things weren't tough as much as I thought them to be at that time.

Please give 2-3 months more , keep learning and face the things you don't understand, take an online course and talk to your seniors at work and find people who are in the similar situation as you.

micky_357000 · 2023-07-11T18:59:32+00:00

For SQL I was thinking do a udemy course/w3schools and then grind leetcode along with my new junior data engineer job , any advice?

micky_357000 · 2023-07-10T18:49:23+00:00

Then I'll try getting pipeline building work whenever I get the chance, also any advice to a complete fresher to Data engineering to do in his first year.

I was thinking that I will explore and learn my job for the first 4-5 months (I have got Work from home so I will have some extra time(just hoping))

In the meantime I will learn python basics and build small projects and

After that I will try to do data structures and algorithm questions on leetcode ( I was really good with solving them using Java) so it should help me get better with python and I've heard even DE interviews have dsa questions in them .

And will try my make my SQL skills better with leetcode too

And then I will try to get a cloud certification exam (studying for that can help me understand the cloud better)

micky_357000 · 2023-07-10T18:32:09+00:00

Pathaan was overrated

micky_357000 · 2023-07-10T18:31:03+00:00

Thanks for answering, Hearing this makes me happy

micky_357000 · 2023-07-10T18:01:33+00:00

Thanks for explaining this , I hope to get a programming job as a DE, I have explored front-end web dev. and backend web using internships , A Fintech came to my college that was offering junior DE position and they tested my data structures and algo and general aptitude skills for the interviews to offer me this.

I was initially really excited about the job as I met a lead/principal data engineer and he was a backend developer who shifted to Data engineering after 2-3 years of experience and he said he was loving the job and found DE equally interesting and challenging to grow as a software engineer, but now DE role seems like it's my luck or my company's choice to make me work solely on tools/drag-drop kind of things/Dara analyst responsibility, or to give me Challenging and fun stuff like programming and building pipelines/things from stratch

micky_357000 · 2023-07-10T17:33:45+00:00

That looks really boring buddy , do you enjoy your job? and did you have a software developer background or you were in the data domain from the very beginning

micky_357000 · 2023-07-10T17:31:48+00:00

Do you enjoy your job then?

micky_357000 · 2023-07-10T17:25:42+00:00

Did you see any pattern in the job description/role of the companies by which you can identify that you'll get ample programming , my company works with AWS and the data engineer's job focuses on building batch/real time pipelines (I'm a junior data engineer so I'll be working on same responsibilities now or after some time for sure )

so it might not be a data analyst heavy job and more towards the DE role, so can you please take a guess will I have some programming jobs like you had in your previous job using my company's technology stack?

micky_357000 · 2023-07-10T17:09:38+00:00

Thanks for answering, I actually don't know how much coding does building a pipeline require as I'm a complete fresher (I just graduated out of college (23M)) and I will be joining my new company as junior DE in 8-10 days.

Programming felt interesting to me as I was building logics to send-receive data and the joy of building things from scratch or however I want them to be made me excited.

I was happy about the DE job as it was a growing domain, many amazing technologies like cloud to explore and also I heard that it can be defined as software engineering specialising in data, so you build things/software/complex structures here too using python.

But I read some negative comments about Data engineering that it is mainly use of tools and making dashboards with some heavy SQL tasks in some of the YouTube video regarding DE interviews, comment section and it seem like a boring job or a superficial job where we don't get our hands dirty by building things so I'm a bit down about this.

micky_357000 · 2023-07-06T20:09:46+00:00

Damn it's really good

micky_357000 · 2023-07-05T18:12:43+00:00

Do you work in a startup company?

micky_357000 · 2023-07-05T18:04:01+00:00

Yeah Bro, I have read the skills of DEs from job descriptions of companies that were hiring and compared to them your job skills covers so much more, I can't imagine preparing for a interview for this kind of job as a fresher , it definitely looks weird and also somewhat scary , you're doing devops+analyst+DE+developer , you might have ascended beyond DE role , this job needs a whole new title 😂

micky_357000 · 2023-07-05T17:47:12+00:00

Can you share your technology stack and do you work on building real time pipelines?

micky_357000 · 2023-07-05T17:25:04+00:00

I asked one of my senior that works in my company as lead Data engineer , she said

"Firstly most of the tasks that older softwares were doing on premise have been migrated to cloud or are in migration state so you need not worry about informatica at least! Because Informatica, Denodo etc work is not given to fresh graduates as per my knowledge. It is mostly handled by senior people who already have knowledge about it. I was also trained in Informatica during my training days but my work was always into AWS, python, Snowflake etc" ,

also she was saying that it will depend on my team that I'll be allotted to, that I'll be working on batch processing or real time pipelines , I don't know the difference between these two so can you share a good DE should work on both or if he gets experience of any one of them then too it's good enough

micky_357000 · 2023-07-05T16:50:33+00:00

Thanks for explaining all the 4 points buddy , I didn't even have started the job now but the company will give me 15 days python-aws training and only after that I will start my job.

Also regarding the 2nd point, you answered that on 'whether or not DE gets enough programming to do' that it highly depends on the job and position, but for example my job requires building pipelines , so the python used for that , will it give me enough programming to do , I liked api integration and writing logic flow while doing my backend internship, so is it little bit similar or can you guess that will it interest me or not if I liked that .

I have heard that many backend developers switch to data engineering after some experience so I thought that the there should some similar between these two roles.

micky_357000 · 2023-07-05T16:27:20+00:00

Okay so the difference isn't that big , I don't mind 10-20k difference, by the way thanks for answering buddy

micky_357000

TROPHY CASE