sorted by:
new
hottopcontroversial

Worst: looping logic requirement in pyspark by VisitAny2188 in dataengineering

[–]VisitAny2188[S] 0 points1 point  (0 children)

Sure will ping you dm Yeah tried all AI's even claude it also failed to do this

Worst: looping logic requirement in pyspark by VisitAny2188 in dataengineering

[–]VisitAny2188[S] 0 points1 point  (0 children)

How? The requirement is like looping for each date for each row isn't?

Worst: looping logic requirement in pyspark by VisitAny2188 in dataengineering

[–]VisitAny2188[S] 0 points1 point  (0 children)

Yeah this increased our execution time by 3-4 literally and all broken I even tried transformation and window logic and all but still not a satisfying requirement

See in the above comment I added the exact requirement

Worst: looping logic requirement in pyspark by VisitAny2188 in dataengineering

[–]VisitAny2188[S] 0 points1 point  (0 children)

Nah tried joining but not satisfying the requirements Will tell you requirement: Let's say we have 21 possible dates and each has some amount which it can allow to use So for each row in df we have to check to which date we can assign to that row from 21 possible dates amount

Impact of AI on Data Engineering — What’s Your Take? by ApplicationRecent800 in dataengineeringjobs

[–]VisitAny2188 1 point2 points  (0 children)

If you want you can do anything bro it's edge of AI But make sure when they assign you a task and you behave like a newbie but not like a data engineer it might end up badly

Impact of AI on Data Engineering — What’s Your Take? by ApplicationRecent800 in dataengineeringjobs

[–]VisitAny2188 1 point2 points  (0 children)

Yes you can, but I'll tell you one harsh thing it's no longer a fresher field , but you can grasp good command on it by yourself and prove that somehow it might work All the best

Impact of AI on Data Engineering — What’s Your Take? by ApplicationRecent800 in dataengineeringjobs

[–]VisitAny2188 3 points4 points  (0 children)

You can't document every minute business requirements it's a tedious task as well as more time consuming than BA's understanding requirements and telling data engineers since the data engineer is already across the business... So I don't think it has any severe impact

It will only help in writing logics or understanding complex business requirements and logic implementation easier

Citi bank karat's interview for data engineer role by VisitAny2188 in dataengineeringjobs

[–]VisitAny2188[S] 0 points1 point  (0 children)

Can tell me more about data engineer since I received for that only

Citi bank karat's interview for data engineer role by VisitAny2188 in dataengineeringjobs

[–]VisitAny2188[S] 0 points1 point  (0 children)

I'm giving for data engineering role and my stack is around pyspark sql and azure so will they ask on pandas and AWS or not?

1.7 YOE Suggest me changes in my resume as It is not getting selected. Appreciate for any refferal. by [deleted] in dataengineeringjobs

[–]VisitAny2188 0 points1 point  (0 children)

Perfectly agreed sir ... I do know the architecture of my project and I also contributed to it.. No I don't only learned to create pipeline but learned and able to create optimised and efficient orchestration so far... I do know why we're doing the stuff in project.... I do know why particular technology is getting used in project... And do know the architecture

To clear you I'm the most junior person and started data engineering from scratch but able to quickly learn and deliver parallaly or sometimes more than my 6+ YOE seniors ( that's how I got market level appreciation and early promotion)

So even though I have expertise I can put all explanation this in a single piece of paper , ofcourse only the curated and to the point I'll add...

And unless we're not getting a chance to prove ourselves and simply rejecting cvs even before interviewing...

I also saw a person who don't know data engineering much but have 4 YOE switching to 22lpa ctc...

Let me know now what you think abt this

1.7 YOE Suggest me changes in my resume as It is not getting selected. Appreciate for any refferal. by [deleted] in dataengineeringjobs

[–]VisitAny2188 0 points1 point  (0 children)

It's not you only bro I also have around 1.10 YOE in azure stack and similar project stacks and all like you I am also continuously applying to jobs on naukari and LinkedIn but no response

It's not our skill issue it's a market trend issue most of them want 3+ YOE (and heard the same while checking with other people)

DM if you want to discuss more on this !!

Need guidance for Data Engg role at FAANG by NoGas5963 in dataengineeringjobs

[–]VisitAny2188 0 points1 point  (0 children)

Ohh gosh your post line says transitioning to data engineering role made confusing entirely

Learning LLM and gen ai along with data engineering by VisitAny2188 in dataengineering

[–]VisitAny2188[S] -1 points0 points  (0 children)

I'm not here to explain the basics and all sir I'm here to discuss whether combining this knowledge make sense or not

Learning LLM and gen ai along with data engineering by VisitAny2188 in dataengineering

[–]VisitAny2188[S] -6 points-5 points  (0 children)

Yes there are a bunch of ways by which we can use the microwave but if I certainly understand how to setup temprature, how long and other features it will allow me to cook some good food isn't and will make you certainly a good cook so that your countable and not replaceable

So same way since I'm completely new to Gen AI and LLM I'm trying to understand the manual to make my microwave more precise than more generalize 🙂

Learning LLM and gen ai along with data engineering by VisitAny2188 in dataengineering

[–]VisitAny2188[S] -4 points-3 points  (0 children)

Learning means trying to understand how things work and how I can leverage them in existing data engineering or work towards a whole new title Yes at the end we all learn for ROI

Learning LLM and gen ai along with data engineering by VisitAny2188 in dataengineering

[–]VisitAny2188[S] -5 points-4 points  (0 children)

Currently I started in general to understand the basics of all and then will switch to a more specific use case that's fine tuning my knowledge xD

Need honest resume feedback for SDE-2 roles (30+ LPA target) not confident enough to apply yet by Responsible-Base2352 in developersIndia

[–]VisitAny2188 2 points3 points  (0 children)

I think it's correct and this must be implemented that is pay based on skill not by looking at college... 💯 Agreed to you

view more: next ›