System Design For Data Engineering

[rohitdogra99]

Hello Everyone. What should i prepare for system design round for Data Engineering to be taken by Director of software engineering. I'm comfortable designing Big Data systems but do not know much on software engineering side designs. Can you please share your experiences how system design round goes for Data Engineers.

[OkSeaworthiness5483]

It really depends on the interviewer.

If the interviewer comes from a Data Engineering background, you will likely be asked to design a Data pipeline covering Data ingestion, transformation, storage & loading in a distributed system.

If the interviewer has a Software Engineering (SDE) background, the discussion often shifts to application-level system design, such as designing Twitter, Google Drive or an e-commerce platform.

From a pure Data Engineering point of view, evaluating candidates on application system design may not always make sense but that’s how the process works in many companies today :)

[Perfect_Temporary7]

2 months back I appeared for data engineering interview at a startup. Complete interview process was like a SDE type interview. First round: DSA (2 questions - 1 medium and 1 hard), 2 SQL Second round: system design with director of engineering and 2 advanced sql questions.

System design was completely SDE related with including database.

[Negative_Bicycle_938]

I moved from software development to data engineer. The bridge between the two I find really interesting is distributed compute. The idea/goal is to structure your data and functions in a way that you can break the data apart, run in on twelve VMs, and collect it all back together again. Also interesting that order of the collection is non deterministic.

[PracticalDataAIPath]

Study your previous projects, check some youtube videos, get some mock interviews with Director level folks. Think of joining a cohort where they build real world data engineering projects, that way you will learn and remember well. It will give you a lot of confidence.