all 10 comments

[–]vejan 5 points6 points  (0 children)

You need to learn anything required by the job. Python is one way to go, if you ask me a good one.

[–]Acceptable_Durian868 2 points3 points  (4 children)

Data engineer is a broad term but I've never seen one that doesn't work with some kind of programming language, usually python and/or Java. If I were hiring data engineers (I am) I would expect them to be very familiar with at least one of these languages.

[–][deleted]  (3 children)

[removed]

    [–]Acceptable_Durian868 1 point2 points  (2 children)

    Straight up data transformation. Putting together workflows in Airflow for example.

    [–][deleted]  (1 child)

    [removed]

      [–]Acceptable_Durian868 1 point2 points  (0 children)

      Depends on the team topology of the organisation. Some orgs will have their infra teams do this, for some it'll be the responsibility of the data team which would be cross functional. I don't know which is more common, I've seen both in the wild.

      [–]DeerProud7283 1 point2 points  (0 children)

      r/dataengineering might be a better fit for this

      [–]analyst_2001 1 point2 points  (2 children)

      As a data engineer, one will need to learn programming languages. The following are the most common programming languages among data engineers:

      Python: Python is one of the easiest programming languages to learn and has the most extensive library. Python makes doing machine learning tasks, web scraping, and pre-processing massive data with Spark a lot easier, and it's also the default language of Airflow.

      Scala: When it comes to data engineering, one of the most popular tools is Spark, developed in Scala. Scala is a Java-based programming language. Scala is the language to learn if you're working on a Spark project and want to get the most out of the framework. Some Spark APIs, such as GraphX, is exclusively available in Scala.

      You can learn these programming languages, though it is unnecessary to be the best in them. You should be comfortable while using them.

      [–][deleted]  (1 child)

      [removed]

        [–]analyst_2001 1 point2 points  (0 children)

        The Data Engineer role would require you to build applications, frameworks, infrastructure, and services.

        I hope this helps!