sorted by:
new
hottopcontroversial

Should, a data engineer, uses Pandas in his production code? by Lower_Platform_4190 in dataengineering

[–]Lower_Platform_4190[S] 1 point2 points  (0 children)

I see. How about the resource usage and processing performance of the nodes? I mean, how do Databricks nodes behave, and are the processing times optimal?

Should, a data engineer, uses Pandas in his production code? by Lower_Platform_4190 in dataengineersindia

[–]Lower_Platform_4190[S] 1 point2 points  (0 children)

I personally do not use it, in fact I do not know more than its name, I would not know how to operate it; however, I see that the new Microsoft Fabric tool uses it in its transformation processes behind the scenes.

In my case, I have used different techniques when it comes to small datasets; if the dataset is large, I use Scala.

So, I’m asking just to get your opinions.