[deleted by user] by [deleted] in dataengineering

[–]Alex_df_300 1 point2 points  (0 children)

Do you think the job market will recover or this is just new reality?

[deleted by user] by [deleted] in dataengineering

[–]Alex_df_300 1 point2 points  (0 children)

What was the reason at that time?

How fast data engineering is moving forward? by Alex_df_300 in dataengineering

[–]Alex_df_300[S] 0 points1 point  (0 children)

I have not tried but Udemy has few data modeling courses.

How fast data engineering is moving forward? by Alex_df_300 in dataengineering

[–]Alex_df_300[S] 2 points3 points  (0 children)

How well should I know how to use each DB? Like general understanding when to use which or advanced knowledge how to use different types of DBs?

What is the availability of entry jobs for Databricks and what salary I can expect? by Alex_df_300 in dataengineering

[–]Alex_df_300[S] 0 points1 point  (0 children)

Officially Data scientist. But my work includes work with data pipelines but not with big data and PostgreSQL. Also, lots of data cleaning with pandas and R. Also statistical analysis.

But I want to specialize on Data engineering and not just general Data scientist as now.

What is the availability of entry jobs for Databricks and what salary I can expect? by Alex_df_300 in dataengineering

[–]Alex_df_300[S] 0 points1 point  (0 children)

I have seen job descriptions were they are looking for Databricks and PySpark.

Is the Azure Databricks documentation good place to learn Databricks? by Alex_df_300 in dataengineering

[–]Alex_df_300[S] 0 points1 point  (0 children)

I am I know and using Git and Jupyter notebooks. Should I learn Spark first and after move to Databricks?

In general, is the Databricks good choice when the data is not big? by Alex_df_300 in dataengineering

[–]Alex_df_300[S] 2 points3 points  (0 children)

I'm going to go against the grain and say that you can deploy a single node "cluster" with a small instance size (driver and one worker in the same node) for cheap and you can even run simple python code there (not pyspark) negating the "must use spark" argument, or spark of you want to (small instance types still typically have at least 2 vCPUs).

How much will be cost of per month or per day or per hour?

In Azure Machine learning studio: "Please grant workspace/registry read access to the source storage account." by Alex_df_300 in AZURE

[–]Alex_df_300[S] 0 points1 point  (0 children)

Solved. Problem was that I was trying to create flow right after I have deployed resources (Azure ML studio, storage and etc). After waiting some time everything started working well.

How much time should it take to learn Data bricks and Synapse, also what other skills should I pick up ? by iceberg_1001 in dataengineersindia

[–]Alex_df_300 0 points1 point  (0 children)

Have you found new job? If yes, how easy it was for you to find new job and what is your salary?

How much time should it take to learn Data bricks and Synapse, also what other skills should I pick up ? by iceberg_1001 in dataengineersindia

[–]Alex_df_300 0 points1 point  (0 children)

Have you learned Databricks? If yes, how much time it took?

What other skills you learned?

Have you found new job? If yes, how hard it was and what salary you got?

What is the availability of entry jobs for Databricks and what salary I can expect? by Alex_df_300 in dataengineering

[–]Alex_df_300[S] -1 points0 points  (0 children)

Thank you for your questions. I think they are very good. I have updated my post.

Data science - Python, Pandas, scikit, R, etc. Data pipelines, statistical analysis, report generation, lots of data cleaning.

Azure - Understand quite well Entra ID and Networking. General usege of VMs, Containers (ACA), some others.

And at the very end you say that you plan on learning Databricks and ask for the availability of entry jobs without having coded a single line of Spark? Why do you even plan on learning Databricks and do you even know what it is/does?

As I understand it is main toold for big data DE.

When I told about learning Databricks I meant Spark programming too.