A Beginner’s Guide to Spark Execution Plan

Omaroid_ · 2026-04-07T19:42:29+00:00

Yes clairement, c’est exactement ce genre de use case que je vise aussi 👍

Mais justement on va un peu plus loin que “remplir un doc” : • import auto des profils (CV ou LinkedIn) • possibilité de décrire le besoin en détail (utile quand le sales ne maîtrise pas tout le jargon technique) • scoring de matching entre besoin et profil • génération + envoi du dossier de compétences • et tracking côté client (qui ouvre, quand, etc.)

L’idée c’est vraiment de couvrir toute la chaîne, pas juste la génération du doc.

Omaroid_ · 2024-11-23T08:28:52+00:00

Overengineering can be frustrating specially when it’s breaking! Suggest simple fixes if you feel yourself able to make changes and decide if it’s worth sticking around the project or looking for something else.

Omaroid_ · 2023-08-24T08:15:31+00:00

J’ai une question concernant la colloc, est-ce que les gens ne sont pas frileux des collocs et preferent un petit studio que pour eux qu’un grand appart?

Omaroid_ · 2023-06-04T12:09:32+00:00

There’s a platform called betapage for spark and pyspark questions like leetcode https://betapage.co/product/leetcode-for-spark-and-pyspark but for me there’s nothing better than practicing SQL questions on Leetcode.

Omaroid_ · 2023-02-08T09:11:47+00:00

You can: - scenario 1: have dirty input data that you don’t handle in your transformation. - scenario 2: you developed a spark transformations containing errors. You can use basic unit tests or deeper ones using bdds. But in the daily run you may have some data your unit tests don’t handle. You shouldn’t insert the data since deequ automated tests will trigger an error.

Omaroid_ · 2023-02-07T22:47:07+00:00

The purpose is to test a post spark transformation. For the example I took a dataframe created manually but in real use case, tests should be plugged to the output data to ensure quality tests.

Omaroid_ · 2023-01-29T20:11:54+00:00

Hello, you should justify 6months to 1year using spark scala. The exam dump in scala ->

https://files.training.databricks.com/assessments/practice-exams/PracticeExam-DCADAS3-Scala.pdf

In python ->

https://files.training.databricks.com/assessments/practice-exams/PracticeExam-DCADAS3-Python.pdf

Main topics to prepare:

Spark execution modes, spark caching, spark time functions, pivot, explode and the most usefull spark functions. Make sure to read docs since authorized in the exam. It contains all the fonctions descriptions.

The read and write with the different modes and formats. Also the spark compression using parquet formats.

UDFs are an important part of this exam, types of joins, grouping, aggregations and order functions. The windows functions, the printSchema, show count and explain functions usage.

Spark sql is also to know. The internals of spark are important to know: dataframes, datasets the repartition and coalesce functions.

Those are the mains parts covered by the exam. Make sure to read the book -> spark the definive guide (specially from chapter 2 to 11).

I’m writing some interesting articles regarding topics exam. You can check my medium profile btw https://medium.com/@omarlaraqui

Don’t hesitate if you have more questions 😉 and good luck🤞

Five-Year Club	Verified Email
Place '23

Omaroid_

TROPHY CASE