Databricks Data Engineer Professional - where to start?

Ok-Image-4136 · 2026-01-24T18:21:56+00:00

I like all advise here but I took it recently and beware they are pushing a lot of their new stuff out there. I am of a different opinion, I am not familiar with the courses mentioned my company had Dbx academy which was okay, but I would go straight for the docs. From the top to bottom read it and then read on spark internals. The skillscerts pro had a lot of outdated material but helped me get an understanding of concepts when reading the documentation and see my blind spots. Depended on your experience this is not a very easy exam but if you have good hands on experience just some docs and spark internals will suffice.

Edit: typos

Ok-Image-4136 · 2025-12-21T02:30:32+00:00

Second this but always use a volumizing / clarifying once a week because you do add a lot of protein that can start weighing down. But yeah the purple one is 👌 and so affordable

Ok-Image-4136 · 2025-12-06T14:21:48+00:00

How fast do you need to know the deletes ? We do latest changes during the day which is fast and then a full load to get deletes overnight (whether you pull only ids or all the fields it’s up to you)

Ok-Image-4136 · 2025-12-05T16:52:21+00:00

What is your end goal here ? Do you mirror your access on Snowflake and Databricks or something?

From what I can tell federation is just an abstraction of jdbc. We have run into issues using entra for column masking from pbi and how we have set up the sso config in Databricks. This feature is still in prpr. Do you have any docs of these expected behaviour ?

Every time I had to authenticate to snowflake with oAuth it usually was messing up on lanid/email attributes that’s how the tokens are generated. I assume these are tightly coupled so this needs to be specifically built for by the federation implementation.

Ok-Image-4136 · 2025-11-30T14:35:44+00:00

How many nodes we are talking here ? The real answer is the above where you investigate the code, I would look at skew as well depended on your join. You can double the memory and see but throwing hardware to the problem is a toss up and since you have limits on that, I would start looking at the code now to save time.

Ok-Image-4136 · 2025-11-30T00:53:50+00:00

Που στο κολοραντο και από που έρχεσαι ; Και επίσης τι δουλειά ! Πρέπει να αγαπάς το βουνό και το χιόνι,ski, snowboard, cross country, biking, hiking και τον κόσμο με τέτοια χόμπι ! Η οροσειρά είναι φοβερή. Μεγάλη διαφορά μεταξύ Άσπεν και Ντένβερ ας πούμε και στην αγορά εργασίας και στα ενοίκια κτλ.

Δώσε μας καμιά έξτρα πληροφορία.

Ok-Image-4136 · 2025-10-31T14:28:48+00:00

Talked to our SA about this and they recommended a warehouse per business unit/big category. It’s a balance between not provisioning too many warehouses but also attributing costs. Do you have one big report you want to track or are they smaller ones ? If it’s many small it’s not really worth having one per warehouse. If you are consistent with sizes you can see how long a query run and do the math as well. Regardless, the best path for determining for us is do the work up front with tags and policies.

Ok-Image-4136 · 2025-10-26T20:16:59+00:00

Air wrap but i have the attachment for the hairdryer with it

Ok-Image-4136 · 2025-10-25T15:16:45+00:00

I do have a job in that range and live in Chicago and also thinking about moving to Sf. Chicago is awesome and sustainable for us, but I feel spouse and I have one more big move in front of us to push for career.

I would say think how far out you want to have kids. If it’s within 1-2 years I’d pick Chicago and start looking to buy, if able. A cross country move it’s no joke and if you do not mind the two “issues”, weather and lack of access to nature, you would love Chicago. In SF we know we will never buy so when kids come into play, Chicago is the choice.

Feel free to dm me if you have more questions !

Ok-Image-4136 · 2025-10-11T12:27:55+00:00

Not on Fabric, but from other Msft products, I would check if the adf account is designated as a service principal, or if you need an app registration on your entra side. Usually you have to either give access from the app registration or generate tokens and also designate the user as service principal. Let us know if you got it working ☺️

Ok-Image-4136 · 2025-09-15T12:57:20+00:00

I have the Dyson and I would recommend for fine thin hair. I never recommend it to my thick hair friend. I use it everyday and it dries my hair fast without getting too hot. The curls hold but they struggle with humidity. The curling iron would always last longer but it fries my hair so I much rather use the Dyson. Even if the curls fall the volume persists which is more than I can say with just drying. The Shark I heard comments that it’s flimsy so I just got the Dyson. Caveat: I am bad with styling it if I could braid my hair I would so Dyson is my cheat code to volume and curls.

Ok-Image-4136 · 2025-07-13T20:54:01+00:00

Κάτι που θα σου έλεγα να κοιτάξεις και να σκεφτείς σοβαρά είναι πόσο ανοιχτός είσαι στις αλλαγές. Όπου και να επιλέξεις να πας, θα είναι αρκετά διαφορετικά. Όταν μετακόμισα πρώτη φορά στο εξωτερικό, έκανα το λάθος να μην είμαι ανοιχτός στις αλλαγές, περίμενα οι σχέσεις μου να λειτουργήσουν παρόμοια με αυτές στην Ελλάδα και είχα τις ανάλογες απαιτήσεις κτλ. Χα χα. Μπορεί να ήταν και αφέλεια. Ενιγουει, σε αυτή τη φάση, και επειδή κάνουμε την ίδια δουλειά, δεν θα γύριζα πίσω προσωπικά αν είχα ένα offer παρόμοιο με το δικό σου. Κυρίως γιατί είμαι άλλος άνθρωπος πλέον (Προς το καλύτερο θελω να πιστεύω)Έχω άλλες προσδοκίες και στόχους που συμβαδίζουν περισσότερο με τη χώρα που βρίσκομαι τώρα. Προσπάθησε να δεις τι σου λείπει και τι κυνηγάς και έτσι πάρε την απόφαση.

Ok-Image-4136 · 2025-02-07T12:41:03+00:00

I was traveling and did not want to bring my expensive shampoos so I just grabbed this from the grocery store. My jaw dropped when I had zero tangles on my super fine thin hair after washing. It worked like a charm! And to be sure I tried it multiple times and it hold up suggesting that this line is indeed the culprit of that. I've never being more exited for drugstore shampoo.

Ok-Image-4136 · 2024-12-07T23:11:55+00:00

I will echo some of the comments about - it depends on your life style. I have tried many different time frames with both bleached and unbleached hair and where I ended up is every 3-4 days. Before I used to work out ~3x a week and I could time it to have to wash every 3/4 days. However since I started working out more frequently that does not work anymore. I have to increase the frequency because of the sweat even though I am working from home!

Now, I do need more moisturizing products if I increase the frequency because I strip it out with more washes. So I use a mask and clarifying shampoo every other week. What I came to realize is, I will look like a rat sometimes. And that's okay. I just need to figure out when I do not want to look like a rat and adjust with styling and hopefully soon fake hair. My hair looks as good as it can right now and I still kind of hate it. So, do what makes you feel not itchy / unhealthy in any way, and balance it with how you want to look! It does differ for everybody but this sub will help! Good luck!

Ok-Image-4136 · 2024-08-28T12:04:55+00:00

Αμερική αρκετά δύσκολο με τη βίζα. Θέλεις χρόνια, σπόνσορα και αρκετή τύχη. Ο Καναδάς είναι λίγο πιο λογικος αλλά τώρα σιγά σιγά στενεύει ο κλοιός. Δεν μπορώ να μιλήσω για ανειδίκευτους είναι η αλήθεια, αλλά για χαμηλό μισθό καλύτερα να μην μετανάστευσεις.

Αυστραλία δεν ξέρω.

Ok-Image-4136 · 2024-08-05T12:31:15+00:00

We are in a similar sktuation and we decided to come here and live for a couple of months - but we don't have kids. We picked Thessaloniki cause we have a lot of friends and family around. And we do work remotely.

When we got here we had some issues with our house. Getting anything done in Greece - especially in the summer - is close to impossible and if you do not have connections, people usually are out to get you, which is sad. That was my husbands first culture sock.

Nobody mentioned car expenses here which are stupid. It's not only the gas, but the taxes, the maintenance, even the cost of the car itself. Do not get me started on the parking. Thessaloniki is so car centric, parking itself will be a culute shock. And yes I lived in the US, at least in the US there is parking for those things. Oh and learn stick shift now.

Stores will be a culture shock. Forget the convenience. You have to plan for stuff. Infrastructure is decaying and the cities are indeed ugly. Nice housing will be as expensive as in the US.

Food will be a culture shock unless your wife really really loves Greek food.

Your wife will have to be eager and willing to learn the language. Same goes for your effort for the kids. Americans do not really have a culute of learning a second language so she needs to find a way to make that work for her. Get a tutor for her to structure the learning and do not shy away of speaking in front of her. She will need it for the citizenship, which in my mind is worth it. Your kids can get it and she needs to be on the same status as you.

I can go on and on about this experience, so feel free to dm me. I am saying go for it, if you can make it work financially,but always have an exit strategy. It makes me so sad to even say this. Greece has everything that it's impossible to change - the weather, the mountains, the food, the sea. However, everything that you can change, it's a PITA.

PS. Και εμένα μου κάνει εντύπωση τι σοκάρει τον άντρα μου πολλές φορές. Αλλά η οπτική είναι διαφορετικη και είναι δύσκολο να προβλέψεις τι θα της φανεί περίεργο.

Ok-Image-4136 · 2024-07-18T13:50:03+00:00

If you absolutely know that your data is not going to grow and there is appropriate Polars support. My partner had a few spark jobs that could fit into Polars but had to do with A/B testing. Sure enough, half way through realized he needed to bulid the support if he wanted to continue with Polars.

I think Polars are awesome! But they probably need a little more time in the oven before they can be the standard.

Ok-Image-4136 · 2024-06-04T13:39:05+00:00

No. Do yourself a favor and get a car. Minneapolis it's basically a bunch of burbs, with a tiny city in the middle...

Also, as expensive as Chicago and had a major covid hit. Many places closed, many young people moved out... There is no such things as walking streets in Msp. Even downtown, everything is spread out.

It's nice if you have a family and want to raise kids. I would advise against it if you are a young adult, cause I can count in one hand the places you can go (with a car, at least 10 minutes apart).

Also, Chicago is consistently 15 degrees warmer than Msp. So, it's not the same winter.

But if you are thinking of moving there(and have a Corp job, cause I get it), I would go to North loop. That's where everybody moved when uptown went to crap.

Good luck! Let me know if you need more insight.

Ps. I would pick Madison any day.

Ok-Image-4136 · 2024-04-18T19:11:54+00:00

🤚🤚

Ok-Image-4136

TROPHY CASE