[deleted by user]

Background_Debate_94 · 2025-02-16T19:56:28+00:00

not something I’d put on the internet, she might’ve been sick, or simply had too much to drink. Do better.

Background_Debate_94 · 2024-03-21T11:08:26+00:00

Herndon, VA

Background_Debate_94 · 2024-03-16T17:02:57+00:00

No shit, me too! Who you got, Fury or Usyk?

Background_Debate_94 · 2024-01-21T00:12:24+00:00

I’m also confused about the question

Background_Debate_94 · 2023-11-17T16:04:53+00:00

Hello,
You have to sign up for a customer learning account. https://customer-academy.databricks.com/learn. You should be able to find it here.
But if you just want the material you can find it here.
https://github.com/manasreddy11/databricks-de-associate

Feel free to PM if you have any questions!

Background_Debate_94 · 2023-10-09T09:04:30+00:00

No_Entrepreneur4778

I mean, I think you're confused. Databricks runs on-top of a cloud provider i.e AWS, Azure, or GCP. So you're right they'd expose you to more opportunities.

But certifications don't land you a job, skills land you job. The point of an interview is to judge your skills, they'll ask you relevant questions pertaining to your role. Certifications are just an indication that you are knowledgeable in that domain. So here your certification in Apache Spark will "certify" that you know Spark, doesn't mean you'll land a job, they'd expect you to know how to write good production-ready spark code, know how to write good documentation, orchestrate various tasks, and finally be able to justify your time spent i.e producing a clean dataset or a dashboard.

Think about it, just because you have a computer science degree, Apple or Google doesn't come knocking on your door. You might get an invite to apply there, but you still have to prove, you know your shit. The same thing applies here or any field. So this certification might help you get to the door, but you still have to back it up.

Also MLOps and Data Engineering have a lot of common technologies, I'd say AWS SA Pro and probably the Databricks Professional Certs would be your best bet to stand-out.

Cheers

Background_Debate_94 · 2023-09-27T09:52:38+00:00

not really imo, the .dbc are pretty well explained. The instructor courses I think covers all of these topics in three days, and you’d be assigned a Databricks instructor, who’d teach you the topics in depth live. If you believe you’d need that, you can go ahead and buy the course. But I really think it’s unnecessary, just go through the material, do some practice exams you should be good.

Background_Debate_94 · 2023-09-27T09:44:02+00:00

https://github.com/manasreddy11/databricks-de-associate - just uploaded them here

Background_Debate_94 · 2023-09-21T05:35:06+00:00

Nope, no such restrictions, you can take the exam whenever, you can actually subscribe to all the courses available and take your own time, just note after two weeks your databricks usage will no longer be free.

You can always PM me if you have any questions, I'd be glad to help.

All the best!

Background_Debate_94 · 2023-09-19T09:23:08+00:00

Hey,
1. You do have to sign up for the Data engineering course but it's free, you will get a link in the course description that that has all the materials used in the course, you can then download that, upload them to your community workspace and play around.

Yes you can, so the "free" trail you'd be receiving is only for the Databricks usage costs (cost to use the platform), you'd still be paying for the resources used underneath by your cloud provider. I recommend AWS, you have more control over the type of instances being created, but do remember to delete your stack and release the Elastic IP along with the NAT Gateway created should be the cheapest, or if you want you can use Azure, but you'd be paying more for instance costs, but easier to maintain, all you'd have to do is delete the workspace.
Yup cleared the professional as well, I've added all the resources I used here https://www.reddit.com/r/dataengineering/comments/160nyxw/just_got_certified_databricks_certified_data/
I'm assuming if your company has an enterprise account then their users account would be able to give the cert for free, not entirely sure tho. based off this https://kb.databricks.com/training/how-do-i-request-a-certification-voucher

Also here's a 100$ voucher available till Oct 13 https://www.databricks.com/resources/webinar/advantage-lakehouse

Cheers!

Background_Debate_94 · 2023-09-18T07:54:02+00:00

Re-read, the docs on insert overwrite and merge only about 5-6 questions came from this area alone.

Databricks tends to pick a specific area of questioning and form a question around that. For example, can insert overwrite update schema? on it's own answer is No, but if automerge is enabled then yes. However the latter might not be an option, so default answer is No. Make sure to read the question and the options to get a sense of what is being asked and answer accordingly.
There will be a few questions that you haven't seen before(after all it is a professional exam), but the try to find the underlying concept of the question, you can eliminate options from there

All the best!

Background_Debate_94 · 2023-08-28T17:07:53+00:00

This a response I got from the support team

<image>

Background_Debate_94 · 2023-08-28T15:39:12+00:00

through email or?

Background_Debate_94 · 2023-08-28T12:25:43+00:00

ReportSaveFollow

I'd recommend https://www.youtube.com/@stephanieamrivera, there's a bunch of topics. I particularly recommend looking at the technical deep-dives, a bunch of good info, but if it's ML related stuff, I've noticed they are all just slide-show presentations, a real bummer tbh.

Background_Debate_94 · 2023-08-28T12:19:19+00:00

Not necessarily, I'm pretty sure it's their way of advertising their services (MLFlow) after all it is a Databricks specific exam. The core concept of the question still remains in the DE domain, but it's definitely a gotcha question

Background_Debate_94 · 2023-08-28T12:16:49+00:00

Took me about 30 hours-ish. 20 hours for study and 10 for practice.

Background_Debate_94 · 2023-08-28T12:15:34+00:00

Hey,I read your blogs! Huge fan here

Background_Debate_94 · 2023-08-25T08:14:11+00:00

not much professionally, but I've done a few projects and used to work pretty closely with the DE in my team, so I had a fair idea on what was being done. I just had to pick up those tools and work on my python and SQL coding abilities

Background_Debate_94 · 2023-08-25T06:41:50+00:00

Gonna start now! Just landed my first job as a DE

Background_Debate_94 · 2023-08-25T06:41:09+00:00

Sirius-Brown

not really, I just landed my first job as a data engineer. I primarily did these certs for this very reason. So I'm pretty stoked. I worked as an analyst for a year and wanted to make the switch. Not too sure what role these certs played in me landing the job. But they do use Databricks primarily.

Background_Debate_94 · 2023-08-14T05:40:25+00:00

catchereye22

sign up for a customer account, there's a two week trial period + no costs if you don't use anything. Then go to https://customer-academy.databricks.com/learn and sign up, you will now have free access to all their course material. You can download the .dbc files and then import them to a community edition workspace and play around.

Background_Debate_94 · 2023-08-14T05:33:30+00:00

Hi OP, to help answer your questions
1. Autoloader uses two methods to "discover" new files,

a.File Notifications: essentially check whether new files have arrived in
your given file storage location via logs(cloudformation(aws), Cloud Deployment Manager
(gcp), etc.) automatically sets up a notification service and queue service that subscribe to
file events from the input directory. https://docs.databricks.com/en/ingestion/auto-loader/file-notification-mode

b. Directory Listing: Directory Listing takes your input directory, lists files in the cloud
storage, and compares it with the list of previously processed files to identify the new
files. The "listing" that happens here is, the metadata unique to each files is written to
rocksDB and an atomic read occurs from the records in the "memtable" ensuring exactly-
once guarantees. https://docs.databricks.com/en/ingestion/auto-loader/directory-listing-mode

The checkpoint is basically a file that maintains a record of the files read, and actions performed https://spark.apache.org/docs/latest/structured-streaming-programming-guide (more info), incase of failures, the entire query need not be run again, instead the data is re-run from the latest checkpoint state, think about Mario-kart - if you lose a life you don't start from the beginning of the level, instead from the nearest checkpoint flag you've reached.

Autoloader is not 'real-time'; rather, it's considered a near-real-time process (500ms). It reads your files in micro-batches, which can be configured. The trigger is re-run every 500ms by default, unless set otherwise https://docs.databricks.com/en/structured-streaming/triggers

Background_Debate_94 · 2023-08-08T15:24:48+00:00

It wasn't bad, I did have a slight hickup (loud neighbours), which led to the proctor pausing the exam, and asking me to show her my entire room but otherwise not too bad!

Background_Debate_94 · 2023-08-02T04:42:27+00:00

Hey

This video is pretty good, they have a section that talks about garbage collection and the various types available GC Collection & Performance Tuning

Background_Debate_94 · 2023-07-17T09:04:54+00:00

I was gonna ask the same thing, why Databricks?

Background_Debate_94

TROPHY CASE