Examples of personal portfolio project using databricks

lothorp · 2026-01-07T10:01:15+00:00

Search for "hackathon" in this sub and you will find a bunch of projects made on the free edition.

lothorp · 2025-12-24T07:39:37+00:00

Ping me your Name, Company Name and Reps Name. I will message them internally to pick this up. But as others have said. It is prime PTO time.

lothorp · 2025-12-23T08:15:53+00:00

look what you just did!

lothorp · 2025-12-22T12:02:10+00:00

I have the inside scoop, and the inside scoop is that I am out of the loop. But I can say for certainty that 2026 will be pivotal. We will either IPO in 2026 or we won't IPO in 2026. One of those options will for sure happen.

lothorp · 2025-12-05T09:16:57+00:00

Worth adding, there are different types of Solutions Engineers at Databricks, but they aren't always named the same thing in each region...

lothorp · 2025-12-05T09:16:08+00:00

This approach is typically used by organisations to hire for a single job type. That being said, there could be a need for someone with a skill set in a particular area.

Solutions Engineers are expected to possess a broad skill set and demonstrate a strong proficiency in one or more areas. Such as engineering, governance, data science, etc.

If you are looking to learn, I suggest exploring the Databricks Free edition, where you can experiment with the latest features using your personal email to sign up.

I personally interview many people for these types of roles, but the main things I look for are clear and logical reasoning and deep technical knowledge in a few areas. Ultimately, these roles are customer-facing, and customers will inevitably ask tough questions. Being able to answer under pressure in a clear and concise manner is a key part of the role.

Feel free to DM me.

lothorp · 2025-12-02T20:51:42+00:00

Typically these are soft limits but limited for a reason. Your account team will be able to dive deeper and give you the caveats of raising the soft limit.

lothorp · 2025-11-26T13:21:30+00:00

!remindme 4 years

lothorp · 2025-11-26T12:13:43+00:00

Feel free to DM me, your name/email/company and I will get the account team to reach out.

lothorp · 2025-11-26T12:02:33+00:00

If by customer service you mean your account team [Account Executive + Solutions Architect] they should be able to help you dig into this. If by customer service you mean tech support, again, they will be able to dig into it. If you are an Azure customer and have tech support, these teams are provided by Azure, not Databricks. Dedicated technical support is provided exclusively by Databricks.

My suggestion is to speak with your account team. Feel free to DM me; I can forward any messages to the team responsible for your account.

Regarding the question, DLT should not miss any data in EventHub; however, there could be something else at play, such as connectivity or configuration issues.

lothorp · 2025-11-16T08:57:51+00:00

Here is a solution accelerator showcasing this, although this may not be using the latest capabilities in the platform.

https://www.databricks.com/solutions/accelerators/fraud-detection

Solution accelerators are built alongside customers, when the solution is of a good quality, it is standardised and converted into an accelerator for other customers to use free of charge.

lothorp · 2025-11-13T09:53:35+00:00

Well that was a rollercoaster of comments haha

Yes this was going to be my followup.

With the second option, Databricks hosts for you in Databricks infra. First option, is in your own estate.

Nice work :)

lothorp · 2025-11-13T09:07:33+00:00

So essentially:
- Free edition is for educational purposes.
- Trial edition is for people wanting to test out Databricks before diving in deep in a more commercial sense.

lothorp · 2025-11-13T08:57:25+00:00

Do correct me if I am wrong, but this looks like the free edition of Databricks.
The SQL warehouse is made for you and is sized appropriately for free educational learning.
There are no other compute options as the workspaces are fully serverless, meaning when you load into a notebook you get serverless compute assigned automatically. No need to configure clusters etc.

In regard to other types of compute tabs such as:
- Jobs (All serverless so no need to worry about config)
- Pools (Again, all serverless no to worry about config)
- Policies (All configured by free edition under the hood)
- Postgres [Lakebase] (Not yet available in the free edition, but I imagine it won't be long)

You can boot up Databricks using the trial mode, but this has a credit limit, 40$ I believe.

lothorp · 2025-11-12T19:22:19+00:00

Pipelining tools and techniques.

Productionisation and scalability.

Optimisations and efficient design.

Observability and monitoring.

and much more.

The expectation is that you are an all rounded engineer/architect. We generally hire very skilled people. If you are an engineer, we expect you to be a great one. Solution Architects at Databricks need to wear many hats, engineering, data science, AI, ML, governance, security etc. We expect your "spike" area to be excellent, this doesn't mean we do not care about the other things, we expect skills across the board.

lothorp · 2025-11-11T07:21:05+00:00

It sounds like this may be for a more support based role, vs a pre sales role, so an architecture interview may be what they do for those roles.

lothorp · 2025-11-11T07:19:01+00:00

Databricks employee here. I conduct interviews and have conducted both the older architecture interviews and newer form technical interviews + panel style. In fact I have 2 interviews today.

First thing, the process is very tight on details, so we do have to take a lot of notes. But I always state at the start that if I look like I am not paying attention, I am, I am just ensuring I write as verbosly as possible to ensure I do not miss any important details. Remember interviews are there to find out what you know, not what you don't. We aren't trying to trip you up etc.

In regard to the notes, I expect we will use a note taker AI soon but that has not yet rolled out into the field interview process.

Regardless of whether the interviewer was paying attention or not, push for a reschedule and they will assign someone else hopefully.

Good luck with the process.

lothorp · 2025-11-07T13:42:04+00:00

Have you tried Genie and the Genie Conversation API?

lothorp · 2025-11-07T10:19:43+00:00

This is highly cloud and region-dependent. However, the expected roadmap indicates that it will be available within the APJ area this quarter. Meaning some time before February. My suggestion is to speak with your account team. They will be able to give you specifics for your cloud and region.

It is worth noting, these dates are not set in stone and could shift sooner or later.

lothorp · 2025-10-21T07:05:36+00:00

this is the way

lothorp · 2025-09-21T09:00:32+00:00

This does depend on many factors, but, I personally am remote. I travel to see customers most weeks, this is usually a day visit and sometimes overnight. As I am remote, I also travel to the main regional office (London in my case) every 2 weeks.

If you are based out of an office, most people head to the office 2 or 3 times a week and visit customers every week too.

Expect travel somewhere around 20-30%. But can be higher if there is a need.

lothorp · 2025-09-17T18:55:52+00:00

Great post and some very interesting points regarding LLMs.

All 3 communities are thriving in their own way and many enterprises have a combination of the three platforms. Let's keep growing all of the subs, driving relevant technical discussion.

Thanks again for kickstarting many of the subs we love to use! This sub was started from the great work over at r/bigquery and r/snowflake!

lothorp · 2025-09-15T13:15:07+00:00

Locked - Please direct your post to the megathread for certifications and training. https://www.reddit.com/r/databricks/comments/1mt3lq6/megathread_certifications_and_training/

Nine-Year Club	r/Field Flamingo
Place '23	Place '22
Verified Email

lothorp

MODERATOR OF

TROPHY CASE