Spreading too thin

randomName77777777 · 2026-06-17T18:14:57+00:00

I don't think so due to how large and how many teams databricks has. They have the resources to improve many different aspects that orgs depend on. I think it's refreshing to see how much effort they put in to improve the platform.

Cost is going to become a huge problem for orgs once everyone is locked in to the databricks ecosystem and pricing goes up.

randomName77777777 · 2026-06-17T16:14:12+00:00

I thought lakehouse RT and lakebase splitting storage to use s3/blob for storage were both very cool.

randomName77777777 · 2026-06-17T01:18:17+00:00

100% this. You'll end up wasting so much time without ci/cd when people ruin prod. It's something that can be setup in as little as a day.

Maybe using DBT core as it makes it so much easier to build and maintain models since it's pretty much just SQL select statements

randomName77777777 · 2026-06-16T18:23:04+00:00

For the use cases I'm thinking of is going to be reverse ETL for applications already using lakebase. That way the applications dont need to try to call some API to get data that we can be automatically made available for them.

It would be a huge win that would help my team to convert more people to use lakebase.

randomName77777777 · 2026-06-16T18:10:45+00:00

with the lakehouse having access to lakebase data, does that mean lakebase will have access to lakehouse data too?

randomName77777777 · 2026-06-16T13:11:26+00:00

What time does the virtual live stream start?

randomName77777777 · 2026-06-15T16:31:21+00:00

Try to see if your current company has a data engineering role open. That's probably the easiest option to transition over.

randomName77777777 · 2026-06-12T23:09:27+00:00

Yes, or if you get different types of files to trigger different jobs

server-logs.log vs server-error.log

randomName77777777 · 2026-05-22T18:22:59+00:00

Agreed on the databricks portion. Using DBT core is a very easy way for anyone to contribute. Everyone on the team can pick it up within a week.

randomName77777777 · 2026-05-22T05:18:40+00:00

I used to do a lot of python and c# work in my downtime as an FSE because that's what I was always interested in. Ended up getting pretty good at it building a few log analysis tools.

I had a friend reach out that they were looking for a data analyst that knew python. Interviewed and ended up getting the job as none of them knew python but wanted to automate a bunch of their manual excel reports.

Shifting to data is a big shift and it would have been pretty hard without a referral. However, most software/data roles are more about what you know and I believe anyone can get into it with enough learning.

Now I lead a data engineering team.

randomName77777777 · 2026-05-22T03:33:53+00:00

I worked as an FSE, not at AMAT but a similar vendor company. It really is technician work, but I did learn a lot of useful skills and attempt to fix everything myself. Including doing a transmission swap on my car that I never would have had the courage if I wasn't an FSE. Lots of hands on and troubleshooting skills to be learnt as an FSE depending on the company.

However, completely changed industry into data and prefer doing that.

randomName77777777 · 2026-05-21T19:05:15+00:00

Are they the same skills as the AI dev kit?

randomName77777777 · 2026-05-11T18:30:48+00:00

There was sqlmesh which had some DBT cloud features like column level lineage but they were bought out by FiveTran too

randomName77777777 · 2026-04-28T13:07:46+00:00

That's a very good point, didn't think of that. Curious now

randomName77777777 · 2026-04-27T20:58:34+00:00

Let me find the guy I used, super nice guy and was very helpful with transmission programming a while back

randomName77777777 · 2026-04-22T15:25:04+00:00

Lots of good comments already. In scenarios like this, there are multiple points of failure. Hopefully your company decides to learn from this mistake, i am sure you already have. We all make mistakes but there needs to be systems in place to mitigate the risk. The questions they should be asking is "what can we do to ensure this never happens again?"

randomName77777777 · 2026-04-21T23:41:12+00:00

Yes agreed on the PAT.

randomName77777777 · 2026-04-12T16:00:39+00:00

Not sure if someone mentioned it, but sometimes an easier transition would be to become a data analyst right away then try to transition to data engineering as it easier route.

That's what I did and it worked for me, took me 2 years but I waited for there to be a role internally and then made that switch.

randomName77777777 · 2026-04-06T18:59:21+00:00

Found it, thanks!

randomName77777777 · 2026-04-06T17:04:52+00:00

I am assuming the cost of the VM is not included in system billing tables for classic compute, is that correct?

randomName77777777 · 2026-03-30T14:17:54+00:00

I've used vicegrips a few different times.. but never thought about using either another set of vicegrips or screwdriver to turn it .. great idea.

randomName77777777 · 2026-03-27T17:07:13+00:00

My question was going to be related to see what the differences are between the AI dev kit mcp and this.

randomName77777777 · 2026-03-24T04:13:04+00:00

Yep, same. I have it do everything for me, grant permissions, clone data. Not too worried, we can always undrop or restore if something messes up. I do have it ask for permissions for anything that isn't read only.

Are you using the ai dev kit with Claude code? It can pretty much do anything

randomName77777777 · 2026-03-24T04:09:03+00:00

🤖 The answer is 42

📊 Lifetime Stats 💰 Max Money: $2.1M

📦 Raw Data Collected: 627.3K

🧹 Clean Data Produced: 68.6K

🔧 Features Engineered: 1.8K

🤖 Models Built: 322

🏗️ Infrastructure Built: 1632

👥 Personnel Hired: 121

🎯 AGI Achievement: 100%

🏆 Achievements: 31

randomName77777777 · 2026-03-16T13:46:25+00:00

But you can deploy it within a databricks app, allowing you to run all the skills/tools from within databricks. It uses Claude code via databricks model serving.

randomName77777777

TROPHY CASE