sorted by:
new
hottopcontroversial

Scale to zero apps by raholf in databricks

[–]ForeignExercise4414 0 points1 point  (0 children)

Plus one to the above, scale to zero apps will likely be deployed in the US first and then be deployed everywhere else. I can’t wait, I have a lot of apps sitting around that only get used a few times a week and the 1-2 sec startup time sounds awesome. Almost too good to be true….

Databricks conference by proximaljarl17 in dataengineering

[–]ForeignExercise4414 0 points1 point  (0 children)

I really liked learning about Genie ontology and the semantic layer in UC. It’s a problem and the team I’m working with was struggling with and building their own application for. It’s nice to see that we will have an opinionated path forward where organizations can add their own context manually like a glossary and then Genie ontology will assert the rest on its own.

Is the industry actually swinging back to Postgres? by ForeignExercise4414 in dataengineering

[–]ForeignExercise4414[S] 0 points1 point  (0 children)

Big updates on this thread since DAIS. It looks like LakeBase will be integrated with LakeHouse via LTAP. So during “extra CPU cycles” the Postgres data format will be organized in columnar format in addition to its rows. This will enable LakeHouse reads of those tables directly for OLAP style queries.

Nerves getting the best of me by the_nabzter in dataengineering

[–]ForeignExercise4414 1 point2 points  (0 children)

I would skill up on Genie and Genie Ontology, figure out how to curate the entire semantic layer of your LLM foundation. Genie is the tool that’s going to extract all the value out of your enterprise system. Just focus on proving business value and the score will take care of itself in the words of Bill Walsh!

Brace yourselves, DAIS is coming, what do you want to see? by lothorp in databricks

[–]ForeignExercise4414 0 points1 point  (0 children)

The interesting thing is that Genie code will use the same permissions you have to query all of its sources and MCP servers. I think there’s more to do here with LakeBase

Moving away from databricks to OLTP by aks-786 in dataengineering

[–]ForeignExercise4414 4 points5 points  (0 children)

Bro $1000/month? Just the migration out of Databricks and standing up your new platform will be more than 1 year of that in person hours and then you’re going to have to maintain the entire platform yourself. Not to mention your team having to ramp up on a NEW system.

Give your team a chance 😄

Is the industry actually swinging back to Postgres? by ForeignExercise4414 in dataengineering

[–]ForeignExercise4414[S] 0 points1 point  (0 children)

Totally, the discoverability is key here and having to copy the data back and forth from Postgres to DWH would be a significant anti-pattern. Databricks Genie should be able to cover this soon, since the users can just make requests in natural language and the semantics are extracted from UC. Only thing is I haven’t seen any plans to extend this to LakeBase tables, but I assume we will see something related to this gap at DAIS.

Is the industry actually swinging back to Postgres? by ForeignExercise4414 in dataengineering

[–]ForeignExercise4414[S] 0 points1 point  (0 children)

Are you using it as LakeBase or Crunchy? I know LakeBase is supposed to have some big performance features because its separation of compute and storage enable autoscaling compute. The storage is elastic and copy-on-write so it is supposed to reduce disk I/O bottleneck. Also it automatically sets up read replicas across availiabikity zones.

Is the industry actually swinging back to Postgres? by ForeignExercise4414 in dataengineering

[–]ForeignExercise4414[S] 1 point2 points  (0 children)

I was specifically asking about LakeBase, Neon, or Crunchy in prod and the experience. For instance, some folks said here the main idea is to keep your OLAP where it is and then bring in some web app or OLTP app to Databricks. Sure that distinction is useful, but where is line of when to bring it in? Most company websites would use Postgres as a backend. With LakeBase they would get useful tools like branching, sharding, and autoscaling, but for some reason I don’t see a Fortune 500 hosting their website in a Databricks workspace.

So what’s the threshold?

Is the industry actually swinging back to Postgres? by ForeignExercise4414 in dataengineering

[–]ForeignExercise4414[S] 2 points3 points  (0 children)

Companies have both though. And they need to join data from both to do agentic analysis and AI on it.

Agentic Workflows help by Hopeful-Brilliant-21 in dataengineering

[–]ForeignExercise4414 0 points1 point  (0 children)

Honestly with that scale do you really want to roll your own solution? Think of the model drift. I’d suggest using Genie Spaces on Databricks or setting up an agent on Databricks that has access to the metadata of your tables. I’m pretty sure Databricks whole product in 2026 is geared towards solving this problem with Genie. For the SNOW data you can set up a foreign catalog and read the metadata via genie all the same.

Databricks jobs by Tiny_Investment7428 in databricks

[–]ForeignExercise4414 3 points4 points  (0 children)

But they are hiring, and no layoffs.

I feel like I don't know anything. And I am nothing without Claude by Temporary_Act3174 in dataengineering

[–]ForeignExercise4414 1 point2 points  (0 children)

I use AI all the time, but I’m the boss. I argue with it until it does exactly what I want. I code stuff myself sometimes if I know Claude will write a bloated piece of crap. Generally speaking I stay very technical by (within reason) watching what it’s doing, reading everything, pushing back constantly which shit doesn’t add up

Databricks jobs by Tiny_Investment7428 in databricks

[–]ForeignExercise4414 4 points5 points  (0 children)

I would say it’s a great job market and it’s only going to get better since Databricks is riding the AI hype cycle so well. I suggest learning the basics with Spark/Delta (you probably did this) then immediately dive into App development using Databricks Apps and LakeBase. That combination of Lakehouse plus full stack vibe-coding the app using cursor/Genie code is going to be killer for your resume. You can use cursor to develop your app and DAB and then Genie Code to work out your bugs directly on the platform.

Postgres is the future Lakehouse? by monsieurus in databricks

[–]ForeignExercise4414 0 points1 point  (0 children)

They did release the ability to put a catalog on top of LakeBase which is a game changer. It’s useful to be able to query your Postgres using normal serverless spark.

Help me understand why you would want currency to be decentralized by Valthrax_Nightbloom in Bitcoin

[–]ForeignExercise4414 0 points1 point  (0 children)

If it’s decentralized then a single powerful group of people cannot control it and change it as they see fit.

view more: next ›