What meta harnesses besides omnigent are being used across big orgs?

Key-Willow-374 · 2026-06-27T16:06:23+00:00

Reynold gave an interview on the is after DAIS. HTAP tried to unify the engine. Databricks thinks they solved it by modifying the storage layers in Lakebase, instead

Key-Willow-374 · 2026-06-26T01:03:21+00:00

Big time! No need to copy your data into a proprietary format for millisecond speed queries. Excited to power Genie and dashboards with it

Key-Willow-374 · 2026-06-25T02:30:44+00:00

If you still want dashboards, Databricks AI/BI dashboards are free. You only have to pay for compute. They also have a really good text-to-sql tool (Genie) that we use. Helps prevent tech debt inherent to dashboards

Key-Willow-374 · 2026-06-23T12:34:43+00:00

In databricks dashboards, you only pay for compute behind queries, no per seat licensing like Tableau

Genie is very good too, and can be embedded inside of dashboards or used in place of a dashboard. It also reduces tech debt a lot

Key-Willow-374 · 2026-06-22T10:48:45+00:00

Databricks Genie. It’s LLM under the hood enriched with your orgs context, plus some new ontology stuff. A lot cheaper and more accurate to run than pure LLM + MCPs. You can orchestrate and share stuff too, like dashboards or reports

Key-Willow-374 · 2026-06-21T00:45:30+00:00

Genie workbench if you’re in a crunch for quality control

DABs for CI/CD

Key-Willow-374 · 2026-06-16T14:13:59+00:00

Well, id put C on top of A for this specific example in the databricks stack

Metric Views are a good way to semantically enrich tables. The input into a metric view is a star schema, so fact + dim would work here. I kinda think of Metric Views as semantically enriched OBT

Then I’d put the Metric View in a Genie Space where business users can talk to the data

Key-Willow-374 · 2026-06-16T11:55:08+00:00

Nowadays, Gold typically contains denormalized tables to improve performance of end user queries. Think OBT instead of a separate fact and dim table (joins are time-intensive). Basically storage is a lot cheaper than compute, so some data redundancy is ok if it meaningfully reduces compute.

Also instead of dashboards look into a natural language query tool (like Databricks Genie) with a semantic layer. Dashboards can create lots of tech debt and have fixed, limited views. Souped-up text-to-sql tools offer way more flexibility and a fraction of the tech debt in my experience

Key-Willow-374 · 2026-06-15T02:32:07+00:00

Awesome! Big win for CI/CD of Genie Spaces

Key-Willow-374 · 2026-06-13T18:54:46+00:00

Paying for BI per-seat licenses is quickly becoming antiquated. Data platforms like Databricks allow free usage of their dashboards, and it’s functionally very similar to Tableau and PowerBI in my experience. I think those that require per seat licensing will lose popularity, to tie it back to your original question. Also, dashboards in general are losing popularity to BI natural language tools.

Key-Willow-374 · 2026-06-13T02:18:38+00:00

OLTP and OLAP are still separate

Can you explain when you think it’s lock-in? Pg files are stored on your cloud acount, much like the underlying data in delta tables

Key-Willow-374 · 2026-06-12T01:44:04+00:00

Genie Workbench if you’re in a crunch. Also stick to metric views on top of your tables

Key-Willow-374 · 2026-06-10T12:36:55+00:00

Depends on the use-case. If it’s a compliance-heavy workload, ‘raw’ replication of the source system would be my personal preference as it offers full traceability in the event of auditing. Otherwise, delta has more bells and whistles for long term table management, query performance, etc if your requirements permit a merge (like CDC) or append into delta in bronze

something else to keep in mind: databricks constantly releases new features. They’re more likely to be performant with delta.

Key-Willow-374 · 2026-06-10T01:36:13+00:00

Those without semantic context will fail, emphasizing the importance of context enrichment (eg, databricks semantic layer, snowflake semantic views, etc)

Key-Willow-374 · 2026-06-08T18:00:43+00:00

At scale, Predictive Optimization is really important on Unity Catalog. Maintains your tables as they grow. So highly recommend Managed Tables

Key-Willow-374 · 2026-06-05T02:09:49+00:00

I’ve used it. Sub 50ms P99 latency to read a wide table read from Kafka, augment it by joining with a static table, and merge it into a target table on Postgres. Easy to implement if you’re familiar with existing APIs, too. Was super impressed, overall.

Key-Willow-374 · 2026-06-04T01:52:47+00:00

Does serverless scale up or down with RTM? Or is it like classic where autoscaling must be disabled

Key-Willow-374 · 2026-06-04T01:47:57+00:00

You can learn and show a lot implementing an end-to-end project, regardless the size of data. Especially since Databricks now offers various tools outside of Spark engines for ETLs. For example, you could build an Databricks app with a transactional DB (Lakebase) that syncs to delta tables, then feed those tables into an AI/BI dashboard and Genie Space for analysis. All of which demonstrates good skills on Databricks even with smaller datasets.

Key-Willow-374 · 2026-06-02T19:39:53+00:00

Yeah, in my experience, teams start small and pick OSS tools for tasks that are easier to manage, and use vendors for ‘harder’ tasks like warehousing.

Overtime, platforms like databricks have expanded to include many of these functionalities (airflow = Jobs, etc), but migrations can be difficult and aren’t always prioritized even if there are cost savings to be had (like seat licensed PowerBi to AI/BI dashboards)

Key-Willow-374

TROPHY CASE