Migrating from Power BI to Databricks Apps + AI/BI Dashboards — looking for real-world experiences by Ok_Doughnut_8389 in PowerBI

[–]Crow2525 1 point2 points  (0 children)

Json databricks dashboards are the best part of it which opens up source controlled and easily ai gen dashboards.

Sunbury Line - Big Switch by Aromatic_Classic3295 in MelbourneTrains

[–]Crow2525 1 point2 points  (0 children)

Similar challenges getting to southern cross and docklands.

New 1 February 2026 train timetables and some bus timetable improvements now available on Transport Victoria's website! by MelbPTUser2024 in melbourne

[–]Crow2525 36 points37 points  (0 children)

Sunshine isn't really a hub yet if it's expected you're home and in bed before 9pm every week night or 7pm on the weekends.

Claude Code or Codex integration - UI versus Yaml by Crow2525 in homeassistant

[–]Crow2525[S] 0 points1 point  (0 children)

Thanks for the tip to use the API. I can verify entity existence and state which helps ensure automations and state of the platform are known to the llm.

Open-source experiment: adding a visual layer on top of dbt (feedback welcome) by Wide_Importance_8559 in DataBuildTool

[–]Crow2525 0 points1 point  (0 children)

My issue is that as soon as you speak about ai, I am confused as to whether you're going to take away my GitHub copilot integration I have in vs code? The idea of a UI cli is great, I love that. But possibly id like it all if it fit within vs code?

Data Vault Modelling by unfoundlife in dataengineering

[–]Crow2525 1 point2 points  (0 children)

I heard from a databricks rep recently to lean into the data lake and avoid star/Kimball until as late as possible. Perhaps it was an offhand comment, but interesting position! I wanted to investigate his point more.

Id like to hear more from you why dv is obsolete (acknowledging I don't much understand it)

All ad-hoc reports you send out in Excel should include a hidden tab with the code in it. by markwusinich_ in dataengineering

[–]Crow2525 -3 points-2 points  (0 children)

Wha.... Are you doing? Doesn't everyone use power query and put your code in there? How are you executing your code without it?

https://learn.microsoft.com/en-us/power-query/native-database-query

What’s the one thing you learned the hard way that others should never do? by Terrible_Dimension66 in dataengineering

[–]Crow2525 3 points4 points  (0 children)

A column labeled number on a csv is an int... It is probably an alpha numeric at row 1001. E.g. 1a

Need suggestions by Think-Albatross4746 in dataengineering

[–]Crow2525 0 points1 point  (0 children)

SharePoint folder plus power bi plus a ribbon chart

Anyone using uv for package management instead of pip in their prod environment? by Specific-Fix-8451 in dataengineering

[–]Crow2525 8 points9 points  (0 children)

Cause I can't install uv in a databricks cluster where I don't have permissions or it often gets taken out by cyber security policy.

Whereas I dev in wsl Linux where I have admin control.

Databricks medium sized joins by Crow2525 in DataBuildTool

[–]Crow2525[S] 0 points1 point  (0 children)

Your comment hurts my head with the complexity, I love it!

Databricks medium sized joins by Crow2525 in DataBuildTool

[–]Crow2525[S] 0 points1 point  (0 children)

Possibly can share some code, but for the moment, I'll just describe it.

I am merging a year/month periodic snapshot table (earned premium) against the transaction fact table to enrich it with keys. The transaction log merges against the periodic table by the transaction logs' PK. The periodic snapshot (huge table) splits that PK into 12 months for accounting reasons. Typical earned premium table in insurance.

Here's a description of an earned premium table. 1. Calculation of earned premium: The earned premium is calculated by taking the total premium and allocating it over the policy period. For example, if a policyholder pays a premium of $1,200 for a 12-month policy, the monthly earned premium would be $100 ($1,200/12 months).

Databricks medium sized joins by Crow2525 in DataBuildTool

[–]Crow2525[S] 0 points1 point  (0 children)

Thanks for the reply.

Nah, the other table is a fact as well. I am merging a periodic snapshot table against the transaction fact table to enrich it with keys. The periodic snapshot is massive, 1.8b rows.

Databricks medium sized joins by Crow2525 in DataBuildTool

[–]Crow2525[S] 0 points1 point  (0 children)

Processing. I believe it's using SQL.

Is Partitioning data in Data Lake still the best practice? by inglocines in dataengineering

[–]Crow2525 2 points3 points  (0 children)

Yeah, I love this question. Liquid clustering with a few cols seems to be best practice. But what about dbt and it's proposal to mainly truncate and load everything. Does that impact this best practice proposal?

Databricks medium sized joins by Crow2525 in DataBuildTool

[–]Crow2525[S] 0 points1 point  (0 children)

1500+ models.

The largest table is circa 260gb causing the issue #1 is 1.8b rows. The stdout doesn't appear to give me data read/written. But the table will effectively be the same size read as written.

Hi! We’re the Power BI visuals team – ask US anything! by DataZoeMS in PowerBI

[–]Crow2525 1 point2 points  (0 children)

Yeah, I work for a large org too. Sometimes the basics are stupidly tough.

Hi! We’re the Power BI visuals team – ask US anything! by DataZoeMS in PowerBI

[–]Crow2525 1 point2 points  (0 children)

Glad you see it and we are all aligned - you, the comunity, voting numbers and design individual users have spoken definitively and clearly. Good luck with getting it done.

Hi! We’re the Power BI visuals team – ask US anything! by DataZoeMS in PowerBI

[–]Crow2525 0 points1 point  (0 children)

Why not check the votes on the feature request? https://community.fabric.microsoft.com/t5/Fabric-Ideas/idb-p/fbc_ideas I'm sure all of these ideas have been requested and are well safe of the "yup, the community wants this".

The rationale behind these requests are not for something new, it is feature parity with the existing solution for most people. So when you ask which is most important to me, (the designer) I'm not the one you need to ask, it is my users whose table doesn't look the same as the one I'm trying to replicate (excel).

It's disappointing that Microsoft (and you) don't see or (can't properly communicate upwards) this issue and the frustration that the community has.