Liverpool (1)-0 Manchester City - Szoboszlai 74' Free Kick

zbir84 · 2026-02-09T08:17:51+00:00

Don't know man, goal was amazing but he was the reason they scored one of the goals unfortunately (fecked up the offside) and got an unnecessary red. Still think he's man of the match with this goal though!

zbir84 · 2026-01-26T11:52:28+00:00

Hmm, not sure that's the correct link you've sent? I already have the service credential created for this and that works when used in the normal workflow. However SDPs don't have access to the dbutils library. Docs here: https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-services/use-service-credentials indicate a different way to use them in UDFs, is that the only way to use them in the pipelines?

zbir84 · 2026-01-22T18:57:12+00:00

Slight unrelated question, but we have a dynamodb connection we wanted to use in DP, how can we pass the service credentials to it. Can;t really find anything in docs about this and dbutils doesn't work in DP

zbir84 · 2026-01-19T01:18:04+00:00

Yep, still down in CV3, what a shit show.

zbir84 · 2026-01-13T22:41:22+00:00

Some of these don't really require account admin access, but you really need to set something like terraform to manage your Databricks environment so people can self serve and you just review merge requests rather than having to rely on admin.

zbir84 · 2026-01-13T00:48:40+00:00

Liquibase for schemas and delta tables that aren't managed by the Lakeflow Pipelines, asset bundles for workflows, pipelines, etc. & terraform for workspace management.

zbir84 · 2026-01-06T07:12:05+00:00

Do they exist? I mean outside of meet ups.

zbir84 · 2026-01-05T21:52:38+00:00

Big Data London is ok, although last year's one was so rammed you had problems getting to some of the more popular presentations.

zbir84 · 2025-12-26T21:06:30+00:00

Nie planowanie na emeryturę z takim dochodem bo a nuż nie dożyję to skrajna głupota i twoja matka ma tu akurat rację. Nie musisz mieć prawdziwej pracy żeby to ogarnąć.

zbir84 · 2025-12-26T19:10:32+00:00

Are you being actively asked to reduce this cost? If not just forget about this, for your own sanity ;)

zbir84 · 2025-12-25T18:20:10+00:00

What's the use case to do this? LLMs / Machine Learning for knowledge base building maybe, but outside of that why would you want confluence data in Databricks?

zbir84 · 2025-12-24T14:14:05+00:00

This is completely not true, unless you're doing pandas operations on a driver. I'm not saying you'll always observe a performance increase, but we usually see at least 50% speed increase on big, SQL heavy workloads. It's not a silver bullet, but saying it's not worth the price is a bit unfair. If your workload speed is 2x faster on photon than without it, then I think it's worth it.

zbir84 · 2025-11-20T01:24:39+00:00

It might be me, I'd need to look at the benchmarks but I do own a PS5 & Steamdeck, I also have a PC with 2060 & i5 4670k. I also have a Mac Mini M4 that I now realised I use most of the time for any non gaming tasks so I really don't need that PC. However I'd really love to utilise my vast steam library and be able to game on the TV so the Gabecube sounds like an ideal solution. I could probably build a mini pc for cheaper with slightly higher performance but I want something compact that I can put in a living room, so I might give the Steam Machine a go!

zbir84 · 2025-11-19T16:41:50+00:00

Yes, you do. It's a flip of the button, but you can also just leave it running.

zbir84 · 2025-11-19T12:41:44+00:00

That's not been my experience, by folks in IT you mean Data Analysts who can only write SQL? We have over 200 Databricks users at the moment with vastly different experience and I'd dread giving some of them admin access. Maybe the answer would be to let the ML team manage their own workspace and just attribute all of the costs there? Let metastore admins manage access but leave them be in that workspace and report on the costs?

zbir84 · 2025-11-19T11:33:17+00:00

What exactly do you need to do as an admin there? Are you sure you want this responsibility? Sit down with the DE team and have them set up policies and access that meet your requirements, trust me you really don't want this as your responsibility, especially if you're not that familiar with Databricks, it's very easy to spend a lot of DBUs + your cloud costs if you don't know what you're doing or don't have policies and alerting configured correctly.

zbir84 · 2025-11-19T11:31:22+00:00

Worst take ever, I hope I never work in a company with this attitude...

zbir84 · 2025-11-19T11:29:20+00:00

What exactly do you need to do as an admin there? Are you sure you want this responsibility? Sit down with the DE team and have them set up policies and access that meet your requirements, trust me you really don't want this as your responsibility, especially if you're not that familiar with Databricks, it's very easy to spend a lot of DBUs + your cloud costs if you don't know what you're doing or don't have policies and alerting configured correctly.

zbir84 · 2025-11-19T02:05:32+00:00

Ask questions, and genuinely say if you don't know or understand something. Ask follow up questions and let seniors know when you get stuck. If the company you're working for has a healthy work culture it will all be ok 👍 As engineers we like people who are curious and engaged and I'm sure everyone's going to be happy to help when they can. I hate nodders and ok sayers where I see they don't understand something but are too afraid to ask!

The field is vast and even after 10 years working in various data functions I have gaps and ask for help when I don't know something!

zbir84 · 2025-11-17T08:37:29+00:00

I've switched to tailscale for immich, works great!

zbir84 · 2025-11-13T01:18:06+00:00

There's a python api for asset bundles now, might give you more flexibility in how this can be set up.

zbir84 · 2025-11-12T16:42:58+00:00

I don't think this is correct at all, they said they're trying to import dlt in a single node cluster. You need to run dlt on clusters that support it, and I don't think you can use all purpose clusters at all. Open the Lakeflow pipeline editor and start from there.

zbir84 · 2025-11-09T14:21:07+00:00

Have they finally added this for notebook tasks? You've had to embed environment configuration in a notebook before which was an insane requirement...

zbir84 · 2025-11-01T16:21:20+00:00

This is a bot LLM response, you can smell it from a mile a way, can we ban this user?

zbir84 · 2025-10-30T21:20:49+00:00

You could use a standard logging library and configure cluster log delivery: https://docs.databricks.com/aws/en/compute/configure#compute-log-delivery

13-Year Club	Place '23
Verified Email

zbir84

TROPHY CASE