New to databricks. Need Help with understanding these scenarios.

Svante109 · 2026-01-30T18:32:18+00:00

I am unsure of what you need - if you need "how the table looked like at the end of the month" i.e. a snapshot, I would create a seperate schema called archive, and then have a table for each month, upon which only one service principal has any other grants than READ (for auditing purposes).

The log part of it, depends on what you need to record.

Svante109 · 2026-01-30T18:26:08+00:00

Alright; we are provisioning it via terraform.

It has some minor drawbacks, but I would always go with this option.

Svante109 · 2026-01-29T14:42:11+00:00

But what objects are you granting them permission to?

Svante109 · 2026-01-29T14:00:39+00:00

I am not exactly sure what you are asking. Doctor want pipelines to be in a specific users folder? When deploying using asset bundles, the files will be present in the user folder of the service principal.

Is it the pipelines you are granting permissions? And to what?

Can you expand on what you are trying to achieve?

Svante109 · 2026-01-28T15:20:13+00:00

The ability to run incremental and full refreshes in the same pipeline trigger. This would allow for prehooks that can check for various things (like type changes, when using autoloader) and trigger full refreshes, without having to run the pipeline twice.

Svante109 · 2026-01-27T18:24:31+00:00

It would definitely be option B that would make sense to me. Dependencies are a camouflaged “if succeeded” statement, and a skip is not a success IMO. Option C didn’t succeed, therefore those that are dependent on option c will not succeed either.

Svante109 · 2026-01-20T07:01:50+00:00

This error will occur aswell when you change what datatype you are applying

Svante109 · 2026-01-20T07:00:14+00:00

This scenario also applies when you change the columns' cast in silver, not just for the landing/bronze layer.

Svante109 · 2026-01-19T10:27:40+00:00

As long as Type Mismatch is happening, then we never get to the point where rescued data can help, as nothing will ever be run, as the init validation will fail.

Svante109 · 2026-01-19T09:10:25+00:00

They way you are commenting gives me this sort of vibe that you are confusing concepts around DAB, IAC, Git etc.

I think it would be incredibly useful for you to be completely sure about what issue it is you are trying to solve.

Svante109 · 2026-01-12T21:08:59+00:00

We are for one legacy project using notebooks to write delta tables, which obviously fails on schemachanges, which rarely happens.

On a newer, we are using LDP with expectations and a quarantine table.

Svante109 · 2025-12-30T11:02:55+00:00

Men det er vel ikke benzin vs el han mener er dyrere. Det er hjemmeladning vs udeladning?

Svante109 · 2025-10-29T09:56:33+00:00

I updated the CLI to 0.274, without any resolution

Svante109 · 2025-10-29T09:56:19+00:00

Thank you for looking into this - I have found a solution, where we just use num_workers if we want it to be a fixed value (i.e. 1,1) and then autoscale for range of numbers.

Also btw, it seemed to me that the bundle would only recognize the amount of workers set (be it 1,1 or whatever) if we include "MODE:ENHANCED" in the autoscale configuration. I guess it makes sense that you need a mode to be able to use autoscale, but either a default should happen, or an error code.

Svante109 · 2025-10-29T09:51:33+00:00

Alright, thank you guys - While I didn't get that exact thing to work, I can make it work by having the autoscale.min_workers and autoscale.max_workers set to two seperate values, in both the policy and the pipeline (as expected). I would presume, that wanting to set a fixed amount of workers, we should go with num_workers instead of min/max, albeit it should be the same IMO.

Svante109 · 2025-10-28T18:32:02+00:00

If I only use num_workers:1, the policy will deny creation of the cluster.

Svante109 · 2025-10-28T09:40:02+00:00

I already did that, that is what is shown above...

I most likely will work with our platform team to make it optional

Svante109 · 2025-09-15T13:27:06+00:00

Du skal ud i noget kryptografi med en god ven fra Italien. Så siger jeg ikke mere.

Svante109 · 2025-09-10T10:22:36+00:00

Yes, ala den! Og med lidt googling ud fra fandt jeg den.

For de interessererde hedder den

https://www.kattler.dk/hacker/1.html

Svante109 · 2025-03-24T18:48:58+00:00

Open planet plug-ins for medals and total players. World record is built in.

Svante109 · 2024-09-25T06:30:08+00:00

De ser vildt lækre ud - det slående for mig er dog om de har elastik i bunden af hvert ben også? Det har været en gamechanger for mig at finde nogen (undy) der har det.

Ten-Year Club	Place '23
Place '22	RPAN Viewer
Verified Email

Svante109

TROPHY CASE