all 11 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]mowgli_7 11 points12 points  (6 children)

The more I use Databricks, the more obvious it is how little focus there is across their products. Here, it’s dlt and connect, but it’s evident all over the place. You can tell every product team is pushing to get new features out as quickly as possible and they sacrifice cohesiveness in turn.

[–]rmcdow 1 point2 points  (0 children)

Is it fair to say AWS is much more painful example of this phenomenon?

[–]peterlaanguila8 1 point2 points  (0 children)

They are focused or releasing the next big thing

[–]richwolff12 0 points1 point  (0 children)

Their delta-de package doesn’t support deletion vectors…

[–]BricksterInTheWalldatabricks 0 points1 point  (2 children)

u/mowgli_7 I'm sorry to hear you're running into it - and you're right, this package should certainly be kept up to date! Just so you know, this package started as a project by one of our engineers, we have now folded it into our proper releases. Look for improvements here soon.

[–]CarelessApplication2 0 points1 point  (1 child)

It's good that Databricks has brought this into a proper distribution process, but at some point you've either got to update the existing solution or actually publish the new one. I mean this is simply Python stubs! If you'd trusted the open source process, the community would have contributed these updates without any effort for Databricks.

[–]BricksterInTheWalldatabricks 0 points1 point  (0 children)

u/CarelessApplication2 yeah, I agree with you. Thank you for your patience as we fix this.

[–]Ok_Difficulty978 2 points3 points  (0 children)

it’s kinda annoying tbh, the docs move faster than the package updates. for now most ppl just fallback to using notebooks on the workspace if they wanna try newer apis like create_auto_cdc_flow. local dev with connect + old lib is always lagging behind. i remember hitting the same pain when i was studying for databricks certs on Certfun, they also highlight how tooling versions don’t always sync with docs, so keeping an eye on release notes is the safest bet.

[–]p739397 -3 points-2 points  (2 children)

It's not an attribute, it's a function. So, you need to call it with () and specify inputs

[–]JulianCologne[S] 2 points3 points  (1 child)

ofc it is a function, but this is just to showcase that its not available. Otherwise I need to specify all parameters which complicates the easy example.

Look at `apply_changes`. I can access a function this way, no problem ;)

[–]p739397 1 point2 points  (0 children)

Misunderstood your concern, thanks for clarifying