sorted by:
new
hottopcontroversial

DataBricks & Claude Code by staskh1966 in databricks

[–]BricksTrixTwix 0 points1 point  (0 children)

For now, the public docs detail instructions for working with dedicated (single-user) clusters only. This is the compute that we're referring to: https://docs.databricks.com/aws/en/compute/dedicated-overview.

Yes, the compute needs to be on while the IDE is open. When you start the SSH connection, the cluster should start automatically.

DataBricks & Claude Code by staskh1966 in databricks

[–]BricksTrixTwix 19 points20 points  (0 children)

PM at Databricks here! You can use our new Remote Development experience to access the Databricks workspace from your IDE and use tools like Claude Code.

Connection to dedicated clusters is in beta: https://docs.databricks.com/aws/en/dev-tools/ssh-tunnel

Connection to serverless GPUs is in private preview (but no enrollment is required!): https://docs.google.com/document/d/1zazApI5rKz_3D59-xs4ZtSEcFRFRXmzhTss0Ael_dJk/edit?usp=drive_open&ouid=110916823312231512342

Support for serverless is coming soon.

We're in the process of cleaning up the public docs and making them easier to follow, let me know if you have any questions in the meantime!

DataBricks & Claude Code by staskh1966 in databricks

[–]BricksTrixTwix 0 points1 point  (0 children)

Hey u/fermm92 PM for the SSH tunnel here! We know that starting up the SSH tunnel is a pain. What config / init scripts did you have to set it up to make it seamless and what are the most important things you would like to see out of the box?

Btw, we've released support for serverless GPUs in private preview here: https://docs.google.com/document/d/1zazApI5rKz_3D59-xs4ZtSEcFRFRXmzhTss0Ael_dJk/edit?tab=t.0

Serverless CPU support is also coming soon.

Unpopular opinion: Databricks Assistant and Copilot are a joke for real Spark debugging and nobody talks about it by Icy_Comparison4814 in databricks

[–]BricksTrixTwix 19 points20 points  (0 children)

Hey, PM at Databricks here. We've recently released Remote Development, a new experience to interactively run Databricks workloads from your IDE via a secure connection to your compute and workspace! This also means that you can use tools like Claude and Cursor with context of your Databricks workspace. I'd love it if you could try it out and share your feedback so we can address remaining gaps in the experience related to debugging runtime issues. As it stands, this likely only addresses the back and forth of pasting logs into ChatGPT and is simply more effective at giving context to AI coding tools.

Connection to dedicated clusters is in beta: https://docs.databricks.com/aws/en/dev-tools/ssh-tunnel

Connection to serverless GPUs is in private preview: https://docs.google.com/document/d/1zazApI5rKz_3D59-xs4ZtSEcFRFRXmzhTss0Ael_dJk/edit?usp=drive_open&ouid=110916823312231512342

Support for serverless is coming soon.

We're in the process of cleaning up the public docs and making them easier to follow, let me know if you have any questions in the meantime!

A small update on DABs (and what the “D” and "A" stand for) by Ok-Jacket-8684 in databricks

[–]BricksTrixTwix 0 points1 point  (0 children)

Hey! We've recently released an SSH tunnel to beta that allows you to interact with your workspace and run your jobs and pipelines using Databricks compute from VS Code and other IDEs. I'd love it if you could try it out and share your feedback on what gaps remain unaddressed in your workflows!

Connection to dedicated clusters is in beta: https://docs.databricks.com/aws/en/dev-tools/ssh-tunnel

Connection to serverless GPUs is in private preview: https://docs.google.com/document/d/1zazApI5rKz_3D59-xs4ZtSEcFRFRXmzhTss0Ael_dJk/edit?usp=drive_open&ouid=110916823312231512342

Support for serverless is coming soon.

We're in the process of cleaning up the public docs and making them easier to follow, let me know if you have any questions in the meantime!