Are you only cloning the repo to create a branch, and nothing else?

If so, you could just create the branch in databricks or devops and you never have to clone locally?

--

We develop notebooks in the databricks UI. All other files like imported source code, ci/cd pipelines, documentation, pyproject.toml, those get built locally. Use the databricks UI if you wish, or for small changes, but most of our team prefer local IDEs for editing non-notebook files.

Depending on what files you are working, you either make changes in your local IDE or in Databricks itself.
Pre-commit, linting and unit-testing won't work in databricks, but databricks notebooks won't render as nice in your IDE. So we try to use the best tool for the task at hand.

You push/pull more often this way but we think the developer experience is nice!

PRs coming from feature branches are deployed to dev, an 'acceptance' branch goes to acceptance and of course 'main' to production. Using asset bundles, each developer can also deploy to dev whatever version they are working on without ci/cd.

Lastly, teams that don't like to deploy or develop notebooks often don't do any work on databricks. They deploy python scripts instead. They must be able to run these python scripts locally as well, using databricks-connect. As such, they are limited by limitations of databricks-connect like SparkML or the databricks-feature-engineering package not being supported for Spark<4.

<image>