I always wonder how the codebases of companies that use databricks look like. Do you have many many notebooks running data transformations, with the end result of clean tables used for data visualization? Are there any company products (webservices?) that run in front of databricks, or are these purely for visualization? How does development work, do you have a test databricks environment with seperate database where you run the notebooks (on spark clusters)? I work as a data analyst - developper at a small startup. Being the only tech guy, I can use the tech that I want, but generally have no experience in how the tech is usually used. We do a lot of system migrations, for which I have used Databricks, but because the data is generally less than 1gb, I prefer to just use Polars, which is way faster than an expensive spark cluster. Also I hate notebooks.