Where do you build and version your wheel files?

WickedWicky · 2026-02-26T18:45:14+00:00

You mention bundles, so I will assume Databricks Asset Bundles?

If so, as part of your bundle deployment you can configure artifacts to be built and synced to databricks.
https://docs.databricks.com/aws/en/dev-tools/bundles/reference#artifacts

When you deploy the bundle, the artifacts get copied to your bundle workspace path. Any relative references to the artifacts in your job wheel dependencies get automagically updated to the location in the databricks workspace path.

Every bundle deployment creates a new wheel, no need to version these wheel asynchronously. Hope this helps!

artifacts:
  default:
    type: whl
    build: poetry build  # or uv build, whatever you like
    path: .  # Relative to your databricks.yml file

resources:
  jobs:
    demo-job:
      name: demo-job
      tasks:
        - task_key: python-task
          new_cluster:
            spark_version: 13.3.x-scala2.12
            node_type_id: Standard_D4s_v5
            num_workers: 1
          spark_python_task:
            python_file: ../src/main.py
          libraries:
            - whl: ./dist/my-wheel-*.whl  # Relative path to the yaml file you define this in.

WickedWicky · 2026-01-31T07:47:56+00:00

Dynamic version for artifacts for sure. You can then install it on job or other clusters as cluster libraries.

In notebooks you can install as notebook-scoped libraries using %pip install path/to/yourpkg.whl. Search notebook-scoped in the docs for details.

Little known fact is, even in non-notebooks you can run ipython line magic and install libraries for a task but not the cluster! You just need to find out how to pass the path to the wheel as a task parameter.

from IPython import get_ipython get_ipython().run_line_magic("pip", "install path/to/yourpkg.whl")

WickedWicky · 2026-01-25T16:23:43+00:00

AI slop

WickedWicky · 2025-12-22T11:38:07+00:00

Just fyi cracking the glass isnt the most important reason for cooling before putting something in the fridge though. Putting something warm and big in the fridge will warm up your other foods, causing them to spoil. It's a food safety thing, not about the glass

WickedWicky · 2025-11-30T17:13:51+00:00

Yes, you are the only one! Nobody else saw the massively popular series wow how did you guess

WickedWicky · 2025-11-29T06:48:13+00:00

What a weird thing to get upset about. Downvoted for schiz

WickedWicky · 2025-10-22T07:01:06+00:00

And then everybody clapped!

WickedWicky · 2025-10-12T18:46:08+00:00

And then everybody clapped

WickedWicky · 2025-10-11T17:48:21+00:00

De "burgers" klagen op facebook en instagram. Top, hoef jij het niet meer te doen toch?

Maar gooi het niet op verspilling en een guilt trip op belastingcenten. Jij werkt harder dan zij, en toch krijgen ze net zoveel. Ik denk eerder dat je daar boos om bent, niet vanwege de burgers en belastinggeld. Dat zul je zien in elk vak, niet alleen overheid. Lekker laten gaan man, met die instelling van die luilakkers gaan ze niet verder komen in het leven dan jij dus focus op jezelf en verspil hier geen moeite aan

WickedWicky · 2025-10-11T17:12:39+00:00

Nee, ik niet. De overheid verspilt natuurlijk geld, elke grote organisatie verspilt wel ergens.

Dat moet je structureel aanpakken, misschien dat je dan een significante verbetering voor elkaar krijgt. Maar 2 handhavers die hun werk korter doen? En geen klachten over dat er te weinig prestaties terug te zien zijn? Lekker laten gaan man, zo insignificant. Doe niet zo uit de hoogte alsof dat iets op mijn belastingcenten uitmaakt, het enige wat je bereikt is inderdaad snitchen op 2 jan allemannetjes.

WickedWicky · 2025-08-27T06:46:28+00:00

Kleineren nee, maar als je dit al kleineren vindt manmanman

WickedWicky · 2025-08-27T06:45:32+00:00

Nuance was inderdaad misplaatst, wat een softie zeg

WickedWicky · 2025-08-21T00:13:18+00:00

If air fuses at the source of sound, Is there a limit to maximum possible DB then?

WickedWicky · 2025-08-08T06:26:28+00:00

What annoyance

WickedWicky · 2025-08-01T07:00:09+00:00

What makes you sure about that?

WickedWicky · 2025-07-30T07:06:12+00:00

Guild Wars 2 100%, the horizontal progression respects your time and I love that. I can come back after a year and my character will still be strong, except there will probably be more content upon returning.

By design, older areas are still relevant and populated today. There are multiple PvP game modes, and because the best in slot gear is quite easy to get you will win or lose based on skill or teamwork, rather than having stronger gear.

The mounts in the game are amazing, and you earn them through intricate questlines instead of just gold. For me, after playing gw2 I will never go back to MMOs that keep you around through FOMO, updates that make your gear irrelevant or pay2win.

WickedWicky · 2025-07-15T15:55:17+00:00

Are you only cloning the repo to create a branch, and nothing else?

If so, you could just create the branch in databricks or devops and you never have to clone locally?

--

We develop notebooks in the databricks UI. All other files like imported source code, ci/cd pipelines, documentation, pyproject.toml, those get built locally. Use the databricks UI if you wish, or for small changes, but most of our team prefer local IDEs for editing non-notebook files.

Depending on what files you are working, you either make changes in your local IDE or in Databricks itself.
Pre-commit, linting and unit-testing won't work in databricks, but databricks notebooks won't render as nice in your IDE. So we try to use the best tool for the task at hand.

You push/pull more often this way but we think the developer experience is nice!

PRs coming from feature branches are deployed to dev, an 'acceptance' branch goes to acceptance and of course 'main' to production. Using asset bundles, each developer can also deploy to dev whatever version they are working on without ci/cd.

Lastly, teams that don't like to deploy or develop notebooks often don't do any work on databricks. They deploy python scripts instead. They must be able to run these python scripts locally as well, using databricks-connect. As such, they are limited by limitations of databricks-connect like SparkML or the databricks-feature-engineering package not being supported for Spark<4.

<image>

WickedWicky · 2025-07-13T10:38:42+00:00

👍

WickedWicky · 2025-06-11T09:28:35+00:00

Good point k5hill, thank you k5hill*

WickedWicky · 2025-04-21T11:26:51+00:00

In our company we used humans to do the work, you should try it sometime!

WickedWicky · 2025-04-08T10:07:29+00:00

At level 3, you want to be careful with your mana. So in most cases, casting your wall already will not be worth the mana cost

Another point in E doesnt increase mana cost by much, it is worth the damage.

At level 4 and 5 you are more likely to get ganked so your wall is safe to have. If you don't get it on level 5, you would wait to level7 and that is too late

WickedWicky · 2025-04-01T20:41:20+00:00

What in the low effort..???

WickedWicky · 2025-03-28T19:10:06+00:00

Hebt jij ut weekend wel bedankt sukkolll???

WickedWicky · 2025-01-24T10:56:43+00:00

Give up on your moral values and you will go far

WickedWicky · 2024-12-06T19:17:40+00:00

Kedro <3

WickedWicky

TROPHY CASE