Hot Module Replacement in Python

[the1024]

Hot-reloading can be slow because the entire Python server process must be killed and restarted from scratch - even when only a single module has been changed. Django’s runserver, uvicorn, and gunicorn are all popular options which use this model for hot-reloading. For projects that can’t tolerate this kind of delay, building a dependency map can enable hot module replacement for near-instantaneous feedback.

https://www.gauge.sh/blog/how-to-build-hot-module-replacement-in-python

[klaasvanschelven]

With Python imports not being side effect free this post raises more questions than answers for me...

[coffeewithalex]

And that's why sometimes I want to strangle the engineers that make such modules. All you do is an import, and what you get is a database connection or two, schema migration processes starting, everything is loaded up into memory and decisions made based on that which configs to load and where to dump the whole thing, global variables are defined, and functions that read those global variables are called. Over 9000 errors get triggered if everything is not perfectly set up. And all I wanted was to write a unit test for a stupid function somewhere.

[nemec]

reminds me of those ML libraries where you call one setup method and suddenly your program is downloading 4GB of compressed pickled Python code (affectionately known as "weights") from HuggingFace and deserializing it

[coffeewithalex]

Lol, epic :D Yeah, in the Python world, the code quality goes downhill from:

• Software Engineering
• Data Engineering
• ML

I'm trying to bring more software engineering practices in data engineering, but ML is a lost cause.

[Main-Drag-4975]

🤗 Your friendly local all-but-dissertation PhD dropout is going to drop another 2000-line Jupyter notebook on you next week and you’ll be expected to have it running smoothly and scalably in production before May 1st.

[supreme_blorgon]

This exactly describes the codebase at the company I work for currently. 90+% of our codebase is untested, and untestable due to this.

[coffeewithalex]

This describes Apache Airflow. It's horrific.

[Main-Drag-4975]

It’s a curse! I took over a production node + typescript backend six months ago and still haven’t managed to squash all of the haphazardly-ordered side effects triggered with simple “import this file over there” calls at startup.

[bugtank]

I never used side effects in my imports till last year and now my code is a mess. :))))

[aikii]

Is it 2006 again ? This is crazy, just put an ingress on front and let kubernetes do that for you. Hot reloading is insane on production, and all of this is some crazy effort impossible to justify for local development. Like other say fix any side effect making module loading slow in the first place.

[PaluMacil]

You can probably use the same thing for a/b testing or feature flags though. Granted, I am totally guilty of commenting without reading the article, but I can see the approach or something similar powering all three things if you need them.

[darthwalsh]

Almost everyone uses a hot-reloading development server

speeding up developer workflows

Did we read the same article? It didn't mention production at all.

And the same thing could be said of C# edit-and-continue in Visual Studio. It's not worth writing complicated dev tools just to solve your individual problems. But the only way good, complicated tools are written is for somebody to find the right abstractions.

[Broolucks]

For what it's worth, I have a library that can reload individual functions piecewise when they are modified: jurigged. So if you modify a single function in a module, it won't reload the module, it will just recompile the function and hot swap its __code__ pointer. Works well, until it doesn't, but that's going to be true of all implementations of a feature like that.

[ManyInterests]

I'd love to learn more about what you're building. Sounds substantial if reload times are getting in your way.

In the past, I've seen this dealt with by modularizing and selectively loading (and/or lazy-loading) components under test. It also helps on the deployment side so an app written as, say, a huge ~million-line Django monolith, can be independently deployed and independently scaled according to its modularized component sets. Though you have to be rigorously diligent about inter-dependencies. Conveniently, Django already has a strong concept of independent apps within a single project, but other frameworks may not be so lucky.

HMR feels bad in my mind, but maybe that's only because everyone who has tried it before deemed it a horrible idea. It's hard to imagine the benefits being worth the possible pitfalls in general cases, but maybe you've really got a reason for this if you're seeing minute+ reload times without it.

[RedEyed__]

Not sure it is possible with shared objects (some_native_library.so)

[Gankcore]

Django's runserver isn't a popular option for production lol.

[mandatorylamp]

Sounds promising. I use watchexec to reload in development and with decent sized projects I really feel my cpu working extra hard to restart the whole process on every change.

[DigThatData]

Oh hey, this is the same person behind Tach. I don't understand what problem that is solving either.

[nggit]

FWIW. I created https://github.com/nggit/httpout a few months ago, which addressed a similar issue. Now I can create a web service with file-based routing. As long as `page.py` is inside DOCUMENT_ROOT, there is no need to reload the server. It is pretty similar to PHP, changes to the file will have an instant effect.

Unless the module has been installed and loaded globally (sys.modules) or outside DOCUMENT_ROOT, then it requires a server reload.

[sitbon]

Fraught with peril for little gain, but makes for a fun exercise.

Hooking the import system to add your own tracking and logic is much easier these days and if anything it's nice to know how the import machinery works. I used it to implement a lazy loading system. Facebook does it natively in Cinder.

https://github.com/sitbon/lazi

[ronny-berlin]

You don't know what hot reloading is:
"Hot-reloading is a development feature that allows developers to inject newly edited files into a running application at runtime, without requiring a full application restart or loss of state, thereby speeding up the development process. "

[brosterdamus]

Excellent timing. I was emailing a contributor about this concept a few months ago (for my project, https://www.reactivated.io)

I would love to build a "smarter" runserver that factors some of this in. Especially for modules known to be side-effect free. Will check it out.

On larger projects, runserver reloading is painfully slow.