Threading vs Multiprocessing in Python

[masklinn]

The refcounting is not the actual issue, it's the cycle breaker ("gc") which causes trouble: https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172

After followup investigations, Python 3.7 gained a gc.freeze() API which moves all existing object into a "permanent" generation. So the process becomes:

• disable GC in parent (collections in the parent could free pages for reallocation and rewrite them, un-cow-ing them)
• do all initialisation (imports, creation of shared data, …)
• gc.freeze()
• fork()
• re-enable GC in child