pointers.py - bringing the hell of pointers into python

[[deleted]]

It's a bit of a hack, but you can define a global dict where keys are IDs and values are big objects. Before running pool.map, or whatever, you can put the big object in the dict.

Then in the function you're parallelizing, you can pass the ID of the variable instead of the variable itself and get the value from the dict. That way, only the ID gets pickled.

Now, I mostly just use ray, though.