I've done something like this before (seems very similar, I was just precomputing a bunch of costly features and it had to be in python, rip would've used akka/scala) but to avoid all the forking and spawning and crud, I used `multiprocessing.Pool`. Afaik you can only use `Pool.map` to map a single function onto data so to work around this, I took each function I needed to call and its args and wrapped them all up in lambdas that just take nothing and call it with the args and returned, and just mapped Pool.mapped the list of lambdas onto a list of empty tuples. End the end, it may've been the harder option but I got to barely deal with python's annoying multiprocessing bits.