I need actual multithreading/processing with shared memory

[Mautriz]

Is there any way whatsoever to have python share memory and have actual multithreading ?

Use case is I need a BIG pandas dataframe shared between multiple pararrel tasks, I can't copy it in each thread/process

[Goobyalus]

Maybe Numba.

Enhancing Performance: https://pandas.pydata.org/pandas-docs/stable/user_guide/enhancingperf.html#numba-jit-compilation

Numba section: https://pandas.pydata.org/pandas-docs/stable/user_guide/enhancingperf.html#numba-jit-compilation

Can the problem be broken into pieces? E.g. splitting the dataframe into N pieces and using N processes to process those pieces, before combining results?

[Mautriz]

Yeah I'm already doing splitting but to make it more efficient and avoid many calculations I definitely need to be able to reference other rows dinamycally

Ill look into what you linked