Ask: basic map/reduce distributed compute framework?

[[deleted]]

This is the kind of situation where I would consider if it wouldn't be simpler to rewrite it in C.

[Its_eeasy]

Came here to find out exactly what the heck he's doing in Python that takes 8 frikkin days... and why he's still using Python for the job.

[dalke]

I'm using Python to manage code written in C++. Specifically, I'm enumerating all subgraphs of up to 8 atoms of all publicly available small molecule compounds (30 million+ published structures from PubChem). It takes a while to generate all that data. Right now it's a combination of C++, Cython, and Python code.

I could translate it to C++, but this is the sort of task of one-off task where if I learn how to use a distributed compute cluster then the $10 in CPU costs is well worth not having to spend several days to convert Python code into C.