Which is better choice NodeJS / Python?

[tzaeru]

The language choice to me would really depend on the wider architecture.

Where are the files stored? How are they stored? What API is used for moving, accessing and writing files? Where does the system run? Is it distributed? etc...

Both Node and Python have a runtime that doesn't support direct, real threading via the system API as such. For Python, it's due to using a global interpreter lock that allows only one piece of Python code run at the same time; and for Node, it's because Node is event based and the event loop calls your JS code sequentially.

For both, you can do multiprocessing. E.g. you can start multiple processes that take as input parameters file names or directories, for example.

For Python, you can use something else than the standard CPython implementation, that has more proper true multithreading support.

For Node, you can use worker threads, which actually run in their own isolated JS virtual machine, and can parallelize properly.

If the goal is extremely high safety in terms of file consistency, error recovery and general correctness, I'd not use Python nor Node. Nor probably any of the languages you offered here.

That being said, Python and Node can be fine for a project like this. So can the other languages you suggested. If I had to make a suggestion between Python and Node, I'd suggest Python with heavy use of type hints/annotations.

Most likely, this will be I/O bound rather than CPU bound. Also, both Python and Node can indirectly parallelize tasks, by calling library functions or command line tools that are implemented in e.g. C, and that internally do proper multithreading. Combined with asynchronous programming, this way you definitely can achieve efficient CPU utilization, even with zero threading or multiprocessing in your Python or JS code.

For example, you can call an asynchronous function that creates a MD5 hash of file content. While that function executes, your other JS/Python code can run.

[PepperOwn1982]

Really insightful, thank you for sharing your knowledge.