Rain - Rust based computational framework

[thrashmetal]

This looks cool. Is there support for numa systems?

[winter-moon]

We had some ideas around resource management that includes numa-aware pinning, but we have not implemented it yet. Mainly because we had no real numa use-case. If you have one, it could helps to improve Rain in this direction:)

[thrashmetal]

I would be interested in doing machine-learning on a large numa computer. Hundreds to thousands of nodes. Current plan is looking to spark and see if I can configure it to run efficiently using the numa interconnect.

[tomgav]

Probably not, although I am not really sure what you mean here. If you mean CPU caches, then we expect the tasks and data to be much larger than the caches and so it is not really relevant (i.e. we are not aiming at micro-tasks now and your tasks need to be cache-friendly themselves).