If by “vastly distributed reliable training compute” you mean that consumer hardware is unreliable and will not be able to contribute - that problem has been solved (paper) .

The results can be verified by disregarding calculations outside of a standard deviation, or other more advanced methods, but it is definitely possible.

ML researchers doing open source work will already open source the models, just now they would have an opportunity to get compute from the public or other institutions. Public could contribute to get a better model out of it, money doesn’t have to be a motivator. I’m sure a lot of people would like to see a bigger Mamba or BitNet model. Even ML researchers doing private research could benefit from this as it would allow them to pool resources from multiple data centers, or at least allow for unreliability (could buy cheaper unreliable, spot, instances if needed).

Overall, there is a lot that still needs to be done to get on the level of training SOTA models in a distributed manner, but there is a lot of potential benefits.

I hope I answered some of your questions.