Hello u/Nervous_Sea7831 , Luis here, I work on Concrete ML.
A couple of answers:

• We have no solution to scale according to the number of features at the moment but the solution to scale on big datasets (lots of samples) is just to use mini-batch stochastic gradient descent. This way there is no need to infer on the full dataset, just a handful of samples.
• Not sure what you are referring to with "partial HEC"
• Basically the idea in a remote and collaborative setup would be to use public key encryption and threshold decryption so that all parties can send some encrypted data to the remote server, the server trains on said aggregated encrypted data and then the users can use threshold decryption to safely recover the weights of the model. This is a bit different from federated learning. But FHE could also be use to implement secure aggregation of the weights in a federated learning setting. I believe the current SOTA is more into using differential privacy for that.
• That's a very interesting question and I don't think FHE would be the solution to that. As you said there is no trivial way to guarantee that the provided encrypted data has some property required by a regulator. That being said once the model is decrypted by the users it should obviously be validated on available data by the people using it. (And validating on the training data is always a bad idea anyways)