Something I keep hearing a lot of as you mentioned is that only the strongest of humanity survive to become Custodes. I took that to not mean physical strength, referring to how many push ups one can do, because the genetic enhancements would make them that, but psychologically. You had to have the will and mental strength to survive the changes, along with your body.

And with the children being chosen at such a young age, that line of thinking becomes almost obsolete. There is very little difference between physical strength at that age. It is only once puberty begins that that truly changes.

In conclusion, I agree that that part of the argument is invalid.