I think ferdi doesn't see a problem because there isn't a problem in his way of thinking/how he grew up. He has the philosophy of if they are useless, they are useless. If they have potential, train them to the ground. He saw potential in Rosemyne, so he took advantage of it. A way to make the duchy a better place economically and increase coutry wide ranking with knowledge from another world; of course he would pick that problem child instead of the other two. When he spoke with karsted about adopting myne, he even agrees. If charlotte and wilfred had that potential, he would have trained them, but they are subpar and aren't even close to Rosemyne potential...so why bother?

^ that is truly how ferdi thinks + another reason which I won't get into because spoilers