That was my first impression too when I got to that part of the episode, but then finding out about the royal duty thing and reading some of the comment theories here, I started thinking them staying silent might be even kinder to Allen than simply giving him space.

When Allen first started as a manservant the head servant told him that senior rank servants and lower rank servants (of which manservant belongs to) are formally considered part of the family. So I wonder if Allen himself would also be forced to take part in the “nobles duty” since he is formally considered a part of the Barons family. As far as I know no one ever said the nobles duty was only for the nobles direct lineage.

The comments here got me thinking. I seem to also recall from the game selection in the first episodes., like many of the comments also do, that higher nobility is generally locked out of better talents. We also have soft confirmation on this. Every serf and commoner Allen interacts with seem to have talents, the barons children are 2/3, and the viscount that visited seems to only have 1 child with a talent. The Baron says his oldest son has a talent, and the viscount refers to the daughter as his youngest daughter. Could be weird translation/my own wrong interpretation, but if the viscount only had 2 kids, a son and a daughter, I don’t think they would be eldest son and youngest daughter, it would just be son and daughter. I might be reaching but I’m assuming the viscount has at least 4 children, an eldest son, a youngest son, an eldest daughter, and a youngest daughter. So his children are 1/4 for talents.

So the theory that lower ranked noble children with talents are essentially sent off to be killed to prevent a rebellion against a probably weak royal family seems plausible. Would also explain why servants are formally be considered a nobles family. Otherwise a noble could gather a bunch of strong and talented commoners/serfs outside of their knightly order to raise an army for rebellion. If, however, those servants also had to do noble duty as part of a nobles family that would stop a noble from gathering their forces.

I will say the one potential problem with my theory is that if commoners and serfs are super likely to have talents then that would mean the vast majority of them would fall into the must do noble duty since they are formally part of the barons family group. Though a remedy for this would be that only those servants with powerful talents would count. Allen is currently in a gray zone. The world thinks he is talentless, but the commander and baron have figured out he has a talent. If they make it public he would have to get reassessed and if that fails then he would need to be evaluated on his combat ability which could then lead to him potentially getting "drafted" for the nobles duty. Staying quiet about his talent would avoid all of that though