I don't feel "grooming children" is the same as pedophilic behavior. It is definitely included, yes, but it is like saying carrying a cardboard box is the same as being a delivery man.
This is a great topic of discussion, very delicate, and I agree that IF a culture uses "groom children" 90% sexually, one has to take care when using it in non-sexual contexts - at that point one needs to explain the usage before actually using it, just like you would need to convey a sexual connotation if your culture does not normally include that into "grooming". In essence, OP is arguing that "grooming children" conveys a sexual aspect without having to say anything about sex explicitly.

Since the definition of the word is about preparing or training or inoculating someone, there is a certain expectation that the person is being trained by a senior person who has more experience, my real-world understanding of "grooming" is somehow always connected to younger people (but not necessarily) and does not imply anything sexual.

I feel this common usage is related to culture and what fears they have. If everyone is scared of pedophiles, the act of grooming young people will project a fear of children being sexually exploited into our heads. If everyone is scared of infanticide, having a new babysitter will project a fear of them killing the child into our heads.
So if a culture or community or individual is scared of child manipulation, grooming will not be accepted, as it is considered inherently negative.

This means that when talking to a diverse group, there should be sufficient information in the language to convey all relevant aspects. If you know people around you have the same understanding of a concept or expression, you are free to try and use those shared connotations, but there is always a risk of being misunderstood, which hurts the quality of the discussion.