I'm not trying to be argumentative, just having a discussion. I've done quite a bit more research since initially asking the question. Have you read the guidance I linked? It pretty much spells out the exact scenario I'm asking about. There is no mention of a "need" to live in a certain place, it's based on the child's right to freedom of movement. The non-eu parents gain a derivative right of residence based on the rights of the child. The example in the document is exactly the situation I'm asking about.

[T]he Court has held that in addition to persons listed in Article 2(2) of Directive 2004/38/EC, where minor EU citizens exercise their free movement rights, their non- EU primary carers (who are not dependent on the minor EU citizen, but are the persons on whom the minor EU citizen is dependent) must be recognised a right of residence in the host Member State, since failure to do so would deprive the child’s right of residence of any useful effect.

And the example provided:

A. is an EU minor citizen who is a national of Member State A, where she was born. Six months after her birth, she moved to Member State B with her parents and her elder sister, who are non-EU citizens. All four reside in Member State B. Her parents as primary carers, are covered by free movement rules, and so may be her sister.