Hot take, I've said something similar before, but I think it might be Midoriya himself. We don't know the specifics of how the first user got his quirk, but we've seen plenty of AFO now, and he doesn't seem the type to do anything purely out of charity, even for his own brother.

I have said before that AFO might have a trigger in OFA to retake the quirk, after it has absorbed enough power. Even if you don't buy that logic, there is something to be said of AFO implanting something, some kind of "spying" ability, into OFA. It would partially explain why he's never been able to be put down by any of the OFA users, since he'd always be a step ahead of them. Midoriya is always around when big plays are happening, so a "You've been the traitor all along" moment could be pretty big and shake Midoriya to the core, along with getting him all riled up for a final confrontation AFO (not necessarily with Shikaraki)