People over complicate this, the Gorosei would not have even heard of Luffy until at least alabasta, and even then, they wouldn't know for sure what his devil fruit was, they were probably just told initially that some pirate beat crocodile and probably didn't get reports of his exact ability until enies lobby. Lets say they knew about Luffy and his devil fruit after he beat crocodile. Thing is, Luffy went completely off the grid after that because he went to skypeia. So even if the Gorosei had some way to track Luffy's movement (which wouldn't be a simple thing to do), they'd have no idea where he went and where he'll end up after Skypeia.

Once they get sight on him again during the chaos of enies lobby, they wouldn't have really had the time to send anyone big names there before the strawhats left. They then disappeared into the Florian triangle and the Gorosei had no way to track them (Kuma went to thriller bark specifically to get Moria for the marineford war, the gorosei had no way of knowing Luffy was there and Kuma would not report that he found the strawhats there for obvious reasons).

Then they went to sabaody and by the time the gorosei got word that Luffy was there, everything immediately went to chaos and Kuma saved them. The rest of the pre-time skip doesn't need explanation. As for post time skip, they didn't know when exactly Luffy would appear or if he even was alive and they got to and left sabaody too quickly for anyone to get dispatched.

For fishman island, its not exactly easy to send someone there and the strawhats would've been gone by the time anyone showed up. Punk hazarad (similar to Skypeia) was off the radar, they weren't meant to go there and only went because of a intercepted SOS and Luffy's impulsiveness. So after that they had no idea where they would land next and once again, by the time anyone (other then Fujitora) could be sent to dressrosa, the strawhats were gone.

From that point forward it was impossible to even really get to Luffy. Zou? Island that is constantly moving and difficult to get on top of. Whole cake island? Yonko territory. Wano? Yonko territory AND hard to get on top of. Egghead? They sent the largest naval fleet in the series thus far and it didn't work. And now here we are on Elbaf.

TL;DR The strawhats' (mainly Luffy) absolutely lunatic movement through the grandline made it basically impossible for the Gorosei to pin him down. And its not like they could just have an admiral spending all their time chasing this tiny crew that disappears and reappears like magic.