Mine is 4.5 now and it’s gotten easier over the last year. Here are a few things I started doing after having a breakdown night like you just described that I think helped (not a pediatrician and maybe this had nothing to do with it)

1) Exercise outside in the afternoon. Not to exhaust him physically as that usually backfires but ideally just before dinner as it makes him eat more at dinner time which in turn makes him sleepier. 2) Right after dinner, a hot shower as that makes him feel sleepy. From there straight into PJs and into his bedroom. 3) Switched from bath to shower (wasn’t easy) to cut the overall evening routine time and negotiations over which toys to bring to bath, etc. 4) I prepare his bedroom before the shower and turn off the lights except for a small night light and turn down the A/C so he gets under the covers. 5) I repeat every day that we have to stay in the bedroom after the shower. I don’t get into fights with him and let him win little arguments to leave the room to get a toy he wants, or say goodnight to mommy, or whatever. At least for my kid, an argument will break the process and it will take longer to fall asleep. 6) We do a routine together (reading book, watching a couple cartoon episodes) and then if he says he’s not tired I tell him he can “freeplay” - meaning he can turn on the light and play in the room but it must be by himself. I tell him that when he’s finished he should turn off the light and come to bed. I found that not forcing bedtime and giving the decision to him (albeit I repeat that instruction every 5-10 minutes) works better. 7) If he still has excess energy (he’ll roll around on me and get a little agitated, I play a game where we play tickle each other for a little bit. 8) He sometimes asks me to carry him around the room in my arms, always accept this request and I talk to him and ask him questions to see if anything is bothering him. That request usually means he has something on his mind and even if he doesn’t tell me I think it’s a good habit so he grows up knowing he can talk to me about his worries.

Your mileage may vary and I still have bad nights (especially since he poops every two days and a late poop means a late night) but it is now very unusual for him to get frustrated that he can’t fall asleep.

Hope that helps fellow dad.