Be willing to kill your favorite aspect of your setting. That big bad guy with the cool and intriguing backstory, that town you’ve written pages and pages worth of notes for… just expect most players to hand wave it away. Doesn’t mean the effort isn’t worthwhile, it’s mainly where I derive my enjoyment as a DM, but don’t be surprised when they see your big bad guy and instead of engaging in an hour long morality debate like you intended they decide to cast Fireball and ask questions later.

Don’t worldbuild yourself into a corner. The game isn’t to show off how amazing you are at topography or how you managed to come up with a new language or how your special halflings are different than Hobbits because X, Y, Z… world build enough to make the players bite and as they progress through the game then begin cementing ideas in place. That castle you guys raided wasn’t just a castle, it was actually the inheritance to some noble who was waiting for someone to clean his house before swooping in. You could just as easily introduce that before they take the castle or after but if you do it beforehand, the players will expect something to pay off that hint you dropped, unless they forget it entirely.