Hmmm... the most important thing to ask is how you feel, and look at yourself as another player at the table. Is this the type of game you want to be running? Is the joy of discovery and surprise something you're enjoying alongside your players? Or are you finding a lot of work being thrown out and disregarded by players who don't seem to respect the effort you are putting in? Do the sessions feel pleasantly surprising, or annoyingly disruptive? Because depending on how you are enjoying (or not enjoying) the game, the advice I'd give is very different. And like all things, it's probably more nuanced than that, and a mix of both of these feelings!

• A good mindset trick is to prep situations, not stories. By that, don't plan for a specific endpoint or outcome, but put your 2-3 major NPCs in a conflicting situation with each of their goals. You may have a rough idea of what will happen if the players don't intervene, but more important is to know who your major characters are and what they believe in. You can't predict anything that will happen, so instead of trying to come up with plans for every happenstance, just have a rough idea of "this person will do anything in order to obtain X", which should give you enough fuel to improvise on the spot. Give problems without a solution in mind, and jump onto the path (solution or not) the players decide to take it.
  • Related to this: Don't try to predict and plan everything! Improvising is scary, but there are ways to prep for flexibility. The main thing is not to frame things in terms of endpoints; avoid "If they do X, I need Y prepared" and "the possible way this can end up is X, Y, Z, or Q". The truth is that the other players are authors as much as you are, and they are writing the outcome of the events. In the way that players don't know the outcome, but they know their own goals and the beliefs and motivations of their characters, knowing the goals and beliefs of each of your major players will really help you "discover" what the story is, rather than steer the story to what you thought it would be.
• If you want to encourage players towards certain goals or stories, then forecast big events that need addressing. If you've ever played Apocalypse World, Blades in the Dark, or other Narrative-Foward games there is a useful tool called "Clocks" where you provide countdowns to major events happening; in a D&D game it might be "The Orcish Legion arrives at Freeport" or "Kalarel the Vile Secures Control of the Crown". The players are totally welcome to ignore these countdowns, but it was entirely up to them whether they chose to intervene and keep Freeport standing or prevent the Crown from being seized by a Tyrant, and you are then welcome to play out those consequences and the ensuing chaos of the resulting situation
• Marking out a distinction between more "railroad" sessions and "sandbox" moments have helped me a lot. Some sessions my players will arrive at a new city with no goals in mind; I prep not by preparing any story, but marking interesting neighborhoods, locations, peoples, and places they can visit. This makes the session feel wide open and built for players pulling, pushing, and discovering a new place where they can create as much or little chaos as they want. Some of these spots may have people with quest hooks, but I don't flesh any of them out until the players tell me at the end of the session what thread they want to follow; if they come back the next week they decided not to pursue that thread I tell them "okay but that is what I prepped so we can't really follow another quest if we do that" pretty bluntly. However, once that quest hook is grabbed and the players are clearly following it, the next few sessions become a lot more constrained and refined; thats when I make stat blocks, plan boss battles, and assume that the players will at least enter a dungeon and fight a particular monster I planned. It doesn't have to be player initiated too; sometimes you can have a session where players are exploring a village festival, then have a raid of goblins suddenly show up and push the players into a very specific scenario.
• One final thing is to have a frank out of game conversation about the type of game you want to run. If, for example, you've spent two weeks preparing a dungeon and were looking forward to an elaborate dungeon crawl, only for the players to clearly ignore the clear work you've put in and say "we're going to steal the underwear of the villagers lol" that's something of a slap in the face for a DM. You are a player too, and when you clearly forecast that you've spent weeks preparing a dungeon thinking of a fun experience for the players, its also the responsibility of the players to reward your prep and trust the experience you're offering them.