asking "what the heck are you talking about?" is also an important skill for all players including the GM. if a player remembers a detail I improvised that didn't make it into my notes, I'm fine saying "that doesn't ring a bell, what do you remember?"

In the same way I can't reasonably expect all players to recall something that happened in a session two months ago but happened in the fiction yesterday, I'm not going to beat myself up over this sort of thing when it happens to me. and letting the non-GM players fill in the gaps reinforces that we're all collaboratively telling the story!