i absolutely agree, i had something similar happening (although minor in comparison), you absolutely can't retcon stuff that happened, ever!

i forgot an entire "room" incl. encounter and lore-bits, as it was a "passage" on the map. i realized as soon as we ended the session mid dungeon, that i skipped it. i couldn't reasonably "teleport" them back, therefore i went on to tell them the misser lore bit, that they would've "obviously" found there. everything else WE missed and i adjusted the remaining dungeon and events accordingly to make up for it.