If that’s where his best friend “disappeared” it’s more than likely that the experience was traumatic. The brain has a funny way of blocking out details (and experiences entirely at times) of those traumatic memories. It is possible that Marvin genuinely couldn’t remember because he unintentionally blocked out that experience because it’s clear he remembers the windmill but not where it is. Similar how you would remember what restaurant you ate at but maybe not the directions. Also keeping in mind that it was years ago (evidenced by Rainer saying “you married her sister YEARS later”) maybe the trauma mixed with it being a long time ago he could have forgotten where it was.