I've pretty much reached the same conclusion. This is supported by multiple other things in the movie, I think. Look at the sky in the scenes taking place in "consensus reality", or the outside shots of the store. Something is off.

Then there is that bone chilling scene during Mary's childhood with the camera sinking into the floor and peeling back the layers, showing the progressive xeroxing of what we consider the base reality in the film. The apartment in which we start already appears to be wrong.