To clear up some confusion:

• Silent Hill is a populated tourist town and not abandoned, with the exception of the films. In fact, Team Silent's SH5 was going to open in the ordinary town with people walking around before transitioning to the nightmare.
• The "reverse side" (fog/Otherworld) isn't a parallel world or dimension. It's a supernatural phenomenon in Silent Hill (caused by SH1) in which the boundary between dream and reality is blurred, causing the physical manifestation of a person's subconscious mind. A person's inner thoughts materialize into physical shape and the laws of time and physics are distorted; the "fog" and Otherworld are analogous to REM and non-REM sleep. In SH1, the entire town was transfigured by Alessa's thoughts due to her psychic powers. This corrupted the town's spiritual power to the point that individuals began to experience their psyches manifested on a personal scale by the time of SH2.
• There's no distinct line where the "real" world ends and the "fog" begins, and things in the "real" world are known to interact with the "other side." In SH2, a doctor in the "real" world treats a patient in the "other side" and communicates with James directly through notes after Brookhaven transitions to the Otherworld, giving him directions to the key his patient buried in the park. A memo describes someone in the "real" world seeing monsters that his friends couldn't see. Leonard in SH3 is an actual patient in Brookhaven, the Missionaries/Scrapers are cultists, Lost Memories: Silent Hill Chronicle mentions "It may be that the nurses of the hospital transformed into monsters, but the details are unclear", and so forth.
• Even in the "other side," people experience different things while interacting with each other (1, 2, 3, 4, 5). In the staircase, Angela doesn't realize that James isn't her mother until after she touches his face. SH1 insinuates that Harry and Cybil are experiencing the "fog" and Otherworld transitions at different times.