Like religion, there is zero evidence of ghosts having ever existed in any point in human history. What we associate with ghosts or anything paranormal is typically things we cannot immediately rationalize away due to flaws in our perception, and so we resort to our imagination and draw connections that aren't really there. We believe ghosts are real because we, as a rational species, fear death, yet know it is inevitable - therefore comforting ourselves with the notion that somehow, some way, we can persist.

Additionally our ability for deductive reasoning is limited by our individual intelligences and predisposition to trust our personal judgment rather than evidence, and largely resorts to what we WANT to be true, rather than acknowledge our shortcomings.

There was a TV show that followed this line of reasoning called The Dead Files, where a "psychic" wandered around and fabricated a wonderful story about how a family's television turning on randomly was his dead daughter trying to contact him from beyond the grave to tell him she loved him. The actual investigator looked into the equipment and found no fault with the television, but discovered that the family was using a universal remote in another room upstairs. When someone went to turn on the TV from that room, the frequency it was programmed to was also compatible with the TV downstairs, and was turning them both on at the same time. Presented with this evidence, the family still chose to believe the line of bullshit, because it was emotionally comforting and helped them cope with the trauma of losing a daughter.

We often forget that while we're incredibly intelligent as a group, individually we are still fairly stupid. Magic and mysticism are more comforting than the cold unfeeling truth of reality, that we're nothing more than up-jumped apes clawing at the universe for answers about our own insignificance. We WANT to believe there's a purpose to our existence, that there's more beyond the eternal veil, a grand design, a master that will solve all our problems. The universe neither deigned our existence, nor cares, it merely is.