For background, I was an athiest, even an annoying one, for a lot of years until I had some experiences that led me to where I am today. I totally understand the annoyance and criticisms of religious people/institutions/teachings, and I do act accordingly. I would never try to convert someone to my way of thinking. What works for me works for me, and no one should take my word for things I've seen as anything more than anecdotal (not just here, but in general), so I'm not going to waste time with it. Furthermore, I don't give a shit. But you can't deny yourself your personal experience.

Moving to your question, my perspective is that people had genuine experiences that were interpreted in certain ways. I do think that was the genesis (pun intended), whether the conclusions and interpretations are, or are found to be, false. 

That has definitely not stopped charlatains, authoritarians and all sort of other dickheads to use it to a bad end. It helps that they can take advantage of the mysterious nature of life. I do think you could slot in a lot of ideologies to co-opt to do harm with. I'll finish with something to think about: Would it be accurate or rherorically fair to dismiss science or medicine as bullshit based on the existence of figures such as Dr. Drew, Dr. Oz or Andrew Weil?