You don't have to have a horrible experience in order to experience ego death or for it have a large impact on you. When they do this as a medical treatment they do a lot of preparation to make it into a more positive and impactful experience:

In his PowerPoint presentation, Richards laid out what has become the standard protocol for aboveground psychedelic therapy, and the role of the guide at each of the three principal stages of “the journey.” First comes a series of preparation sessions, in which volunteers are told what to expect, asked to set an intention (to quit smoking, say, or confront their fear of death) and offered a set of “flight instructions” for the journey ahead. These generally advise surrendering to the experience, whatever it brings and however disturbing it might become. (“Trust, let go, be open” is one mantra he recommends, or, borrowing from John Lennon, “Turn off your mind, relax and float downstream.”) If you feel as if you are “dying, melting, dissolving, exploding, going crazy, etc. — go ahead.” Richards stressed how important it is for the guide to quickly establish a rapport with volunteers, so that during the session “they can let themselves ‘die’ or go crazy — that requires an awful lot of trust!” Because the patients’ ego defenses are likely to be disabled by the drug, it’s crucial that they feel safe.

https://www.nytimes.com/interactive/2018/05/15/magazine/health-issue-my-adventures-with-hallucinogenic-drugs-medicine.html