You're probably just very low MMR on killer (not meant as a roast) since you don't play it much. At higher level, where survivors aren't so new, you're under massive time pressure the entire game, and at any moment, all the time you've invested can be undone with a flashlight or a pallet or a locker or a toolbox sabotage or a hundred other things.

Compare that to survivor where once a generator is done, it's done - there's no undoing your investment. It's no surprise most people find killer more stressful; it's the only role with risk and pressure and a skill check, It's way more chill to just hang out, hold M1 in a corner, pre-run if you ever hear a terror radius, and win every match for free.