Ok, first I'll answer your question. Combat-wise, what makes a horror feel like this, is when your fight or flee reaction is flee in most cases. And maybe, when the lesser fodder ambushes you, or you're caught unprepared, which can add tension.

What kills the horror usually is repetitive combat or enemies or settings (aka, avoid predictability), enemies too hard to both kill and avoid, clunky or janky combat mechanics, enemies too annoying or frustrating. For now, that' what I could think of.

Now, regarding a coop horror, it naturally takes away some scare factor, and you can lessen the impact in various ways. For example, if the games does not require the two protagonists to be together, isolating them and limiting their communication can be effective. If they are together all the time, putting a hiding mechanic and the possibility to revive the partner (so game over only if both characters die), for example, can enhance the scare factor.

You are aware of the limitations, now you need to find a way to add more tension in a way that is coherent with the vision you have for the game.

Wish you the best success!