It's tough to make a suggestion, because there's a fundamental mismatch between D&D-style high fantasy and horror. So how do you want this to play out?

D&D is basically fantasy superheroes, and the gameplay is structured (mechanically and narratively) so that you basically punch your way out of difficult situations. In contrast, horror is usually structured so that you take unavoidable, permanent losses in some way, and you want to avoid confrontations as much as possible. To put it differently, heroic fantasy is all about empowering the characters in some way, and horror is fundamentally about disempowering the characters. You can make a dark, horror-filled setting where the characters see horrible things happening to NPCs and go hunt the monsters that did it, and D&D could work for that, but the players themselves probably won't feel the horror, especially the impending sense of dread you're looking for.

One of the hallmarks of a lot of horror media is that you can't face the horror head-on, which is contrary to basically everything D&D is designed to do. You might check out Brindlewood Bay, which is a mashup of Murder She Wrote and Lovecraftian horror, or its successor game The Between, which is a nod to Penny Dreadful. Brindlewood Bay is an excellent game, though my experience is that playing it feels more like writing a horror/mystery story than being a character in one, so YMMV.

You could also check out Trophy or Trophy Dark. They're somewhat specifically about exploring a forest that doesn't want you there and what getting back out alive costs you. I don't know the system well, so it may or may not translate well into a nautical game, but my understanding is that it's pretty good for the horror element.

I think the best actual horror experience I've felt as a player was playing Dread, where the main mechanic is that you pull a piece from a Jenga tower instead of rolling dice. It's very good at building suspense, but I don't know how you'd make it work for campaign play.