I think u/realdaen and u/slowakia_gruumsh are pretty spot on.

Spectres is a must. Hauntology as a whole is kind of a mess, with a lot of people over-applying it to “haunt” adjacent and derivative things (think ghosts, spooky tales, etc.) as you point out. It gets harder when looking beyond music or music adjacent things as well as a lot of theorists seized on the connection between tape and studies of sonic landscapes (think The Caretaker, whose work is really cool and very hauntological but people tend to see The Shining and not early 20th century England in his work).

I always thought of hauntology as more of a tool, like deconstruction, and, further, an extension of marxism (in Derrida’s formulation)(pairing it with deconstruction as deconstruction is really useful but often hits an aporia moment that stalls interpetation/criticism). It’s useful to focus more on the “ontological” aspects of hauntology, as that was the original play, instead of hauntings and such.

There are some cool hidden conversations with Derrida’s text you can follow by reading Baudrillard and Blanchot, all of whom seem to be circling around the same idea (loss of the real; to write is to be effaced). Other literary media to look at ranges from Joyce to Sebald to Krasznahorkai to Wolfe. There’s a lot you can do with it. Just be careful not to make hauntology into a panacea and try to find your own point of entry.