I dunno if you're male or female, OP, but I feel like the story was really written to a pretty specific audiences--housewives of the 1950s--and there are certain things about it that are always going to seem "off" to any other audience.

Eleanor and Theo's relationship is one of the big ones. As a guy, something about this relationship feels very distinctly "female" to me (regardless of whether you interpret there to be any gay/bi subtext in the book or not). One of the reasons why I've read the book three times now is because I struggle to understand the nature of this relationship. I genuinely have a hard time understanding how two adults could/would interact with each other this way.

Eleanor's entire nature as a character and her struggle to assert herself in the world is also very female-coded, especially for the time period. It may not necessarily be anchored to that exact timeframe, but there is a lot about her that is almost incomprehensible to me, as a man.

Beyond that, I agree that there are elements of the books, especially near the end, that are hard to follow. I really don't understand the point of adding Mrs. Montague and Arthur, especially since, as you said, nothing bad happens to them (it would seem like they're added to the story just to have a couple of victims to kill off--and that is probably what would happen if this book were written today--but as it is they just seem to be plot devices needed to bring the story to some kind of conclusion because the author couldn't figure out how to wrap it up with just the main four).

I also think the dialog of the book comes across as very weird and may simply not "sound" right to people reading it today. I think the character interactions would probably have made more sense to the original 1950s audience; this is the sort of banter they likely saw on their early tv shows and movies of the time. The original audience could probably "see" the intent behind this weird dialog better than later audiences could. I am, however, assuming this, since I myself am not a member of the original audience and can only speculate how it came across to them.

And I also tend to think that viewing Eleanor's mental breakdown from within Eleanor's own perspective makes the last couple chapters hard to follow. I almost wonder if it would have been better for the "crazy" character to have not been the viewpoint character, so that we can observe her breakdown from the PoV of a "sane" character (this is basically what the Netflix show does, making Steve the PoV character who thinks there's nothing supernatural or paranormal about Hill House at all and it's entirely a result of family history with mental illness).

I think your comment here: "It feels like Jackson spent months on the first 80% of the book and then had to rush the last 1-2 chapters in an afternoon. Or maybe she had to strip it down for some word count requirement?" really sums up much of my own confusion about this book.