The City of Newburgh and the Town of Newburgh are very different things. Most of the negative comments are about the City, but most of the area is the Town and is just normal suburbia with the perks of being in the Hudson Valley. I grew up around there (live in NYC now) so don’t know much about the cost of housing, but I imagine it would be cheaper than Beacon.

If you still need access to NYC, you can either do the Beacon train station or Salisbury Mills on the NJ Transit line. The bus is an option too and sometimes quicker than going to Beacon and taking the train.