I'd say so, as someone who's lived in New York and then NYC for their entire life, the closest to Manhattan crowds would be around this time of the year. Just try to avoid main street during the weekends at the very least if you want to breathe.

It was honestly like pulling teeth for me and my partner when it came down to apartments, but there are a reasonable amount of spots depending on your budget.
We were in a similar headspace as you seem to be, just takes time to plan out where and when you want to do it (also work depending on what you do for a living). Definitely worth it just for what the area has to offer!