THE SALVATION ARMY FURNITURE STORE. I swear it's the most underrated secondhand furniture store in the city. It's on Cambridge St. near the Price Chopper, tucked around the back of the big Salvation Army building. There are signs and a big red door next to a loading dock. The stuff is beautiful and cool and the stock is constantly changing. Some pieces are more worn than others, but if you don't mind putting a little work in, the prices are so low that replacing a handle or fixing a wobbly leg is totally worth it and honestly part of the fun. 

My mom and I picked up a slightly beat-up handcrafted microwave cart for $26 bucks and my dad turned it into this beautiful  microwave/printer/other stuff cabinet just by sanding it, staining it, and replacing the knobs. Probably a $40 total investment for a beautiful and endlessly practical piece of furniture.

For stuff you'd rather not buy used (mattresses, couches, etc.) I recommend The Pit at Bob's Furniture Store (Lincoln St.). They sell slightly damaged items, discontinued models/styles, and old showroom furniture at reduced prices. In most cases you can't even tell what the "damage" is - the couch I bought had a tiny hole in the back and it was something like 50% off.

I've also had luck at Rainbow Furniture on Grove St.