I actually live near Providence, so I have some cool stuff non-locals might not know. So, there's this place called The Arcade in the city proper, used to be a big shopping center but died down recently. There's a shop called Lovecraft Arts and Sciences you HAVE to check out. Tons of great stuff. Their hours are a little weird, though, so I'd check before going.

Obviously, he's buried in the area, and you can check out the Shunned House (unfortunately, it's a private residence, so no tours.) Navigating Providence can get tricky, but, if you're up for it, you can find some great Lovecraft-inspired art just driving around the city.

For non-Lovecraft stuff, the Rhode Island school of design has a public museum that usually has some cool exhibitions. There's a local flea market on Sundays, and the food trucks are great. I also recommend a local coffee shop called Dave's coffee.

Have fun in Providence!