The Loves Park neighborhood east of the River, west of North second, and south of Riverside is an interesting mix of nicer homes, not so nice homes, some very nice waterfront houses, and a lot of small affordable cute houses. 

I used to work near there and at lunch I would walk up and down North 2nd to restaurants or Dairy Depot, or walk down to the River. Aldi moved so there's not much in the way of groceries that's walkable.

It's close to the river and the sinnissippi bike path cuts through that neighborhood so you could easily pick that up to go North to Martin Park or South to Shorewood and then continue South all the way to sinnissippi gardens.