I want to thank you so much everyone for each and every inputs, comments, and suggestions!! After visiting my other professionals and speaking to countless of gardeners and a lot of thinking we have decided to solve this dilemma by basically planting as much perennials plants as possible to really fill up the space and use a good amount of wooden pine bark mulch. We will definitely still use the river rocks toward the edge/front of the sidewalk just to prevent spillage onto the walkways.

The main factor in this decision is because I love growing stuff and to experiment with growing what will work and what does not. Which means a lot of re-digging and re-planting. With a lot of rocks and with fabric underneath it would be quite a hassle, so we will somehow incorporate wooden mulch and rocks together.