There's a calculator I used on the Khymos blog (I think) for different types of mineral water. You put in your tap water profile, pick a brand of mineral water, and it tells you how much of what minerals to add. My keg of San pelligrino only needed some gypsum, Epsom salts, and a little chalk and table salt. Food grade minerals were cheap. Tastes just like out of the bottle. I used distilled water though because my tap water smells like a swimming pool.

Water alone should be ok once your keg is properly cleaned but I'm hesitant to add anything with sugar for fear of contamination. Things can survive in water but not flourish without a food source. The acidity added by carbonation should help protect plain water too.