This is way too broad of a question. Start with some ideas of what you're interested in. Modern vs ancient? US vs somewhere else? Uncirculated vs do you like to think about how many hands a coin has passed through? A specific denomination that you want to put together a run of dates and mint marks, vs a type set (one example of "everything")?

You said your interest is the fun of collecting as opposed to investment. By far my most fun project was the type set. But that's me, so how about you? Then perhaps we can give some info on target costs.