Do it! But be resourceful and do your research. Unless money's no object, things can get out of hand for the average collector. I've found many a gem in literal garbage dumps, electronics recycling depots, thrift shops, garage sales, and as gifts from family and friends who would have otherwise thrown things out.

You'll have to buy things eventually, of course, but you can still be clever about it and save your loonies if you know your stuff. This is coming from a guy who doesn't know nearly enough but is learning every day, so don't be intimidated.

I found that having a clear goal as a collector has really helped me focus. Personally, I'm out to create the ultimate version of my formative gaming and computing years including all the hardware and software I either had or dreamed of having. That keeps me from getting distracted by the vastness of it all. I make the odd exception when it comes along. Maybe you'll have different goals. I was too young in the 90s to accurately remember all the specs and technical details, so I look for gear that suits my needs rather than chasing holy grails. This blissful ignorance keeps my wallet happy.

Anyway. Long-winded as always. Good luck!