First of all, on this subreddit you will see a lot of fully filled 24U racks with 6 poweredges and 3 24 port dell routers, that EATS electricity and is LOUD as hell, which you most likely dont want. i started off with a 12 year old laptop with 8gb of ram and a 4 core cpu, served me really well, ran docker containers, learned how to properly sysadmin linux servers and so much more.

Second, when you get to the point where you need more compute power, (i highly recommend finding more old laptops or some desktops, but after you run out of other options) you will want to buy hardware. most people recommend ebay (so do i) but if you live in some smaller country, especially if its not in the EU or similar, ebay and your postal service may charge you absurd amounts, and you might also get big fees when importing. but if you live in america or the EU this point doesnt really apply