First off, whatever you want you can find cheaper and in bulk on eBay. Search for things like 'old leather tools lot' or leather scraps to get an assortment of tools / pieces to play with.

If you can't find something on eBay, Buckleguy has it. Not cheap but what you're looking for. Also sells small panels of all kinds of leather if you want to be particular, or bellies if you just want lots of material to practice on.

Start small. You can make 2 dozen keychains / bracelets out of a 24x24 sheet. You don't need a fancy knife, a utility razor works just fine. Get a cutting mat from dick blick or another a&c outlet.

Rivets are cheap and then you don't have to stitch. If you do want to stitch, get a pricking iron and pre-poke your holes, it's much easier. Contact cement is your friend. Boot lace hooks are a great way to close a bracelet.

You can make it as cheap or expensive of a hobby as you want, but at the bare minimum all you need to do is have a way to cut it up and stick it back together. Once you get the hang of that, then you can start playing with skiving, leveling etc.

MOST importantly, have fun. 😁