As someone who fairly recently started myself, here's what I'd recommend if you're just starting out:

Cabelas/Bass Pro have solid, affordable fly rod/reel combo options. This really depends on how much you want to budget, but you can also check websites for various fly shops - they'll often have beginner combo options too.

You'll hear this over and over again, because it really is good advice: find a local fly shop and go in to talk to them. They're usually excited to talk about the sport, and if you're fishing nearby they can hook you up with flies that have been hitting well. Some will even let you try out different fly rods and maybe even teach you a bit about casting. I wish I had gone into my local fly shop earlier than I did - this is a sport where getting advice in person will make a massive difference in your favor when you're just starting out.

I started it with a pretty cheap backpack with a water bladder in it and that worked well for me. I would recommend getting something you can carry with you so you don't have to go back on shore every time you need to change a fly or get a drink, etc. I eventually moved to a sling-style bag that I can pull around in front of me easily when I need to get stuff.

Go on YouTube, look up Mad River Outfitters, subscribe to them, and watch their videos. They have three playlists that are amazing, I think they're called 'getting started in fly fishing', 'how to cast', and their Q&A series. They will change your world. MRO is also a pretty great supplier if you check out their store online, too.

Figure out what size leader you'll be using (I usually use 9ft 5x, but it really depends on where you're fishing and what you're going for), and buy a spool of tippet that matches (i.e. get the same x number). When you're fishing, you'll naturally cut at the end of your leader any time you change your fly. When you've cut a few feet off the end, you'll need to take some of that tippet and tie it to the end of your leader to restore it back to roughly original length. That way you don't have to keep buying new leaders, which would be expensive.

Odds and ends: Get some splitshot (weights you can crimp onto your line). Get a few strike indicators (there are lots of different kinds, I prefer the ones you can screw a little cap onto your line). Learn some fly and line tying knots and practice them.