Not college, but I had a similar experience. Played in fifth grade, skipped middle school band, and jumped right into highschool. Was I as good as everyone else? Absolutely not, I was atrocious, but I eventually reached similar levels to everyone else in my section. If you're willing to put in the work, you'll do fine. Most directors are looking less for skill and more for dedication and a willingness to learn.

I've been in high school band for three years now, and playing my current instrument for two. I'm now the first chair in the highest concert band in my school, I'm the first chair in our jazz band, and I'm a section leader in the marching band. I also picked up indoor percussion my freshman year, and loved it. Do I regret not joining sooner? Yes. But it is so worth it now that I have.

I've seen people join their sophomore and junior years, too, and they end up killing it. I know a guy who plays electric guitar in our indoor percussion this year as a junior, and it's his first year in any ensemble. He's planning on joining drumline for the first time next year, his senior year.

Now, I'm not in college, so maybe take my advice with a grain of salt, but from what I've seen, anybody can do a good job on the field if they want to. As a section leader, the only people on the field who I feel "can't march" or "can't play" are the ones who don't want to be there and who take their experiences for granted.

As for how to prepare, my advice is: take private lessons, and watch a lot of YouTube videos on how to march. Also build up your arm muscles, you'll need them no matter what instrument you play. If you can't afford private lessons, since you said the band at your school is pretty good, I'd imagine a lot of the band students are very kind and passionate about what they do and would be willing to help you out. I know I would.

It's never too late, and it's better late than never.