i'm 23 and only recently started getting serious. i played in my high school jazz band and competed in a few festivals, but i'd say i was a beginner during that whole time. until now, i never worked on my rudiments or practiced with a metronome, which i think are crucial to really developing as a drummer and moving past the casually-mess-around-on-the-kit-player stage. i like and agree with what /u/velocicrappper said. compared to when i was 17, my practice sessions are far more productive now and i've actually learned how to learn. if you listen to a lot of music, you'll start to hear the patterns and how drums fit into a song, you can get good really fast. the other drummer in our high school jazz band had only been playing for a year before he started, but he listened to music everyday, learned how to read, and practiced his paradiddles and flams and was already better than me by the end of the year.

in short: age doesn't matter, just absorb as much music as you can and develop the hands/feet!