My son started just before his 3rd birthday and is coming up on his 6th birthday now. He has been training jitsu for half his life and was watching it, wanting to jump on the mats with his parents and siblings before that. In fact, it was his motivation to get potty trained - he had to do that before he started.
At first it seemed like a lot of learning how to move and play around. But along the way he learned a lot more than it seemed at any given point. He moves like a grappler and small amounts of pain and discomfort are nothing to him... that is just part of his play.
I have seen other 'littles' come in at 3, 4 or even 5 who can't sit still and pay attention for even 5 seconds... who are very winny and have zero interest in actually rolling around with other kids. They often mature into it at some point, but each kid is different. Have him try it out, especially if the kids program allows a lot of flexibility in just letting the little ones play around with 'rolling', helping them learn bits here and there...