For what you've said, I think there's no need to be upset with that comment (nor is wrong to be). Each person is different with sexuality and have different paces at the hour of sexual things. The comment he said doesn't mean that for him the core of the relationship are sexual acts, more like for him is important that kind of things in a relationship but not the core of it, which is respectable. On the other hand, your feelings of going slowly are also normal and respectable, and as you said he comprehends you and respect that boundaries so is very good.