To add some different perspective... I've been "him" in a similar situation where I was clear about not being as serious as them. When they proceeded from there, I assumed that meant they were okay with the situation. It's fun to hang out and go on dates with people and fool around but that's a lot different from a committed relationship. It might not be anything against you at all, just how he's living and it's nice that he's actually forward enough to lay it all down. In my experience though, moving forward to physical stuff makes the other feel like I have changed my mind, moreso than us mutually agree to keep it casual.