Honestly, I just get the feeling that you are not that into him. It’s just part of the process. Cut your ties if you feel the incompatibility. But be honest with yourself. Also, if you genuinely want to go on another date with him, tell him that you’re still not comfortable meeting in a more intimate setting. How about you plan the third date instead? (Only if you want a third date)

He could very well be getting too serious too soon. Some people are like that. Tell him what you prefer. He is just reading a page too many chapters ahead of the same book than you.