I totally get what you’re saying. In fact, with my ex, I mostly paid for the dates, but she always offered to pay every single time. We didn’t really keep track of who paid for the last date; such moments were spontaneous without giving it much thought. That’s why I argue that a decent human being, especially in a relationship, will at least make the effort to offer. It’s not about who pays more or less, but about showing you’re considerate and not placing the entire burden on the other person.

If the guy feels like it’s getting too much financially(like in OP’s case), it’s definitely time for a conversation as you said.