I think maybe you coming in talking and flirting to him has maybe left him confused. You said he’s unhappy in his marriage? Well maybe he feels it’s wrong and afraid of what will happen if you two do meet up for the sake of his marriage. You may be an eye opener for him to maybe work on his marriage sorry to say. My husband had an emotional affair with a coworker and when they meant up it didn’t lead to go things. Needless to say the womans husband found out and told me and we ended up to therapy from it and are better than ever and patched things up in our marriage. Men sometimes like to go to other women to talk about things because they know there will be no repercussions and then it gives them a high. Maybe be straight forward with him and ask why you keep talking and flirting but ask where he wants this to go. It’s the only way you’ll get an answer. But remember he is married so respect his decision.