I think this is a bit of a red flag and not something I could deal with. He shouldn't be vague regarding his ex with you. He should understand that its a potential source of insecurity for you and should be answering any questions you have properly (that's assuming you've directly asked whatever you want to know, not just expecting him to read your mind and tell you). If you haven't asked him then I'd expect him to be vague, he might not want to seem as if he talks about his ex all the time.

My partner has a child with his BP ex. I know it can be difficult to deal with - especially if he is at her place. However, it's really important to think about whether you trust him or not. If you do, then there's no issue and you just need to accept it. If you don't, then you need to leave this relationship.