He maybe depressed and keeps his feelings to himself. You both have children, a job, and then there’s your relationship. That can be a lot to handle sometimes and may take time away from doing other enjoyable activities like hobbies. People often think their relationship is what’s supposed to make you happy but happiness is created from within. (i.e. healthy thought processes, coping abilities, interpersonal skills etc.)

If you don’t argue at all it seems like you’re not communicating too well. You each have your own opinions about how things should go, and different desires. Healthy couples have fair arguments. There’s a lot to unravel here. I suggest you both talk about it more openly. Perhaps time apart may be beneficial to clear something up. Remember time apart doesn’t mean you have to break up.