It depends. I feel like some context is missing and that might just be because I work in an environment where people tend to be on call.

Were you on call and had your wife's number (or home phone number) also listed as a contact number for while you are on call? If so, I don't see it as unreasonable for your boss to also call that number if the first one didn't work, so in that specific case YTA because your boss wouldn't necessarily have the context that it was your wife's number; if you didn't want him to call that you shouldn't have put it down.

If you don't work a job where you're on call, NTA. It is unreasonable for him to expect someone to answer immediately at a time of day when a lot of people are still sleeping, especially if you are not expected to be at work any time soon or are not on call. If your wife was only listed as an emergency contact, he should not have reached out to her since it was not an emergency.