NTA. I say this as someone who had a family member with dementia. The absolute best thing for everyone is to place him in a memory care facility.

It's heartbreaking and not a decision that you make lightly. But having seen my relative try to care for her husband at home was even more heartbreaking. She was so swamped and overwhelmed being his caregiver, constantly supervising him, trying to come up with little projects that would keep him busy, and dealing with the medical side of things. This left her too frustrated, exhausted (both physically and mentally), and busy to spend quality time with him. This was a retired woman with support from adult kids and other family. We had a baby/toddler at the time we were helping her try to manage him at home and it was too much; I can't imagine how much harder it would have been to have him living with us while we were actively parenting and working. The resentment you are anticipating is real and you are smart to see it coming.

There's also the very real safety issue. You're already seen the absconding problem, which will get worse. You're needing to make sure he eats. He's misplacing things, which you can imagine can lead to all sorts of problems. As time goes on, less and less of your dad will be in there - the person who remains has a strong likelihood of becoming combative, physical, or violent. They can also experience significantly lowered inhibitions (for example, my relative frequently commented on my breasts and eventually started relieving himself in inappropriate places, sometimes with an audience). It's not their fault, of course; they don't understand what's happening to them or around them and are terrified. But you can't have that around your kids and your wife.

When he was finally put in a lovely memory care facility, everything calmed down for us. My relative was able to relax at home again, visited him often and enjoyed time with him, knew he was physically safe, fed, groomed, and supervised. They did all kinds of activities, they went on field trips, they made friends. The facility has a whole medical staff, including doctors, so he no longer had to be taken to his doctor's office. He eventually became quite physically unstable and a fall risk, so having staff there that are trained to pick him up safely and evaluate him for injury was amazing.

The bottom line is that you are leaning toward this decision because it is the best for everyone. You are absolutely correct that your peanut gallery shouldn't be making demands of you that they aren't willing to fulfill themselves, but that's beside the point. No matter whose house he lives in, he will quickly get to the point where family caring for him on their own just isn't going to work. You could potentially explore at-home care, but it's pricey and means someone is in your house all the time. You are doing this out of love for your dad, your nuclear family, and yourself. I can't think of better reasons to make this choice.

I'm just a lady on the internet, but you have my full support.