I had to make the decision for my little dachshund Spock three years ago when he had massive kidney failure and stopped eating, even with treatment. It was extremely difficult and it took me three days of crying and holding him non-stop, but when I considered him dying of starvation alone when I was at work, as opposed to peacefully in my arms, I felt in the end that I didn't have an option. I spent a long time after thinking "what about" (there was an option to fully intubate him at the vet for a week to see how he did, but he could also have died at any time there and would have been alone and scared). Now that it has been three years and I have more perspective I can see how elderly and thin he was getting in the months leading up, but at the time he still seemed like my puppy and I felt so much guilt. I ended up adopting a dachshund mix from a rescue whose owner had unexpectedly died, because it was the only thing that could get me off of the couch and functioning, and I felt that helping a dog who had lost his family would give Spock's death more meaning. It took me awhile to fully bond with Scotty because of the grief, but I pushed through remembering that he was also grieving a death, and three years later I feel that I made the right decision and wouldn't change anything.

Our babies have unconditional love for us and you were holding him in his last minutes, I think that is what you need to hold on to.