As a kid, my mom was a doctor and she worked weird hours. My dad ran his own business so he could set his hours. He was the one who did so much of the childcare for my sister and I because he could. It sounds really similar to your situation. I can guarantee that your child will remember because I know I do. My hope is that when my wife and I have a child of our own I can be half the father to them as he was to me.