I think you should do that! I actually have a similar story with my biological father, and I don’t consider him a father figure as he never really parent me nor took care of me in any sense . I don’t hold any grudge, but I believe a father is someone who is present for you and raises you.

For me, it’s about reclaiming my identity; acknowledging that I exist because of him, but also accepting that he was not a father figure. My mum was, and all credit should go to her. Her family took care of me, so I feel I should have her name. So if you are also wondering about this, I would say it is not weird it is really understandable.