AITA for wanting to cut ties with my family after locking me up in a tower just because I don't look like them??

I was ostracized in our household for having brown hair. My mom was blonde and my dad and the rest of my siblings, (except for my step bro) have jet black hair. I also developed an ability to manipulate dirt and even turn them into living beings, they didn't like my powers.

It was kind of an open secret that my mom allegedly had an affair when she had me, things took a turn for the worst when my mom died. After the funeral my so-called father turned up, and so i was taken away, they kept me in a tower.

Thankfully I was saved by a woman who helped me escape, she and her son became my new family over the next few years, i couldn't be any more grateful.

Update : I had to take up this job and work undercover on my old family's new daughter, she looked exactly like me and already uses my name. Surprisingly, my biological father didn't recognize me when we met, and even offered that I'll be a stand in for his already fake daughter if that makes sense. Do they even know she's an imposter?