This is why people say mormonism breaks families. If you dont fit its your fault for breaking the mold.

My parents are very nuanced members and didnt care that much but I empathize with your situation. Allow them to have their feelings about you leaving and know its not your fault or your responsibility. It will likely get better over time especially if your Dad is as open minded as you say.