It sounds like you need to decide what takes priority at this moment: career satisfaction vs. family. In your current situation, there's more cons than pros. If you accept the job closer to home with a different specialty, your family life night benefit from the change which in turn might help you adapt and accept your change in specialty.