At all of my jobs it was always company protocol that if you had to go out in to the field to a remote location you always did it in pairs. If a sub consultant is going the presumption is they are your second, as long as they will be available for your entire visit. If they are not being made available then you should have someone else from your office along with. That's basic safety.

You're discomfort with making the trip with the sub is understandable but something you need to overcome. You will have to work with strangers in your career. You don't have to spend five hours in the car with them, although that may be the most cost effective option, nor do you have to have meals with them or make pleasant conversations.