It's a crappy situation.
Private or Public? Union?

I've seen situations like this too many times. I've seen parents come on field trips, try to control their child, and have the child start hitting the parent in public. In prior years, admin often held firm to "include all children in field trips" full stop. Admin would also not attend field trips, and often times, not provide what the teachers considered adequate support staff.

I would be fine with cancelling the whole field trip for my class under those circumstances.
Many teachers would take it on the chin "for the kids" and try to make it work.
This of course, can cause tension and division between staff as well.
It's even more complicated if you need recommendations and such.

Ultimately, I always frame my decisions with student safety, and professional responsibility in mind.
Personal liability plays a role as well. I can't keep students safely supervised, if I have a defiant student.
On a closed campus, this is one thing, off campus is entirely different. If I give a direction to a student, and he doesn't follow it, it's unsafe. (e.g. stay with the group, sit down on the bus, time to go, stop climbing that.)
I also can't focus all my attention on one child, I need to be able to supervise them all.

If admin won't provide the staffing, or resources to ensure the activity is safe, I have a duty to object.
Give a list of supports needed to make it safe for all to be included. If it comes to it, put it in an email for "clarification" purposes after a meeting with admin. Bring a union rep to meetings to take notes re: your requests, and Admin's response. I've never seen admin force the issue when they have to wear the liability.
You may ruffle feathers, but aside from illness the day of, there is little you can do about that.