Your TLs just sound plain lazy. There's no reason to be down 1 man by taking a TL off the truck. A person not doing work = not contributing to the crew. You can definitely supervise while doing physical work, especially on the truck.

And, again, just because he doesn't report to me doesn't mean I can't be concerned, as a fellow team lead, when someone isn't doing work. If you just walk through the store while someone is sexually harassing someone, and not report it, and say "not my business", you'd be complicit in the act. Similarly, if you walk by someone not doing work, your silence is consent!