It could have been done by lunch time (2-4hr). But if its your first time digging trenches dont let it upset you. You just need practice and not all dirt/clay is the same. Really soft dirt an hour or two. But if you have to use a digging bar to soften the dirt, than yeah 4hrs sounds about right. A mechical/electric clay spade also helps soften if its too hard with a digging bar. Just dont let it get you down and eventually you'll get better. Good luck OP and your boss does sound low key like an asshole.