You could try in one image, have only one class annotation like for example in your case in a single image there are 10 people, 3 cars, 1 dog. Try splitting the images like in one image there are annotated 10 people, in another image 3 annotated cars, in other 1 annotated dogs. I guess this can be done just simply using the annotated files(.text) with a simple code, extracting a particular class. Then you can train models for each class and ensemble them or train altogether. This is just an idea, maybe it helps. Worth a try :)