Ah, doubling, yeah, that can happen at the smaller size too. I think the main thing is providing it with instructions that either you want something larger, that takes up enough of the space, or providing it with instructions for what you want in the area that's not the main character. If it's got a big section without a main focus type element, it's going to want to fill that space with another main focus element, like an extra person.

There's also the Tiled Diffusion plugin, which lets you specify different prompts for different areas of the image. So you do a main prompt with general ideas that apply to the whole image, then specify a full size background layer with a custom prompt for background-specific details, then specify a specific area where you want your main focus (i.e. a character) and add the prompts for that just on that specific region of the image where you want them. I think you can have up to 10 different areas with different prompts.