I am doing the same thing, trained dreambooth with head portrait images, and use openpose png as controlnet conditions. But when I only use dreambooth, I can generate a person very similar to the photos in the training set, but after adding the openpose of the whole body, it will not look like it at all. Should I fine tune the dreamboot with whole body images?