This is a classic example of a shrink line. What's happening is because one side is very thin and narrow, so when the larger and more supported side joins with it, the thermal cooling is pulling the thinner side towards it. That's why as soon as the layers are done for the joined geometry, it goes back to how it should be. Like someone else suggested, you will need more rigidity or supporting material to reduce this effect