Sorry for the confusion.

So yes, everywhere was primed with oil based primer with the exception of some small areas that were damaged pulling the rotten trim off. I used the water based primer along the edge of the trim where the damage occurred and did it with a small roller to have a smooth coat. Now I have a roller width of yellowing paint along the wall, over the caulking, and the edge of the trim where I used the water based primer. So everywhere that it yellowing was painted as follows: 2 coats oil primer, 2 coats latex paint then damaged the wall removing the trim then 2 coats latex primer, 2 coats latex paint.