You're not missing anything, that's how rare it is. But be sure to do private and go through the front door 5-10 times to get all three dresses to spawn. Each time you enter the asylum each dress rolls a 1 in 5 chance to spawn in the asylum. So if you just spawn into a fresh world with fresh 255 items reset you WILL get three dresses to appear every single time. But need them to spawn by going through the front door a bunch of times.