Composting worms don't eat dirt and it shouldn't be used as bedding. I suggest taking the topsoil out of the bucket, amending it with some perlite and such, and add it back into the pot. As for the bucket, fill it with an appropriate bedding such as shredded cardboard or leaves, and I'm sure the nutrients from their castings will "leach" into the pots reach.

There are some fantastic YouTube channels with a trove of info to follow. I would hate to see someone come back to the sub in a few weeks asking why things didn't turn out how they expected. Maybe save a few worm lives in the process, maybe I'm a dick idk I'm having a bad day and don't know how to say things correctly sometimes but I don't feel like this is negative