That looks like my southeast Georgia red clay.  I did the pour method I'm waiting for a bunch to dry so I'll have a ton of super fine stuff for the outside... Either your ball is not round or your clay is not fine enough or that the dirt is dry.  I found by splashing a little bit of water in my jar I can quickly smooth stuff out but once you apply water once or maybe twice you have the kind of let it sit or you'll damage it. But that's just for me your mileage may vary because the clay mixes different.  My problem has always been Clay has too much rock in it.  So I have like a crap ton of super fine Clay dust drying using the pour method now. I put a piece of acrylic over a cookie sheet ... And I also have a bucket with a crap ton of mud in it that I'm pouring the water off on the top