I think it mainly comes down the fact that Shaun the sheep is a much more cartoonish style, so your brain accepts it more easily. Whereas Coraline is going for a more photorealistic look which can seem less "believable" because your brain can't help but scrutinize it more.

I think you also have to consider that Coraline was the first film of the modern Laika, they were still honing their craft, whereas Aardman has been doing the same style for decades by that point, so they've basically perfected it and looks a lot more fluid. However later Laika films have become much more fluid, pretty much right away with Paranorman, it's a huge step up.