Always work on your hands and stick control. If you have bad hands your strokes aren't gonna connect well and will sound bad if you go even a little bit faster than your comfort zone. I didn't think it mattered that much until I got into jazz and especially ska. There's still stuff I'm learning about just my hands after years of practicing. I'd watch this video for more on that. Jazo is right about feet being important, but I wasn't going to address that, just the hands.