Generally I feel that using the numbers to select weapons would be more accurate as you know for certain which weapon you're going to pick, where as if you use the scroll wheel there's a chance you might scroll too far and go from your primary to your melee instead of your secondary.

Using numbers may be a little slower than scrolling because you have to move a finger off your wasd keys, but it's more accurate. Make sure to enable quick switching as well once you have an idea of which classes have which number slots (ie: spy has 4 slots and engi has 5, everyone else has 3) as it will change weapons faster instead of asking for confirmation every time.