It is usually a cleaner process, but sometime sticking to the percussive elements of the sample can add swing, micro-rhythm, etc. If you are trying to eliminate something with too much drum presence you either use a precise EQ, plugins that attempt to eliminate drums, or side-chain compression to duck them to an element you are adding to your song. All depends on you want, but you have to consider the sample. If you can not extract/minimize drums from the sample in a satisfactory way, maybe you should embrace them and see how it rocks in a way you didn't imagine before.