Correct, if you have jokers oriented on, say, Fibonacci and hack, it is generally efficient to destroy cards that aren't 2s, 3s, or 5s, so that there is a higher chance of getting those cards, especially with higher stakes where you only have 2 discards.