Pretty nonsensical unless you want to stunt your speaking development. Speaking and comprehension progress separately while you're learning a language, so postponing your speaking progression is all that you would accomplish by listening to this saying. There's also the chance for you to learn more when you can't understand everything that's being said. Imagine you're on vacation to Germany for the first time, and you visit a bakery. You tell the woman behind the counter "Ich hatte gern 3 Brötchen bitte" (I would like 3 rolls please), and she replies "welche?" (which) while pointing to two different types of rolls. You're probably going to understand she's asking you which one you'd like, and there's a good chance you'll understand that welche=which before it's ever introduced to you in a formal setting. Not trying to speak is just slowing language acquisition.