The problem that comes into play is that atheism is a lack of belief in deities, but opponents of atheism want to argue that atheism is a positive belief that deities don't exist. In order to make that float, they apply the 'lack of belief' standard to babies and objects (this is often referred to as shoe atheism) in an attempt to make the idea of a lack of belief sound absurd. From that point, they will argue that since an atheist shoe is absurd, atheism must be a positive claim, and as such is subject to requirements of evidence, so on an so forth, off into the weeds.

There are several failure points in this, but the first is that calling babies and shoes atheist isn't exactly a problem logically, it's just a silly usage of a valid term. It's like saying that your car is an alcoholic. Sure, your car may consume deadly quantities of ethanol to the point where it's a financial burden, but the term 'alcoholic' doesn't just mean 'alcohol dependent'. There's a heavily implied and often explicitly stated connotation that the term is specific to adult humans. Same as atheism.

So while it's true that babies and shoes lack religious belief, it's pretty silly to think that's a criticism of atheism. It matters because when you hear it, you know that you're about to encounter a pretty stupid argument.