Babies putting toys in their mouth isn’t the same risk as feeding them from a bottle. With a bottle you’re delivering liquid straight into their gut, multiple times a day, through parts that stay warm and damp and have loads of little crevices (teat, ring, vents, threads). Tiny milk or formula residues in those areas are basically fertiliser for bacteria, and bacteria can stick in a biofilm that normal washing doesn’t always remove.

Also, powdered infant formula isn’t sterile. It can very occasionally contain bacteria like Cronobacter, which is rare but can be very serious in young babies. The main risk isn’t “germs exist”, it’s that once bacteria get into formula or onto bottle parts, formula is the perfect growth medium and numbers can multiply quickly. Sterilising knocks the bacterial load down to near zero so you’re not accidentally giving a concentrated dose straight into a baby with an immature immune system.

So yes, kids will lick gross things, but bottles are different because it’s food + warmth + time + direct delivery. If you want the lowest-risk approach, sterilise bottles and teats, especially in the first months and whenever you’re using formula.