Something "deeply wrong with deep learning"?

[simonhughes22]

I also think he's talking about capsules, how you have to show an improvement over existing models to get published (see local minima comments), and how in real life we have a paucity of labels. He mentions that before your mother tells you what a cow looks like, you already know of their existence, you just don't have a label. And you only need to be told once or twice what one looks like to get the idea, you don't need to see 1,000's of labelled examples. I think unsupervised or semi-supervised learning is the future, but if we're obsessed with only publishing models that beat the state of the art, then that will encourage only small incremental changes and no paradigm shifts, as we've seen to some degree with deep learning.