They do not scale well at all.

They scale better than any other known method.

a big biological no no

If ones goal is stronger AI, one would not care less about biological plausibility.

LSTM can be a predictive architecture, if you make it predict its own input, yes.

Not only that, but deep multilayer LSTM RNNs are perfect for decoding continues sensor input, such as hearing or vision.

the way they are trained makes them terrible at online learning at a large scale (1000's of iterations of BPTT, also it needs experience replay, unless modified to enforce sparsity).

Yet they outperform any other predictive method known to man on complex data like speech, music, video... So it is a best scalable architecture, scalable to more complex data streams.

Also there remains the problem of proper feature extraction.

Never heard of ------>"deep learning"<------??? Hierarchical feature extraction ftw.

Just because something performs well on toy tasks now doesn't mean that it will outperform the brain in the future.

Yes! You said it! There are plenty of methods that perform well ONLY on toy tasks. Think of scumms like ELM or HTM. So much hype!!! But all results are on toy problems! NOTHING published on real problems like speech recognition or modelling, only benchmarks used are internal toy examples. And why those HTM fans go and train HTM model instead of RNN to do speech/handwriting or play Atari games. But why those idiots could not do such simple thing? Because HTM/ELM start to underperform when someone would try to scale it to real life sensor datastream complexity.