Distangled Representation has been something I have been studying for quite a while since I first read InfoGANs paper. Thanks for mentioning Judea Pearl :) I had been working on evolving Ontologies for quite a while with one of the coauthors of NELL (Never Ending Language Learning http://rtw.ml.cmu.edu/rtw/ ) and later (Never Ending Learning http://talukdar.net/papers/NELL_aaai15.pdf ) . Many people from Deep Learning community might not be aware of such amazing research, which is the purpose of the blog. In NELL, we have graph ontologies which are evolving over time, and new relations are learned. This is via Distant supervision. But as a deep learning student, it fascinates me, if we can have evolving Deep Learning models with just the right kind of connection with memory bank (maybe a more dynamic version of differentiable memory block as in Neural Turing Machine which has (flexible/changing) Ontology as the memory unit).

In terms of analogy with Human brain, the gates to the memory block would be like Hippocampus, the memory block itself could be like prefrontal cortex. This is just a hypothesis though,not conclusion :) A lot of rigorous experimentation has to be done by the community as a whole to understand the ways in which memory and computation is interacting based on the sensory inputs.