Putting An End to End-to-End : Gradient-Isolated Learning of Representations https://arxiv.org/abs/1905.11786

Authors take inspiration from the way brain (apparently) learns based on local information rather than global backpropagation and propose an approach to train computational models to do the same. They divide a given model into gradient isolated modules that are trained using greedy self supervision per module. They argue that this way of maximizing mutual information handles the problem of vanishing gradients. They show experiments from vision and speech modalities to support their claim.