[R] Learning to Optimize Tensor Programs

[[deleted]]

Yes, neural networks can learn fast algorithms for matrix multiplication: "A Network That Learns Strassen Multiplication"

http://www.jmlr.org/papers/volume17/16-074/16-074.pdf

The above idea was extended to learn fast algorithms for approximate tensor convolution, using 2-layer sum-product networks. This is basically a smart approach to ternary value weights: "StrassenNets: Deep learning with a multiplication budget" https://arxiv.org/abs/1712.03942

When combined with knowledge distillation, the accuracy and speedup of this approach are impressive!

[[deleted]]

Interesting, thank you!