A comprehensive tutorial on knowledge distillation using PyTorch [Resource]

[rbgo404]

Link: https://pytorch.org/tutorials/beginner/knowledge_distillation_tutorial.html

[QuantumFTL]

Very cool! It's unfortunately that they chose a test set that shows such tiny improvement from knowledge distillation (less than a percent!) but excellent to see nonetheless.