Teacher-Student Architecture for Knowledge Learning: A Survey

By Chengming Hu et al

Read the original document by opening this link in a new tab.

Table of Contents

1. Introduction
2. Learning Objectives
2.1 Knowledge Distillation
2.2 Knowledge Expansion
2.3 Knowledge Adaptation
2.4 Multi-Task Learning
3. Knowledge Formulation
3.1 Knowledge Construction
3.2 Knowledge Optimization

Summary

This paper provides a comprehensive survey on Teacher-Student architectures for knowledge learning objectives, including knowledge distillation, expansion, adaptation, and multi-task learning. It discusses the construction and optimization of knowledge during the learning process, detailing various architectures and learning schemes. The paper also covers applications of Teacher-Student architectures in classification, recognition, and generation tasks, and explores future research directions in knowledge learning.