Read the original document by opening this link in a new tab.
Table of Contents
Abstract
1 Introduction
2 Related Work
3 Capacities of a Perceptron
4 Information Theoretic Model
5 Learning Method
5 Networks of Perceptrons
Summary
This paper discusses a practical approach to estimating the maximum size of a neural network based on MacKay's information theoretic model of supervised machine learning. It presents rules to determine network architectures' capacity and proposes a heuristic method for estimating network capacity requirements. The paper also explores the consequences of incorrect network sizing, emphasizing the importance of memory and computation requirements in learning tasks. The authors draw on previous work on perceptrons, VC dimension, and Rademacher complexity to analyze neural network capacities and generalization. The paper concludes with future research directions in machine learning theory.