Read the original document by opening this link in a new tab.
Table of Contents
ABSTRACT
INTRODUCTION
I NTRODUCTION
A PAC L EARNABILITY FRAMEWORK FOR IN-CONTEXT LEARNING
GUARANTEES ON IN-CONTEXT LEARNING
THE ANALYZED LATENT CONCEPT HYPOTHESIS CLASS
Summary
In-context learning is a surprising and important phenomenon that emerged when modern language models were scaled to billions of learned parameters. This paper proposes a PAC based framework for in-context learnability and provides finite sample complexity results for the in-context learning setup. The study reveals how frozen pretrained models learn from prompts that do not resemble their pretraining distribution. The analysis focuses on a latent task inference perspective of in-context learning and sheds light on the effectiveness of in-context learning in pretrained language models.