Read the original document by opening this link in a new tab.
Table of Contents
1. INTRODUCTION
2. EMPIRICAL OVERVIEW
3. LINEAR MODELS OF SPACE AND TIME
4. ROBUSTNESS CHECKS
Summary
The paper explores the capabilities of large language models (LLMs) in learning coherent representations of space and time. It discusses the linear representations of space and time across various datasets, showing evidence that LLMs can encode spatial and temporal coordinates. The study uses probing experiments to analyze the quality of representations and their sensitivity to prompting. Results indicate that LLMs can learn spatial and temporal information linearly and are fairly robust to changes in prompting. The paper also includes robustness checks to evaluate the generalization of the learned features and discusses the implications of the findings.