Memory Augmented Large Language Models Are Computationally Universal

By Dale Schuurmans et al

Published on Jan. 10, 2023

Read the original document by opening this link in a new tab.

Table of Contents

1 Introduction
2 Stored instruction computer
3 Universal Turing machine
4 Simulating U15,2 with a prompt program

Summary

This paper discusses the computational universality of transformer-based large language models when augmented with external memory. It explores the concept of simulating a universal Turing machine using a specific large language model, Flan-U-PaLM 540B, combined with an associative read-write memory. The paper presents a prompt program designed to drive the system to simulate a universal Turing machine U15,2. By establishing computational universality for the Flan-U-PaLM 540B model augmented with memory, it shows the potential to simulate any algorithm on any input without modifying the pre-trained weights of the language model.