Read the original document by opening this link in a new tab.
Table of Contents
1. Introduction
2. Related Work
3. Model
4. Inputs and Outputs Handling
5. Discretization
6. Experiments
Summary
Neural Random-Access Machines (NRAM) propose a new neural network architecture capable of manipulating and dereferencing pointers to an external variable-size random-access memory. The model is trained using backpropagation from input-output examples. It shows promising results in solving algorithmic tasks that involve pointer manipulation and dereferencing, operating on data structures like linked-lists and binary trees. The paper discusses the model's core primitives and its potential for learning algorithms. It also highlights the challenges of training extremely deep and nonlinear models, emphasizing the importance of optimization methods for better results. Related work in the field of learning algorithms is reviewed, showcasing models like Neural Turing Machines and Stack-Augmented Recurrent Neural Networks. The NRAM model is described in detail, including the controller, registers, and memory tape. The paper outlines the inputs and outputs handling, as well as the discretization process for efficient computation. The training procedure involved Adam optimization and curriculum learning to train deep networks effectively on complex problems.