Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition

By X. Zheng et al

Read the original document by opening this link in a new tab.

Table of Contents

1. Introduction
2. Transformer-Based LMs
3. Methodology
4. Experimental Setup
5. Experimental Results

Summary

The paper discusses the adaptation of GPT, GPT-2, and BERT language models for automatic speech recognition (ASR). It presents results using fine-tuned GPT, GPT-2, and their combination for ASR tasks. The study compares unidirectional and bidirectional LMs, highlighting the importance of accurate language prior probabilities. Experimental results show significant improvements in ASR performance using the combined models. The paper provides insights into LM combination techniques and bidirectional LM output probability conversion for ASR applications.