Multimodal Chain-of-Thought Reasoning in Language Models

By Zhuosheng Zhang et al

Published on May 10, 2024

Read the original document by opening this link in a new tab.

Table of Contents

1. Introduction
2. Background
3. Challenge of Multimodal-CoT
4. Multimodal-CoT

Summary

Multimodal Chain-of-Thought Reasoning in Language Models explores the incorporation of language and vision modalities into a two-stage framework to enhance reasoning capabilities. The study delves into the challenges of CoT reasoning in different modalities and proposes a method that leverages vision features to generate effective rationales and improve answer inference accuracy. The proposed Multimodal-CoT framework demonstrates state-of-the-art performance on the ScienceQA benchmark, highlighting the benefits of using multimodal information for better reasoning outcomes.