A Benchmark for Multimodal Fact Verification with Explainability Through 5W Question-Answering

By Megha Chakraborty et al

Published on Oct. 31, 2023

Read the original document by opening this link in a new tab.

Table of Contents

Abstract
Introduction
FACTIFY 3M - An Illustration
5WQA: Explainability
Related Works - Data Sources and Compilation
Paraphrasing Textual Claims

Summary

The document introduces FACTIFY 3M, a dataset of 3 million samples focusing on multimodal fact verification with explainability through 5W question-answering. It highlights the importance of combating disinformation and the need for efficient fact verification in the era of social media. The dataset includes textual claims, paraphrased claims, associated images, visual paraphrases, pixel-level image heatmaps, and 5W QA pairs. The document discusses the complexities of multimodal fact verification and the lack of substantial effort in this area. It also mentions the generation of adversarial fake news stories to benchmark fact verification systems. Additionally, it touches upon related works in automatic fact verification and the compilation of datasets for support, neutral, and refute categories. The document emphasizes the importance of QA-based explanation, heatmap-based image explainability, and adversarial assertion in the context of multimodal fact verification.