Adaptive Multi-Modalities Fusion in Sequential Recommendation Systems

By H. Hu et al

Published on Oct. 21, 2023

Read the original document by opening this link in a new tab.

Table of Contents

1. INTRODUCTION
2. RELATED WORK
3. PRELIMINARIES
4. APPROACH
4.1 Multimodal Sequence Graph Construction
4.2 Node Feature Aggregation
4.3 Dual Attention Mechanism
5. EXPERIMENTAL EVALUATION
6. CONCLUSIONS
7. REFERENCES

Summary

Adaptive Multi-Modalities Fusion in Sequential Recommendation Systems proposes a graph-based approach named MMSR to fuse modality features in an adaptive order, enabling a spectrum from early to late modality fusion. The method outperforms state-of-the-art models across six datasets and demonstrates robustness to missing modalities. By constructing Multi-modal Sequence Graphs for each user, an adaptive merging order facilitates simultaneous consideration of both sequential and cross-modal aspects. The dual attention mechanism independently aggregates heterogeneous and homogeneous node information, enhancing the efficacy of fusion.