Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems

By Benjamin Coleman et al

Read the original document by opening this link in a new tab.

Table of Contents

1 Introduction
2 Contributions
3 Preliminaries
4 Analysis of Feature Multiplexing
5 Parameter Efficiency in The Dimension-Reduction Framework
6 Trajectories of Embeddings During SGD for Single-Layer Neural Networks

Summary

Learning high-quality feature embeddings efficiently and effectively is critical for the performance of web-scale machine learning systems. Unified Embedding introduces a simple yet highly effective framework, Feature Multiplexing, where one single representation space is used across many different categorical features. This approach leads to significant improvements in offline and online metrics compared to competitive baselines across web-scale search, ads, and recommender systems. The paper presents detailed theoretical analysis and experimental results supporting the effectiveness of the Unified Embedding approach.