Once-For-All: Train One Network and Specialize It for Efficient Deployment

By Han Cai et al

Published on June 10, 2020

Read the original document by opening this link in a new tab.

Table of Contents

ABSTRACT
INTRODUCTION
RELATED WORK
METHOD
PROBLEM FORMALIZATION
ARCHITECTURE SPACE
TRAINING THE ONCE-FOR-ALL NETWORK
Progressive Shrinking
CONCLUSION
ACKNOWLEDGEMENTS
REFERENCES

Summary

This paper presents the concept of training a once-for-all (OFA) network to support diverse architectural settings for efficient deployment. The challenge of efficient inference across various devices and resource constraints, especially on edge devices, is addressed by proposing a solution that reduces the computational cost. The OFA network allows quick selection of specialized sub-networks without additional training. A progressive shrinking algorithm is introduced for training the OFA network, enabling support for different dimensions such as depth, width, kernel size, and resolution. The paper demonstrates the effectiveness of OFA in outperforming state-of-the-art NAS methods on various edge devices while reducing GPU hours and CO2 emissions. Overall, the OFA approach offers a flexible and efficient solution for deploying deep neural networks on diverse hardware platforms.