AutoHint: Automatic Prompt Optimization with Hint Generation

By Hong Sun et al

Read the original document by opening this link in a new tab.

Table of Contents

1. Introduction
2. Related Work
3. Proposed Method
3.1. Overview
3.2. Inference and Get Residual Data
3.3. Hints Generation
3.4. Sampling Strategies
3.5. Hints Summarization
4. Experiments
4.1. Experimental Setup
4.2. Experiment Results and Analysis
4.3. More Iterations
4.4. Cost Analysis
5. Conclusion

Summary

AutoHint is a framework that aims to optimize prompts for Large Language Models by generating hints automatically. It leverages zero-shot and few-shot learning to enhance the quality of prompts, with a focus on enhancing task-specific instructions. The method involves sampling incorrect predictions, deducing hints, summarizing hints, and refining prompts iteratively. Experimental results on the BBII dataset show significant improvements in accuracy and balanced accuracy across various tasks. The framework demonstrates effectiveness under zero-shot settings and provides insights into sampling strategies for generating informative summaries. Cost analysis highlights opportunities to optimize expenses in the process. Overall, the proposed AutoHint framework shows promise in automatic prompt optimization for language models.