Tuning Techniques for Cost Optimization

Summary

This chapter focuses on methods to enable efficient fine‐tuning of LLMs to downstream tasks through cost optimization. These techniques are collectively referred as parameter‐efficient fine‐tuning, or PEFT. The goal of PEFT is to tune only a small subset of parameters in the LLM instead of all parameters like in full fine‐tuning. This provides substantial reductions in computation, memory, and storage without significantly impacting model performance. The chapter discusses the landscape of methods, analyzes their connections, and discusses practical considerations. It provides an in‐depth discussion of parameter‐efficient modules such as adapters, which encapsulate task‐specific representations to reduce the optimization footprint during fine‐tuning, such as prompt tuning, prefix tuning, and P‐tuning. The chapter also discusses some cost implications of deciding to train or fine‐tune our own LLMs.