This chapter covers
- Why you might need to fine-tune language models
- The product manager’s role in the fine-tuning process
- Creating data for fine-tuning
- Domain, supervised, and instruction fine-tuning
In the previous two chapters, you learned about prompt engineering and retrieval-augmented generation (RAG)—two powerful techniques to supply a language model (LM) with specialized knowledge during inference. However, if your app requires expert-level LM performance, you might soon hit the limits of these techniques. Prompt engineering will quickly start to feel like consulting a high-school graduate who possesses solid general knowledge and can converse across many topics, but struggles with highly specialized or nuanced subjects. RAG is like giving an encyclopedia to that same person. Now, they can offer more specialized responses, but once you dig deeper, you find gaps in their terminology, reasoning, and overall understanding. Thus, Alex observes an alarming drop in usage as users soon grow frustrated by the need to repeatedly tweak and refine the model’s outputs.