Supervised Fine-Tuning (SFT)
What is Supervised Fine-Tuning (SFT)? Supervised fine-tuning is a training strategy where a pre-trained language model is further refined on a carefully curated dataset of prompt-response pairs. The primary goal is to “teach” the model how to generate appropriate, contextually relevant, and human-aligned responses. Key points about SFT include: Data Curation: The model is exposed to a dataset that contains high-quality examples—often created by human annotators—that demonstrate the desired behavior (e.g., step-by-step reasoning, correct coding outputs, or helpful dialogue responses). Instruction Following: By training on these examples, the model learns to interpret prompts as instructions and produce answers that mimic the reasoning and style of the training data. Limitations: While SFT works well to instill basic response quality, it is typically limited by the dataset’s scope and may not encourage the model to “think” beyond what is explicitly provided. Furthermore, excessive fine-tuning can lead to overfitting and reduce the model’s ability to generalize to unseen tasks. For many contemporary language models, SFT is the standard method used to bridge the gap between raw pre-training and interactive, user-facing performance. ...