- Process of adjusting the parameters of a pretrained model on a specific dataset to enhance its performance on a specific task.
- Used to transform a foundation model into a specilized one for a particular use case.
- Start by selecting an LLM that has been pretrained on a broad dataset.
- Due to its initial training phase, the selected model already possesses a wide-ranfing understanding of language.
- The next step is to identify the specific task that requires improvement.
- These tasks can range from improving a model's knowledge of a specific domain to enhance its conversational skills.
- The model is then further trained or fine-tuned using a dataset closely related to the desired task.
- This dataset should be representative of the scenarios in which the model will be applied, containing examples of the input-output patterns it’s expected to learn.
- During the fine-tuning process, the model’s parameters are adjusted to minimize errors in its output compared to the expected results.
- This process is iterative, with multiple cycles of training and adjustment continually refining the model’s ability to perform the chosen task with increased precision.
- The fine-tuning is complete when the model achieves a satisfactory level of performance, as determined by relevant metrics and evaluations.
- InstructGPT - Fine-tuned by OpenAI
- ChatGPT
- Koala - Fine-tuning meta's LLaMA
- StarCoder - Fine-tuned ona diverse and extensive code from GutHub on variety of programming language.
Next: ➡️ Phases of LLms