Fine Tuning LLMs

• Adapting pre-trained model to specific tasks.
• Upates model weights

Requirements:

1. Training data format: json – <prompt, completion>
2. Hyperparameters:

Challenges:

• High Quality Data; Data Format
• Setting hyper parameters carefully

Parameter Tuning:

1. Retrain all parameters
2. Transfer Learning
3. Parameter Efficient Fine-tuning (PEFT)
   • Low-Rank Adaptation (LoRA): https://github.com/svpino/lora-vit/blob/main/lora.ipynb
   • QLoRA: Efficient Finetuning of Quantized LLMs: https://github.com/artidoro/qlora

References:

1. https://learn.microsoft.com/en-us/ai/playbook/technology-guidance/generative-ai/working-with-llms/fine-tuning