Fine-Tuning LLMs
Prompting gets you 80% of the way. Fine-tuning gets you the rest. Learn when to fine-tune versus using RAG, how parameter-efficient methods like LoRA and QLoRA work, how to curate training data, run training jobs, evaluate results, and deploy custom models into production.
What you'll learn
Course outline
Free — start now
Full course — $59 one-time
Hugging Face and Training Setup
Running and Monitoring Training
Evaluating Fine-Tuned Models
Deploying Custom Models
RLHF and DPO: Aligning Model Behaviour
Get the full course
All 8 lessons. Train custom language models with LoRA, QLoRA, and DPO — from dataset to deployment.
Written by the RadarTrek editorial team · Reviewed June 2026
About this course
Fine-tuning a large language model means taking a pre-trained model — GPT-4o mini, Llama 3, Mistral, Claude — and training it further on your own dataset so it learns the specific patterns, tone, vocabulary, or reasoning style your application requires. Most applications do not need fine-tuning: prompt engineering and RAG (retrieval-augmented generation) handle the majority of customisation needs far more cheaply and flexibly. But when you have a domain-specific task where quality consistently falls short despite careful prompting, or when you need to reduce token costs by using a smaller model that has been specialised for your use case, fine-tuning becomes the right tool.
Fine-Tuning LLMs for Builders is a practical course for engineers who want to move beyond prompt engineering and understand what fine-tuning actually involves: preparing training data in the correct format, choosing the right base model and fine-tuning method (full fine-tune vs LoRA vs QLoRA), running training jobs via the OpenAI API or open-source alternatives like Axolotl, evaluating your fine-tuned model against your baseline, and deploying it to production. By the end of this course you will have a complete pipeline from raw examples to deployed model, and you will understand the tradeoffs that determine when fine-tuning is worth the investment.
Frequently asked questions
When should I fine-tune instead of using prompt engineering?
Fine-tuning is worth considering when: your task has a very specific format or style that cannot be reliably achieved with prompting, you have hundreds or thousands of high-quality input-output examples that would take too many tokens to include as few-shot examples, you need a smaller cheaper model to perform at a larger model's quality on a narrow task, or you need to bake in domain knowledge too voluminous for a context window. Fine-tuning is NOT worth it when: you have fewer than a few hundred examples, your use case is general enough that a large frontier model handles it well, your requirements change frequently (fine-tunes take time to retrain), or the main issue is factual accuracy rather than style (use RAG instead). Most applications should exhaust prompt engineering and RAG before considering fine-tuning.
What format does training data need to be in?
OpenAI fine-tuning requires training data in JSONL format — one JSON object per line, each containing a "messages" array with the system, user, and assistant turns that represent your desired input-output pairs. The minimum recommended dataset size is 50 examples, with 100-500 being typical for meaningful improvement. Quality matters far more than quantity — 100 high-quality, diverse, correctly labelled examples outperform 1,000 noisy ones. For open-source fine-tuning frameworks (Axolotl, LlamaFactory), formats vary but most support Alpaca-style (instruction + input + output) or ShareGPT-style (multi-turn conversation) JSONL. Always hold out 10-20% of your data as a validation set — do not train on everything.
What is LoRA and how is it different from a full fine-tune?
A full fine-tune updates all the parameters of the model, which requires significant GPU memory and compute — fine-tuning a 7B parameter model with full fine-tuning needs 40-80GB of VRAM. LoRA (Low-Rank Adaptation) instead inserts small trainable matrices into the model's attention layers and only trains those, leaving the original weights frozen. This reduces trainable parameters by 99%+ while achieving most of the performance of full fine-tuning. QLoRA (Quantized LoRA) additionally quantises the frozen base model weights to 4-bit, enabling fine-tuning of 7B-13B models on a single 24GB GPU. For most practical applications, QLoRA gives you 90%+ of the benefit of full fine-tuning at a fraction of the cost.
How do I evaluate whether my fine-tuned model is actually better?
Evaluation is the most commonly skipped step in fine-tuning projects, and the most important. Before training, define your success metric with held-out test examples that the model never sees during training. Common evaluation approaches: automated metrics (BLEU, ROUGE for text generation, accuracy for classification), LLM-as-judge (using a capable model like GPT-4o to evaluate your fine-tuned model's outputs on a rubric), and human evaluation (if the task requires human judgment, there is no substitute). Compare your fine-tuned model against your best prompting approach on the same test set. If the fine-tuned model does not clearly outperform careful prompting, reconsider whether fine-tuning was the right investment.
How much does fine-tuning a model cost?
OpenAI fine-tuning (the easiest starting point): training costs vary by model — gpt-4o-mini fine-tuning costs approximately $0.30 per million tokens of training data. A typical training run with 1,000 examples of 500 tokens each costs roughly $0.15 to train. Inference on your fine-tuned gpt-4o-mini model is priced like the base model plus a small hosting surcharge. For open-source fine-tuning: a 4-hour QLoRA run on a 7B model costs $8-20 on RunPod or Lambda Labs (A100 GPU). Deploying the resulting model on your own infrastructure costs $0.40-1.50/hour for a GPU instance that handles a few requests per second. For most applications, the OpenAI API fine-tuning is the fastest path to evaluate whether fine-tuning helps before investing in open-source infrastructure.