AI Model Fine-tuning: The Basics

What Is Fine-Tuning?

Fine-tuning is the process of customizing a pre-trained AI model to specialize in a specific task or speak in a particular style. Instead of training a model from scratch (which costs millions of dollars), you take a ready model like Llama 3 or Mistral and retrain it on specific data.

Practical example: Llama 3 knows Arabic, but it does not know your company policies, products, or communication style. After fine-tuning, the model becomes an expert in your specific domain.

When Do You Need Fine-Tuning?

Need	Best Solution	Reason
Answering questions from your documents	RAG (no Fine-Tuning)	Easier and cheaper — data speaks
Custom writing style	Fine-Tuning	Model learns the style
Classifying data by your criteria	Fine-Tuning	Needs to understand your context
Automated technical support	RAG first, Fine-Tuning if needed	Start with the easier option
Specialized translation	Fine-Tuning	Terminology differs by domain
Data extraction from documents	Fine-Tuning	Needs to learn the format
General conversation	Nothing needed	Base models are sufficient

Golden rule: Try RAG first. If results are unsatisfactory, move to Fine-Tuning.

Types of Fine-Tuning

1. Full Fine-Tuning

Updates all model parameters. Best results but requires:

• GPU with massive memory (A100 80GB or more)
• Large training data (thousands of examples)
• Long time (hours to days)

2. LoRA (Low-Rank Adaptation)

Updates only a small fraction of parameters. The practical choice:

• Works on smaller GPUs (RTX 4090 is sufficient)
• Less data needed (hundreds of examples)
• Much faster
• Results very close to full fine-tuning

3. QLoRA (Quantized LoRA)

LoRA with a quantized model (4-bit). Most efficient:

• Works on 24 GB VRAM
• Nearly same quality as LoRA
• 60% less memory consumption

Type	VRAM Required (7B)	VRAM Required (13B)	Result Quality
Full Fine-Tuning	28+ GB	52+ GB	Best
LoRA	16 GB	24 GB	Excellent
QLoRA	8 GB	16 GB	Excellent

Preparing Training Data

Data Format

The most common format is JSONL (one JSON line per example):

{"instruction": "What is your return policy?", "output": "Our return policy allows returns within 14 days of purchase provided the product is in its original condition."}
{"instruction": "How long does shipping take?", "output": "Shipping within Riyadh takes 1-2 business days. Other cities take 3-5 business days."}
{"instruction": "Do you support installment payments?", "output": "Yes, we support installments through Tamara and Tabby with 0% interest for 4 months."}

Required Data Volume

Task	Minimum Examples	Ideal Examples
Text Classification	100	500-1,000
Question Answering	200	1,000-3,000
Writing Style	500	2,000-5,000
Data Extraction	300	1,000-2,000

Data Quality Rules

1. Diversity: Examples covering all expected cases
2. Accuracy: Every answer must be 100% correct
3. Format: Same format across all examples
4. Balance: Similar number of examples per category
5. Language: If the target is Arabic, all examples in Arabic

Practical Fine-Tuning Steps (QLoRA)

1. Environment Setup

pip install torch transformers datasets peft bitsandbytes trl accelerate

2. Load Model in 4-bit

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(
 load_in_4bit=True,
 bnb_4bit_quant_type="nf4",
 bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
 "meta-llama/Meta-Llama-3-8B",
 quantization_config=bnb_config,
 device_map="auto",
)

3. Configure LoRA

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
 r=16,
 lora_alpha=32,
 target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
 lora_dropout=0.05,
 bias="none",
 task_type="CAUSAL_LM",
)

model = get_peft_model(model, lora_config)

4. Training

from trl import SFTTrainer, SFTConfig

training_config = SFTConfig(
 output_dir="./results",
 num_train_epochs=3,
 per_device_train_batch_size=4,
 gradient_accumulation_steps=4,
 learning_rate=2e-4,
 logging_steps=10,
 save_strategy="epoch",
)

trainer = SFTTrainer(
 model=model,
 train_dataset=dataset,
 args=training_config,
)

trainer.train

5. Save the Model

model.save_pretrained("./my-finetuned-model")
# Convert to GGUF format for use with Ollama
# python convert_hf_to_gguf.py ./my-finetuned-model

Evaluating Results

After training, evaluate the model:

1. Manual testing: Ask 20-30 questions not in the training data
2. Comparison: Compare answers with the original model
3. Classification accuracy: If the task is classification, measure accuracy on a test set
4. Human review: Have someone review a sample of answers

Common Mistakes

• Overfitting: Model memorizes training data instead of learning — Solution: increase diversity and reduce epochs
• Too little data: Less than 100 examples is usually insufficient — Solution: collect more data
• Dirty data: Errors in answers transfer to the model — Solution: manual review
• High learning rate: Model forgets its prior knowledge — Solution: start with 2e-5 or 2e-4

Server Requirements

For QLoRA fine-tuning on a 7B model:

• 8 vCPU minimum
• 32 GB system RAM
• GPU with 24 GB VRAM (or CPU only with patience)
• 100 GB+ fast SSD

A Cloud Server from CLOUD (cloud.net.sa) on the Enterprise plan with additional RAM provides the necessary resources. Your data stays in Riyadh — critically important if your training data is sensitive.

Bottom Line

Fine-tuning is not for everyone or every use case. Start with RAG, and if that is not enough, move to QLoRA. Prepare clean and diverse data, start small, and evaluate results carefully. A well-tuned model outperforms a model 10 times its size — if the data is right.

---

Get Started

Ready to run AI on your own server? Explore AI hosting solutions or configure your server with the specs you need.