Chapter 19 · Hands-on: build real projects
Fine-tune a small model with LoRA
LoRA fine-tuning trains small add-on weights on your examples, so a model learns your style without retraining everything.
Chapter 19 · Hands-on: build real projects
LoRA fine-tuning trains small add-on weights on your examples, so a model learns your style without retraining everything.
BeforeA cache is a place where a computer stores data.
After your fine-tuneShort answer: a small, fast memory. Tip: it speeds up repeat work.
A big model, already trained, with its weights frozen…
You will not retrain a whole model. LoRA freezes it and trains only small add-on layers on a few examples of how you want it to answer. Then you compare its answers before and after.
The LoRA lesson explained the idea: instead of changing every weight, train a few small extra ones. Now you do it. We use a very small open model so it works on an ordinary computer, with Hugging Face libraries.
Fine-tuning is good at teaching style and format: how to answer, the tone, the layout. It is poor at teaching new facts. For facts, use RAG from Chat with your PDFs: a simple RAG app.
Create a folder lora-lab with its own virtual environment (see Set up Python and a virtual environment), then install PyTorch, Transformers (Hugging Face’s model library) and PEFT (its LoRA library). PyTorch can be a large download; the PyTorch website’s install selector offers a smaller CPU-only command.
mkdir lora-lab
cd lora-lab
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install torch transformers peft Check: The line starts with (.venv) and the install ends without a red error.
Training data is a text file where every line is one example with a prompt (the question) and a response (the answer you want). This format is called JSON Lines. Save the lines below as train.jsonl in lora-lab. Every answer here has the same shape: Short answer: … Tip: …. That steady pattern is what the model will pick up.
Later, replace them with your own: a tone you like, a format, a fixed layout. Use the checker below the tutorial to catch typos. Twenty or more examples show a change; a few hundred are better for real work.
{"prompt": "What is a vector database?", "response": "Short answer: a store that finds text by meaning. Tip: try searching your own notes first."}
{"prompt": "What is an embedding?", "response": "Short answer: a list of numbers for a piece of text. Tip: similar meanings get similar numbers."}
{"prompt": "What is a chunk?", "response": "Short answer: a small piece of a longer document. Tip: keep the page number with it."}
{"prompt": "What is RAG?", "response": "Short answer: search first, then answer from what was found. Tip: always check the cited page."}
{"prompt": "What is a token?", "response": "Short answer: a small piece of text the model reads. Tip: about four English letters make one."}
{"prompt": "What is a model?", "response": "Short answer: a program whose numbers were learned from examples. Tip: more data usually helps."}
{"prompt": "What is loss?", "response": "Short answer: a number that says how wrong the model is. Tip: training tries to make it fall."}
{"prompt": "What is a prompt?", "response": "Short answer: the text you send to a model. Tip: clear instructions give clearer answers."}
{"prompt": "What is an agent?", "response": "Short answer: a model in a loop that can use tools. Tip: always cap the number of steps."}
{"prompt": "What is MCP?", "response": "Short answer: a standard way for AI apps to use tools. Tip: only connect servers you trust."}
{"prompt": "What is LoRA?", "response": "Short answer: small add-on weights trained on your examples. Tip: the big model stays frozen."}
{"prompt": "What is a virtual environment?", "response": "Short answer: a private box of packages for one project. Tip: activate it before you install anything."}
{"prompt": "What is overfitting?", "response": "Short answer: learning the examples instead of the pattern. Tip: test on questions it has not seen."}
{"prompt": "What is a parameter?", "response": "Short answer: one learned number inside a model. Tip: bigger models have more of them."}
{"prompt": "What is a context window?", "response": "Short answer: how much text a model can read at once. Tip: put the most useful text inside it."}
{"prompt": "What is temperature?", "response": "Short answer: a setting for how adventurous the writing is. Tip: lower is safer and more repetitive."}
{"prompt": "What is hallucination?", "response": "Short answer: a confident answer that is made up. Tip: ask for sources and check them."}
{"prompt": "What is a GPU?", "response": "Short answer: a chip that does many sums at once. Tip: a CPU can still train tiny models."}
{"prompt": "What is quantisation?", "response": "Short answer: storing a model's numbers with fewer bits. Tip: it saves memory."}
{"prompt": "What is an epoch?", "response": "Short answer: one full pass over the training examples. Tip: too many can cause overfitting."}
{"prompt": "What is a dataset?", "response": "Short answer: the examples a model learns from. Tip: clean examples matter more than many."}
{"prompt": "What is inference?", "response": "Short answer: using a trained model to make an answer. Tip: it needs far less compute than training."}
{"prompt": "What is a neural network?", "response": "Short answer: layers of simple units with learned connections. Tip: start tiny to see how it works."}
{"prompt": "What is fine-tuning?", "response": "Short answer: training a model a little more on your own examples. Tip: begin with a small model."}Check: The file has one complete JSON object per line, with no blank lines in the middle and no trailing commas.
Save this as finetune.py. The key parts:
HuggingFaceTB/SmolLM2-135M here, as an example; use any small model that fits your computer.get_peft_model adds LoRA layers (r=8 is their small size) and freezes everything else. print_trainable_parameters shows how few numbers are actually trained.-100), and nudges only the LoRA weights. It makes 3 passes (EPOCHS).my-lora.import json
import torch
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer
BASE = "HuggingFaceTB/SmolLM2-135M" # or any small model that fits your computer
EPOCHS, LR = 3, 2e-4
TEST_QUESTIONS = ["What is a loss function?", "What is a pipeline?", "What is a cache?"]
torch.manual_seed(0)
tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE)
def ask(question, max_new_tokens=40):
ids = tokenizer(f"Question: {question}\nAnswer:", return_tensors="pt")
with torch.no_grad():
out = model.generate(**ids, max_new_tokens=max_new_tokens, do_sample=False,
pad_token_id=tokenizer.eos_token_id)
text = tokenizer.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True)
return text.strip().split("\n")[0]
print("=== BEFORE fine-tuning ===")
for q in TEST_QUESTIONS:
print(f"{q}\n -> {ask(q)}")
# Add small trainable LoRA layers; the original weights stay frozen
model = get_peft_model(model, LoraConfig(r=8, lora_alpha=16, lora_dropout=0.05,
target_modules="all-linear", task_type="CAUSAL_LM"))
model.print_trainable_parameters()
examples = [json.loads(line) for line in open("train.jsonl", encoding="utf-8")]
optimiser = torch.optim.AdamW([p for p in model.parameters() if p.requires_grad], lr=LR)
model.train()
for epoch in range(1, EPOCHS + 1):
total = 0.0
for ex in examples:
prompt = f"Question: {ex['prompt']}\nAnswer:"
full = prompt + " " + ex["response"] + tokenizer.eos_token
ids = tokenizer(full, return_tensors="pt")["input_ids"]
labels = ids.clone()
labels[:, : len(tokenizer(prompt)["input_ids"])] = -100 # learn the answer, not the question
loss = model(input_ids=ids, labels=labels).loss
loss.backward()
optimiser.step()
optimiser.zero_grad()
total += loss.item()
print(f"epoch {epoch} average loss {total / len(examples):.3f}")
model.eval()
print("\n=== AFTER fine-tuning ===")
for q in TEST_QUESTIONS:
print(f"{q}\n -> {ask(q)}")
model.save_pretrained("my-lora") # only the small add-on weights are saved
print("\nSaved the adapter to the folder my-lora") Check: The file is saved next to train.jsonl.
The first run downloads the model, which needs the internet once. Training then takes a minute or a few on a laptop CPU. You may see harmless warnings.
python finetune.py Check: You see three answers under BEFORE, a line like trainable params: … || trainable%: … showing a small share of the model, the average loss falling over three epochs, then three answers under AFTER.
Read both sets of answers. The “after” answers should follow your pattern, here the Short answer: … Tip: … layout, even for questions that were not in the file. The content may be thin or wrong, because the model is tiny; what changed is the style. That is what LoRA is good at.
Check: You can point to one way the after answers copy your format. Exact words vary between runs and computers.
Edit one value at a time and run again: EPOCHS = 1 (too little change?), EPOCHS = 6 (more change, maybe more copying), LR = 1e-4 (gentler), or your own examples in a different style. Add your own test questions to TEST_QUESTIONS. A folder named my-lora holds the adapter, which is small compared with the model, as the file sizes show.
Check: You can say what changing each setting did to the “after” answers.
Paste or edit lines of train.jsonl. The checker finds broken lines, counts examples and tells you if your style is steady. Nothing is trained here.
4 examples. Aim for 20 or more for a visible change; a few hundred is better for real work.
The checker runs on this page only. Your text is not sent anywhere. It checks the file format, not whether the answers are right.
What do you see?
What is trained in LoRA fine-tuning?
What is fine-tuning best at teaching?
After training, the model copies your examples word for word. What now?